[Openstreetmap] Re: PostgreSQL perl script and GPS protocol question

[ronan at roasp.com]

>
> [Petter Reinholdtsen]
>> To avoid having to investigate how to get the Java program to store
>> points into a database to work on my woody installation (kaffe
>> didn't seem up for the job), I rewrote the collecting task in Perl.
>
...
>
> When running 'select * from track order by time', the DBD/Perl scritp
> tries to load the complete database into memory.  This is doomed to
> fail, and does so.
>
> I consider this to be a bug in DBD::Pg, but accept the design
> decision.  I'm going to test if I can use cursors directly within the
> DBD driver.  If this fail, I will have to change to another language.
>

It appears to be related more to DBI and maybe also to the underlying
client application for PG than to DBD::Pg.

> JFYI.
>
> Anyone know if the Python or C drivers uses cursors by default?
>

The doc on the net claim the Python also fails on cursors (and so does
php). I have not corraborated this.

I would bet this is a problem with binding to the OS-specific postgres
drivers by DBD::Pg rathen than an inherent DBD::Pg problem. It might be
worth your while to drop the DBD::Pg mailing list a note asking for
further info and showing interest...

http://gborg.postgresql.org/pipermail/dbdpg-general/2004-October/001222.html

If your efforts are memory-driven, beware... I am not sure cursors will
save you anything memory-wise. Maybe there is a more efficient way to
solve your problem than cursors? In 10 years of app dev using DBs, I have
never found a case where cursors added more value than they cost me
elsewhere.

Finally, if you really, really want to use a cursor, you can try the
following:

Try to use the pure-perl implementation of DBD::Pg called DBD::PgPP. It
may support cursors.
http://search.cpan.org/perldoc?DBD::PgPP

All the best,

Ronan