[OSM-dev] Reducing osm2pgsql memory usage using a database method
frederik at remote.org
Sun Mar 11 16:37:02 GMT 2007
> There are several perl interface to osm, but working with planet.osm in perl
> - takes 40 minutes only for reading
> - takes 100Bytes of memory for each node stored in memory.
> Which is more than 10 times more than really needed.
That's not accurate. My script for determining the last-modified time of
level-12 tiles processes the full planet file - actually more of it,
because it first seeks to the begin of the way portion, then re-starts
and seeks to the segment portion, then reads nodes, all to save memory -
in less than 15 minutes.
And storing a node with lat/lon as float and it's id will need at least
12 bytes in any programming language I know, PLUS the overhead incurred
by any hash map structure you decide to use... granted, you'll not be
able to live on 12 bytes for a Node in Perl, but you cannot hold Perl
responsible for the thoughtless way in which people write their scripts.
Frederik Ramm ## eMail frederik at remote.org ## N49°00.09' E008°23.33'
More information about the dev