[OSM-dev] Reducing osm2pgsql memory usage using a database method

[Frederik Ramm]

Hi,

> There are several perl interface to osm, but working with planet.osm in perl 
>  - takes 40 minutes only for reading 
>  - takes 100Bytes of memory for each node stored in memory. 
>    Which is more than 10 times more than really needed.

That's not accurate. My script for determining the last-modified time of 
level-12 tiles processes the full planet file - actually more of it, 
because it first seeks to the begin of the way portion, then re-starts 
and seeks to the segment portion, then reads nodes, all to save memory - 
in less than 15 minutes.

And storing a node with lat/lon as float and it's id will need at least 
12 bytes in any programming language I know, PLUS the overhead incurred 
by any hash map structure you decide to use... granted, you'll not be 
able to live on 12 bytes for a Node in Perl, but you cannot hold Perl 
responsible for the thoughtless way in which people write their scripts.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00.09' E008°23.33'