[OSM-dev] SOTM EU Imposm presentation: osm2pgsql memory usage with large extracts

[Paul Norman]

The Imposm presentation states that larger extracts in osm2pgsql require 
comparable resources to a full planet import due to the cache structure 
used. Having not observed this when importing, I collected the data to 
test it while performing an import of the Germany Geofabrik extract from 
June 1st. At a size of 2.2GB (10^9 bytes/GB), this is a large extract.

osm2pgsql reports the storage efficiency of the node cache, in this case 
51.85%. This is a typical efficiency for extracts.

This works out to a minimum memory of 2.7GB to keep all nodes in the 
memory cache. It is of course possible to import with less, but then 
osm2pgsql will fetch node locations from the database. I would generally 
question if a machine with under 4GB of RAM is suitable for rendering 
such a large area, regardless of OSM to database conversion tools used.

While re-checking the presentation for memory usage, I noted a mistake 
about flat-nodes. Flat nodes is not like normal mode, it is part of slim 
mode, but with the nodes table moved from PostgreSQL to on disk.