[OSM-dev] SOTM EU Imposm presentation: osm2pgsql memory usage with large extracts

Jukka Rahkonen jukka.rahkonen at latuviitta.fi
Thu Jun 19 12:02:14 UTC 2014


Paul Norman wrote 2014-06-18 19:44:
> The Imposm presentation states that larger extracts in osm2pgsql
> require comparable resources to a full planet import due to the cache
> structure used. Having not observed this when importing, I collected
> the data to test it while performing an import of the Germany
> Geofabrik extract from June 1st. At a size of 2.2GB (10^9 bytes/GB),
> this is a large extract.
> 
> osm2pgsql reports the storage efficiency of the node cache, in this
> case 51.85%. This is a typical efficiency for extracts.
> 
> This works out to a minimum memory of 2.7GB to keep all nodes in the
> memory cache. It is of course possible to import with less, but then
> osm2pgsql will fetch node locations from the database. I would
> generally question if a machine with under 4GB of RAM is suitable for
> rendering such a large area, regardless of OSM to database conversion
> tools used.

OSM data are not only used for rendering. With GDAL I can easily import 
germany-latest.osm.pbf into PostGIS on a hosted server with only 700 MB 
of RAM and it is perfectly suitable for serving data out through WFS 
service. I do use the same machine for rendering into WMS with Mapserver 
and it works also but honestly, it is not very fast. PostgreSQL+PostGIS 
runs also on the same tiny server. Importing with osm2pgsql is not 
possible and if I want to play with osm2pgsql tables I must do import 
with home computer and transfer the tables with pgdump/restore which is 
not very handy but with my budget I must tolerate it.

-Jukka Rahkonen-





More information about the dev mailing list