[OSM-dev] SOTM EU Imposm presentation: osm2pgsql memory usage with large extracts
Jukka Rahkonen
jukka.rahkonen at latuviitta.fi
Thu Jun 19 12:02:14 UTC 2014
Paul Norman wrote 2014-06-18 19:44:
> The Imposm presentation states that larger extracts in osm2pgsql
> require comparable resources to a full planet import due to the cache
> structure used. Having not observed this when importing, I collected
> the data to test it while performing an import of the Germany
> Geofabrik extract from June 1st. At a size of 2.2GB (10^9 bytes/GB),
> this is a large extract.
>
> osm2pgsql reports the storage efficiency of the node cache, in this
> case 51.85%. This is a typical efficiency for extracts.
>
> This works out to a minimum memory of 2.7GB to keep all nodes in the
> memory cache. It is of course possible to import with less, but then
> osm2pgsql will fetch node locations from the database. I would
> generally question if a machine with under 4GB of RAM is suitable for
> rendering such a large area, regardless of OSM to database conversion
> tools used.
OSM data are not only used for rendering. With GDAL I can easily import
germany-latest.osm.pbf into PostGIS on a hosted server with only 700 MB
of RAM and it is perfectly suitable for serving data out through WFS
service. I do use the same machine for rendering into WMS with Mapserver
and it works also but honestly, it is not very fast. PostgreSQL+PostGIS
runs also on the same tiny server. Importing with osm2pgsql is not
possible and if I want to play with osm2pgsql tables I must do import
with home computer and transfer the tables with pgdump/restore which is
not very handy but with my budget I must tolerate it.
-Jukka Rahkonen-
More information about the dev
mailing list