[OSM-dev] osm2pgsql slow slim import
sly (sylvain letuffe)
liste at letuffe.org
Thu Dec 1 10:03:27 GMT 2011
> Can you try if using the switch --number-processes=1? That should disable
> any parallelisation and make it more or less the same as before. (Although
> there is a slight difference in transaction handling)
It's x2 faster with --number-processes=4 instead of 1 but still far from older
version
> Otherwise, can you check if you are getting 100% hit ratio from the node
> cache?
yes, 100%
extracted output :
$ osm2pgsql --number-processes=4 -C 2000 -s -S default.style -m -d gis2
monaco.osm.bz2
Using 4 helper-processes
processing way (0k) at 0.00k/s
All child processes exited
Pending ways took 6s at a rate of 158.17/s
node cache: stored: 10409(100.00%), storage efficiency: 62.61% (dense blocks:
1, sparse nodes: 10401), hit rate: 100.00%
Osm2pgsql took 9s overall
$ osm2pgsql --number-processes=1 -C 2000 -s -S default.style -m -d gis2
monaco.osm.bz2
Using 1 helper-processes
processing way (0k) at 0.00k/s
All child processes exited
Pending ways took 15s at a rate of 63.27/s
node cache: stored: 10409(100.00%), storage efficiency: 62.61% (dense blocks:
1, sparse nodes: 10401), hit rate: 100.00%
Osm2pgsql took 18s overall
(using revision 26892)
$ osm2pgsql -C 2000 -s -S default.style -m -d gis2 monaco.osm.bz2
Going over pending ways
processing way (0k) at 0.00k/s
Going over pending relations
node cache: stored: 10409(100.00%), storage efficiency: 2.32%, hit rate:
100.00%
Osm2pgsql took 3s overall
The old version didn't display time spent in processing way, but for such a
small extract it's a few ms
--
sly
qui suis-je : http://sly.letuffe.org
email perso : sylvain chez letuffe un point org
More information about the dev
mailing list