[OSM-dev] osm2pgsql slow slim import

sly (sylvain letuffe) liste at letuffe.org
Thu Dec 1 10:03:27 GMT 2011


> Can you try if using the switch --number-processes=1? That should disable
> any parallelisation and make it more or less the same as before. (Although
> there is a slight difference in transaction handling)

It's x2 faster with --number-processes=4 instead of 1 but still far from older 
version

> Otherwise, can you check if you are getting 100% hit ratio from the node
> cache?
yes, 100%

extracted output :
$ osm2pgsql --number-processes=4 -C 2000 -s -S default.style -m -d gis2 
monaco.osm.bz2

Using 4 helper-processes
processing way (0k) at 0.00k/s
All child processes exited
Pending ways took 6s at a rate of 158.17/s
node cache: stored: 10409(100.00%), storage efficiency: 62.61% (dense blocks: 
1, sparse nodes: 10401), hit rate: 100.00%
Osm2pgsql took 9s overall

$ osm2pgsql --number-processes=1 -C 2000 -s -S default.style -m -d gis2 
monaco.osm.bz2

Using 1 helper-processes
processing way (0k) at 0.00k/s
All child processes exited

Pending ways took 15s at a rate of 63.27/s
node cache: stored: 10409(100.00%), storage efficiency: 62.61% (dense blocks: 
1, sparse nodes: 10401), hit rate: 100.00%
Osm2pgsql took 18s overall


(using revision 26892)
$ osm2pgsql -C 2000 -s -S default.style -m -d gis2 monaco.osm.bz2

Going over pending ways
processing way (0k) at 0.00k/s
Going over pending relations
node cache: stored: 10409(100.00%), storage efficiency: 2.32%, hit rate: 
100.00%
Osm2pgsql took 3s overall

The old version didn't display time spent in processing way, but for such a 
small extract it's a few ms


-- 
sly
qui suis-je : http://sly.letuffe.org
email perso : sylvain chez letuffe un point org



More information about the dev mailing list