[OSM-dev] osm2pgsql-64 and benchmarks

Frederik Ramm frederik at remote.org
Tue Jun 21 12:44:39 BST 2011


    as you know I've recently made the osm2pgsql-64 branch which does 
not use intarray. The osm2pgsql-64 branch is capable of handling 64-bit 
IDs but can also be defined to use 32-bit IDs like the classic osm2pgsql.

I would encourage everyone to move to the osm2pgsql-64 branch as soon as 
practicable. If you do not want to support 64-bit IDs then you can 
simply remove the "#define OSMID64" from osmtypes.h and you have a plain 
standard 32-bit system. (Maybe we should do that by default?)

I would suggest that we soon make this branch the standard osm2pgsql 
version so that any further development takes place on this and not on 
the old intarray version.

I have done some timings with various builds of osm2pgsql and compared 
Postgres 8.4 and 9.0. I used the settings

--slim -C4000 -r primitive -S default.style

and parsed a germany.osm.gz file (1.85 GB compressed, 16.23 GB 
uncompressed) on a plain simple desktop machine (64bit processor) with a 
single standard SATA hard disk.

The results (all times in minutes):

                                   Postgres 8.4    Postgres 9.0
old osm2pgsql (intarray)              401              308
osm2pgsql-64 with 32bit IDs           407              297
osm2pgsql-64 with 64bit IDs           415              302

This was not a scientific experiment but I took some care to make sure 
the machine wasn't doing anything else while importing. What is lacking 
from this comparison is of course diff updates (no patience for that at 
the moment).

I am surprised by the results. I was expecting Postgres 9 to perform 
better on BIGINT operations because they have by-reference passing of 
these values, but I did not expect Postgres 9 to be so much better in 
the 32bit department as well.


More information about the dev mailing list