[OSM-dev] osm2pgsql-64 and benchmarks
Frederik Ramm
frederik at remote.org
Tue Jun 21 12:44:39 BST 2011
Hi,
as you know I've recently made the osm2pgsql-64 branch which does
not use intarray. The osm2pgsql-64 branch is capable of handling 64-bit
IDs but can also be defined to use 32-bit IDs like the classic osm2pgsql.
I would encourage everyone to move to the osm2pgsql-64 branch as soon as
practicable. If you do not want to support 64-bit IDs then you can
simply remove the "#define OSMID64" from osmtypes.h and you have a plain
standard 32-bit system. (Maybe we should do that by default?)
I would suggest that we soon make this branch the standard osm2pgsql
version so that any further development takes place on this and not on
the old intarray version.
I have done some timings with various builds of osm2pgsql and compared
Postgres 8.4 and 9.0. I used the settings
--slim -C4000 -r primitive -S default.style
and parsed a germany.osm.gz file (1.85 GB compressed, 16.23 GB
uncompressed) on a plain simple desktop machine (64bit processor) with a
single standard SATA hard disk.
The results (all times in minutes):
Postgres 8.4 Postgres 9.0
--------------------------------------------------------------------
old osm2pgsql (intarray) 401 308
osm2pgsql-64 with 32bit IDs 407 297
osm2pgsql-64 with 64bit IDs 415 302
This was not a scientific experiment but I took some care to make sure
the machine wasn't doing anything else while importing. What is lacking
from this comparison is of course diff updates (no patience for that at
the moment).
I am surprised by the results. I was expecting Postgres 9 to perform
better on BIGINT operations because they have by-reference passing of
these values, but I did not expect Postgres 9 to be so much better in
the 32bit department as well.
Bye
Frederik
More information about the dev
mailing list