[OSM-dev] speeding up loading an OSM dump into PostGIS?

[Ákos Maróy]

Hi,

I wonder what ways are there to speed up importing an OSM planet file
into a PostGIS database?

What I've tried so far is importing the current planet-XXX.osm.bz2 file
into PostGIS via osm2pgsl, which I have used with the --slim option, as
without it the memory load exceeded the 16GB memory I had in my system
significantly (it was using about 26GB when I shut the process down).
this way, it took about a week to import the current planet.osm file, on
a Xeon 4 core system running 64 bit Linux.

After this succeeded, I wanted to try to replicate this database, so I
created a pg_dump using the -Fc switch, and then import this using
pg_restore with --disable-triggers - but this process is taking rather
long as well, and it is obvious that it's quite I/O heavy, as CPU load
is rather low, and the pogres process is spending most of its time
waiting on I/O.

thus, I wonder, what good ways are there to speed up this process?

or, to put it in the other perspective: what hardware would make be
needed to make this process faster?


Akos