[OSM-dev] speeding up loading an OSM dump into PostGIS?
Ákos Maróy
akos at maroy.hu
Wed Nov 30 06:25:25 GMT 2011
Hi,
I wonder what ways are there to speed up importing an OSM planet file
into a PostGIS database?
What I've tried so far is importing the current planet-XXX.osm.bz2 file
into PostGIS via osm2pgsl, which I have used with the --slim option, as
without it the memory load exceeded the 16GB memory I had in my system
significantly (it was using about 26GB when I shut the process down).
this way, it took about a week to import the current planet.osm file, on
a Xeon 4 core system running 64 bit Linux.
After this succeeded, I wanted to try to replicate this database, so I
created a pg_dump using the -Fc switch, and then import this using
pg_restore with --disable-triggers - but this process is taking rather
long as well, and it is obvious that it's quite I/O heavy, as CPU load
is rather low, and the pogres process is spending most of its time
waiting on I/O.
thus, I wonder, what good ways are there to speed up this process?
or, to put it in the other perspective: what hardware would make be
needed to make this process faster?
Akos
More information about the dev
mailing list