[OSM-dev] speeding up loading an OSM dump into PostGIS?

[Jukka Rahkonen]

Kai Krueger wrote:

> On 01/-10/-28163 12:59 PM, Jukka Rahkonen wrote:

>> It can be slow but it is not always a bad idea. I have a very lean Linux
>> virtual server with about 700 MB of memory and it is very slow to import
>> even Finnish excerpt with osm2pgsql. In addition import tends to fail
>> totally about every second time. However, the virtual box has no
>> troubles
>> if I run osm2pgsql at home, upload PG dump files into server and run
>> restore there. Even running restore from my home computer with pgAdmin
>> III
>> through a 1 Mb line upwards is faster than making import with osm2pgsql
>> on
>> the Linux box.  Ogr2ogr from the home PostGIS to remote PostGIS has also
>> worked reliably for me.
>
> Can you describe a bit more of what your setup is, and where it fails?
> Which version of Postgresql, PostGis and Osm2pgsql? 32bit or 64bit?
> 700Mb of ram really is very low, but I would like osm2pgsql to work with
> little resources even if very slow.

For me it takes many hours with the Finnish dataset and if it fails it
happens in some "Going over pending ways" phase. I will need to make some
further tests some day so I can give you better information.  I know that
it is Ubuntu 10.04 and PostgreSQL 9.0 with PostGIS is running on the same
machine.  I believe it is 32-bit and osm2pgsql is half an year old. I have
not used it many times because of failures.
My Windows laptop in not much faster but it does not fail. Osm2pgsql
version is the newest that exists for Windows, it is rather old (April 9,
2010). Your 20 minutes total time feels amazing for me. But my server is
very basic and I do not wait very much for 27 euros per month. But when
the data are in it works well as a WFS server.

-Jukka-

> Kai