[OSM-dev] osm2pgsql hangs.

[Richard Ive]

Hi All,

I've had an open street map database set up on a server for a while now.
I've just got two new servers that are much, much more powerful (16GB Ram.
Two quad cores..). What I'm trying to do is have the database on one
machine, and mapnik on the other.

Last night I imported the latest planet using osm2pgsql in slim mode - it
took about 20 hours (I didn't use time so I'm guessing here). My old machine
used to take about 3 to 4 days, so I'm happy with the results there. I
assume the database has been populated correctly because I received no
errors, and 70GB of my disc space has been used up.

I've now hit an issue though. I'm about 6 days behind the planet from last
week. So I'm trying to import the daily diffs from last Thursday. The
command I've always used is this one:

*gunzip -dc /home/postgres/osm/daily/20091118-20091119.osc.gz |
/tmp/osm2pgsql/osm2pgsql -S /tmp/osm2pgsql/default.style --slim -C 2700 -a
-m -d gis -*

It has always worked so I tried it on my new machine, but it seems to hang
on "Reading file". Here's the output so far, after about 1.5 hours:

[postgres at mapsdb bin]$ gunzip -dc
/home/postgres/osm/daily/20091118-20091119.osc.gz | /tmp/osm2pgsql/osm2pgsql
-S /tmp/osm2pgsql/default.style --slim -C 2700 -a -m -d gis -
osm2pgsql SVN version 0.69-18773

*Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
NOTICE:  table "planet_osm_point_tmp" does not exist, skipping
Setting up table: planet_osm_line
NOTICE:  table "planet_osm_line_tmp" does not exist, skipping
Setting up table: planet_osm_polygon
NOTICE:  table "planet_osm_polygon_tmp" does not exist, skipping
Setting up table: planet_osm_roads
NOTICE:  table "planet_osm_roads_tmp" does not exist, skipping
Mid: pgsql, scale=100, cache=2700MB, maxblocks=345601*8192
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels

Reading in file: -*


Top shows a postgresql process running at 99% cpu, so it must be doing
something?
*
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
32669 postgres  25   0  141m 133m 132m R 99.6  2.3   4:07.58 postgres*


Process ID 32669 is:

*postgres 32429 32427  0 15:21 ?        00:00:00 postgres: writer process
postgres 32430 32427  0 15:21 ?        00:00:00 postgres: wal writer process
postgres 32431 32427  0 15:21 ?        00:00:00 postgres: autovacuum
launcher process
postgres 32432 32427  0 15:21 ?        00:00:00 postgres: stats collector
process
postgres 32661 32404  0 15:40 pts/0    00:00:00 gunzip -dc
/home/postgres/osm/daily/20091118-20091119.osc.gz
postgres 32662 32404  0 15:40 pts/0    00:00:00 /tmp/osm2pgsql/osm2pgsql -S
/tmp/osm2pgsql/default.style --slim -C 2700 -a -m -d gis -
postgres 32664 32427  0 15:40 ?        00:00:00 postgres: postgres gis
[local] idle in transaction
postgres 32665 32427  0 15:40 ?        00:00:00 postgres: postgres gis
[local] COPY
postgres 32666 32427  0 15:40 ?        00:00:00 postgres: postgres gis
[local] COPY
postgres 32667 32427  0 15:40 ?        00:00:00 postgres: postgres gis
[local] COPY
postgres 32668 32427  0 15:40 ?        00:00:00 postgres: postgres gis
[local] idle in transaction
postgres 32669 32427 99 15:40 ?        00:04:50 postgres: postgres gis
[local] UPDATE
postgres 32670 32427  0 15:40 ?        00:00:00 postgres: postgres gis
[local] idle in transaction*

I hope I've provided enough information. If I haven't please drop me an
email.

Any help would be greatly appreciated.

Thank you.
-- 
Regards,
Richard Ive.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20091125/c3ebfca3/attachment.html>