[OSM-dev] Osm2pgsql failure with low-end server

[Jukka Rahkonen]

Hi,

I had a new try with importing Finnish osm excerpt with brand new
osm2pgsql into PostgreSQL 8.4 on a very modest Linux server (700 Mb of
memory). Import seems to be much faster in the beginning. It breaks still
at the same place than with older osm2pgsql. Error seems to be something
general server error. All that can be done in this case is to start again.
Sooner or later it is going through but with this server I like more to
create a PG dump file with another computer and install it with PG
restore. It never fails.

osm2pgsql -d gis -p osm2 -P 5432 -s  -k -E 3067 --drop finland.osm.bz2
osm2pgsql SVN version 0.80.0 (32bit id space)

Using projection SRS 3067 (EPSG:3067)
Setting up table: osm2_point
NOTICE:  table "osm2_point" does not exist, skipping
NOTICE:  table "osm2_point_tmp" does not exist, skipping
Setting up table: osm2_line
NOTICE:  table "osm2_line" does not exist, skipping
NOTICE:  table "osm2_line_tmp" does not exist, skipping
Setting up table: osm2_polygon
NOTICE:  table "osm2_polygon" does not exist, skipping
NOTICE:  table "osm2_polygon_tmp" does not exist, skipping
Setting up table: osm2_roads
NOTICE:  table "osm2_roads" does not exist, skipping
NOTICE:  table "osm2_roads_tmp" does not exist, skipping
Allocating memory for dense node cache
Allocating dense node cache in block sized chunks
Node-cache: cache=800MB, maxblocks=0*102401, allocation method=8192
Mid: pgsql, scale=100
Setting up table: osm2_nodes
NOTICE:  table "osm2_nodes" does not exist, skipping
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index
"osm2_nodes_pkey" for table "osm2_nodes"
Setting up table: osm2_ways
NOTICE:  table "osm2_ways" does not exist, skipping
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index
"osm2_ways_pkey" for table "osm2_ways"
Setting up table: osm2_rels
NOTICE:  table "osm2_rels" does not exist, skipping
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index
"osm2_rels_pkey" for table "osm2_rels"

Reading in file: finland.osm.bz2
Processing: Node(5020k 31.8k/s) Way(0k 0.00k/s) Relation(0 0.00/s)WARNING:
Found Out of order node 727762854 (34265137,934) - this will impact the
cache efficiency
Processing: Node(10558k 30.0k/s) Way(887k 1.97k/s) Relation(10548 32.36/s)
 parse time: 1129s

Node stats: total(10558318), max(1534413174) in 352s
Way stats: total(887645), max(140040529) in 451s
Relation stats: total(10548), max(1889635) in 326s
Committing transaction for osm2_point
Committing transaction for osm2_line
Committing transaction for osm2_polygon
Committing transaction for osm2_roads

Going over pending ways

Using 1 helper-processes
processing way (68k) at 0.46k/sWARNING:  terminating connection because of
crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and
repeat your command.
way_done failed: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
(7)
Arguments were: 32041909,
Error occurred, cleaning up

-Jukka Rahkonen-