[OSM-dev] Osm2pgsql and failed planet import

Francisco.Matamala francisco.matamala at weblakes.com
Fri Aug 26 18:18:37 BST 2011


Hi everyone,

  I've been attempting to perform a planet import for mapnik rendering for
the past few weeks without success.  I've setup my PostGreSQL database with
PostGis correctly, including the various parameter tweaks.  I setup
PostGreSQL on a separate Windows Server 2008 box with 8GB RAM, in order to
separate processes and not have osm2pgsql have missing memory issues.  This
box has plenty of space for the final database.

  I setup a virtual box on our virtualization server using a CentOS
distribution, obtained osm2pgsql from source, turned on the 64bit
compilation flags and compiled it.  This virtual box has 16GB memory.

  I've attempted several times to import, without success.  The last attempt
I made was using a boundary box, hoping osm2pgsql would have an easier time
just importing North America.  I'm using slim mode and a 14GB cache.

Here is the command line and the output:

./osm2pgsql64 -c -s -m -C 14000 --bbox -165,13,-45,90 -U gis -W -H
192.168.1.131 -P 5432 -S ./default.style ./planet-110810.osm.bz2
osm2pgsql SVN version 0.80.0 (64bit id space)

Password:
Using projection SRS 900913 (Spherical Mercator)
Applying Bounding box: -165.000000,13.000000 to -45.000000,90.000000
Setting up table: planet_osm_point
NOTICE:  table "planet_osm_point_tmp" does not exist, skipping
Setting up table: planet_osm_line
NOTICE:  table "planet_osm_line_tmp" does not exist, skipping
Setting up table: planet_osm_polygon
NOTICE:  table "planet_osm_polygon_tmp" does not exist, skipping
Setting up table: planet_osm_roads
NOTICE:  table "planet_osm_roads_tmp" does not exist, skipping
Mid: pgsql, scale=100, cache=14000MB, maxblocks=1792001*8192
Setting up table: planet_osm_nodes
NOTICE:  table "planet_osm_nodes" does not exist, skipping
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index
"planet_osm_nodes_pkey" for table "planet_osm_nodes"
Setting up table: planet_osm_ways
NOTICE:  table "planet_osm_ways" does not exist, skipping
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index
"planet_osm_ways_pkey" for table "planet_osm_ways"
Setting up table: planet_osm_rels
NOTICE:  table "planet_osm_rels" does not exist, skipping
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index
"planet_osm_rels_pkey" for table "planet_osm_rels"

Reading in file: ./planet-110810.osm.bz2
Unknown node type 8
Processing: Node(1173423k) Way(104443k) Relation(1070274)  parse time:
175361s

Node stats: total(1173423751), max(1393174131)
Way stats: total(104443727), max(125432779)
Relation stats: total(1070274), max(1706327)

Going over pending ways
processing way (26160k)way_done failed: ERROR:  out of memory
DETAIL:  Failed on request of size 419430400.
(7)
Arguments were: 75277817,
Error occurred, cleaning up

  Now the error occurs with or without 64bits, with or without the boundary
box.  It seems like the pending ways section of the code is behaving oddly,
or I've setup something incorrectly.  The size of the memory request is odd
as well.  Anybody have any ideas?

  Best regards to all,
  Francisco


--
View this message in context: http://gis.638310.n2.nabble.com/Osm2pgsql-and-failed-planet-import-tp6729516p6729516.html
Sent from the Developer Discussion mailing list archive at Nabble.com.



More information about the dev mailing list