[OSM-dev] osm2pgsql failing with low cache size

[Michael Corey]

Hello all:

Maybe this just isn't going to work, but I'm trying to use osm2pgsql to 
process a small set of XML that I've downloaded through the Overpass API.

I'm running this on an EC2 micro instance, which supposedly has up to 
613 MB of RAM. That's not a lot, but the XML I'm trying to import is 
only 23 ways made up of 191 nodes. This will grow, but not by orders of 
magnitude.

So I'm running:

osm2pgsql ./data/latest.xml -d border_processing -S 
/home/border/osm_scraper/mystyle.style --slim --cache 128 --extra-attributes

Which fails, returning at the end:

Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=128MB, maxblocks=16385*8192, allocation method=11
Mid: pgsql, scale=100 cache=128
Setting up table: planet_osm_nodes
NOTICE:  table "planet_osm_nodes" does not exist, skipping
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index 
"planet_osm_nodes_pkey" for table "planet_osm_nodes"
Setting up table: planet_osm_ways
NOTICE:  table "planet_osm_ways" does not exist, skipping
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index 
"planet_osm_ways_pkey" for table "planet_osm_ways"
Setting up table: planet_osm_rels
NOTICE:  table "planet_osm_rels" does not exist, skipping
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index 
"planet_osm_rels_pkey" for table "planet_osm_rels"

Reading in file: ./data/latest.xml
StartElement: Unknown element name: note
Unknown node type 3
EndElement: Unknown element name: note
StartElement: Unknown element name: meta
EndElement: Unknown element name: meta
Segmentation fault (core dumped)

If I up the cache to anything over 500 MB then I run out of memory on 
the EC2 instance.

How much memory does osm2pgsql need for something seemingly small like 
this? Can I reduce the load significantly by cutting out other 
attributes in my config file? Any other suggestions?

Thanks much,

-- 
Michael Corey
News Applications Developer
Center for Investigative Reporting
510.809.3178

/2012 Recipient of the MacArthur Award for Creative and Effective 
Institutions/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20120619/a5f4f90c/attachment.html>