[OSM-dev] Osmosis crashing due to lack of memory

[Emilie Laffray]

>
> Are you sure to be using DMA for HDD-access?
> It should be IO-bound for such simple tasks.
>
> What do you mean with "maxed out"? "99% user" or
> just just "0% idle"?
> IO-processes (Windows: [x]show kernel times) use
> cpu-time too.
>
> (It's sad that Java-NIO does not get async file-io until
>  Java7.)
>

Well, whether it is IO bound or not in this case doesn't really
matter. Osmosis just crashed again on me, while I increased the VM to
1024MB.
The exact message that i am getting is this:

INFO: Processing Way 15717214.
26-Feb-2009 12:45:54 com.bretth.osmosis.core.progress.v0_5.EntityProgressLogger
process
INFO: Processing Way 15815970.
26-Feb-2009 12:46:00 com.bretth.osmosis.core.pipeline.common.ActiveTaskManager w
aitForCompletion
SEVERE: Thread for task 1-read-xml failed
java.lang.OutOfMemoryError: Java heap space
        at com.bretth.osmosis.core.filter.common.ListIdTracker.extendIdList(List
IdTracker.java:66)
        at com.bretth.osmosis.core.filter.common.ListIdTracker.set(ListIdTracker
.java:117)
        at com.bretth.osmosis.core.filter.v0_5.UsedNodeFilter.process(UsedNodeFi
lter.java:89)
        at com.bretth.osmosis.core.container.v0_5.WayContainer.process(WayContai
ner.java:60)
        at com.bretth.osmosis.core.filter.v0_5.UsedNodeFilter.process(UsedNodeFi
lter.java:58)
        at com.bretth.osmosis.core.filter.v0_5.WayKeyValueFilter.process(WayKeyV
alueFilter.java:90)
        at com.bretth.osmosis.core.container.v0_5.WayContainer.process(WayContai
ner.java:60)
        at com.bretth.osmosis.core.filter.v0_5.WayKeyValueFilter.process(WayKeyV
alueFilter.java:53)
        at com.bretth.osmosis.core.progress.v0_5.EntityProgressLogger.process(En
tityProgressLogger.java:84)
        at com.bretth.osmosis.core.xml.v0_5.impl.WayElementProcessor.end(WayElem
entProcessor.java:103)
        at com.bretth.osmosis.core.xml.v0_5.impl.OsmHandler.endElement(OsmHandle
r.java:109)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endEleme
nt(Unknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp
l.scanEndElement(Unknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp
l$FragmentContentDriver.next(Unknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(U
nknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp
l.scanDocument(Unknown Source)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(U
nknown Source)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(U
nknown Source)
        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown So
urce)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Un
known Source)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.p
arse(Unknown Source)
        at javax.xml.parsers.SAXParser.parse(Unknown Source)
        at javax.xml.parsers.SAXParser.parse(Unknown Source)
        at com.bretth.osmosis.core.xml.v0_5.XmlReader.run(XmlReader.java:109)
        at java.lang.Thread.run(Unknown Source)
26-Feb-2009 12:46:00 com.bretth.osmosis.core.Osmosis main
SEVERE: Execution aborted.
com.bretth.osmosis.core.OsmosisRuntimeException: One or more tasks failed.
        at com.bretth.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pi
peline.java:146)
        at com.bretth.osmosis.core.Osmosis.run(Osmosis.java:85)
        at com.bretth.osmosis.core.Osmosis.main(Osmosis.java:30)

The command line was:
java -Xmx1024m -jar e:\osmosis-0.30\osmosis.jar --read-xml
file="planet-090218.osm" enableDateParsing="no" --log-progress
interval="10" --way-key-value
keyValueList="highway.motorway,highway.motorway_link,highway.trunk,highway.trunk_link,highway.primary,highway.primary_link,highway.secondary,highway.tertiary,highway.unclassified,highway.road,highway.residential,highway.living_street,highway.service,highway.track,highway.pedestrian,highway.bus_guideway,highway.path,highway.cycleway,highway.footway,highway.bridleway,highway.byway,highway.steps,highway.toll_booth,highway.ford,cycleway.lane,cycleway.track,cycleway.opposite_lane,opposite_track,cycleway.opposite,tracktype.grade1,tracktype.grade2,tracktype.grade3,tracktype.grade4,tracktype.grade5,waterway.stream,waterway.river,waterway.riverbank,waterway.canal,waterway.dock,waterway.weir,waterway.dam"
--used-node --migrate --write-xml-0.6
file="E:\world\dump\roads\roads.osm"

I am a bit at a loss on why it is failing here.

Emilie Laffray