[OSM-dev] Osmosis crashing due to lack of memory

D Tucny d at tucny.com
Thu Feb 26 12:03:42 GMT 2009


2009/2/26 Emilie Laffray <emilie.laffray at gmail.com>

> Hi,
>
> I have been trying to play with the planet file and extract
> information out of it. However, it crashed on me with an out of memory
> memory regarding heap space.
> I used the following command:
> osmosis --read-xml file="planet-090218.osm" enableDateParsing="no"
> --way-key-value
>
> keyValueList="highway.motorway,highway.motorway_link,highway.trunk,highway.trunk_link,highway.primary,highway.primary_link,highway.secondary,highway.tertiary,highway.unclassified,highway.road,highway.residential,highway.living_street,highway.service,highway.track,highway.pedestrian,highway.bus_guideway,highway.path,highway.cycleway,highway.footway,highway.bridleway,highway.byway,highway.steps,highway.toll_booth,highway.ford,cycleway.lane,cycleway.track,cycleway.opposite_lane,opposite_track,cycleway.opposite,tracktype.grade1,tracktype.grade2,tracktype.grade3,tracktype.grade4,tracktype.grade5,waterway.stream,waterway.river,waterway.riverbank,waterway.canal,waterway.dock,waterway.weir,waterway.dam"
> --used-node --migrate --write-xml-0.6
> file="E:\world\dump\roads\roads.osm"
>
> I would love to avoid as much as possible to break the planet into
> bounding boxes, for the sake of simplicity initially. Maybe it is
> unrealistic but I would like to keep it that way.
>

In osmosis.bat is the following line...
REM # JAVACMD_OPTIONS - The options to append to the java command, typically
used to modify jvm settings such as max memory.

so, either modify osmosis.bat or, create a new file called osmosis.bat in
the all users profile directory or your profile directory, to include a 'set
JAVACMD_OPTIONS = -Xmx1024M' line (to set maximum memory usage to 1GB, if
you have enough RAM)...



> I am using the planet file uncompressed with osmosis 0.30.
> Is there something I can do? Spending hours and then seeing your task
> crashing is pretty disheartening.
> Also, is there a way to use multicore efficiently? I have looked at
> the pages but I didn't see any convincing number. I was thinking of
> maybe of getting the file to be read in one drive and writing to the
> other one, therefore avoiding the IO hit of reading and writing at the
> same time on the same drive.
>

Using a gzipped planet input and gzipped output may be quicker if your
facing an IO bottleneck... the compression and decompression will use more
CPU, but, significantly reduce the IO...

d
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20090226/e8c2fbd3/attachment.html>


More information about the dev mailing list