[OSM-dev] Osmosis woes, something not right - can someone pls. see why?

[Brett Henderson]

2008/7/26 Karl Newman <siliconfiend at gmail.com>

>
> Well, I ran it for a while until it failed because of lack of heap space (I
> had -Xmx512M for the Java command but it wasn't enough for the
> ListIdTracker). But regardless, it already generated a 18.6 GB uncompressed
> OSM file (and that's only nodes), so I suspect there's something wrong with
> the creation of your segment_earth.osm. Make sure it's not failing silently.
>
> A few optimizations you could use:
>
>    1. Use planet.gz, not bz2 because Osmosis is much faster at gzip (it
>    has native code).
>    2. Don't enable date parsing.
>    3. Add the parameter "idTrackerType=BitSet" to the bounding box task.
>    It's more efficient for large bounding boxes.
>
> Karl
>

Apologies for the slow reply, I've had a lot on.

I had similar issues with out of memory errors and had to switch to the
BitSet id tracker.  I ended up with a 3.25 GB gzip compressed file for the
segment file.

Another optimisation is to stop using the log progress task.  I haven't
tried running with and without logging recently but I suspect it adds a lot
of overhead.  There's probably a much smarter way of doing this but the
current implementation isn't very efficient.

I'll run the second step now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20080730/2a8c238b/attachment.html>