[OSM-talk] Osmosis running forever with completeWays=yes?

Karl Newman siliconfiend at gmail.com
Sun Feb 1 21:24:00 GMT 2009


On Sun, Feb 1, 2009 at 11:11 AM, Nop <ekkehart at gmx.de> wrote:

>
> Hi!
>
> I am trying to cut 10 areas out of the planetfile for germany. When I
> set the parameter completeWays=yes for these areas, osmosis (V0.29)
> seems to run forever. After 8,5 hours of CPU time, osmosis had created a
> number of large temporary files, one output file was at about 15% the
> expected size and all others were still empty. Without the parameter
> completeWays=yes the job is completed in less than 30 minutes.
>
> Is completeWays=yes broken? Or does it really take this long to process?
> Is there anything I can do differently? I want the cut areas to be
> seamless, that is the reason I switched on completeWays=yes.
>
> All hints welcome
>
> thanks
>
>        Nop
>

The short answer is: don't use completeWays with tee, or if you do, limit
the number of tees and start with a smaller extract (not the entire planet).

completeWays is not (completely) broken. Since Osmosis processes entities in
a stream, and the ways only have node references, not their locations, the
completeWays function uses a temporary file to store the nodes until the
ways are being read. Then it processes the ways, marks the extra nodes it
needs and stuffs the ways into a temporary file. The stored nodes are then
streamed out and filtered as needed, then the stored ways are streamed out
and filtered as needed.

Anyway, the tee can choke things up with all the temporary files. It would
be nice to be able to share the stored node and ways files between tee
tasks, but I haven't created that infrastructure yet.

Karl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk/attachments/20090201/1766bade/attachment.html>


More information about the talk mailing list