[osmosis-dev] deadlock when extracting bounds with 0.41 but not 0.40.1

[Brett Henderson]

Hi Eric,


On 13 December 2012 23:18, Eric Fernandez <zeb at zebulon.org.uk> wrote:

> Hi,
>
> This is a follow-up to the thread here:
>
> http://lists.openstreetmap.org/pipermail/osmosis-dev/2012-September/001333.html
>
> I am experiencing the same hanging issue with osmosis 0.41 with the GB
> map from , which I did not have with osmosis 0.40.1.
>
> The command I use is the same one (as described in
>
> http://wiki.openstreetmap.org/wiki/Mkgmap/help/options#Using_precompiled_bounds_for_the_address_index
> ):
>
> ./osmosis-0.41/bin/osmosis --read-pbf file=great_britain.osm.pbf
> outPipe.0=1 --tee 2 inPipe.0=1 outPipe.0=2 outPipe.1=3 --buffer
> inPipe.0=3 outPipe.0=4 --buffer inPipe.0=2 outPipe.0=5 --tag-filter
> accept-relations boundary=administrative,postal_code inPipe.0=4
> outPipe.0=6 --used-way inPipe.0=6 outPipe.0=7 --tag-filter
> reject-relations inPipe.0=5 outPipe.0=8 --tag-filter accept-ways
> boundary=administrative,postal_code inPipe.0=8 outPipe.0=9 --used-node
> inPipe.0=9 outPipe.0=10 --used-node inPipe.0=7 outPipe.0=11 --merge
> inPipe.0=10 inPipe.1=11 outPipe.0=12 --write-pbf
> file=great_britain-boundaries.osm.pbf omitmetadata=true
> compress=deflate inPipe.0=12
>
> In his answer, Andrew Byrd explains there may be a deadlock problem
> with the inputs for the merge, but this command was working with
> osmosis-0.40.1, and normally is dealing with this issue already. Is
> there any reason that the new 0.41 version leads to the issue?
>

Andrew is correct, the deadlock is expected behaviour with your pipeline
structure.  Splitting data with a --tee and then recombining with a --merge
should not be done in the same pipeline.

As to why it works in 0.40.1 and not 0.41 I'm not sure, but there were
major changes made to the DataPostbox class which is used as a data buffer
between threads.  It is possible that these changes have triggered your
deadlock condition, although I'm not aware of anything specific.  It *may*
be possible to avoid the deadlock by specifying a large bufferCapacity on
your --buffer tasks, but this isn't recommended as you'll probably still
run into deadlock conditions at some point in the future.

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmosis-dev/attachments/20121229/22e653c7/attachment.html>