[OSM-dev] XAPI and other solutions

[Igor Podolskiy]

Hi,

On 26.04.2011 22:29, Frederik Ramm wrote:
> You could probably save some time by doing something like this
> (shortened by dropping arguments):
>
> curl ... -o - | osmosis --rb /dev/stdin --tee 3 --tf --tf -tf --wx --tf
> --tf --un --wx --tf --un --uw --wx
>
> that way you would start processing while the download was still
> ongoing, and process all three filters at the same time. Logically it
> would even be possible to add the merge step to the back of the chain
> but I remember some past discussion on the Osmosis list about potential
> deadlocks if you combine what you have split in the same task.
exactly, to do tee+merge in a single process you'll need --buffer/--b 
tasks after the tee or you will deadlock for sure. So the pipeline would be:

osmosis --rb /dev/stdin --tee 3 --b --b --b --tf --tf -tf --wx --tf --tf 
--un --wx --tf --un --uw --merge -merge --wx

(Though if I'd be doing this for real, I'd spell out the at least the 
out/in-pipe names, in my experience the pipe stack is quite easy to 
break and quite hard to debug in big pipelines like this operating on 
data sets >1G).

Best regards
Igor