[OSM-dev] osmosis bz2 performance

[Stefan Baebler]

Martijn van Oosterhout wrote:
> On Feb 7, 2008 2:01 PM, Stefan Baebler <stefan.baebler at gmail.com> wrote:
>>>> Would "-" have a meaning somewhere else? read xml change, write xml
>>>> ... all file operations perhaps?
>>> Yep, ideally this should be done for all tasks with a file argument.
>> It still won't be possible to use 2 input streams (eg planet.osm +
>> planet.osc) on stdin, but at least one (probably bigger source) as it
>> was now with /dev/stdin. Named pipes could solve that, but AFAIK they
>> are not supported in Java. Sockets could work, but are not as
>> convenient for scripting.
> 
> Actually, named pipes would work, since osmosis doesn't seek. Something like:
> 
> mkfifo /tmp/buffer
> bzcat x.bz2 >/tmp/buffer &
> osmosis .... --read /tmp/buffer
> rm /tmp/buffer
> 
> Should do fine.
interesting, tnx!
This could be a workaround for the newly introduced aliases.

I've added the "-" alias to the other xml file reading and writing tasks:
http://trac.openstreetmap.org/changeset/6844
Now it is possible to do
wget -O - http://... |  bzcat -c | osmosis --rx - --bb ... --wx - | 
bzip2 > excerpt.osm.bz2
which performs considerably better than Apache's bzip implementation in 
Java, which is built into osmosis.

Enjoy,
Stefan