[OSM-dev] osmosis bz2 performance
brett at bretth.com
Wed Feb 6 20:08:50 GMT 2008
Stefan Baebler wrote:
>> convenience aid, but if I'm processing large files I use the native
>> bzip2 command line tools and pipe the data into osmosis. For example.
> Probably a brief note in the Wiki could save quite a bit of someone's
> time. Added to wiki.
>> bzcat planet.bz2 | osmosis --rx /dev/stdin --wn
> Nice on unixes, but I can't seem to read "/dev/stdin" in Cygwin.
> I tried that to pipe the planet straight from wget -> osmosis without
> writing the planet to file.
> putting native bzip in between would improve perofrmance quite a bit.
> wget -O - http://.... | bzip -c | osmosis -rx /dev/stdin ...
I've been expecting someone to complain about this eventually :-)
> Can Osmosis get some alias for /dev/stdin (eg "-" would mean stdin on
> reading tasks and stdout on writing tasks, or we can reserve and
> handle differently "stdin" and "stdout" file names) ?
> If we agree on this design i can implement it.
Yep, if you're prepared to make the change then I'm more than happy to
accept a patch. "-" sounds like a good approach to me. I don't think
it would be a complicated change, but if you have any questions about
how osmosis is structured just ask.
More information about the dev