[OSM-dev] Slow Osmosis

[Brett Henderson]

Frederik Ramm wrote:
> Hi,
>
> Andreas Kalsch wrote:
>   
>> Osmosis is very slow on my server.
>>     
>
> The Java implementations of gzip/bzip are notoriuosly slow. If you are 
> working with compressed data, you might see an improvement if you first 
> uncompress the file and then use osmosis to process it in raw form; 
> later use an external utility to compress the output if applicable.
>   
The gzip implementation is pretty fast.  In fact during some testing the 
other day it seemed to only add approximately 10% overhead on top of 
normal xml processing in uncompressed form and has the advantage of 
considerably less disk activity.  But yes, bzip is horrendously slow and 
is much faster to use an external program.

It's worth noting that you can pipe direct from native bzip to osmosis 
like this to avoid uncompressed files on disk:
bzcat myfile.osm.bz | osmosis --read-xml - --write-xx...