[OSM-dev] Compression types in PBF Format

[Anthony]

On Wed, Dec 1, 2010 at 10:24 AM, Stefan de Konink <stefan at konink.de> wrote:
> On Wed, 1 Dec 2010, Anthony wrote:
>
>>> Did you benchmark what pbf + lzma did or did you embed lzma in osmosis?
>>
>> xz uses lzma.  I made an uncompressed pbf file (florida.osm.rawpbf)
>> and then compressed it with xz (florida.osm.rawpbf.xz).  This isn't
>> the same as making a pbf file which uses lzma, but it should be a good
>> approximation of the compression achievable by embedding lzma in the
>> pbf.
>
> Yeah, but your lead basically shows we are talking about more than 10%...

Yeah, probably, but at the expense of more complicated code, greater
memory usage, etc.

I'm interested now in seeing how the full history compression goes,
though.  If it can achieve 70, 80, 90% on top of zlib, then it might
be worth embedding the compression as opposed to just using it for
transfer over the Internet.