[OSM-dev] Compression types in PBF Format

[Stefan de Konink]

On Wed, 1 Dec 2010, Anthony wrote:

> On Tue, Nov 30, 2010 at 11:03 PM, Anthony <osm at inbox.org> wrote:
>> On Tue, Nov 30, 2010 at 8:29 PM, Stefan de Konink <stefan at konink.de> wrote:
>>> If any of gzip/bzip2/lzma in the general give better compression ratio's
>>> (20% smaller), then this compression scheme should become the default
>>> format.
>>
>> Depends on the performance.  If all you want is max compression
>> without regard to performance, you're almost surely better off using
>> raw and then compressing the entire file with LZMA (e.g. 7zip or xz).
>
> LZMA vs. zlib actually makes less of a difference than I thought it would:
>
> -rw-r--r-- 1 a a 103M 2010-12-01 08:07 florida.osm.bz2
> -rw-r--r-- 1 a a 129M 2010-12-01 08:32 florida.osm.gz
> -rw-r--r-- 1 a a  74M 2010-12-01 08:19 florida.osm.pbf
> -rw-r--r-- 1 a a 169M 2010-12-01 08:15 florida.osm.rawpbf
> -rw-r--r-- 1 a a  62M 2010-12-01 08:15 florida.osm.rawpbf.xz
> -rw-r--r-- 1 a a  86M 2010-11-25 11:29 florida.osm.xz
>
> I suspect it would make *much more difference* when it comes to the
> full history .osm, though.  Does PBF support full history files?  Does
> Osmosis?

Did you benchmark what pbf + lzma did or did you embed lzma in osmosis?


Stefan