[OSM-dev] Writing PBF

[marqqs at gmx.eu]

Hi Peter & Jochen,

> I encountered shrinkage of around 1-2% of the overall block size while 
> comparing the intermediate string ids with the sorted ones.

sounds plausible. Sorting the strings might not be worth the effort if circumstances allow you to increase block size. I just ran a comparison (File germany.pbf had been downloaded from Geofabrik at November 22):


$ time ./osmconvert germany.pbf -o=germany_oc.pbf
real	2m32.438s

$ ls -l germany*
-rw------- 1 pt pt 1089521558 2011-12-13 18:53 germany_oc.pbf
-rw-r--r-- 1 pt pt 1150359237 2011-11-22 04:02 germany.pbf


Result:
8000 objects/block (download file)
-> n objects/block, but 30..31 MB/block (osmconvert)
saves about 5% file size.

Markus


-------- Original-Nachricht --------
> Datum: Tue, 13 Dec 2011 16:39:00 +0100
> Von: "Peter Körner" <osm-lists at mazdermind.de>
> An: dev at openstreetmap.org
> Betreff: Re: [OSM-dev] Writing PBF

> Am 13.12.2011 15:04, schrieb Jochen Topf:
> > Has anybody any data on how much smaller the files get with sorted vs.
> unsorted
> > string table? Sorting the string table means there is a lot more work
> and a lot
> > more main memory needed for the PBF writer.
> I encountered shrinkage of around 1-2% of the overall block size while 
> comparing the intermediate string ids with the sorted ones.
> 
> Peter
> 
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev