[OSM-dev] Writing PBF

[marqqs at gmx.eu]

Hi Jochen,

> Osmium now checks whether the buffer is 95% full and
> then writes it out (the 95% should leave more than enough space for the
> string table).

This is great news, thanks!
If you limit the number of objects only to maximum block size (32 MiB), you will get smaller PBF files. I noticed this effect when writing the PBF-write module for osmconvert. The files are much smaller although I did not care about sorting the string table (which had been recommended by PBF format description).

Markus

-------- Original-Nachricht --------
> Datum: Tue, 13 Dec 2011 11:46:34 +0100
> Von: Jochen Topf <jochen at remote.org>
> An: dev at openstreetmap.org
> Betreff: [OSM-dev] Writing PBF

> Hi!
> 
> The PBF format has maximum block sizes. Osmium used to just write 8000
> objects
> into each block and hope that there is enough space. Turns out that there
> are
> cases where that doesn't work. If there are a lot of very large relations,
> the
> block can overflow. Somebody noticed that problem and Peter Körner fixed
> it
> in Osmium (Thanks!). Osmium now checks whether the buffer is 95% full and
> then writes it out (the 95% should leave more than enough space for the
> string table). Its still not a perfect strategy, you could create an OSM
> file
> that would overflow the buffer, but it should not happen in practice.
> 
> So if you see this problem, upgrade to a newer version of Osmium.
> 
> And if you have implemented a PBF writer, you should probably check
> whether
> it suffers from the same or similar problem.
> 
> Jochen
> -- 
> Jochen Topf  jochen at remote.org  http://www.remote.org/jochen/ 
> +49-721-388298
> 
> 
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev