[OSM-dev] Binary OSM; the first pass encoder

[Chris Jones]

Stefan de Konink wrote:
> Hi All,
>
>
> Because I am getting more and more disappointed with the current state 
> of affairs with respect to the downloading of OSM content some people on 
> the Dutch OSM IRC channel thought of an alternative way of distribution 
> that could potentionally get binary diffs after any possible download in 
> the past.
>
> I wrote the first implementation of it in the last couple of hours and 
> tested it on the Dutch dataset. The current gzip compressed data is 
> about 135MB. Extracted it represents 1.4GB of XML.
>
> The binary file is completely analogue to the XML, no shortcuts what so 
> ever. The first reduction to binary format containing only data reduced 
> the set to 418MB and allows a bzip2 compression to 78MB.
>   
How does gzip do on the binary format? or How does bzip2 do on the xml 
format.

You cant sensibly compare the two if you apply different compression 
methods.

Given how compression works I suspect you'd do better to tweak things 
like the block size used at compression time than come up with some 
binary format.

-- 
Chris Jones, SUCS Admin
http://sucs.org