[OSM-dev] Binary OSM; the first pass encoder
unix at rullzer.com
unix at rullzer.com
Wed Nov 12 11:41:54 GMT 2008
Hi,
Compression is always good when transporting large amounts of data. Binary
data is usually smaller than structured data. Combining the two is better
(like Stefan did). Since that way we get the smallest size.
So you are right to compare them the same compression should be used. But
it is still very clear that the binary format is much smaller than the one
in XML.
So your claim that it would be better to spend time on compression setting
does not really make sense to me. It seems hard to believe that we could
get a 50 MB smaller file by changing the compression settings.
But feel free to tweak the compression settings of course since that could
make it even smaller :)
--Roeland
> Stefan de Konink wrote:
>> Hi All,
>>
>>
>> Because I am getting more and more disappointed with the current state
>> of affairs with respect to the downloading of OSM content some people on
>> the Dutch OSM IRC channel thought of an alternative way of distribution
>> that could potentionally get binary diffs after any possible download in
>> the past.
>>
>> I wrote the first implementation of it in the last couple of hours and
>> tested it on the Dutch dataset. The current gzip compressed data is
>> about 135MB. Extracted it represents 1.4GB of XML.
>>
>> The binary file is completely analogue to the XML, no shortcuts what so
>> ever. The first reduction to binary format containing only data reduced
>> the set to 418MB and allows a bzip2 compression to 78MB.
>>
> How does gzip do on the binary format? or How does bzip2 do on the xml
> format.
>
> You cant sensibly compare the two if you apply different compression
> methods.
>
> Given how compression works I suspect you'd do better to tweak things
> like the block size used at compression time than come up with some
> binary format.
>
> --
> Chris Jones, SUCS Admin
> http://sucs.org
>
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>
More information about the dev
mailing list