[OSM-dev] Release of osmpbf version 1.2.1

marqqs at gmx.eu marqqs at gmx.eu
Tue Nov 1 14:48:49 GMT 2011


Hi Scott,

> The decoding loop of osmconvert scares me a bit.

Don't worry, it scares me too! :-)
The PBF DEcoding procedure is a section in the source code I'm really not proud of. It's the result of a couple of experiments. Meanwhile this procedure works quite reliable, but if I had to write it anew I surely would choose a different structure, maybe that one from my PBF ENcoder - although somewhat tells me that you might not like that code either.

> The reason I chose protobuf was to *avoid* anyone,
> including myself, from having to implement their own serialization
> code.

As you can see, this didn't help. ;-) Even if you try to encrypt the data, there always will be some strange guys who like to "understand" every bit in the file.

> How much benefit do you get from reading the raw bytes this way,
> versus going through google protobuf?

Well, to be honest: I never really tried to use google protobuf. Once my first attempt to link that stuff had failed, I decided to try it without it.

There appear to be two advantages:

First, the Users need not to care about the protobuf library. All they need is already available on their machine.

Second, the program seems to be significantly faster than any other program.

Scott, I really did not want to _complain_ about insufficient documentation. We all know that 99.9 % of OSM is a work of volunteers. Therefore I'm very happy that there are people like you who spend a lot of time and effort in contributing to the project!

Some even think that I would not like PBF format because I developed a different binary format. This also isn't the case. I developed my format for certain use cases, I never intended to completely replace PBF format. Fact is that I think highly of your format creation, it's great work!

> Also, if we're talking about PBF, do we want to open a conversation
> about OSM metadata, in both the PBF and XML formats?

In my opinion we first should define which (file level) meta data are needed. For example:

- border box
- timestamp min
- timestamp max
- writing program
- data source
- license
- ...

Then we could decide how to store them. There already are at least two different ways to store border boxes in XML. PBF (and .o5m) have a well defined format for storing certain meta objects - but not for all of them.

Regards
Markus



More information about the dev mailing list