[OSM-dev] Simpler binary OSM formats

Andrew Byrd andrew at fastmail.net
Sat Feb 6 21:35:27 UTC 2016


Hello OSM developers,

Last spring I posted an article discussing some shortcomings of the PBF format and proposing a simpler binary OSM interchange format called VEX. There was a generally positive response at the time, including helpful feedback from other developers. Since then I have revised the VEX specification as well as our implementation, and Conveyal has been using this format in our own day-to-day work.

I have written a new article describing of the revised format:
http://conveyal.com/blog/2016/02/06/vex-format-part-two <http://conveyal.com/blog/2016/02/06/vex-format-part-two>

The main differences are 1) it is more regular and even simpler to parse; and 2) file blocks are compressed individually, allowing parallel processing and seeking to specific entity types. It is no longer smaller than PBF, but still comparable in size.

Again, I would welcome any comments you may have on the revised format and the potential for a shift to simpler binary OSM formats.

Regards,
Andrew Byrd


> On 29 Apr 2015, at 01:35, andrew byrd <andrew at fastmail.net> wrote:
> 
> Hello OSM developers,
>  
> Over the last few years I have worked on several pieces of software that consume and produce the PBF format. I have always appreciated the advantages of PBF over XML for our use cases, but over time it became apparent to me that PBF is significantly more complex than would be necessary to meet its objectives of speed and compactness.
>  
> Based on my observations about the effectiveness of various techniques used in PBF and other formats, I devised an alternative OSM representation that is consistently about 8% smaller than PBF but substantially simpler to encode and decode. This work is presented in an article at http://conveyal.com/blog/2015/04/27/osm-formats/ <http://conveyal.com/blog/2015/04/27/osm-formats/>. I welcome any comments you may have on this article or on the potential for a shift to simpler binary OSM formats.
>  
> Regards,
> Andrew Byrd
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20160206/fb18d653/attachment.html>


More information about the dev mailing list