[OSM-dev] Simpler binary OSM formats

[andrew byrd]

On Wed, Apr 29, 2015, at 07:03, Paul Norman wrote:
> How does it work with diffs and history files?

These features are not implemented, as this is a prototype format whose
main purpose so far is to seed a discussion. The way OSM diffs are
usually handled (full information is given for modified entities,
overwriting the existing entities) I am confident that the format would
require only minor amendments to handle diffs.

> PBF comes with relatively common support and libraries with a
> reasonable interface for most languages. How is this with your
> proposed format?

Being merely a proposal at this point, there is no support for VEX
beyond a prototype implementation in our OSM loading library. However
you can see from that example that the code to read and write the format
is quite simple. If I see interest in a more polished version of the
format, I would of course port the library to some common languages.

> Parse times for an extract with osm2pgsql are 13s for PBF (334 MB) and
> 18s for o5m (698 MB). I don't have any large o5m files sitting around
> so I can't check for a larger extract.

You are right to emphasize processing speed, especially considering the
huge size of many OSM data sets. I will need to do a follow up
addressing speed and memory usage. However, the real motivation for my
work on the VEX format was to aim for _simplicity_ while maintaining
speed and size at least as good as PBF. Anecdotally, for certain
operations in a C-language utility I was seeing processing speeds about
2x those for PBF. I will need to test that methodically.

-Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20150429/ffb7825d/attachment.html>