[OSM-dev] Call to mobile developers: OSM binary file format

[Stefan de Konink]

On Fri, 8 Aug 2008, Marcus Wolschon wrote:

> On Fri, 8 Aug 2008 10:34:54 +0200 (CEST), Stefan de Konink <skinkie at xs4all.nl> wrote:
> > A binary XML format would be better to parse for an XML processor. But OSM
> > data is pretty structured. The point is that for line transfers a pretty
> > efficient format can be choosen that is totally unparsible by embedded
> > devices.
>
> Well, if it's binary it's no longer XML. wbXML is a binary encoding of XML
> but it is no longer XML and it's parser is no longer an XML-parser.

binary XML is a standard that takes XML trees and converts them to
something non human-readible, but efficient machine readible it *IS* XML.
(With all other XML niceless)

> > Since the subject suggest mobile developers; they might only be interested
> > in a derrivative format that even *not* includes some information, and has
> > a high focus on routing. Now I was always teached that if something
> > focusses on one thing it will need to make compromises.
>
> So, do we agree that the focus is routing and navigation -software?
> If so, we could move on to finding out what data we actually need to store
> and what we can leave out.
> Once we know what we are storing, we can start on the "how" to store, retrieve
> search, merge and update it.

See my other email.

> > Binary XML is cute if there is an efficient parser. But I presume it would
> > work as great as any other database format specification. I wonder if even
> > OSM in SQLite will give better line speeds.
>
> You got me confused.
> What do database-formats have to do with transfer-formats?
> You would never use wbxml for storage on a mobile device (you have to load it
> into ram and decode instead of indexing into the file) and databases
> not for transfer.

It is all about structuring information. Since you could even claim that
network lines work like old fashion 'tapes' it makes sense to have
geographical 'close' information to be close in the information stream. If
you choose an efficent storage method, or something that is general enough
to store information, it is structured somehow, efficient for lookup
properties for example. While reinventing the wheel looks great, it will
be only more efficient if it is tied specifically to your applications
memory layout, for any other format the lookup is the most important
question.

So by default; if you don't want to invent a format + processing, take a
look to formats that are already efficient in processing and take their
format. If you don't care about formats, ignore everone else and write
something to import the *default* osm-xml, and output it to your
application data structures.


Stefan