[OSM-dev] Timestamp in PBF files

[Scott Crosby]

On Tue, Nov 20, 2012 at 1:09 PM, Jochen Topf <jochen at remote.org> wrote:

> On Tue, Nov 20, 2012 at 06:57:50PM +0100, Stephan Knauss wrote:
> > Frederik Ramm writes:
> >
> > >I really don't mind *how* it's done but I would really love to
> > >have one agreed way to place a timestamp in a PBF instead of
> > >everyone rolling their own.
> >
> > I would prefer epoch timestamps. That's a widely accepted way of
> > storing time information without the need to worry about time zones
> > and such.
>
> The other timestamps in PBF files (at all the objects) use 64 bit integers
> with
> seconds since epoch. So it would make sense to use the same format.
>
>
Not quite. The granularity of timestamps can go down to the milliseconds.

https://github.com/DennisOSRM/OSM-binary/blob/master/src/osmformat.proto#L96


> You can have optional fields in protobuf, but unfortunately this doesn't
> help us in this case. There are two ways nodes can be stored in PBF files:
> as a series of "Node" objects or as "DenseNode" objects. "Node" objects
> have required fields "lat" and "lon". We could change this to be optional.
> There would be a has_lat() or has_lon() call to check for this.
>
> Unfortunately in most cases the more space efficient "DenseNode" objects
> are
> used. In this case the latitude and longitude of all nodes of a block are
> stored in a special delta encoding. This doesn't allow for optional fields.
> As far as I can see we could either add a boolean for each node in a block
> that
> defines whether the coordinate field is valid or use a special value for an
> invalid coordinate.


Correct. There is no way in the current DenseNodes format to encode 'no
value' for a latitude or longitude. Changing the message buffer to include,
(say) a boolean array for the hasLatitude()/hasLongitude() would be a
breaking format change, and would add about 18-40 bytes to each block of
8000 nodes.

How many nodes in the planet lack a latitude or longitude? Using a MAXINT
encoding will cost about 8 bytes for each missing latitude or longitude.
 It's possible to reduce this to 2-3 bytes, but the format gets
uglier/hackier. IMHO, probably not worth that cost.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20121120/7446560e/attachment-0001.html>