[OSM-dev] PBF for History-Files (was: visible-Flag in PBF)

Jochen Topf jochen at remote.org
Sun May 8 10:50:32 BST 2011

Thinking about the inclusing of the invisible-Flag I think we have to step
back a bit:

There are two use-cases: One is reading an OSM file with no history
information. Thats what most people do. The other is reading an OSM file with
full (or in the future maybe partial) history information. Those are two very
different things and the application has to be aware of which kind of file it
is reading. History information does not just mean that there is now an
invisible flag, it means the other data has to be interpreted differently. For
instance the ID of an object is no longer unique.

So it doesn't make any sense to have a normal OSM file with optional history
information that some applications would read and others don't. I think an
approach with blocks that interleave current information with historic
information which would allow a reader that only nows about current OSM data to
skip some blocks, doesn't make sense.

In a way "current OSM" and "history OSM" are two very different formats. They
share ther basic building blocks, like id, version, timestamp, etc. plus one
visible flag thats only needed for the history OSM. But the semantics of this
data is different.

We could now have to different kinds of blocks for every object type. One for
current OSM data and one for historic OSM data. The only difference would be
the existence of the invisible flag. No PBF file would ever contain both types
of these blocks. But that seems like a lot of overhead. Every reader library
now has to parse both those kinds of blocks although they are nearly the same.
If we just add the visible flag the low-level reader libraries for current
and historic OSM data are the same, only upper levels of an applications
have to take these differences into account.

So I propose the following:

1. Add the visible flag as I have already proposed:
--- a/src/osmformat.proto
+++ b/src/osmformat.proto
@@ -126,6 +126,7 @@ message Info {
    optional int64 changeset = 3;
    optional int32 uid = 4;
    optional uint32 user_sid = 5; // String IDs
+   optional bool visible = 6 [default = true];
 /** Optional metadata that may be included into each primitive. Special dense format used in DenseNodes. */
@@ -135,6 +136,7 @@ message DenseInfo {
    repeated sint64 changeset = 3 [packed = true]; // DELTA coded
    repeated sint32 uid = 4 [packed = true]; // DELTA coded
    repeated sint32 user_sid = 5 [packed = true]; // String IDs for usernames. DELTA coded
+   repeated bool visible = 6 [packed = true];

2. Add some kind of type field that says "This is an OSM file with current data"
or "This is an OSM file with historic data". (We should also think about adding
a "diff" file type, which is also needed, but I'll leave that for a different

I am not sure whether this type field should just be a "required_feature" or
something else. Its not really a feature. But it would make sense to use this

The PBF documentation would then explain that your are not supposed to set the
visible flag on "current OSM" files and have to ignore its setting when reading
those, but you have to set and read it for "historic OSM" files.

Jochen Topf  jochen at remote.org  http://www.remote.org/jochen/  +49-721-388298

More information about the dev mailing list