[OSM-dev] full history files on Geofabrik server

[Sebastian Kürten]

Hi,

I took this as an opportunity to check osm4j's support for history
files. From what I understand the main difference to ordinary data files
is that elements with the same id appear multiple times and they carry
the 'visible' attribute. During testing I noted that I would not find
any nodes with the visibility set to false, which appears strange to
me? Can this be true, or is there likely something wrong with either
the data or the code that processes it?

I've set up a minimal example that counts and sums up the visibility of
each entity type in Bremen:

https://github.com/topobyte/osm4j-examples/blob/master/src/main/java/de/topobyte/osm4j/examples/history/HistoryVisibilityStats.java

Here's the output:

    nodes: 2389608 visible, 0 invisible
    ways: 729714 visible, 27604 invisible
    relations: 67500 visible, 1632 invisible

The data file used is this:
http://download.geofabrik.de/europe/germany/bremen.osh.pbf

Thank you for any hints,
Sebastian

On Mon, 20 Feb 2017 01:18:00 +0100
Frederik Ramm <frederik at remote.org> wrote:

> Hi,
> 
>    "full history" files are kind of a niche interest, but they can be
> useful to analyze some things like "how many different people have
> added a certain tag", or "how has an object evolved over time", and
> so on. They can also be used to reconstruct the complete data set for
> any given timestamp in the past (minus, of course, redactions).
> 
> I'm happy to announce that the Geofabrik download server now has full
> history files for all regions served. These files (called ".osh.pbf")
> can for example be processed with the Osmium library and the osmium
> command line tool. If you want to run analyses on them without C++
> coding, Osmium can convert them to a text-based format ("OPL") that is
> then accessible for grep et al.
> 
> The full history files will be updated weekly.
> 
> Bye
> Frederik
>