[OSM-dev] OSM History Retriever

Brett Henderson brett at bretth.com
Wed Jul 21 14:42:11 BST 2010


I haven't looked at the full history dump to be honest so I'm not in a great
position to comment, but I'll comment anyway ;-)

I'm curious what the format of the full history dump is.  I'd like to
understand how nodes, ways and relations are represented in the file but I
can't do so without downloading the whole thing and decompressing it.

A full history dump would presumably require at least the addition of the
"visible" attribute to the standard set of XML attributes.  Currently
Osmosis has support for normal entity streams, change streams, and dataset
streams (random access to data, not in common use).  Entities with visible
attributes would require a new stream type which is not terribly difficult,
but requires a few new interfaces and task managers to be defined.

The bigger task is then writing tasks to support these new data types.  In
particular the existing --bounding-box task can't be used because it assumes
that only a single version of each entity exists, and that nodes reside in a
single location.  With full history files you need to take into account that
each way may refer to several different versions of nodes through time
depending on timestamp and that each version of a node might reside in a
completely different location.  It's not as simple as the current bounding
box task which just tracks which nodes it has included and then includes
ways which reference them.

I'll do my best to answer any questions if somebody wants to take this on
but it doesn't sound trivial.  Not much existing code could be re-used other
than the generic pipeline management.

On Wed, Jul 21, 2010 at 8:53 PM, Andy Allan <gravitystorm at gmail.com> wrote:

> On Wed, Jul 21, 2010 at 10:06 AM, Martijn van Exel <mvexel at gmail.com>
> wrote:
>
> >> Well we don't really want to be running that script lots of times for
> different extents either - the idea would be take the dump that produces and
> process it to produce subsets of the data as people do with the ordinary
> planet dumps.
> >>
> > Are there any existing tools that could do the processing though? Would
> osmosis for example be able to extract a bbox-defined subset of the
> history.osm file?
>
> Not as far as I'm aware, unless osmosis happens to magically work!
> It's the best tool for the job though, so I'd think some extra osmosis
> tasks (--read-history, --write-history) would be the best approach.
> I've no idea how much internal plumbing would be required though to
> support this - anyone want to comment?
>
> Cheers,
> Andy
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20100721/0fff7498/attachment.html>


More information about the dev mailing list