[OSM-dev] OSM History Retriever

Brett Henderson brett at bretth.com
Thu Jul 29 11:56:38 BST 2010


Hi Martijn,

On Thu, Jul 29, 2010 at 7:02 PM, Martijn van Exel <mvexel at gmail.com> wrote:

> Hi Brett,
>
> What kind of trouble do you envisage when doing a bbox operation on a full
> history dump? I guess the movement of features over time makes an accurate
> determination of what is and what isn't in the bounding box less trivial,
> but for my purposes having all the historical data for which the current
> version is within the given bbox would be adequate.
>

This is educated guesswork because I haven't actually tried it ...

Yep, the movement of features over time is one issue.  The current bounding
box task will only send node versions through that are inside the bounding
box so it could potentially include some versions of a node and not others.
The next problem is that it will track all node ids that have been included
regardless of how many versions of the node were included.  These tracked
node ids are then used during way processing to determine which ways to
include.  The way processing won't take into account which version of a node
was included so some ways may be included that shouldn't be.  Finally the
relation processing may also have issues where relations are included that
point to ways for which only some versions have been included.  It might
mostly work because most data won't shift drastically during its lifetime
but there will be exceptions.

The bigger issue is that Osmosis has no concept of a "visible" attribute.
Osmosis deals with two main types of data, entity streams (ie. nodes, ways
and relations) and change streams (as per entity streams but with an action
create, modify or delete).  Entity streams are what is used for processing
osm files, and change streams deal with osc files.  The full history file
has much more in common with an osc file than an osm file.  The use of a
"visible" attribute is another way of representing what Osmosis already
represents using create, modify and delete actions on change streams.

Trying to use Osmosis to process these full history files as if they were
normal osm files may kind of work but entity streams are not designed to
work this way.  I think a better solution is to write a new XML change
reader for Osmosis that reads these full history files and uses the visible
attribute to determine if it's a create, modify or delete.  It can then
sends them into the pipeline as a normal change stream.  The next step after
that is to write a new bounding box task that can deal with change streams.

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20100729/9b1f66ae/attachment.html>


More information about the dev mailing list