[OSM-dev] TRAPI status

[Lars Francke]

>> I started working on a streaming XML output plugin for Osmosis. I was
>> intending to take advantage of PuSH/PubSubHub messaging and maybe even XMPP
>> (so that you get a 1-min delayed IM when someone changes something in your
>> bbox).
>> Anyway, TRAPI could use this same plugin to apply updates to their
>> database.
>
> I've also spent a fair bit of time thinking about this type of thing.  When
> I first started work on the replication diffs I had in mind a server-side
> daemon (using Osmosis internally) that would push changes to all connected
> clients.  It would allow a client to connect, specify which replication
> number it was up to, receive all updates in a single stream, then continue
> to receive live changes as they occurred.

These are things we talked about at the recent FOSSGIS conference. I
planned to do a more detailed write-up but I don't know when I'll get
to that so this is not finished. We too agreed that some kind of
streaming/subscription to changesets would be a good idea. Our focus
though was on the german dev servers so that not everyone would need
to write an .osc parser and download the changeset files etc.

> Several issues preventing this are:
> 1. Writing it.  I don't have much time to do it therefore I went with the
> simplest approach that would work at the time.

I'm spending some time on something like this as part of the OSMdoc rewrite.

> 2. Cacheability.  A push approach is non-cacheable and therefore would
> increase the amount of bandwidth required.  I suspect diff updates are small
> compared with planet downloads though so perhaps this is a non-issue.

I'd just don't care about that. We have enough resources. So I agree
that this is a non-issue.

> But I would really like to see it happen :-)

I currently use AMQP (RabbitMQ) for message processing and it works
very well. It is very flexible and it'd be easy to extend it with a
PubSubHubBub or XMPP output.

Mitja (of OpenStreetBugs) proposed just yesterday a filter that
filters changes by the tags/changes involved so it would be very easy
to subscribe to only the events you are interested in. This can be
implemented in just a few lines of code.

While we only thought about doing this on the german dev servers I've
since gotten multiple requests/questions and suggestions that this
should be integrated into the main OSM site. All that'd be needed
would be a call in the Ruby API that sends a message (asynchronously,
very fast) once a change has been made. This would make the generation
of the diff files a lot easier and everything more flexible. I haven't
yet asked anyone if this would be a possible. I know that this isn't
the right topic (although the TRAPI could also use this system) but I
wanted to take the opportunity to inform about our ideas.

Cheers,
Lars