[OSM-talk] Live Data - all new Data in OSM

[Ian Dees]

On Wed, May 13, 2009 at 2:41 AM, Tom Hughes <tom at compton.nu> wrote:

> Ian Dees wrote:
>
>  I'd like to continue this part of the thread. As was discussed by
>> Frederik, I think the end goal should be a real-time OSM stream of what's
>> getting applied to the database. Doing that in a performant way is
>> relatively difficult (which is why we're using Osmosis and minutely diffs
>> right now), but I think we should be striving for having a realtime XML
>> feed.
>>
>
> I have to say I don't see any great reason to strive for it.


Because it's there? Why are we striving to cover the globe with map data? :)


> I don't think anybody has ever given a use case which requires such a
> stream and can't work with the diffs.


I agree, but the point is that minutely-diffs are a minute old. At some
point in the future someone will want to see the data in real time as a
stream. The only reason I can currently think of is because they don't want
to have to deal with downloading the minutely diffs and would rather read a
stream of XML messages, applying each one to their database somehow as they
came in.

Given that such a stream is uncacheable (and hence requires much higher
> bandwidth outgoing from the core servers)


The stream would be uncacheable, but could be repeated by others outside of
the core server so that the bandwidth load was spread amongst the community.


> and much more fragile than the diffs, it is not obvious that we should put
> what would undoubtedly be a huge amount of effort into creating and
> maintaining such a system rather than into doing other things.


Ok, this I'll agree on. My original post was just to talk about it... not
really to do it. But it sounds like we should take "baby" steps. Let's work
on the minutely diffs first and if some crazy person comes up with a good
use case for streaming, we can talk about it then.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk/attachments/20090513/0e2dd9ac/attachment.html>