[OSM-dev] Minutely changeset feed

[Matt Amos]

On Fri, Nov 2, 2012 at 6:34 AM, Toby Murray <toby.murray at gmail.com> wrote:

> Since I have been playing with changeset metadata[1] a bit, Paul
> Norman recently pointed out to me that there are now minutely diffs of
> changesets available from planet.osm [2]. Would anyone care to share
> some juicy details like where it came from and how it works?
> Apparently it was mentioned in an EWG meeting. I should really check
> one of those out...
>
> Anyway, some observations/questions:
>
> I see the state file is in a different format than all the other
> replication state files. Probably not a big deal but is there value in
> consistency?
>

perhaps. ultimately, i'd like this to be rolled into the osmosis diff
production processes, but doing that now might cause some breakage. in the
interim, i thought a YAML file is probably standard enough that not
re-using the non-standard osmosis parser would be easy enough.


> Are there any existing tools to use this stream?
>

i don't know of any, but i'd be very happy to learn of new tools / support
in existing tools :-)


> It looks like changesets don't show up until they are closed. This
> makes sense since then you don't have to worry about information
> changing.
>

hmm... this shouldn't be the case - changesets should show up when they're
opened and again when they're closed. if they're opened and closed in the
same period, it will only show the closed state. this seems to be what
happens with most changesets - they open and close within a few seconds.


> However, what will it do in the edge case where a changeset is closed
> but then reopened? Not possible you say? I once had an upload take
> over an hour to process. It did eventually succeed but the changeset
> was marked as closed in my changeset list for several minutes before
> the upload finished and then it was suddenly open again. I am assuming
> in this case the changeset would show up twice in the minutely diffs
> and cause the INSERT queries of all consumers to explode violently :)
>

yeah... i don't think the code is able to handle this at the moment...
maybe it would see the re-opening, but probably not. any specific examples
of this would be great, btw, to help me start building up some test cases.

cheers,

matt



> [1] https://github.com/ToeBee/ChangesetMD
> [2] http://planet.openstreetmap.org/replication/changesets/
>
> Toby
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20121107/136ed233/attachment.html>