[OSM-dev] Deriving Change Sets
frederik at remote.org
Fri Jun 29 14:45:50 BST 2007
> I'm trying to derive a change set within a time interval using the
> history stored in a mysql database. The end purpose of this is to
> a mechanism for regular updates to be fed from the main osm
> database to
> other data destinations without requiring complete planet dumps.
This is something I would really love to see get off the ground. (I
remember an evening at the Essen meeting where I complained about the
weekly dump, and Nick Black said something like "well we could do it
daily", and I said "daily is not enough", and he went "well one could
do hourly with proper equiment" and I said "dumps, dumps, dumps, I
don't want no stupid dumps, I want live data..." - a discussion
ensued about what you'd possibly need live data for, but until today
I maintain that we should just provide data as live as possible
without asking what people want to use it for.)
But I always thought - as long as "near live" feeds are what one
wants - it would be much cheaper in terms of processing power to
simply log each change as performed by the API.
Your approach would be required if there were other ways to change
the data but in our situation where anything that changes data has to
go through rails anyway, why not have rails log these things and
simply process the log files?
That way you wouldn't have to ask the database for anything, and
unlike something based on DB replication or triggers, you would be
independent of the target system.
On the other hand your method probably has an advantage in cases
where the client has "lost" a few updates in between; it should be
easier to re-synchronize based on a database comparison like you're
aiming at than trying to figure out which bits of a serial stream of
change events have been missed.
Frederik Ramm ## eMail frederik at remote.org ## N49°00.09' E008°23.33'
More information about the dev