[OSM-dev] Deriving Change Sets

Frederik Ramm frederik at remote.org
Fri Jun 29 14:45:50 BST 2007


> I'm trying to derive a change set within a time interval using the
> history stored in a mysql database.  The end purpose of this is to  
> build
> a mechanism for regular updates to be fed from the main osm  
> database to
> other data destinations without requiring complete planet dumps.

This is something I would really love to see get off the ground. (I  
remember an evening at the Essen meeting where I complained about the  
weekly dump, and Nick Black said something like "well we could do it  
daily", and I said "daily is not enough", and he went "well one could  
do hourly with proper equiment" and I said "dumps, dumps, dumps, I  
don't want no stupid dumps, I want live data..." - a discussion  
ensued about what you'd possibly need live data for, but until today  
I maintain that we should just provide data as live as possible  
without asking what people want to use it for.)

But I always thought - as long as "near live" feeds are what one  
wants - it would be much cheaper in terms of processing power to  
simply log each change as performed by the API.

Your approach would be required if there were other ways to change  
the data but in our situation where anything that changes data has to  
go through rails anyway, why not have rails log these things and  
simply process the log files?

That way you wouldn't have to ask the database for anything, and  
unlike something based on DB replication or triggers, you would be  
independent of the target system.

On the other hand your method probably has an advantage in cases  
where the client has "lost" a few updates in between; it should be  
easier to re-synchronize based on a database comparison like you're  
aiming at than trying to figure out which bits of a serial stream of  
change events have been missed.


Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00.09' E008°23.33'

More information about the dev mailing list