[OSM-dev] Planet diff's revisited

[Brett Henderson]

Thanks for getting this started.  Not only do weekly diffs save space 
and bandwidth, they can be used more effectively for maintaining copies 
of OSM data.  Weekly diffs are a good start for getting people used to 
the idea of deltas and hopefully working on tools to consume them.  The 
next step is to reduce the interval between diffs.

Sebastian Spaeth wrote:
> I won't be able to do daily diff's though as I still would have to work 
> from the weekly planet dump Steve generates. I wonder if you could have 
> some MySQL queries (WHERE modified_ago < 24h) which would enable you to 
> do "diff dumps". But I am not going to look into that myself.
>   
I am actively working on deltas directly from the database at the 
moment.  This problem is exactly what Osmosis is attempting to solve.

My aim is to do a lot better than daily dumps, although daily would be a 
good start.  Osmosis should scale down to very small intervals such as 1 
per minute if necessary.

If you can give me a bit longer I should have something working in the 
next week or so.  Query tuning is the final hurdle.  Some additional 
indexing on the timestamp columns of the database will be required.
>
> What do people think? Would diffs be useful? SHould we start providing 
> them? In addition to full planets or rather instead of some of the older 
> ones?
>
> Thanks to Jon for creating the planetdiff tool though, in any case. It 
> is a nice piece of work (although the UTF8 sanitizer could probably be 
> removed now that we have proper UTF8 data).
>   
Jon and I probably should talk, we've created very similar delta file 
formats.  If we could agree on a format the two tools could be used 
interchangeably, Jon's when processing raw planet files (I believe it 
will be quicker than Osmosis), Osmosis when producing diffs directly off 
a database.
> spaetz
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
>