[OSM-dev] Hourly diffs are missing edits (too)

[Tom Hughes]

On 07/10/09 09:35, Frederik Ramm wrote:

>      so we know that the minutely diffs are broken (and we have created
> the minutely-replicate diffs instead; something that I had completely
> missed until it was mentioned casually a few days ago - can osmosis
> --rci be run on them or do they require special treatment?).

I think there's a different switch for handling them.

The reason they haven't been publicised widely is that Brett had only 
just got them going when he went away on holiday so they're a bit 
betaish at the moment.

> But until now I had assumed that at least the hourly diffs were correct
> which probably was naive - I guess that it is just *less likely* for
> them to miss out on something than the minute diffs, but not impossible?

The duration the diff covers isn't really the issue - the issue is how 
long we wait after the end of the period the diff covers before we 
generate it. So it we wait until 0300 to generate the 0100-0200 diff 
then is will probably contain almost everything but if we generate it at 
0201 then it probably won't.

Essentially if any transaction that started before the end of the period 
the diff covers is still running when the the diff is created then 
changes in that transaction may be missed out as they won't have been 
committed yet.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/