[OSM-dev] Minute Diffs Broken

[Brett Henderson]

Tom Hughes wrote:
> Brett Henderson wrote:
> I have to say I don't fully understand what the issue you're seeing is 
> as I only skimmed the vast amount of discussion that went on overnight.
>
> Are you really seeing a five minute delay? Or just a delay that 
> happens to be over a five minute boundary?
>
> In other words are you saying that an edit timestamped at 17:50:00 is 
> still not visible in the database at 17:55:00? Or just that you look 
> at the database at 17:55:00 and don't see an edit which is timestamped 
> at 17:54:50 which then appears when you look again at 18:00:00?
It's the first of the two.  I'm seeing a full five minute delay.  At 
17:55:00 I will query for data in the period 17:49:00 to 17:50:00.  Some 
large changesets still haven't committed in this 5 minute period.
>
> It's certainly possible that a very large changeset might take some 
> minutes to process I guess - if you can give me a specific changeset 
> number from the last few days then I can have a look and see how long 
> it took.
I've created a report at:
http://planet.openstreetmap.org/minute/audit/report-2009-05-05_11-23.txt

Some of the impacted changesets are:
1081602
1080775
1077469

I'm going to try to set up an automated audit process producing reports 
like the above.  I'm not sure if I'll get it done tonight though.
>
>> 2. What appear to be the current system bottlenecks?  Is the database 
>> already approaching processing capacity or is rails the limiting factor?
>
> Nothing specific in terms of the database - that largely seems to be 
> fine. The issues are, as always, mostly about rails and the amount of 
> memory it uses and getting the daemons to restart cleanly when they 
> get too big.
Great.