[OSM-dev] Minute Diffs Broken

[Brett Henderson]

Brett Henderson wrote:
> Tom Hughes wrote:
>> Brett Henderson wrote:
>>
>>> I've created a report at:
>>> http://planet.openstreetmap.org/minute/audit/report-2009-05-05_11-23.txt 
>>>
>>>
>>> Some of the impacted changesets are:
>>> 1081602
>>
>> OK. That one (which has 13225 nodes and 1810 ways) took 566.46090 
>> seconds to process, which is just short of 10 minutes.
>>
>> > 1080775
>>
>> That took 1380.05972 seconds, or 23 minutes. It only has 1402 nodes, 
>> 265 ways and 11 relations, which makes it quite surprising that it 
>> took so long.
>>
>> > 1077469
>>
>> That is a Potlatch changeset, so didn't use bulk upload as such and I 
>> don't think I have any way to find out how long any given edit to it 
>> took.
> Okay, thanks.  I've run out of time to do much this evening but I'll 
> try to rig up an audit process tomorrow.  I'm not sure what I should 
> do next though.  Any suggestions?  I'm happy to sit back and wait if 
> necessary :-)
It didn't take as long as I thought.  This directory now contains 
reports that run at 50 minutes past every hour.
http://planet.openstreetmap.org/minute/audit/

It will only include problems detected since the last invocation so 
they'll often (I hope ...) be 0 bytes in size.  It keeps track of what 
it's already detected by creating a <diff>-broken file in the actual 
minute directory where <diff> is the name of the diff with missing data.

For curiosity's sake I'm also auditing the minute-fast diffs which are 
trailing the API by 1 minute.  It is missing significantly more data.
http://planet.openstreetmap.org/minute-fast/audit/

The audit is comparing results against yet another minute process 
running 30 minutes behind the API so it is possible that I'm missing 
transaction that take longer than 30 minutes to complete.  I'm hoping 
there aren't too many of those!

Brett