[Tilesathome] Inconsistent data from OSMXAPI

[Brett Henderson]

That problem has been created by potlatch.  It's a known issue that 
RichardF is working on but I think he's been struggling to get to the 
bottom of it (haven't heard an update for a while).  If you look at the 
times you'll see that the node was deleted 11 seconds prior to the way.  
It appears to be triggered by slow network links where potlatch updates 
take some time to reach the main db.  I don't know if potlatch is doing 
referential integrity checks on the client or server ...

There are plenty more errors like it in the main db.  This link is a 
referential integrity report I ran a couple of weeks ago.  It's 43K 
gzipped and contains over 27545 records.
http://www.bretth.com/osm/integrity-report-20071106.csv.gz

Brent Easton wrote:
> Hi 80n,
>
> Comments below,
>
> Regards,
> Brent.
>
> *********** REPLY SEPARATOR  ***********
>
> On 20/11/2007 at 8:21 AM 80n  wrote:
>
>   
>> On Nov 20, 2007 1:28 AM, Brent Easton <b.easton at exemail.com.au> wrote:
>>
>>     
>>> I am seeing inconsistent data being returned by OSMXAPI when doing
>>> lowzooms. Ways are referencing nodes that are not included in download.
>>>       
>> When
>>     
>>> you go an view the actual data, that node no longer exists. This seems to
>>> indicate that it is some sort of timing issue. The end result is that
>>>       
>> Batik
>>     
>>> crashes trying to render the lowzoom tiles.
>>>
>>>       
>> Osmxapi *should* be consistent providing its sources are consistent.  I
>> know
>> planet.osm is not guaranteed to be consistent but I think that the planet
>> diffs are supposed to be.  Brett can you confirm that?
>>
>> I ran a full consistency check on osmxapi just a couple of days ago.  At
>> that point it seemed ok.  Brent, can you give me some example way Ids so
>> that I can track down where the problem came from please?
>>     
>
>
> Way 5403535 is good one to start, while trying to run
>
> perl tilesgen.pl xy 235 150 8
>
> Generates
>
>   <way id='5403535' timestamp='2007-09-02T10:39:00+01:00'>
>     <nd ref='35298243'/>
>     <nd ref='38879056'/>
>     <nd ref='38879058'/>
>     <nd ref='38879060'/>
>     <nd ref='38879063'/>
>     <nd ref='38879066'/>
>     <nd ref='38879069'/>
>     <nd ref='38879072'/>
>     <nd ref='38879076'/>
>     <nd ref='38879079'/>
>     <nd ref='38879084'/>
>     <nd ref='38879088'/>
>     <nd ref='38879091'/>
>     <nd ref='38879093'/>
>     <nd ref='38879096'/>
>     <nd ref='38879098'/>
>     <nd ref='38879100'/>
>     <nd ref='38879103'/>
>     <nd ref='38879107'/>
>     <nd ref='38879111'/>
>     <nd ref='38879115'/>
>     <nd ref='38879119'/>
>     <nd ref='38879123'/>
>     <nd ref='38879127'/>
>     <nd ref='38879132'/>
>     <nd ref='38879137'/>
>     <nd ref='38879139'/>
>     <tag k='highway' v='secondary'/>
>     <tag k='ref' v='15'/>
>   </way>
>
> in the data-pid-4.osm file, but all except the first node do not appear in the file.
>
>
>
>   
>>> Is this a known issue? I am presuming the planet.osm dump is
>>>       
>> inconsistent.
>>     
>>> Where should we look at solving this problem :-
>>>
>>> 1. Planet.osm dump is inconsistent, fix that?
>>>
>>>       
>> Osmxapi uses planet diff files so fixing planet.osm won't help much.
>>
>>
>>     
>>> 2. Fix OSMXAPI to not emit nodes in ways that have not been included in
>>> the download file?
>>>       
>> It's intentional behaviour that if the ways references a non-existant node
>> then the node will not be in the output, but the way will still contain the
>> offending <nd> tag.  It didn't seem to make sense to remove it as the
>> client
>> cannot then detect that the way is incomplete.  The client should choose
>> how
>> to process such error situations, although this fact should be mentioned in
>> the docs.
>>
>>
>>     
>>> 3. Have Osmarender ignore undefined nodes? Is this possible with XSLT?
>>>
>>>       
>> It used to be able to deal with missing nodes, but maybe this broke during
>> the 0.4 to 0.5 migration.
>>     
>
> The problem is not so much Osmarender as Batik. Inkscape handle errors by ignoring them, Batik fails.
> I will see if I can see anything in old versions of the source and also run it past Dodi.
>
> Thanks,
> Brent.
>
>
> ____________________________________________________________
> Brent Easton                       
> Analyst/Programmer                               
> University of Western Sydney                                   
> Email: b.easton at uws.edu.au
>
>