[Tilesathome] Inconsistent data from OSMXAPI
80n
80n80n at gmail.com
Tue Nov 20 18:51:10 GMT 2007
On Nov 20, 2007 11:30 AM, Brent Easton <b.easton at exemail.com.au> wrote:
> Thanks Brett,
>
> That makes sense. Ultimately we will want to clean up the DB. In the
> meantime, looks like 2 approaches:-
>
> 1. Stop OSMXAPI from emitting non-existent nodes in ways. Since we know
> that it represents a DB inconsistency, I think it is reasonable that xapi
> suppress them rather than emitting them. While we're at it, I think xapi
> should also suppress ways that contain only one node. Is the source for
> osmxapi in the SVN somewhere?
>
IMHO Osmxapi shouldn't do this. It should mimic as exactly as possible what
the main API does. If the main api stops emitting non-existent nodes then
osmxapi will also.
>
> 2. Have Osmarender filter out the extraneous nodes. Dodi, can we do this
> in Osmarender.xsl?
>
Osmarender should do this. It should be tolerant of missing nodes and make
an appropriate decision when it encounters this case. It should certainly
alway produce valid SVG so as to not break Batik or anything else.
>
> Regards,
> Brent.
>
>
>
> *********** REPLY SEPARATOR ***********
>
> On 20/11/2007 at 9:55 PM Brett Henderson wrote:
>
> >That problem has been created by potlatch. It's a known issue that
> >RichardF is working on but I think he's been struggling to get to the
> >bottom of it (haven't heard an update for a while). If you look at the
> >times you'll see that the node was deleted 11 seconds prior to the way.
> >It appears to be triggered by slow network links where potlatch updates
> >take some time to reach the main db. I don't know if potlatch is doing
> >referential integrity checks on the client or server ...
> >
> >There are plenty more errors like it in the main db. This link is a
> >referential integrity report I ran a couple of weeks ago. It's 43K
> >gzipped and contains over 27545 records.
> >http://www.bretth.com/osm/integrity-report-20071106.csv.gz
> >
> >Brent Easton wrote:
> >> Hi 80n,
> >>
> >> Comments below,
> >>
> >> Regards,
> >> Brent.
> >>
> >> *********** REPLY SEPARATOR ***********
> >>
> >> On 20/11/2007 at 8:21 AM 80n wrote:
> >>
> >>
> >>> On Nov 20, 2007 1:28 AM, Brent Easton <b.easton at exemail.com.au> wrote:
> >>>
> >>>
> >>>> I am seeing inconsistent data being returned by OSMXAPI when doing
> >>>> lowzooms. Ways are referencing nodes that are not included in
> download.
> >>>>
> >>> When
> >>>
> >>>> you go an view the actual data, that node no longer exists. This
> seems
> >to
> >>>> indicate that it is some sort of timing issue. The end result is that
> >>>>
> >>> Batik
> >>>
> >>>> crashes trying to render the lowzoom tiles.
> >>>>
> >>>>
> >>> Osmxapi *should* be consistent providing its sources are consistent.
> I
> >>> know
> >>> planet.osm is not guaranteed to be consistent but I think that the
> >planet
> >>> diffs are supposed to be. Brett can you confirm that?
> >>>
> >>> I ran a full consistency check on osmxapi just a couple of days ago.
> At
> >>> that point it seemed ok. Brent, can you give me some example way Ids
> so
> >>> that I can track down where the problem came from please?
> >>>
> >>
> >>
> >> Way 5403535 is good one to start, while trying to run
> >>
> >> perl tilesgen.pl xy 235 150 8
> >>
> >> Generates
> >>
> >> <way id='5403535' timestamp='2007-09-02T10:39:00+01:00'>
> >> <nd ref='35298243'/>
> >> <nd ref='38879056'/>
> >> <nd ref='38879058'/>
> >> <nd ref='38879060'/>
> >> <nd ref='38879063'/>
> >> <nd ref='38879066'/>
> >> <nd ref='38879069'/>
> >> <nd ref='38879072'/>
> >> <nd ref='38879076'/>
> >> <nd ref='38879079'/>
> >> <nd ref='38879084'/>
> >> <nd ref='38879088'/>
> >> <nd ref='38879091'/>
> >> <nd ref='38879093'/>
> >> <nd ref='38879096'/>
> >> <nd ref='38879098'/>
> >> <nd ref='38879100'/>
> >> <nd ref='38879103'/>
> >> <nd ref='38879107'/>
> >> <nd ref='38879111'/>
> >> <nd ref='38879115'/>
> >> <nd ref='38879119'/>
> >> <nd ref='38879123'/>
> >> <nd ref='38879127'/>
> >> <nd ref='38879132'/>
> >> <nd ref='38879137'/>
> >> <nd ref='38879139'/>
> >> <tag k='highway' v='secondary'/>
> >> <tag k='ref' v='15'/>
> >> </way>
> >>
> >> in the data-pid-4.osm file, but all except the first node do not appear
> >in the file.
> >>
> >>
> >>
> >>
> >>>> Is this a known issue? I am presuming the planet.osm dump is
> >>>>
> >>> inconsistent.
> >>>
> >>>> Where should we look at solving this problem :-
> >>>>
> >>>> 1. Planet.osm dump is inconsistent, fix that?
> >>>>
> >>>>
> >>> Osmxapi uses planet diff files so fixing planet.osm won't help much.
> >>>
> >>>
> >>>
> >>>> 2. Fix OSMXAPI to not emit nodes in ways that have not been included
> in
> >>>> the download file?
> >>>>
> >>> It's intentional behaviour that if the ways references a non-existant
> >node
> >>> then the node will not be in the output, but the way will still
> contain
> >the
> >>> offending <nd> tag. It didn't seem to make sense to remove it as the
> >>> client
> >>> cannot then detect that the way is incomplete. The client should
> choose
> >>> how
> >>> to process such error situations, although this fact should be
> >mentioned in
> >>> the docs.
> >>>
> >>>
> >>>
> >>>> 3. Have Osmarender ignore undefined nodes? Is this possible with
> XSLT?
> >>>>
> >>>>
> >>> It used to be able to deal with missing nodes, but maybe this broke
> >during
> >>> the 0.4 to 0.5 migration.
> >>>
> >>
> >> The problem is not so much Osmarender as Batik. Inkscape handle errors
> >by ignoring them, Batik fails.
> >> I will see if I can see anything in old versions of the source and also
> >run it past Dodi.
> >>
> >> Thanks,
> >> Brent.
> >>
> >>
> >> ____________________________________________________________
> >> Brent Easton
> >> Analyst/Programmer
> >> University of Western Sydney
> >> Email: b.easton at uws.edu.au
> >>
> >>
> >
> >
> >--
> >No virus found in this incoming message.
> >Checked by AVG Free Edition.
> >Version: 7.5.503 / Virus Database: 269.16.0/1139 - Release Date:
> 19/11/2007 12:35 PM
>
>
> ____________________________________________________________
> Brent Easton
> Analyst/Programmer
> University of Western Sydney
> Email: b.easton at uws.edu.au
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20071120/34e05623/attachment.html>
More information about the Tilesathome
mailing list