[Tilesathome] Corrupted data in TRAPI database

[Jeremy Adams]

>
> Hi List!
>
> There seem to be corrupted data in the TRAPI database. Currently T at H has
> problems compiling tiles around Hawaii (e.g.
> http://tah.openstreetmap.org/Browse/details/tile/12/275/1818/).
> Rendering that tile throws a lot error messages like these:
>
> > Use of uninitialized value in multiplication (*) at
> /home/tilesathome/client-no-auto-1/osmarender/orp/orp-drawing.pm line 366.
> > Use of uninitialized value in numeric lt (<) at
> /home/tilesathome/client-no-auto-1/osmarender/orp/orp-drawing.pm line 403.
>
> but the rendering continues... Finally maplint crashes with the
> corrupted data:
>
> > Modification of non-creatable array value attempted, subscript -1 at
> /home/tilesathome/client-no-auto-1/osmarender/orp/orp-drawing.pm line 403.
>
> I ran T at H in debug mode for that tile and found that the downloaded file
> from API was only 1308 bytes of size whereas the downloaded data from
> TRAPI was 225051 bytes of size. Examing the TRAPI data I found that
> there are ways where some nodes are missing in the file (e.g.
> http://www.openstreetmap.org/browse/way/12222823). As you can see when
> you click on that link the way was deleted on 2009-11-23.
>
> Can someone please examine what went wrong? I assume that there was an
> error in the script updating the TRAPI database. As this should really
> never happen this error should be fixed ASAP.
>
> While the XAPI servers have not yet recent data I would suggest to let
> TRAPI online and try to find the error first. When the error is found
> and fixed (and XAPI servers have recent data) then TRAPI should possibly
> rebuild its database from scratch.
>
> Regards
> Andre
>
>
The Tiger imports that are happening (happened?) in that area are creating
changesets that take longer than 30 minutes to apply.  When this happens,
the TRAPI servers can miss data since they run 30 minutes behind the main
db.  I'm not sure exactly why this happens - I would assume that as the
changeset continues to modify data, those changes would show up in
subsequent minute diffs but it seems this is not the case.

The fix seems to be using the new replication diffs, but TRAPI doesn't
support those yet.

As a bandade fix, I have a second TRAPI server that is loading data right
now.  My plan is to take down the old one for a db rebuild once the new one
is up to date.

-Jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20091130/c024a540/attachment.html>