[Tilesathome] Corrupted data in TRAPI database

[Brett Henderson]

On Tue, Dec 1, 2009 at 9:17 AM, Jeremy Adams <milenko at king-nerd.com> wrote:

>
>>
> The Tiger imports that are happening (happened?) in that area are creating
> changesets that take longer than 30 minutes to apply.  When this happens,
> the TRAPI servers can miss data since they run 30 minutes behind the main
> db.  I'm not sure exactly why this happens - I would assume that as the
> changeset continues to modify data, those changes would show up in
> subsequent minute diffs but it seems this is not the case.
>

It depends.  If the changeset is uploaded in small chunks to the API, each
is written as a separate database transaction and the data will be spread
across multiple minute diffs.  If the data is uploaded in a single chunk it
is written within a single database transaction and can possibly take longer
than 30 minutes to process.  The data only becomes visible to Osmosis after
the transaction is committed by which time Osmosis has moved past that time
interval and misses the data entirely.


>
> The fix seems to be using the new replication diffs, but TRAPI doesn't
> support those yet.
>

Yes, they don't use timestamps at all so don't suffer from problems with
long running transactions.  If you see any missing data in those files,
please let me know.

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20091201/716fb6f8/attachment.html>