[osmosis-dev] Invalid actions during replication

[Brett Henderson]

On 14 November 2012 22:42, Brett Henderson <brett at bretth.com> wrote:

> Hi Paweł,
>
> On 11 November 2012 21:24, Paweł Paprota <ppawel at fastmail.fm> wrote:
>
>> Hi all,
>>
>> On Friday I have implemented a simple validation in the OWL plugin that
>> checks whether given entity action (create/modify/delete) can be applied to
>> the current database state. See implementation: [1].
>>
>> By "invalid action" I mean a situation when there is a "delete" action in
>> the pipeline but there is no entity in the database to delete. Or "modify"
>> action with version X when there is no version X - 1 in the database but
>> some other version.
>>
>> I've been running replication for over a day now (in LOG mode, on a
>> database with planet imported) and results are quite interesting. There is
>> more invalid actions reported than I thought there would be. Some random
>> examples:
>>
>> INFO: Invalid action: Node 1342627235 - Trying to modify entity that does
>> not exist!
>> INFO: Invalid action: Node 964649444 - Trying to modify entity that does
>> not exist!
>> INFO: Invalid action: Way 11133255 - Trying to modify entity that does
>> not exist!
>> INFO: Invalid action: Way 9105159 - Trying to modify entity that does not
>> exist!
>>
>
I just looked at the errors more closely.  Are you always getting the same
error?  Is it always trying to modify an entity that doesn't exist.

I'd need to dig into the code again, but that *may* be expected behaviour.
The replication code doesn't make a distinction between create and modify,
it processes them the same way.  This is to allow replication data to be
replayed onto a database that already contains a later snapshot of data.
Replaying replication data that already exists occurs after the initial
import (ie. you start replication at a point earlier than the planet file
to avoid missing data), and also occurs if problems occur with replication
and the process needs to be restarted.

What I can't remember off the top of my head is if something in the
replication pipeline is converting all creates to modifies, or whether
there are some scenarios where I can't properly determine if a change is a
create or modify.

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmosis-dev/attachments/20121114/9fcda2b7/attachment.html>