[osmosis-dev] Invalid actions during replication

[Paweł Paprota]

Hi all,

On Friday I have implemented a simple validation in the OWL plugin that 
checks whether given entity action (create/modify/delete) can be applied 
to the current database state. See implementation: [1].

By "invalid action" I mean a situation when there is a "delete" action 
in the pipeline but there is no entity in the database to delete. Or 
"modify" action with version X when there is no version X - 1 in the 
database but some other version.

I've been running replication for over a day now (in LOG mode, on a 
database with planet imported) and results are quite interesting. There 
is more invalid actions reported than I thought there would be. Some 
random examples:

INFO: Invalid action: Node 1342627235 - Trying to modify entity that 
does not exist!
INFO: Invalid action: Node 964649444 - Trying to modify entity that does 
not exist!
INFO: Invalid action: Way 11133255 - Trying to modify entity that does 
not exist!
INFO: Invalid action: Way 9105159 - Trying to modify entity that does 
not exist!

I have not yet analyzed closely those entities that are reported but in 
general there should be no invalid actions at all on a full planet 
database with correct replication setup, am I correct in this assumption?

Also, if anyone's interested, this functionality would be trivial to 
port over to the pgsnapshot Osmosis plugin (--write-pgsql-change) since 
the OWL plugin is built from that plugin. I can do this if you think it 
could be useful.

[1] 
https://github.com/ppawel/osmosis/blob/owldb-plugin/owldb/src/main/java/org/openstreetmap/osmosis/owldb/v0_6/impl/ChangeWriter.java#L256

Implementation notes: basically there is a "invalidActionsMode" 
paramater for the --write-owldb-change task (which is derived from 
--write-pgsql-change but with OWL-specific additions) that can be IGNORE 
(does nothing), LOG (logs information about the action) or BREAK (stops 
processing when invalid action occurs).

Paweł