[OSM-dev] preconditions_ok? in app/models/relation.rb and way.rb

Matt Amos zerebubuth at gmail.com
Tue Aug 24 00:22:16 BST 2010


On Sun, Aug 22, 2010 at 5:24 PM, Mitja Kleider <mitja at kleider.name> wrote:
> Richard Fairhurst wrote:
>> I think you're misreading how the relation integrity checks work. The
> API
>> only checks whether an entity is still visible when the entity is
> _added_
>> to a relation.
>> For performance reasons, it does not check the entities
>> formerly in the relation.
>
> Thanks for clarifying. That is probably the cause.
>
>
>> It assumes that they're ok, which in the case of historic
>> (pre-integrity check) data may not be the case. See preconditions_ok? in
>> app/models/relation.rb in the Rails port for the code.
>
> When were the integrity checks introduced?

many of them were in-place before API 0.6 went live on 2009-04-21, but
0.6 (i think) included far more of them and in a more consistent
manner.

> Are you sure that checking only _added_ entities is enough? I read it like
> that: for performance reasons the checks are done when the entity is
> deleted (way node in the next case).

when a referenced entity is deleted it is checked for membership in
any way or relation and, if it's a member of another entity, the
deletion is disallowed. when a new entity is created, or elements
added to an existing entity, then those members are checked to ensure
they're not deleted. in a fully serialised model this should be enough
to prevent any referential integrity problems. but...

> Looking at the other example
> ("http://www.openstreetmap.org/api/0.6/way/60263972/1"), the deleted nodes
> contained in that way (e.g.
> "http://www.openstreetmap.org/api/0.6/node/338392109/5") were deleted May
> 2010 (2010-05-26).
>
> In my understanding it still should not have happened, or am I missing
> something?

unfortunately, we don't have a fully serialised model and many
uploads, particularly from potlatch, execute in parallel. this leads
to the case above where the node is being deleted and the way created
at very similar times, but in different transactions. the timeline
probably looks like:

1. (transaction A) request is received for node deletion. checks in
software indicate that it isn't part of any ways or relations. the
node is deleted.
2. (transaction B) request is received for way creation. checks in
software indicate that the nodes are all not-deleted (since
transaction A hasn't committed yet). the way is created.
3. (transaction B) commits without problems, as the FK constraint is satisfied.
4. (transaction A) commits without problems.

since nodes are stored in the database with a "visible" flag to
indicate whether they're deleted or not, the FK constraint can't catch
this error. it's a similar story with relation members, as the current
schema can't be FK checked.

API 0.6 drastically improved things over 0.5 in its referential
integrity, but we didn't manage to catch all the problems. hopefully
we'll get them in 0.7. until then, when you find one of these it's
best just to fix it.

cheers,

matt



More information about the dev mailing list