[OSM-dev] Incomplete diffs?

[Frederik Ramm]

Hi,

On 11/05/2011 06:58 PM, marqqs at gmx.eu wrote:
> Meanwhile I found out that this node simply did not appear in the daily diffs:
> http://www.openstreetmap.org/browse/node/1470178889
>
> It was crated at 2011-10-16T23:58Z by a large changeset along with 23.000 other nodes.
> Neither the 16/17 nor the 17/18 daily diff contain this node whereas the hourly diff from October 17 01:00 does.

There are two types of diffs; "replication diffs" and normal diffs. A 
replication diff contains everything that happened between two 
timestamps, including multiple changes of the same object, whereas a 
normal diff only contains the information required to get from state 1 
to state 2.

Also, replication diffs are created in a relatively fail-safe process 
with Osmosis whereas the normal diffs can miss changes in some cases 
when a long-running database transaction that was created before 0:00 
extends past the time when the diff is created. (There was a time when 
we had only "normal" diffs, and it was near impossible to make sure the 
minutely/hourly ones did not miss anything.)

For minutely and hourly diffs, we only offer replication diffs these 
days. For daily diffs, we have the normal ones under 
planet.openstreetmap.org/daily, as well as the replication diffs under 
planet.openstreetmap.org/history.

The normal diff indeed lacks the node in question, but the daily 
replication diff under history/2011/1016-1017.osc.gz has it.

So, if you want to use daily diffs but avoid the danger of missing 
edits, use the replication diff.

Frankly I don't know why the normal daily diffs are still created at 
all; if one really wanted to offer a reduced-traffic version of the 
replication diffs then it would indeed make sense to simply deflate the 
replication diff using Osmosis' --simplify-change task.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00'09" E008°23'33"