[OSM-dev] Incomplete diffs?

[marqqs at gmx.eu]

Hi Frederik,

thanks for the explanation!

The _replication_ diffs are the right choice if you want to update a full history file.

Most people who update their OSM files on a regular basis do not need replication diffs, they are satisfied with the newest version of each object which has been changed.

> So, if you want to use daily diffs but avoid the danger of missing 
> edits, use the replication diff.

Very good advice.

Until today I chose NOT to use the diffs in the planet.openstreetmap.org/history/ directory because they are outdated. They usually come with a delay of 25 hours. Du you know if the creation process could be accelerated somehow?

Now I will attend to osmupdate and try to change from daily normal diffs to daily replication diffs. Seems to be better than loosing objects once in a while.

Meanwhile people can use Osmosis, or run osmupdate with the --hourly option which will restrict the program to replication diffs.

Markus

-------- Original-Nachricht --------
> Datum: Sat, 05 Nov 2011 22:56:01 +0100
> Von: Frederik Ramm <frederik at remote.org>
> An: dev at openstreetmap.org
> Betreff: Re: [OSM-dev] Incomplete diffs?

> Hi,
> 
> On 11/05/2011 06:58 PM, marqqs at gmx.eu wrote:
> > Meanwhile I found out that this node simply did not appear in the daily
> diffs:
> > http://www.openstreetmap.org/browse/node/1470178889
> >
> > It was crated at 2011-10-16T23:58Z by a large changeset along with
> 23.000 other nodes.
> > Neither the 16/17 nor the 17/18 daily diff contain this node whereas the
> hourly diff from October 17 01:00 does.
> 
> There are two types of diffs; "replication diffs" and normal diffs. A 
> replication diff contains everything that happened between two 
> timestamps, including multiple changes of the same object, whereas a 
> normal diff only contains the information required to get from state 1 
> to state 2.
> 
> Also, replication diffs are created in a relatively fail-safe process 
> with Osmosis whereas the normal diffs can miss changes in some cases 
> when a long-running database transaction that was created before 0:00 
> extends past the time when the diff is created. (There was a time when 
> we had only "normal" diffs, and it was near impossible to make sure the 
> minutely/hourly ones did not miss anything.)
> 
> For minutely and hourly diffs, we only offer replication diffs these 
> days. For daily diffs, we have the normal ones under 
> planet.openstreetmap.org/daily, as well as the replication diffs under 
> planet.openstreetmap.org/history.
> 
> The normal diff indeed lacks the node in question, but the daily 
> replication diff under history/2011/1016-1017.osc.gz has it.
> 
> So, if you want to use daily diffs but avoid the danger of missing 
> edits, use the replication diff.
> 
> Frankly I don't know why the normal daily diffs are still created at 
> all; if one really wanted to offer a reduced-traffic version of the 
> replication diffs then it would indeed make sense to simply deflate the 
> replication diff using Osmosis' --simplify-change task.
> 
> Bye
> Frederik
> 
> -- 
> Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00'09" E008°23'33"
> 
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev