[OSM-dev] Diff issues: minutely overwritten by later data/incompl. daily diffs

mmd mmd.osm at gmail.com
Sun Aug 14 09:56:55 UTC 2016


Hi there,

I'm currently struggling with two different issues when processing daily
and minutely diff files.

Originally I planned to set up a full Overpass API instance using daily
diffs. During processing, some ways/nodes were found to be missing,
which shouldn't have happened in the first place. Now trying again with
minutely diffs, I noticed some minutely diff files got overwritten by
much later data on the server. The two separate issues in more detail:


1/ First 234 OdBl minutely diffs overwritten by 2016 data
---------------------------------------------------------

This currently seems to affect the files
http://planet.openstreetmap.org/replication/minute/000/000/001.state.txt
http://planet.openstreetmap.org/replication/minute/000/000/001.osc.gz
to
http://planet.openstreetmap.org/replication/minute/000/000/234.state.txt
http://planet.openstreetmap.org/replication/minute/000/000/234.osc.gz

For some reason those minutely diffs now include data from May 2016 (!)

Where did the original minutely diffs from September 2012 go? Can they
be restored from backup?


2/ Some daily diffs incomplete
------------------------------

Way 326324100, version 1 was somehow not included in a daily diff,
although the respective minutely diff includes that version.

<way id="326324100" version="1" timestamp="2015-02-04T06:59:27Z"
uid="400383" user="Kalaallit Nunaat" changeset="28604887">

Respective files I checked:

http://planet.openstreetmap.org/replication/minute/001/252/157.osc.gz
http://planet.openstreetmap.org/replication/day/000/000/876.osc.gz

I thought daily diffs should always include all minutely diffs for that
day. Was this caused by some osmosis hiccup? Maybe this affects other
daily diffs as well.

Processing daily diffs is in fact a very convenient to load 4 years
worth of OSM data (needed for full history), rather than downloading
more than 2 Mio. minutely diffs.

Thanks for looking into this.

regards,




More information about the dev mailing list