[OSM-dev] Minute replication hiccup

michael spreng mailinglist at osm.datendelphin.net
Sun Feb 14 16:32:12 UTC 2016


On 13/02/16 10:26, Sarah Hoffmann wrote:
> On Fri, Feb 12, 2016 at 12:50:32PM +0000, Tom Hughes wrote:
>> On 12/02/16 12:17, Roland Olbricht wrote:
>>
>>> something strange has happened to
>>> http://planet.osm.org/replication/minute/001/788/263.osc.gz
>> Yes, exactly the same thing as the last N times you asked ;-)
>>
>> The machine crashed, and because osmosis doesn't fsync the state
>> file the last state was lost and Matt had to reset it.
> Wouldn't it be better to leave an empty diff file for the broken
> sequence id and reset so that it continues with the subsequent id?
> Most clients should be able to cope with replayed data and
> hopefully also with empty diff files.
> In any case, even a crashing client is preferable to silently
> loosing data.
>
> Sarah
>
I think so as well. I would like to have a more resilient data
consumption path. I have thought of how to make data consumers more
resilient against such data loss, and I think Sarah has found a good
solution. The solutions I came up with are all inferior I think.

So could we have strictly increasing state numbers as a policy, please?
Which means clients have to cope with repeating changes and empty change
files, which most probably do.

Michael



More information about the dev mailing list