[OSM-dev] change replication and replication lag

[Stephan Knauss]

Hello,

I am looking into replacing osmose replication handling with something 
more modern, given that osmose is nearing its end-of-life.

I thought about using pyosmium-get-changes, which seems to do the 
process of creating change.osc files quite well.

A missing functionality is a way to detect the replication lag to 
monitor for stuck replication.

osmose stored the full state.txt file which had a timestamp entry 
scripts could parse.

pyosmium doesn't do this and stores only the sequence ID.

I could do a request of the matching state file and grep for the 
timestamp, but this sounds a bit excessive:

# wget -q -O - 
https://planet.openstreetmap.org/replication/minute/004/448/577.state | 
grep timestamp
timestamp=2021-03-09T22\:14\:57Z

What are others using? switch2osm still refers to a mod_tile supplied 
script using osmose.

Would it sound feasible to extend pyosmium to store the full server 
state file instead of just the sequence? Tools used to the osmose 
behavior would then still work.
Pyosmium already fetches the "newest" state file. As long as diffs fit 
within size, this is then the state file which could be persisted. 
Otherwise it would require to fetch the state file of the then resulting 
sequence number.

Any ideas regarding this before filing an extension request?

Stephan