[Tile-serving] OSM Tile Expiration

[Christian Quest]

Le 17/07/2020 à 15:42, Lynn W. Deffenbaugh (Mr) a écrit :
> Greetings all,
>
> I saw a reference somewhere that the main OSM tile servers don't use 
> osm2pgsql's tile expiration lists, but use something different for (at 
> least) relations.  Is this approach documented anywhere?
>
> I've been running a planet-wide tile server for years now and have 
> been doing the 
> osm2pgsql/openstreetmap-tiles-update-expire/render_expired approach 
> successfully for the most part.  But my updates sometimes fall behind 
> and take a long time to get current.  Most of the time is spent doing 
> the pending_relations.  As far as I can tell, this is normal.
>
> I realize a different expiration approach won't change the updates, 
> but I'm ready to learn if there's a new/better way of maintaining a 
> planet-wide tile server.
>
> Lynn (D)
>
> PS.  The munin graphs of my tile server (and IPFS server) are at:
>
> http://ldeffenb.dnsalias.net:14171/munin/localdomain/localhost.localdomain/index.html 
>
>
> Just ignore the Hot Water Temperature graph at the top!
>
> Oh, and I've been messing with hosting metatiles on IPFS which is why 
> the forced renderings are going on.  My tile server doesn't really 
> serve out that many tiles! 


I've also been looking recently for some documentation about the 
expiration process on the OSMF tile servers and only found (so far) the 
chef cookbooks as answers.

After the osm2pgsql run, a python script takes the .osc file just 
processed by osm2pgsql and does the expiration on the metatiles based on 
it and the flatnodes file (to get the location of the nodes that are not 
in the .osc file).

My question is why doing it that way and not the expire_list / 
render_expired way ?

Is it faster ?

Does it invalidates metatiles in a better way ? (for example less tiles 
invalidated for nothing)


On OSM-FR server, we face some saturation on the tile regeneration 
process and in order to find what to improve first, I'm investigating 
each piece of the puzzle, from postgres setup, SQL queries, style sheet, 
cache policy, expiration, etc...

I've done a lot of statistics to understand the ratio of tiles rendered, 
cached, requested, purgeable. By analyzing the renderd log, I found that 
some tiles get rendered up to 25 times a day !

-- 
Christian Quest - OpenStreetMap France