[Tilesathome] Questions on tile updating process

[Andre Hinrichs]

On Samstag 18 April 2009, Maarten Deen wrote:
> The only way I see for an external user is to load the details page for
> every tile (http://tah.openstreetmap.org/Browse/details/tile/12/x/y/) and
> get the "last modified" date.
> But I don't think that would be very nice to the tah server.

Hmm.. since the tiles are processed sequentially I don't think, that would be 
a real problem. To be honest, I've alread finished writing a small bash script 
which is doing this. It's just finished processing Spain and Germany and it 
seems to work fine.

Will not post it here, since it is partly a really dirty hack and it would not 
be a good thing if too many people run this script in parallel.
If someone is really interested, I will send it directly...

Here are the checks it does in order:
1.) Tile has an unhandled request -> ignore
2.) Tile has an active request -> ignore
3.) Tile is empty and was recently finished -> ignore
4.) Tile is empty and sea -> ignore
5.) Tile is empty and land -> rerender
6.) Tile is empty and mixed -> rerender
7.) Tile is empty and unknown -> ignore with warning
8.) Tile is no older than a given period -> ignore
9.) Tile is older than a given period -> rerender
10.) Tile is not empty and has no date -> ignore with warning

Requests are being sent with priority 4 and source 'tile-reliability-checker'.
Max age is currently set to one month since the tiles seem to have been 
restored from a backup of 2009-03-04. At least I cannot see tile dates older 
that date and younger than 2009-04-14.

This situation results in rather high rate of render requests. I think in the 
future the situation will get better.

The script takes approx. 0.4 seconds for each tile. Means, that for a complete 
world check with 16 mio tiles it would take approx. 77 days to finish. Isn't 
this a good result if the max age for a tile is set to two months???

Well, yes, there could be some more optimization done. A perl script would 
possibly a bit more fast and could also possibly run on a system which has 
direct database access so that the API is not stressed. Or the API would give 
the possibility to get the necessary data in a better form rather than the 
HTML code which has to be parsed... etc...

> I have always felt that there should be a process that enters the oldest
> tiles in the queue (with low priority) so that every tile will get
> rerendered with new rules every once in a while.

Me too...

> That said: there are still wrong rules or renderers that use out-of-date
> rules. I believe the issue of large names of waterfeatures and landfeatures
> in lower zoomlevels has surfaced before, but it is still an issue. See tile
> 12, 2111, 1353 at
> <http://informationfreeway.org/?lat=51.957&lon=5.570&zoom=12&layers=B0000F0
>00F> where the text "Neder Rijn" (and "Lingekanaal" in a neigbouring tile)
> should not be there in zoom 12 and gets smaller in zoom 13 and 14 and gets
> larger again from zoom 15.
> I have requested rerenders of other tiles to see if it is a current
> problem.

That looks to me like a very easy problem to fix... will look at it tomorrow... 
err... today... after lunch...


Regards
Andre