[OSM-talk] Mapnik tileset coherency issues

[Tom Hughes]

In message <47178D1C.1010500 at arjam.net>
          "Robert (Jamie) Munro" <rjmunro at arjam.net> wrote:

> Patrick Weber wrote:
> > Are we just running into a stone wall here of the current server being
> > overwhelmed ? Or can we manage to minimise these inconcistencies by some
> > clever algorithm as proposed?
> 
> The clever algorithm is to merge the osmosis daily dump into the mapnik
> PostGIS DB. Then for high zoom tiles (say zoom levels over 12), work out
> the bounding boxes of all the ways that have changed, and purge from the
> cache any tiles that intersect any of those bounding boxes.

What does doing daily merges have to do with anything? That's entirely
orthognal to the question of rerendering and consistency.

Of course we could find out which tiles have changed and mark them
dirty when planet is updated, but that would in all probability just
give us a huge render queue that would take days to clear.

I think you are seriously underestimating how many tiles change
each day/week and would therefore need rerendering.

> For zoom levels below a certain number (perhaps 12), I'd just re-render
> the whole lot once a week or something.

That may well be possible.

> I'm not convinced by the need for a queue for the high zoom tiles - I
> think we should either render straight away or not at all. I think the
> probability of any individual high level tile being re-visited is quite low.

Umm.... Without the queue lots of people would just get blank space
on the map. At peak times there is no way we can render all the needed
tiles on demand and if they're not queued then they will still not be
there the next time somebody looks at that area.

> A queue would only be helpful if we can instruct OpenLayers to try again
> after 30 seconds or however long we expect that tile to be rendered in
> (it may be as simple as setting a meta-refresh header on the http of the
> tile download). If we can't instruct OpenLayers or the browser to retry
> after a delay, it seems to me that many tiles in the queue will never be
> looked at again, and the server is better off spending it's time
> re-rendering the lower zoom levels. Maybe the hybrid queue is worth
> doing for zoom levels 13 and 14 or something.

I think your theory that nobody will come back and look at the same
area is completely false. Areas which are being actively mapped get
looked at all the time, other areas get looked at less often.

A thirty second delay is also useless as at busy times it will take
much longer than that.

> If we only delete tiles that we know are no longer valid, hopefully we
> won't need to re-render tiles often as they will already be cached.

We don't delete any tiles unless they've been around a long time, and
in all probability the mod_tile based system we are working on will
not delete them at all.

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/