[OSM-dev] osm2pgsql tile expiry freaks me out

[Steve Hill]

On Thu, 5 Nov 2009, Frederik Ramm wrote:

> I suspect that at least the large results for relations are due to an
> inefficiency; probably the whole circumference of the relation is marked
> dirty if a little bit changes here or there, something which would not
> be necessary. (In theory, of course, a rendering rule depending on the
> polygon area could flick over and render a whole country pink instead of
> gray just because its area has changed minimally...)

Yes, (unless someone has changed it since I wrote it) the expiry code 
could be a lot smarter.  If an object changes, it expires all the nodes 
that the old version of the object covered and all the nodes that the new 
version covers - in the case of geometry changes, comparing the geometries 
of the two versions of the object and figuring out what tiles actually 
have changes on them would significantly reduce the expiry count.

In the case of polygons, it tries to be slightly smart: for polygons under 
a certain size it expires everything within the polygon (since you might 
need to rerender everything within the whole polygon), but for large 
polygons it switches to treating it like a normal way and just expiring 
the perimeter.  This is a tradeoff - on the one hand, expiring just the 
perimeter means that you might end up with out of date tiles within the 
polygon that don't get marked as expired, but on the other hand, expiring 
an entire continent could be a really really Bad Thing.

> But as for tagging changes, we're quickly getting into terrain where
> expiry and render rules intermingle; if someone changes the "source" tag
> on a very large polygon way, do I really need to expire half a million
> tiles? But what if the same way's landuse tag is changed?

Yes, this is definately a problem - the current expiry system expires far 
too much stuff and really needs some knowledge about the rendering rules. 
As well as the examples you cite, theres also the fact that osm2pgsql 
doesn't know if a polygon will be rendered as a filled object or as an 
outline - just knowing whether you need to expire the whole polygon or 
just the outline could make a big difference.

To integrate osm2pgsql with the rendering rules, we either need another 
rendering rule definition file, or we need to be able to parse the mapnik 
rules and automatically water them down to some simple data about how we 
are treating each tag/value pair.

-- 

  - Steve
    xmpp:steve at nexusuk.org   sip:steve at nexusuk.org   http://www.nexusuk.org/

      Servatis a periculum, servatis a maleficum - Whisper, Evanescence