[OSM-dev] Vector tile based raster rendering - toolchain architecture

Frederik Ramm frederik at remote.org
Wed Jun 25 08:25:43 UTC 2014


Hi,

   if you have followed recent developments then you know that both
Mapbox and Andy Allan have moved, or are moving, away from producing
raster tiles the old fashioned way, to producing raster tiles on-the-fly
from pre-computed vector tiles.

For the time being (i.e. until we have reached a point where every
possible client can simply process vector tiles directly), this seems to
be an elegant solution that allows great flexibility in styling at a
manageable cost.

For those of you who are not familiar with the concept, I would suggest
you review Dane Springmeyer's presentation at SOTM US 2013 and 2014, and
Andy Allan's talk at SOTM EU 2014; all talks are recorded and available
online.

As far as I can see, the current implementations of this concept are
based on the idea: "Let's make it so that we always have a world-wide
set of current vector tiles, and render raster tiles from that on the fly."

In order not to require too many vector tiles, vector tiles are usually
produced for zoom levels no higher than 14; and the z14 vector tile will
contain all information required for rendering any higher zoom tile
below it.

(There's also the technical possiblity of combining vector tiles from
different zoom levels to produce a raster tile, e.g. you could load
woodland boundaries from a z12 vector tile and road geometries from a
z14 tile but I'll ignore that for now - Dane explains in his 2014 talk.)

With our old-fashioned raster tile servers, based on renderd or tirex,
the updating and even the production of tiles is demand-driven. You are
usually well advised to do some pre-rendering on a tile server but you
don't *have* to; likewise, tiles don't have to be updated as a diff
comes in, they are just marked as outdated, then when someone asks for
them next time they get updated. (And if updating doesn't work fast
enough, an old tile is delivered and the update happens in its own time.)

I wonder if this old architecture could somehow be combined with the
shiny new world of vector tiles, something along these lines:

1. Browser requests tile from Apache
2. Apache (mod_tile) checks if the raster tile is in the memory cache
3. if yes, deliver from memory cache
4. if no, mod_tile checks if the required vector tile(s) is/are on disk
5. if yes and vector tile is current, render raster tile, store in
memory cache, and deliver
5. if yes and vector tile is outdated, request new vector tile from
tirex/renderd and proceed as above, falling back to using the outdated
vector tile if response takes too long
6. if no, request new vector tile from tirex/renderd and proceed as
above, returning 404 if response takes too long

This would require a significantly beefed-up mod_tile (that includes
mapnik for turning vector tiles into raster tiles) and only a slightly
modified tirex/renderd (able to produce vector tiles in much the same
way they now produce raster tiles). A lot of work that has gone into
tirex/renderd regarding different storage backends, queue management
etc. would probably be valuable for this use case too.

This architecture would also do away with nodejs altogether, although
whether a C/C++ Apache module and a C/C++ and/or Perl based backend is
any better is of course very much a matter of taste.

I'd be interested to hear your opinions on this.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00'09" E008°23'33"



More information about the dev mailing list