[OSM-dev] Vector tile based raster rendering - toolchain architecture

Wed Jun 25 13:29:38 UTC 2014

Hi,

in the OpenScienceMap project we actually use Tirex in our vector-tiles
stack. For this I wrote a simple Tirex backend that can render any kind of
TileStache layers[1].
I can definitely recommend Tirex for renderqueue management, though I
havent looked what is provided by the Tilelive components yet.

To produce Mapbox vector-tiles Mapzen implemented a TileStache VecTiles
provider[2].

So for those who enjoy python the first part to create vector-tiles
together with Tirex is somewhat working. So far I was only concerned with
client-side vector rendering. Not sure whether Mapnik (without nodejs) can
use mvt to render bitmap tiles yet.

Regards,
Hannes

[1]https://github.com/hjanetzek/TirexStache
[2]https://github.com/mapzen/TileStache/tree/integration-1
https://github.com/mapzen/vector-datasource/wiki/Mapzen-Vector-Tile-Service

On Wed, Jun 25, 2014 at 10:25 AM, Frederik Ramm <frederik at remote.org> wrote:

> Hi,
>
>    if you have followed recent developments then you know that both
> Mapbox and Andy Allan have moved, or are moving, away from producing
> raster tiles the old fashioned way, to producing raster tiles on-the-fly
> from pre-computed vector tiles.
>
> For the time being (i.e. until we have reached a point where every
> possible client can simply process vector tiles directly), this seems to
> be an elegant solution that allows great flexibility in styling at a
> manageable cost.
>
> For those of you who are not familiar with the concept, I would suggest
> you review Dane Springmeyer's presentation at SOTM US 2013 and 2014, and
> Andy Allan's talk at SOTM EU 2014; all talks are recorded and available
> online.
>
> As far as I can see, the current implementations of this concept are
> based on the idea: "Let's make it so that we always have a world-wide
> set of current vector tiles, and render raster tiles from that on the fly."
>
> In order not to require too many vector tiles, vector tiles are usually
> produced for zoom levels no higher than 14; and the z14 vector tile will
> contain all information required for rendering any higher zoom tile
> below it.
>
> (There's also the technical possiblity of combining vector tiles from
> different zoom levels to produce a raster tile, e.g. you could load
> woodland boundaries from a z12 vector tile and road geometries from a
> z14 tile but I'll ignore that for now - Dane explains in his 2014 talk.)
>
> With our old-fashioned raster tile servers, based on renderd or tirex,
> the updating and even the production of tiles is demand-driven. You are
> usually well advised to do some pre-rendering on a tile server but you
> don't *have* to; likewise, tiles don't have to be updated as a diff
> comes in, they are just marked as outdated, then when someone asks for
> them next time they get updated. (And if updating doesn't work fast
> enough, an old tile is delivered and the update happens in its own time.)
>
> I wonder if this old architecture could somehow be combined with the
> shiny new world of vector tiles, something along these lines:
>
> 1. Browser requests tile from Apache
> 2. Apache (mod_tile) checks if the raster tile is in the memory cache
> 3. if yes, deliver from memory cache
> 4. if no, mod_tile checks if the required vector tile(s) is/are on disk
> 5. if yes and vector tile is current, render raster tile, store in
> memory cache, and deliver
> 5. if yes and vector tile is outdated, request new vector tile from
> tirex/renderd and proceed as above, falling back to using the outdated
> vector tile if response takes too long
> 6. if no, request new vector tile from tirex/renderd and proceed as
> above, returning 404 if response takes too long
>
> This would require a significantly beefed-up mod_tile (that includes
> mapnik for turning vector tiles into raster tiles) and only a slightly
> modified tirex/renderd (able to produce vector tiles in much the same
> way they now produce raster tiles). A lot of work that has gone into
> tirex/renderd regarding different storage backends, queue management
> etc. would probably be valuable for this use case too.
>
> This architecture would also do away with nodejs altogether, although
> whether a C/C++ Apache module and a C/C++ and/or Perl based backend is
> any better is of course very much a matter of taste.
>
> I'd be interested to hear your opinions on this.
>
> Bye
> Frederik
>
> --
> Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00'09" E008°23'33"
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
>

-- 
Hannes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20140625/70761d39/attachment.html>