[OSM-dev] Compressed vector tiles with mod_tile?

[Kai Krueger]

Hello everyone,

I was playing around with vector tile rendering and in particular with 
Cover[1], which was a GSoC project this year. It can act as a tirex 
backend plugin and thus enable the standard tool chain of mod_tile/tirex 
to serve json vector tiles, including the caching and expiry, queuing 
and load management mechanisms that are part of that stack.

Overall this works very well in combination with e.g. KothicJS[2]. 
However, as text based json is not very space efficient, Cover stores 
the json compressed in the metatiles.

Mod_tile can't yet natively deal with this and doesn't automatically set 
the HTTP header Content-Encoding: gzip and I'd like to fix this.

The question is, should it simply be another configuration option in 
tile layer specification, or should the meta-tile format be changed to 
include this information?

In the latter case the easiest way to store this information would be to 
include it in the first 4 bytes. Instead of "META" they could then be 
e.g. "METZ" to indicate the content of the metatile is gzip encoded. 
This would allow things to be completely backwards compatible and not 
interfere with raster tiles served by mod_tile.

If a bigger version change of the meta-tile format would be the best 
idea, are there any other changes that would be sensible to do at the 
same time?

For example, one could potentially store the md5 sum of each tile in the 
metatile header. If tiles are in memory, I think calculating the md5 is 
at least partly the bottleneck in serving tiles. On the other hand, I am 
not sure if that really is any real concern, as I can get over 2 Gbit/s 
at 20000/s tiles out of my laptop in benchmarks. So I suspect in real 
world scenarios other elements of the system bottleneck long before the 
speed of mod_tile becomes an issue.

Kai


[1] https://github.com/mdaines/cover
[2] https://github.com/kothic/kothic-js