[Tile-serving] [openstreetmap/osm2pgsql] Rewrite tile expiry (#709)

[Michael Reichert]

I am currently rewriting the tile expiry of osm2pgsql because its code is not nice, difficult to understand and it seems to me that it does not what it is expected to do. As part of my master thesis I wrote a [database import tool](https://github.com/Nakaner/Cerepso) which has a tile expiry. @lonvia asked me a few months ago to port its tile expiry to osm2pgsql.

The work is done in my fork of this repository in the `expire-as-set` branch.

I have already replaced the self-made tree structure by an unordered set of `uint64_t`.

Because the new tile expiry is not fully backward-compatible, I'm opening this issue. The format of the written expiry file is the same – one tile per line, each line follows the pattern `zoom/x/y`. In difference to the old code, my code writes out all tiles on all requested zoom levels if something in the tile[1] has changed. However, the old tile expiry writes out a tile on a zoom level if all its subtiles are expired or it is a tile on the maximum zoom level.

If you [convert](https://github.com/Nakaner/expiries2shp) the expiry log files into shape files and visualize them using QGIS, it will look like this (area fill is semi-transparent):

![osm2pgsql's current tile expiry converted to a shape file and visualized using QGIS](https://cloud.githubusercontent.com/assets/3611273/23796488/7287ade2-059b-11e7-8ede-75c9d56480d7.png)

My code would generate an expiry list which will look like this (rendered without an area fill):

![asset-expiry](https://cloud.githubusercontent.com/assets/3611273/23796487/7286f9c4-059b-11e7-9917-c71448cdc612.png)

The raw data of the images can be [downloaded here](https://michreichert.de/share/public/osm2pgsql/expiry-example-shapes.zip).

I can see only one advantage in this behaviour – reducing the size of the file. But this behaviour is not documented anywhere and there is a disadvantage: Every program which reads the tile expiry list, has to derive the list of subtiles on its own. I think that the size of the file does not matter that much because you usually don't transfer it over a network.

[1] i.e. the method `expire_tiles::expire_tile(uint32_t x, uint32_t y)` has been called. The selection of tiles is also handled by my rewrite but out of scope of this issue.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/709
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20170310/4686d3a3/attachment.html>