[OSM-talk] Mapnik tileset coherency issues

Dirk-Lüder Kreie osm-list at deelkar.net
Fri Oct 19 04:32:32 BST 2007


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jon Burgess schrieb:
[...]
> There have been several good ideas in this thread and I won't reply to
> them all individually but I'll certainly try to remember them as I
> continue working on the mod_tile development.

Tiles at home currently has more than 26M tiles in the osmarender layer
from z0 to z17, not counting empty land and sea tiles, z18 is not
rendered at all. (I'm estimating another 50-70M tiles in that layer
would it be rendered).

What we learned from T at H that storing tiles in blobs for non-static
tiles is a bad idea, because the tiles tend to grow in complexity and
thus filesize, so the table apparently becomes fragmented.
Storing tiles in a filesystem is not too great either, if you just stick
to paths like ./z/x/y.png.
Instead you should split the paths up further so there aren't that many
files in the directories, and hide that fact with a mod_rewrite rule, so
you get nice "like google" URLs back for the tiles.
For example the tile 130543,80370 zoom17 is stored as
./17/130/543/080/370.png
ensuring that no directory has more than 1000 entries.

As for marking tiles "dirty" by proximity: if you start doing it for
Zoom direction, only do it for *lower* zooms, or you will get into
trouble pretty fast, as each higher zoom layer adds 3n+1 tiles, so more
than triples the work to be done for each zoom added.

As already said, the tiles should only be marked "dirty" if not rendered
since last mapnik db update (i.e. last planet/planetdiff import).

At some point in the future you might want to give up on storing each
tiles metadata individually, but grouping tiles from several layers
together to "tilesets" which get marked "dirty" in one go, (i.e. 18-16,
15-13, 12-10, each 21 tiles) which will automatically give some manner
of z-coherency and xy-coherency at the higher zooms.

At some other point in the future one might want to split the load up
between multiple tileservers, for example 4 tileservers in the following
manner:
server0: evenX evenY
server1: evenX oddY
server2: oddX  evenY
server3: oddX  oddY
Although that would make the coherency issue harder to solve, if each
tileserver keeps it's own copy of the dirty db for performance reasons.


- -> dev ml
- --

Dirk-Lüder "Deelkar" Kreie
Bremen - 53.0952°N 8.8652°E

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHGCVQFUbODdpRVDwRAtBaAKCBLOzXiX1+cMavivDt5AjakyEtYQCbBpNb
hJ7Rv4Z2gG+6GXc54Z6g8KA=
=ok7r
-----END PGP SIGNATURE-----




More information about the talk mailing list