[OSM-dev] [OSM-talk] Mapnik tileset coherency issues

Christopher Schmidt crschmidt at metacarta.com
Fri Oct 19 12:43:37 BST 2007

On Fri, Oct 19, 2007 at 08:44:50AM +0100, Tom Hughes wrote:
> In message <47182550.4030208 at deelkar.net>
>         Dirk-L?der Kreie <osm-list at deelkar.net> wrote:
> > What we learned from T at H that storing tiles in blobs for non-static
> > tiles is a bad idea, because the tiles tend to grow in complexity and
> > thus filesize, so the table apparently becomes fragmented.
> MySQL is definitely a bottleneck for the mapnik layer as well, hence
> the plan to move to the filesystem with mod_tile.
> > Storing tiles in a filesystem is not too great either, if you just stick
> > to paths like ./z/x/y.png.
> > Instead you should split the paths up further so there aren't that many
> > files in the directories, and hide that fact with a mod_rewrite rule, so
> > you get nice "like google" URLs back for the tiles.
> > For example the tile 130543,80370 zoom17 is stored as
> > ./17/130/543/080/370.png
> > ensuring that no directory has more than 1000 entries.
> So long as you are using indexed directories it shouldn't be too much
> of a problem to have so many files in a directory, and recently created
> ext3 filesystem should be using indexed directories.

I believe ext3 gets upset at more than 32000 'links' pointing to a
directory. (This is why TileCache has super-deep directory structure.)
So although lots of files in a directory is fine, lots of directories in
a directory was not... but my memory is poor. This was ext3 ... ah, yes:

"Apparently when using ext2 or ext3 there's a limit on the number of
subdirectories you can create within a directory. This is a hardcoded
number and seems to be set to about 2^15 ~= 32k."


Christopher Schmidt

More information about the dev mailing list