[OSM-dev] osm2pgsql, zoom range for -e option?

Igor Podolskiy igor.podolskiy at vwi-stuttgart.de
Wed Aug 10 13:26:17 BST 2011


> The expired-tiles list which is written by osm2pgsql does not contain any redundant information. This is intended that way because he wanted to save memory: the quad-tree in the algorithm should not grow too much.
> For this reason there are two groups of tiles excluded from the list:
> 1. Next lower zoom levels
> For example, if a tile at zoom level 18 is considered as dirty, you will need to render the related tile at level 17 too. The list will only contain the level-18 tile.
> 2. Next higher zoom levels
> If four neighboring tiles at zoom level 18 are dirty and they fit into a single level-17 tile, only the level-17 tile is marked as dirty, the four level-18 tiles are not.
> Of course I can understand the need to save memory while the expired-tiles list is created but in my opinion this list should be written as a complete list of expired tiles...
aha, so it's not a bug, it's a feature... good to know.

I worked around this issue (didn't have time to dig further in 
expire-tiles.c then) with the following Python function:

def expire_lower_zoom(filename, lowest_zoom):
     for line in open(filename, 'r'):
         z, x, y = [int(i) for i in line.strip().split('/')]
         for zz in xrange(z, lowest_zoom-1, -1):
             yield (zz, x, y)
             x, y = x / 2, y / 2

Or, avoiding duplicates by storing everything in a hashset (in memory):

def expire_lower_zoom_no_dup(filename, lowest_zoom):
     result = set()
     for line in open(filename, 'r'):
         z, x, y = [int(i) for i in line.strip().split('/')]
         for zz in xrange(z, lowest_zoom, -1):
             result.add((zz, x, y))
             x, y = x / 2, y / 2
     return result

You could do the same for a stdin stream or any Python iterable, for 
that matter, the procedure for invalidating higher zoom levels is almost 
the same. You can even avoid writing the function for higher zoom levels 
by passing "-e 17" (or whatever your highest zoomlevel is) to osm2pgsql 
to make it only expire tiles on your highest zoomlevel, and the above 
function will then compute all the lower zoom levels. It worked for me 
for a while now.

Or am I missing something?

Hope that helps

More information about the dev mailing list