[OSM-dev] osm2pgsql, zoom range for -e option?
marqqs at gmx.eu
marqqs at gmx.eu
Wed Aug 10 15:40:49 BST 2011
My new patch cares about issue 1 and 2:
$ diff -C 5 -p expire-tiles_old.c expire-tiles.c
*** expire-tiles_old.c 2011-05-21 15:13:49.000000000 +0200
--- expire-tiles.c 2011-08-10 16:23:05.000000000 +0200
*************** static int _mark_tile(struct tile ** tre
*** 105,114 ****
--- 105,146 ----
*/
static int mark_tile(struct tile ** tree_head, int x, int y, int zoom) {
return _mark_tile(tree_head, x, y, zoom, 0);
}
+ #if 1
+ static void output_dirty_tile(FILE * outfile, int x, int y, int zoom, int min_zoom) {
+ // writes a tile into output file, including all subsequent tiles
+ // with higher zoom levels;
+ int x_max, y_max;
+ int tile_size;
+
+ tile_size= 1;
+ while (zoom <= Options->expire_tiles_zoom) {
+ if(zoom>=min_zoom) {
+ x_max = x + tile_size;
+ while (x < x_max) {
+ y_max = y + tile_size;
+ while (y < y_max) {
+ if ((outcount++ % 1000) == 0) {
+ fprintf(stderr, "\rWriting dirty tile list (%ik)", outcount / 1000);
+ fflush(stderr);
+ }
+ fprintf(outfile, "%i/%i/%i\n", zoom, x, y);
+ y++;
+ }
+ y-= tile_size;
+ x++;
+ }
+ x-= tile_size;
+ }
+ zoom++;
+ x<<= 1; y<<= 1;
+ tile_size<<= 1;
+ }
+ }
+ #else
static void output_dirty_tile(FILE * outfile, int x, int y, int zoom, int min_zoom) {
int y_min;
int x_iter;
int y_iter;
int x_max;
*************** static void output_dirty_tile(FILE * out
*** 131,148 ****
--- 163,191 ----
}
fprintf(outfile, "%i/%i/%i\n", out_zoom, x_iter, y_iter);
}
}
}
+ #endif
static void _output_and_destroy_tree(FILE * outfile, struct tile * tree, int x, int y, int this_zoom, int min_zoom) {
int sub_x = x << 1;
int sub_y = y << 1;
FILE * ofile;
if (! tree) return;
+ #if 1
+ if(this_zoom >= min_zoom) {
+ if ((outcount++ % 1000) == 0) {
+ fprintf(stderr, "\rWriting dirty tile list (%ik)", outcount / 1000);
+ fflush(stderr);
+ }
+ fprintf(outfile, "%i/%i/%i\n", this_zoom, x, y);
+ }
+ #endif
+
ofile = outfile;
if ((tree->complete[0][0]) && outfile) {
output_dirty_tile(outfile, sub_x + 0, sub_y + 0, this_zoom + 1, min_zoom);
ofile = NULL;
}
If you change the two "#if 1" to "#if 0", the module will return to its old behaviour.
Markus
-------- Original-Nachricht --------
> Datum: Wed, 10 Aug 2011 15:16:22 +0200
> Von: marqqs at gmx.eu
> An: Igor Podolskiy <igor.podolskiy at vwi-stuttgart.de>, dev at openstreetmap.org
> Betreff: Re: [OSM-dev] osm2pgsql, zoom range for -e option?
> Hi Igor,
>
> thanks!
>
> Do your python functions recalculate all the missing tile paths?
>
> Example:
> The dirty_tiles file contains just one tile:
>
> 16/100/100
>
> Would the output of your function produce the following result?
>
> 16/100/100
> 17/200/200
> 17/200/201
> 17/201/200
> 17/201/201
> 18/400/400
> 18/400/401
> 18/400/402
> 18/400/403
> 18/401/400
> 18/401/401
> 18/401/402
> 18/401/403
> 18/402/400
> 18/402/401
> 18/402/402
> 18/402/403
> 18/403/400
> 18/403/401
> 18/403/402
> 18/403/403
> 15/50/50
> 14/25/25
> 13/12/12
> 12/6/6
> 11/3/3
>
> Markus
>
> -------- Original-Nachricht --------
> > Datum: Wed, 10 Aug 2011 14:26:17 +0200
> > Von: Igor Podolskiy <igor.podolskiy at vwi-stuttgart.de>
> > An: dev at openstreetmap.org
> > Betreff: Re: [OSM-dev] osm2pgsql, zoom range for -e option?
>
> > Hi,
> >
> > > The expired-tiles list which is written by osm2pgsql does not contain
> > any redundant information. This is intended that way because he wanted
> to
> > save memory: the quad-tree in the algorithm should not grow too much.
> > >
> > > For this reason there are two groups of tiles excluded from the list:
> > >
> > > 1. Next lower zoom levels
> > > For example, if a tile at zoom level 18 is considered as dirty, you
> will
> > need to render the related tile at level 17 too. The list will only
> > contain the level-18 tile.
> > >
> > > 2. Next higher zoom levels
> > > If four neighboring tiles at zoom level 18 are dirty and they fit into
> a
> > single level-17 tile, only the level-17 tile is marked as dirty, the
> four
> > level-18 tiles are not.
> > >
> > > Of course I can understand the need to save memory while the
> > expired-tiles list is created but in my opinion this list should be
> written as a
> > complete list of expired tiles...
> > aha, so it's not a bug, it's a feature... good to know.
> >
> > I worked around this issue (didn't have time to dig further in
> > expire-tiles.c then) with the following Python function:
> >
> > def expire_lower_zoom(filename, lowest_zoom):
> > for line in open(filename, 'r'):
> > z, x, y = [int(i) for i in line.strip().split('/')]
> > for zz in xrange(z, lowest_zoom-1, -1):
> > yield (zz, x, y)
> > x, y = x / 2, y / 2
> >
> > Or, avoiding duplicates by storing everything in a hashset (in memory):
> >
> > def expire_lower_zoom_no_dup(filename, lowest_zoom):
> > result = set()
> > for line in open(filename, 'r'):
> > z, x, y = [int(i) for i in line.strip().split('/')]
> > for zz in xrange(z, lowest_zoom, -1):
> > result.add((zz, x, y))
> > x, y = x / 2, y / 2
> > return result
> >
> > You could do the same for a stdin stream or any Python iterable, for
> > that matter, the procedure for invalidating higher zoom levels is almost
> > the same. You can even avoid writing the function for higher zoom levels
> > by passing "-e 17" (or whatever your highest zoomlevel is) to osm2pgsql
> > to make it only expire tiles on your highest zoomlevel, and the above
> > function will then compute all the lower zoom levels. It worked for me
> > for a while now.
> >
> > Or am I missing something?
> >
> > Hope that helps
> > Igor
> >
> > _______________________________________________
> > dev mailing list
> > dev at openstreetmap.org
> > http://lists.openstreetmap.org/listinfo/dev
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
More information about the dev
mailing list