[OSM-dev] Multithread generate_tiles.py, generate_images.py and question about attribution.
jburgess777 at googlemail.com
Mon Oct 5 19:05:02 BST 2009
On Mon, 2009-10-05 at 15:57 +0200, Radek Bartoň wrote:
> Dne neděle 04 Říjen 2009 05:09:51 Dane Springmeyer napsal(a):
> > I meant to ask if you had yet compared your generate_tiles.py
> > threading support to the recent additions in svn?
> > I assume not since you wrote yours, as you say several months ago, but
> > it would be interesting to test the speed of each version.
> Now I tried and the results are quite interesting. I tested on 3GHz Intel Core
> 2 Duo machine with 2GB RAM and using 2 threads.
> Result for OpenStreetMap generate_tiles.py was:
> real 42m10.608s
> user 21m39.037s
> sys 4m36.265s
> Result for OpenTrackMap (mine) generate_tiles.py was:
> real 56m57.562s
> user 32m7.568s
> sys 8m24.184s
> Then I noticed that OSM version doesn't use any overlapping.
It does. It uses the Map.buffer_size feature of the newer Mapnik which
provides the benefits of an overlap with less overhead.
> When I removed
> overlapping from mine version the result (even when full color version of tile
> is stored to the disk and then converted to 256 color version using
> ImageMagick) was:
> real 40m35.201s
> user 21m42.249s
> sys 6m42.197s
> Anyway, OSM version should use some overlapping because then there are some
> artifacts on tile borders. Maybe the best solution would be to render for
> example 16x16 (or more) tiles image with small border and then cut the image
> to separated tiles. This could bring also another performace gain because of
> better memory utilization or less access to DB.
This meta tile scheme is implemented in more advanced tools like the
mod_tile renderd.py. If there is interest i'll port that across to
generate_tiles.py too... I have been reluctant to do this in the past
because generate_tiles.py was just meant as a simple example script,
never a fully developed tool.
> Mine version has small benefit compared to OSM version - it can be interrupted
> with CTRL+C keys, while mine version doesn't use job queue so it probably
> doesn't scale as well as OSM version.
The KeyboardInterrupt handler would be useful and is just a couple of
lines of code.
More information about the dev