[OSM-dev] How is Mapnik Performances ?

[Jon Burgess]

On Tue, 2009-01-06 at 00:58 +0000, Roberto Navoni wrote:
> Jon Burgess ha scritto:
> > On Mon, 2009-01-05 at 20:46 +0000, Roberto Navoni wrote:
> >   
> >> Hi,
> >> I have a question for OSM-Developer ;)
> >> So I'm trying mapnik for render italian OSM planet , but the performance 
> >> on my sandbox is not so god . I'm using an old one hp laptop that use 
> >> Ubuntu 8.04 as SO and as hardware use P4 3 Ghz 1 GByte of ram . I see 
> >> that actualy are rendering only two  tile for second.  I'm usining 
> >> postgress 8.3 as db. Are there some optimization for speed up the render 
> >> time.
> >> How many mapnik i can start on a single cpu ? The performance is better ?
> >>     
> >
> > Are you using generate_tiles.py for rendering your tiles?
> >   
> Yes I use generate tiles
> > Is the web server under your control or are you serving static files via
> > a 3rd party web host?
> >
> >   
> I'm doing some test on my server ... I have complete control on it , I'm 
> using Ubuntu 8.04 as Operating systems ..
> > The main tile.openstreetmap.org server uses mod_tile[1] which uses
> > several optimisation tricks to improve the performance but you can only
> > get the full benefit of this if you can install a custom Apache module.
> >
> >   
> Ok ... now  I try It.
> > - multithreaded rendering (should exploit improve efficiency if you have
> > hyperthreading in your CPU).
> > - rendering and storing tiles in "meta-tiles" (8x8 at once).
> > - rendering on-demand, avoiding the need to pre-render lots of tiles
> >
> > If you can't install the Apache module then the mod_tile render daemon
> > can be used standalone with the speedtest & convert_meta tools to
> > generate a tree containing plain PNG files for uploading to a 3rd party
> > web host. I can provide more details if you are interested in this.
> >
> > Last time I benchmarked the tile.openstreetmap.org server it was
> > rendering approximately 600 tiles/s using 2 x dual core Opteron CPUs.
> >   
> 
> > The osm.xml file has got significantly more complex since I did those
> > tests so the numbers are probably a bit slower now. Even so, your P4
> > should manage more than 10 times your current performance.
> >
> >   
> Strange .. is possible that there're some problem on db configuration ?

Possibly but postgres is not normally the bottleneck.

>  
> With top I see the cpu usage by thread and i see that postgress using 15 
> % of cpu and mapnik use only 12% of cpu ... is possible that there're 
> some setting to change on postgress or on mapnik ?

Some CPU is taken by the "convert" step. Since this is a short lived
process it does generally appear in top.

If you want to test, simply comment out the following lines in
generate_tiles.py and see how much faster it goes:

    command = "convert  -colors 255 %s %s" % (tile_uri,tile_uri)
    call(command, shell=True)

For serving tiles you really want to use 256 colours since the tiles are
much smaller. If you are using a recent build of Mapnik then this
includes a faster colour conversion algorithm, just change the format
from png to png256 a line earlier:

    view.save(tile_uri,'png256')

Jon