[OSM-dev] Improvement of tile rendering

Jon Burgess jburgess777 at googlemail.com
Sat Mar 21 09:39:00 GMT 2009


On Sat, 2009-03-21 at 02:25 +0100, Udo Giacomozzi wrote:
> Hello Jon,
> 
> Friday, March 20, 2009, 7:48:42 PM, you wrote:
> JB> If you want to mess around with a slightly higher level implementation,
> JB> I recently re-implemented the mod_tile render daemon into python [1].
> 
> Unfortunately I'm not familiar with Python.

You could use it as an example for writing something else. Mapnik
currently only has bindings to support C/C++/Python so you will need to
write your own bindings if you want to render tiles with another
language.

> 
> JB> It is functionally equivalent to the C render daemon.
> 
> BTW, where can I find the source code of the C daemon?

http://trac.openstreetmap.org/browser/applications/utils/mod_tile/

> 
> JB> It still requires the
> JB> C mod_tile module, but I am considering writing a python version of that
> JB> too.
> 
> 
> You might want to use FastCGI instead of messing around with Apache
> modules. It offers a number of advantages:

> - in this case a Apache module is probably not really faster than a
>   FastCGI implementation

Maybe, but you need both implementations to do a fair comparison.
Serving the tiles directly from the Apache thread should be faster than
going via the FCGI interface.

> - FastCGI would allow you to keep rendering daemon and request
>   handling at one place (in one process)

Splitting the low level request handler from the rendering daemon was a
deliberate design decision. This allows for greater scalability. The
rendering is quite heavyweight and slow, we don't want to burden the
request handling threads with job of doing the rendering.


> - FastCGI is more portable, as it is not bound to Apache (consider
>   Lighttpd)

When I wrote the current code I was happy to just support Apache. There
is nothing to prevent someone developing an fcgi based mod_tile
implementation. 

> - you have full control over threading, like limiting the number of
>   concurrent requests. This is very important for optimizazion.

I already have control of the threading in the rendering daemon. I was
happy to let Apache provide the flexible threading model for the
incoming request handling.


> - your FastCGI server can be restarted/replaced or crash at any time
>   without bringing the whole web server down

The rendering daemon can be restarted like this today. The mod_tile code
can be reconfigured while running. The mod_tile code itself is fairly
simple and does not need to be replaced very often.

> - the FastCGI server runs with it's own system user, which makes file
>   access (caching...) easier
> 

The rendering daemon runs as a non-Apache user too. I don't see much
difference here.



I did initially try FastCGI before I opted for a custom module. I just
found the fcgi code I wrote 18 months ago. I can not see why I dropped
this approach but I think I was unhappy with the FCGI_ C interface and
found it easier to work directly with Apache.

> JB> The purpose of these is to allow easier experimentation. If some good
> JB> new ideas develop then I may adopt them into the C code. Another
> JB> possibility is mixing the two implementations together by linking an
> JB> embedded python interpretor into the C code.
> 
> ...I'm sure Python supports FastCGI. If you are unfamiliar with it,
> you should have a look at it. It's very simple (which makes it so
> good).

I have not looked into the python fcgi support.


> Anyway, I would suggest to do any dynamic processing only in case a
> tile is missing (i.e. installing it as a error 404 handler), because
> most requests will probably go to already rended tiles. In this case
> the web server will just have to serve a static file without having to
> invoke Python or anything.

You are right that serving up pre-rendered tiles is the common case that
we need to optimise. Unfortunately storing the plain pngs on the
filesystem has its own scaling problems. This also provides quite
limited abilitiy to http expiry headers and triggering re-rendering of
old tiles.

	Jon






More information about the dev mailing list