[Tilesathome] How often...

[Florian Lohoff]

On Tue, Feb 10, 2009 at 07:47:12AM +0000, 80n wrote:
> How well will mapnik scale under load?
> 
> As I understand it, most mapnik tiles are generated on-the-fly.  This is
> fine when you have a big server and only a few users.  But when you have a
> very large number of users (think Google maps) then would the mapnik
> approach cope with the load?
> 
> It seems to me that T at H is likely to be more scaleable when there are large
> numbers of users. The T at H machine will be mainly concerned with serving
> images to the user rather than rendering and then serving.

My guess is that mapnik can scale very good with little code added.
With the CPU power beeing the limit you can either limit the maximum
number of requests or when appropriate cacheing is applied limit the
currentness of the map.

Pseudo code like this:

	if (!datachanged() && cachedtileavail) {
		send(cached-older-tile)
	} else {
		if (length(renderqueue)*timetorender > 500ms) {
			queuererenderrequest(prio 2)
			if (cachedtileavail)
				send(cached-older-tile)
			else
				send(More OSM Coming)
			return
		}
		rendertile(prio 1)
		updatecachewithnewtile
		send(new-tile)
	}

Then one could use a squid to proxy to multiple mapnik backends based
on URL (e.g. tile xy number - read: geographic region) so one could
use a couple smaller machines which do not need to keep the full
database which makes the working set smaller and thus the database
faster. Once you are at google scale you have a machine per Z12 ;)

Annother approach would be to seperate database, render engine and web
frontend. Rendering is typically CPU intensive and the database is
memory and io bandwidth intensive - So the renderer would have just a
"normal" amount of memory like 1-2 GB but tons of cores. The Database
would have tons of memory and tons of spindles but not necessarily a lot
of CPU.

The only problem with mapnik would be a tons of user panning around
on previously unrendered territory which obviously cant happen to t at h.

Flo
-- 
Florian Lohoff                  flo at rfc822.org             +49-171-2280134
	Those who would give up a little freedom to get a little 
          security shall soon have neither - Benjamin Franklin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20090210/a273581a/attachment.pgp>