[Tilesathome] How often...

[80n]

On Tue, Feb 10, 2009 at 8:07 AM, Florian Lohoff <flo at rfc822.org> wrote:

> On Tue, Feb 10, 2009 at 07:47:12AM +0000, 80n wrote:
> > How well will mapnik scale under load?
> >
> > As I understand it, most mapnik tiles are generated on-the-fly.  This is
> > fine when you have a big server and only a few users.  But when you have
> a
> > very large number of users (think Google maps) then would the mapnik
> > approach cope with the load?
> >
> > It seems to me that T at H is likely to be more scaleable when there are
> large
> > numbers of users. The T at H machine will be mainly concerned with serving
> > images to the user rather than rendering and then serving.
>
> My guess is that mapnik can scale very good with little code added.
> With the CPU power beeing the limit you can either limit the maximum
> number of requests or when appropriate cacheing is applied limit the
> currentness of the map.
>
> Pseudo code like this:
>
>        if (!datachanged() && cachedtileavail) {
>                send(cached-older-tile)
>        } else {
>                if (length(renderqueue)*timetorender > 500ms) {
>                        queuererenderrequest(prio 2)
>                        if (cachedtileavail)
>                                send(cached-older-tile)
>                        else
>                                send(More OSM Coming)
>                        return
>                }
>                rendertile(prio 1)
>                updatecachewithnewtile
>                send(new-tile)
>        }
>
> Then one could use a squid to proxy to multiple mapnik backends based
> on URL (e.g. tile xy number - read: geographic region) so one could
> use a couple smaller machines which do not need to keep the full
> database which makes the working set smaller and thus the database
> faster. Once you are at google scale you have a machine per Z12 ;)
>
> Annother approach would be to seperate database, render engine and web
> frontend. Rendering is typically CPU intensive and the database is
> memory and io bandwidth intensive - So the renderer would have just a
> "normal" amount of memory like 1-2 GB but tons of cores. The Database
> would have tons of memory and tons of spindles but not necessarily a lot
> of CPU.
>
> The only problem with mapnik would be a tons of user panning around
> on previously unrendered territory which obviously cant happen to t at h.
>

Well, yes, that is the scenario that I'm describing.  What percentage of the
planet's tiles have actually been pre-rendered on the mapnik tile server?



>
> Flo
> --
> Florian Lohoff                  flo at rfc822.org             +49-171-2280134
>        Those who would give up a little freedom to get a little
>          security shall soon have neither - Benjamin Franklin
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
>
> iD8DBQFJkTXXUaz2rXW+gJcRAsi6AJ9c4SIGOffn3vRkya2BrKcY6ABe0gCggtY6
> 6K7MO25FtPuU7bdqrYCafzs=
> =SqVF
> -----END PGP SIGNATURE-----
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20090210/9ca218d7/attachment.html>