[Tilesathome] Some First Day Stats

[Brent Easton]

One problem that needs to be addressed is that some tile sets can not be rendered by all machines, depending on the amount of RAM installed versus the complexity of the tile..

This is a known problem with Inkscape and my testing with Batik indicates the same problem. I have 1GB of RAM and can render most tilesets successfully. However, once a tile reaches a certain complexity (e.g. around Rotterdam), then it is not possible to render it successfully on a 1GB machine - thrashing sets in, %cpu drops off to 0 and no work is done. Under these cirumstances, Inkscape tends to crash, while Batik keeps going, but never finishes the tile.

The problem is with the Zoom 16 and Zoom 17 SVG's, they require too much RAM to process. By changing my MaxZoom down to 15, I was able to render Z12 to Z15 of Rotterdam with Batik, (Z15 started to thrash lightly), but Z16 and Z17 are impossible.

I'm not sure how we go about fixing this. Somehow, some tilesets need to be indentified as 'high resource' and only be distributed out to clients that indicate that can process these. Similiarly, slients with < 1GB can usefully render many tiles, but will fail to render 'medium' or 'high' resource tiles. It is very wasteful to even hand them out to low RAM clients.

Regards,
Brent.



*********** REPLY SEPARATOR  ***********

On 8/11/2007 at 11:24 PM Christopher Schmidt  wrote:

>On Thu, Nov 08, 2007 at 11:11:59PM -0500, Matthias Julius wrote:
>> So we have to think of ways to fill it up again to see when we can
>> bring the server to its knees ;-)
>
>Well, right now I'm moving 1.4TB of data off the disk where tiles are
>currently hosted  -- I made 120GB of space earlier, but I figured we
>could use more. So right now, the server seems to top out somewhere
>between 800/900 tilesets an hour, while I'm copying other data off the
>disk at 200MB/s.
>
>So, there are definitely limits to what is doable, but the current rate
>of tile edits is about 250 an hour (1000 tilesets every four hours);
>based on deelkar's script: this means that we have probably more than
>four times the capacity to keep up with 'live' editing of tiles. 
>
>Of course, this brings up something deelkar was talking about earlier,
>which is that so long as there is a long outstanding queue,
>multithreading via different rendering processes works, but as soon as
>that's no longer the case -- that is, most tilesets being requested are
>immediately given out to renderers -- we want to multithread the actual
>rendering itself, since that results in faster turnaround time to users. 
>Even if the queuing server is instantaneous, if each tileset takes an
>hour to render, we can't make the turnaround time get faster.
>
>However, I do think it means that we can start filling in the gaps that
>we have, perhaps by having a 404 handler which queues tiles, like
>mapnik's rendering currently does. I think that would probably be handy
>until we get the tileset back up to 'full' like we had it before.
>(Certainly, it would make me feel better to know that if I bump into an
>empty tile, something will automatically be taking care of it for me.
>
>Regards,
>-- 
>Christopher Schmidt
>MetaCarta
>
>_______________________________________________
>Tilesathome mailing list
>Tilesathome at openstreetmap.org
>http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/tilesathome
>
>
>-- 
>No virus found in this incoming message.
>Checked by AVG Free Edition. 
>Version: 7.5.503 / Virus Database: 269.15.26/1119 - Release Date: 8/11/2007 5:55 PM


____________________________________________________________
Brent Easton                       
Analyst/Programmer                               
University of Western Sydney                                   
Email: b.easton at uws.edu.au