[Tilesathome] [server]

[Florian Lohoff]

On Sun, Mar 28, 2010 at 11:04:04AM +0200, Patrick Kilian wrote:
> But in the end we have to accept that the ~110 active clients (many of
> them multicore and quite fast) can render batches of simple prio4 tiles
> which are expired by my oldtile script or the oldtile checker faster
> then the server can handle the upload. And no removal of code checking
> for that condition is going to fix it. All we (might) get is a hung
> server drowned in upload requests.

As i already mentioned i think a year ago - When looking at the load
of my renderers it seems very "spiky" - so when we have the hourly?
render prio 2 stuff coming in the clients start rendering - as soon
as the prio 2 are done we come back to the prio4 and lots of clients
stall because of the incoming queue.

Here is a munin of a 8 Core 16GB Renderer sitting idle >50% of the
time:

https://hydra.gt.owl.de/munin/lab.rfc822.org/bs4.lab.rfc822.org-cpu.html

In the end it comes to trying to equally load the clients more instead
of the spiky cpu usage, and to equally load the server.

I would see 2 solutions/optimizations - let the prio2 rerender
stuff run much more often - like depend on the minutely replication
diffs. So we sprinkel prio2 into the queue much more often so that
halve of the clients render prio2 and half of them render the oldtile
stuff all the time.

Another would be to make it not a prio1 then prio2 then
prio3 4 ... but rather a kind of "weighted fair queue". So once we have
a bunch of prio1/2 in the queue we not start swamping the clients with them
but rather try to feed some clients prio4 which come back fast (typically)
and let the incoming not drain.

Flo
-- 
Florian Lohoff                                                 f at zz.de
"Es ist ein grobes Missverständnis und eine Fehlwahrnehmung, dem Staat
im Internet Zensur- und Überwachungsabsichten zu unterstellen."
- - Bundesminister Dr. Wolfgang Schäuble -- 10. Juli in Berlin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 827 bytes
Desc: Digital signature
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20100328/cce9af90/attachment.pgp>