[Tilesathome] Thoughts about smoothing the load on dev

[Martijn van Oosterhout]

Currently the load on dev swings wildly and some of the code there
currently contributes to this effect. I have some thoughts about how
we could make the response a little smoother:

1. Currently the backoff time doubles every time an upload fails, but
goes to zero when it succeeds. This causes a "thundering herd" problem
as soon as the load drops below three. I propose we change the
calculation to cut the failure count in half on successful upload
rather than setting it to zero.

2. Currently the only way clients get feedback that the server is busy
is when an upload fails. I propose that the server accept uploads but
return an extra code when the load is high (say >2.5) to tell the
client to start backing off before all goes crazy.

3. When the server load is continuously high (say after four
consectutive failures) I propose that the client, instead of just
uploading again, uses the go_nogo script to determine when it is safe
to upload. This should save bandwidth and processing time on the
server for uploads it's going to reject anyway.

If these ideas sound good I'm willing to implement them. My SVN
account should be coming any week now...

Have a nice day,
-- 
Martijn van Oosterhout <kleptog at gmail.com> http://svana.org/kleptog/