[Tilesathome] Thoughts about smoothing the load on dev

[Martijn van Oosterhout]

On 6/6/07, Sebastian Spaeth <Sebastian at sspaeth.de> wrote:
> Martijn van Oosterhout wrote:
> > 1. Currently the backoff time doubles every time an upload fails, but
> > goes to zero when it succeeds. This causes a "thundering herd" problem
> > as soon as the load drops below three. I propose we change the
> > calculation to cut the failure count in half on successful upload
> > rather than setting it to zero.
>
> Yes, this is what an exponential backoff obviously should do. If it
> isn't implemented already, I'm all for doing it. No need to make it more
> complex with random variation and stuff. *2 and /2 should work just fine.

Someone is working on this.

> > 2. Currently the only way clients get feedback that the server is busy
> > is when an upload fails. I propose that the server accept uploads but
> > return an extra code when the load is high (say >2.5) to tell the
> > client to start backing off before all goes crazy.
>
> Sounds good to me. It is a just a different HTTP return code you were
> thinking of?

Yes, I can't find an appropriate status code to reuse, so I propose
something like "299 Accepted, Slow down". It should be backward
compatable with current clients. And if this code is received, don't
double the count, instead do "$failurecount += 0.1" (I also propose
renaming the variable to something other than failurecount, but
anyway....).

> > 3. When the server load is continuously high (say after four
> > consectutive failures) I propose that the client, instead of just
> > uploading again, uses the go_nogo script to determine when it is safe
> > to upload. This should save bandwidth and processing time on the
> > server for uploads it's going to reject anyway.
>
> +1

This shouldn't be too hard.

Have a nice day,
-- 
Martijn van Oosterhout <kleptog at gmail.com> http://svana.org/kleptog/