[Tilesathome] Thoughts about smoothing the load on dev

[Martijn van Oosterhout]

On 6/5/07, Dirk-Lüder Kreie <osm-list at deelkar.net> wrote:
> A reset to 0 should be OK, if not everyone waiting for upload resumes
> uploading in a short timeframe, so just randomizing the upload delays a
> bit should smooth things out already.

I disagree. We have, on the whole, a load of clients trying to upload
faster than the server can handle. To reduce the load we add delays on
the client side. It may be that the optimal upload pattern is for
every client to delay 8 seconds between each upload. But the system
can never stabilise to that, because as soon as one upload goes
through it starts uploading as fast as possible.

The simple suggestion is maybe too simple, but it allows the system to
stabilise to a delay other zero, which the current code can't do.

> There is another problem with the failed downloads: It's entirely
> possible newer data gets uploaded for an area while old data is still
> delayed by an overload some time ago, and when that upload finally
> succeeds the tiles are overwritten with stale data.

Agreed, clients should delete stuff rendered more than five hours ago.

> Basically we need to switch to a mode where the server accepts every
> upload and then separately processes the incoming zip files, stopping
> uploads only when the queue gets too long.

Agreed, it just needs to be written. Problem is, no-one has...

Have a nice day,
-- 
Martijn van Oosterhout <kleptog at gmail.com> http://svana.org/kleptog/