[Tilesathome] "done" down and "active" through the roof

[Dirk-Lüder Kreie]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Matthias Julius schrieb:
> Hi all,
> 
> The "done" queue is now down to about half of what it was a few days
> ago.  Did something happen to cause this?  Or do people now circumvent
> the request queue and send in unsolicited uploads?

the OSM api was down for several hours last night, and since "done" is a
48 hour average (normalized to 1 hour) it went down with it.

> At the same time "active" is at 16k and my client is idle.  There must
> be clients out there that grab every job they can while having a huge
> backlog.

this was also partly due to the API downtime. I would estimate 10-15k
jobs there to be "lost".

> Should the server keep track of who is doing which job and not hand
> out new ones while there are too many pending?

We could use a better queue management, yes. before the server overload
we had a system that re-requested the active jobs that hadn't been
turned in after 6 hours. This is obviously only useful if clients can
upload shortly after rendering.

Instead of that approach I suggest we use a job-reinsertion scheme that
moves the oldest X "active" jobs to "new" to avoid swamping the queues
with re-requests. X to be chosen depending on load and queue length
(10-100 maybe?).

- --

Dirk-Lüder "Deelkar" Kreie
Bremen - 53.0952°N 8.8652°E

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGkk5CFUbODdpRVDwRAkKOAJ0cF5VG+HNfe6AdhP07ljQ2KTuxTQCdGg+a
sWbNyjZetQZPcoLNI3Rh2Hw=
=4sJv
-----END PGP SIGNATURE-----