[Tilesathome] Active queue

[Matthias Julius]

Dirk-Lüder Kreie <osm-list at deelkar.net> writes:

> The "active" queue is currently inflated by the fact that the API did
> not respond for about an hour today and thus all the jobs failed. Since
> there is currently no mechanism for the client to tell the server the
> job has failed the active queue has to requeue the job somehow. The
> average for the active queue is ~580 jobs according to munin which is
> well within the limits of what can be processed.

OK, so it was just a coincidence that I happened to look at those
numbers today.  Active is still climbing (1.69k).

Another thing the client could do is to retry the download after a
while.  Since problems there are not so uncommon it might be
worthwhile.  But, having the client tell the server that a job jas
failed is probably the better way to go.

>
> Another thing is that if a job takes just over 6 hours to be uploaded it
> doesn't get counted *at all* even when it's inserted into the tileset. I
> think the "done" field should also encompass the jobs not found in
> "active" and perhaps be able to remove jobs from the "new" queue
> (recently timed out "active" jobs) but not the "pending" queue (jobs
> requested through the request API)

Yes, I just realized that, too.  Only active jobs are treated by the
Run script.

If the client would somehow tell the server when it has downloaded its
data for an upload and jobs could be removed from the pending queue if
they were requested before that.

Matthias