[openstreetmap/openstreetmap-website] Use a reduced priority for batch trace uploads (PR #4537)

[Andy Allan]

@gravitystorm commented on this pull request.

+1 for making the queue more fair.

I think the risk with this approach is that if someone adds 1000 traces, and then keeps adding more batches 50 at a time before the original 1000 are complete, then the final traces from the original batch (e.g. with priorities 990-999) might sit in the queue for a long long time, while subsequent traces are processed first (since they go in the queue with e.g. 400-450).

I'm not sure if I'm overthinking this though, maybe it's not a real problem.

https://www.salsify.com/blog/engineering/queue-fairness touches on this same topic (this PR implements "the first solution" and it doesn't seem straightforward to implement a better algorithm at the point of dequeuing (rather than setting the priority at the point of enqueuing).

I'm surprised there's no gems to do this, it seems quite a common topic to discuss fair dequeuing for multi-user queues



-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/pull/4537#pullrequestreview-1899545332
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/openstreetmap-website/pull/4537/review/1899545332 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20240224/f0bc5fa0/attachment-0001.htm>