[Tilesathome] Status 'new' vs. 'pending'?

[Robert (Jamie) Munro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Christopher Schmidt wrote:
> Earlier today, the tiles_queue database grew a bit too big for its
> britches, and as a result, it started resorting to a filesort for
> finding the next tile to request. (ouch.) As a result, I changed the
> code, but I'm realizing now that I changed it incorrectly, and it iwll
> need to go back to the way it was.
> 
> Specifically, the problem I'm having is that the test of finding 'high
> priority tiles' where status <= 1 is expensive (but finding all the
> tiles in status = 1 is easy and status = 0 is easy). 
> 
> Can someone explain to me what the difference between 'pending' and
> 'new' is supposed to be? It seems like rendering from 'new' will let
> clients sit without being rendered for a time, but always rendering from
> pending will cause requests which don't get picked up in the first hour
> to get missed as more come in behind them.
> 
> Why are these queues seperate? What does this help?

I think it was supposed to be a way of prioritising before someone added
the priority

> And how should we
> accomodate for the fact that this really is not cutting it for
> performance? (Should we drop the low priority request count to get the
> table back to being tiny and fast again? Is there a way to index a
> column such that 'status <= 1' actually hits a key?) 

It won't hit the key unless the results are sorted by status, not priority.

I wondered about merging status and priority, so levels of priority
would just be statuses higher than 10 or something.

Looking at when I changed the file, I tried doing to queries for the
status, and merging them with a UNION statement in order to make maximum
use of the keys. I don't know if this fully worked, but it lasted from:
http://trac.openstreetmap.org/changeset/2937
to:
http://trac.openstreetmap.org/changeset/4508

Which is quite a long time.

Looking at an SVN blame of the current file, all that is left of my
contributions is whitespace :-)

Robert (Jamie) Munro
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHPDLYz+aYVHdncI0RAkaRAJ9zsoqsmKyb8Q5j4ff2FS/v1uxB5QCghOQo
tDU0TrdVzA9kZQM0GPLJnGk=
=UC7x
-----END PGP SIGNATURE-----