[Tilesathome] Rate Limiting & be gentle with you requests

[Robert (Jamie) Munro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

spaetz wrote:
> On Fri, Aug 10, 2007 at 12:26:02PM +0100, Robert (Jamie) Munro wrote:
>> It's not the number of active jobs that's important. It's the number
>> that are in the "requesting data from the API" stage of the process.
> 
> Sure, but I have no way of finding that out. I hand out a request for
> a tileset to a renderer and get back some images later (or not). I
> cannot possibly lknow how many blank tiles are involved, nor whether
> they are all rerequesting data because the API was down for some time
> or if they are all happily rendering along. I don't even know if a
> renderer simply took a job and discarded it (because the corresponding
> computer was e.g. shut down). So I have to live and deal with some
> number I have, and that is the number of currently actuve jobs.
> 
>> I think we need to concentrate on directly making sure that T at H clients
>> don't overload the API.
> 
> Sure, patches accepted.
>  
>> When T at H clients request data from the API, they need to request it
>> low-priority. The API should be able to not give it to them if it is a
>> bit busy.
> 
> Agreed. However that is nothing I cn control from the t at h server side. I agree, it should be tackled though.

I'm trying to say that I agree and I don't think it's the T at H servers
job. Work on something else (*). :-)

The T at H server is finally fast enough that it isn't the bottleneck in
the rendering process. That's great news. Let's keep improving it, not
make it slower again in an attempt to fix a problem somewhere else.

> Right, but we have to time out requests at some point in time, if they
> aren't handled or we will simply have a render the whole world queue all
> the time. Or are you suggesting, requests should be resent to clients
> infinitely until one fulfills that request?

What's wrong with a queue to do the whole world? If a T at H client
requests a tile and the queue is completely empty, it's just wasting
it's time and capacity. T at H should have the whole world rendered and
ready to browse.

Robert (Jamie) Munro

* One thing to improve on the server is the PNG blank sea tile database
thing that we have. Personally, I think that when a T at H client renders
an area with a coastline in it, it should be able to upload to the
server "this is a mixed tile", and upload that there is sea to the left
of it and, land to the right and mixed above and below (or whatever).
You can work this out by looking at the colour of the last pixel along
each edge of the z12 tile. The server should be able to use that
information to update the "which z12 tiles are blank" database, rather
than that have to be updated by fiddling with PNGs inside SVN.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGvGY0z+aYVHdncI0RAvblAKCXbxUtPxR2faWtlxBf/mKkX/UitgCguyH8
JfW0lxhSVYHIEDurhkFV15M=
=E5kG
-----END PGP SIGNATURE-----