[Tilesathome] changetracker / flohoff-ct prio3 requests

[Florian Lohoff]

Hi,
just for the curious - i am doing some experiments to get the overall
throughtput of the t at h clients/server up a bit.

The very high numbers of processed tiles e.g. 3-4Ktiles/h are caused by
a lot of prio4 tiles beeing rendered. Those are mostly empty and the
clients returns them within 10-20 seconds.

So in case all clients render prio4 requests the server queue overflows
and is the limiting factor and we see idling clients (stuck in upload
queue)

In case all clients render prio1/2 requests which are mostly full and
take time to render the clients are busy but the queue is empty. 

So the idea is to mix those two - so keep the clients busy with prio1/2
queries and mix prio4 requests e.g. missing tiles into it up to the max
of the queue.

The culprit i think i have identified is the changedtiles requests which
seem to run every 6 hours. So we run like 3-4 hours with "pure" prio4 and
2 hours with prio2 queries.

The current idea is to queue changedtiles requests all the time e.g.
every minute but only a few.

For this i wrote some lines of perl to download the minutely diffs. Then
i store some informations on disk to debounce the changes e.g. users
making successive changes to the very same tile in multiple uploads.
This should help not to requeue the same query over and over again.

So currently i debounce 15 minutes. So when i see a change i store the
tile to rerender and when it did not receive further changes within the
next 15 minute i queue the tile are pri3.

I used to queue with "flohoff-changetracker" but i changed it to mention
the number of changes i have seen and the userid of the last changing
user. Its just a gimmick and not really needed.

It'll take an hour or two to get into a mixed state as there is some
prio2 and some prio3 queued up.

We'll see how it works out - if it doesnt help i'll turn it off again.

Flo
-- 
Florian Lohoff                  flo at rfc822.org             +49-171-2280134
	Those who would give up a little freedom to get a little 
          security shall soon have neither - Benjamin Franklin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20080906/b971dc89/attachment.pgp>