[Tilesathome] Long upload times

[Jiri Klement]

I understand that queue length is compromise between speed and latency.

If you think current queue length is alright then keep it. I
personally think that it could be reduced, it didn't get bellow 200
tilesets last week.

Anyway the maximum length of queue should be increased. It doesn't
matter whether tiles wait in queue or in client computer waiting for
upload.

I suggest to give jobs only when queue contains less than 600 tilesets
but allow uploads until queue is filled with 1200 tilesets.

On 6/12/08, Dirk-Lüder Kreie <osm-list at deelkar.net> wrote:
> Jiri Klement schrieb:
>
>
> > Hi,
> >
> > It still takes quite a long time to upload tiles, even now when server
> > doesn't give a job when upload queue is full. Could server be changed
> > to give job only when queue is at most 50% full?
> >
> > Long upload times are anoying for me because I run t at h client on
> > computer I switch off at night. In the evening I have to either wait
> > or waste already rendered tiles.
> >
>
>  The problem is that this is an optimization problem, as you can't have
> everything.
>
>  The current priority is to get the bottleneck never idle, that's the
> server, currently. For that we need a queue, because the tileset rendering
> times vary greatly (a couple of seconds to several hours).
>
>  So to reasonably throttle the clients, the server would need to know
> beforehand, when a tileset would be returned, and how long it would need to
> process it. Because that is impossible we have a queue.
>
>  Because the queue is full almost all of the time, we throttle render-jobs,
> so the clients can have a little shorter upload queues.
>
>  The current value is again chosen so, that the queue of the server will not
> run empty.
>
>  Now we could make the queue on the server bigger, but that would increase
> latency, which is not really wanted, because long latency is what we don't
> like with mapnik.
>
>  We could decrease the amount of jobs handed out while the queue is filled
> to a certain percentage, but we did have some near-underruns already, so we
> shouldn't do that for the sake of tile throughput.
>
>  So to ensure that A) we get the fastest overall rendering speed and B) the
> time from render to processed upload is not too long we need the queue
> system as it is.
>
>  So the current workaround (until new server hardware arrives) would be to
> stop the client *in time* before bedtime.
>
>  If you stop it via stopfile.txt (or ./tilesGen.pl stop) it will finish
> rendering the current tileset and then upload it before exiting.
>
>  --
>
>  Dirk-Lüder "Deelkar" Kreie
>  Bremen - 53.0952°N 8.8652°E
>
>
>