[Tilesathome] Long upload times

[Dirk-Lüder Kreie]

Jiri Klement schrieb:
> Hi,
> 
> It still takes quite a long time to upload tiles, even now when server
> doesn't give a job when upload queue is full. Could server be changed
> to give job only when queue is at most 50% full?
> 
> Long upload times are anoying for me because I run t at h client on
> computer I switch off at night. In the evening I have to either wait
> or waste already rendered tiles.

The problem is that this is an optimization problem, as you can't have 
everything.

The current priority is to get the bottleneck never idle, that's the 
server, currently. For that we need a queue, because the tileset 
rendering times vary greatly (a couple of seconds to several hours).

So to reasonably throttle the clients, the server would need to know 
beforehand, when a tileset would be returned, and how long it would need 
to process it. Because that is impossible we have a queue.

Because the queue is full almost all of the time, we throttle 
render-jobs, so the clients can have a little shorter upload queues.

The current value is again chosen so, that the queue of the server will 
not run empty.

Now we could make the queue on the server bigger, but that would 
increase latency, which is not really wanted, because long latency is 
what we don't like with mapnik.

We could decrease the amount of jobs handed out while the queue is 
filled to a certain percentage, but we did have some near-underruns 
already, so we shouldn't do that for the sake of tile throughput.

So to ensure that A) we get the fastest overall rendering speed and B) 
the time from render to processed upload is not too long we need the 
queue system as it is.

So the current workaround (until new server hardware arrives) would be 
to stop the client *in time* before bedtime.

If you stop it via stopfile.txt (or ./tilesGen.pl stop) it will finish 
rendering the current tileset and then upload it before exiting.

-- 

Dirk-Lüder "Deelkar" Kreie
Bremen - 53.0952°N 8.8652°E

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20080612/cd74bbcf/attachment.pgp>