[OSM-dev] Tiles at home and the job queue

Dirk-Lüder Kreie osm-list at deelkar.net
Sat Mar 24 22:13:22 GMT 2007


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lars Aronsson schrieb:
> Frederik Ramm wrote:
> 
>> Two possibilities: Either the RSS automatism has in fact picked 
>> up your change and some t at h client has taken the request but not 
>> yet responded; or your change has not been picked up and there 
>> never was a request in the queue. The link above should shed 
>> light on this!
> 
> Is there any attempt to debug the RSS feed?  Do we know how often 
> it misses edits?

As far as I know, no, I'd estimate it at 10-30% also depending on wether
you are editing at peak times or at times where there's relatively
little activity.

>> We'll be doing that over the next few weeks.
> 
> Great to hear that.  I hope I can help.

You will see the "no work" message quite regularly, but you can always
run your own tile-jobs with the xy parameter (don't do nonsensical
stuff, it'll only load the main OSM server because of the data requests)

>>> suitable timeout could be 10 minutes.
>> Not so; a complex tile like x=2143,y=1406 can require over half 
>> an hour, even on decent hardware.
>
> OK, so maybe set it at 3 hours.  But is there any timeout at all 
> today?  I think many people can install tilesGen.pl and try it 
> out, without ever requesting a password for uploading.  It is a 
> pity if this blocks anybody else from rendering the same tile.

Nobody blocks anybody from rendering any tile by just taking it out of
the queue, however chances that the tile gets rendered sink somewhat,
unless the experimenting person re-requests the tile through the
requests api.[1]

> Ever since I got involved in OSM, tile rendering has been a great 
> problem.  Now with Tiles at home we have a method where more people 
> can help, and this is great.  I think we need to look at the whole 
> chain of events from editing the map, to seeing the new tiles: How 
> often does the chain fail (e.g. RSS misses the edit) and when it 
> does succeed, how long does it take?  If we can measure these 
> parameters, we can start to make improvements.

There are mechanisms to ensure every tile gets into the queue
eventually, (once a week) but so far no mechanism to ensure it actually
gets rendered after it has been handed out to a client. I used to take
care of this by means of watching the requests log[2] by means of a
script and simply re-requesting everything that hung there for 6 hours
without moving to "completed" but since that doesn't work at the moment
there is no way of dealing with unfinished requests so far.

As for the times taken for the process these tend to vary greatly, not
only dependent on the area rendered, but also on the different changes
of the tiles at home client which sometimes evolves quite fast.

The rendering time of a tile in London for example could easily reach 2
hours or more before revision 2032[3] (didn't have version names back
then, they were introduced in revision 2100[4]) but dropped to under
half an hour then, now are back at approximately an hour due to the
Frollo preprocessing (which personally I don't like, because I feel data
repair shouldn't be done by the renderer).

This lossy rendering is not seen as a high priority right now, because
eventually every tile in need of updating should get caught and be
updated, and most of the more active editors run tiles at home clients as
well, so with a little knowledge about tilenames (see wiki) and a
calculator[5] they can see to it themselves that their area gets rendered.

Others (like me) run their own queue (small, to keep updates timely),
which gets fed from the server to be able to ensure every tile requested
(in this case by the local queueing) is rendered and uploaded.

I even plan to track "bad" and complicated tiles that don't render
correctly or render only on very decent hardware, others are working on
getting a Maplint layer to tiles at home, but if you have fixes, the code
for the website should be in svn.[6]


Dirk-Lüder "Deelkar" Kreie

[1]
http://wiki.openstreetmap.org/index.php/Tiles%40home/APIs#Requesting_updates
[2] http://dev.openstreetmap.org/~ojw/Log/Requests/
[3] http://trac.openstreetmap.org/changeset/2032
[4] http://trac.openstreetmap.org/changeset/2100
[5] http://almien.co.uk/OSM/Tools/Coord/
[6] http://svn.openstreetmap.org/browser/websites/tilesAtHome


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGBaKBFUbODdpRVDwRAm6pAJ9UVhA8H9pgiXfoqz6ZHab+pr4lpACfQaeE
4Lzd1XfUBBkz7XfcDg1bjDw=
=Onsq
-----END PGP SIGNATURE-----




More information about the dev mailing list