[Tilesathome] Proposal: New T at H Server structure

[Gerhard Schmidt]

Florian Lohoff schrieb:
> On Sun, Jun 01, 2008 at 10:00:22AM +0200, Gerhard Schmidt wrote:
>> 1. A Client requests a tile from t at h
>>
>> One of the Reflector server (choose per round robin or by some
>> intelligent code in the OSM Javascript code) gets the request looks the
>> tile up in the Database and returns the location of the tile via
>> redirect the the client who connects to the indicated TileServer and
>> gets the Tile there.
>>
>>
>> 2. A T at H client uploads an Tile package
>>
>> The T at H client requests an upload from an web service at one of the
>> Database servers. The web service responds with an upload cookie and the
>> address of one of the TileServer which is able to store more tiles.
>>
>> The T at H client starts the upload of his tile to this server and goes on
>> with the next render request.
>>
>> The TileServer extracts the tiles in a new directory on the server and
>> updates the location for the uploaded tiles in the Main Database.
>>
>> Each TileServer runs a garbage collectiont at some interval to find
>> tiles that are no longer referenced in the Database an delete them.
> 
> Every request for a tile going through a database sounds horrible from
> the point of speed in my eyes.

It is not if some basic rules are followed. As stated in some other
Mails I've done something like that already with very high performance.

1. You have to pool your MySQL connection (opening an authentication
takes the most time)
2. all columns in the where clause have to be in one index
3. keep it to one max 2 simple queries (no joins or views)

> I'd wish the redirector could easily determin the location by e.g.
> hashing the tiles x and y taking the first 2 bytes e.g. 2^16 chunks
> of data and deciding which tileserver contains this chunk of tiles the
> mechanism of this could even be embedded into the map javascript code
> so no need for a redirector.

You break every usage of the Tiles without JavaScript this way.

> So in the end you split 2^16 tiles to n machines - you now only need a
> map with 2^16 entries describing where to find this location. The point
> in using a hash is a better load and data distribution or even much more
> hackish - put the map into the DNS e.g.
> 
> th0000.ts.informationfreeway.org
> th0001.ts.informationfreeway.org
> ...
> thffff.ts.informationfreeway.org

Updates to DNS takes ages to be distributes as some DNS servers ignore
the given ttl an cache far longer than that (most of the Large ISPs do
that) I have done something like hat too and hit the wall quite fast.

> Now requesting the tile is made by the javascript code by running e.g.
> an MD5 or even cheaper hash over the tiles x and y and using the data
> to construct the url. Adding more machines would mean put them into the
> cluster, deciding which chunks of data to move, copying the data and
> once done switch the DNS over to the new machine. Adding more power
> to a single chunk could be done with multiple ip addresses to the dns
> entry. DNS is most likely one of the oldest distributed data systems
> which has proven to work very reliable.

why hashing xyz is already an hash for each tile. You don't gain
anything by running md5 over them.

> Uploading would work as currently established except the upload server
> only pushing the tiles out to the appropriate tileserver(s) noted in the
> hash map.

as Noted by someone OS doesn't have some spare servers in the Attic so
the new system has to run on the given hardware but must be able to
scale if need and new hardware is available. (which my system will do)
as all 3 Parts can be run on the same Machine at first. Still solving
the Upload Problem be eliminating the need to replace tiles in place.

> From what i understood the bootleneck is currently disk i/o or better
> metatadata io as the files get unpacked. This could either be solved
> by distributing on different filesystems or even on different machines.

I think the main problem is replacing one file in a directory populated
by a large number of files. Which will my system solve because ever
upload get its own directory (and n subdirectories to prevent them from
getting to large)

> I dont think fixing the upload queue problem by loading it up to a more
> or less random tileserver and letting the frontend fix the location
> problem is a long term dead end. Given the success of OSM in the long
> term will shift load from updating to serving tiles whereas the latter
> will get more expensive with this approach.

Sure serving tiles will be slightly more costly but in the long run
because of the better scalability we will gain performance instead of
loosing it.

Regards
   Estartu
-- 
-------------------------------------------------
Gerhard Schmidt       | E-Mail: schmidt at ze.tum.de
TU-München	      |
WWW & Online Services |
Tel: 089/289-25270    |
Fax: 089/289-25257    | PGP-Publickey auf Anfrage