[Tilesathome] Distributing tiles at home server chores?

[Robert (Jamie) Munro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frederik Ramm wrote:
> Hi,
> 
>     prepare for a solid round of backseat driving ;-)
> 
> The dev server cannot cope with running the tiles at home server. I don't 
> know why exactly that is. Let's summarize what the tiles at home server does:
> 
> A. Managing the re-render queue
> 
> A1. accepting re-render requests
> A2. handing them out to clients
> A3. watching whether they really get completed
> 
> B. Accepting tile uploads
> 
> B1. unzip files being delivered
> B2. sort them into the filesystem, update metadata MySQL
> 
> C. Serving tiles to web browsers
> 
> C1. serving through Apache from file system
> C2. serving through 404 handler script (if Apache doesn't find the file, 
> do a database lookup)
> 
> D. Statistics and cleanup jobs
> 
> Tasks "A" and "D" are negligible; "A" could conceivably be run on a 
> totally different server with only little overhead, but it probably 
> isn't even worth the effort to outsource it as the request queue only 
> receives and dispatches one request every five seconds (on average).
> 
> "B" is very i/o intensive. There is considerable potential for 
> optimising the code, but still "B" will have to create thousands of 
> files on the hard disk for every tileset uploaded, and in addition write 
> stuff to the database. The load doesn't become smaller if we consider 
> that numerous processes do "B" at the same time, needlessly vying for 
> resources.
> 
> "C" has massively improved since tiles are on the hard disk as opposed 
> to in a BLOB in MySQL. I assume, but cannot prove, that "C" is only slow 
> because there's so much "B" going on.
> 
> So in my eyes, "B" is the main culprit.
> 
> Unless we manage to drastically optimise the software at this point, we 
> should think about distributing "B" to more than one server. How could 
> this be done?

I think a drastic optimisation may be possible. Looking at Munin, it's
MySQL REPLACE queries that are the problem, not the tiles themselves. If
these were replaced with UPDATE queries, only INSERTing when needed (by
checking that affected rows=1), it would probably help a lot - REPLACE
means DELETE then INSERT. UPDATE changes the record in place. Also there
will be no need to do any reindexing as the primary keys won't have
changed).

Alternatively, we could just drop those queries all together - I don't
see why we need to keep information about each tile separately rather
than just keep the information about when each level 12 was rendered as
a whole. Or maybe we could just write the _details files as files.

> I could imagine that tiles could be distributed in a "striped" fashion - 
> one server takes all level-12 tiles where x mod 3=0, one those with x 
> mod 3=1, and one those with x mod 3=2 and so on. But how would a slippy 
> map then load tiles from these servers? It could be done via a proxy, 
> but it will probably be difficult to configure an Apache or Squid with 
> the rules required to find the right server of origin for a tile:
> 
> * if z==12 compute x mod 3 and select server 0,1,or 2
> * if z>12 compute relevant level-12 tile x coordinate (x/2**(z-12)) mod 
> 3 and select server 0,1,or 2
> * z<12 - special case, probably only have one server with all these tiles
> 
> Any thoughts?

OpenLayers is able to distribute requests across several servers, using
an algorithm like that one, so it would make sense to use that facility.
That avoids the need to configure a proxy or anything like that, because
openlayers would just do it for us, but as the problem is mysql, not the
tiles themselves, I don't think we need to go down that road.

Robert (Jamie) Munro
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGTOggz+aYVHdncI0RAtkqAJ4nAMXLl3hlym84ZekCUGcJeHLVTgCfT65h
94+2ZBjCfnxen7gmnphRIg0=
=Paq+
-----END PGP SIGNATURE-----