[Tilesathome] Distributed server idea

[80n]

On 7/19/07, Frederik Ramm <frederik at remote.org> wrote:
>
> Hi,
>
> > I sketched an idea, and put notes at:
> > http://wiki.openstreetmap.org/index.php/Tiles%40home/
> > Distributed_Server
>
> I heard a talk by Richard Stallman once in which he said that the
> idea of user accounts and passwords was inherently fascist.
>
> While I'd say that that was a bit over the top, I still detect a
> desire to "control" in the whole GPG key business here. The reason
> for this is mistrust vis-a-vis the participants in the project, which
> I believe is inappropriate and hurting the climate.
>
> I do understand that such measures have to be put in place where
> system security is at risk, e.g. when you download packages from a
> Debian repository you really want to make sure the package is from a
> trusted source.
>
> But we're talking about map images here, and the worst thing that can
> happen is someone defacing the map, which can be rectified quickly.
> It is like a public forum where there's a danger of people posting
> porn or inappropriate texts. It is like Wikipedia where you can
> replace an article by a porn image anytime. That can happen, and will
> happen, but there's no reason to build a whole control and
> authentication and verification structure just because of that. (Or,
> at least, we could leave the decision on how "fascist" they want to
> be with the tile servers because they are in charge, legally.)
>
> Regarding distribution, yes, DNS is a cool idea.
>
> The weak point with any distribution is knowing where to get the data
> from. You can either have the client request data from one specific
> server, and have the server act as a proxy/cache (this is what you
> would do if you had a proper server farm at one location), or you can
> have the server return HTTP redirects to where the content actually
> is (makes more sense in a world-wide distributed environment).
>
> DNS has a proven record of good, fast database lookups. A DNS lookup
> is much, much faster than a HTTP server connection that get answered
> with a redirect message. So this is good. In detail it would probably
> look like this:
>
> If renderer X renders tileset 1234,2345 and uploads to server Y, with
> Y being just any simple Apache setup anyone can install, then Y would
> afterwards notify a central DNS server that it now has this tileset.
> The central DNS server would from then on return Y's IP address for
> any queries like "1234-2345.tiles.openstreetmap.org". The clients
> (i.e. OpenLayers) would then have to use the proper host names for
> every tile, so whenever they request a tile that is within tileset
> 1234,2345 they will request e.g. http://
> 1234-2345.tiles.openstreetmap.org/tiles/13/2468/4690.png, which
> automatically resolves to server Y.
>
> This requires OpenLayers et al. to put that extra arithmetic in
> (calculating the hostname of a host where the tile will be found).


OpenLayers has a getURL method which is invoked for each tile.  It can be
overridden by a custom method and this is already done for OSM URLs.

It should be trivial to implement this although there seems to be some
additional mechanism (for local tile caching?) that might be affected by
dynamically changing the server component of the url.



To support clients that cannot use the extra logic required here, we
> could still provide a redirection service (such a service can be run
> anywhere, by anyone) that answers to the standard HTTP requests like
> http://tiles.freds-own-server.com/tiles/13/2468/4690.png, does the
> computing and then EITHER fetches and returns the tile OR returns a
> HTTP redirect.
>
> In a world like this, we would have:
>
> * Rendering client - renders as usual, uploads to any tile server;
> either you have your own tile server, or you use a friends's, or some
> tile server that has been set up for your area, or whatever. I think
> that for the time being we can work with configured tile servers in
> the rendering clients, and only later may we require a sort of
> "please tell me where I can upload to" mechanism.
>
> * Tile server - accepts uploads, notifies central server that it now
> has data for tileset so-and-so (perhaps also with timestamp so that
> central server can find out who's the most current) - optionally
> mirrors full tilesets from other servers
>
> * Central server ("Load server") - really only one big DNS server
> that contains a database with one entry per tileset, pointing to IP
> number of server carrying that tile; perhaps also records
> alternatives and does availability checks/round robin; accepts
> notifications from tile servers
>
> * Key server - not required as it's fascist ;-) leave it up to the
> tile servers whether they want to accept upload from just any
> anonymous client or whether they want to agree on some sort of
> authentication with their peers - not our business
>
> The main development issues here would be:
>
> * Create a "base" tile server for people to run, ideally a Debian
> package not dependent on PHP and MySQL, just a simple CGI that
> accepts an upload and stuffs it into the file system. People will of
> course be free to run more complex tile servers with lots of bells
> and whistles, but this will serve as the entry level thingie everyone
> can use.
>
> * Create the custom DNS server we'll need for the Load Server. This
> is definitely a C/C++ task. Needs MySQL as a backing store but must
> keep all addresses in memory. Should be doable with a sub-200 MB
> memory footprint. Cannot use bind. Database-backed DNSes exist; would
> have to check if they can be told to also have big memory cache.
>
> Bye
> Frederik
>
> --
> Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00.09' E008°23.33'
>
>
>
> _______________________________________________
> Tilesathome mailing list
> Tilesathome at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/tilesathome
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tilesathome/attachments/20070719/6adece92/attachment.html>