[OSM-dev] API/XAPI caching proxy server

[Graham Jones]

Wyo,
Last time there was serious trouble with XAPI servers I started to think
about setting up my own server (at least for a limited area), but got stuck
with the technology that XAPI uses.
I did start to write my own version of it but didn't get far (I made the
mistake of trying to use an off the shelf parser writer, rather than just
writing the URL parser myself).   Then someone fixed the servers (thank
you!), so I lost interest.

My little (low load) web application uses XAPI to get bits of up to date OSM
data on demand.  This works fine while XAPI is working, but makes the
application very unreliable if XAPI is struggling.  Therefore I would be
interested in helping to develop some sort of distributed XAPI system.

I don't know how your proxy proposal would work, but the way I thought it
could be done is if users run a XAPI server for a particular region, they
could register their servers with a 'master' one that actually receives user
requests.  The master server could chose which machine to send the requests
to based on the area covered and their load.

Obviously that distribution architecture would need some careful design, and
it would have to be fairly easy for users to set up their own instances (I
would have thought a postgresql/osmosis based set up?).

Let me know if you are going to try to set something up and I will see what
I can do to help (but I don't know much about network technologies).

Graham.

On 11 December 2010 10:41, Wyo <otto.wyss at orpatec.ch> wrote:

> After being told to create my own API/XAPI instance (
> http://gis.638310.n2.nabble.com/XAPI-server-unusable-tt5796609.html) I've
> looked at how to set up a server. Well the current procedure using osmosis
> etc is simply impossible for me and I guess for a lot of others as well. IMO
> if it were easier to set up a server, many more would do it, meaning a lot
> less load on the current infrastructure. IMO a API caching proxy would do
> the job as well and doesn't need to handle Gigs of files.
>
> Lets say such a proxy exists, than requests to the API are directed at the
> proxy and only if the proxy doesn't contain the data, the request is
> forwarded to an API server. Assuming further such proxy exists on the
> internet anyone allowed could use such a proxy. So in the future only
> proxies may access the API server directly, any user has to connect via a
> proxy. And if this still means too much load on the API server, primary and
> secondary proxies could be created etc. This works similar as a P2P network.
>
> To have as many as possible proxies running in the internet means to stick
> with technologies easily available. That limits the choice almost entirely
> to a PHP/MySQL solution, at least for the beginning.
>
> Wyo
>
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>



-- 
Graham Jones
Hartlepool, UK.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20101211/53635376/attachment.html>