[OSM-dev] C++ implementation of the API

Ludwig ludwigbrinckmann at gmail.com
Mon May 26 16:05:11 BST 2008


My concern is that even if a reimplementation of the API in some other form
will not fundamentally solve the scalability problem if all data requests
will have to be served by a single machine.

Will not the introduction of the 0.6 API offer a solution in the sense that
will will with far greater reliability allow the separation of read and
write requests? With versioned map elements all read requests could be
redirected to some mirror server, while only write requests will continue to
hit the real (master) server. With read requests removed from the main
server, it should be able to handle serve more active mappers better than at
the moment.

Mirror servers, in the form of OSMXAPI (which claims to be only a few
minutes behind the real DB nowadays), could then handle the bulk of the
requests.

Could the effort for a API reimplementation be directed to some sort of
smart mirror service using HTTP redirects, which would shield client
applications from the existence of the mirror services?
A first version could redirect read-request to a single read-only server,
while future versions could smarten up and be specific on bounding boxes,
allowing for a regionalisation of mirror servers, which would not have to
host the entire world...
Much of that could probably be achieved by URL rewriting in Apache.


Ludwig




2008/5/26 Tom Hughes <tom at compton.nu>:

> In message <1211658532.3899.18.camel at test.asus>
>           Joachim Zobel <jzobel at heute-morgen.de> wrote:
>
> > Am Donnerstag, den 22.05.2008, 00:06 +0100 schrieb Tom Hughes:
> > > > Writing Apache modules in C is hard, and I don't think using mod_cpp
> > > > will make it much easier. Doing Apache modules in Perl (mod_perl has
> > > API
> > > > access including filters) is a lot easier.
> > >
> > > Ye gads no. We want to keep the memory footprint under some sort of
> > > control so mod_perl is a non-starter.
> >
> > Everybody seems obsessed with low memory footprint. Why? Memory is
> > cheap, and we are not planning an embedded system. What is the number of
> > concurrent requests we need to handle?
>
> I'm not obsesses with memory footprint, but with our current code
> memory is far and away the biggest problem.
>
> Currently we can handle about six simultaneous requests to the data
> heavy API calls - that is basically map data downloads, gps data
> downloads and potlatch calls.
>
> The reason is that we have to allow about 600Mb or so for each call
> and that quickly mounts up as you try and add extra simultaneous
> accesses.
>
> Factor in the fact that many of these calls may take a minute or
> more and you have a serious limitation to the load we can support.
>
> Of course we can just add hardware, but we should be able to reduce
> the menory footprint dramatically as well.
>
> Tom
>
> --
> Tom Hughes (tom at compton.nu)
> http://www.compton.nu/
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20080526/f78e4745/attachment.html>


More information about the dev mailing list