[OSM-dev] API/XAPI caching proxy server

[Emilie Laffray]

On 17 December 2010 00:45, Stefan Keller <sfkeller at gmail.com> wrote:

> Hi Brett
>
> Thanks very much for your detailed instructions.
>
> > In my experience the biggest limitation in performance is disk seeking,
> > rather than the amount of data returned.
>
> If that's the bottleneck (or the amount of data returned before
> processing), then pl/pgsql or pl/python could help, since stored
> procedures are close to the data.
>

Having done some test some time ago, having functions in pl/pgsql doesn't
have much impact in terms of performance overall. The major impact is still
by far the IO. In my environment, I partially resolved the issue by the fact
that our search pattern in the query are going to be localized.
I ended up writing a multithreaded component that would rearrange queries to
stopped working in FIFO mode. The queries were rearranged by proximity and
send to the database in that order, in order to maximize the effects that we
got Postgresql caching the data from the first query. Depending on the type
of load we were getting, this was giving us some non negligible boost, while
in the worst case scenario, it was giving slightly less performance (due to
the reorganizing of queries).
I know that the OSM admins are looking into the replication from postgresql
9.0. There are still some little issues to be solved due to some temporary
tables not usable on replicated database.

Emily Laffray
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20101217/ba40c54f/attachment.html>