[OSM-dev] Proxy for the API

Dave Hansen dave at sr71.net
Mon Apr 16 19:13:33 BST 2007


Is most of the load on the servers a result of database queries?  Is
most of the database traffic from reads, or writes?

I was wondering if a caching layer in front of the database would be a
good idea.  If most of the load came from API queries that were somewhat
redundant, then it makes sense to start caching the results of those
queries and serving those up instead of querying the database again.

So, as a result of each database query for a rectangle of OSM data, you
take the results of that query (probably the raw XML that would have
gone back to the client) go make an entry in the cache.  On each API
query, you go see if any entries in the cache are completely encompass
the request.

At first, it can be simple, and only return cache entries for the
*exact* same query.  But, it could be expanded to do a bit more
comprehensive searches.

The other portion comes from the update side of the API.  Whenever a
write occurs, you have to go and look to see if any parts of the cache
should be invalidated.

The key to all of this is making a quick translation between the API
requests and the cache entries.  Hopefully, there is a scheme to do it
without very much overhead, possibly only a quick translation function
and a filesystem lookup.

I did something like this for a web server log analysis program that I
wrote, and it works pretty well.  I'm honestly very green, and know very
very little about the unique demands on the OSM infrastructure.

-- Dave





More information about the dev mailing list