[OSM-dev] OSM API Performance (Was: OSM Database Infrastructure)

[Christopher Schmidt]

On Sun, Jan 20, 2008 at 10:17:21AM +0000, Tom Evans wrote:
> Christopher Schmidt wrote:
> > I'm working on creating interfaces that display OSM data as you 
> > drag
>  the map around. The interface requests data from OSM 
> > "on the fly" -- so every time the map stops moving, the old features 
> > are removed, and a new request is sent out. 
> 
> I had thought about something similar, but I must admit I hadn't even 
> considered sending the requests to the main database API.  Data a 
> few days to a week old is adequate, so surely OSMXAPI or even a 
> server with pre-tiled static .osm files (if the volume really picks up) 
> would be better.  
> 
> Scaling read-only requests with multiple servers is easy, but read/write 
> much harder.  Better to avoid hitting the core database unless you're 
> using an editor and actually need to check the up-to-date data.

Let me state some of the use cases I'm currently working against:
 * The Coastline viewer displayed a small section of disconnected
   coastline after last night's update. These are disconnected nodes
   that I'm having problems finding in JOSM. I want to display the
   section of coastline, select them in OpenLayers, and delete them.

   Potlatch won't work, because I can't see the coastline errors in
   Potlatch, and the section of map I'm working in is simply a total
   mess coastline wise, and I can't find the section that coastline
   editor is complaining about. 

 * I'm going on vaccation in a couple weeks, so I've been tracing out
   my destination against Yahoo!'s imagery. Then, when I'm there, I want
   to use my OpenLayers editor (displaying tiles from my locally loaded
   planet) to edit the streets I haven't gotten yet. I can download data
   from the local db or from osmxapi -- but as soon as I want to edit a
   name of a road, I have to switch to the real API, because otherwise I
   might be editing against old data.

 * I'm editing and want to understand the tagging scheme that someone
   used on a particular set of data, so I can mimic it because it seems
   like what I'm looking for. I want to download the tags, click them to
   view them, and then use them i my other editor.

   This case is obviously served by the osmxapi without any problems.

> Having said that, I agree it does pause a bit sometimes.  But scaling 
> the read-only access and keeping the read/write access stable and 
> bug-free seems more sensible to me.

Sure. Maybe the answer is "Sorry, waiting 5s for a request to start is
the penalty you pay for needing read-write access." If so, that's fine.
But if the reason that that is is because we don't have hardware -- that
is, it scales trivially, as compared to the database being slow, which
clearly does not scale trivially -- then it's good to at least know
that's the reason.

Regards,
-- 
Christopher Schmidt
MetaCarta