[OpenStreetMap] #4501: Include API usage policy technical limitations into Capabilities

[Kai Krueger]

On 7/31/2012 11:57, Frederik Ramm wrote:
> Hi,
>
> On 07/31/12 19:13, OpenStreetMap wrote:
>> #4501: Include API usage policy technical limitations into Capabilities
>
> [...]
>
>>   We are currently working on a change in JOSM in order to download data
>>   with several concurrent threads, up to to the maximum number allowed by
>>   the OSM API usage policy (currently 2):
>
> My suggestion: Reduce maximum number allowed to 1, and save you the hassle.
>
> Assuming that the API is not idle, I think any multi-threading
> performance gain on the client side is at the expense of other clients
> who then have to wait longer.
>
> I think everyone should just use API connection at a time.
>
> If performance gains are possible by querying the database in parallel,
> then these could be implemented in the API itself (thereby benefiting
> all users and not only those of one specific editor).

Which API calls are we talking about? In case of map calls, the limiting 
factor is likely the db and so indeed increasing the number of threads 
would probably be at the cost of others. This would then possibly 
balance it self out again (depending on the ratio of JOSM to other API 
users)

For some of the simpler API calls, like a single node, the bottleneck 
might be the network latency, in which case having multiple threads 
might actually help without hurting others to much.

On the other hand some of the more complex queries, like relation/full, 
the bottleneck is likely the rails processing. In that case, as long as 
there are still some spare CPU cycles left on the frontend/backend 
servers, again adding multiple threads might be a benefit. However, the 
backend servers will then run out of steam much sooner. Probably the 
better approach in this case is to move the relation/full API call into 
cgimap, as that gives roughly a factor ten in reduced CPU usage (server 
side) and a reduction to about 1/3 total processing time (at least last 
time I benchmarked things).

Kai

>
> I like the general idea of adding policy stuff to the capabilities
> response (see
> http://wiki.openstreetmap.org/wiki/User:Frederik_Ramm/Ideas_for_API_0.7#Extend_.22Capabilities.22_Response_with_Policy_Information
> which I incidentally wrote only recently), but I don't like the idea
> that some clients should grab a larger share of API resources than others.
>
> Bye
> Frederik
>