[OSM-talk] Overpass Quotas

Roland Olbricht roland.olbricht at gmx.de
Mon Apr 27 20:27:31 UTC 2015


Dear all,

in the last weeks the server overpass-api.de has seen a flood of extra 
requests. Unfortunately, most of these requests have been from a few 
clumsy clients.

The standard pattern has been apparently a server that fires as much 
requests as possible, often some or most of them syntactically correct, 
but semantically nonsense. For example, from one IP adress was sent the 
same query for objects with name "Lipkenskothen" every 5 seconds (It is 
very unlikely that these few results change more often than once per 
month, hence a frequency of once per day would be more than enough).

While I may block such extreme cases with "iptables" (i.e., the server 
then appears unreachable from that IP), I would like to avoid manual 
blocking as much as possible. Beside the quite hard challenge of being 
unbiased in blocking decisions, it is also a lot of annoying work.

For that purpose I have changed the quota calculation. I will uphold the 
basic rule that a client shall not send more than one request in 
parallel from an IP. For the other details, I'm trying an A/B-strategy 
for traffic management, hence they may change quite often at the moment.

If you encounter a serious service degradation, in particular sustaining 
HTTP 429 responses, please contact me by mail such that I can adjust the 
algorithm to make fewer or no false positives.

Cheers,

Roland



More information about the talk mailing list