[OSM-dev] Overpass-API 0.7.52 : HTTP 429 error even after kill_my_queries

Pierre Béland pierzenh at yahoo.fr
Sun Jun 12 17:32:54 UTC 2016


Hi François,
Like you, I do not need immediate results. Last year, I did not succeed querying for large regions. This is easier now, but I experiment similar problems as you. 

I have better response if I query after midnight UTC. It would be interesting if the OverPass team could provide us hourly statistics of the server usage. 

 When the request bounce,  I wait a minimum of 5 minute before retrying. Sometimes, it works, sometimes, it bounce again. Then I wait a longer time before retrying, presumably because the server is too busy.

Pierre 

      De : François Lacombe <fl.infosreseaux at gmail.com>
 À : mmd <mmd.osm at gmail.com> 
Cc : dev at openstreetmap.org
 Envoyé le : Dimanche 12 juin 2016 12h19
 Objet : Re: [OSM-dev] Overpass-API 0.7.52 : HTTP 429 error even after kill_my_queries
   
Hi and thank you both for this information :)

Indeed, changelog told me 7.52 introduces this new rate limitation rule.
No problem for me : I've added a new waiting time in case of 429 error to retry up to 4 times (delay is increased as much as failed tries).
The involved script is just a daily (or weekly) pull component which doesn't require to get instant data as you may need to. So I can wait couple of minutes to retry.

The issue I see is since we can't determine how high is our load on the server side, we can't choose the best time to send a request.
429 error shouldn't return plain html but a time to wait until next call (for the same request if the time to wait depends on the size of the request).
Basically, plain html output isn't easy to handle for a machine. oAPI may look at Accept header to choose the best way to interact with the client.
I'll post it in the github pull request.


Cheers

François


François Lacombe

fl dot infosreseaux At gmail dot com
www.infos-reseaux.com
@InfosReseaux
2016-06-12 11:51 GMT+02:00 mmd <mmd.osm at gmail.com>:


> there are basically two issues here:
>
> 1. kill_my_queries currently has no effect. The query keeps on running
> on the server.
>

Small addition: in your case, kill_my_queries confirmed that there's no
query running on the server.

If you ever get a message, that the query was killed, you can assume
that the query still continues running on the server (at least as long
as the bug is not fixed).

In that case, I would wait for some time, try to kill the query again,
until you finally get a response that no query is running anymore, etc.

--




_______________________________________________
dev mailing list
dev at openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev



_______________________________________________
dev mailing list
dev at openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20160612/9af2044f/attachment.html>


More information about the dev mailing list