[Imports] [Imports-us] [Talk-us-newyork] Update: New York State GIS SAM Address Points Import

[Skyler Hawthorne]

Kumi is the only option that doesn't heavily rate limit. I believe most others limit to 1 request per second. My importer performs on average 3 requests per address point, each request takes about 20–30 seconds, and each county has tens to hundreds of thousands of addresses.

I usually run my importer with 100 simultaneous requests at a time, and even so, with Kumi, a whole county can take between 10–15 hours. With a rate limited endpoint, it would simply be intractable.

I'm starting to regret my decision to use the Overpass. Had I known how poorly it performs, I would have done more research to figure out how to do the geospatial queries locally.

Feb 28, 2021 13:06:02 Mateusz Konieczny via Imports <imports at openstreetmap.org>:

> I had some issues with Kumi instance
> (I was lucky to have outright crashes).
> 
> I would recommend using instance
> used by Overpass Turbo by default.
> 
> At least for imports where it may
> produce bugs in data.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20210228/bfaea7e7/attachment.htm>