[OSM-talk] Exceeded API bandwidth limit, now what?

[Michal Migurski]

On Sep 15, 2010, at 8:45 AM, SteveC wrote:

> On Sep 14, 2010, at 12:02 PM, Michal Migurski wrote:
> 
>> Thanks guys. I understand about the extracts, I've used them extensively for years.
>> 
>> I'm experimenting with a way to get at smaller areas of OSM data (generally city-sized) for a possible update to http://tiledrawer.com, and I'm hoping to understand how to both work within the API limitations and be able to piecemeal together a town-sized area without requiring end-users to deal with bzip files or osm2pgsql on their own.
>> 
>> The code I'm developing is here:
>> 	http://github.com/migurski/TileStache/blob/osm-mirror/TileStache/Goodies/Providers/MirrorOSM.py
>> 
>> It's a provider class for Tilestache that mirrors OSM on a tile-by-tile basis.
>> 
>> Is there any interest here in publishing the OSM API via tile-like URLs? For example, being able to make a request like this to pull a chunk of bounded XML cached out of the OSM API:
>> 	http://tile.openstreetmap.org/14/2627/6331.xml  <---- note "xml" on the end
>> 
>> The advantages with this should be plainly obvious: a source of data that's trivially cacheable, on the order of hours-to-days old, and available for specific areas of the world, without the massive download and parse overhead of OSM extracts.
> 
> Can't you do the hours-to-days old with diffs or your own api server based on diffs? The key is that it's out of band of the main API. Granted you appear to be breaking new ground here - perhaps what you *really* want to work on is something like tiledrawer - where tiledraw wakes up on an ec2 image, grabs data and begins rendering tiles (from my limited knowledge) - you want APIdrawer which wakes up and starts serving an up to date read-only API automatically? For use with tilestache clients on the net, or whatever.


Yeah, that's actually quite close. Or maybe, more accurately, a read-only database slave whose sole function is to serve up a GET-only API to enable the creation of things like tiledrawers (or just very local renderings). I don't want to just make another layer of abstraction on EC2, I'm more interested in OSM itself supporting something like reliable XML retrieval. Even now, the extracts have a "temporary" smell about them, coming as they do from non-openstreetmap.org domain names.

Things like the TRAPI linked earlier are absolutely on the right track, though a bit slow with requests. I could also build all this myself, though I'm not quite ready to go a-yak-shaving with this particular idea. Probably the best thing for me would be to shelve it until OSM.org feels prepared to support on-demand XML at a city scale, at whatever speed is sustainable.

-mike.

----------------------------------------------------------------
michal migurski- mike at stamen.com
                 415.558.1610