[OSM-talk] Downloading lots of tiles from tile.openstreetmap.org

[Jon Burgess]

On Fri, 2007-11-09 at 22:44 -0700, Jason Reid wrote:
> Jason Reid wrote:
> > Jon Burgess wrote:
> >   
> >> We don't have any specific terms of service for downloading tiles from
> >> tile.openstreetmap.org and people often download thousands of tiles with
> >> scripts. Quite often people often submit requests for completely invalid
> >> tile co-ordinates. A couple of examples from this evening:-
> >>
> >>
> >> [09/Nov/2007:20:15:35 +0000] "GET /mapnik//12529/11592.png HTTP/1.1" 
> >>
> >> * The zoom level is missing from this request.
> >>
> >>
> >> [09/Nov/2007:22:40:08 +0000] "GET /20/528358/383233.png HTTP/1.1"
> >>
> >> * This client is trying to fetch zoom=20 tiles which have never existed.
> >> The highest zoom for the Mapnik layer is 18
> >>
> >>
> >> The examples have been repeated 100,000+ times from 2 IP addresses
> >> today. I don't want to restrict people from downloading lots of tiles
> >> but if people keep submitting bad requests then it is a complete waste
> >> of bandwidth and server resources.
> >>
> >> If you write tools to grab lots of tiles please at least try to ensure
> >> you are fetching valid tiles URLs!
> >>
> >> 	Jon
> >>
> >>   
> >>     
> >
> > One thing that I will check is if any are coming from the newest release 
> > of Maemo-Mapper for the Nokia tablets, the 2.0.3+ release increased the 
> > maximum zoom levels so its possible that it may be trying to fetch 
> > beyond what we have. I will test and confirm later tonight once I find 
> > where I left my N800.
> >
> > Jason
> >
> >   
> Alright, I can confirm that Maemo-mapper will try to access zoom 19 and 
> 21 tiles if you zoom into the maximum amount that it allows. I will 
> contact the author of Maemo-mapper and bring it to his attention. I can 
> see that there may be a large number of requests from this, as its not 
> uncommon to preload huge areas at once (its easy to ask for it to 
> preload over 500mb of tiles, which would likely include a lot large 
> percentage of tiles that are beyond our rendering limit).

I think it is Maemo responsible for attempting to download these zoom 20
tiles. Maemo uses gnome_vfs to download the tiles and has a loop where
it will attempt the download 3 times if it fails to get a good tile and
this matches what appears in the server logs:

"GET /20/596639/307644.png HTTP/1.1" 200 - "-" "gnome-vfs/2.12.0.19 neon/0.24.7"
"GET /20/596639/307644.png HTTP/1.1" 200 - "-" "gnome-vfs/2.12.0.19 neon/0.24.7"
"GET /20/596639/307644.png HTTP/1.1" 200 - "-" "gnome-vfs/2.12.0.19 neon/0.24.7"

Even though we return 200 OK, the returned tile is 0 size.

I have updated the tile serving script to return a 403 error for an
invalid zoom request but the Maemo code will still try 3 times
regardless.

Thanks for doing this Jason.

	Jon