[OSM-talk] tile downloader

Grant Slater openstreetmap at firefishy.com
Thu Oct 21 15:27:49 BST 2010


On 21 October 2010 14:32, MP <singularita at gmail.com> wrote:
>> Maybe we could work around this by automatically changing the link for
>> the stored tiles? This would also harm "friendly" projects with small
>> tile-download-rates though. If it is technically possible to identify
>> this application they could also be filtered out.
>
> But what prevents the "evil" applications to set same user agent as good ones?
> Not a good idea.
> This would break many things, for example the slippy map preview in
> JOSM, without managing to do what is intended (stop bulk downloaders -
> as if one downloader does not work, user will just pick another that
> will - or wait for update to existing one)
>

The tile server already automatically rate limits client connections
after they have bypassed a particular rate and usage limit.
Scraper apps which download huge uncached areas will run fine for a
few minutes until they hit these limits after which their connections
are slowed down. Within a few minutes of backing off the rate limits
are removed.

Technical: Large set of squid delay pools in a reverse proxy
configuration. High bucket size with a low top-up rate.

Regards
 Grant
 Part of OSM sysadmin team.



More information about the talk mailing list