[OSM-dev] Paging the tile server admins

[Grant Slater]

Hi NopMap

On 27 April 2011 09:29, NopMap <ekkehart at gmx.de> wrote:
>
> Hi!
>
> Paging the admins of the OSM tile servers. I'd like to get in touch about
> the topic of mass downloaders. It appears that some problems first become
> visible on small tile servers like my own, but hit the main tile servers a
> while later, so it might be a good thing to share experiences. E.g. MOABC
> was an issue for my server in december and had to be blocked on the main
> servers about three months later.
>

Yes, the MOBAC app is becoming a serious problem for the tile.osm.org
server. Earlier versions fake a Firefox user-agent. I have had to
reduce max tile QOS limit downward in an attempt to control the
situation. Allowing mass tile scraping is not sustainable.
A lively thread when I restricted the Faked user-agent:
http://sourceforge.net/projects/mobac/forums/forum/861096/topic/4423122

tile.osm.org traffic has recently peaked around 300Mb/s.

> A few days ago a new mass downloading application has appeared on my server,
> already causing tile requests in the order of millions with up to 600000
> tiles for a single client. The user agent is "Profile/MIDP-2.0
> Configuration/CLDC-1.1" which indicates some sort of handy software, but a
> search in the net brought up no clues as to what exactly this might be. Do
> you have any suggestions?
>

I too am perplexed by the MIDP user-agent. I believe the user-agent is
from Java Mobile Edition. Some? BlackBerry's prefix their user-agent
eg: "BlackBerry9800/6.0.0.246 Profile/MIDP-2.0 Configuration/CLDC-1.1".

Basic tile.osm.org user-agent stats:
http://grant.dev.openstreetmap.org/tmp/logs/yevaud-user-agents-16042011.txt.bz2

Regards
 Grant