[OSM-dev] Something wrong with the renderer?

Grant Slater openstreetmap at firefishy.com
Thu May 5 17:26:11 BST 2011


On 5 May 2011 16:15, Manuel Reimer <Manuel.Spam at nurfuerspam.de> wrote:
> Hello,
>
> currently it often takes days until I see a change, I uploaded, in the
> official Mapnik rendered map.
>
> In the past, rendering was much faster. Often changes were rendered within
> minutes. Is something wrong with the renderer?
>

Something wrong: no... But the server is completely overloaded yes.

Apps which scape masses of tiles for offline usage is becoming a big
problem. Some of these apps fake their User-Agent and other headers.
I've politely asked the heaviest of these commercial apps to move to
other tile servers as per our tile usage policy:
http://wiki.openstreetmap.org/wiki/Tile_Usage_Policy

Google Maps has been sending legal letters to tile scraping apps for
being in violation of their terms of service. eg: gmapcatcher, MOBAC,
NaviComputer, Locus Free and likely others, the apps have been
switching to using tile.openstreetmap.org as the default tile layer.

The scrapers cause problems because they normally attempt to download
all zoom layers available. Zoom 0 to 18. Only 6.69% of z16 has ever
been viewed (and therefore rendered). z17 2.50% and z18 0.90% [1].
When a request comes in for one of these layers the server normally
has to specially renders a copy, the server is only powerful enough to
render a few tiles per second. It is this rendering of tiles for
scrapers that is blocking up the render queue from serving fresh tiles
to mappers.

I've employed some techniques in an attempt to limit the scaper's
download rate while keeping things responsive to mappers. Token bucket
rate limiting [2]. There are also some upgrades in the pipeline.

1: http://wiki.openstreetmap.org/wiki/Tile_Disk_Usage (In total only
1.79% of the theoretical number of tiles have ever been viewed.)
2: http://en.wikipedia.org/wiki/Token_bucket

Regards
 Grant
 Part of the sysadmin team.



More information about the dev mailing list