[Osmf-talk] Data Was: Re: Tiles

[Steve Coast]

On Jun 3, 2023, at 9:10 AM, Peter Gervai via osmf-talk <osmf-talk at openstreetmap.org> wrote:
I noticed in this thread that some people are lack the actual data
required for this decision: some thought the said traffic is a real
burden on the foundation servers while others say that it is almost
negligible.

This is an interesting question.

TL;DR; OSM ships approaching a billion tiles a day and OSM is only 10-20% of the traffic.


If you look at the tile stats - https://planet.openstreetmap.org/tile_logs/hosts-2023-06-01.csv

Then the first few lines look like this:

$ head hosts-2023-06-01.csv
"openstreetmap.org",1736.1112962962964,481.8022800925926
"localhost",247.9595023148148,26.77423611111111
"openrailwaymap.org",172.78620370370368,22.316759259259253
"kadastr.live",170.78546296296295,35.2583912037037
"bidenlaptopmedia.com",163.11689814814815,1.26375
"myrouteapp.com",119.45278935185185,24.105104166666667
"mondialrelay.fr",108.20900462962963,3.983298611111111
"qualp.com.br",95.50988425925927,0.7505092592592592

So you can see osm.org<http://osm.org> + localhost are the top two with roughly 2k requests a second (the second column), or roughly 173 million tiles a day.

The other hosts (571 in total) quickly drop off as you can see from 1736 requests a second down to only 95 requests a second by the 6th non-core OSM website, “qualp.com.br”. If you look, we’re supporting many large companies public and private, and huge governments with our free service.

When we’re told that other sites are a small fraction this is indeed true but sidesteps a well known problem in long tail statistics - what happens when you add up all the other sites? If you add them all up there are ~10k/requests a second across the data given.

That means that OSM is only using 19-20% of our tiles. We support the worldwide internet with a free minutely updated fileserver with 80% of our traffic.

I’ve tried a few random days and it’s always the same, we are using ~20%, we give away ~80% to bidenlaptopmedia.com<http://bidenlaptopmedia.com> et. al.

However, the localhost HTTP referrer is most likely people using QGIS, developing websites on local machines, etc etc So not OSM. The logs don’t include requests below 5/second or various errors…. So we can guesstimate that the actual number is closer to 10-15% OSM traffic.

So it’s fair to say OSM is giving away ~80-90% of our tiles to non-OSM users. I wonder if our hosts actually know this?

Which brings us neatly to the theory that it’s all fine because someone is doing this for free. It strikes me this is like saying the roads are free because the government is paying for them. But more precisely, OSM(F) has an opportunity cost where donations (80-90% in this case could) be better managed or spent. It has a cost in terms of relationship and management. It’s not free at all.

Best

Steve

PS - it’s arguable obviously that some key sites like openrailwaymap should be allowed to use OSM resources
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmf-talk/attachments/20230605/10cb0e95/attachment.htm>