[Openstreetmap-dev] Cache the whole world.

[Lars Aronsson]

Immanuel Scholz wrote:

> After reading Lars's measurements, I came up to the (probably very old)
> idea of having the tile server cache the background images fetched from
> NASA. ;)

Yes, I think this has been suggested before.  But Steve also plans 
to download the entire image dataset and generate satellite image 
tiles locally, instead of calling NASA's server.  I have no idea 
what effect that would have on performance.  I guess it depends on 
how much CPU power is needed to generate the satellite image 
tiles.  If it adds CPU load, your suggestion to cache satellite 
image tiles might still make sense.  I also have no idea about 
Steve's time plan or how we can help him.

But still: Suppose that satellite image tile generation (at NASA) 
takes 2 seconds and that drawing roads adds 1 second.  Serving a 
tile from a Squid cache apparently takes 0.15 seconds.  Thus, 
caching the satellite images could reduce the response time from 3 
seconds to 1.15 seconds.  Our currently most important problem, 
however, is that roads that should be drawn are only drawn in 18% 
of the cases.  What we have is essentially a (more or less slow) 
viewer for NASA's satellite images, not a viewer for OSM road 
maps.  The slowness is bad, but it is not our worst problem.

> Lars: Can you rerun your script, this time not accessing the 
> tileserver from OSM, but directly the URL from the NASA server 
> where the tile server get its pictures from? And better: can you 
> do this from a shell from the development server (which should 
> be the same performance than the tile server ,since they are 
> placed in the same network, right?)

I see your point, but I'd like to see the 18% problem (ticket 134) 
solved before we go further into performance issues.  And perhaps 
Steve can tell us about the plans to store the satellite images 
locally?

> Aren't the 48 hours because some people complaining that they 
> are note able to correctly invalidate the cache and don't see 
> their map changes drawn in OSM?

The 48 hours is the worst kind of compromise.  It's too long when 
updates are made, and it's too short when no updates are made.
What we need is a way to invalidate the cache on updates.  Then 
the expiration time can be set to two weeks, months or years.
I don't see any trac ticket for this need.  I was almost certain 
it had been suggested already.  Should I add a ticket?  Wasn't it 
mentioned that Squid has some API for this?


-- 
  Lars Aronsson (lars at aronsson.se)
  Aronsson Datateknik - http://aronsson.se