[OSM-dev] Proposal: Database accelerator and dirty tile marker based on simple algorithm.

[Lars Aronsson]

Nick Hill wrote:

> A server upgrade myself and Steve completed a few months ago 
> gave us good performance for a short time. The system then 
> became extremely slow as demand caught up with supply. I 
> recently isolated an SQL query which I posted an improvement 
> for, which was soon improved again. This has led to the current 
> state of affairs where the database and API systems are easily 
> able to cope with current demands. I have made further database 
> optimisations which provide further growth potential.
> 
> It is only a matter of time before demand and OSM growth rate 
> bring the system back to it's knees.

What I see here is urban legends, myths and guesswork.  Yes, you 
made some changes to the SQL, for example on August 31.  But what 
was the load before that and how did it change afterwards?  I see 
no statistics at all on how many requests are served.  So how can 
you say that "demand" has increased?  My guess is that since the 
system appears to be slower, you guess that demand has increased. 
Since I don't like to guess, I'm asking: What do you base this 
statement on?

Now the tile server works so-so on some days and is completely 
dead on other days, but the reason is a complete unknown.  All I 
know for sure is that *I* am not trusted with a login account on 
the tile server, so *I* cannot look in the log files what's going 
on.  I'm strapped to the backseat, so you can blame me for being a 
backseat driver all you want.  It's quite frustrating.

> Looking for method changes which can improve performance 
> possibly by another order of magnitude is really what we need.

No, what we need is measurement.  How many calls are served, how 
much time does each call take, and what part of the call is taking 
time?  Without measurement, every change is a guesswork.


-- 
  Lars Aronsson (lars at aronsson.se)
  Aronsson Datateknik - http://aronsson.se