[OSM-dev] Report on wiki.osm

[Nick Hill]

Yesterday and today, wiki has had some down time.


A few days ago, I received a report that mediawiki was running out of memory and 
a suggestion to increase the memory available to scripts running under PHP.

I increased the memory available to PHP from 8Mb to 12Mb. Looking at the failure 
this morning, it was clear from logs that the kernel ran out of logical memory, 
causing applications to close, or go catatonic.

I have responded by squeezing MySQL process memory and reducing the maximum 
number of apache instances. I have also set apache so that it will not spawn 
apache processes to handle peaks in demand. The rationale being that we'll have 
apache and MySQL using a good deal of available resources anyway and spawning 
new processes will only cause problems with further I/O and CPU loading.

I had acted on suggestions in October to increase the memory for MySQL caching 
to improve mediawiki performance. I have reverted the settings in favour of 
preserving more available memory. My subjective tests this afternoon show no 
appreciable negative impact, and should make the system more robust by 
maintaining adequate reserve memory for the system to stay up in periods of peak 
demand. It is better that at times of very high demand, the odd page fails to be 
delivered, than the machine being dragged down.

Given the resources we are running mediawiki on are probably adequate but 
finite, and we can have very sharp spikes in demand, and the machine mediawiki 
is on serves other clients, I don't see any reason to consider occasional pages 
not being delivered as a fault - it is part of the design - the system is not 
designed to handle any level of demand. Of course, if it becomes common, then it 
is a fault.

Given that all visitors tot he domain www.openstreetmap.org are redirected to 
the wiki front page, it may be an idea to take snapshots, perhaps every half 
hour, of the wiki front page as the front page for openstreetmap. This will 
reduce wiki load. We can use a base href tag to make sure links in the page 
remain unbroken.