[Tile-serving] mod_tile causes segfault on debian 7.0
kakrueger at gmail.com
Wed May 22 16:47:14 UTC 2013
On 05/22/2013 05:49 AM, Tom Hughes wrote:
> On 22/05/13 12:34, Andy Allan wrote:
>> On 22 May 2013 12:06, Sven Geggus <lists at fuchsschwanzdomain.de> wrote:
>>> I get segmentation faults in mod_tile on a more or less regular
>>> basis which
>>> will leave apache in a 100% CPU on all cores state!
>> Ah, that's interesting, I'm seeing something very similar. I'm using
>> 1341e129e5 (22 April) with apache2-mpm-event 2.2.22 (on Ubuntu 12.04)
>> I'm not sure that commit is necessarily involved, since it doesn't
>> change anything for apache 2.2, but perhaps the bug was introduced
>> elsewhere and that commit is just a symptom? I'm just using renderd
>> with a normal disk backend, if that helps.
> The point is that before the pluggable storage backends were added
> that code didn't exist at all.
> Once they were added the use of the process pool in a function that
> might be called in multiple threads at once was not robust, so that
> commit was added to make things robust.
> Unfortunately there is no way (that Kai has found) to get a per-thread
> memory pool on 2.2 so the old code is still used there, which will
> quite likely crash if you are using a threaded mpm.
That summarizes the issue with that commit fairly well.
In apache 2.2. the storage backends are keyed off of the process memory
pool and are initialised once per process. Both the worker and event
mpms however use a combination of processes and threads. As indeed I
haven't found a way to tell if you are in a new thread (Apache 2.4 fixes
that flaw), I could only do the storage backend initialisation per process.
However, this should only effect storage backends that need a unique
context per thread and their underlying libraries are not thread safe.
This should not be the case for the file backend, as it does not have
any per storage backend context, and all of the file system api's
should be thread safe.
The main problem is the memcached storage backend, as you need to pass
in a memcached context to each libmemcached function and those need to
be unique per thread, as libmemcached itself does not do any locking on
the passed in context.
So, unless you are using the memcached backend, this commit in theory
should be fine and I have done a bunch of load testing with ab to
confirm this. (Although limited ab testing can't properly simulate real
world load). I'll have another look through the file system backend, if
I can see anything that might be thread unsafe.
There is a good chance though that it is caused by something else entirely.
Is there any chance you could get a more detailed backtrace (with debug
symbols) to identify where in mod_tile it seg_faults?
More information about the Tile-serving