[Tile-serving] [openstreetmap/mod_tile] memory consumption (#181)

[Stephan]

I think in this issue two different issues are mixed, both around memory not freed. The initial report was about the RSS of renderd growing.

I have with my recent setup (using tirex) a similar problem like described by @duerk-de. In that case the RssAnon of the postgres backends is growing. A single backend grow with a rendering queue fully busy during this time leads to a growth of roughly 3GB within 7 hours for each rendering instance:

RssAnon:  from 352.532 to 3.343.852, increase by 2.991.320

resetting the database connection frees this memory. I also have the suspicion that this is related to mapnik using persistent connections to postgis. By looking at the source code I think this is used in the same way in renderd and tirex.

A workaround with tirex is to send SIGHUP to the rendering backend manager. I would feel better if the root cause is understood.

Postgresql developer documentation suggests dumping the allocation structure with gdb, but I fear I won't understand the output without deeply digging into Postgresql.

> Examining backend memory use
> PostgreSQL's palloc is a hierarchical memory allocator that wraps the platform allocator. See #Why do we use palloc() and pfree() to allocate memory?.
> 
> Memory allocated with palloc is assigned to a memory context that's part of a hierarchy rooted at TopMemoryContext. Each context has a name.
> 
> You can dump stats about a memory context and its children using the MemoryContextStats(MemoryContext*) function. In the most common usage, that's:
> 
>    gdb -p $the_backend_pid
>    (gdb) p MemoryContextStats(TopMemoryContext)
> The output is written to stderr.
> 
> This may appear in the main server log file, a secondary log used by the init system for before PostgreSQL's logging collector starts, journald, or on your screen if you are running a backend directly without a postmaster.
> 

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/mod_tile/issues/181#issuecomment-808962223
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20210328/f3cd1347/attachment.htm>