[Tile-serving] [openstreetmap/mod_tile] `render_list` not using all cores (Issue #378)

[John Coggeshall]

I am building tiles with `raster_list` and `meta2tile`, and I have noticed a performance issue I can't figure out why is happening that maybe someone here can shed some light on.

I am using a `c5d.12xlarge` instance on AWS which has 48 cores and 96G of memory backed by a 2TB SSD drive. I am rendering the North America dataset as raster tiles, and this is the `render_list` command I am using:

```
render_list \
    --all \
    --force \
    --min-zoom=0 \
    --max-zoom=18 \
    --max-load=$(nproc) \
    --map=current \
    --num-threads=$(nproc) \
    --tile-dir=/osm-data/renderd/tiles
```

Things run fine, but I've noticed that my load never has gotten past 32, about 60% of my total CPU capacity. I know that, for example, if you don't specify `--max-load` then `render_list` will throttle itself pretty aggressively at 16 cores... but is there some other setting (compiled in or otherwise) that is limiting `render_list` to 32 cores? I've even tried increasing `--num-threads` to 2x my cores to absolutely no effect. 

Other facts:
  - I have plenty of free RAM (consuming 32G of 92G available)
  - My Disk Writes seem to not impact at all... sometimes my writes are like 100-1000 K/s sometimes they are in the MB but neither has any impact on my load.
  
 Unless I'm misunderstanding something (possible), it seems like I'm not disk bound, or memory bound -- yet `render_list` won't take advantage of the extra 16 vCPUs available to it.

Thoughts anyone?

-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/mod_tile/issues/378
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/mod_tile/issues/378 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20240122/e55080c8/attachment.htm>