[OSM-dev] Slow rendering

[michael spreng]

For SOSM we use a similar, but probably a lot older server. 20 CPU cores
but also 256G RAM. Have you tuned your postgresql config? we use
something like

shared_buffers = 64GB
work_mem = 512MB
maintenance_work_mem = 4GB
checkpoint_completion_target = 0.9
wal_buffers = 16MB
default_statistics_target = 500
random_page_cost = 1.1
effective_io_concurrency = 200
min_wal_size = 4GB
max_wal_size = 16GB
max_worker_processes = 20
max_parallel_workers_per_gather = 10
max_parallel_workers = 20
max_parallel_maintenance_workers = 4

fsync = on
synchronous_commit = off
full_page_writes = off

(don't read too much into the exact values, it's just that some postgres
defaults can be way too low on such a huge machine)

We also use NVMe, as with SATA SSD io was usually saturated. RAM gets
almost immediately used up for caching. Not sure if your curve includes
disk caching or not.
Renderd should not crash with 64G RAM and no swap. If renderd starts
eating excessive RAM, restart renderd. I have seen it some times eating
all the memory.

We limit rendering to 10 threads. I just tested, at z15 I get about 1.5
meta tiles per second. Most of the work seems to be with postgres. CPU
was at around 1000% (10 cores saturated)



On 04.05.21 18:49, Alex Chirigiu wrote:
> On 04-May-21 17:32, Stephan Knauss wrote:
>> I hope that Alex is going to share some figures and optimization
>> results here on the list.
>> So @Alex: Even with you potentially paying someone for optimizing your
>> rendering stack, would be great if you could share the results so
>> others could benefit in the future as well. 
> -------------------
> 
> It might also be ESXI related. If I will find something, I will share it.
> That machine acts strange since it won't saturate anything else than RAM.
> 
> What we noticed to this point:
>     - eventually, it will eat as much RAM as you trow on it
>     - if you won't define a swap, it will crash *(so if you experience
> that, try to tweak that **value to 75% of the actual size of RAM)*
>     - CPU load is under <50% no matter what (we've tried anything from
> 12 to 32 cores)
>     - disk usage is around 1 mb/s with some spikes
> 
> This is what it will do a few hours after the render started
> *1) Huge spike on RAM and it will stay there till will go up again* (it
> wont get down again for nothing)
> 
> *2) Huge spike on disk and it will come back *(the controller and that
> RAID should sustain a lot more so I don't think this is the reason for a
> potential bottleneck)
> 
> *3) Nothing at all on the CPU *(this is nailed down at 39.xx% usage)
> 
> 
> 
> 
> Same settings on that Z600 (but without ESXI) will produce different
> results.
> 
> So the main questions here is this: *Does anyone have some experience
> with ESXI machines?
> 
> 
> *Best regards,
> Alex Chirigiu
> 
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
>