[Tile-serving] [openstreetmap/osm2pgsql] limit the number of buffers that are in the copy queue (#960)

[Sarah Hoffmann]

I don't understand your comment. The issue here was that osm2pgsql was producing data for postgresql at a faster rate than postgresql can consume. The Max_buffers parameter has been explicitly introduced to limit the number of data buffers we queue for postgresql. There is not much point in making it larger as it only has to account for the jitter when producer and consumer are working at about the same speed.

Imports without flatnode files are hit especially hard because producing the data for `planet_osm_nodes` is blazingly fast. You pretty much always end up with the larger part of the node data in RAM or with an out-of-memory condition. There is a chance though to run into an OOM at other stages as well when postgresql is badly configured or the disks are very slow.

I'm against adding more obscure performance parameters. People are struggling with -C enough already.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/pull/960#issuecomment-541929948
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20191014/d7ec4e04/attachment.html>