[Tile-serving] [openstreetmap/osm2pgsql] Improve performance of processing diff files (#1528)

[doskabouter]

When processing diff files, the performance of processing nodes is dramatically (0.2k/s for append vs 800k/s for create) slow.

Based on my investigations the bottleneck is the 
PREPARE mark_ways_by_node(int8) AS  SELECT id FROM "southamerica"."planet_osm_ways" w    WHERE $1 = ANY(nodes)      AND "southamerica"."planet_osm_index_bucket"(w.nodes)       && "southamerica"."planet_osm_index_bucket"(ARRAY[$1]);
query. because it maxes out 1 core of my databaseserver. 
Adding --number-processes 5 didn't help here, is this part easily rewritten as multithreaded so at least this part of the process uses multiple threads for getting way ids for the changed nodes?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/1528
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20210616/077eb8fe/attachment-0001.htm>