[Tile-serving] [osm2pgsql-dev/osm2pgsql] Any reasons why osm2pgsql doesn't leverage parallelization during huge maps import (eg: whole world)? (Discussion #2090)

[mboeringa]

>PS: the system I'm running on is a 22 cores 120GB RAM VM running on a server with 2 x Xeon v2 (24 cores total)

>On my side i have 2 x XEON E5-2630 v2.

You first suggested you had 22/24 real cores, but that processor only has 6 real cores, 12 threads. It also has pretty low single thread performance, which willl hurt, and multi-threaded performance is also nowhere near the top of its generation of processors:

https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E5-2630+v2+%40+2.60GHz&id=2052

Upgrading your processors (these v2 processors should be dirt cheap nowadays on the second hand market), to something like the Xeon E5-2697 v2, should give you a considerable boost and cost little:

https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E5-2697+v2+%40+2.70GHz&id=2009

It is not that difficult to replace CPUs if you have any experience with hardware. I swapped my dual Xeon E5-2680 v4 14C/28T processors for two Xeon E5-2699 v4 22C/44T processors and got about a 40%-50% boost in multi-threaded processing.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/osm2pgsql-dev/osm2pgsql/discussions/2090#discussioncomment-7194079
You are receiving this because you are subscribed to this thread.

Message ID: <osm2pgsql-dev/osm2pgsql/repo-discussions/2090/comments/7194079 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20231004/c7526654/attachment-0001.htm>