[Tile-serving] [openstreetmap/osm2pgsql] Slow import (#1397)

[trohsb]

Hi,

I'm trying to do a planet import on my machine but the Way processing is quite slow. 

`Processing: Node(6616414k 1154.7k/s) Way(133713k 5.37k/s) Relation(0 0.00/s)`

Processing the Nodes is CPU bound and processing the Ways is IO bound.

My osm2pgsql command is:

```
osm2pgsql -d gis --create --slim -G --hstore --tag-transform-script /home/osm/openstreetmap-carto/openstreetmap-carto.lua -C 32000 --number-processes 8 -S /home/osm/openstreetmap-carto/openstreetmap-carto.style planet-210111.osm.pbf
```


PostgreSQL settings:
```
shared_buffers = 4GB
autovacuum_work_mem = 2GB
effective_io_concurrency = 2
fsync = off
synchronous_commit = off
full_page_writes = off
checkpoint_timeout = 10min
max_wal_size = 1GB
min_wal_size = 80MB
checkpoint_completion_target = 0.9
autovacuum = off
effective_cache_size = 24GB
```

Hardware:
CPU: Intel® Core™ i9-9900K
RAM: 96GB 3200MHz
SSD: Force Series™ MP510 1920GB M.2

My SSD gives the following speeds from hdparm:

```
sudo hdparm -Tt /dev/nvme0n1
/dev/nvme0n1:
 Timing cached reads:   38672 MB in  2.00 seconds = 19369.32 MB/sec
 Timing buffered disk reads: 4018 MB in  3.00 seconds = 1338.89 MB/sec
```
Is there anymore I can do to speed this up on my hardware?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/1397
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20210120/0558f1f1/attachment.htm>