<p>I am currently having huge problems importing the whole planet. We are currently already waiting more than 1.5 days and it's still not done.<br>
And our machine is quite fast. We have 250GB RAM and 32 Cores. And all is running on SSDs...</p>
<p>I've read that the <code>--slim</code> should be used for the whole planet. Without this param we went out of memory quite fast.</p>
<p>So this is how our import statement looks currently:</p>
<pre><code>osm2pgsql -S my.style -c -H xxx-d x -U xxx --slim --drop --cache 100000 --number-processes 8 planet.osm.pbf
</code></pre>
<p>The problem is that currently there seems to be a lot of postgres threads going on. <code>CREATE TABLE</code>, <code>COMMIT</code>, <code>PREPARE insert_node/way/rel</code>... those kind of queries.</p>
<p>But those are stuck waiting for IO.<br>
We did define that osm2psql can use about 100GB of ram, but it's currently "only" using about 50GB?!?</p>
<p>Here's an extract of our postgres-config:</p>
<pre><code>shared_buffers = 4GB
work_mem = 2MB 
maintenance_work_mem = 2GB
dynamic_shared_memory_type = posix
effective_io_concurrency = 200
max_worker_processes = 30
max_parallel_workers_per_gather = 15
wal_buffers = 16MB
min_wal_size = 4GB
max_wal_size = 8GB
checkpoint_completion_target = 0.9
effective_cache_size = 12GB
default_statistics_target = 500
</code></pre>
<p>Is there an obvious mistake we are making? We are happy about every hint.</p>

<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/openstreetmap/osm2pgsql/issues/962?email_source=notifications&email_token=AA6353UKY4EXLVFJJTSNMT3QPCB25A5CNFSM4JB3RZC2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HSQCUGA">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AA6353SXOS3XTKLCXP2GHSTQPCB25ANCNFSM4JB3RZCQ">unsubscribe</a>.<img src="https://github.com/notifications/beacon/AA6353VW6DQPCXZD2EXCWV3QPCB25A5CNFSM4JB3RZC2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HSQCUGA.gif" height="1" width="1" alt="" /></p>
<script type="application/ld+json">[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/openstreetmap/osm2pgsql/issues/962?email_source=notifications\u0026email_token=AA6353UKY4EXLVFJJTSNMT3QPCB25A5CNFSM4JB3RZC2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HSQCUGA",
"url": "https://github.com/openstreetmap/osm2pgsql/issues/962?email_source=notifications\u0026email_token=AA6353UKY4EXLVFJJTSNMT3QPCB25A5CNFSM4JB3RZC2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HSQCUGA",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]</script>