[Tile-serving] [openstreetmap/osm2pgsql] Why so slow, even after upgrading to full SSD system and doubling RAM to 32Gb? (#883)

[mdadduzio]

Hi pnorman,

thanks for replying.

I was expecting an improvement at the parse step after the HW upgrade, considering that now everything is using only RAM and SSD.
Why the ways (and relations later I'd expect) are being processed at the same speed as when RAM was just the half and HDD 7200rpm was used instead of the new SSD?

It looks upgrading the HW made no difference.

I restarted it later adding --unlogged and --drop but no difference again (perhaps they'll make it in a later stage).
As of now it's still running and processing ways at 9.45k/s, as I remember it was around the middle of the ways processing in the previous attempt when RAM was the half (16Gb) and disk was not an SSD...
That makes me expect another 6 days for relations and then? At least another 12 or more days for indexes as it was before?

I've read benchmarks with HW comparable to mine, or even less powerful, processing ways in the order of tens or hundreds k/s and same for relations  (not K of course for the latter).
They complete the parse stage in less the 12 hours and the full import in less than 2 days.

That's why I was expecting an improvement, at least 5 or 6 times if not an order of magnitude or more, after the HW upgrade, but having nothing, literally nothing at all, looks wrong to me.

What can I do to detect the bottle neck (which is obviously not the RAM or the SDD)?

Thanks for help

P.S.: the fact that it's not using the swap partition is good, I know, but anyway, in case needed, being it on the SSD too, should be quite fast, even if not as fast as if osm2pgsql didn't require swapping.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/883#issuecomment-443420887
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20181201/da446b36/attachment-0001.html>