[Tile-serving] [osm2pgsql] Extreme slowdown while importing ways (#534)

[ekr3peeK]

I know that importing of ways supposed to be slower then that of nodes, but this slowdown seems a bit extreme to me. Analyzing some of the benchmarks (which indeed were made with better machines then the one I am using), they doesn't encounter such a signficant speed loss when importing the ways. With these speeds, and based on OSM data, to import the whole planet file, it would take about 2-3 months (considerng that relations would be even slower then ways, and we have yet to index the data).

Don't get me wrong, I am not expecting miracles here, I know that the import would take a long time, but 2-3 months is just a bit too long for me. I was expecting a 15-18 days data import, and I would be okay with that.

I don't know how to check correctly the speed of the hard drive, cause I am rather new to Linux, but I've used this command I've found on the net with the following results:

hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   11444 MB in  2.00 seconds = 5729.45 MB/sec
 Timing buffered disk reads: 264 MB in  3.01 seconds =  87.79 MB/sec

(Note, that the import process was still running while I've used the command, I don't know if that affects the performance of hdparm or not)

---
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/534#issuecomment-189215100
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20160226/a2d3667d/attachment-0001.html>