[Tile-serving] [openstreetmap/osm2pgsql] Long import of Europe data file (#676)

[Hartmut Holzgraefe]

On 31.01.2017 23:31, Koles500 wrote:
> It looks like the process has stopped, because it is using 0% of
> processor (and ~3,3 GB RAM). Could you please say something more about
> these messages : ?
[...]
> Creating indexes on osm_line finished
> All indexes on osm_line created in 103561s
> Completed osm_line`

building indexes for osm_polygon has not finished yet,
which is a process that mostly requires random IO
which classic HDDs are especially not good at.

So while CPU use is minimal you'll probably see at least
one postgres process with high IO wait, and this process
also being near the top of iostat output all the time.

IO throughput is probably low, nowhere near what the disk
subsystem could do when doing a linear file copy for example,
that's due to lot of (slow) read/write head moving time
due to the random IO pattern caused by updating large index
tree structures on disk.

If you'd be near the actual machine you'd be able to hear
that the disks are actually quite busy.

SSDs (and large postgres workmem) really make a big difference
in this step of the import.

-- 
hartmut