[OSM-dev] osm2pgsql slow on update import

[Markus Wagner]

Hello again...

Quoting Andy Allan <gravitystorm at gmail.com>:
>> How can I investigate what the bottleneck is? So far, I can see, that there
>> is a lot of I/O going on and not much CPU usage. The diff import is stuck
>> upon prompting "Going over pending relations","processing relation (...)".

> In my experience it's all about the disks. What disks do you have,
> speed of them etc? I found that 7.2k SATA disks were just about not
> quite fast enough, whereas 10k SATA, SAS or SSDs seem to be fine.

Ok, so I finally set up a (software) RAID-0 with three disks. Nothing  
too fancy, but still, I am not completely satisifed. I now can barely  
keep up with the update rate, but Postgres doesn't seem to utilize the  
RAID completely (CPU load is next to 0% and RAM usage is also not  
exhausted). Looking at iostat and device Utilization, all drives are  
at about 30-35% (compared to 100% with the single drive setup. But  
when writing a file to the raid with dd I get 100%.

Do I have to tell Postgres, that there are more discs to do  
optimizations or do I have to reimport, to rearrange the data in a  
smarter way?

Any pointers are highly appreciated :-)

Markus