[OSM-dev] osm2pgsql slim mode, postgis, and hard disk spindles

Stephan Knauss osm at stephans-server.de
Tue Sep 8 21:42:51 BST 2009


Richard Weait wrote:
> Jaunty, osm2pgsql from svn,
> p4, 3.4GHz
> single 150GB hard drive
> 50 hours.

Wow, 50 hours to apply the changes from a day.

I would be interested in having a wiki page to compare the typical 
runtime. It would help a lot to determine what performance to expect 
from a machine and to compare the own setup, so discover 
misconfiguration better.
In case we could get some comparable results I would set up a wiki page.

Here my measurements again. I did run an incremental update:
*** Old mapnik db timestamp
2009-09-07T07:00:00Z
*** Start update
08:37
INFO: Osmosis Version 0.31.2
osm2pgsql SVN version 0.66-16423M
Mid: pgsql, scale=100, cache=3800MB, maxblocks=486401*8192
Node stats: total(3264066), max(489240074)
Way stats: total(106990), max(40421741)
Relation stats: total(2780), max(233957)
*** End update
15:52
*** New mapnik db timestamp
2009-09-08T06:00:00Z

So that are 435 minutes. i7 920, 12GB, HW-Raid with high sequential speed

I tried to tune the DB with these settings (others are default):
effective_cache_size 8000MB
random_page_cost 6.0
maintenance_work_mem 2000MB
shared_buffers 2000MB
temp_buffers 100MB
work_mem 100MB
max_fsm_pages 204800
fsync off
wal_buffers 128kB
checkpoint_segments 64
autovacuum off


With databases that huge as ours I suspect the disk subsystem being the 
limiting factor. What would be a typical workload so to set up an 
iometer (www.iometer.org) bench to compare the results?

Stephan




More information about the dev mailing list