[OSM-dev] osm2pgsql update

[Kai Krueger]

Frederik Ramm wrote
> 
> I think it would be great to share results of osm2pgsql runs among users 
> - how long does it take to import X on infrastructure Y?
> 
> I've made a start here, please add/modify as you see fit:
> 
> http://wiki.openstreetmap.org/wiki/Osm2pgsql/Benchmarks
> 
Great, that could be a useful resource for anyone wanting to run their own
tileserver and see what kind of specs they need.
Perhaps one could in addition also devise a standardized tile rendering
benchmark to compare the setups on that level as well. Such a benchmarking
tool might also be useful to identify performance issues when creating style
sheets.


Frederik Ramm wrote
> 
> I introduced a "--drop" flag that makes osm2pgsql drop the temporary 
> tables after import, and also does not create the indexes on way id and 
> relation id that a --slim import normally created. 

Although not creating some of the indexes can have a huge impact (In some of
my benchmarks creating the planet_ways_nodes index took more than half of
the overall time for import) and thus the --drop option can be very helpful,
do the indexes on way id and relation id count to those important indexes?

I experimented with not creating the primary key indexes before import on
the osm_ids of the slim mode tables and defer them to the index creation
phase. While deferring the primary index on the planet_osm_nodes had a
significant effect, pushing import speed from ~150k nodes/s to ~300k nodes/s
(although about half of the benefit was lost again by having the extra time
at the end creating the index), doing the same on the way or relations
tables had basically no effect. This is presumably because while the node
import phase is bottlenecked on the CPU usage of postgresql, the ways and
relations are either bottlenecked on the CPU of osm2pgsql or on read
performance of retrieving nodes in postgresql. Do you see benefits from
dropping the indexes on the output tables? I guess it does save you some
space on disk which can be useful. 

Kai


--
View this message in context: http://gis.638310.n2.nabble.com/osm2pgsql-update-tp7027563p7030055.html
Sent from the Developer Discussion mailing list archive at Nabble.com.