[OSM-dev] Merging Changes into Planet

[Jochen Topf]

On Thu, Jul 14, 2011 at 09:04:03AM +0100, Andrew Ayre wrote:
> If I download the latest set of hourly changes and merge them into
> planet.osm.pbf using osmosis, this always takes about two hours to complete:
> 
> osmosis --read-xml-change changes.osc --read-bin planet-latest.osm.pbf
> --apply-change --write-bin planet-new.osm.pbf
> 
> This is on an Ubuntu-based quad core server with 8GB of RAM so I think
> the limitation is disk speed.

Its almost certainly CPU. Try
dd if=planet-latest.osm.pbf of=copy.osm.pbf bs=1M
and you'll see how fast the files can be copied. Thats the time needed for the
disk. Everything else is CPU.

> Does this merge time seem right/reasonable? Are there any approaches or
> tricks I am missing that can speed it up?

Try larger buffers:
$OSMOSIS --read-xml-change $OSCFILE --read-bin $BASEDIR/var/current-planet.osm.pbf --buffer bufferCapacity=12000 --apply-change --buffer bufferCapacity=12000 --write-pbf $NEWPLANET

This takes about an hour every day on a 800MHz machine.

Jochen
-- 
Jochen Topf  jochen at remote.org  http://www.remote.org/jochen/  +49-721-388298