[OSM-dev] Merging Changes into Planet

[marqqs at gmx.eu]

Hi Andy,

there is another program which I would expect to be significantly faster. However, it has two disadvantages:
1. It's new, it still may have bugs.
2. It lacks in the ability to write PBF and uses .osm or the efficient but less common .o5m format instead.

Nevertheless, it could be worth a try. There is an application example in the Wiki:
http://wiki.openstreetmap.org/wiki/Daily_update_an_OSM_XML_file

Markus

-------- Original-Nachricht --------
> Datum: Thu, 14 Jul 2011 12:38:02 +0100
> Von: Andrew Ayre <andy at britishideas.com>
> An: dev <dev at openstreetmap.org>
> Betreff: Re: [OSM-dev] Merging Changes into Planet

> On 7/14/2011 9:26 AM, Jochen Topf wrote:
> > Its almost certainly CPU. Try
> > dd if=planet-latest.osm.pbf of=copy.osm.pbf bs=1M
> > and you'll see how fast the files can be copied. Thats the time needed
> for the
> > disk. Everything else is CPU.
> > 
> >> Does this merge time seem right/reasonable? Are there any approaches or
> >> tricks I am missing that can speed it up?
> > 
> > Try larger buffers:
> > $OSMOSIS --read-xml-change $OSCFILE --read-bin
> $BASEDIR/var/current-planet.osm.pbf --buffer bufferCapacity=12000 --apply-change --buffer
> bufferCapacity=12000 --write-pbf $NEWPLANET
> > 
> > This takes about an hour every day on a 800MHz machine.
> > 
> > Jochen
> 
> Thanks everyone for the suggestions. I am using osmosis to generate a
> single changes.osc file already. The server is in a server farm so I
> don't have control over the disks.
> 
> The dd command took 6 minutes 40 seconds so that proves Jochen right.
> 
> With my 766MB change file the original merge command that I posted takes
> 106 minutes.
> 
> By using Jochen's merge command with buffers and the same change file it
> now takes 58 minutes!
> 
> I knew about the possibility of buffers but I never tried them because
> [1] appeared quite negative regarding them. Maybe it is out of date?
> 
> Thanks, Andy
> 
> [1] http://wiki.openstreetmap.org/wiki/Osmosis/Benchmarking
> 
> -- 
> Andy
> PGP Key ID: 0xDC1B5864
> 
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev