[OSM-dev] Planet diff's revisited

Brett Henderson brett at bretth.com
Wed Jul 25 13:06:43 BST 2007


The diff tool was posted to this list by Jon Burgess on 6/04/2007 9:34 PM.

Osmosis can also generate diff files (actually object changes, not line 
changes) but I suspect Jon's diff tool will be a lot quicker if it works 
at a raw text file level rather than an OSM entity level.  Then again, 
his use of an xml delta file extension suggests otherwise.

The content of his email was:

Hi all,
	I've developed a diff/patch tool for handling planet.osm dump files.
This allows an incremental diff to be generated between 2 planet.osm
dumps. This diff file can then be applied on top of the old planet.osm
to generate the new one. 

If we published these diff files on planet.openstreetmap.org then it
could make the downloads to obtain a new planet.osm dump much smaller. I
think the typical weekly diff file is in the order of 5MB to 10MB when
bz2 compressed.


The usage summary is:

Patch creation:
$ planetdiff planet-070307.osm.bz2 planet-070321.osm.bz2 > delta.xml

Applying a patch:
$ planetpatch planet-070307.osm.bz2 delta.xml > planet-070321a.osm



The tool is in SVN under utils/planetdiff/...

See the SVN readme.txt for further details or
http://trac.openstreetmap.org/browser/utils/planetdiff/readme.txt


	Jon



spaetz wrote:
> Hi all, I was looking at the planet generation and archiving proces. Currently we archive them both as bz2 and as 7z files. Download stats tell that in 8 days the bz2 has been retrieved nearly 15000 times while the 7z was retrieved about 500 times. Should we continue to bother with 7z, given that disk space on the dev server is not unlimited?
>
> Also I would like to raise the question of planet diff's again. Would people appreciate 4-weekly full dumps and planet diff's in between? As most of the thing remains the same, we could save quite a bit of disk space with that, I guess.
> The catch is IMHO, that the files are too big to be handled with std diff tools, so we (you) would have to use one that can cope with those files (somebody posted them previously, I forgot who).
>
> What do people think?
>
> spaetz
>   
> ------------------------------------------------------------------------





More information about the dev mailing list