[OSM-dev] New Replication Functionality
Brett Henderson
brett at bretth.com
Mon May 26 14:13:37 BST 2008
Hi All,
I've just released osmosis 0.27 which adds some additional support for
downloading and applying changesets from the planet server.
As you may or not be aware there are currently three changesets
available on the planet server. These are the daily, hourly and minute
changesets.
I have added two new tasks to osmosis for consuming these files. These are:
--read-change-interval - This provides the ability to download a set of
changes, merge them into a single changeset and pass the data to the
next task in the pipeline. It maintains a working directory where it
stores its configuration file and tracks the latest downloaded timestamp.
--read-change-interval-init - This is necessary to initialise the
working directory used by the --read-change-interval task.
I'll illustrate using an (untested) example. This example will keep an
xml file containing a small bounding box up-to-date.
1. Create the working directory with a timestamp initialised to the
start of today.
$ osmosis --read-change-interval-init "myworkingdirectory"
"2008-05-26_00:00:00"
2. Edit the config file.
The configuration file will contain suitable defaults for downloading
the hourly changes and hopefully won't require any modifications.
3. Extract an extra of the world that you're interested in using a
planet that is newer than the start of today (the original source must
contain later data than you initialise the initial timestamp to).
osmosis --rx planet.osm.bz2 --bb left=0 top=10 right=10 bottom=0 --wx
myextract.osm
4. Apply the latest changes to the extract, produce an updated extract
and rename new file the old name.
osmosis --read-change-interval "myworkingdirectory" --rx myextract.osm
--ac --bb left=0 top=10 right=10 bottom=0 --wx myextract-updated.osm
mv myextract-updated.osm myextract.osm (Note that the error code
returned by osmosis should be checked in the previous step)
Some limitations exist:
Currently only the hourly and minute changesets are supported due to the
bzip2 encoding used by the daily changesets. I should fix this soon but
it will take a little bit of rework in the task design, it's currently
hard coding a gzip encoding.
There is no limits to the number of files downloaded, I should add a
configuration option to limit the number of files (and subsequent
processing threads) processed in a single invocation.
I hope people find it useful. Any feedback appreciated.
Cheers,
Brett
More information about the dev
mailing list