[OSM-dev] New Replication Functionality
Brett Henderson
brett at bretth.com
Tue May 27 00:47:15 BST 2008
I stuff3d up the 0.27 release. 0.28 is up now and fixes a major runtime
problem preventing it from ever launching.
Brett Henderson wrote:
> Hi All,
>
> I've just released osmosis 0.27 which adds some additional support for
> downloading and applying changesets from the planet server.
>
> As you may or not be aware there are currently three changesets
> available on the planet server. These are the daily, hourly and
> minute changesets.
>
> I have added two new tasks to osmosis for consuming these files.
> These are:
> --read-change-interval - This provides the ability to download a set
> of changes, merge them into a single changeset and pass the data to
> the next task in the pipeline. It maintains a working directory where
> it stores its configuration file and tracks the latest downloaded
> timestamp.
> --read-change-interval-init - This is necessary to initialise the
> working directory used by the --read-change-interval task.
>
> I'll illustrate using an (untested) example. This example will keep
> an xml file containing a small bounding box up-to-date.
> 1. Create the working directory with a timestamp initialised to the
> start of today.
> $ osmosis --read-change-interval-init "myworkingdirectory"
> "2008-05-26_00:00:00"
>
> 2. Edit the config file.
> The configuration file will contain suitable defaults for downloading
> the hourly changes and hopefully won't require any modifications.
>
> 3. Extract an extra of the world that you're interested in using a
> planet that is newer than the start of today (the original source must
> contain later data than you initialise the initial timestamp to).
> osmosis --rx planet.osm.bz2 --bb left=0 top=10 right=10 bottom=0 --wx
> myextract.osm
>
> 4. Apply the latest changes to the extract, produce an updated extract
> and rename new file the old name.
> osmosis --read-change-interval "myworkingdirectory" --rx myextract.osm
> --ac --bb left=0 top=10 right=10 bottom=0 --wx myextract-updated.osm
> mv myextract-updated.osm myextract.osm (Note that the error code
> returned by osmosis should be checked in the previous step)
>
> Some limitations exist:
> Currently only the hourly and minute changesets are supported due to
> the bzip2 encoding used by the daily changesets. I should fix this
> soon but it will take a little bit of rework in the task design, it's
> currently hard coding a gzip encoding.
> There is no limits to the number of files downloaded, I should add a
> configuration option to limit the number of files (and subsequent
> processing threads) processed in a single invocation.
>
> I hope people find it useful. Any feedback appreciated.
>
> Cheers,
> Brett
>
>
More information about the dev
mailing list