[osmosis-dev] Wrapper scripts for streaming replication

[Brett Henderson]

Hi Paul,

On 25 May 2013 15:15, Paul Norman <penorman at mac.com> wrote:

> > From: Brett Henderson [mailto:brett at bretth.com]
> > Sent: Friday, April 19, 2013 4:12 AM
> > Subject: Re: [osmosis-dev] Wrapper scripts for streaming replication
> >
> > It'd be great to see the replication streaming getting some usage.  It
> doesn't seem to be used at the moment.  In fact it's been down for a couple
> of weeks and it's trying to catch up again now.
>
> Do you think it's stable enough to base a service on, or am I better off
> sticking with minutely diffs?
>

> The minutely diffs are used by enough services including OSMF hosted ones
> that you can be fairly confident that any stoppages will be found and fixed
> quickly, but I'm not sure the same is true for streaming diffs.
>

That's right, we have a chicken and egg problem here.  Minutely diffs are
more stable because they've been worked on for several years with a large
number of consumers.  Replication streaming is fairly new and likely to
take a while to iron out the issues, but that won't occur unless people use
it.

However I'm largely unavailable at the moment with other commitments, so I
don't have much time to help fix things.


>
> > There are a couple of example Osmosis command lines on the following
> wiki page in case you haven't already seen them.
> >
> http://wiki.openstreetmap.org/wiki/Osmosis/Replication#Client-side_Streaming
>
> The pgsnapshot example would work from the shell, but do you have any
> examples for an unattened server?
>

I don't have any scripts that I can share.  It should be sufficient to
write a wrapper script that redirects output to log files and launches the
osmosis instance.  For extra points it might be worth having it restart
Osmosis if it fails (with some delays included), however Osmosis should
theoretically handle error conditions such as network failures gracefully.


>
> Also, what's the best way to switch to streaming replication from diffs,
> and the reverse?
>
> It's a similar question to switching from hourly to minute diffs where
you'll have to figure out the sequence number to begin from, and doing that
will require figuring out an approximate date that is earlier than your
current replication point.  Note that the streaming server does provide
some assistance where it can provide you with a state file matching a
particular timestamp.  A curl example is provided in the URL you've already
read, but more details on all the URLs are available here:
http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage#--send-replication-data_.28--srd.29

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmosis-dev/attachments/20130720/d7afd824/attachment-0001.html>