[osmosis-dev] Migration of Osmosis Changesets

[Brett Henderson]

On Fri, Nov 13, 2009 at 9:40 PM, Lennard <ldp at xs4all.nl> wrote:

> Peter Körner wrote:
>
> > It seems the minute-replication diffs are broken -
> > http://planet.openstreetmap.org/minute-replicate/state.txt
> > shows 07:22:02 UTC for soome minutes now and this is about 1h behind.
> >
> > Can someone confirm this?
>
> Confirmed. Breet also mentioned "the hourly, minute-slow (...), and the
> hourly", and I wonder if one of those hourly ones was meant to be
> minute-replicate. Currently, hourly and minute-slow is showing updates.
>

Sorry, that was just a typo in my email.  The permissions were okay on the
directory but there were a couple of files that Osmosis couldn't write to.
I've deleted and re-created the files under the correct user and all is
fine.  The changesets are running again.

One thing you may notice is that the changesets don't catch up in a big
bang, they only extract up to 25000 transactions per invocation.  You can
tell this by comparing the txnMaxQueried value to the txnMax value in the
state.txt files.  If they are identical the process has caught up.  The
timestamp in the file is taken from the maximum entity timestamp if the
process is behind, but uses the database system timestamp if the system has
caught up.

This 25000 limit was previously due to a limitation in the way the queries
were generated but I think the limit may now be unnecessary.  I used to pass
all transaction ids in the where clause which had limits but the queries are
range based now which means the queries are much the same regardless of how
many transactions are being queried (I'm getting a bit vague, it's been a
couple of months since I last looked at it).  I'll leave the 25000 limit in
for now though because the production rate of changesets is much slower than
that and a throttle might be a good thing when recovering from outages.

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmosis-dev/attachments/20091113/7ff5a739/attachment.html>