[OSM-dev] Missing changes in minute-replicate

[Brett Henderson]

On Wed, Nov 11, 2009 at 8:59 AM, Brett Henderson <brett at bretth.com> wrote:

> Tom Hughes wrote:
>
>> On 10/11/09 00:09, Matt Amos wrote:
>>
>>  possibly it's a coincidence, but changeset 3073502 was committed in
>>> transaction 174978321, which is the txnMax of replication diff 84538.
>>> i don't see anything special about that number, though.
>>>
>>
>> Ah... I investigated an issue like this a couple of weeks back. I suspect
>> there is an off by one error somewhere that means the last transaction in
>> each diff isn't getting processed by osmosis when it builds the replicate
>> diffs.
>>
> That's highly likely :-)  The id logic is a bit funky so I might have made
> some mistakes in there.  I'll do my best to take a look at it.  Probably
> won't get to it for a few days though ...
>

I've taken a look and believe I've fixed the issue (in svn only for now).
The code was retrieving a transaction snapshot from the server which among
other things includes the xMax value.  This value is the next *unassigned*
transaction identifier.  Osmosis was then building a query which attempted
to retrieve data with that transaction identifier included even though it
hadn't been committed yet.  This means that the transaction identifier would
some time very soon after get committed to the db but Osmosis would never
query for that id again.  Not sure what I was thinking when I did that but I
guess I just didn't read up on the meaning of xMax properly.

I think I've fixed the code and I've checked in my changes.  I'll deploy a
new version of Osmosis to Services/Horntail hopefully in the next 12 hours.
I'll probably create a minute-replicate2 initially but will update the
existing minute-replicate once it looks like it's working and kill the
minute-replicate2.

The replication is reasonably simple in theory but gets tricky in practice
due to wrapping txn ids, use of unsigned integers (which aren't directly
supported in java), and reserved transaction ids.  Hopefully it's one step
closer to working properly now.

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20091116/57c3b733/attachment.html>