[OSM-dev] Minute Diffs Broken
Tom Hughes
tom at compton.nu
Tue May 5 10:42:42 BST 2009
Brett Henderson wrote:
> If you're referring to multi-mastered clustered databases then that is a
> whole different problem that shouldn't be confused with what I'm trying
> to achieve. I'm simply trying to provide a way for people to access
> regular updates in a read-only fashion where data integrity is the
> highest priority. By allowing delays in delivery (I'd like to get it
> down to a couple of minutes but I'm not aiming for anything like
> real-time) it becomes a simpler problem with hopefully a simpler
> solution. Any multiple database system is likely to be a long way off
> so I can't wait for that.
I made my comments because people seemed to be suggesting using one or
other of the clustering solutions to provide some sort of change stream
to you.
> 1. Are there any known issues with the current API that could cause
> delays in excess of 5 minutes? Or is it just a fact of life with large
> changesets? I guess what I'm asking is longer than 5 minutes a regular
> occurrence with a large changeset or is something strange going on in
> rare cases?
I have to say I don't fully understand what the issue you're seeing is
as I only skimmed the vast amount of discussion that went on overnight.
Are you really seeing a five minute delay? Or just a delay that happens
to be over a five minute boundary?
In other words are you saying that an edit timestamped at 17:50:00 is
still not visible in the database at 17:55:00? Or just that you look at
the database at 17:55:00 and don't see an edit which is timestamped at
17:54:50 which then appears when you look again at 18:00:00?
It's certainly possible that a very large changeset might take some
minutes to process I guess - if you can give me a specific changeset
number from the last few days then I can have a look and see how long it
took.
> 2. What appear to be the current system bottlenecks? Is the database
> already approaching processing capacity or is rails the limiting factor?
Nothing specific in terms of the database - that largely seems to be
fine. The issues are, as always, mostly about rails and the amount of
memory it uses and getting the daemons to restart cleanly when they get
too big.
Tom
--
Tom Hughes (tom at compton.nu)
http://www.compton.nu/
More information about the dev
mailing list