[Rebuild] Another issue to consider: Osmosis replication
penorman at mac.com
Thu Feb 16 20:57:15 GMT 2012
> -----Original Message-----
> From: Frederik Ramm [mailto:frederik at remote.org]
> Subject: [Rebuild] Another issue to consider: Osmosis replication
> (CC'ing this to Brett Henderson, Osmosis inventor+maintainer)
> most of you will be familiar with the standard way of running one's
> own replicated OSM database - all based on Osmosis' --rri task which
> automatically loads the hourly or minutely diffs since it was last run
> and does with them whatever you ask it to.
> I've asked Grant for the figures today, and according to the access logs
> there seem to be approximately 150 replicated instances of OSM "out
> there" - the majority of them, one would assume, for rendering, but some
> do other things - for example, Matt Amos runs his "OWL" database off of
> such diffs, or MapQuest's Nominatim will likely do the same.
> Many of these databases run on auto pilot - they just have Osmosis
> polling for new stuff all the time and if our server hands out something
> then they'll process it.
> Now what happens after we change the license and boot up our servers?
> Will we produce one gigantic "diff" that encapsulates all the changes we
> had to make to get from the last CC-BY-SA version to the first ODbL
> version of our database, so that these users can simply continue to
> gobble up our diffs and they will magically be ODbL after the switch?
> Or do we tell them: This is all too complicated, you will have to kill
> your old databae and do a fresh import if you want to be on the ODbL
> Is it even possible, legally, that there is someone out there who has
> our current database (which is licensed CC-BY-SA only), and we provide
> them with a diff that deletes 5% of their objects and modifies anohter
> 5%, and after that their database has magically become ODbL?
> If this isn't possible or feasible, then some of them will face quite a
> bit of work - someone running a world-wide nominatim would have to
> schedule about two weeks of data import time after the license change
> during which they would have to serve old data. Also, if sticking to the
> current mode of operations is not an option, then maybe we need to
> rename the download directories of everything so that we can make sure
> that there's not a CC-BY-SA database somewhere updating itself on auto
> pilot with new ODbL diffs, creating all sorts of trouble for everyone
Do we have any idea how many weeks of edits such a diff would be equivalent
to? I know that for my pgsnapshot database if it falls too far out of sync
it is quicker to reimport than allow it to catch up with diffs. Not counting
download times and the fact that planetfiles are only generated weekly, if
my DB were to fall more than about 10 days behind, I'd be better off
re-importing than allowing it to catch up.
More information about the Rebuild