[Rebuild] Strategy for running the bot in "regions"

Paul Norman penorman at mac.com
Tue Jun 26 04:54:30 BST 2012


> From: Andy Allan [mailto:gravitystorm at gmail.com]
> Sent: Monday, June 25, 2012 8:28 AM
> Cc: rebuild at openstreetmap.org
> Subject: Re: [Rebuild] Strategy for running the bot in "regions"
> 
> So putting all the ideas together, and thrashing it through with Matt,
> here's the proposal for how the bot should approach handling the entire
> planet. The key points I think are that it uses a "candidate list" to
> avoid processing entities only ever touched by acceptors, and that it
> uses regions to both order the work and allows us to run multiple copies
> in parallel.

When writing the code for developing clean coastlines I found that the most
useful candidate criteria was to pass over objects which were v1 by
acceptors. This covers a huge number of objects and going beyond this
requires much more expensive full history calls. Full history calls would be
less expensive for the bot with its more direct access access to the data.
You'd also have to deal with dirty changesets by accepting users which I
ignored.

If you were to filter the candidate list farther and remove objects where
all versions were by acceptors you might find that it doesn't offer much of
a speed advantage over the full redaction logic. I guess it might depend
exactly on what is limiting, IO or CPU




More information about the Rebuild mailing list