[Rebuild] Update on plan after feedback from weekend

Dermot McNally dermotm at gmail.com
Tue Mar 27 02:05:37 BST 2012

Dear Rebuilders,

There has been much feedback on how the rebuild plan should or should
not proceed. I indicated that the progress of Matt's weekend tests
would help inform us better to choose from the available options, in
particular where benchmarking is concerned. I have been discussing the
outcome with Matt and we have indeed come to the conclusion that a
change of approach is desirable.

To recap, the plan as documented was to take the API into read-only
mode and to use this window for both the migration of the database to
a new server and the offline rebuild process. In this way we hoped to
achieve a complete rebuild over the course of a few days.

In the event, the population of the test database alone took a number
of days. To add the additional cost of reprocessing problem objects
would cause the read-only phase to be simply too long. This
makes the live rebuild option more appealing. It can be expected to
take longer than the offline approach, but on balance the disruption
now seems certain to be less.

Note that, since the migration of the production database to the new
server will still require 3-4 days of read-only operation, this phase
will still need to be scheduled. We now propose a timeframe along
these lines:

* 31st March: Declare last CC planet, halt diff generation (to be
resumed from a different diff file location after ODbL declaration)

* 1st April: Enter Read-only mode

* 3rd-4th April: Enter Read-write mode on new server. Request community
to avoid imports or other very large scale edits to avoid conflicts
during live rebuild

* 5th-6th April: bbox-based live-data tests of rebuild logic (as per
gradual rebuild approach previously discussed)

7th April: Start automated processing of all remaining non-clean objects.

9th April: Progress report (particularly of indications of remaining
processing time)

On completion of the processing, subject to satisfaction with the
outcome we can redeclare the dataset to be ODbL. Immediately afterwards,
a new planet file will be generated/published, and diff creation will resume
from a different location to the CC diffs.

Please provide feedback on the rebuild list. I will, during Tuesday, update the
wiki-based plan to reflect these changes.


Igaühel on siin oma laul
ja ma oma ei leiagi üles

More information about the Rebuild mailing list