Could an AI model update affect comparisons between annual board evaluations?

Yes. An AI model update can affect comparisons between annual board evaluations, because changes in how questions are interpreted, responses are analysed or findings are summarised can shift outputs even when board behaviour has not changed. The risk is manageable. Boards that document the update and test for consistency can compare years with confidence.
The effect depends on where the AI sits in the process. A model that only drafts summaries poses a different risk than one that scores responses or ranks themes. The questions below set out what changes, how to verify it and what a Company Secretary should require before a year-on-year comparison goes to the Chair.

Which parts of an evaluation shift when an AI model changes?

The parts most exposed are thematic coding of open-text comments, sentiment and priority weighting, summary wording and the generation of recommendations. Numerical ratings supplied directly by directors are unaffected, but anything the model interprets, classifies or condenses can move when the underlying model changes.

In practice, three areas deserve attention. First, qualitative analysis: a new model may group the same comments under different themes or assign different emphasis to them. Second, scoring logic: where AI converts responses into composite indicators, a change in weighting can alter results without any change in director opinion. Third, narrative output: the tone and framing of findings can shift, which affects how a board reads the same underlying data.

Questionnaire design is also relevant. If a platform generates or adapts questions, a model update may alter wording. Small wording changes affect how directors respond, which undermines like-for-like comparison before any analysis begins.

How can you tell if score changes are real?

Score changes are real when they persist after controlling for methodology. The most reliable test is to re-run the prior year’s raw responses through the updated model and compare the output with the original. If the results differ materially, the movement reflects the model rather than the board.

Several checks strengthen that conclusion:

  • Parallel analysis. Process both years’ data under the same model version and compare the outputs directly.
  • Triangulation. Test whether movements in AI-derived findings are corroborated by director interviews, observed board dynamics and direct ratings.
  • Stable anchor items. Retain a core set of unchanged questions so that year-on-year trends rest on identical inputs.
  • Plausibility review. Ask whether the change matches known events, such as a new Chair, director turnover or a major strategic decision.

A sudden, uniform shift across unrelated areas is a warning sign. Genuine board change is usually uneven and explicable.

What should the vendor disclose about model updates?

A vendor should disclose when a model changes, what changed, which outputs it affects and how the impact on comparability was tested. Boards should also expect version identification for every analysis and notice before an update takes effect rather than after results are delivered.

A Company Secretary can build these expectations into the procurement and renewal process:

  1. Advance notice of material model or prompt changes, with the effective date.
  2. A plain-language description of the change and the outputs it touches.
  3. The model version used for each evaluation cycle, recorded in the report.
  4. Evidence of back-testing against prior-year data.
  5. Confirmation of how board data is handled, retained and kept out of model training.

Vendors unable to provide this information limit the board’s ability to defend its conclusions to regulators, investors or a successor Chair.

Should prior-year results be restated after an update?

Restate prior-year results when the update materially changes how the same data is analysed and the board intends to draw trend conclusions. Where the effect is negligible, retain the original results and disclose the change. Never present figures from two methodologies as one continuous series without explanation.

The decision turns on materiality and purpose. If the board uses year-on-year movement to judge progress against a development plan, restating the earlier year on the new basis preserves integrity. Keep the original figures on record as well, so the board can see both versions and the reason for the difference.

Where restatement is impractical, for example because the earlier raw data is unavailable, present the years separately and state plainly that a direct comparison is not supported. An honest break in the series is more useful than a misleading trend line.

How should methodology changes be documented for the board?

Document methodology changes in a short, dated record that states what changed, why, how comparability was tested and what limits remain. The record should accompany the evaluation report and be retained in the board’s governance file so that future boards can interpret historic results correctly.

A sound methodology note covers:

  • The model, version and analytical steps used in each cycle.
  • Any changes to questions, scales, weightings or reporting formats.
  • The outcome of back-testing and any adjustments applied.
  • Known limitations on year-on-year comparison.
  • Who approved the change and when.

This documentation also supports disclosure. Boards that describe their evaluation process in annual reports are better placed when the process has a clear, auditable record. The Company Secretary is the natural custodian of that record.

Can external evaluators help keep year-on-year comparisons credible?

Yes. An external evaluator provides continuity of judgement that does not depend on any single tool. Experienced facilitators can interpret AI output against what they observe in the boardroom, flag discontinuities and anchor comparisons in consistent professional assessment rather than in model behaviour alone.

Independence matters here. An outside party has no stake in the story the numbers tell, and a track record across many boards gives context for what a given movement means. Technology scales the analysis; human judgement validates it. The strongest evaluation programmes combine both, and treat AI output as evidence to be tested rather than a verdict to be accepted.

For boards weighing the role of an external board effectiveness evaluation, consistency of method across cycles is a legitimate selection criterion alongside depth of experience.

How The Board Practice helps with board evaluation consistency

The Board Practice has refined its evaluation methodology over more than 19 years, with Dr. Victor Prozesky having led or contributed to board performance programmes on more than 120 occasions across industries and continents. Its AI-powered platform, launching on 1 August 2026, pairs that expertise with scalable technology so that analysis is transparent and comparable from one cycle to the next.

  • Evaluations tailored to the Chair’s priorities and the board’s specific dynamics.
  • Honest, forward-looking findings that identify strengths as well as development areas.
  • Clear documentation of method, so year-on-year comparisons withstand scrutiny.
  • Two- to three-year development plans, monitored with the Chair.
  • Expert human judgement alongside AI-powered analysis and recommendations.

To discuss how your board can protect the integrity of its annual comparisons, contact The Board Practice.

Related Articles