AI GxP validation for regulated life sciences
AI Validation

When the AI Model Changes: Change Control and Revalidation

A validated AI stays validated only while it stays the same. Which changes need revalidation, which do not, how to size the retest, and what to do when your vendor updates the model without asking you.

Cybroscape Technologies5 min read

Key takeaway

A validated AI stays validated only while it stays the same. Which changes need revalidation, which do not, how to size the retest, and what to do when your vendor updates the model without asking you.

Here's the uncomfortable thing about validating AI. The validation is only true for the exact model you tested. Change the model, and strictly speaking you're looking at a different system. Traditional software changes when someone ships a release. AI can change when a vendor swaps the model underneath, when a prompt or template is edited, or when the kind of documents going in slowly shifts.

So change control for AI needs a few extra triggers. Here's how we think about it.

What counts as a change

Add these to your change control procedure by name. If they aren't written down, they get treated as routine maintenance.

  • A new model version — from your vendor or your own team.
  • A change to prompts, instructions or templates that shape the output. Small wording changes can shift behaviour more than you'd expect.
  • New types of input — a new document template, a new site, a new language.
  • A wider intended use — using it for something it wasn't validated for.
  • Monitoring going out of bounds — performance drifting below your limits. See continuous monitoring.

How much to retest

Not every change means starting over. This is where keeping your test set pays off — you already have approved cases and agreed answers, so retesting is a rerun, not a rebuild.

  • New model version: rerun the full test set against the same acceptance criteria. Compare results side by side with the last validated run. Look at which cases changed, not just the overall score.
  • Prompt or template change: rerun the cases the change could affect, plus a sample of the rest.
  • New input type: add new test cases for it. The old set can't tell you about inputs it never contained.
  • Wider intended use: this is really a new validation for the new use. Update the intended use statement first.

Watch the side-by-side closely. A new model can score the same overall while getting different cases wrong — maybe the ones that matter more. The average hides that.

When the vendor changes the model without asking

This is the one that worries quality teams most, and fairly so. Some AI services update the model behind the scenes. If that happens, your validated state has quietly expired and you didn't know.

Three things help. First, the contract: ask for version pinning, advance notice of changes, and a window to stay on the old version while you retest — covered in qualifying an AI vendor. Second, record the model version with every output, so you can always tell which version produced what. Third, monitoring: a small regular check against a few known cases will catch a silent change even if nobody tells you.

And if a vendor can't tell you which version you're running, that's your answer about whether you can use it for GxP work.

Keep the paper trail simple

For each change: what changed, the impact assessment, what you retested, the results next to the previous ones, and who approved it. That's it. Link it to the model version so an inspector tracing an old output can see exactly which validated version made it.

The last piece of the series is about keeping AI where it belongs in testing: why AI should draft your tests, not run them. For the full framework, see GxP AI.

Where to go next

Explore Cybroscape GxP Copilot for AI-native validation, Cybroscape TraceDraft for source-traceable clinical documentation, or book a demo to see either on your own data.

Part of our guide to GxP AI — the regulatory framework, validation expectations and platform landscape. For the detail on this specifically, see GxP AI validation.

Validated state
Periodic reviewA set cadence confirms the system still does what it was validated to do
Change controlEvery change is assessed for its impact before it is made
Supplier reassessmentVendor releases and evidence are checked when they change
Deviations and incidentsProblems are investigated and fed back into the record
Validation does not end at go-live. Each of these can move a system out of its validated state, and each brings it back with evidence.
See it on your own systemBring a URS or a system. We run GxP Copilot against it live on the call.
ai model revalidationai change control gxprevalidate ai after updateai model update validationai version control pharma
FAQ

Frequently Asked Questions

Prefer to talk first? Chat with us on WhatsApp or book a call with an expert.

When does an AI system need revalidation?

When something changes that could alter its behaviour: a new model version, edits to prompts or templates that shape output, new input types such as a new template or language, a wider intended use, or monitoring results drifting outside limits. Name these triggers explicitly in your change control procedure.

Does every AI change require full revalidation?

No. A new model version usually means rerunning the full test set against the same criteria. A prompt change can mean rerunning affected cases plus a sample. A new input type needs new test cases. A wider intended use is effectively a new validation. Keeping the approved test set makes most of this a rerun rather than a rebuild.

Why compare results case by case after a model update?

Because a new model can score the same overall while getting different cases wrong — possibly the ones that matter more. The average hides that, so compare each case against the last validated run.

What if an AI vendor updates the model without notice?

Your validated state may have expired silently. Protect against it with contract terms for version pinning and advance notice, by recording the model version with every output, and with a small routine check against known cases that catches silent changes. If a vendor cannot say which version you are running, it is not suitable for GxP work.

What records does AI change control need?

What changed, the impact assessment, what was retested, results beside the previous validated run, and the approval — all linked to the model version so any past output can be traced to the validated version that produced it.

Keep reading

All articles
AI ValidationBuilding the Test Set: How to Validate an AI With Your Own DataYour test set decides whether an AI validation means anything. How to choose real, messy examples, agree the right answers, keep test data apart from training data, and avoid the shortcuts that make results look better than they are. · 5 min readAI ValidationHuman-in-the-loop is not a footnote: designing HITL that actually worksQuorum, segregation of duties, escalation, override auditing: HITL patterns that hold up under inspection. · 6 min readAI ValidationValidating an AI assistant that answers from your SOPsAn SOP assistant becomes a GxP system once people act on its answers. Require cited sources, effective document versions only and honest ’not covered’ answers, and test with real questions before go-live. · 3 min readAI ValidationHow Good Is Good Enough? Acceptance Criteria for AI in GxPSetting acceptance criteria for an AI system without picking a number out of the air: linking accuracy to the cost of an error, comparing against today's manual process, and counting the right kind of mistake. · 5 min readAI ValidationYour validation software uses AI. What do you need to validate?If your validation software uses AI, the key question is whether its output is a draft a person reviews or a decision nobody checks. Test that review is enforced, recorded and visible in the audit trail. · 3 min readAI ValidationHow to Write an Intended Use Statement for an AI SystemEvery part of validating an AI depends on one short document. What to put in an AI intended use statement, what to leave out, how it sets your GAMP category and risk, and the vague wording that causes trouble later. · 6 min readAI ValidationWhy AI Should Draft Your Tests, Not Run ThemUsing AI for test execution in GxP is where teams get into trouble. A simpler pattern: AI drafts the steps, a person approves them, and a fixed non-AI engine runs exactly what was approved, so the evidence holds up. · 6 min readAI ValidationThe risk-based control strategy Annex 22 will ask forIf the scope goes technology-neutral, you demonstrate controls rather than argue about model type. What that control strategy has to contain, under ICH Q9. · 8 min read

Bring a system. We'll show you the package.

Book a demo, or ask a validation lead a quick question on WhatsApp.

  • We reply within one business day
  • We run your URS live on the call
  • You get a clear plan for your system
Chat on WhatsApp

Book a demo

Four fields. The right specialist replies within one business day.

Your details stay private. We never share them.