AI Validation Framework for GxP (Draft Annex 22)
How to validate AI used in GxP processes, in 26 checks: intended use and model type, documentation, acceptance criteria, human oversight, explainability, monitoring and change control.
Validating AI in a GxP process means proving the system is fit for a defined intended use and keeping it that way as the model changes: a stated intended use, acceptance criteria set in advance, an independent test set that gates every release, human oversight where it is the control, and monitoring and change control in operation. The draft EU Annex 22 (not in force) sets these out for critical GMP uses.
Define the intended use narrowly
Most AI validation difficulty traces back to an intended use written broadly enough to sound impressive. A narrow statement is easier to test and easier to defend.
- 1
Intended use statement
What the model does, on what inputs, producing what output, used for what decision, by whom.
- 2
Explicit out-of-scope uses
What the model must not be used for. Without this, scope creep happens silently after go-live.
- 3
Criticality determination
Whether the output has a direct impact on patient safety, product quality or data integrity. That decides whether the draft Annex 22 applies.
- 4
Model type fits the use
As drafted, critical uses allow only static models with deterministic output. Generative AI, large language models and models that learn in use belong in non-critical work, with a qualified person responsible for the output.
- 5
Deterministic alternative considered
Whether a rules-based approach would do the job. If it would, use it: it is cheaper to validate and easier to explain.
Data and model documentation
The model card is the equivalent of a design specification. An inspector will ask what the model was trained on and what it was tested against.
- 6
Model card
Model type, version, training data provenance, known limitations, and the conditions under which performance degrades.
- 7
Training data governance
Where the data came from, what rights you have to it, how it was cleaned, and whether any regulated data is involved.
- 8
Test set held out and frozen
Never used in training or tuning, representative of real inputs including awkward ones, and version-controlled.
- 9
Bias and edge-case analysis
Where the model performs worse, assessed deliberately rather than discovered in production.
- 10
Version pinning
The exact model version in production is recorded. A silently updated vendor model is an uncontrolled change.
Acceptance criteria and the release gate
This is the part most AI projects skip, and the part that makes the difference between a validated system and a pilot that escaped.
- 11
Performance thresholds set before testing
Defined in advance and tied to the consequence of error, not chosen after seeing the results.
- 12
Error-type weighting
A false negative and a false positive rarely cost the same. State which matters more and why.
- 13
Evaluation run against the frozen set
Every candidate release evaluated against the same held-out set, with results recorded.
- 14
Release blocked on failure
Enforced by the process, not by goodwill. A gate that can be waived informally is not a gate.
Human oversight
Where a person reviewing the output is the control, the draft Annex 22 expects that review to be part of the system and recorded. The design question is where that person sits and whether they can realistically intervene.
- 15
Human-in-the-loop policy
Which outputs require human review before use, which are advisory, and which are logged only.
- 16
Reviewer sees the evidence
The reviewer gets the source material and the model's citation, not just its conclusion. Review without evidence is rubber-stamping.
- 17
Segregation of duties preserved
The person approving AI-assisted output cannot be the person who generated it, exactly as with human-authored work.
- 18
Override is possible and recorded
Reviewers can reject or amend, and the rate of override is tracked as a quality signal.
- 19
Deterministic guardrails on outputs
Risk scores, classifications and calculations computed in code, not generated. An inspector will ask how a number was produced.
- 20
Explainability and confidence
Where the decision warrants it, the features behind an output can be shown; a confidence score is recorded with each result and low-confidence results are flagged rather than passed through.
Monitoring, drift and fallback
The obligation that separates AI validation from software validation: the thing can get worse without anyone changing it.
- 21
Input drift monitoring
Detects when live inputs stop resembling the test set, usually the earliest warning available.
- 22
Output and performance monitoring
Tracks acceptance rates, override rates and, where ground truth arrives later, accuracy over time.
- 23
Re-evaluation cadence
A defined period after which the model is re-run against the test set regardless of whether anything appeared wrong.
- 24
Documented fallback
The process runs without the AI. Written down, tested, and known to the people who would have to use it. This comes from Annex 11 business continuity (clause 16) rather than Annex 22.
- 25
Rollback capability
A previous model version can be restored, and the decision to roll back has a named owner.
- 26
Change control covers model updates
Retraining, fine-tuning and vendor model upgrades go through change control like any other change to a validated system.
The draft EU Annex 22 (July 2025, not in force) is the GMP annex for AI used in critical applications. Read it alongside Annex 11 rather than instead of it, the electronic records and signature obligations still apply to AI-assisted records.
The single most useful design decision is to keep generative output away from anything that must be reproducible. Let the model draft and summarise; compute classifications, risk scores and calculations deterministically so they can be explained and repeated.
This is a working aid, not a regulatory document, and it does not replace your own quality system procedures.
Explore related resources, products and services: GxP Copilot, TraceDraft, validation services, AI transformation.
Sources
Primary documents referred to on this page, from their official publishers.
- EudraLex Volume 4: EU Good Manufacturing Practice guidelines · European Commission
- EU GMP Annex 11: Computerised Systems (current version) · European Commission
- Consultation on EU GMP Chapter 4, Annex 11 and draft Annex 22 (Artificial Intelligence) · European Commission
Common questions
Prefer to talk first? Chat with us on WhatsApp or book a call with an expert.
How do you validate an AI system for GxP use?
Define a narrow intended use, document the model and its training data, hold out a frozen test set, set acceptance thresholds before you evaluate, gate each release against that set, require human review of consequential outputs, monitor performance and inputs in production, put model changes under change control, and keep a documented fallback so the process runs with the AI disabled.
What is EU Annex 22?
Annex 22 is the EU GMP annex covering artificial intelligence in regulated manufacturing. It is still a draft (consultation closed in October 2025 and it has not been adopted) but it is the clearest signal of what regulators will expect. It applies to AI in critical GMP applications, where it allows only static models with deterministic output and keeps generative AI out. It expects a detailed intended use, acceptance criteria set before testing, independent and representative test data, explainability and confidence where relevant, and change control, monitoring and recorded human review in operation. It complements Annex 11 rather than replacing it.
Can AI make a GMP-critical decision on its own?
As drafted, only a static model with deterministic output, validated against the annex, may be used in a critical GMP application; generative and continuously learning models may not. Most companies still keep a qualified person deciding and signing, with the model drafting, summarising or proposing, and that person should see the underlying evidence rather than only the model's conclusion.
What is model drift and why does it matter for validation?
Drift is the degradation of model performance over time as live inputs diverge from the data the model was evaluated against. It matters because it breaks the usual assumption of software validation: the system can stop being fit for its intended use without anyone changing it. Monitoring and periodic re-evaluation are the controls.
Does a vendor updating their model break our validated state?
Potentially, yes. An AI component whose model changes underneath you is an uncontrolled change to a validated system. Pin the model version where the vendor allows it, require notification of changes contractually, and route any model update through change control with re-evaluation against your frozen test set.
Put this to work on your system.
Bring a URS. We show you risk-sized testing on your own requirements.
- We reply within one business day
- We run your URS live on the call
- You get a clear plan for your system