An AI validation plan is the document that says, before any testing starts, what the AI is for, how good it has to be, how you will prove it and how you will keep it that way. Most of the trouble in AI validation comes from deciding those things after the results are in.
The plan below follows the order the work actually happens in. It sits inside your normal validation approach, not beside it: the system still gets the usual controls for access, audit trail, backup and change. These sections cover what is different because the system contains a model. The wider method is on our GxP AI validation page.
1. Intended use, written narrowly
Say what the AI does, for which records, in which process step, and who acts on its output. "Supports deviation triage" is too wide to test. "Suggests a severity category for new deviations in the packaging area, which a QA reviewer confirms or changes before the record is committed" can be tested and inspected.
Also say what it does not do. The exclusions are often what an inspector reads first. More on this in how to write an intended use statement for an AI system.
2. Risk and the human decision point
Rate the risk of the decision the AI influences, not the risk of the technology. Then name the person who makes the final decision, what they see when they make it and what happens if they disagree with the AI.
If nobody can be named, the AI is making the decision, and the plan has to treat it that way.
3. Acceptance criteria, set in advance
Write the numbers down before you test: the error rate you can accept, which errors matter most, and what happens to cases the model is unsure about. Criteria chosen after the results are seen are not criteria. See acceptance criteria for AI in GxP for how to set them.
4. The test set
Describe where the evaluation data comes from, who prepared the agreed answers, and why it represents real use, including the difficult and unusual cases. It must be kept apart from anything used to build or tune the model. The detail is in building the test set.
5. Testing and the report
List the tests: accuracy against the test set, behaviour on bad or missing inputs, the uncertain-case path, and the human review step working as designed. The report then compares results with the criteria from section 3, without moving them.
6. Running it: monitoring, change and fallback
The plan is not finished at go-live. It should say how performance will be checked in use, what triggers a review, how changes to the model, prompts or settings are controlled, and how the process runs if the AI is switched off. A model that drifts quietly is the most common way a validated AI stops being validated. See continuous monitoring for AI and when the AI model changes.
The draft EU GMP Annex 22 covers the same ground: intended use, test data, acceptance criteria, change control and monitoring. It is still a consultation draft, so treat it as direction rather than a rule, but a plan built this way will not need rewriting when it is finalised.
Where to go next
Explore GxP Copilot for AI-native validation, TraceDraft for source-traceable clinical documentation, or book a demo to see either on your own data.