Validation software now commonly includes AI: it drafts requirements, suggests risk ratings, writes test cases and summarises results. That is useful. It also raises a question many buyers skip: if the tool you use to validate other systems uses AI, what do you need to validate about the tool?
The tool is a GxP system
Validation software produces GxP records: requirements, risk assessments, test evidence, approvals. Those records support decisions about whether other systems are fit for use. So the software is in scope for validation itself, usually as configurable off-the-shelf software with the vendor doing much of the work.
Adding AI does not change that. It adds one more set of functions whose behaviour you need to understand and control.
Separate what the AI does from what it decides
The most important question is not how good the AI is. It is what happens to its output. If the AI drafts a test case that a qualified person reviews, edits and approves, the AI is a drafting aid, and the control is the review. If the AI decides a risk rating that nobody checks, or marks a test as passed, the AI is making a GxP decision, and the bar is much higher.
A well-designed tool keeps the AI on the drafting side of that line and makes the human decision visible and recorded. Our view, set out in AI drafts, a human approves, a machine runs, is that the AI should never be the one executing or passing tests. We explain why in AI should not run validation tests.
What to ask the vendor
- Which functions use AI, and for each, is the output a draft for review or a decision?
- Can a reviewer see what the AI produced versus what a person changed?
- How was the AI evaluated before release, and can you see the evidence?
- How will you be told when the AI model changes, and what testing happens first?
- Does any of your data leave your environment or get used to train models for others?
- Can the AI features be switched off, and does the tool still work without them?
What to test yourself
Not the AI's general quality; that is the vendor's job and you assess their evidence. Test the controls around it in your configuration: that AI output cannot become an approved record without human review, that the review is recorded with a real signature, that the audit trail shows what the AI drafted and what changed, and that switching the AI off leaves a working tool.
Then monitor in use. If reviewers accept nearly every AI draft without changes, that is either a very good model or a review that has stopped happening. Sample and check which.
Where we stand
We build GxP Copilot on exactly this line: AI drafts, a qualified person reviews and approves, and execution is deterministic. Our human-in-the-loop reviews page shows how the review is recorded. If you are assessing tools, our guide on choosing a validation platform goes further.
Where to go next
Explore GxP Copilot for AI-native validation, TraceDraft for source-traceable clinical documentation, or book a demo to see either on your own data.