AI GxP validation for regulated life sciences
AI Validation

Building the Test Set: How to Validate an AI With Your Own Data

Your test set decides whether an AI validation means anything. How to choose real, messy examples, agree the right answers, keep test data apart from training data, and avoid the shortcuts that make results look better than they are.

Cybroscape Technologies5 min read

Key takeaway

Your test set decides whether an AI validation means anything. How to choose real, messy examples, agree the right answers, keep test data apart from training data, and avoid the shortcuts that make results look better than they are.

You can have a perfect intended use statement and sensible acceptance criteria, and still end up with a validation that means nothing — because the test set was too easy. It happens more than anyone likes to admit. The vendor's demo documents are clean. Yours aren't.

The test set is where an AI validation earns its credibility. Here's how we put one together.

Use real material, including the ugly bits

Pull examples from your actual work. Not the best ones — a fair spread. That means including the scanned document that's slightly crooked, the requirements list written by someone who left three years ago, the template somebody modified without telling anyone.

A good rule: if a case gives your own team trouble, it belongs in the test set. The AI will meet those documents in real use, so it had better meet them in testing first.

  • Normal cases, in roughly the mix you see day to day.
  • Hard cases — messy, long, unusual.
  • Edge cases right at the boundary of the intended use.
  • A few out-of-scope cases, to check it says "not sure" instead of guessing.

Agree the right answer first

For every test case you need to know what a correct output looks like — before you see what the AI produces. This is ground truth, and it's the most time-consuming part. It's also the part you can't skip.

Have two qualified people prepare it independently where you can, then compare. Where they disagree, that's valuable: it tells you the task has some genuine judgement in it, and your acceptance criteria should allow for that. If experts can't agree, you can't fairly fail the AI for disagreeing with one of them.

Keep test data away from training data

If a document was used to train or tune the model, it can't be in your test set. The model has effectively seen the answers, so the result will look better than it really is. This sounds obvious. It's surprisingly easy to get wrong when the same folder of examples gets reused for everything.

Record where every test case came from, and lock the test set once it's approved. If you use a vendor's model, ask them to confirm your data isn't used for training — covered in data privacy in GxP AI — otherwise your test set may not stay independent.

Shortcuts that make results look better than they are

  • Testing only what the vendor supplied. Curated examples predict curated performance.
  • Too few cases. Ten documents can't tell you much about a rare but serious mistake. Size the set around the error you most need to catch.
  • Marking by impression. "Looks good" isn't a result. Compare each output against the agreed answer, item by item.
  • Quietly dropping failures. A case that fails is data. Removing it after the fact is exactly the kind of thing an inspector looks for.
  • Running it once. Generative models can answer differently each time. Run key cases more than once and record whether the output stays consistent. See validating generative AI.

Keep the test set after go-live. You'll need it again the moment the model changes — which is the next step, change control and revalidation. The wider framework is in GxP AI.

Where to go next

Explore Cybroscape GxP Copilot for AI-native validation, Cybroscape TraceDraft for source-traceable clinical documentation, or book a demo to see either on your own data.

Part of our guide to GxP AI — the regulatory framework, validation expectations and platform landscape. For the detail on this specifically, see GxP AI validation.

1Bounded intended useWhat the model does, and what it must not be used for
2A frozen evaluation setHeld-out data, version-controlled, agreed before testing
3A release gate that holdsAcceptance criteria set in advance, results recorded
4A person decides what mattersConsequential calls are made and signed by a named person
5Drift monitoring in productionInputs, outputs and overrides watched, with authority to suspend
6A fallback that actually runsThe process continues with the AI switched off
Annex 22 · consultation draft
The six parts of validating an AI system for GxP use. They map onto what the EU’s draft Annex 22 asks for. Annex 22 is still a consultation draft, not yet in force.
See it on your own systemBring a URS or a system. We run GxP Copilot against it live on the call.
ai validation test setai test data gxpground truth dataset validationindependent test data aihow to test an ai model pharma
FAQ

Frequently Asked Questions

Prefer to talk first? Chat with us on WhatsApp or book a call with an expert.

What should an AI validation test set contain?

Real material from your own work in a fair mix: normal cases in roughly day-to-day proportions, hard and messy cases, edge cases at the boundary of the intended use, and a few out-of-scope cases to confirm the AI flags uncertainty rather than guessing.

What is ground truth in AI validation?

The agreed correct output for each test case, prepared before seeing what the AI produces. Ideally two qualified people prepare it independently and compare; where they disagree, the task involves genuine judgement and the acceptance criteria should allow for it.

Why must test data be separate from training data?

If the model was trained or tuned on a document, it has effectively seen the answer, so results look better than real performance. Record the source of every test case, lock the approved test set, and confirm with any vendor that your data is not used for training.

How many test cases do you need to validate an AI?

Enough to detect the error you most need to catch. Ten documents cannot tell you much about a rare but serious failure. Size the set around your most important error type rather than a round number.

Should AI tests be run more than once?

For generative models, yes. They can produce different output for the same input, so run key cases several times and record whether the output stays consistent. Keep the test set after go-live too — you will rerun it whenever the model changes.

Keep reading

All articles
AI ValidationHow Good Is Good Enough? Acceptance Criteria for AI in GxPSetting acceptance criteria for an AI system without picking a number out of the air: linking accuracy to the cost of an error, comparing against today's manual process, and counting the right kind of mistake. · 5 min readAI ValidationHow to Write an Intended Use Statement for an AI SystemEvery part of validating an AI depends on one short document. What to put in an AI intended use statement, what to leave out, how it sets your GAMP category and risk, and the vague wording that causes trouble later. · 6 min readAI ValidationWhen the AI Model Changes: Change Control and RevalidationA validated AI stays validated only while it stays the same. Which changes need revalidation, which do not, how to size the retest, and what to do when your vendor updates the model without asking you. · 5 min readAI ValidationWhy AI Should Draft Your Tests, Not Run ThemUsing AI for test execution in GxP is where teams get into trouble. A simpler pattern: AI drafts the steps, a person approves them, and a fixed non-AI engine runs exactly what was approved, so the evidence holds up. · 6 min readAI ValidationThe risk-based control strategy Annex 22 will ask forIf the scope goes technology-neutral, you demonstrate controls rather than argue about model type. What that control strategy has to contain, under ICH Q9. · 8 min readAI ValidationFDA's AI credibility framework: seven steps and one idea worth borrowingFDA’s 2025 draft guidance asks for a seven-step, risk-based credibility assessment of AI that supports regulatory decisions, with risk set by model influence and decision consequence. · 3 min readAI ValidationHow to Validate an AI System for GxP Use, Step by StepA practical guide to validating AI for GxP use: what regulators expect from the FDA, EU Annex 22 and GAMP, how AI validation differs from classic CSV, an eight-step method, the deliverables, and the mistakes that fail inspections. · 9 min readAI ValidationGxP Validation AI: Where AI Fits Across the Validation LifecycleA stage-by-stage map of AI in GxP validation (planning, requirements, risk, test design, execution, traceability and periodic review) showing what AI drafts, what humans decide, and what regulators expect at each gate. · 12 min read

Bring a system. We'll show you the package.

Book a demo, or ask a validation lead a quick question on WhatsApp.

  • We reply within one business day
  • We run your URS live on the call
  • You get a clear plan for your system
Chat on WhatsApp

Book a demo

Four fields. The right specialist replies within one business day.

Your details stay private. We never share them.