AI GxP validation for regulated life sciences
Test Automation

What an Automated Execution Has to Capture to Count as Evidence

A green tick is not evidence. The specific record an automated run needs to leave behind so that someone reading it in three years can tell what was tested, against what, and by whose authority.

Cybroscape Technologies4 min read

Key takeaway

A green tick is not evidence. The specific record an automated run needs to leave behind so that someone reading it in three years can tell what was tested, against what, and by whose authority.

A green tick is not evidence. It is a claim that evidence exists somewhere. The gap between those two things is where automated validation gets into trouble.

The standard to aim for is not "the tool recorded a result". It is: could a competent person, three years from now, read this record and tell what was tested, against what, and on whose authority — without asking anyone who was there.

The record every step should leave

Per step, not per run. This is the distinction that matters, because a run-level summary cannot tell you where something went wrong.

  • The step as approved — its text, from the approved protocol version, not a paraphrase generated at run time.
  • Expected and actual, both recorded, both verbatim. A step that records only "pass" has thrown away the thing that made it a pass.
  • A contemporaneous timestamp, at the step. ALCOA+ asks for contemporaneous; a single timestamp for a forty-minute run is not.
  • Captured evidence where the step type implies it — a screenshot for a UI assertion, the log region for a service check.

That is the per-step floor. Everything below is run-level context that makes the steps interpretable.

The run-level context

  • Which protocol, at which version, and the signature that approved it. A result against "the OQ protocol" is ambiguous the moment the protocol is revised.
  • Which system, at which version, in which environment. Test evidence from an environment nobody can identify is not evidence.
  • Who initiated the run, and under what authority.
  • Start and end, and the overall outcome.
  • The tool and its version — relevant because the tool is itself qualified, and a result is only as good as the qualified version that produced it.

Immutability, and what it really requires

Under FDA 21 CFR Part 11 compliance, the record has to be protected from alteration, and "we do not allow editing in the UI" is not the same thing as the record being protected.

Three properties worth checking in any tool, including one you built:

  • Results are written once. A completed run cannot be re-opened and amended. A correction is a new record that references the old one.
  • Evidence is bound to the step. Screenshots and logs are stored so that they cannot be swapped for others without it being visible.
  • The audit trail covers the record itself, not only the system under test — who viewed, exported, or signed a result, and when.

A quick way to test your own system: ask how you would correct a result entered against the wrong protocol version. If the answer involves editing the record, the record is not protected.

The readability test

All of the above can be satisfied by a system that produces something nobody can read. Evidence is for a human being, usually one under time pressure who was not involved.

So the last check is editorial rather than technical: print one execution record and give it to a colleague who was not on the project. If they can follow what was tested and what happened without a guided tour, the format works. If they need you to explain the columns, it does not — and an inspector will not ask you to explain them, they will simply write down that the records were difficult to follow.

Where to go next

Explore Cybroscape GxP Copilot for AI-native validation, Cybroscape TraceDraft for source-traceable clinical documentation, or book a demo to see either on your own data.

High riskFailure could affect patient safety or product quality
Scripted testingWritten steps, expected results, an independent reviewer
Step-by-step results, screenshots and signatures
Medium riskFailure is noticed and contained
Unscripted testingExploratory or ad-hoc testing against the intended use
A record of what was tested, by whom, and the result
Low riskStandard function, supplier-tested
Supplier evidenceThe vendor’s own testing, assessed and accepted
The assessment and the reason it is enough
Computer Software Assurance in one picture: test effort follows the risk of each requirement. Based on FDA’s Computer Software Assurance guidance.
See it on your own systemBring a URS or a system. We run GxP Copilot against it live on the call.

Sources

Primary documents referred to on this page, from their official publishers.

automated test evidencevalidation evidence requirementsalcoa automated testingtest execution recordspart 11 automated execution

Keep reading

All articles
Test AutomationYour Test Automation Tool Is Also a GxP SystemIf a tool produces the evidence that another system is validated, the tool is in scope. The recursive problem nobody wants, and the proportionate way through it. · 4 min readTest AutomationWhen an Automated Test Fails: From Red Result to DeviationA failing automated test is the moment automation either proves itself or embarrasses you. What the run has to capture, when a failure becomes a deviation, and why a re-run that passes is not an explanation. · 4 min readTest AutomationOff by Default: Controlling the Blast Radius of Validation AutomationAutomation that can reach anything is automation you have to validate against everything. Shipping it switched off, and behind an explicit flag, is a scope decision more than a security one. · 4 min readTest AutomationWhy a GxP Test Engine Should Not Be Able to Run Arbitrary CommandsThe fastest way to automate a protocol is to let the engine run a shell command. It is also the fastest way to lose the argument with an inspector. Why a closed vocabulary of test steps is the safer design, and what you give up for it. · 5 min readTest AutomationAutomating IQ/OQ/PQ Protocols: What Works, What Fails, and What Inspectors ExpectWhere test automation adds genuine value in IQ/OQ/PQ execution, where it introduces new compliance risk, and how to build an audit-ready automated protocol record. · 5 min readTest AutomationGxP Test Automation: The Complete Technical Guide Under CSA in 2026How to build a GxP test automation strategy under Computer Software Assurance: what to automate, what not to, how to document it, and how to keep it defensible under inspection. · 6 min readTest AutomationGxP Test Automation and 21 CFR Part 11: The Audit Trail Requirements That Most Teams MissThe Part 11 requirements that apply specifically to automated test execution (audit trail, e-signature, result attribution, and data integrity) with a defensible record structure. · 5 min readTest Automation7 Best GxP Validation Test Automation Tools (2026): CSA, UI, API & Equipment AdaptersA technical comparison of GxP test automation tools, including Cybroscape GxP Copilot (12 closed-vocabulary adapters across Web, API, SQL, OPC UA & data loggers), Tricentis Tosca, Vera, Playwright and Selenium, scored on 21 CFR Part 11 evidence, maintenance cost, and FDA CSA alignment. · 6 min read

Bring a system. We'll show you the package.

Book a demo, or ask a validation lead a quick question on WhatsApp.

  • We reply within one business day
  • We run your URS live on the call
  • You get a clear plan for your system
Chat on WhatsApp

Book a demo

Four fields. The right specialist replies within one business day.

Your details stay private. We never share them.