Medical Devices · AI Validation

AI/ML validation for a SaMD manufacturer — Annex 22 posture stood up ahead of MDR submission

A Class II SaMD manufacturer had an ML-based clinical decision-support module ready to ship, but no defensible AI assurance stack. We stood up an EU Annex 22-aligned assurance package — governance, benchmark set, safeguards, human-in-the-loop policy, drift monitor and fallback — in time for their MDR submission.

Quick chat on WhatsAppHuman review at every gatePart 11 readyISO 9001 & 27001 certified

ClientA medical device manufacturer (Class II SaMD)

Size~250 employees

GeographyEurope + North America

StageSeries C

Results

What changed

GxP Copilot with Cybroscape delivery leads.

See GxP Copilot

6/6Assurance components signed

13 wkTo submission-ready

First passAI dossier accepted without rework

100%Fallback workflow test coverage

Pain points before the engagement

The model performed well in evaluation, but there was no published governance around what it was for and what it wasn't for
No frozen benchmark existed to gate the next model release
Human-in-the-loop was 'clinician judgement' as a policy phrase, not enforced anywhere in the product
Drift monitoring existed on the engineering side but had no clinical escalation path
There was no working fallback for the clinical workflow when the model was unavailable

Problem

The company had validated the medical device software under IEC 62304, but the AI/ML functionality was a different regulatory conversation. EU Annex 22 (AI in GMP-adjacent contexts) and the FDA's evolving AI/ML SaMD guidance both wanted answers the product could not currently give. The MDR submission window was three months out, and the AI assurance stack had to be more than a set of engineering artefacts.

Approach

We built the six-part assurance stack in parallel with the submission drafting. Published governance stated exactly what the model was intended for and what clinical decisions were out of scope. A frozen benchmark set gated every release. Output safeguards constrained what the model could emit. Human-in-the-loop was enforced at the product level, not left as policy language. A drift monitor with a defined clinical escalation path went live, and a documented fallback kept the clinical workflow moving even when the model was offline.

Week by week

Timeline

1Assurance stack designWeeks 1–3
Six-part stack scoped, evidence templates built
2Governance + benchmarkWeeks 3–6
Model governance signed; benchmark set frozen and versioned
3HITL + safeguardsWeeks 5–9
Human-in-the-loop enforced in product; safeguards deployed
4Drift + fallbackWeeks 8–11
Drift monitor live; fallback workflow tested end-to-end
5Submission draftingWeeks 10–13
MDR submission package delivered on schedule

Outcome

Results

Full Annex 22-aligned assurance stack in place ahead of MDR submission
Model governance published — clinical decisions in and out of scope explicit
Benchmark set frozen and versioned; every release gated against it
Human-in-the-loop enforced by the product, not by policy language
Drift monitor with a defined clinical escalation path in production
Documented workflow fallback tested and signed for regulatory review

Compliance frameworks

EU MDREU Annex 22FDA AI/ML SaMD guidanceIEC 62304

We knew the model worked. What we did not have was the story regulators would need to hear about how it was governed. Cybroscape built the whole assurance stack alongside our submission, and the reviewer questions were about substance, not gaps.

Head of Regulatory AffairsMedical device manufacturer (Class II SaMD)