FDA's AI credibility framework: seven steps and one idea worth borrowing

What it covers

The guidance covers AI used to produce information or data that supports regulatory decisions about safety, effectiveness or quality. That includes uses in clinical development, manufacturing and post-market work. It does not cover AI used only in drug discovery, or AI used for operational efficiency that does not affect patient safety, product quality or the reliability of study results.

The seven steps

• Define the question of interest. What decision is the AI helping to answer?
• Define the context of use. Exactly what the model does, what it produces and how that output is used, including what other evidence sits alongside it.
• Assess model risk. A combination of model influence, meaning how much the output drives the decision, and decision consequence, meaning how bad a wrong decision would be.
• Plan the credibility assessment. What evidence will show the model is fit for this context, scaled to the risk.
• Execute the plan.
• Document the results and any deviations from the plan.
• Decide whether the model is adequate for the context of use, and what to do if it is not.

The idea worth borrowing

Even if your AI never touches a regulatory submission, the risk model is useful. Model influence and decision consequence are two questions anyone can answer about any AI use. An AI whose output is one of several inputs to a reversible decision is low risk. An AI whose output alone decides something with patient impact is high risk. Most real uses sit in between, and the framework makes you say where.

This lines up closely with the factors in the draft EU GMP Annex 22 discussions, which we cover in the four factors for judging AI risk.

Context of use is the hard part

Most weak AI validation starts with a vague purpose. Supports quality decisions is not a context of use. Flags batch records with missing entries for human review, where every flag and every unflagged record in a sample is checked by a reviewer, is. The second can be tested. The first cannot. Our piece on writing an AI intended use statement goes into how to write one.

Talk to FDA early

The guidance encourages early engagement with FDA about the credibility assessment plan, especially for higher-risk uses. That is good advice in its own right. Agreeing the context of use and the evidence expected before generating it is much cheaper than discovering a gap during review.

For lower-risk uses, the same discipline still helps. Writing the context of use, the risk assessment and the credibility plan down before testing makes the work easier to defend and much easier to repeat when the model changes.

Lifecycle, not a one-off

The guidance expects credibility to be maintained, not just shown once. Models change, data drifts, and the context of use can widen quietly. Plan how you will monitor performance and what will trigger a fresh assessment. When an AI model changes covers the triggers, and our GxP AI validation page the wider approach.