EU GMP Annex 22 Checklist: Free 39-Point AI Readiness Check (Draft)

01 Scope and type of model

Draft Annex 22 section 1. Critical domain.

• 1

  Have you listed every AI model used in GMP and classed each use as critical (direct impact on patient safety, product quality or data integrity) or not?

  § 1

  If not: Build an inventory of AI models in GMP processes and record, for each use, whether it directly affects patient safety, product quality or data integrity.

• 2

  Are the models in critical uses static, meaning frozen and not learning or adapting during use?

  § 1

  If not: Freeze models used in critical applications and release changes only as new tested versions; keep adaptive models out of critical uses.

• 3

  Do models in critical uses give deterministic output, the same output for the same input?

  § 1

  If not: Confirm and document that each critical model returns identical outputs for identical inputs, and move probabilistic models to non-critical uses.

• 4

  Are generative AI and large language models kept out of critical uses, and used elsewhere only with a qualified person checking the output?

  § 1

  If not: Restrict generative AI to non-critical uses and require a trained person to check each output for its intended use (human in the loop).

02 Principles and Annex 11 basis

Draft Annex 22 section 1, 2.

• 5

  Do process experts, QA, data scientists and IT work together through algorithm selection, training, testing and operation, with defined responsibilities?

  § 2.1

  If not: Name the process SME, QA, data science and IT roles for each model, with responsibilities and access levels, and involve them from algorithm selection onward.

• 6

  Do you hold and review the model documentation, including where a supplier trained and tested the model?

  § 2.2

  If not: Obtain the supplier's training, validation and test documentation and review it as the regulated user before relying on the model.

• 7

  Are AI activities scaled by quality risk management, based on risk to patient safety, product quality and data integrity?

  § 2.3

  If not: Apply a documented risk assessment to each model and use it to set the depth of testing, review and monitoring.

• 8

  Is the computerised system the model runs in validated under Annex 11?

  § 1

  If not: Validate the host system under Annex 11 first; Annex 22 is drafted as additional guidance on top of Annex 11, not a replacement.

• 9

  Are suppliers of AI models or platforms assessed, with agreements that cover training, testing, documentation and changes?

  § 2.2

  If not: Add AI-specific questions to supplier assessment and agreements: training data, test evidence, documentation access and change notification.

03 Intended use

Draft Annex 22 section 3. Critical domain.

• 10

  Is the intended use described in detail, including the input data and all its common and rare variations (the input sample space)?

  § 3.1

  If not: Write an intended use description that characterises the input data, including rare variations, based on knowledge of the process.

• 11

  Does it identify the model's limitations and possible erroneous or biased inputs?

  § 3.1

  If not: Add a section on limitations and on inputs that could be wrong or biased, and how each is handled.

• 12

  Has a process subject matter expert approved the intended use description before acceptance testing starts?

  § 3.1

  If not: Make SME approval of the intended use a gate before the test plan can be approved.

• 13

  Is the input sample space divided into relevant subgroups, such as decision outcome, site, equipment, material or defect type?

  § 3.2

  If not: Define subgroups from the characteristics that matter for the task, so performance can be tested and accepted per subgroup.

• 14

  Where a person makes the decision using the model's input, does the intended use set out that operator's responsibility?

  § 3.3

  If not: Describe the operator's role and responsibility in the intended use, and monitor their training and performance like any manual process.

04 Acceptance criteria

Draft Annex 22 section 4. Critical domain.

• 15

  Are test metrics defined to suit the intended use, such as a confusion matrix, sensitivity, specificity, accuracy, precision or F1 score?

  § 4.1

  If not: Choose metrics that match the task, for example sensitivity and specificity for an accept or reject classifier, and record why.

• 16

  Are acceptance criteria set for those metrics, per subgroup where needed, and approved by a process SME before testing?

  § 4.2

  If not: Set numeric acceptance criteria, per subgroup where relevant, and have the process SME approve them before acceptance testing.

• 17

  Must the model perform at least as well as the process it replaces, and is that process's performance known?

  § 4.3

  If not: Measure the current manual or automated process first, and set acceptance criteria no lower than that baseline.

05 Test data

Draft Annex 22 section 5.

• 18

  Is test data representative of the full intended use, stratified across all subgroups and rare variations, with the selection rationale documented?

  § 5.1

  If not: Select test data to cover every subgroup and rare variation, and document the selection criteria and rationale.

• 19

  Is the test set large enough, overall and per subgroup, to calculate the metrics with adequate statistical confidence?

  § 5.2

  If not: Calculate the sample size needed for the required confidence per subgroup and collect test data to meet it.

• 20

  Are test data labels verified to a very high degree of correctness, for example by multiple independent experts or laboratory tests?

  § 5.3

  If not: Verify labels through independent review by more than one expert, validated equipment or laboratory testing, and record the verification.

• 21

  Is pre-processing of test data pre-specified and justified, and is any cleaning or exclusion documented and justified?

  § 5.4, 5.5

  If not: Specify pre-processing before testing with a rationale, and record every excluded or cleaned item with its justification.

• 22

  Do you avoid generating test data or labels with generative AI, or fully justify it where used?

  § 5.6

  If not: Use real data and human-verified labels for testing; where synthetic data is unavoidable, document the full justification.

06 Test data independence

Draft Annex 22 section 6. Critical domain.

• 23

  Do technical or procedural controls make sure test data is never used in development, training or validation of the model?

  § 6.1

  If not: Capture test data after training ends, or split it off before training starts, and control it so developers cannot use it.

• 24

  Is test data held in an access-controlled repository with an audit trail of access and changes, and no copies elsewhere?

  § 6.2

  If not: Move test data into a controlled repository with access control and an audit trail, and remove copies held anywhere else.

• 25

  Is it recorded which data was used for testing, when, and how many times?

  § 6.3

  If not: Log each test run against the test dataset version used, so repeated use of the same test data is visible.

• 26

  Are people who have seen the test data kept out of training and validating the same model, or paired with a colleague who has not?

  § 6.5

  If not: Separate test data custodians from model developers, or apply the four-eyes rule the draft describes where separation is impossible.

07 Test execution

Draft Annex 22 section 7.

• 27

  Is a test plan approved before testing, covering intended use, metrics, acceptance criteria, test data, the test script and how metrics are calculated?

  § 7.2

  If not: Use a test plan template with these sections, involve the process SME, and approve it before any test is run.

• 28

  Does the testing show the model generalises to new data, including checks for over- or underfitting?

  § 7.1

  If not: Test on independent data from the intended use and compare against training performance to detect over- or underfitting.

• 29

  Are deviations from the test plan, failures against acceptance criteria and unused test data documented, investigated and justified?

  § 7.3

  If not: Route every test deviation through your deviation process and record why any test data was not used.

• 30

  Are test documentation, the test data and the records of access to it retained like other GMP records?

  § 7.4

  If not: Archive the test plan, results, test data, physical test objects where relevant and access logs under your GMP retention rules.

08 Explainability

Draft Annex 22 section 8.

• 31

  During testing, does the system record which features drove each decision, for example with SHAP values, LIME or heat maps?

  § 8.1

  If not: Add feature attribution to the test set-up for critical models and keep its output with the test results.

• 32

  Are those features reviewed for relevance as part of approving the test results?

  § 8.2

  If not: Make a review of decision features a step in test result approval, so a model deciding on irrelevant features is caught.

09 Confidence

Draft Annex 22 section 9.

• 33

  Does the system log a confidence score for each prediction or classification?

  § 9.1

  If not: Configure the system to record the model's confidence score alongside each output.

• 34

  Is there a confidence threshold, with low-confidence results flagged as undecided rather than forced into a decision?

  § 9.2

  If not: Set and justify a threshold, and route results below it to an 'undecided' outcome for human handling.

10 Operation

Draft Annex 22 section 10.

• 35

  Are the model, its system and the process under change control before go-live, with every change assessed for retest?

  § 10.1

  If not: Bring model, system and process under change control, including changes to physical inputs, and justify any decision not to retest.

• 36

  Is the deployed model under configuration control, with measures that detect unauthorised changes?

  § 10.2

  If not: Version and checksum the deployed model and alert on any change that did not come through change control.

• 37

  Is the model's performance monitored regularly in operation against its defined metrics?

  § 10.3

  If not: Set a monitoring schedule using the test metrics, with limits that trigger investigation.

• 38

  Is input data monitored for drift outside the sample space, with defined drift metrics?

  § 10.4

  If not: Define drift metrics for the input data and review them regularly against the intended use.

• 39

  Where people review the model's output, are records of that review kept, covering every output where the risk requires it?

  § 10.5

  If not: Write a review procedure that sets which outputs are reviewed, by whom, and how the review is recorded.

Adding an AI visual inspection model to a packaging line

This is a critical use with a clear accept or reject output, which is the case the draft was written for. Define the intended use and subgroups, set acceptance criteria no lower than the current inspection, and protect the test images from the developers. Start with checks 1, 10, 13, 16, 17, 23 and 34.

Switching on an AI feature inside existing software

Vendors increasingly ship AI features in LIMS, MES and QMS products. Decide first whether the feature is used in a critical way, then obtain and review the supplier's training and test evidence before relying on it. Start with checks 1, 4, 6, 9, 35 and 36.