Prompt Change Control: When Editing a Prompt Needs Revalidation

Treat prompts as configuration

Give every prompt an identifier and a version, store it where changes are recorded, and link each version to the validation evidence that covers it. The same applies to the settings around it: the model version, temperature, the documents it is allowed to search and any rules applied to its output.

Record which prompt and model version produced each output. Without that, you cannot answer the question "was this document drafted by the version we validated?"

Decide what counts as a minor change

Not every edit needs full retesting. Agree categories in your procedure, for example:

• Cosmetic: wording of the output format, with no change to the task or the facts used. Re-run a small regression set.
• Behavioural: a new instruction, a changed rule, a different source set. Re-run the full test set against the original acceptance criteria.
• Scope: a new task or a new type of record. This is a change of intended use and needs its own validation.

The categories should be decided by the risk of the decision the output supports, not by how small the edit looks.

Keep a regression set ready

A fixed set of inputs with agreed answers lets you check any prompt change quickly. Keep it with the validation records and add to it whenever a real error is found, so the same mistake cannot come back unnoticed.

Watch for changes you did not make

If the model comes from a vendor, it may change underneath your prompt. Your supplier agreement should require notice of model changes, and your monitoring should notice a shift in output even if the notice never arrives. See change control and revalidation when the AI model changes and Annex 11 Chapter 7 and your AI vendor.