Keeping a SaaS system validated when the vendor releases on its own schedule

The shift in mindset

With on-premise software, you decide when to upgrade and validate before you do. With SaaS, the vendor decides and you assess before the change reaches your production environment. Validation becomes a recurring routine rather than a project.

Teams that try to treat each release as a full revalidation exhaust themselves within a year. Teams that ignore releases discover in an inspection that the system they validated no longer exists. The workable answer is in between.

A routine that works

• Get the release notes early. Most enterprise vendors publish them weeks ahead. Make receiving them someone's job.
• Screen for relevance. Which changes touch functions you use, and of those, which touch GxP-relevant functions?
• Assess the risk. A new report you do not use is no risk. A change to how signatures or audit trails work is high risk.
• Test what matters in a preview environment. Run your regression tests for the affected functions before production is upgraded.
• Record the decision. What changed, what you assessed, what you tested, and the result.

Most releases end at step two or three. That is fine, as long as the assessment is recorded.

Use the vendor’s testing

A capable vendor tests each release thoroughly. Use that evidence rather than repeating it. What you should test is your configuration and your use of the system: workflows you built, integrations you rely on, reports that feed decisions. The vendor cannot test those for you, because they do not know how you use the product.

Assess the vendor's release process once, in depth, through an audit or detailed questionnaire. Then rely on it, with periodic checks that it still works the way they described. Our supplier qualification page covers how.

Automate the regression set

A small set of automated regression tests covering your critical GxP functions pays for itself quickly in a SaaS setting. Run them against every preview environment, and against production after each upgrade. Then the question each release is not whether to test, but whether the tests passed. Our guide to GxP test automation goes into how.

When a release goes wrong

Sometimes a release breaks something you rely on, and you find out after it reaches production. Have a plan for this before it happens. Who decides whether to stop using the affected function? What is the manual workaround? How do you assess whether records created since the release are affected? How do you work with the vendor on a fix?

Treat it as a deviation, with impact assessment on data created during the period. The vendor's fix is then a change like any other, with its own assessment and testing.

Contract terms that help

Some of this is decided before you sign. Ask for advance notice of releases, access to a preview environment, a way to delay non-critical changes for a short period, and clear notice of anything affecting regulated functions. These are reasonable requests that good vendors already meet. Our cloud & SaaS validation page covers the rest of the shared responsibility.