Data integrity breaks at the joins between systems

Why the joins get missed

Validation is usually organised by system. Each project has a scope, a team and a sign-off, and the scope ends at the system boundary. Interfaces are tested, but often only to show that data arrives, not that it arrives complete, unchanged and attributable.

Ownership follows the same lines. Each system has an owner. The transfer between two systems often belongs to nobody, or to an IT team that sees it as plumbing rather than as part of a GxP record.

What goes wrong in practice

The failures are mundane, which is why they persist.

Data exported to a file that can be edited before import, with no record that it was. A transfer that drops records when a field is empty, silently. Units or decimal places changed by a format conversion. Time stamps shifted between time zones. An interface that retries after an error and creates duplicates. A manual re-keying step that nobody thinks of as a system at all.

None of these show up when each system is validated alone. All of them show up when someone follows a single result from where it was created to where it was used.

Follow the data, not the system

The most useful exercise a quality team can do is to pick a few critical results and trace them end to end. Where was the value created? Every place it was copied, transformed or re-entered? Who could change it at each step, and would anyone know?

This produces a data flow map, and the map usually shows the risk clearly. Steps with manual handling, editable files or no reconciliation are the weak points.

Controls that help

Reduce manual steps where you can. Every re-keying or file handover is a place for error and manipulation.

Where data moves automatically, check it arrived complete: counts reconciled, and a sample compared at both ends. Make transfers and their failures visible, with logs someone actually reviews.

Give each critical data flow an owner, in the same way each system has one. And include the flow, not just the systems, in validation scope and in periodic review.

The broader lesson

ALCOA+ applies to data, not to systems. A value must be attributable, accurate and complete wherever it is, including in transit. Regulators increasingly look at data across its whole life, from creation to archive. Companies that still organise data integrity around individual systems will keep finding the same problems in the gaps between them.

Start with one critical result. Follow it. The map you draw will tell you where to look next.

For the practical detail, see hybrid systems and data integrity.