A month ago, I published The Validation Boundary. Two weeks ago, I published The Inheritance Boundary. Today, I close out the House Boundary series with The Data Boundary.
The Validation Boundary covered the horizontal boundary: how the probabilistic system as the unit of validation applies at layers 1-5 of the House. The Inheritance Boundary covered the vertical boundary: how the type of evidence that may be inherited, either from external sources or from other internal sources, changes as you walk up the floors of the house. The Data Boundary caps the series by covering the flow of data through the house, both inbound and at re-entry.
Data governance sits at the base of the house alongside regulatory infrastructure, because a strong data foundation is required for the house to bear weight. While data governance is a mature discipline, the flow, provenance, and quality of data carry additional importance when probabilistic AI systems enter the picture.
The House of AI Trust was designed to govern any AI use case, regardless of technology or system property. This piece covers the flow of data within a probabilistic system. Data passes through the validation boundary as the system runs, and it touches the inheritance boundary when its provenance is in question. The data boundary is not a third boundary; it is defined by where data crosses the existing ones.
Why Mature Data Governance Doesn’t Cover the Full Data Boundary
Data governance is a mature discipline, but it was built on the assumptions that deterministic systems carry: records are created by people, or by systems, where the same input leads to the same output. With probabilistic AI systems, system outputs may re-enter the corpus, and errors in data sources may both propagate and compound into AI outputs.
In some instances, these data errors are misclassified as “hallucinations” or LLM/agentic errors. Sambasivan et al. outline both the causes and consequences of data propagation more broadly in “Everyone wants to do the model work, not the data work.“ The propagation problem makes the case for a data inheritance gate, where data provenance, quality, and traceability are verified prior to initiating the risk assessment for a probabilistic AI system.
In probabilistic systems, including large language models and agentic AI, data flows not only during training time, but during runtime also. Regulatory direction suggests that data should be governed at runtime as well as during training: Principle 6 of the FDA-EMA Guiding Principles of Good AI Practice states that “data source provenance, processing steps, and analytical decisions are documented in a detailed, traceable, and verifiable manner, in line with GxP requirements . . . appropriate governance, including privacy and protection for sensitive data, is maintained throughout the technology’s life cycle.”
The Runtime Data Challenge: The Precedent Loop
Deterministic systems require governed data. Additional system properties, including probabilism, dynamism, and adaptiveness each add unique requirements. Here, I focus primarily on probabilism.
The coupling bifurcates; probabilism couples data governance to the layers above it, in two ways:
Validation coupling (Layer 1 ↔ Layer 3): During validation assurance activities, the validated boundary is established via testing against representative datasets (separate from the training datasets). Data provenance, ownership, and cleanliness must be established prior to initiating the AI risk assessment (step 2 of the Probabilistic Validation Lifecycle). The data inheritance gate prevents error propagation downstream.
Runtime coupling (Layer 1 ↔ Layers 2 and 4): During production, the runtime data is monitored by data governance. However, monitoring is not equivalent to control. The named system owner is responsible for ensuring that the system remains in a state of control while it is in production.
In addition, the domain expert must justify why the chosen validation boundary is acceptable for a given use case. When runtime context drifts outside of the boundary, the domain expert may also assist in documenting the rationale for domain-specific failure modes, such as subgroup performance or calibration bias.
Runtime Retrieval
Ingestion / indexing: this portion of the retrieval corpus is owned by traditional data governance.
Retriever: the retriever lives in the harness, and is a separate risk surface from ingestion. The harness is a component of the validated boundary, which is owned by the House of AI Trust Governance and Control layers (Governance during production runtime, and Control during validation assurance execution).
Context assembly: the context retrieval crosses layers 1-3; during validation execution, the probabilistic AI system retrieves relevant context from layer 1 (Data Governance), typically a knowledge graph or something similar. Errors during context assembly could therefore originate with either the knowledge layer itself or with the retrieval mechanism.
Model/HITL: the model and human-in-the-loop similarly cross layers 1-3 during validation execution. The human-in-the-loop should be demonstrated to be qualified for the role, via human performance qualification.
2. Re-entry
During boundary re-entry, the approved output enters the GMP record (if a HITL is involved, the approved output is the one finalized by the human). In the future, this is then retrieved from the corpus.
Approved output → GMP record → retrieval corpus → retrieved as precedent. Lineage and provenance questions arise: is it flagged as AI-drafted, which model version, and what did the reviewer change? A deterministic capture layer (like the Unified Capture Layer) is needed to record these data transactions.
Once approved outputs return into the system as precedent, the runtime evidence stream is no longer independent of the system it is meant to monitor.
The Data Inheritance Gate
To demonstrate why the data inheritance gate is required, take the deviation triage use case from The Inheritance Boundary.
The gate sits upstream of Step 1 (Retrieval corpus).
The deviation triage example demonstrates that the gate allows you to inherit ALCOA++ data integrity. Accuracy of the root causes in the retrieved historical records is what must be verified at the gate. However, a human must still verify the accuracy of the proposed root causes within the context-of-use during runtime.
The gate alone cannot close the loop. The gate runs once, before the risk assessment in the Probabilistic Validation Lifecycle. However, re-entry adds records continuously while the system is in production. This necessitates an instrument to monitor the re-entry process and ensure data provenance in higher risk use cases. A deterministic instrument, such as the Unified Capture Layer (UCL), may serve this purpose. The system needs a mechanism to ensure the runtime context remains within the validated boundary at all points where data flows.
The Probabilistic Failure Mode Taxonomy and Controls
When someone identifies an error in the system’s output, it initially looks like a model or reviewer error. No one suspects that the error may have originated with the data source until the team is several days into the root cause analysis investigation.
This is both a case for the data inheritance boundary, and a case for multi-cell attribution during risk assessment and CAPA generation using the Probabilistic Failure Mode Taxonomy. The inaccurate root cause analysis is found to map to the following origination points: data, retrieval, and human-in-the-loop.
Established controls don’t cover the full data boundary of a probabilistic AI system: Part 11/draft Annex 11 revision/draft Annex 22 cover audit trails, step-level tracing, human-in-the-loop requirements, and drift/performance monitoring. Tooling mapped to these requirements partly covers workflow errors.
But what about retrieval, generation, re-entry, and human reviewer error detection?
Detection: ICH Q9(R1) explicitly calls out detectability. Several failure mode detection mechanisms may be specified in the URS or supplier assessment for organizations implementing probabilistic AI-based workflows, including large language models and agentic AI.
Provenance Fields: provenance should be captured as part of the record: any AI-assisted flags shown to the human reviewer, model identifier and version, prompt/harness configuration version, document identifier and version of retrieved documents, the unedited draft, the identity of the human reviewer and the human-AI interaction. These items should be explicitly documented in the user requirements and supplier assessment.
Deterministic Capture Lineage: Deterministic capture mechanisms, including the Unified Capture Layer, allow QA/the system owner to trace the record of actions committed by both the AI and the human, across the supplier-sponsor inheritance boundary, to assist in the process of determining the source of an observed failure mode.
Retrieval Logs: the retrieval logs are an additional source of error detection; by tracing the lineage and provenance of the data the system actually pulled during retrieval, retrieval can either be ruled in or ruled out as a cause of the observed failure mode. If retrieval logs are not retained with the GMP record, retrospective impact assessment for a misclassified or undetected deviation becomes impossible. This is a data integrity issue.
Human-in-the-loop: draft Annex 22 §3.3 states that where validation testing is lower because a human makes the final decision in the workflow, “training and consistent performance of the operator should be monitored like any other manual process,” and that “records should be kept from this process.” Records require demonstrable evidence, including acceptance rates, edit rates, time-on-task, and seeded challenges.
LLM-as-a-judge: The LLM judge will check the draft against the same retrieved context as the original LLM, in most cases; it catches fabrication, but not retrieval error. All large language models are probabilistic by design, and LLM monitors require their own validation controls, including measurement against ground truth. This mechanism should be reserved for low-risk, low impact use cases as the only control. Layered controls are only effective when the layers are independent of one another.
2. Remediation: Once a wrong root cause has propagated, does CAPA fix one record or every downstream draft that retrieved it?
Each downstream record is its own approved GMP record requiring its own documented correction.
Digital validation / AI-enabled tools:
Less fragmented, linked data turns "which records did this touch?" from rhetorical into a query.
Digital validation and some AI-enabled tools consolidate and organize data records.
Consolidation cuts both ways: integration makes propagation faster and traceable. Lineage is what makes it safe.
Keep AI-enabled tools deterministic (lineage, traceability), consistent with the UCL; probabilistic detection of bad records needs its own boundary.
The Validation Boundary, The Inheritance Boundary, and The Data Boundary: The Interdependent Threads
The data boundary is where the validation and inheritance boundaries must stay coupled while the system runs. Monitoring is not equivalent to control.
The FDA-EMA released ten Guiding Principles of Good AI Practice in Drug Development on January 14th, 2026; one of these ten principles was multidisciplinary expertise. Robust, controlled, and assured probabilistic AI workflows cannot be implemented with teams working in silos. Coupling across the house layers means the teams that own those layers have to coordinate.
A follow-up piece to “You Can’t CAPA What You Can’t Classify,” walking through an example of a failure mode with multi-cell attribution is in progress ("You Can't CAPA What You Can't Locate”). Once you can trace a failure across the boundary, how do you attribute it?
