When the Check Runs Somewhere Else
A guardrail fires inside a vendor's infrastructure. The refusal lands in their logs. What reaches you is an output with no verdict attached — indistinguishable from a guardrail that crashed, and indistinguishable again from one that was never configured.
Three different states, one signal.
Evidence crosses the boundary — incompletely, and on the vendor's terms. Accountability, however, lives on only one side of the boundary: the regulated user's. The burden of proof sits with the party furthest from the proof.
Outsourcing is Not a Binary
Outsourcing isn't a binary; it's a gradient. Sponsors can choose between fully in-house models, models hosted solely by the vendor, and several options that sit between these two ends of the spectrum:
In-house
Hosted with sponsor-built harness
Vendor platform with configurable guardrails
Guardrail-as-a-service
Full-AI-as-a-service (fully vendor-hosted)
Sponsors need to watch for which layers of the harness sit outside the internal infrastructure. The harness drifts, and every layer requires monitoring:
1. Retrieval Layer — how the system finds and assembles grounding
2. Instruction Layer — how the system is told to reason
3. Action Layer — what the system is permitted to do (agentic)
4. Control Layer — the guardrails that catch failure
5. Human Performance Layer — the reviewer's performance as a monitored surface
This is why the validation boundary must be drawn around the system, not just the model. The harness drift impacts the model's output; therefore, it must be validated and monitored, just like the model. And the two layers that migrate first are also the two that generate the evidence you need most — what the model was shown, and whether the check fired.
Three Places the Boundary Fails, and What Closes Them
The boundary breaks in three areas: detection, root cause origin identification during investigation, and reviewer oversight:
Detection: If your supplier silently updates your foundation model from version 5.1 to 5.2, and its performance subtly shifts, how do you detect it? The same way a raw materials supplier change triggers a standing test you re-run on each lot, a foundation model update should trigger a fixed challenge test against each model version.
Root cause: Without an independent referent, you can't classify what you can't see. If the true classification is invisible, trending in the QMS becomes skewed. Supplier-origin failures are misclassified as training data or model inference errors. This means time and resources are allocated towards corrective actions that don't address the core issue. This is why continuous validation becomes critical with generative AI; regularly scheduled challenge sets allow you to trace output shifts to a point in time. With this data, narrowing down a root cause during a deviation investigation becomes a more efficient and robust exercise.
Reviewer oversight: The vendor can own the model, the retrieval, even the guardrails. But the moment they own the reviewer interface, the draft Annex 22 condition stops being evidenceable. Draft Annex 22 allows probabilistic models in non-critical GMP applications only on the condition of qualified human oversight. If the record of the human-AI interaction lives with the vendor, the evidence that the implementation of the models hinges on doesn't live on the sponsor's side of the wall. The independent test set is doing double-duty: it qualifies both the model and the reviewer on an ongoing basis.
The Hidden Trust-Boundary Story
The OpenAI-Hugging Face incident has been heavily covered as an AI safety story — which it is — but it is also a trust-boundary story. The models (GPT-5.6 Sol and a more advanced model in development) used a zero-day exploit in Jfrog Artifactory, a package-registry proxy, to escape their sandbox. They inappropriately accessed the Internet to find the answers to a cybersecurity assessment. The guardrails were intentionally dialed down to allow the models to complete the assessment. The containment failed anyways.
Hugging Face detected and contained the intrusion themselves, and reconstructed more than approximately 17,000 attacker actions from logs on their own side of the line. To do so, they used an open-source model, GLM 5.2, because the foundation models refused to respond on safety grounds (they could not distinguish an adversarial attack from a reconstruction for What they could not produce was the other side. Five days later, OpenAI supplied the attribution — the models were its own — and when Delangue publicly asked for the agents' execution traces, OpenAI pointed to a forthcoming report instead.
Procurement due diligence for agentic tooling in GxP contexts must establish what execution records exist, who can read them, and how quickly. Inherited evidence either arrives on the vendor's terms, or not at all.
Beyond SOC 2
SOC 2 attests that controls exist and are described; it doesn't tell you whether a specific guardrail fired on a specific transaction. ISO 42001 governs the vendor's management system, not the evidentiary path across the line. While both are necessary, neither allows you to reconstruct the transaction-level actions of a model sitting behind the vendor's wall.
Compressing the Boundary
There are three types of methods that the sponsor should employ to compress the trust boundary: prospective, architectural, and data-level. They are additive and layered, not substitutive.
Prospective: This is the responsibility allocation declared before deployment. By identifying upfront, in writing, which evidence you own as the sponsor and which you inherit, you give procurement a structured method to assess potential vendors, and you give the qualification team a clear roadmap to assess the validation package against. As Tanya Sharma, cofounder of Assurea and ISPE GAMP Americas Secretary phrases it, “Good supplier assurance isn’t about collecting more documentation. It’s about knowing which evidence you can rely on and where you still need to build your own assurance.”
Architectural: The unified capture layer essentially functions as the bridge, capturing across the boundary rather than depending on the vendor's log. When we can't test reproducibility of outputs from inputs, we need an instrument that allows us to trace each step of a recorded run, including both the machine's actions and the human reviewer's decisions. Without the UCL, that trace is truncated architecturally when any part of the infrastructure sits behind the vendor's wall.
Data-level: Content-addressed capture. Recent industry trends demonstrate a shift toward contemporaneous, system-generated evidence, consistent with data integrity expectations. Multiple binding and near-binding regulatory documents have shifted toward risk-based, lifecycle, and ongoing-performance evidence: FDA Computer Software Assurance (CSA), the FDA's January 2025 draft risk-based credibility assessment framework, and EU draft Annex 11.
An article in the January/February 2026 issue of Pharmaceutical Engineering (Canterbury, Sharma, and Killingsworth) titled "Content Addressability in the Future of Knowledge Management" highlights content-addressable storage (CAS) with cryptographic fingerprints yielding "mathematically provable version control."
Where deterministic software has artifacts that are portable by construction, allowing the trust boundary to be stitched together by construction, probabilistic AI has no equivalent. This is where the UCL becomes necessary as a method of reconstructing the decision path.
The Empty Cell Error
A boundary between the model and its evidence is a liability; a boundary between whoever built the system and whoever attests to it is load-bearing. Most sponsors place them backwards.
The qualification record's job isn't to prove the vendor is trustworthy, it's to make explicit which evidence you inherit, which you generate, and which cell is empty. Empty cells are documented residual risk or they're findings — there's no third option.
