A sample OOS result is reported for a recent batch in a clinical manufacturing facility. The quality assurance professional assigned to the investigation uses a piloted deviation triage assistant computerized system for intake. The system processes the scenario using a large language model, and based on the content, accesses a sandbox/non-production version of LIMS via its tool capabilities to verify the result. It then proceeds to use a RAG architecture to verify the investigation procedure for an OOS result in the relevant SOP. The final record is produced and formatted in accordance with the SOP:
Based on the organization’s human review procedure, the professional verifies the final report, adds a justification in the sandbox QMS, and submits it as the pilot’s record of disposition. When the broader team reviews the pilot results, an error in the final report is identified: the wrong batch was pulled. The reviewer only saw the final output, not the current version of the SOP or the LIMS results. The error is classified as a contextual misapplication caused by a RAG error. The model pulled a closely related historical version of the SOP which pointed it to a different batch.
This case study represents the potential consequence of an agentic error committed by a computerized system.
Agentic is Not a New Type of System
How do we validate this computerized system?
To answer that question, we first have to identify what the system is. Agentic describes the colloquial label we apply to the technology. The architectural characteristics of the system, and how they modify the system’s behavior, define what needs to be validated.
Context of use (CoU) frames the system. Per the glossary, a probabilistic system is defined as any architecture composed of:
System = Model × Harness × HITL/HOTL within its CoU
with the capacity for output variation. The agentic system described in the case study is consistent with this definition, buts its harness carries significantly more operational weight.
In the table below, we walk through the non-agentic probabilistic system vs. the agentic one:
The Action Surface
If the system is defined the same way, what changes?
Agentic architectures often have autonomous capabilities, including:
Tool Invocation: Executing read/write API requests against external systems (e.g., LIMS, QMS, ERP).
Authority & Permission Scoping: Enforcing identity, authorization checks, and write-action boundaries at the harness level.
Multi-Step Chaining: Executing sequential workflows where the output of step N dynamically determines the tool call for step N+1.
Evolving Checkpoints: Shifting from passive end-state reviews to active checkpoint gating, ensuring human-in-the-loop oversight is architecturally enforced before state-changing operations occur.
Agents and the Probabilistic Validation Lifecycle
An agentic system can be validated with the same lifecycle as any other probabilistic system:
↓1. Context of Use: Who uses the model, for what decision, with which failure modes? The context of use fixes the intended purpose and the consequences of getting it wrong. Everything downstream is scoped from here.
↓2. Risk Tiering: Risk is tiered by consequence to patient safety and data integrity, not by software category. The tier sets how much evidence each later step must produce, so a high-consequence use earns deeper validation than a low-stakes one.
↓ 3. Evaluation Design: Traceable, domain-specific evaluation sets with lineage (ALCOA+), leakage controls, and real edge cases. The Error Taxonomy maps candidate failure modes to consequence-weighted outcomes so the eval targets what actually matters.
↓ 4. Acceptance Criteria + Validation Execution: Probabilistic systems don't pass or fail on a single run. Acceptance criteria are defined as bounded-variance thresholds calibrated to the risk tier — what range of behavior is acceptable, and how confident must we be in it.
↓ 5. HITL/HOTL Controls: Human-in-the-loop and human-on-the-loop controls are designed to match the risk tier: where a person reviews, where they supervise, and where the system runs unattended. Oversight is engineered, not assumed.
↓ 6. Continuous Monitoring: Post-deployment, performance is monitored against the acceptance criteria with drift detection on inputs, outputs, and the operating context. Monitoring turns validation from a moment into a standing state.
↓ 7. Change Control: When the model, data, or context changes — or a monitoring trigger fires — change control decides what must be re-evaluated and routes the system back to the relevant step. This is the loop that closes back to context of use.
↺ (back to 1. Context of Use)
The Case Study: What the Agent Changes
Context of Use: System may process deviation intake information to produce the most likely cause for a root cause investigation. Agentic change: system may access an approved set of tools (to be specified by the organization), may produce a draft report that must be reviewed prior to submitting as the GxP record. ↓
Risk Tiering: Decision consequence x model influence; failure mode taxonomy risk assessment; determination on the type and depth of assurance evidence. Agentic change: risk assessment must consider level of autonomy (model influence); agentic failure mode origination point; ↓
Evaluation Design: Test the system on a representative set of 75 historical deviations (across years, sample types, batches) and compare the output to the “ground-truth” final classification. Agentic change: Add adversarial testing design and action ability. ↓
Acceptance Criteria + Validation Execution: Based on risk-tier (medium-high), set bounded acceptance criteria (e.g., correct root cause classification in 90% of the test set, across three test runs). Agentic change: Additional security testing, verifying adherence to autonomy scope restrictions. ↓
HITL/HOTL Controls: Human performance qualification on defined competencies; in-production testing using historical deviations with seeded errors prior to reviewing tasks in production; monitoring via pre-defined metrics (e.g. time-on-task, seeded challenge). Agentic change: Implementing gated checks before autonomous actions are allowed to proceed. ↓
Continuous Monitoring: Ensuring the in-production RCA identification accuracy remains within the pre-defined validated envelope and that the failure modes identified remain in a state of control. Agentic change: Ensuring the action scope remains within the pre-defined validated envelope; the system has not accessed any tools that are out of scope. ↓
Change Control: Pre-defining an acceptable change control based on validation testing when a system update or context of use expansion occurs outside of the validated boundary (e.g., the foundation model inside the system updates to a new version); opening a change control when an update occurs outside of that defined envelope.
The Floor: Making the Trajectory Inspectable with the UCL
Agentic architectural components add complexity to a probabilistic system. The system may take multiple different pathways for the same request. A reviewer must verify that each step the system took is within the defined context of use.
A Unified Capture Layer (UCL) makes this possible via:
Dynamic audit trails: Logging model reasoning, selected tools, queried endpoints, and step-by-step parameter payloads.
Provenance tracking: Linking every generated claim directly to source data (e.g., exact LIMS records and active SOP document IDS).
The reviewer is able to verify the execution capture against the defined context of use. This transforms human oversight from passive trust into active verification.
What To Ask When You Implement an Agent
Can your human reviewer replay and inspect the exact execution trace before signing off on the record?
