Bounding the Uncertainty: An Error Taxonomy for Non-Deterministic AI in Regulated Life Sciences

The Probabilistic Problem

An LLM deployed for deviation triage identifies a root cause with 86% confidence. Upon investigation, the output contains an error: the root cause is incorrect. The quality team flags it. But when they write the deviation report, they face a question the existing quality system has no answer for: what kind of error was it?

In traditional computerized systems, errors are well understood. The system is deterministic: it either executes the instruction correctly or it does not. When it fails, the failure is classifiable within existing frameworks — software bug, configuration error, data-integrity failure, hardware malfunction. The ISPE GAMP AI Guide (led by Brandi Stockton, Eric Staib, and Martin Heitmann) and the ML Risk & Control Framework (Blumenthal, Erdmann, Heitmann, Lemettinen, & Stockton, 2024) establish the risk-management process; this error taxonomy provides the failure-mode inventory that process operates against, specifically for the generative and probabilistic AI systems that do not cleanly fit the traditional ML lifecycle model.

Probabilistic AI systems break this model. An LLM can produce output that is:

• Entirely fabricated (hallucination)

• Partially correct but misleadingly assembled

• Technically accurate but contextually inappropriate

• Right in content but wrong in confidence (miscalibration)

• Within scope on most outputs but exceeding scope on edge cases

• Accurate in aggregate but inaccurate for underrepresented subpopulations (population bias)

Each of these is a fundamentally different failure mode with different root causes, different risk profiles, and different corrective actions. Treating them all as “the AI was wrong” is the equivalent of classifying every deviation as “equipment malfunction”: it tells you nothing actionable and prevents meaningful trending.

The pharmaceutical industry needs a standardized error taxonomy for probabilistic AI systems in GxP drug-development workflows — one that classifies failures by type and by origin point, maps them to severity, and enables the same kind of systematic root-cause analysis and trending we apply to every other quality event logged in the QMS.

QA professionals will recognize the structure: it is FMEA-shaped, decomposing failure modes by manifestation (class) and cause (origin), with risk tiering applied through the FDA seven-step framework. The familiarity is intentional. The substantive content addresses what FMEA was not built for: probabilistic systems where failures do not have stable identities and reproducibility is bounded rather than absolute. To the author’s knowledge, this is the first public two-dimensional matrix for non-deterministic/probabilistic AI anchored in GxP context.

The Two-Dimensional Probabilistic Error Taxonomy: Error Type × Origin

Classifying output errors from probabilistic systems requires a shared vocabulary for understanding what “wrong” looks like.

Probabilistic errors do not have fixed identities, so classifying them by surface category alone obscures meaningful information about the error. Hence the two-dimensional taxonomy: error type and origin. This enables organizations to address each error with the appropriate architectural control. Instead of labeling an LLM-generated adverse event a “fabrication,” we label it a fabrication and then track where in the pipeline the error occurred. A fabrication caused by training data poorly representative of the use case requires a different root-cause investigation than a fabrication caused by a model-inference error.

Dimension 1: Error Type — What Went Wrong

Class 1 — Fabrication. These are the classic “hallucinations.” In regulated settings, one method of preventing this class of error is Retrieval-Augmented Generation (RAG); see the GAMP AI Guide’s RAG infrastructure controls.

• 1a: Citation/reference fabrication

• 1b: Data-point fabrication (dosages, study results, AE data)

• 1c: Entity fabrication (drugs, conditions, sites, identifiers that do not exist)

Examples:

• “Made-up” citations or references (e.g., an SOP that does not exist)

• Invented data points (e.g., dosage, manufacturing yield)

• Hallucinated adverse events not present in patient reports

The infrastructure-qualification principles in GAMP 5 Second Edition (2022) Appendix M11 (IT Infrastructure) — backup/restore testing, integrity checks — and the data-governance provisions of the ISPE GAMP AI Guide (2025) extend to the vector store, which is appropriately treated as a regulated record store requiring IQ/OQ verification.

Class 2 — Misinterpretation. Misinterpretation includes recognition errors, reasoning errors, and omission.

• 2a: Recognition errors (entity confusion, semantic similarity)

• 2b: Reasoning errors (logical/inferential failures from correct recognition)

• 2c: Omission (failure to surface available relevant information)

Examples:

• An LLM used to flag environmental-monitoring limit alerts fails to identify an excursion because it conflates the action limit with the alert limit.

• Jin et al. (2024) reported that “GPT-4V frequently presents flawed rationales in cases where it makes the correct final choices (35.5%), most prominent in image comprehension (27.2%).”

• In Wong et al. (2021), the Epic Sepsis Model reported a sensitivity of just 33% — meaning it failed to identify roughly two-thirds of sepsis cases in its external-validation cohort of 38,455 hospitalizations, despite the relevant data being present in the EHR.

Omission is classified here as a subclass of Misinterpretation because it represents a failure in processing available information. However, practitioners should note that omission requires distinct test strategies — specifically, completeness testing against a defined reference set — and may warrant elevation to a standalone class as the taxonomy matures.

Class 3 — Contextual Misapplication. The model produces results outside the bounds of appropriate contextual constraints, while perhaps appearing “correct” on the surface.

• 3a: Population/context mismatch (pediatric vs. geriatric, jurisdiction)

• 3b: Temporal validity (withdrawn guidance, version drift, knowledge cutoff)

• 3c: Specification non-compliance (output does not match instruction/format/scope)

Examples:

• Clinical LLMs citing withdrawn or superseded FDA guidance documents

• A model including data from an irrelevant reference population (e.g., including both EU and US data when queried for EU data only)

• A model omitting required regulatory section headings from a submission document

While 3a and 3b involve failures of contextual judgment, 3c involves failure to adhere to explicit procedural constraints. It is included here because the underlying mechanism is the same: the model produced output outside the bounds of its specified operating context.

Class 4 — Confidence Miscalibration. The model hedges where certain, is confidently wrong, or collapses ambiguous evidence into a clean binary.

• 4a: Overconfidence (high confidence on fabrication or error)

• 4b: Under-confidence/hedging (excessive uncertainty on established facts)

• 4c: Failure to flag conflicting evidence

Examples:

• Confidence overweighting: a maximum-confidence statement on fully fabricated content

• Under-confidence/hedging: a model states it is 32% certain of a deviation root cause when its empirical accuracy on such cases is closer to 88%

• A model summarizing a regulatory dossier encounters two sources that contradict each other on a stability specification and weights one more heavily without flagging the conflict

Class 5 — Boundary Violation. The model acts outside its sanctioned scope, authority, or safety boundary.

• 5a: Scope creep

• 5b: Authority creep

• 5c: Adversarial boundary breach

Examples:

• Weissman et al. (2025) found that LLMs can and often do produce device-like clinical decision support even when instructed not to.

• LLM recommendations superseding professional judgment — for example, a study in which UK GPs switched from a correct to an incorrect prescription in 5.2% of cases where clinical decision support issued incorrect advice (Goddard et al., 2014).

• Lee et al. (2025), in a controlled simulation of 216 patient–LLM dialogues (108 injection, 108 control), found that prompt-injection attacks succeeded in 94.4% (102 of 108) of injection trials, including 91.7% of extremely high-harm scenarios, across six commercial LLMs.

Class 6 — Population Bias. The model produces responses that may be accurate in aggregate but inaccurate for specific subpopulations less well represented in training data.

• 6a: Demographic subgroup bias

• 6b: Pharmacovigilance/safety-signal bias

• 6c: Site/transferability bias

Examples:

• Larrazabal et al. (2020) found a “consistent decrease in performance for underrepresented genders when a minimum balance is not fulfilled.”

• Spontaneous-report systems under-report adverse drug reactions from populations with lower healthcare usage, meaning an LLM signal-detection tool trained on this data may infer false safety in those subgroups.

• Models trained on one site’s process data perform poorly when deployed at a second site in the network.

Dimension 2: Origination Point — Where It Went Wrong

Six error origins exist broadly: Training Data, Retrieval/RAG layer, Model Inference, Human–AI Interface, Agent Orchestration, and Supplier.

Origin 1 — Training Data. Training data is the furthest-upstream source of error for generative AI used in drug-development workflows. Any generative AI application built on a foundation model inherits provenance risk. This is more vulnerable than it may appear: replacing just 0.001% of training tokens with synthetic medical misinformation produced models that propagate harmful content significantly more often than baseline, while remaining indistinguishable from clean models on standard medical benchmarks (Alber et al., 2025).

• 1a: Data poisoning

• 1b: Data privacy

• 1c: Distribution shift

• 1d: Source error

Examples:

• Invented data points (e.g., dosage, manufacturing yield)

• Hallucinated adverse events not present in patient reports

• Memorization of PII/PHI

• Memorization of copyrighted literature

Origin 2 — Retrieval/RAG Layer. RAG is the dominant architectural pattern in regulated pharma generative AI because it enables grounding in version-controlled, GxP-aligned knowledge bases (protocols, SOPs, CSRs, labels, regulatory guidelines). This class of failures originates in the retrieval pipeline of RAG-based systems.

• 2a: Chunking failures

• 2b: Embedding drift

• 2c: Citation fabrication / grounded hallucination

• 2d: Multi-hop/complex query

• 2e: Retriever–generator misalignment

Examples:

• Allamraju et al. (2025) show that retrieval performance is highly domain-dependent: their learned semantic-chunking methods — Projected Similarity Chunking (PSC) and Metric Fusion Chunking (MFC), trained on PubMed data — substantially outperform naïve recursive and fixed-length splitting.

• Changing the embedding model, tokenizer, or preprocessing without re-embedding the entire corpus produces a vector store with mixed-provenance vectors and outputs that drift without a clear trace in the logs.

• RAG reduces but does not eliminate grounded-hallucination errors; fabricated citations occur when the model extrapolates beyond retrieved context.

• A query requiring multiple retrieval instances is more error-prone — for instance, “Which of our Phase 2 protocols use the same secondary endpoint as Study A, and what was their dropout rate?”

• A regulatory Q&A system retrieves the correct FDA guidance section but generates its response from parametric memory anyway.

Origin 3 — Model Inference. Failures at runtime even when training and retrieval are sound. These failure modes survive downstream of quality data and accurate retrieval.

• 3a: Hallucination (intrinsic vs. extrinsic)

• 3b: Sycophancy / prompt sensitivity

• 3c: Numerical errors

• 3d: Non-determinism

• 3e: Structured-output failures

• 3f: Prompt injection

Examples:

• Ji et al. (2023) formalized the intrinsic/extrinsic distinction: an intrinsic hallucination occurs when a model summarizing a clinical trial report states the control arm showed superiority when the source states the opposite; an extrinsic hallucination occurs when the same model introduces an adverse-event finding that appears nowhere in the source document.

• Perez et al. (2022) demonstrated that LLMs systematically shift their answers to align with the user’s implied opinion when questions contain leading framing, even when the model “knows” the correct answer under neutral prompting.

• Frieder et al. (2023) systematically evaluated GPT-4 on graduate-level mathematics and found consistent failures in multi-step arithmetic, symbolic manipulation, and modular arithmetic.

• At identical temperature settings and system prompts, repeated inference runs on the same clinical vignette can produce materially different outputs — a phenomenon documented in the behavioral-drift study of Chen, Zaharia, & Zou (2023).

• LLMs tasked with producing JSON, XML, or tabular output frequently generate syntactically invalid structures.

• Lee et al. (2025) found that prompt-injection attacks against six commercial LLMs succeeded in 94.4% (102 of 108) of injection trials within a controlled simulation of 216 patient–LLM dialogues.

Origin 4 — Human–AI Interface. This class of failures arises from how humans interact with the AI system.

• 4a: Automation bias

• 4b: Confidence miscommunication

• 4c: Deskilling / workflow integration

• 4d: Prompt design

Examples:

• Goddard, Roudsari, & Wyatt (2014) found that in 5.2% of prescribing cases, UK GPs switched from a correct to an incorrect answer after receiving clinical decision support advice.

• When an LLM output is rendered in a polished UI with citations formatted as hyperlinks, users may interpret the presentation as evidence of retrieval grounding; these limitations should be made explicit to end users.

• Over repeated use, operators who initially verified AI outputs begin to skip verification steps as trust accumulates.

• Weissman, Mankowitz, & Kanter (2025) demonstrated that two widely used LLMs readily produced device-like clinical decision support across a range of scenarios despite system-prompt instructions to remain compliant with FDA non-device guidance.

Origin 5 — Agent Orchestration. Failures originating in the orchestration layer of multi-agent or multi-step LLM systems — errors that arise when multiple agents, tools, or function calls compose into a failure that none of the individual components would have produced alone.

• 5a: Quality degradation across iterations

• 5b: Multi-step planning failures

• 5c: Compounding propagation (multiple errors interact across stages)

• 5d: Tool-use errors

Examples:

• In multi-agent document-generation pipelines, iterative revision loops can degrade output quality.

• The model sequences tasks in the wrong order, creating downstream dependency issues — for example, generating the statistical analysis plan before receiving protocol endpoints from another agent.

• In multi-hop reasoning chains, per-step accuracy declines significantly by the fourth or fifth step.

• An agent selects an inappropriate tool — for example, calling a pharmacovigilance database API with an incorrect MedDRA code.

Origin 6 — Supplier. Failures rooted in components provided by an upstream vendor — foundation models, vendor data, vendor model updates, SLA failures, and third-party platform changes. The ISPE GAMP AI Guide surfaces this concern explicitly with a supplier-quality planning section.

• 6a: Silent model updates

• 6b: Deprecation

• 6c: Multi-tenancy leakage

• 6d: DPA drift

Examples:

• Silent retraining (Chen, Zaharia, & Zou, 2023): comparing the March and June 2023 versions of GPT-4, USMLE accuracy decreased from 86.6% to 82.4%, underscoring the need for continuous monitoring.

• For a sponsor that validated a regulatory-submission workflow against a specific model endpoint, deprecation forces an unplanned revalidation; pharma organizations rely on predictability to plan change control.

• Training data can be extracted from language models through targeted prompting, raising concerns around patient-level data and proprietary parameters.

• When a foundation-model provider silently migrates inference infrastructure across regions — routing API calls through a new data center to manage capacity — data-residency and processing terms may shift without the regulated customer’s knowledge.

GAMP 5 (Second Edition) suggests Category 5 treatment for foundation models, given their potential for autonomous decision-making impact.

The Worked Example: Mata v. Avianca

Mata v. Avianca is not a pharma case, and that is why it is worth your time. In 2023, two New York attorneys submitted a legal brief containing six fabricated court decisions generated by ChatGPT, complete with fake quotes attributed to real federal judges. When one of the attorneys asked ChatGPT whether the cases were real, it confirmed they were. They were not.

The lawyers were sanctioned $5,000 (a joint-and-several Rule 11 sanction against the attorneys and the law firm collectively). The headlines focused on the embarrassment. The substantive lesson, for any regulated industry, is what the failure mode actually was — because “the AI hallucinated” does not tell you anything actionable.

Mapped through the taxonomy:

• Class 1 (Fabrication) at Origin 3 (Model Inference): the model generated case names and citations with no grounding in any retrieved source.

• A secondary failure mode is Class 1 × Origin 4 (Human–AI Interface): verification would have prevented the error from surfacing in the official brief.

Two different controls would have caught it. A retrieval-grounding requirement (a citation-grounding fix at the RAG layer) addresses Class 1 at Origin 3. A forced verification workflow with structured event logging (a Human–AI Interface fix) addresses Class 1 at Origin 4.

How This Fits Within Existing Standards

The taxonomy is not a parallel discipline competing with classical quality risk management. It is an extension of it into territory the foundational standards did not originally anticipate but whose principles still cover.

The FDA’s seven-step credibility assessment framework (draft, January 2025) defines the validation envelope within which any specific AI deployment lives. Context of use is established, model influence is characterized alongside decision consequence, and the resulting credibility plan is scaled to the risk tier. A high-influence deployment with high decision consequence demands the highest evidence tier — independent test sets, blinded human evaluation, statistically powered comparisons — while a lower-tier deployment requires proportionate but lighter evidence. The framework remains in draft as of June 2026, but it is sufficiently mature to adopt today; waiting for finalization is no longer defensible given the volume of AI entering regulated workflows.

The error taxonomy operates inside this envelope. The FDA framework tells you how much rigor a given deployment requires; the taxonomy tells you what kinds of failures that rigor must address. They are orthogonal and complementary. A higher-tier deployment requires explicit treatment of more cells in the matrix and deeper evidence for each; a lower-tier deployment may need explicit controls only for the highest-probability cells. The connection is operational: the FDA framework sets the evidentiary bar, and the taxonomy specifies what evidence is needed at each failure point.

ICH Q9(R1), effective July 2023, provides the risk-management spine within which both the FDA framework and the taxonomy sit. Its classical triad of severity, probability, and detectability maps directly onto this architecture. Severity is drawn from the FDA framework’s model-influence × decision-consequence classification. Probability is characterized per cell of the taxonomy through bounded-variance testing: the same input replayed N times to characterize the stochastic envelope, with failure rate estimated from the resulting distribution. Detectability is established through the continuous-monitoring layer, which determines whether a given failure type surfaces at the output or requires external validation to catch. ICH Q9 was written for deterministic systems; this framework extends its principles into probabilistic systems where reproducibility is bounded rather than absolute. The R1 revision’s explicit acknowledgment that risk-management decisions involve transparent subjectivity is directly relevant: bounded variance is a transparent acknowledgment of subjectivity in evaluation, and it earns its regulatory standing under R1.

GAMP 5 (Second Edition, 2022) provides the lifecycle and categorization layer. Probabilistic AI systems typically fall under GAMP Category 5 — custom applications — because they are neither off-the-shelf nor merely configured. The taxonomy does not replace this categorization; it sits inside Category 5 as the deeper decomposition GAMP 5 does not provide for probabilistic systems specifically. The Britt Probabilistic Validation Lifecycle that operationalizes the taxonomy maps structurally onto the GAMP 5 V-model: context of use corresponds to concept and planning; risk tiering to risk assessment; evaluation design to configuration and verification; acceptance criteria to qualification; HITL/HOTL controls to operational use; and continuous monitoring and change control to ongoing change management. Same skeleton, different muscle — adapted for the failure characteristics of probabilistic systems.

The ISPE GAMP AI Guide (July 2025) establishes the risk-management process for AI-enabled computerized systems but does not provide an explicit failure-mode framework for non-deterministic systems specifically. This taxonomy is intended to operate inside that space. The supplier-qualification work extends Chapter 7 and Appendix M2 with the four-pillar VALID Trust decomposition; the error taxonomy extends the failure-analysis discipline that GAMP 5 inherits from FMEA into the probabilistic systems that classical FMEA was not built to address. The framework is GAMP-aligned in its skeleton and FMEA-shaped in its decomposition, but its substantive content is new, because probabilistic systems break the reproducibility assumptions both of those frameworks were built on.

The structural familiarity is intentional. QA professionals should be able to use the taxonomy with the same operational discipline they bring to deviation investigation, root-cause analysis, and FMEA. The substantive divergence is what makes the work necessary: the matrix gives quality teams something to hold onto when the failure they are investigating does not have a stable identity — which, in probabilistic systems, is most of the time.

For non-deterministic AI systems, the state of control is demonstrated not by establishing and maintaining a fixed validated state, but by continuously verifying the absence of defined failure modes within an accepted tolerance. This shifts the validation-evidence model from point-in-time qualification to continued process verification — a concept already established in process validation (FDA 2011 guidance; EU Annex 15) but not yet extended to AI-enabled computerized systems. This taxonomy is meant to enable sponsors to build robust, defensible acceptance criteria for non-deterministic systems.

A Living Framework

This taxonomy is the initial version of a framework I expect to develop publicly over the next several years. The six origin categories are stable enough to be useful now; the cell-level mappings between origins and output types will need refinement as more case studies accumulate. I am particularly interested in contributions in three areas where the published literature is thinnest: retrieval-origin failures in regulatory-submission workflows, agent-orchestration failures in GxP manufacturing, and supplier-origin failures involving foundation-model updates. If you have encountered a failure that does not fit cleanly into this structure, or if you think the structure itself needs reconsidering, I want to hear about it. Refinements will be published with attribution.

References

Alber, D. A., Yang, Z., Alyakin, A., Kane, E., Rigney, I., Roytman, M., … Oermann, E. K. (2025). Medical large language models are vulnerable to data-poisoning attacks. Nature Medicine, 31(2), 618–626. https://doi.org/10.1038/s41591-024-03445-1

Allamraju, A., et al. (2025). Breaking it down: Domain-aware semantic segmentation for retrieval augmented generation. arXiv. https://doi.org/10.48550/arXiv.2512.00367

Blumenthal, R., Erdmann, N., Heitmann, M., Lemettinen, A.-L., & Stockton, B. (2024). Machine learning risk and control framework. Pharmaceutical Engineering, 44(1). ISPE.

Chen, L., Zaharia, M., & Zou, J. (2023). How is ChatGPT’s behavior changing over time? arXiv. https://doi.org/10.48550/arXiv.2307.09009

European Commission. (2015). EudraLex Volume 4, Annex 15: Qualification and validation. European Commission.

Frieder, S., Pinchetti, L., Griffiths, R.-R., Salvatori, T., Lukasiewicz, T., Petersen, P. C., Chevalier, A., & Berner, J. (2023). Mathematical capabilities of ChatGPT. arXiv. https://doi.org/10.48550/arXiv.2301.13867

Goddard, K., Roudsari, A., & Wyatt, J. C. (2014). Automation bias: Empirical results assessing influencing factors. International Journal of Medical Informatics, 83(5), 368–375. https://doi.org/10.1016/j.ijmedinf.2014.01.001

International Council for Harmonisation. (2023). ICH harmonised guideline: Quality risk management Q9(R1) (Step 4 version, 18 January 2023). ICH.

International Society for Pharmaceutical Engineering. (2022). GAMP 5: A risk-based approach to compliant GxP computerized systems (2nd ed.). ISPE.

International Society for Pharmaceutical Engineering. (2025). ISPE GAMP guide: Artificial intelligence. ISPE. https://doi.org/10.1002/9781946964854

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248. https://doi.org/10.1145/3571730

Jin, Q., Chen, F., Zhou, Y., Xu, Z., Cheung, J. M., Chen, R., … Lu, Z. (2024). Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine. npj Digital Medicine, 7, 190. https://doi.org/10.1038/s41746-024-01185-7

Larrazabal, A. J., Nieto, N., Peterson, V., Milone, D. H., & Ferrante, E. (2020). Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proceedings of the National Academy of Sciences, 117(23), 12592–12594. https://doi.org/10.1073/pnas.1919012117

Lee, R. W., Jun, T. J., Lee, J.-M., Cho, S. I., Park, H. J., & Suh, J. (2025). Vulnerability of large language models to prompt injection when providing medical advice. JAMA Network Open, 8(12), e2549963. https://doi.org/10.1001/jamanetworkopen.2025.49963

Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023) (No. 1:22-cv-01461 (PKC)).

Perez, E., Ringer, S., Lukošiūtė, K., Nguyen, K., Chen, E., Heiner, S., … Kaplan, J. (2022). Discovering language model behaviors with model-written evaluations. arXiv. https://doi.org/10.48550/arXiv.2212.09251

U.S. Food and Drug Administration. (2011). Process validation: General principles and practices (Guidance for industry). FDA.

U.S. Food and Drug Administration. (2025). Considerations for the use of artificial intelligence to support regulatory decision-making for drug and biological products (Draft guidance for industry). FDA.

Weissman, G. E., Mankowitz, T., & Kanter, G. P. (2025). Unregulated large language models produce medical device-like output. npj Digital Medicine, 8, 148. https://doi.org/10.1038/s41746-025-01544-y

Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., … Singh, K. (2021). External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Internal Medicine, 181(8), 1065–1070. https://doi.org/10.1001/jamainternmed.2021.2626