Probabilistic Failure Mode Taxonomy V2.0 — Britt Biocomputing
Architecture · Failure classification

The Probabilistic Failure Mode Taxonomy, V2.0

Probabilistic failures don't have stable identities, so classifying them by what went wrong is only half the picture. The taxonomy adds a second dimension — where in the system the failure first became observable — and, in V2.0, the logic for working through the investigation: what must be true before the matrix applies, how to tell an intrinsic failure from a locatable one, and how to record a failure that occupies more than one cell.

A Control Layer (L3) component of the House of AI Trust · feeds evaluation design & monitoring

V2.0 · November 2026 · What changed from V1.0 ↓
The logic at a glance

From baseline to observed failure to disposition

The taxonomy is more than the matrix. Two preconditions come before it, an operational test sits between intake and recording, and every recorded failure feeds trending, which in turn feeds the next version. Each step below links to the section that explains it.

A revision updates the classification guide. Baseline rates stay tied to the taxonomy version they were coded under.

Select any step to jump to its section.

Two orthogonal dimensions

Error type is the what. Origination point is the where.

The two axes are orthogonal: the same error type can arise at several origination points, and the same origination point can produce several error types. That's why the framework is a matrix of cells rather than a one-to-one list — and why it mirrors how quality teams already investigate deviations. The taxonomy's scope is limited to probabilistic-specific errors in a probabilistic system: the model × harness × human-in-the-loop or human-on-the-loop, within a defined context of use.

Dimension 1 · Error type

What went wrong

The error type is what you count. Baseline rates are set per error type and input class.

  1. Fabrication — citation, data-point, and entity fabrication
  2. Misinterpretation — recognition errors, reasoning errors, omission
  3. Contextual misapplication — population mismatch, temporal validity, spec non-compliance
  4. Confidence miscalibration — overconfidence, hedging, unflagged conflicts
  5. Boundary violation — scope creep, authority creep, adversarial breach
  6. Subgroup performance failure — accurate in aggregate, less accurate for subpopulations within the larger group: demographic subgroup bias, pharmacovigilance/safety-signal bias, and site/transferability bias
Dimension 2 · Origination point

Where it first became observable

The origination point is the first observable point: the earliest place in the execution trace where the failure can be attributed. It is not the root cause, and it is not necessarily where the control goes.

  1. Runtime data New in V2.0 — input-class shift, malformed or incomplete inputs, upstream source error, stale reference data
  2. Retrieval / RAG layer — chunking, embedding drift, grounded hallucination, multi-hop
  3. Model inference — ungrounded generation, sycophancy, non-determinism, prompt injection
  4. Human–AI interface — automation bias, confidence miscommunication, deskilling
  5. Agent orchestration — multi-step planning, compounding propagation, tool-use
  6. Supplier — silent model updates, deprecation, multi-tenancy leakage, DPA drift
Before the matrix applies

The data inheritance gate and the validated baseline

V1.0 listed training data as an origination point. A sponsor can rarely observe a failure there, and can almost never correct one there. V2.0 moves it upstream. Data defects that are inherited silently look like chance variation once the system is live; the gate is where they are caught, and the baseline is what every later investigation is compared against. The matrix and the operational test apply only to a system that has passed both.

Precondition 1

The data inheritance gate

  • What it covers — supplier attestations of training-data provenance (which can be inherited), and the fitness of your own data and retrieval corpus for the context of use (which must be owned)
  • Who does what — data governance produces the evidence, validation verifies and inherits it, and the business process owner owns the gate decision
  • How much — proportionate to the risk of the context of use
  • Regulatory anchor — step 4 of FDA's draft credibility framework
Precondition 2

The validated baseline

  • Baseline rates — the occurrence of each error type per input class, reported with an uncertainty bound
  • A classification guide — so the same failure is coded the same way at validation and at investigation
  • A limits table — alert and action levels per failure mode, tiered by severity
  • Execution-time capture — inputs, retrieved content, and the as-run configuration, recorded when the system runs
The 6 × 6 matrix

Same error class, different origination point, different fix

A fabrication is the same error class whether it first appears at model inference, in the retrieval layer, or at the human review point — but the corrective control is different in each case. In V2.0 one incident can occupy several cells: a primary cell at the first observable point, used for trending, and secondary cells for each contributing origination point. Two cases are worked that way below: Mata v. Avianca, and an AI-assisted batch record review. The fully populated 36-cell matrix is the client deliverable.

Runtime data Retrieval / RAG Model inference Human–AI interface Agent orchestration Supplier
Fabrication
Misinterpretation
Contextual misapplication
Confidence miscalibration
Boundary violation
Subgroup performance failure
Featured multi-cell case Worked example Part of the full client matrix
Mata v. Avianca · primary cell

Fabrication × Model inference

The canonical hallucination. In Mata v. Avianca (S.D.N.Y. 2023), the model fabricated six federal case citations in a single inference run — correct citation structure, invented substance — with no grounding in any retrieved source.

Corrective actionFabrication at inference can be reduced but not engineered out, so the control is scope restriction: the model is never the source of record for citations. Every citation it produces is checked automatically against an authoritative database before it reaches a reviewer, and anything that doesn't resolve is blocked.
Mata v. Avianca · secondary cell

Fabrication × Human–AI interface

The same case, one layer out. When the attorney asked whether the cases were real, the model confirmed they were — and no independent check stood between that answer and the filed brief.

Corrective actionForced verification workflow with structured event logging before sign-off.
New in V2.0 · multi-cell attribution worked end to end

AI-assisted batch record review: one incident, three cells, three dispositions

An agent reviews executed batch records by exception. For a multi-strength product it checks a process limit against a specification that is current and approved, but for the wrong strength. The result falls inside that limit, the record is not routed to QA as an exception, and QA approves it. The trace shows retrieval returned the specifications for both strengths with no strength filter. Replayed five times against the captured context, the failure recurs twice.

Primary cell · Contextual misapplication × Retrieval / RAGRetrieval is where the trace first went wrong, so this is the cell used for trending. Disposition: corrective action. Filter retrieval on the batch's product-strength metadata, then rerun the validation set for the input class and confirm the rate falls within the baseline interval.
Secondary cell · Contextual misapplication × Model inferenceGiven both specifications, the model sometimes applied the wrong one, which is why only two of five replays failed. This is the intrinsic part. Disposition: no corrective action; record it and trend it.
Secondary cell · Confidence miscalibration × Human–AI interfaceThe record reached the reviewer with nothing prompting a second look. Disposition: add a control. Show on the review screen which strength's limit was applied.
Compensating control · located somewhere elseA deterministic, rule-based check after generation confirms the limit applied matches the strength in the batch record header. It sits in the output-verification step, not in retrieval where the failure began.

The incident counts once, under contextual misapplication, for trending; all three cells are recorded for routing. Illustrative of a non-critical use with qualified human review: the consultation draft of EU GMP Annex 22 excludes LLMs and other probabilistic models from critical GMP use.

Worked example

Fabrication × Retrieval / RAG

RAG reduces but doesn't eliminate fabrication. Grounded hallucination occurs when the model extrapolates beyond the retrieved context — producing a citation the source doesn't actually support.

Corrective actionCitation-grounding checks that bind every claim to retrieved text.
Worked example

Misinterpretation × Model inference

Right data, wrong reasoning. Cabral et al. (2024) documented a model that correctly identified every relevant clinical finding from a vignette, then drew an incorrect inferential conclusion from it.

Corrective actionReasoning-chain audits and inferential checks, not just fact-presence tests.
Worked example · moved from Training data

Contextual misapplication × Model inference

Right-looking, wrong-context. A clinical LLM cites a withdrawn or superseded FDA guidance document because its knowledge predates the change. The cause sits in training data the sponsor cannot reach; the failure is first observable when the model answers from parametric memory instead of a current source.

Corrective actionTemporal-validity checks and currency guards against a maintained reference set, with every regulatory reference required to resolve to it.
Worked example

Contextual misapplication × Supplier

The model changed underneath a validated workflow. Between the March and June 2023 GPT-4 versions, USMLE accuracy fell from 86.6% to 82.1% with no announcement (Chen, Zaharia & Zou, 2023) — enough to break a workflow validated against the earlier endpoint.

Corrective actionContinuous monitoring, version pinning, and change control on supplier endpoints.
Worked example

Confidence miscalibration × Model inference

Maximum confidence on fabricated content. The Mata model didn't hedge — it asserted the invented cases were real, collapsing uncertainty exactly where it mattered most.

Corrective actionCalibration and uncertainty quantification; stated confidence must track accuracy.
Worked example

Boundary violation × Model inference

Guardrails that don't hold under pressure. Lee et al. (2025) found prompt-injection attacks succeeded in 94.4% of 216 controlled medical dialogues, including most high-harm trials.

Corrective actionAdversarial / red-team testing and input sanitisation at the inference boundary.
Worked example · moved from Training data

Subgroup performance failure × Model inference

Same clinical picture, different answer by demographic. Zack et al. (2024) found that GPT-4's differential diagnoses for standardized clinical vignettes were more likely to include diagnoses that stereotype certain races, ethnicities, and genders — a bias encoded in the model, invisible in aggregate accuracy, and observable only in what it generates.

Corrective actionCounterfactual subgroup testing: run the same inputs with only demographic attributes varied, and set acceptance criteria per subgroup, not just in aggregate.
Worked example · illustrative · new in V2.0

Subgroup performance failure × Runtime data

Validated on one population of inputs, used on another. A newly onboarded site submits records in a template that was not among the validated input classes. Aggregate accuracy holds; the error rate for that site's records does not.

Corrective actionInput-class gating: inputs outside the validated classes are detected and routed to manual handling until validation is extended to the new class.
The rest of the grid

Available in engagement

Every remaining intersection carries its own trace signature, confirming test, and default disposition — the fully populated 36-cell matrix, with coded per-cell logic, is delivered as part of a client engagement rather than published here.

36 cellsPer-cell failure mode, coded logic + control.
The operational test · new in V2.0

Intrinsic, locatable, or indeterminate?

A probabilistic system is stochastic by design, so some failures are intrinsic to it and cannot be corrected at a single point. Others come from a locatable, correctable defect in a component. Four questions, asked in order from cheapest and most deterministic to most expensive and most judgment-dependent, separate the two. The logic is adapted from the out-of-specification investigation framework: verify the result is real first, fix the replay count in an SOP in advance, and never let a passing replay invalidate the original result without an assignable cause.

Q0

Is the failure real, and is the evidence preserved?

Yes to both: go to Q1. Not real: this is not the right tool. Evidence not preserved: the priority is recovering what evidence is available.

Q1

Has the configuration been changed?

Configuration includes the model version, prompts, harness settings, index contents, tool definitions and thresholds, on the sponsor's side or the supplier's. Yes: replay the validated configuration. A change is a suspect, not a cause. If an anomaly appears, trace the earliest place it shows up to identify the primary origination point. If the validated configuration also produces the failure, go to Q2. No: go to Q2.

Q2

Under SOP-fixed replay conditions, is this input's failure rate elevated against the validated baseline?

Yes: the failure may be locatable. This sets severity and urgency. No: the failure is consistent with the validated rate for its input class, which alone does not establish it as intrinsic. Either way, go to Q3.

Q3

Does the execution trace show a step whose correction prevents the failure, in a harness component the sponsor or supplier controls?

Verified by counterfactual replay where feasible. Yes: locatable. No: intrinsic. No trace: indeterminate.

Outcome

Locatable

A corrective action is appropriate. Assign the origination point(s) from the matrix and route each to its control.

Outcome

Intrinsic

Monitoring is the appropriate response. Record the failure and trend it against the baseline and its limits.

Outcome

Indeterminate

A missing trace is itself a data integrity finding, and is treated as one before any disposition.

Recording the result · new in V2.0

Multi-cell attribution and the coding rules

The conceptual shift is from one root cause mapped to one control, to a primary origination point and any secondary ones, each mapped to its own downstream action. Where a failure appears is not necessarily where it gets fixed, and one failure can need several dispositions.

Attribution

Primary and secondary cells

  • Primary cell — the first observable point. It keeps that meaning and is the cell used for trending
  • Secondary cells — contributing origination points, each recorded with a required disposition
  • Counting — the incident counts once, under the primary error type; every cell is recorded for routing
  • Status — an origination point is a conclusion of investigation, recorded as suspected, confirmed, or unresolved
Disposition

Three downstream actions

  • Corresponding control — a control at the origination point itself
  • Compensating control — a control placed elsewhere in the system, where the failure can be caught
  • Accepted risk — with a documented justification
Coding rules

So two reviewers code alike

  • Origination rule — code the earliest point where the trace diverges, not where the failure became visible
  • Observation rule — code what the trace shows, not what you infer about why
  • Re-coding rule — the error type recorded at intake is provisional; correct it at RCA closure
  • Unclassifiable — don't force a fit. Use the code, and treat a rising count as a trigger to revise the taxonomy

Where this sits in the architecture

The taxonomy is a Control-Layer component of the House of AI Trust. It gives the Probabilistic Validation Lifecycle's evaluation-design step (03) a structured map of failure modes to test against, and gives the continuous-monitoring step (06) a vocabulary for classifying what it catches. The data inheritance gate connects it to Layer 1, where data governance lives.

License the Failure Mode Taxonomy

Want to run the full 6 × 6 matrix — coded logic, trace signature, confirming test and default disposition for every cell — in your own validation work? Book a licensing conversation.

Revision history

Versioned & under change control

Baseline rates are tied to the taxonomy version used to code them. Records coded to a V1.0 Training data cell should be re-coded under V2.0 before they are trended alongside new records.

VersionDateChanges
2.0 Nov 2026
  1. Origination point 1 changed from Training data to Runtime data. Training-data provenance moves upstream of the matrix, into the data inheritance gate.
  2. Origination point defined as the first observable point in the execution trace, distinguished from the root cause and from the location of the control, and recorded as suspected, confirmed, or unresolved.
  3. Two preconditions added: the data inheritance gate and the validated baseline (rates per error type and input class, a classification guide, a limits table, and execution-time capture).
  4. Operational test added (Q0–Q3), with three outcomes: locatable, intrinsic, and indeterminate.
  5. Multi-cell attribution added: a primary cell for trending, secondary cells for routing, three dispositions, and a rule that an incident counts once.
  6. Coding rules added.
  7. Worked examples: the two formerly at Training data moved to Model inference; Mata v. Avianca restated as primary and secondary cells; the batch record review and a Runtime data example added.
  8. Scope stated: probabilistic-specific errors in a probabilistic system.
1.0, amended Oct 2026 Error class 6 renamed Subgroup performance failure (previously Population Bias). No structural change.
1.0 May 2026 Initial publication: six error types × six origination points, with worked examples.
Cite this work

Versioned & citable

The taxonomy is published as a preprint with a permanent DOI, currently under peer review — citable in submissions, papers, and validation documentation.

Britt, K. (2026). Establishing AI State of Control: An Error Taxonomy and Assurance Framework for Transformer-Based Probabilistic Systems in Regulated Life Sciences [Preprint]. Research Square. https://doi.org/10.21203/rs.3.rs-10476545/v1

The preprint describes V1.0: it lists Training data as the first origination point, and class 6 appears as "Population Bias" (since renamed Subgroup Performance Failure). To cite V2.0, cite this page with its version number.

License the Probabilistic Failure Mode Taxonomy

The 6×6 taxonomy (error type × origination point) is open under CC BY 4.0. Use it freely, with attribution.

Licensing covers the material that puts it into practice:

  • Worked examples for each cell
  • Classification coding rules
  • SME criticality capture, so later calls stay consistent
  • Training for teams applying the taxonomy

Licenses are scoped to how you’ll use the taxonomy: in your own deviation and CAPA work, in client engagements, or built into your product. Licensing doesn’t imply endorsement or certification.

Found a failure mode or worked example that should be added or corrected? Send a suggested revision.