Catch a model’s confident mistakes before they ship.
Latent reads your model’s internal activations while it generates and scores whether the answer is about to be wrong. High risk gets interrupted, escalated, or replaced. For teams that hold their own weights or run their own inference.
Real calibration run on public filings. No signup, nothing to install. Security and deployment detail →
Replay of a real held-out generation from the shipped trace set, shaded by the per-token risk the probe assigned. Drag the threshold control below to change which tokens escalate.
A fluent wrong answer leaves no trace in the text.
Output-level checking asks a second model, a search index, or a person to re-read what was already written. That works when the mistake is visible on the page. It fails on the case that costs you money: an answer that reads correctly, asserts nothing obviously false, and is wrong.
It also fails structurally when the field needed to verify an answer is the field you are required to strip before sending it anywhere. Once the patient age or the account identifier is redacted, the checker no longer holds the thing that decides whether the answer was right. Nobody notices, because the checker still returns a confident pass.
The model’s own internal state carries that signal before the first token is emitted. Latent reads it where it already exists, inside your serving process.
One line into vLLM, then a control you can actually staff.
Install
Registers through vLLM’s standard general_plugins entry point. No fork, no
patched engine, no change to how you serve. A read-only forward hook on the scored layer.
Calibrate
Fits on your model and your traffic, on your hardware. Selects the layer, the probe, and a threshold matched to the review budget you can actually staff. The artifact stays local.
Operate
Risk-ranked escalation queue, token-level traces, policy tied to review capacity, and an append-only audit record. Reviewer decisions feed the next recalibration.
Probes are per model and per domain. A fine-tune, a version bump, or a shift in traffic changes what the correct escalation threshold is.
What your team operates.
The escalation queue, ranked by risk. These are real rows from the finance calibration run published in the calibration report: real questions, real scores, real verdicts. 65 of 173 held-out requests crossed the threshold.
Inside the forward pass, inside your VPC.
Output-level checking reads the finished text, which is why it goes blind the moment the field it needs is redacted. Latent reads the layer while the answer is still being written, and routes on what it finds. Nothing crosses the boundary.
Every scored request reconstructs.
An action log says a decision happened. A model risk examiner asks whether it can be rebuilt: which probe version scored this request, at which threshold, against which artifact, and which spans of the input produced the tokens that got flagged. Two layers, and they are at different stages.
How a flagged claim was assembled: the input spans that fed each dictionary feature, and the output tokens those features wrote. Hover a node.
Share of attribution mass entering the probe read, by residual-stream band.
One append-only entry per scored request, written locally. Prompt and output are stored as digests by default, so the record is reconstructible without retaining text.
The audit record ships today and is what an SR 11-7 reviewer asks for. The attribution engine is the layer above it, built on sparse dictionary features, activation oracles and natural-language explanations of what a feature fires on.
The failure mode is documented, and internal state is where it shows up.
The labs publishing this work read internal state to find these failures, and that capability sits behind their APIs. It never reaches a team serving a fine-tuned 8B in its own VPC. We fill this gap.
Measured, held out, published in full.
Every number is reproducible from the calibration report on this site. Self-generated answers, judge-labeled, company-disjoint splits, no teacher forcing.
| Measurement | Result | What it means |
|---|---|---|
| Catch rate at a 10% review budget | 3.5x | Against 10% for random sampling of the same traffic. |
| Label quality, judge against gold | 94.7% | Balanced accuracy of the labeller against 150 HaluEval gold pairs, zero failed calls. The labels the probe is fit on are validated, not assumed. |
| Discrimination, held out | 0.916 | AUROC, replicated across five folds on a Llama-3.1-8B deployment. |
| Replication at 70B | 0.92 | Llama-3.3-70B, no hand-tuning, layer chosen by the same automated procedure. |
| Added latency, every request | 1–2% | Measured under continuous batching, not extrapolated from a single-request benchmark. |
Inside your environment, and only there.
Teams that hold the weights and run the serving process.
AI vendors deploying into customer environments
Your contract puts the model inside the customer's environment with nothing reporting back to you, so a wrong answer is either caught by someone reading outputs one at a time, or it reaches the customer and you answer for it.
Regulated enterprises running fine-tunes
Banks, insurers, and health systems running their own models on internal data, where a reviewer queue is a headcount line and an audit record is a supervisory expectation.
Air-gapped and classified deployments
Environments where a managed API is not an option at all, and every control has to run inside the boundary with a record that survives an audit.
Start with a scoped pilot.
One model, one review workflow, calibrated on your representative traffic. Runs on a sandbox model and never touches production, which is scoped to clear as an evaluation rather than a full third-party risk assessment.