Benchmarks · measured 18 to 28 September 2026
Hallucination detection, calibration on new workloads, cost against an LLM judge and serving overhead. Each entry names its data, its sample, who labeled it and the date, and the full list downloads as CSV.
Want the full detail? The complete register, with every condition and experiment, is part of the evidence pack we share during an Enterprise engagement. Ask for it.
How well Latent separates failed answers from good ones on other models' answers read by its reader model.
When Latent audits other models' RAG answers and flags 10% of them, 96% of its flags are real failures; those flags catch 21% of all failures, where 46% of answers fail.
Calibrated on 10,000 human-labeled RAG answers, the probe finds 2.7 times as many real failures as random review at the same 10% budget.
What one calibration run changes on a new kind of traffic, what it costs and how fast it goes live.
After calibration on a finance workload, reviewers working Latent's top 10% find failures at 4.2 times the rate of random review.
latent calibrate artifact, Llama-3.1-8B-Instruct via vLLM 0.19.1On finance filings, one latent calibrate run on 1,000 of the model's own answers lifts detection from AUROC 0.586 to 0.851, in 453 seconds for $6.88 of judge calls.
One latent calibrate run labels about 1,000 answers and fits a new artifact in five to eight minutes, for $6.22 (batch mode, half price) to $7.46 of judge calls.
A newly calibrated artifact goes live at the plugin's next policy check, every 30 seconds by default, with no restart; in our run both swaps took effect within six seconds.
Latent against a frontier judge model reading every answer, on the same human-labeled answers, and the two working together.
Judging every answer with Claude Opus 5 at its September 2026 list price costs $10.61 per 1,000 requests and 3.1 seconds per call at the median; the sidecar probe reads an answer in 33 ms for $0.02 per 1,000.
Let the probe pick the riskiest 10% and have Opus judge only those: precision 0.927 for $1.06 per 1,000 requests, against 0.818 for $10.61 when Opus judges everything.
What scoring every answer costs a vLLM server, and whether the stack keeps every record through load, restarts and artifact swaps.
With Latent reading every answer, vLLM serves 14.56 requests per second against 14.66 without it, and first tokens arrive in 298 ms against 316 ms, a difference inside run-to-run noise.
The plugin adds 16 MiB of GPU memory at idle and at most 58 MiB at peak, whatever the prompt length.
Over an 8-hour soak of 28,800 requests, every request was stored at every layer of the stack, with zero drops.
Through service, gateway and engine restarts and a live artifact swap under traffic, every one of 10,654 scored events was accounted for; two lost in flight at a gateway restart were recovered from the local audit file.
Each audit event is 1.7 KB in the log and carries hashes of the prompt and answer, never the text.
The drift alert stays silent on steady traffic and fires within 120 to 181 answers when a shift breaks the cut.
Other engines, a smaller reader, and the models and GPUs every number was measured on.
Latent runs beside SGLang with zero changes to SGLang, scoring every finished answer through the sidecar with the same verdicts as the vLLM plugin.
Latent is measured in serving on Llama-3.1-8B-Instruct and Hermes-3-Llama-3.1-8B, with Llama-3.2-3B as a small sidecar reader.
Every serving number here was measured on NVIDIA A100 40 GB or H100 GPUs, and two 8B models run side by side on one A100 40 GB.