LLM hallucination detection from inside the model.

How Latent detects hallucinations

A language model computes layers of activations for every word it writes, and those internal states carry a signal of whether the answer is grounded. Latent reads that signal and turns it into a risk score for every answer. Why internal state covers the research behind it.

Read

A probe, a small classifier fit on labeled answers, reads the activations behind each answer. With an open-weight model on vLLM in your environment, Latent reads your model word by word as it writes. With a model behind an API, Latent's reader model re-reads the finished prompt and answer, and Latent scores the reader's activations.

Score

Each answer gets a risk score and a verdict against a cut set for your review budget. The riskiest sentence and the words that drove the score are marked, so a reviewer knows where to look.

Act

Deliver the answer and log it, hold it and send your fallback text, or ask a stronger model. A streamed answer can be cut at the first unsupported sentence. LLM guardrails covers each action.

Calibrate

A judge labels a sample of your own answers against your sources, and Latent tunes its read and its cut to them. When Latent hosts it, calibration runs for you.

Benchmarks

Every detection number below was read on answers held out of the fit. Each one names its data, its sample and who labeled it. The benchmarks page lists every number we quote, with a CSV.

Latent and a judge model

A judge model reads each finished answer and writes out a verdict, which takes seconds and a frontier-model call for every answer. Latent reads activations and writes no text, so it can check every answer you serve.

Teams that keep a judge can send it only Latent's riskiest 10%. On the same 1,498 human-labeled RAGTruth answers, that cut the judge bill from $10.61 to $1.06 per 1,000 answers, and the share of flags that were really wrong rose from 82% to 93%. Latent also works alongside the guardrails and tracing you already run. Read the full comparison or see Latent as an LLM-as-a-judge alternative.

Use cases

Latent fits wherever a wrong answer costs more than a slow one. We have calibrated it on finance filings, medical questions and legal contracts.

Integration

Install the Python SDK and add one line at startup. Latent scores every answer from your OpenAI, Anthropic, Gemini and Ollama clients, and your calls go to your provider exactly as before.

pip install runlatent
export LATENT_API_URL=https://app.runlatent.ai
export LATENT_API_KEY=lk_...

import runlatent
from openai import OpenAI

runlatent.auto_instrument()  # once at startup

client = OpenAI()
answer = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "What is our refund window?"}],
)
# Latent scores the answer off-thread; flagged answers wait in your review queue

The quickstart walks through setup and the first scored answer. Serving your own model on vLLM? The Latent plugin reads it inside the engine: see hallucination detection in vLLM.

Questions

What is LLM hallucination detection?

Checking a language model's answers for claims that are made up or unsupported by the sources it was given, so they can be caught before they reach users. Latent does it by reading the model's internal activations as well as the text.

Does it work with GPT, Claude and Gemini?

Yes. For models behind an API, Latent's reader model re-reads each finished answer. For open-weight models on vLLM, Latent reads your own model as it writes.

Does it slow my app down?

No. The SDK sends each answer off-thread after the call returns, so your requests wait on Latent only when you choose to hold flagged answers. Inside vLLM, the nightly benchmark shows no measurable overhead.

Where does my data go?

Hosted by Latent, prompts and answers are scored in the United States under the retention you set. In your VPC, nothing leaves it. Security

How do I start?

Sign up free and add one line of Python. Pricing lists the plans.

Research

• Why internal state: four published results on what a model's activations reveal that its output does not.
• Latent against an LLM judge: the time and cost of checking every answer, measured.
• Latent catches two-neuron evil Qwen before it writes a word: 245 of 245 steered prompts flagged from the model's state.
• We rebuilt the OpenAI x Hugging Face incident and stopped it before it happened: 99% of cheating agents stopped before the command ran.