Latent
Start free

RAG hallucination detection

Catch RAG answers your sources don't support.

Retrieval puts the right documents in front of a model, and the model can still write claims they don't contain. Latent reads the model with your sources in context, scores every answer in milliseconds and marks the sentence that isn't supported.

Start free Book a call
Flags that are real failures96 in 100
Failures found on finance filings, against random review4.2×
Time to check an answer33 ms
One calibration run on your documentsAbout $7

Models with the right documents still write claims the documents don't contain

In RAGTruth, a public benchmark of RAG answers labeled by people, 46% of the test answers from six models, GPT-4 among them, contain at least one claim their sources contradict or never make. The wrong claim reads as grounded, because the right documents were in the prompt.

Latent reads the model with your sources in context

Context
Pass the retrieved passages with the answer, or let the SDK capture each call as your app makes it. Latent scores the answer with what the model was given.
Read
With an open-weight model on vLLM, Latent reads your model as it writes. With a model behind an API, Latent's reader model re-reads the finished prompt and answer.
Sentence
Each sentence gets its own risk and its position in the answer, so your interface or your reviewer can mark the exact claim that isn't supported.
Act
Deliver the answer and log it, hold it with your fallback text, or ask a stronger model. LLM guardrails covers each action.

Latent finds RAG failures at 2.7× to 4.2× the rate of random review

Every number below was read on answers held out of the fit. The benchmarks page lists each one with its data, sample and labels.

96%

of the answers Latent flags are real failures, when it flags 10% of other models' RAG answers.

RAGTruth held-out set: 3,498 answers to 583 prompts from GPT-4, GPT-3.5, Llama 2 7B, 13B and 70B, and Mistral 7B. Human span labels. Read by Latent's reader, Llama 3.1 8B Instruct. September 2026.
2.7×

as many real failures found as random review at the same 10% review budget, after calibrating on 10,000 human-labeled RAG answers.

RAGTruth, 2,700 held-out answers, split by prompt. Human span labels. September 2026.
4.2×

the failure rate of random review in Latent's top 10% on finance filings, after one calibration run on 1,000 of the model's own answers. Detection rose from AUROC 0.586 to 0.851, in 453 seconds for $6.88 of judge calls.

FinQA 10-K excerpts: 187 held-out answers about 24 companies the fit never saw. Labels from a strict Claude Opus judge. Llama 3.1 8B Instruct on vLLM. September 2026.
33 ms

median time for Latent's reader to check an answer on the same machine, and 150 to 170 ms over a network. Claude Opus as a judge took 3.1 to 3.4 seconds.

1,498 RAGTruth answers, every checker given the source document. September 2026. How we measured

One calibration run fits Latent to your documents

A judge model labels about 1,000 of your model's own answers against their sources, and Latent fits its read and its cut to them. A run takes five to eight minutes and $6 to $7.50 of judge calls, and the new fit goes live without a restart. We have run it on finance filings, health questions and legal contracts. When Latent hosts it, calibration runs for you.

Check an answer against its sources in three lines

Install the Python SDK. runlatent.auto_instrument() scores every answer from your OpenAI, Anthropic, Gemini and Ollama clients, or check one answer with its retrieved passages before you return it:

pip install runlatent
export LATENT_API_URL=https://app.runlatent.ai
export LATENT_API_KEY=lk_...
import runlatent

r = runlatent.check(question, answer, context=retrieved_passages, detail=True)
if r.flagged:
    answer = FALLBACK_TEXT
for s in r.sentences or ():
    print(s.risk, s.text)  # each sentence's risk, at answer[s.start:s.end]

The quickstart walks through setup and the first scored answer.

Questions

What is a RAG hallucination?
An answer from a retrieval-augmented system that states something its retrieved sources contradict or never say. It is hard to spot because the answer cites the right documents.
How is this different from asking a second model to check the answer?
A judge model re-reads the sources and the answer and writes a verdict, which takes seconds and a model call per answer. Latent reads the model that wrote the answer in milliseconds, so it can check every answer and send a judge only the risky ones.
Does it work with my retriever and vector database?
Yes. Latent works at the model call, so the retriever, vector database and chunking stay as they are.
Does it work with GPT, Claude and Gemini?
Yes. For models behind an API, Latent's reader model re-reads each finished answer. For open-weight models on vLLM, Latent reads your own model as it writes.

Research