Reads the model
A probe reads the activations behind each answer. On your own model in vLLM, Latent reads it word by word. For a model behind an API, Latent's reader model re-reads the finished answer.
How Latent detects hallucinationsCatch wrong LLM answers before your users do.
Latent reads model internals to flag unreliable answers at runtime.
Backed by Y Combinator
Watch Latent catch
a made-up answer.
As your model writes each word, Latent reads what is happening inside it. When the answer is finished, it sends it where your policy says. Pick an answer and a policy, and watch it run.
Pick an answer
When Latent flags it
Customer asks
Latent is reading·
Your model
The answer
Customer sees
Your stronger model
Standing by
Review queue
Nothing waiting
Built for LLM reliability in production.
Latent is a hallucination detection layer for LLM apps. It reads the activations your model computes as it writes, scores every answer, and acts on the risky ones before your users see them.
Scores every answer
Each answer gets a risk score in milliseconds, with the riskiest sentence marked. Latent calibrates its cut on your own traffic.
BenchmarksActs on the risky ones
Hold a flagged answer and send your fallback text, ask a stronger model, or deliver it and log it for review.
LLM guardrailsLabels every answer
Latent is the only scalable way to label every answer your model gives, so your training data grows with your traffic. Export it as fine-tuning or preference data for your model.
How it improvesLatent checks an answer in milliseconds. A judge model takes seconds.
We compared Claude Opus as a judge with Latent's reader, each given the source document. Latent reads activations and writes no verdict, so it keeps up with every answer you serve.
Answers checked in the same time176× faster
A judge model1 checked
Latent, API models176 checked
Latent, your own modelas it writes
On your own model, Latent reads each answer while it is written, so the check adds no measurable time.
Median per answer: a judge model 4.4 s, Latent about 25 ms.
Cost per 1,000 answers660× cheaper
A judge model$10.61
Latent, API models$0.016
Latent, your own modelno extra cost
Claude Opus at its September 2026 list price. Latent's reader on one A100 you run.
Want a judge anyway?
Let Latent send only its riskiest 10% to the judge. On the same human-labeled answers, the judge bill falls tenfold and more of what gets flagged is really wrong.
10×smaller judge bill
$1.06 against $10.61 per 1,000
$1.06 against $10.61 per 1,000
93%of flags correct
against 82% judging everything
against 82% judging everything
Judge routing measured on 1,498 human-labeled RAGTruth answers, September 2026. How we measured
A layer the rest of your stack can't see.
Judges, guardrails and dashboards look at what your model said. Latent reads the model while it writes, and it works alongside all of them if your team wants to keep them.
Works with what you already run
Latent sits at runtime, between the input and the answer, where the rest of your stack can't see. Keep your judge, guardrails and traces on the output. Latent feeds them through a Prometheus endpoint, OpenTelemetry trace ids and labeled answers for fine-tuning.
See why it flagged
Each flag points to the sentence and the words that drove it, the sentence in your source the answer leaned on most, and the kind of failure, such as an unsupported number, a wrong date or an invented name. Your reviewers know where to look.
Alerts where your team already works
Slack, PagerDuty, email or a webhook, when flags spike or stop, confirmed accuracy drops, scoring slows, the review queue backs up or your traffic drifts.
From one line of code to a working review queue.
- 01Add the SDK
One line at startup sends each finished answer from your OpenAI, Anthropic, Gemini or Ollama calls to Latent. Your calls go to your provider exactly as before.
Rather not install a package? An OpenAI-compatible endpoint is coming.
You're on the list. We'll email you when the endpoint is ready.
- 02Collect your first answers
Latent scores every answer from the first call, and keeps a sample of what your model actually served for calibration.
- 03Calibrate on your traffic
Latent runs calibration for you. A judge labels the sample against your sources, Latent tunes its threshold and its read to your traffic, and the update goes live with nothing to do on your side.
- 04Review what it flags
Flagged answers wait in a queue with the words that gave them away. Your reviewers confirm the failure or clear it, and each decision is kept as a labeled example.
pip install runlatent ✔ Successfully installed runlatent-0.3.7export LATENT_API_URL=https://app.runlatent.aiexport LATENT_API_KEY=lk_••••••••••••# app.py, once at startupimport runlatentrunlatent.auto_instrument()# after the first model callrunlatent.flush(); print(runlatent.stats()){"enabled": True, "captured": 1, "posted": 1, "failed": 0, ...}
Off-threadposts after each call returns, so no network round trip on your request path
Fail-openSDK errors are counted in
stats() and never raised into your appZero depspure Python, covering sync, async and streaming calls
Sample from a finance assistant1,000 of 1,000 answers
Latent keeps this sample's text for calibration under your retention settings.
calibration run · 1,000 labeled answers · run by Latent
96%of the answers Latent flags are really wrong, checking other models' answers
Checked at random46%
Flagged by Latent96%
Under an hourfor the whole run
Judge includedno key of yours needed
Nothingto do on your side to go live
Latent's reading, word by word
Every review makes it better.
Every answer Latent scores is kept as a labeled example, and each call your reviewers make sharpens it. The more traffic you serve, the more data you post-train your model on.
FlaggedLatent holds an answer
ReviewedConfirmed or released
LearnedKept as a labeled example
Flag accuracyHow often a flagged answer is really wrong
At install
Training data for your modelValidated answers, ready to export
Recalibrating live, no restart
Latent recalibrates on your reviewers' decisions while it runs, with no restart. Every validated answer can also be exported as fine-tuning or preference data for your model.
In your VPC, or hosted by us.
Security and deployment detail
Hosted by Latent
Your model serves exactly as it does today. After each call returns, the SDK sends the prompt and answer to Latent's reader, which scores every token of the finished answer. In your VPC, Latent reads your model itself, word by word.
Your appwith the runlatent SDK
Latent readerscores each finished answer
Review consolequeue, alerts and history
Your model
Serves as it does today.
Latent
Keeps what your retention settings allow.
Hosted by Latent
Your calls go to your provider as before. After each one returns, the SDK sends the prompt and answer to Latent's reader, which scores every token of the finished answer.
Your appwith the runlatent SDK
Latent readerscores each finished answer
Review consolequeue, alerts and history
Your model provider
Sees what it sees today.
Latent
Keeps what your retention settings allow.
Your VPC or your own hardware
On the GPUs that already serve your model. Latent reads the model itself, word by word.
Your app
Your inference enginewith the Latent plugin
Review servicequeue and audit log
Latent
When Latent runs in your environment, nothing leaves it: prompts, answers and activations stay on your infrastructure.
Your VPC or your own hardware
The reader runs on one GPU of yours and checks every finished answer.
Your app
Latent gatewayin front of your API calls
Latent readera small open model
Review servicequeue and audit log
Your model provider
Sees what it sees today.
Latent
Receives nothing.
You decide what leaves
In your VPC, nothing leaves it, and the only outbound calls are ones you configure yourself: the model download, an alert webhook, a stronger model you name. Hosted by Latent, prompts and answers are sent to Latent's cloud for scoring, under the retention and redaction settings you choose.
Your weights stay yours
Latent never receives or copies model weights, and calibration does not need them.
A record of every answer
Each scored request writes a 1.7 KB append-only record with hashes of the prompt and answer, never the text.
Questions teams ask first.
Anything else, ask us. Get in touch
We call our model through an API. What do we need?
The runlatent Python SDK and an API key, when Latent hosts it. In your VPC, one GPU for Latent's reader model, the smallest of which fits in about 10 GB. Either way, the reader scores each finished answer and returns its verdict in a fraction of a second. Choose Hold in the console, and the SDK in your app holds a flagged answer before your user sees it, replacing it with your text or a stronger model's answer. A streamed answer is cut at the first unsupported sentence. It covers OpenAI, Anthropic, Gemini and Ollama clients; with any other provider, one call in your code does the same.
Which models does it work with?
Any model, including closed-weight models behind an API, which Latent checks with its own reader model. With open-weight models served on vLLM in your VPC, Latent reads your own model's activations per token as it writes, and can act on an answer mid-stream.
What hardware does it need?
None when Latent hosts it. In your VPC, it runs on the GPU already serving your model: the plugin adds 16 MiB of GPU memory at idle and at most 58 MiB at peak, whatever the prompt length. Every serving number on this site was measured on NVIDIA A100 40 GB or H100 GPUs.
Does it change what my customers see?
Only when you tell it to. Out of the box Latent scores and records every answer. To hold a flagged answer, choose Hold in the console, and the SDK in your app replaces it with your text or a stronger model's answer. In your VPC, Latent's gateway in front of your model does the same. You can also act on the verdict in your code.
How good is it at finding wrong answers?
Reviewers working from Latent's flags find 2.7 to 4.2 times more wrong answers than checking at random, and when it checks other models' answers, 96 in 100 of the answers it flags are really wrong. Calibrating on your own traffic is what moves you to the top of that range.
How long does calibration take?
When Latent hosts it, Latent runs calibration for you, judge included. A run takes under an hour for a few hundred to a thousand answers, and the update goes live with nothing to do on your side. In your VPC, one run over about 1,000 answers takes five to eight minutes and $6 to $8 of judge calls on your own key, and goes live at the plugin's next policy check, within 30 seconds by default, with no restart.
Can it run air-gapped?
Yes. After the model download on first start you can block all egress from the host, and the stack keeps running.
What happens when our traffic changes?
A drift alert watches the scores. It stays silent on steady traffic and fires within 120 to 181 answers when a shift matters, so you recalibrate when you need to.