LLM guardrails that stop wrong answers before users see them.

Latent guards each answer from inside the model

Most guardrails read text. Rules and classifiers match patterns in the prompt or the answer, and a judge model reads the finished answer and writes a verdict. Latent reads the activations behind each answer, where the signal of an unsupported claim is present before any text is written, and turns them into one risk score per answer.

Read

With an open-weight model on vLLM, Latent reads your model as it writes. With a model behind an API, Latent's reader model re-reads the finished prompt and answer.

Score

Each answer gets a risk score and a verdict against a cut you set. The console shows what each cut means on your traffic: the share flagged, how many flags are right and how many failures are caught.

Act

Your policy decides what happens to each flagged answer, for a whole project or for one model.

Record

Every scored answer, flagged or not, goes to your audit log with its score and the decision taken.

Your policy decides what happens to a risky answer

Set it once on the console's Policy page. The default delivers the answer and queues the flag for review. The others change what your user sees.

Act on a flagged answer in the docs covers each action and where it runs.

Latent catches real failures in milliseconds

Every number below was read on answers held out of the fit. The benchmarks page lists each one with its data, sample and labels.

When a score is late, a hold policy holds the answer too

While your policy holds flagged answers, an answer whose score does not arrive in time is held as well, with your fallback text, because releasing an unchecked answer is the failure the policy exists to prevent. Under the other policies it is delivered and marked unknown. Each of these decisions is recorded.

Latent runs beside the guardrails you already have

Filters for personal data, unsafe topics and prompt injection look for known patterns, and they do that job well. Latent covers the failure a pattern cannot describe: an answer that reads well and is wrong. Keep your filters on the same request path and add Latent for correctness.

Add it in one line, then pick a policy

Install the Python SDK and turn it on at startup. Choose Hold on the console's Policy page, and the SDK holds flagged answers in your app before it returns them.

pip install runlatent
export LATENT_API_URL=https://app.runlatent.ai
export LATENT_API_KEY=lk_...

import runlatent
from openai import OpenAI

runlatent.auto_instrument()  # once at startup

client = OpenAI()
answer = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "What is our refund window?"}],
)
# with Hold on, a flagged answer comes back as your fallback text

To decide in your own code, runlatent.check() scores one answer and returns its verdict before you deliver it:

r = runlatent.check(prompt, answer, context=sources)
if r.flagged:
    answer = FALLBACK_TEXT

The quickstart walks through setup and the first scored answer.

Questions

What are LLM guardrails?

Checks around a language model that decide whether a prompt or an answer may pass. Common guardrails filter personal data, block unsafe topics, enforce output formats or ask a second model to judge the answer. Latent is a guardrail for correctness: it decides whether an answer is likely wrong.

Can guardrails stop hallucinations?

Rules only catch the errors someone wrote a rule for, and a judge model costs a model call and seconds on every answer. Latent reads the model itself, so it scores every answer in milliseconds and can hold the risky ones before they ship. How Latent detects hallucinations

Does it work with GPT, Claude and Gemini?

Yes. For models behind an API, Latent's reader model re-reads each finished answer, and the SDK holds or reroutes it in your app. Stopping an answer mid-generation needs an open-weight model on vLLM.

Does holding answers add latency?

Holding an answer waits for its score, about 50 ms on a single GPU and more under load. Without a hold policy, scoring runs off-thread and your app waits on nothing.

What about streamed answers?

Latent can cut a stream at the first unsupported sentence. Sentences already shown stay on screen, so to hold a streamed answer completely, buffer it or turn off streaming for that route.

Research

• LLM hallucination detection: methods compared: judges, NLI models, sampling, token probabilities and internal-state probes, with what each needs and costs.
• Latent against an LLM judge: the time and cost of checking every answer, measured.
• We rebuilt the OpenAI x Hugging Face incident and stopped it before it happened: a guardrail for agents that holds a tool call before it runs.