Latent reads your model's internal state as it writes. Made-up answers are held before a customer sees them.
Backed by Y Combinator
As your model writes each word, Latent reads what is happening inside it. When the answer is finished, it sends it where your policy says. Pick an answer and a policy, and watch it run.
You can. It is slow, it is expensive, and every answer has to leave your environment to be checked. Latent reads the model that wrote the answer, so it checks output validity in real time.
On your own model, Latent reads each answer while it is written, so the check adds no measurable time.
Let Latent choose which answers the judge reads. The judge bill falls tenfold, and more of what gets flagged is really wrong.
Judges, guardrails and dashboards look at what your model said. Latent reads the model while it writes, and it works alongside all of them if your team wants to keep them.
Latent sits at runtime, between the input and the answer, where the rest of your stack can't see. Keep your judge, guardrails and traces on the output. Latent feeds them through a Prometheus endpoint, OpenTelemetry trace ids and labeled answers for fine-tuning.
Each flag points to the sentence and the words that drove it, the sentence in your source the answer leaned on most, and the kind of failure, such as an unsupported number, a wrong date or an invented name. Your reviewers know where to look.
Slack, PagerDuty, email or a webhook, when flags spike or stop, confirmed accuracy drops, scoring slows, the review queue backs up or your traffic drifts.
One line at startup sends each finished answer from your OpenAI, Anthropic or Gemini calls to Latent. Your calls go to your provider exactly as before.
Latent scores every answer from the first call, and keeps a sample of what your model actually served for calibration.
Latent runs calibration for you. A judge labels the sample against your sources, Latent tunes its threshold and its read to your traffic, and the update goes live with nothing to do on your side.
Flagged answers wait in a queue with the words that gave them away. Your reviewers confirm the failure or clear it, and each decision is kept as a labeled example.
pip install runlatent ✔ Successfully installed runlatent-0.3.0export LATENT_API_URL=https://app.runlatent.aiexport LATENT_API_KEY=lk_••••••••••••# app.py, once at startupimport runlatentrunlatent.auto_instrument()# after the first model callrunlatent.flush(); print(runlatent.stats()){"enabled": True, "captured": 1, "posted": 1, "failed": 0, ...}
stats() and never raised into your appLatent keeps this sample's text for calibration under your retention settings.
calibration run · 1,000 labeled answers · run by Latent
Each decision your reviewers make is kept as a labeled example. Latent learns from them, and the more answers your team validates, the better the data you post-train your model on.
Latent recalibrates on your reviewers' decisions while it runs, with no restart. Every validated answer can also be exported as fine-tuning or preference data for your model.
Serves as it does today.
Keeps what your retention settings allow.
Sees what it sees today.
Keeps what your retention settings allow.
When Latent runs in your environment, nothing leaves it: prompts, answers and activations stay on your infrastructure.
Sees what it sees today.
Receives nothing.
In your VPC, nothing leaves it, and the only outbound calls are ones you configure yourself: the model download, an alert webhook, a stronger model you name. Hosted by Latent, prompts and answers are sent to Latent's cloud for scoring, under the retention and redaction settings you choose.
Latent never receives or copies model weights, and calibration does not need them.
Each scored request writes a 1.7 KB append-only record with hashes of the prompt and answer, never the text.
Anything else, ask us. Get in touch
The runlatent Python SDK and an API key, when Latent hosts it. In your VPC, one GPU for Latent's reader model, the smallest of which fits in about 10 GB. Either way, the reader scores each finished answer and returns its verdict in a fraction of a second. Turn on an automatic response in the console, and the SDK in your app holds, replaces or reroutes a flagged answer before your user sees it. It covers OpenAI, Anthropic and Gemini clients; with any other provider, one call in your code does the same.
Any model, including closed-weight models behind an API, which Latent checks with its own reader model. With open-weight models served on vLLM in your VPC, Latent reads your own model's activations per token as it writes, and can act on an answer mid-stream.
None when Latent hosts it. In your VPC, it runs on the GPU already serving your model: the plugin adds 16 MiB of GPU memory at idle and at most 58 MiB at peak, whatever the prompt length. Every serving number on this site was measured on NVIDIA A100 40 GB or H100 GPUs.
Only when you tell it to. Out of the box Latent scores and records every answer. To hold, replace or reroute a flagged answer, turn on an automatic response in the console, and the SDK applies it in your app, or Latent's gateway in front of your model does. You can also act on the verdict in your code.
Reviewers working from Latent's flags find 2.7 to 4.2 times more wrong answers than checking at random, and when it checks other models' answers, 96 in 100 of the answers it flags are really wrong. Calibrating on your own traffic is what moves you to the top of that range.
When Latent hosts it, Latent runs calibration for you, judge included. A run takes under an hour for a few hundred to a thousand answers, and the update goes live with nothing to do on your side. In your VPC, one run over about 1,000 answers takes five to eight minutes and $6 to $8 of judge calls on your own key, and goes live at the plugin's next policy check, within 30 seconds by default, with no restart.
Yes. After the model download on first start you can block all egress from the host, and the stack keeps running.
A drift alert watches the scores. It stays silent on steady traffic and fires within 120 to 181 answers when a shift matters, so you recalibrate when you need to.