Every answer your model gives becomes a labeled example

Your evals test the questions you thought to ask. Your users ask everything else, and the answers your model gives them in production are the training data that would make it better. Almost none of that data gets labeled, because labeling it has always taken too long or cost too much.

Latent labels every answer. It scores each one in milliseconds and keeps it with its risk score and its verdict, flagged or passed. Your labeled dataset grows at the speed of your traffic: a million answers a month is a million labeled examples, from your users, on your documents, in your model's own words.

Why production answers go unlabeled

Labeling production answers has meant people or a judge model. People are accurate and slow, so they cover a sample. A judge model covers more, at a price: Claude Opus takes 3.1 seconds and costs $10.61 for every 1,000 answers it checks. At a million answers a month, that's $10,610 and about 860 hours of judge calls, so teams judge a sample there too, and the rest of their traffic goes unlabeled.

Latent is the only scalable way to label all of it and break down every failure.

Labeling a million answers a month	Cost	Time per answer
Claude Opus as judge	$10,610	3.1 seconds
Latent's reader, on a GPU you run	about $23	33 ms
Latent inside your own model on vLLM	No extra cost	No measurable overhead

The full measurements are in Latent against an LLM judge.

Every failure, in detail

Latent keeps more than a pass or fail. Each answer it labels carries:

• its risk score, and the chance the answer is wrong
• the riskiest sentence, marked. On human-labeled answers, it lands on the sentence a person marked 71 to 84% of the time.
• the kind of failure: a claim the source doesn't support, wrong arithmetic on correctly quoted figures, or a criterion you define
• the model and the detector version that scored it

Across a million answers, that adds up to a map of where your model goes wrong: on which questions, with which kinds of mistake, in which sentences. That's what post-training needs, and a sample can't show it to you.

The loop

1. Latent scores every answer in milliseconds and labels it flagged or passed.
2. Every answer is kept as a labeled example, with its request id, the model, the risk score and the verdict.
3. Latent's judge labels a sample of your answers, and Latent recalibrates on them, so its labels keep fitting your traffic.
4. Your team can review any flagged answer if you want a person's call. Theirs takes precedence over Latent's.
5. You export the dataset and train the next version of your model on it.

No reviewer required

The loop runs without anyone on your team touching it. Latent labels every answer as it scores it, and its judge labels a sample on its own to keep those labels calibrated to your traffic. Review is there when you want it: a person can confirm a flagged answer as a failure or release it, tag the kind of error, and those decisions also come out as ready-made fine-tuning and preference files.

What you export

Export	What's in it
The full log	Every answer Latent scored, with its risk score and verdict, keyed by request id so it joins your own logs
Fine-tuning (SFT)	Reviewed answers: released ones as demonstrations, confirmed failures marked to leave out
Preference (DPO)	Reviewed answers: confirmed failures as rejected answers, released ones as chosen. Pair each rejected answer with a better one for the same prompt

Download any of them from the console whenever you like.

When your team does review, it reviews the right answers

Checking answers at random, a reviewer spends most of the day on answers that are fine. Working from Latent's flags, reviewers find 2.7 to 4.2 times as many real failures as random review, reading the same number of answers. When Latent checks other models' answers, 96 in 100 of the answers it flags are really wrong. Every hour of review goes into the answers your model learns the most from.

It keeps up when your model changes

When you ship the retrained model, its answers change, and Latent follows. A drift alert stays silent on steady traffic and fires within 120 to 181 answers when a shift matters. Recalibrating on the new model's answers takes under an hour, and the new detector goes live with no restart. Then the loop starts again on the new model's answers.

Start the loop

Add one line to your app, and every answer your model gives starts landing in your dataset, scored and labeled. Start free, or book a call and we'll set up the loop with you.