Get started
Quickstart
Install the Python SDK, add one line at startup, and see your first scored answer.
Latent scores every answer your app gets from a model, wherever the model runs. The Python SDK captures each finished call and sends the prompt and the answer to Latent's scoring API. Your app's calls never get an exception from Latent, and until you change When Latent flags an answer from Deliver it and log it on the console's Policy page, they are never slowed by a network round trip and never see a different result.
Before you start
- A Latent API key. Sign in to the console with your email and password, an email link, Google, or GitHub. After your first sign-in the console shows your project's key once, starting with
lk_, with the install line filled in. To make another later, open Settings > Project > API keys. A key lasts one year, and your admins get an email 14 days before it expires. - A Python app that calls a model. The SDK captures calls made through the
openai,anthropic,google-genaiandollamaPython SDKs, including any OpenAI-compatible server you reach with theopenaiclient. Any other client takes one call per answer: see Ollama and other clients.
Set up with your coding agent
Paste this into Claude Code, Codex, Cursor, or another coding agent working in your project:
Add Latent hallucination scoring to this project.
Read https://runlatent.ai/docs/quickstart and https://runlatent.ai/docs/sdk before changing any code.
1. Install the runlatent package from PyPI (pip install runlatent) and add it to the project's dependencies.
2. Read LATENT_API_URL and LATENT_API_KEY from the project's environment configuration. Add both to the example env file if there is one, with LATENT_API_URL=https://app.runlatent.ai. Do not write a real key into the code.
3. In the app's entry point, import runlatent and call runlatent.auto_instrument() once at startup, before any OpenAI, Anthropic, Google GenAI or Ollama client is created, including an openai client pointed at another endpoint by base_url.
4. If LATENT_API_KEY is not set, stop here and ask me to create a key at https://app.runlatent.ai and add it to the project's environment myself. Never ask for the key in this chat.
5. Run the app once so it makes one model call, then call runlatent.flush() and print runlatent.stats(). Confirm that "posted" is at least 1 and "failed" is 0.
Change nothing else.
The agent's change is three lines and two environment variables. The coding agent guide explains each step.
Set up by hand
1. Install the SDK
pip install runlatent
The package has no dependencies of its own. It patches whichever of openai, anthropic, google-genai and ollama your app already uses.
2. Set your API key
export LATENT_API_URL=https://app.runlatent.ai
export LATENT_API_KEY=lk_...
Nothing is sent while neither LATENT_ nor its fallback LATENT_ is set: auto_ logs one warning and patches nothing. So you can ship the line below everywhere and turn Latent on per environment.
3. Add one line at startup
import runlatent
runlatent.auto_instrument()
That covers every OpenAI, Anthropic, Google GenAI, and Ollama client in the process, including clients a framework builds for you. Your model calls stay exactly as they are:
from openai import OpenAI
client = OpenAI()
answer = client.chat.completions.create(
model="gpt-4.1",
messages=[{"role": "user", "content": "What is our refund window?"}],
)
4. Verify
Run the app so it makes at least one model call. Posting runs on a background thread, so flush the queue before you read the SDK's counters:
runlatent.flush()
print(runlatent.stats())
# {"enabled": True, "captured": 1, "posted": 1, "failed": 0, "dropped": 0, ...}
posted counts the answers delivered to Latent. Each posted answer appears in your project's console with its risk score.
For a short script that exits right after its last call, the SDK waits up to 2 seconds at exit for queued answers. In a server, call runlatent. in your shutdown path.
Give Latent your sources
Latent checks each answer against the source it was written from, so send the source with the call. Any one of these counts:
-
Tag your documents around the call. Nothing in the call changes. From runlatent 0.3.4, the sources tagged when the call is made travel with it however the stream is consumed, so a stream returned from inside the block and read outside it, such as a FastAPI
StreamingResponse, keeps them:with runlatent.context(docs): # a string, a list of strings, or [{"title": ..., "text": ...}] client.chat.completions.create(model="gpt-5", messages=messages) -
A
Knowledge:line opening the user turn, with the question on the line after it. -
The system message, or earlier turns of the conversation.
On hosted plans, an answer that comes with none of these, a bare question on its own, gets no grounding verdict, since there is no source to check it against. It is stored and counted toward your plan, and the console marks it No source.
The no-source head scores those answers instead, on every hosted project by default. It is a second read, fit on answers written with no source at all. On held-out answers it separates wrong answers from right ones at AUROC 0.833 on Llama 3.1 8B's answers and 0.813 on Claude Haiku 4.5's, and at its shipped cut it flags about one right answer in ten while catching about half of the wrong ones. Once 30 of your own sourceless answers are scored, its cut sets itself from your traffic at your review budget and keeps adjusting as traffic moves; a cut you pin under Policy wins. To switch it off, clear Policy > Advanced > No-source head. Where you have a source, pass it: the grounding read is the stronger check.
Ollama and other clients
Ollama. runlatent 0.3.1 or later captures Ollama's own Python client as it is, with no other setup:
pip install -U runlatent
import runlatent
runlatent.auto_instrument()
import ollama
ollama.chat(model="llama3.2", messages=messages) # captured, like ollama.generate, ollama.Client and AsyncClient, streams included
Latent reads the answer's text, so a model running on your own machine is scored like any other, on every plan, Free included.
An OpenAI-compatible server. vLLM, LM Studio, and most other model servers speak the OpenAI API, and so does Ollama at /v1. Point the openai client at the server with base_url, and auto_ captures its calls like any other:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") # Ollama's OpenAI-compatible endpoint
Any other client, such as in-process transformers generation or your own HTTP code: score each answer with runlatent. once your model returns it.
import runlatent
r = runlatent.check(prompt, answer, context=sources, model="llama3.1") # sources: the documents the answer draws on
Latent reads the answer's text, so this works wherever the model runs, on every plan, Free included. Without a source, the no-source head gives the verdict, on by default: see Give Latent your sources. check() also returns the verdict, so your code can hold a flagged answer: see Hold an answer before your user sees it. From another language, send the pair to POST /.
Troubleshooting
enabled is False. Neither LATENT_ nor LATENT_ is set in the process that makes the calls.
captured grows but posted stays at 0. A rejected or unreachable post increments failed, and the SDK logs one warning per status every 60 seconds naming the URL and the HTTP status. A 401 means the key is unknown, revoked, or expired: create a new one in the console. A 4xx other than 429 is final after one attempt. stats()["errors"] is a different channel: it counts failures inside your process by stage, such as capture:, redact: and stream_.
patched is []. No supported SDK could be imported in this process, and one warning is logged.
failed grows after your plan's included answers. Free scores up to 5,000 answers a month. Pro and Team keep scoring past their included volume with usage billing, which is on from the upgrade, and pause there if an admin turns it off. Above the limit, the scoring API answers 429 limit_, and its reason says which limit: Free's monthly cap, the included volume, or your usage-billing spend limit. The SDK counts each pair as failed after three tries.
Nothing is captured. Pass arguments by keyword, as the provider SDKs require. with_, with_ and OpenAI's parse() are not captured; see Python SDK limits.
Note Gemini thinking models count their thought tokens against
max_. A small budget produces empty answers, which are not posted, and truncated answers, which are posted asoutput_ tokens lengthand, under a new project's default review rule, routed to review. Give thinking models a budget that covers the thinking, or turn thinking off, before you read the review queue as a quality signal.
Note OpenAI's
chat.can raisecompletions. stream() LengthFinishReasonErrorafter the stream has finished. The answer is still sent to Latent, marked as cut off at the length limit, and your app still gets the exception.
Next steps
- Hold an answer before your user sees it: one call returns the verdict before you return the answer.
- Python SDK reference: every option, per-client wrappers, streaming, and shutdown.
- What Latent receives: the exact fields sent for each call, and what is never sent.
- Calibration: fit Latent to your traffic once answers are flowing.
Was this page helpful?
Updated 3 October 2026