accept or review. A person can label a review later. The label does not change the evaluation.
Quickstart
Publish a pack and judge a ticket over REST.
Concepts
Ticket, pack, version, evaluation, and label.
Judge over REST
POST /api/judge returns the stored evaluation.MCP
Judge a published pack from an MCP client.
How a judgment runs
1
You send one ticket
subject and customerMessage are required. Earlier turns, metadata, and your own id are optional. See Concepts.2
The ticket is redacted
Email addresses, phone numbers, and payment-like numbers are replaced before the ticket is stored and before the model sees it. See Redaction.
3
The model answers the pack
One evaluation call asks every question on the same ticket. Questions are
choice, score, or boolean. They stay observational. Thresholds stay in the pack.4
Code decides the result
Each question has a review rule. Any rule that fires, or a model failure, stores
review. Otherwise the result is accept. See Results.5
The evaluation is stored
The row keeps the pack version, the redacted ticket, the answers, the model id, usage, and latency. It is append-only.
Four ways to call it
Simulations use the same published version. Create keys and the completion URL on Integrations.
What you get back
A successful judge call returns the stored evaluation id, the pack version that ran, and the result:reviewReasons names the question and the rule that fired. Read the full shape on Judge a ticket.