Choice Questions: Pick One from a Fixed Set of Options
The materials were prepared in the previous lesson; this lesson starts on asking questions. Jev Choice is the first of the five question types, and the most direct one: you list the options clearly, Jev picks one from them and returns it along with the probability of each option and an overall confidence—the answer is always one of your options, and you never have to write a single line of text-parsing if in your code.
In fact, you already used it in your first request: the department in route_ticket.py was a Choice question that picked technical. This lesson covers it thoroughly, in 10 minutes.
What You'll Be Able to Do
- Write a complete Jev Choice question using a
criteriaoptions map (option name → one-line description), knowing that both option names and descriptions are sent to the model - Read the three returned fields
choice,probabilities,confidence, and map the highest-probability option directly to a code branch - Ask multiple Choice questions in parallel in one request (up to 255 options per question), ignoring answers to speculative questions you don't use
- Add an
otheroption when the list may not be exhaustive, or follow the official recommendation to pair a companion Noul as a fallback - Self-check answer stability with "shuffle the options and run it again"—option order may affect the answer
Your Current Pain Points
- You ask GPT to classify tickets, but it returns a different category name every time—"refund," "return and refund," "wants money back"—and the downstream
ifcan never keep up with its self-invented vocabulary - To know how certain it is, you can only ask "how sure are you," and you get a paragraph of prose like "I think it's quite likely," which can't go into code
- One ticket requires several judgments (which team, what reason, what the customer wants); chaining three requests is slow and expensive, and cramming them all into one prompt makes the answers interfere with each other
When to Use This Technique
When you need:
- The answer to be a set of options you define yourself: which team to route to, which category a product belongs to, what language the code is written in
- The answer to become a code branch directly—one option, one path, no parsing from generated text
- To ask several choice questions in a row about the same material, while sending only one request
And you don't want:
- The model inventing new answers outside your list on the fly
- Writing a long prompt for a one-second judgment like "who owns this ticket"
- Letting it run automatically even when the model isn't confident
🎒 Prerequisites
- Completed State: you know that materials go in
state(strings, JSON objects, and text arrays all work) and judgments are written in questions - Got the first request working:
.venvhastypesafe-sdkinstalled and the environment variableTYPESAFE_API_KEYis set - Remember roughly what the
departmentquestion inroute_ticket.pylooked like—if you've forgotten, go back and skim it
Core Idea
The official one-sentence definition: use Choice when the answer is one of a fixed set of options. Typical questions from the docs: which team handles this ticket, which category this product belongs to, what language this code is written in.
What makes Choice great is one hard guarantee—the answer is constrained to the options you provide:
Every answer is constrained to the options you provide. The model returns a probability distribution over your options and never a value outside them. Your code never has to fish answers out of generated prose.
A Choice question looks like this:
| Field | What to write | Notes |
|---|---|---|
| ID (the question's key) | Your choice, e.g. department | Used only in your code, not sent to the model; answers are retrieved by it |
type | Fixed "choice" | — |
instructions | The full question | Write the entire question content here; don't count on the ID to relay it for you |
criteria | Options map: option name → description | Option names and descriptions are both sent to the model; up to 255 options per question |
How to choose among the three question types (official criteria):
| Type | What it answers | Usage in code |
|---|---|---|
| Choice (this lesson) | Which fixed option to pick, with no ordering among options | One option, one code path |
| Score | Where it falls on a spectrum, where you can describe each level | Threshold checks, ranking |
| Noul | clean yes/no, probabilities themselves are useful | an if |
The answer is: a spectrum (how severe, how angry) → Score; a clean yes/no (whether a refund was mentioned) → Noul. When both seem to fit, the official advice is to pick the one whose answer can be acted on directly by code: refund / rebook / information — three options that map directly to three code paths; that is the shape of a Choice.
Follow along with me
Step 1: Define the options map and write your first Jev Choice question
Why: A Choice question has only three fields: type, instructions, and criteria — both the option names and descriptions are sent to the model, so the descriptions should be written to distinguish the options from one another.
Using the example from the official docs: a support ticket for an online shoe store, asking which team should handle it. Create choice_first.py:
# 运行前:export TYPESAFE_API_KEY="<你的API Key>"
from typesafe_sdk import Choice, TypeSafeClient
with TypeSafeClient() as client:
response = client.system_one(
state="My running shoes arrived in the wrong size. Can I swap them for a size 10?",
questions={
"department": Choice(
instructions="Which team should handle this?",
criteria={
"returns": "Exchanges, wrong or damaged items",
"shipping": "Delivery status, delays, lost packages",
"billing": "Charges, invoices, payment problems",
},
),
},
)
print(response.answers["department"].choice)Run it:
.venv/bin/python choice_first.pyYou should see:
returnsWrong shoe size, wants to exchange for another pair — a matter for the returns team; in the official example recording, the model put everything on returns.
Three key points for writing criteria:
The option name is the answer value. The
choiceyour code receives is the option name string; use stable English identifiers (returns), not display text that may changeThe purpose of descriptions is separation. In the official words: both option names and descriptions are sent to the model, "so write the descriptions so the options are distinguishable from one another." When the name itself is clear enough, the description can be
null(Step 3 has a ready-made example)Give the full list, not a shortlist. A Choice question allows at most 255 options, and adding options only costs a few extra tokens — give the model all teams, categories, and products instead of pre-filtering
I got
returnsand know this value comes from an option name incriteria
Step 2: Read the returned choice, probabilities, and confidence
Why: choice is just the peak of the distribution; in real projects you will most likely branch on probabilities and confidence rather than just taking a single name.
The full response for the request in Step 1 (official example):
{
"model": "jev-1.13.0",
"answers": {
"department": {
"type": "choice",
"choice": "returns",
"confidence": 1.0,
"probabilities": {
"shipping": 0.0,
"returns": 1.0,
"billing": 0.0
}
}
},
"usage": {
"input_tokens": 328,
"output_tokens": 34
}
}You should see: besides type, the answer object contains three more values — let's break them down one by one:
| Field | Value in this example | How to read it |
|---|---|---|
choice | returns | the option with the highest probability |
probabilities | returns 1.0, the rest 0.0 | the distribution over all options, summing to about 1; the keys are option names |
confidence | 1.0 | compresses the shape of the distribution into a single number in 0–1: all on one option = 1, spread evenly = 0 |
Three insights worth remembering:
- The answer is always within the options. For a question with three options,
probabilitieshas only these three keys — there is no such thing as "the model added a fourth on its own" - confidence only looks at the highest probability. For three options the formula is
(3 × 最大概率 − 1) / 2(i.e., (3 × max probability − 1) / 2), so both(0.6, 0.3, 0.1)and(0.6, 0.2, 0.2)yield a confidence of 0.4. How the distribution shape is computed into this number and how to use the three tiers are covered in detail in the Confidence lesson - confidence describes the distribution; it is not a guarantee the answer is correct. This ticket is simple, the probability is all on
returns, so the confidence is naturally 1.0; a ticket mentioning both a wrong size and a duplicate charge would split the probability betweenreturnsandbilling, and the confidence would drop — Step 3 has exactly such a ticket
The "model": "jev-1.13.0" in the example is the version from when the official example was recorded; if you run it now with jev-latest, this field shows the current model version.
- I can point at the response and say:
choiceis the peak,probabilitiesis the distribution (summing to about 1),confidenceis the shape of the distribution
Step 3: Ask five Choice questions in one request (parallel evaluation)
Why: The official recommendation is to ask all the Choice questions you might need in one go: all questions are evaluated in parallel and independently against the same state — adding questions barely increases response time, and your code can just ignore the answers it doesn't need.
This time, switch to a more ambiguous ticket: two weeks late, wrong size, and charged twice — all three teams are implicated, and the customer didn't say what they want. For this ticket, the official example asked five Choice questions at once:
# 运行前:export TYPESAFE_API_KEY="<你的API Key>"
from typesafe_sdk import Choice, TypeSafeClient
TRIAGE_QUESTIONS = {
"department": Choice(
instructions="Which team should handle this?",
criteria={
"returns": "Exchanges, wrong or damaged items",
"shipping": "Delivery status, delays, lost packages",
"billing": "Charges, invoices, payment problems",
},
),
"return_reason": Choice(
instructions="If the customer wants to return something, why?",
criteria={
"wrong_size": "The item doesn't fit",
"wrong_item": "A different product was delivered",
"damaged": "The item arrived broken or faulty",
"changed_mind": "The item is fine, the customer no longer wants it",
"other": "A return reason that fits none of the above",
},
),
"shipping_issue": Choice(
instructions="If this is a shipping problem, which kind is it?",
criteria={
"not_delivered": "The package never arrived",
"delayed": "The package is late but still on the way",
"wrong_address": "The package went to the wrong place",
"damaged_in_transit": "The package arrived damaged",
"other": "A shipping problem that fits none of the above",
},
),
"requested_resolution": Choice(
instructions="What does the customer want to happen?",
criteria={
"exchange": "Swap the item for a different one",
"refund": "Money back",
"replacement": "The same item sent again",
"information": "Just an answer, no action needed",
},
),
"tone": Choice(
instructions="What is the customer's tone?",
criteria={"calm": None, "frustrated": None, "angry": None},
),
}
ticket = "Shoes arrived two weeks late and in the wrong size. Also I see two charges of $120 on my card. What are you going to do about this?"
with TypeSafeClient() as client:
response = client.system_one(state=ticket, questions=TRIAGE_QUESTIONS)You should see (official example response, all five answers arrive together; this request used 589 input / 212 output tokens in total):
| Question | choice | confidence | Probability highlights |
|---|---|---|---|
department | returns | 0.42 | returns 0.61, billing 0.35, shipping 0.04 |
return_reason | wrong_size | 1.0 | wrong_size 1.0, the rest 0.0 |
shipping_issue | delayed | 0.67 | delayed 0.74, other 0.26 |
requested_resolution | refund | 0.2 | refund 0.40, replacement 0.34, exchange 0.24 |
tone | frustrated | 0.76 | frustrated 0.84, angry 0.16 |
The official interpretation of each answer is worth reading line by line:
departmentsplit: returns 0.61, billing 0.35 — the duplicate charge gave billing over thirty percent of the probability. This ticket spans two teams, and the confidence of 0.42 reflects exactly thatreturn_reasonis clean: "wrong size" is stated plainly, everything onwrong_size, confidence 1.0shipping_issueis a speculative question: the answer splits between delayed and other, butdepartmentis not shipping at all — your code can just ignore it (Step 4 does exactly that)requested_resolutionis low-confidence: refund 0.40, replacement 0.34, exchange 0.24 — the duplicate charge suggests wanting money back, the wrong size suggests wanting an exchange, and the customer never said which one they wanttone's descriptions are allnull: calm / frustrated / angry are self-explanatory names that need no descriptions—this is exactly the case Step 1 refers to with "if you can't write a distinguishing description, give null"
Notice that of these five questions, two are speculative questions: return_reason is only useful when grouping into returns, and shipping_issue is only useful when grouping into shipping. Ask first, then use as needed—this is the standard practice for Jev: the official benchmark of 13 questions vs. 13 separate requests came out 12.2 times cheaper and 10.0 times faster, with unchanged answers (covered in the parallel questions lesson). The official smart-home demo is even more extreme: every request evaluates a long list of Choice questions (category, room, device, action) all at once; most answers go unused, and the code only takes the few it needs.
Step 4: Write routing branches with choice, probabilities, and confidence
Why: the answers are all typed values—choice maps directly to code paths, low confidence goes to human review, and if the second-place option's probability crosses a threshold, it gets CC'd. One request, five answers, and everything after that is plain if.
The official complete routing code that follows Step 3:
def triage(ticket: str) -> None:
with TypeSafeClient() as client:
response = client.system_one(
state=ticket,
questions=TRIAGE_QUESTIONS,
)
answers = response.answers
department = answers["department"]
if department.confidence < 0.3:
# Not clear which team to send to. Let a person decide.
send_to_manual_triage(ticket)
return
if department.choice == "returns":
# return_reason answer is only used here
assign(ticket, team="returns", issue=answers["return_reason"].choice)
elif department.choice == "shipping":
# shipping_issue answer is only used here
assign(ticket, team="shipping", issue=answers["shipping_issue"].choice)
else:
assign(ticket, team="billing")
# A second team with a real share of the probability gets a copy
for team, probability in department.probabilities.items():
if team != department.choice and probability > 0.25:
notify(ticket, team=team)
resolution = answers["requested_resolution"]
if resolution.confidence < 0.5:
# The customer hasn't said what they want. Ask, don't guess.
ask_customer_what_they_want(ticket)
elif resolution.choice == "refund":
flag_for_refund_approval(ticket)
if answers["tone"].choice == "angry":
flag_for_senior_agent(ticket)(assign / notify / send_to_manual_triage etc. are illustrative functions; replace them with your own routing logic.)
What you should see—for the ticket from Step 3, the official behavior is:
assigned to returns team with issue wrong_size
billing team notified (0.35 > 0.25)
customer asked what they want (confidence 0.20 < 0.5)All three gate values come from the official examples: 0.3 (below it, even the group can't be settled, so go to human review), 0.25 (the second-place option's probability share; above the line, CC it), 0.5 (the grasp of the customer's request; below it, ask instead of guess). shipping_issue's answer was never read at all—that's exactly the point of speculative questions.
Turning "intent classification + confidence gating" into a complete routing system (three-tier routing: deterministic logic / expert LLM / human) is covered in the smart routing lesson.
- I can clearly explain, for this ticket: why billing was CC'd, why the customer is being asked, and why shipping_issue wasn't used
Step 5: Two fallbacks for incomplete lists: the other option and a companion Noul
Why: the option list may not cover all inputs—a customer raises a request you didn't anticipate, and the model can only force a pick from the options you gave. The official docs provide two fallbacks.
Fallback one: the other option. Official recommendation: when the list may not cover all inputs, add an other or none of the above option so the model can say "none of these apply." Step 3's return_reason and shipping_issue both include an other option, with the description stating it is "the way out when none apply."
Fallback two: a companion Noul (officially preferred). The official docs note that none of the above is not necessarily reliable in many scenarios; the alternative is to include a Noul question in the same request that determines "does any option apply," for example:
Choice: which category does this request belong to?
Noul : does `message` mention any of `our_products`?The code only reads the Choice answer when the Noul is yes—if nothing applies, the whole group is skipped. This is exactly how the live-chat filter works: one request with three questions per message, and anything that doesn't match gets passed straight through. Noul itself is covered in the next lesson.
What you should see: neither fallback needs a new request—other is one extra option in criteria, and a companion Noul is one extra question in the same questions; together they still stay within the "one request" frame.
- I can say when to use each of the two fallbacks: add
otherwhen the list occasionally misses items; to "block even completely unrelated input," pair with a companion Noul
Checkpoint ✅
- My
criteriais an options map (name → description); I know both get sent to the model and the question ID does not - I know the answer is always within the given options;
probabilitiescovers all options and sums to approximately 1 - I know multiple questions in one request are evaluated in parallel and independently; a speculative question's answer can simply go unused
- I know confidence only looks at the highest probability (three options:
(3×最大概率−1)/2, i.e., (3 × max probability − 1) / 2); it describes the distribution, not a guarantee of correctness - I remember the two self-checks: reorder options to verify consistency; if you only need the best pick, just take the highest probability
Pitfall warnings
❌ Two options' descriptions can't be told apart
Symptom: with near-synonym options like return_policy / return_status, where the ticket mentions both return and refund, the model keeps wavering between the two and confidence never rises.
Why it's wrong: both option names and descriptions get sent to the model. When two descriptions point to the same kind of input, the model has no basis for comparison and can only split the probability between them.
Correct approach: nail down the boundaries in the descriptions—what this option covers and what it does not (the neighboring option's job). That's exactly how the official docs write return_policy / return_status: each description explicitly states the boundary of "whether and how an item can be returned"; with contrasting examples added, the official example achieves return_status and confidence 1.0. When you need to write more, use an object directly as the option value (see the first advanced technique).
❌ Change the order, and the answer changes
Symptom: shuffle the order of the three options in criteria, and the answer changes accordingly.
Why it's wrong: this is a known failure mode listed in the official pitfall guide (applicable to jev-1.13)—the order of Choice options affects the answer, and the model favors options listed earlier.
Correct approach: the official advice is one sentence—"reorder the options, run again, and check that the answer is consistent." Make it a fixed pre-launch self-check, especially for questions with few options and short descriptions. See the Jev 1.13 pitfall guide for the full list of failure modes.
❌ Using Choice to make the model "write its own answer"
Symptom: instructions say "what date is mentioned in this message," expecting the model to fill in the date.
Why it's wrong: Jev is not a generative model; generating text is the LLM's job, and that's what LLMs are trained for. Open-ended value questions fail both ways: the model can't make anything up (the answer is constrained to the options), yet you haven't given it any options to choose from.
Correct approach: if the answer space is bounded, enumerate it as options. Dates are a classic case: one Choice each for year, month, and day—months are only twelve, days at most thirty-one—plus a "not mentioned" option for each, so missing parts get reported honestly rather than guessed. The code assembles the parts into a real date and handles all comparisons and arithmetic. The full recipe for value extraction (regex to find candidates, Jev to pick the right one) is in the value extraction lesson.
❌ Setting confidence thresholds everywhere
Symptom: a confidence threshold on every answer and every branch, more and more of them, impossible to tune.
Why it's wrong: the official Agent Skill FAQ calls out this habit—when you only care about picking the best option, just take the one with the highest probability; no need for confidence thresholds everywhere. When you genuinely need statistical algorithms, what you should use is probabilities.
Correct approach: use choice for routing branches; use probabilities for distribution analysis; set confidence thresholds only on critical gates like "not sure whether to execute automatically" (Step 4's 0.3 / 0.5 are exactly that), and let them scale with risk—the full method is in the Confidence lesson.
Advanced techniques (optional)
🔧 When an option can't be written in one line, use an object. When two or three options keep getting confused and a one-line description can't make the boundary clear, replace the description with an object: one field says what the option covers, one says what it doesn't, and one gives a few examples. The official way of writing return_policy / return_status (raw request sent directly from the Playground, with the response being return_status and confidence 1.0):
official structured criteria example (raw JSON)
{
"state": "I sent the shoes back a week ago. When do I get my money?",
"questions": {
"return_topic": {
"type": "choice",
"instructions": {
"question": "Which returns topic is the customer asking about?",
"focus": "Classify the information the customer wants."
},
"criteria": {
"return_policy": {
"what": "Whether and how an item can be returned",
"not_for": "Progress of a return already sent",
"examples": [
"Can I return shoes I've worn once?",
"How long do I have to return an order?"
]
},
"return_status": {
"what": "Progress of a return already sent",
"not_for": "Whether and how an item can be returned",
"examples": [
"Has my return arrived yet?",
"When will my refund be paid?"
]
}
}
}
}
}Two key points: instructions can also be structured this way (question + focus); the field names what / not_for / examples are not API reserved words, so you name them yourself—the model sees both field names and values, so just name the fields with short words that say "what the following text labels."
🔧 Deep category trees: ask one level at a time. When the categories are deep (patent classifications, retail catalogs, directory trees of source code repositories), chain Choices together: each level's answer determines which options the next level's request offers. The official hierarchical classification cookbook runs beam search on top of this—using Choice's probabilities to keep the best K candidate paths at each level instead of betting on just one per level. See Hierarchical Classification for the full approach.
🔧 Bypassing the SDK to call the API directly. POST https://api.typesafe.ai/v1/systemone, with three top-level fields in the request body state / model / questions; the raw JSON for the department question looks like this:
{
"model": "jev-latest",
"state": "My running shoes arrived in the wrong size. Can I swap them for a size 10?",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"returns": "Exchanges, wrong or damaged items",
"shipping": "Delivery status, delays, lost packages",
"billing": "Charges, invoices, payment problems"
}
}
}
}For the complete definition of field constraints (criteria requires at least 1 and at most 255 options, with a four-field answer), see API Quick Reference.
Lesson Summary
✅ How to write a Jev Choice: criteria is an options map (name → one-line description; at most 255 options), so the answer always lands within your options and the code never parses text.
✅ Read the three return fields together: choice is the highest-probability option, probabilities is the distribution over all options, and confidence is a single number describing the shape of the distribution.
✅ Multiple questions per request run in parallel and are evaluated independently; ask speculative questions first and let the code use them as needed—13 questions combined into one request measured 12.2x cheaper and 10.0x faster.
✅ Lists may be incomplete: add an other escape hatch, or pair it with a companion Noul judging "is there anything related."
Review of key configuration fields:
| Field | Recommended way to write it | Notes |
|---|---|---|
type | "choice" | Fixed value |
instructions | The full question | The ID isn't sent to the model, so the question must be written out in full |
criteria | Options map: name → one-line distinguishing description | Give null as the description when the name is self-explanatory; provide the full list, not a shortlist |
| Answer access | response.answers["id"].choice | Read together with .probabilities and .confidence |
To see how Choice is used in real scenarios: Hierarchical Classification (beam search over a category tree using Choice probabilities), Live Chat Filtering (three questions in one request per message), and Smart Routing (Choice for intent classification, confidence for gating).
Next Lesson Preview
The other end of fixed options is "to what degree": Score Rating Questions: Rate on a Scale You Define.
You will learn:
criteriaswitches from an options map to an ordered list of levels, where the level number is the array indexscorecan land between two levels (e.g., 1.43), and must be read together with the probability distribution- Split multi-dimensional judgments into multiple Score questions, then normalize, weight, and merge them in code
- When to switch from Choice to Score