Skip to content

Choice Questions: Pick One from a Fixed Set of Options ​

The materials were prepared in the previous lesson; this lesson starts on asking questions. Jev Choice is the first of the five question types, and the most direct one: you list the options clearly, Jev picks one from them and returns it along with the probability of each option and an overall confidence—the answer is always one of your options, and you never have to write a single line of text-parsing if in your code.

In fact, you already used it in your first request: the department in route_ticket.py was a Choice question that picked technical. This lesson covers it thoroughly, in 10 minutes.

What You'll Be Able to Do ​

  • Write a complete Jev Choice question using a criteria options map (option name → one-line description), knowing that both option names and descriptions are sent to the model
  • Read the three returned fields choice, probabilities, confidence, and map the highest-probability option directly to a code branch
  • Ask multiple Choice questions in parallel in one request (up to 255 options per question), ignoring answers to speculative questions you don't use
  • Add an other option when the list may not be exhaustive, or follow the official recommendation to pair a companion Noul as a fallback
  • Self-check answer stability with "shuffle the options and run it again"—option order may affect the answer

Your Current Pain Points ​

  • You ask GPT to classify tickets, but it returns a different category name every time—"refund," "return and refund," "wants money back"—and the downstream if can never keep up with its self-invented vocabulary
  • To know how certain it is, you can only ask "how sure are you," and you get a paragraph of prose like "I think it's quite likely," which can't go into code
  • One ticket requires several judgments (which team, what reason, what the customer wants); chaining three requests is slow and expensive, and cramming them all into one prompt makes the answers interfere with each other

When to Use This Technique ​

When you need:

  • The answer to be a set of options you define yourself: which team to route to, which category a product belongs to, what language the code is written in
  • The answer to become a code branch directly—one option, one path, no parsing from generated text
  • To ask several choice questions in a row about the same material, while sending only one request

And you don't want:

  • The model inventing new answers outside your list on the fly
  • Writing a long prompt for a one-second judgment like "who owns this ticket"
  • Letting it run automatically even when the model isn't confident

🎒 Prerequisites ​

  • Completed State: you know that materials go in state (strings, JSON objects, and text arrays all work) and judgments are written in questions
  • Got the first request working: .venv has typesafe-sdk installed and the environment variable TYPESAFE_API_KEY is set
  • Remember roughly what the department question in route_ticket.py looked like—if you've forgotten, go back and skim it

Core Idea ​

The official one-sentence definition: use Choice when the answer is one of a fixed set of options. Typical questions from the docs: which team handles this ticket, which category this product belongs to, what language this code is written in.

What makes Choice great is one hard guarantee—the answer is constrained to the options you provide:

Every answer is constrained to the options you provide. The model returns a probability distribution over your options and never a value outside them. Your code never has to fish answers out of generated prose.

A Choice question looks like this:

FieldWhat to writeNotes
ID (the question's key)Your choice, e.g. departmentUsed only in your code, not sent to the model; answers are retrieved by it
typeFixed "choice"—
instructionsThe full questionWrite the entire question content here; don't count on the ID to relay it for you
criteriaOptions map: option name → descriptionOption names and descriptions are both sent to the model; up to 255 options per question

How to choose among the three question types (official criteria):

TypeWhat it answersUsage in code
Choice (this lesson)Which fixed option to pick, with no ordering among optionsOne option, one code path
ScoreWhere it falls on a spectrum, where you can describe each levelThreshold checks, ranking
Noulclean yes/no, probabilities themselves are usefulan if

The answer is: a spectrum (how severe, how angry) → Score; a clean yes/no (whether a refund was mentioned) → Noul. When both seem to fit, the official advice is to pick the one whose answer can be acted on directly by code: refund / rebook / information — three options that map directly to three code paths; that is the shape of a Choice.

Follow along with me ​

Step 1: Define the options map and write your first Jev Choice question ​

Why: A Choice question has only three fields: type, instructions, and criteria — both the option names and descriptions are sent to the model, so the descriptions should be written to distinguish the options from one another.

Using the example from the official docs: a support ticket for an online shoe store, asking which team should handle it. Create choice_first.py:

python
# 运行前:export TYPESAFE_API_KEY="<你的API Key>"
from typesafe_sdk import Choice, TypeSafeClient

with TypeSafeClient() as client:
    response = client.system_one(
        state="My running shoes arrived in the wrong size. Can I swap them for a size 10?",
        questions={
            "department": Choice(
                instructions="Which team should handle this?",
                criteria={
                    "returns": "Exchanges, wrong or damaged items",
                    "shipping": "Delivery status, delays, lost packages",
                    "billing": "Charges, invoices, payment problems",
                },
            ),
        },
    )

    print(response.answers["department"].choice)

Run it:

bash
.venv/bin/python choice_first.py

You should see:

text
returns

Wrong shoe size, wants to exchange for another pair — a matter for the returns team; in the official example recording, the model put everything on returns.

Three key points for writing criteria:

  • The option name is the answer value. The choice your code receives is the option name string; use stable English identifiers (returns), not display text that may change

  • The purpose of descriptions is separation. In the official words: both option names and descriptions are sent to the model, "so write the descriptions so the options are distinguishable from one another." When the name itself is clear enough, the description can be null (Step 3 has a ready-made example)

  • Give the full list, not a shortlist. A Choice question allows at most 255 options, and adding options only costs a few extra tokens — give the model all teams, categories, and products instead of pre-filtering

  • I got returns and know this value comes from an option name in criteria

Step 2: Read the returned choice, probabilities, and confidence ​

Why: choice is just the peak of the distribution; in real projects you will most likely branch on probabilities and confidence rather than just taking a single name.

The full response for the request in Step 1 (official example):

json
{
  "model": "jev-1.13.0",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "returns",
      "confidence": 1.0,
      "probabilities": {
        "shipping": 0.0,
        "returns": 1.0,
        "billing": 0.0
      }
    }
  },
  "usage": {
    "input_tokens": 328,
    "output_tokens": 34
  }
}

You should see: besides type, the answer object contains three more values — let's break them down one by one:

FieldValue in this exampleHow to read it
choicereturnsthe option with the highest probability
probabilitiesreturns 1.0, the rest 0.0the distribution over all options, summing to about 1; the keys are option names
confidence1.0compresses the shape of the distribution into a single number in 0–1: all on one option = 1, spread evenly = 0

Three insights worth remembering:

  • The answer is always within the options. For a question with three options, probabilities has only these three keys — there is no such thing as "the model added a fourth on its own"
  • confidence only looks at the highest probability. For three options the formula is (3 × 最大概率 − 1) / 2 (i.e., (3 × max probability − 1) / 2), so both (0.6, 0.3, 0.1) and (0.6, 0.2, 0.2) yield a confidence of 0.4. How the distribution shape is computed into this number and how to use the three tiers are covered in detail in the Confidence lesson
  • confidence describes the distribution; it is not a guarantee the answer is correct. This ticket is simple, the probability is all on returns, so the confidence is naturally 1.0; a ticket mentioning both a wrong size and a duplicate charge would split the probability between returns and billing, and the confidence would drop — Step 3 has exactly such a ticket

The "model": "jev-1.13.0" in the example is the version from when the official example was recorded; if you run it now with jev-latest, this field shows the current model version.

  • I can point at the response and say: choice is the peak, probabilities is the distribution (summing to about 1), confidence is the shape of the distribution

Step 3: Ask five Choice questions in one request (parallel evaluation) ​

Why: The official recommendation is to ask all the Choice questions you might need in one go: all questions are evaluated in parallel and independently against the same state — adding questions barely increases response time, and your code can just ignore the answers it doesn't need.

This time, switch to a more ambiguous ticket: two weeks late, wrong size, and charged twice — all three teams are implicated, and the customer didn't say what they want. For this ticket, the official example asked five Choice questions at once:

python
# 运行前:export TYPESAFE_API_KEY="<你的API Key>"
from typesafe_sdk import Choice, TypeSafeClient

TRIAGE_QUESTIONS = {
    "department": Choice(
        instructions="Which team should handle this?",
        criteria={
            "returns": "Exchanges, wrong or damaged items",
            "shipping": "Delivery status, delays, lost packages",
            "billing": "Charges, invoices, payment problems",
        },
    ),
    "return_reason": Choice(
        instructions="If the customer wants to return something, why?",
        criteria={
            "wrong_size": "The item doesn't fit",
            "wrong_item": "A different product was delivered",
            "damaged": "The item arrived broken or faulty",
            "changed_mind": "The item is fine, the customer no longer wants it",
            "other": "A return reason that fits none of the above",
        },
    ),
    "shipping_issue": Choice(
        instructions="If this is a shipping problem, which kind is it?",
        criteria={
            "not_delivered": "The package never arrived",
            "delayed": "The package is late but still on the way",
            "wrong_address": "The package went to the wrong place",
            "damaged_in_transit": "The package arrived damaged",
            "other": "A shipping problem that fits none of the above",
        },
    ),
    "requested_resolution": Choice(
        instructions="What does the customer want to happen?",
        criteria={
            "exchange": "Swap the item for a different one",
            "refund": "Money back",
            "replacement": "The same item sent again",
            "information": "Just an answer, no action needed",
        },
    ),
    "tone": Choice(
        instructions="What is the customer's tone?",
        criteria={"calm": None, "frustrated": None, "angry": None},
    ),
}

ticket = "Shoes arrived two weeks late and in the wrong size. Also I see two charges of $120 on my card. What are you going to do about this?"

with TypeSafeClient() as client:
    response = client.system_one(state=ticket, questions=TRIAGE_QUESTIONS)

You should see (official example response, all five answers arrive together; this request used 589 input / 212 output tokens in total):

QuestionchoiceconfidenceProbability highlights
departmentreturns0.42returns 0.61, billing 0.35, shipping 0.04
return_reasonwrong_size1.0wrong_size 1.0, the rest 0.0
shipping_issuedelayed0.67delayed 0.74, other 0.26
requested_resolutionrefund0.2refund 0.40, replacement 0.34, exchange 0.24
tonefrustrated0.76frustrated 0.84, angry 0.16

The official interpretation of each answer is worth reading line by line:

  • department split: returns 0.61, billing 0.35 — the duplicate charge gave billing over thirty percent of the probability. This ticket spans two teams, and the confidence of 0.42 reflects exactly that
  • return_reason is clean: "wrong size" is stated plainly, everything on wrong_size, confidence 1.0
  • shipping_issue is a speculative question: the answer splits between delayed and other, but department is not shipping at all — your code can just ignore it (Step 4 does exactly that)
  • requested_resolution is low-confidence: refund 0.40, replacement 0.34, exchange 0.24 — the duplicate charge suggests wanting money back, the wrong size suggests wanting an exchange, and the customer never said which one they want
  • tone's descriptions are all null: calm / frustrated / angry are self-explanatory names that need no descriptions—this is exactly the case Step 1 refers to with "if you can't write a distinguishing description, give null"

Notice that of these five questions, two are speculative questions: return_reason is only useful when grouping into returns, and shipping_issue is only useful when grouping into shipping. Ask first, then use as needed—this is the standard practice for Jev: the official benchmark of 13 questions vs. 13 separate requests came out 12.2 times cheaper and 10.0 times faster, with unchanged answers (covered in the parallel questions lesson). The official smart-home demo is even more extreme: every request evaluates a long list of Choice questions (category, room, device, action) all at once; most answers go unused, and the code only takes the few it needs.

Step 4: Write routing branches with choice, probabilities, and confidence ​

Why: the answers are all typed values—choice maps directly to code paths, low confidence goes to human review, and if the second-place option's probability crosses a threshold, it gets CC'd. One request, five answers, and everything after that is plain if.

The official complete routing code that follows Step 3:

python
def triage(ticket: str) -> None:
    with TypeSafeClient() as client:
        response = client.system_one(
            state=ticket,
            questions=TRIAGE_QUESTIONS,
        )
    answers = response.answers

    department = answers["department"]
    if department.confidence < 0.3:
        # Not clear which team to send to. Let a person decide.
        send_to_manual_triage(ticket)
        return

    if department.choice == "returns":
        # return_reason answer is only used here
        assign(ticket, team="returns", issue=answers["return_reason"].choice)
    elif department.choice == "shipping":
        # shipping_issue answer is only used here
        assign(ticket, team="shipping", issue=answers["shipping_issue"].choice)
    else:
        assign(ticket, team="billing")

    # A second team with a real share of the probability gets a copy
    for team, probability in department.probabilities.items():
        if team != department.choice and probability > 0.25:
            notify(ticket, team=team)

    resolution = answers["requested_resolution"]
    if resolution.confidence < 0.5:
        # The customer hasn't said what they want. Ask, don't guess.
        ask_customer_what_they_want(ticket)
    elif resolution.choice == "refund":
        flag_for_refund_approval(ticket)

    if answers["tone"].choice == "angry":
        flag_for_senior_agent(ticket)

(assign / notify / send_to_manual_triage etc. are illustrative functions; replace them with your own routing logic.)

What you should see—for the ticket from Step 3, the official behavior is:

text
assigned to returns team with issue wrong_size
billing team notified (0.35 > 0.25)
customer asked what they want (confidence 0.20 < 0.5)

All three gate values come from the official examples: 0.3 (below it, even the group can't be settled, so go to human review), 0.25 (the second-place option's probability share; above the line, CC it), 0.5 (the grasp of the customer's request; below it, ask instead of guess). shipping_issue's answer was never read at all—that's exactly the point of speculative questions.

Turning "intent classification + confidence gating" into a complete routing system (three-tier routing: deterministic logic / expert LLM / human) is covered in the smart routing lesson.

  • I can clearly explain, for this ticket: why billing was CC'd, why the customer is being asked, and why shipping_issue wasn't used

Step 5: Two fallbacks for incomplete lists: the other option and a companion Noul ​

Why: the option list may not cover all inputs—a customer raises a request you didn't anticipate, and the model can only force a pick from the options you gave. The official docs provide two fallbacks.

Fallback one: the other option. Official recommendation: when the list may not cover all inputs, add an other or none of the above option so the model can say "none of these apply." Step 3's return_reason and shipping_issue both include an other option, with the description stating it is "the way out when none apply."

Fallback two: a companion Noul (officially preferred). The official docs note that none of the above is not necessarily reliable in many scenarios; the alternative is to include a Noul question in the same request that determines "does any option apply," for example:

text
Choice: which category does this request belong to?
Noul  : does `message` mention any of `our_products`?

The code only reads the Choice answer when the Noul is yes—if nothing applies, the whole group is skipped. This is exactly how the live-chat filter works: one request with three questions per message, and anything that doesn't match gets passed straight through. Noul itself is covered in the next lesson.

What you should see: neither fallback needs a new request—other is one extra option in criteria, and a companion Noul is one extra question in the same questions; together they still stay within the "one request" frame.

  • I can say when to use each of the two fallbacks: add other when the list occasionally misses items; to "block even completely unrelated input," pair with a companion Noul

Checkpoint ✅ ​

  • My criteria is an options map (name → description); I know both get sent to the model and the question ID does not
  • I know the answer is always within the given options; probabilities covers all options and sums to approximately 1
  • I know multiple questions in one request are evaluated in parallel and independently; a speculative question's answer can simply go unused
  • I know confidence only looks at the highest probability (three options: (3×最大概率−1)/2, i.e., (3 × max probability − 1) / 2); it describes the distribution, not a guarantee of correctness
  • I remember the two self-checks: reorder options to verify consistency; if you only need the best pick, just take the highest probability

Pitfall warnings ​

❌ Two options' descriptions can't be told apart ​

Symptom: with near-synonym options like return_policy / return_status, where the ticket mentions both return and refund, the model keeps wavering between the two and confidence never rises.

Why it's wrong: both option names and descriptions get sent to the model. When two descriptions point to the same kind of input, the model has no basis for comparison and can only split the probability between them.

Correct approach: nail down the boundaries in the descriptions—what this option covers and what it does not (the neighboring option's job). That's exactly how the official docs write return_policy / return_status: each description explicitly states the boundary of "whether and how an item can be returned"; with contrasting examples added, the official example achieves return_status and confidence 1.0. When you need to write more, use an object directly as the option value (see the first advanced technique).

❌ Change the order, and the answer changes ​

Symptom: shuffle the order of the three options in criteria, and the answer changes accordingly.

Why it's wrong: this is a known failure mode listed in the official pitfall guide (applicable to jev-1.13)—the order of Choice options affects the answer, and the model favors options listed earlier.

Correct approach: the official advice is one sentence—"reorder the options, run again, and check that the answer is consistent." Make it a fixed pre-launch self-check, especially for questions with few options and short descriptions. See the Jev 1.13 pitfall guide for the full list of failure modes.

❌ Using Choice to make the model "write its own answer" ​

Symptom: instructions say "what date is mentioned in this message," expecting the model to fill in the date.

Why it's wrong: Jev is not a generative model; generating text is the LLM's job, and that's what LLMs are trained for. Open-ended value questions fail both ways: the model can't make anything up (the answer is constrained to the options), yet you haven't given it any options to choose from.

Correct approach: if the answer space is bounded, enumerate it as options. Dates are a classic case: one Choice each for year, month, and day—months are only twelve, days at most thirty-one—plus a "not mentioned" option for each, so missing parts get reported honestly rather than guessed. The code assembles the parts into a real date and handles all comparisons and arithmetic. The full recipe for value extraction (regex to find candidates, Jev to pick the right one) is in the value extraction lesson.

❌ Setting confidence thresholds everywhere ​

Symptom: a confidence threshold on every answer and every branch, more and more of them, impossible to tune.

Why it's wrong: the official Agent Skill FAQ calls out this habit—when you only care about picking the best option, just take the one with the highest probability; no need for confidence thresholds everywhere. When you genuinely need statistical algorithms, what you should use is probabilities.

Correct approach: use choice for routing branches; use probabilities for distribution analysis; set confidence thresholds only on critical gates like "not sure whether to execute automatically" (Step 4's 0.3 / 0.5 are exactly that), and let them scale with risk—the full method is in the Confidence lesson.

Advanced techniques (optional) ​

🔧 When an option can't be written in one line, use an object. When two or three options keep getting confused and a one-line description can't make the boundary clear, replace the description with an object: one field says what the option covers, one says what it doesn't, and one gives a few examples. The official way of writing return_policy / return_status (raw request sent directly from the Playground, with the response being return_status and confidence 1.0):

official structured criteria example (raw JSON)
json
{
  "state": "I sent the shoes back a week ago. When do I get my money?",
  "questions": {
    "return_topic": {
      "type": "choice",
      "instructions": {
        "question": "Which returns topic is the customer asking about?",
        "focus": "Classify the information the customer wants."
      },
      "criteria": {
        "return_policy": {
          "what": "Whether and how an item can be returned",
          "not_for": "Progress of a return already sent",
          "examples": [
            "Can I return shoes I've worn once?",
            "How long do I have to return an order?"
          ]
        },
        "return_status": {
          "what": "Progress of a return already sent",
          "not_for": "Whether and how an item can be returned",
          "examples": [
            "Has my return arrived yet?",
            "When will my refund be paid?"
          ]
        }
      }
    }
  }
}

Two key points: instructions can also be structured this way (question + focus); the field names what / not_for / examples are not API reserved words, so you name them yourself—the model sees both field names and values, so just name the fields with short words that say "what the following text labels."

🔧 Deep category trees: ask one level at a time. When the categories are deep (patent classifications, retail catalogs, directory trees of source code repositories), chain Choices together: each level's answer determines which options the next level's request offers. The official hierarchical classification cookbook runs beam search on top of this—using Choice's probabilities to keep the best K candidate paths at each level instead of betting on just one per level. See Hierarchical Classification for the full approach.

🔧 Bypassing the SDK to call the API directly. POST https://api.typesafe.ai/v1/systemone, with three top-level fields in the request body state / model / questions; the raw JSON for the department question looks like this:

json
{
  "model": "jev-latest",
  "state": "My running shoes arrived in the wrong size. Can I swap them for a size 10?",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": {
        "returns": "Exchanges, wrong or damaged items",
        "shipping": "Delivery status, delays, lost packages",
        "billing": "Charges, invoices, payment problems"
      }
    }
  }
}

For the complete definition of field constraints (criteria requires at least 1 and at most 255 options, with a four-field answer), see API Quick Reference.

Lesson Summary ​

✅ How to write a Jev Choice: criteria is an options map (name → one-line description; at most 255 options), so the answer always lands within your options and the code never parses text.

✅ Read the three return fields together: choice is the highest-probability option, probabilities is the distribution over all options, and confidence is a single number describing the shape of the distribution.

✅ Multiple questions per request run in parallel and are evaluated independently; ask speculative questions first and let the code use them as needed—13 questions combined into one request measured 12.2x cheaper and 10.0x faster.

✅ Lists may be incomplete: add an other escape hatch, or pair it with a companion Noul judging "is there anything related."

Review of key configuration fields:

FieldRecommended way to write itNotes
type"choice"Fixed value
instructionsThe full questionThe ID isn't sent to the model, so the question must be written out in full
criteriaOptions map: name → one-line distinguishing descriptionGive null as the description when the name is self-explanatory; provide the full list, not a shortlist
Answer accessresponse.answers["id"].choiceRead together with .probabilities and .confidence

To see how Choice is used in real scenarios: Hierarchical Classification (beam search over a category tree using Choice probabilities), Live Chat Filtering (three questions in one request per message), and Smart Routing (Choice for intent classification, confidence for gating).

Next Lesson Preview ​

The other end of fixed options is "to what degree": Score Rating Questions: Rate on a Scale You Define.

You will learn:

  • criteria switches from an options map to an ordered list of levels, where the level number is the array index
  • score can land between two levels (e.g., 1.43), and must be read together with the probability distribution
  • Split multi-dimensional judgments into multiple Score questions, then normalize, weight, and merge them in code
  • When to switch from Choice to Score
gaodong.aiAboutPrivacyContact

Independent tutorials, not affiliated with any product mentioned. Product names and trademarks belong to their owners. Check official pages for prices and policies.