Noul yes/no questions: get true probabilities from 0 to 1

The Score from the previous lesson measures "to what degree". But many judgments don't even need a degree asked—they're a single cut of yes or no: should this message be escalated to a human, does this resume mention distributed systems, does this comment contain personal information.

Hand these questions to Jev Noul: pose a yes/no question, and Jev returns the probability that "the answer is yes"—a number between 0 and 1.

In fact, you already used it in your first request—the is_urgent in the ticket example was a Noul, and the "API fully 500" ticket got back 0.99. This lesson covers it thoroughly, in 8 minutes.

What you'll be able to do after this lesson

Write a clean Jev Noul yes/no question: one yes/no per question, values near 1 reliably mean "yes", and when the boundary is subtle use criteria to define true / false clearly
Read noul values: near 1 a strong "yes", near 0 a strong "no", near 0.5 a coin flip, and explain why there is no separate confidence
Turn probabilities into code branches with 0.7 / 0.3 starting thresholds, adjusted up or down based on the cost of errors
Send a batch of Noul in one request for checklist verification, using structured instructions so code generates the questions in batch
Explain clearly when to use Noul and when to switch to Score—0.5 is not "moderate degree"

Your current predicament

Ask GPT a yes/no question and it analyzes in three paragraphs before concluding, often saying "overall yes, but not entirely"—unusable in code
Ask the same message twice, get "yes" once and "no" the other, with no stable probability, so the threshold can never be settled
Force it to answer with numbers only, and it will say both 0.9 and 0.6, but no one knows whether those two numbers mean the same kind of "certainty", so what to review is pure guesswork

When to use this technique

When you need:

Clean yes/no judgments: whether a message is asking for a refund, whether a resume mentions distributed systems, whether a comment contains personal information
Probabilities going straight into code: pass a threshold to enter if, sort by probability, weight after combining multiple questions
Ask a whole series of yes/no conditions at once: checklist-style verification, one condition per question

And you don't want to:

Parse a chat model's freeform "Yes, but..."
Mix yes/no standards into the prompt, where the standard drifts as soon as the model version changes
Let it execute automatically even when the model itself is unsure (the value falls near 0.5)

🎒 Preparation before starting

completed State: you know the content to judge goes in state (strings, JSON objects, and text arrays all work)
ran the first request successfully: .venv has typesafe-sdk installed, and TYPESAFE_API_KEY is in the environment variables

Core idea

Noul answers "is it or isn't it". Official applicable scenarios: whether a message is asking for a refund, whether a resume mentions distributed systems, whether a comment contains personal information—there are only two answers, and the probability itself is a useful signal. This probability is calibrated: Jev is trained for calibrated decisions, with probabilities aligned to real outcomes, so a number like 0.7 can stand up to being used as a threshold.

**A TypeSafe Noul question has only three fields: **

Field	What to write	Example
`type`	Fixed `"noul"`	—
`instructions`	The complete yes/no question, or a statement for the model to judge	Is the customer asking for a human agent?
`criteria`	Optional: `true` / `false` one sentence each, defining what counts as "yes" and what counts as "no"	see Step 1

Pick your own question ID (e.g., is_human_escalation); it is not sent to the model—the full question must be written in instructions, even if the ID already seems self-explanatory.

The return is just one number. noul = the probability that the answer is "yes": 0 no, 1 yes. There is no separate confidence—a yes/no has only two outcomes, so one probability describes the entire distribution; Choice / Score spread their probabilities across multiple options or levels, which is why they need confidence to summarize the shape of the distribution.

How to choose among the three question types.

Question type	Answer form	Usage in code
Choice	Pick one from fixed options, with no ordering among the options	Maps to N code paths
Score	A position on a spectrum, where you can describe each level	Thresholding, ranking
Noul (this lesson)	A clean yes/no, where the probability itself is the signal	A single `if`

Official criteria: fixed options with no ordering → Choice; a spectrum where each level can be described → Score; a clean yes/no where the probability is useful → Noul. When two seem to fit, pick the one whose answer can be acted on directly by code.

noul is not a degree scale. It only measures the probability that "the proposition you asked about is true." 0.5 = a fifty-fifty split, not "medium"—when "strong" in "is the candidate strong at Python" is not clearly defined, 0.5 says nothing. To measure degree, use Score's defined levels (see item 3 of the pitfall reminders for the official comparison experiment).

Follow along

Step 1: Write a clean Jev Noul yes/no question (instructions and optional criteria)

Why: the meaning of a Jev yes/no question's noul value is determined entirely by its wording. Two conditions crammed into one question, a reversed direction, a fuzzy boundary—the value becomes unreadable, and any threshold you set is wasted.

Official question-writing guidelines:

One Noul asks only one yes/no question. "Is the customer both angry and demanding a refund?" is two judgments; the model can only estimate them together, muddying the meaning of the value. Split it into two questions and combine them in code (that's exactly what Step 4 does)
Make values close to 1 consistently mean "yes." Does the message contain personal data? is clear; Is the message free of personal data? has the direction reversed, and the code reading the value will be flipped across the board
Both declarative and interrogative forms work. The customer is requesting a refund—close to 1 means the statement is true. Try both wordings on your own data and keep the one that performs better
Leave no middle ground at the boundary. The any in Does this candidate have any Python experience? pins down the boundary; when the boundary is subtle, add criteria to spell out what counts as true / false

An official example of a subtle boundary—the "repeat contact" question with criteria added:

python

from typesafe_sdk import Noul, NoulCriteria

Noul(
    instructions="Has the customer contacted support about this before?",
    criteria=NoulCriteria(
        true="Mentions a prior attempt, ticket, or that they have asked before",
        false="No sign of any previous contact",
    ),
)

Two additional notes:

Official wording: instruction is enough for most Nouls—one sentence suffices for most questions; run a version without criteria first, and add it only if the answers aren't satisfactory
When state is a structured object, use a backtick-wrapped path to name the field to judge: Does ticket.messages[0].text request a refund?

What you should see: a cleanly worded yes/no question—only one judgment, values close to 1 clearly mean "yes," and the boundary is unambiguous.

My question contains only one yes/no judgment, and values close to 1 clearly mean "yes"

Step 2: Send your first Noul request with typesafe-sdk

Why: first run the official two-question example as-is to establish a "known-correct" baseline—so that when you swap in your own questions, you have a reference for troubleshooting.

Create noul_support.py and paste in the official example (a customer service message requesting human handoff, asking two things at once):

python

from typesafe_sdk import Noul, NoulCriteria, TypeSafeClient

with TypeSafeClient() as client:
    response = client.system_one(
        model="jev-latest",
        state="I have asked three times now. Can I please just talk to a real person?",
        questions={
            "is_human_escalation": Noul(
                instructions="Is the customer asking for a human agent?",
            ),
            "is_repeat_contact": Noul(
                instructions="Has the customer contacted support about this before?",
                criteria=NoulCriteria(
                    true="Mentions a prior attempt, ticket, or that they have asked before",
                    false="No sign of any previous contact",
                ),
            ),
        },
    )

    print(response.answers["is_human_escalation"].noul)
    print(response.answers["is_repeat_contact"].noul)

Run:

bash

.venv/bin/python noul_support.py

What you should see:

text

0.99
0.93

The official interpretation: the message explicitly says talk to a real person, so "wants a human" is 0.99; I have asked three times now matches the true description of is_repeat_contact (mentioning having asked before), so it is 0.93. The example values may change with the Jev version, so don't worry about differences after the decimal point.

I got two numbers close to 1 (example values 0.99 / 0.93)

💡 Can Chinese content be judged

Yes. The official documentation says: English is the primary training language and gives the best accuracy; other languages, including Chinese, work but with lower accuracy. Before going live with Chinese scenarios, run a batch on your own data first; for Noul, which has no confidence, focus on how many values fall into the 0.3–0.7 uncertainty band (Step 4).

Step 3: Read noul values: what 0, 0.5, and 1 each mean

Why: a single number is unreadable without a frame of reference. The official docs recorded answers for the same question (is_human_escalation) across six different messages, which is the most intuitive yardstick.

First, look at the full return from the Step 2 request:

json

{
  "model": "jev-1.13.0",
  "answers": {
    "is_human_escalation": {
      "type": "noul",
      "noul": 0.99
    },
    "is_repeat_contact": {
      "type": "noul",
      "noul": 0.93
    }
  },
  "usage": {
    "input_tokens": 360,
    "output_tokens": 39
  }
}

What you should see: each answer object contains only the type and noul fields, and noul matches the 0.99 / 0.93 you printed. "model": "jev-1.13.0" is the version used when the official example was recorded; if you run it now with jev-latest, this shows the current model version.

Side-by-side of the six messages (officially recorded jev-1.13.0 answers; messages are the original English state):

Customer message (state)	`noul`
Thanks, that fixed it!	0.02
How do I reset my password?	0.07
I need this sorted today, whatever it takes.	0.26
Are you a bot?	0.40
Is there any way to speak to someone about my invoice?	0.84
I have asked three times now. Can I please just talk to a real person?	0.99

How to read it:

This number combines the answer and the certainty. Close to 1 is a strong "yes," close to 0 a strong "no," and close to 0.5 means the model gives roughly equal probability to yes/no—it is itself uncertain
0.26: urgent, but not asking for a person. I need this sorted today conveys urgency but never asks for a human—the two judgments are separable, so the value is low
0.40: implied, not stated. Are you a bot? hints at wanting a person without saying so directly, and the model is almost fifty-fifty
The official docs point out: values that "fall in the middle," like 0.26 and 0.40, are precisely the messages that must be adjudicated by the thresholds in your code

Why is there no confidence? Choice / Score spreads its probability across multiple options or levels, and confidence summarizes the shape of that distribution into a single number; a yes/no question has only two outcomes, so a single noul already fully describes the distribution. If you want to use the same thresholds as Choice / Score, you can compute it yourself—see |2p − 1| in the advanced tips.

💡 Another way to fetch the value

Quick Start lesson's example uses response.nouls["is_urgent"].noul, while this lesson's primitives example uses response.answers["id"].noul—both entry points lead to the same set of answers, so just copy the official example.

Step 4: Set noul probability thresholds to turn probabilities into code branches

Why: A probability only counts as being used once it turns into action. The most common way to use Noul is to slice the probability into a boolean with a threshold and attach an if.

Simplest form (official example):

python

wants_human = response.answers["is_human_escalation"].noul > 0.9

if wants_human:
    route_to_agent(ticket)
else:
    route_to_bot(ticket)

Where should the threshold sit? The official starting points and adjustment rules:

Rule	What to do
Starting point	yes ≥ 0.7, no ≤ 0.3
Only care about "yes"	0.7 one threshold is enough
A false "yes" is costly	Raise it—for example, waking up the on-call person, issuing a direct refund
Missing a true "yes" is costly	Lower it—for example, missing a safety issue flag
Between 0.3–0.7	don't take the code path; hand off to a human
Avoid	fine-tuning the threshold to "all correct" against just a handful of test cases

Full routing example (official): both questions asked in one go, either question landing in the middle triggers a human handoff, and repeated contact raises the priority:

python

from typesafe_sdk import Noul, NoulCriteria, TypeSafeClient

SUPPORT_QUESTIONS = {
    "is_human_escalation": Noul(
        instructions="Is the customer asking for a human agent?",
    ),
    "is_repeat_contact": Noul(
        instructions="Has the customer contacted support about this before?",
        criteria=NoulCriteria(
            true="Mentions a prior attempt, ticket, or that they have asked before",
            false="No sign of any previous contact",
        ),
    ),
}

YES = 0.7
NO = 0.3


def route(message: str) -> None:
    with TypeSafeClient() as client:
        response = client.system_one(
            model="jev-latest",
            state=message,
            questions=SUPPORT_QUESTIONS,
        )
    answers = response.answers

    wants_human = answers["is_human_escalation"].noul
    repeat = answers["is_repeat_contact"].noul

    if NO < wants_human < YES or NO < repeat < YES:
        # The model isn't sure either way. Let a person decide.
        send_to_review(message)
        return

    priority = "high" if repeat >= YES else "normal"
    if wants_human >= YES:
        route_to_agent(message, priority=priority)
    else:
        route_to_bot(message, priority=priority)

route_to_agent / route_to_bot / send_to_review are your own handler functions; the official example only demonstrates the decision logic.

What you should see:

The "asked three times" message: 0.99 / 0.93 both pass YES → hand off to a human, and repeat ≥ 0.7 raises the priority to high
How do I reset my password?: both questions are 0.07 → goes to the bot

How to adjust after launch? The official guidance gives two directions, plus one invariant:

Reviewers can't keep up → narrow the gap between NO and YES so more messages land in the code path
Misrouted messages increase → widen the gap so more messages go to a human
Want to add a new check (does it mention payment, does it contain personal information) → add another Noul to SUPPORT_QUESTIONS; the request count stays at 1
My code has explicit YES / NO thresholds, and values between 0.3–0.7 have a human escape hatch

Step 5: Send a batch of Nouls in one request: checklist batch verification and structured instructions

Why: Many judgments are naturally checklists—whether each record in a résumé and candidate database is the same person, whether each pair of adjacent lines in a document was broken by a line wrap. The official approach: one Noul per condition, all packed into a single request; questions are evaluated in parallel, adding questions barely increases response time and only costs a few extra question tokens (for measured gains from sending 13 questions in one batched request, see parallel questions).

When each question also needs to carry data, write instructions as an object: the question stays fixed in one field, the data goes in other fields, and it's generated in bulk by code. The official résumé deduplication example—comparing a newly arrived résumé one by one against records in the candidate database that might be the same person:

python

from typesafe_sdk import Noul, TypeSafeClient

SAME_PERSON = "Is the resume for the same person as `potential_duplicate`?"


def duplicate_questions(candidates: list[dict]) -> dict[str, Noul]:
    """One Noul per candidate record, all asking the same question."""
    return {
        f"same_as_record_{candidate['id']}": Noul(
            instructions={
                "potential_duplicate": {
                    "name": candidate["name"],
                    "location": candidate["location"],
                    "last_employer": candidate["last_employer"],
                },
                "question": SAME_PERSON,
            },
        )
        for candidate in candidates
    }


def find_duplicates(resume: dict, candidates: list[dict]) -> list[str]:
    with TypeSafeClient() as client:
        response = client.system_one(
            model="jev-latest",
            state={"resume": resume},
            questions=duplicate_questions(candidates),
        )
    return [
        question_id
        for question_id, answer in response.answers.items()
        if answer.noul > 0.7
    ]

Three details:

Question IDs are assembled by code from the record's database ID (same_as_record_18), and answers are attached back to the same ID as-is
The question field is identical across all questions; only the data in potential_duplicate changes
find_duplicates uses 0.7 to filter for "yes"—middle values should likewise go to a human, same idea as Step 4

The official example's return (three candidate records):

json

{
  "model": "jev-1.13.0",
  "answers": {
    "same_as_record_18": {
      "type": "noul",
      "noul": 0.74
    },
    "same_as_record_42": {
      "type": "noul",
      "noul": 0.09
    },
    "same_as_record_77": {
      "type": "noul",
      "noul": 0.08
    }
  },
  "usage": {
    "input_tokens": 535,
    "output_tokens": 58
  }
}

Official interpretation: record 18 has a differently spelled name, but location and employer both match → 0.74; record 42 has the same name, different city and employer → 0.09; record 77 has a similar name, same location, different employer → 0.08.

You should see: find_duplicates() returns ['same_as_record_18']—only 0.74 passes the 0.7 threshold.

The same structured instructions can also verify a cheap model's extraction results field by field—the cascade extraction scenario page expands on this usage (the same set of questions for every field, with main_question holding the question text, and field_spec, extracted_field varying per field).

Checkpoint ✅

I know noul is the probability that the answer is yes: near 1 is a strong yes, near 0 a strong no, near 0.5 a coin flip
I ran the official example's 0.99 / 0.93 and can say which sentence in the message each corresponds to
I know Noul answers have no separate confidence, and why
My starting thresholds are yes ≥ 0.7, no ≤ 0.3, with 0.3–0.7 going to a human, adjusted according to the cost of errors
Each of my Nouls asks a single yes/no question; for batch verification, one question per condition, all sent in one request

Pitfall reminders

❌ Cramming two conditions into one Noul

text

instructions: "Is the customer angry and asking for a refund?"

"Angry" and "wants a refund" are two separate judgments; squeezed into one question, the model can only estimate them together: a message where one condition is true and the other false has nowhere to go, and 0.6 can't tell whether it's "a bit angry but not asking for money" or "not angry but asking for money".

The right approach: one Noul per condition, sent in parallel in one request, combined in code—the route() in Step 4 is an example of combining the two questions "needs a human × repeated contact".

❌ Asking in reverse, so near 1 becomes "no"

text

Counterexample: Is the message free of personal data?        # near 1 = no personal data, opposite to the reading intuition
Good example: Does the message contain personal data?      # near 1 = has personal data

The code still runs if the direction is flipped, but three months later whoever reads this code (including you) will intuitively treat a high value as "there's a problem"—the whole logic gets inverted. Official rule: word it so that near 1 consistently means "yes".

❌ Using noul as a degree scale: 0.5 is not "moderate"

Want to measure "how strong the candidate is in Python"? The official docs ran a comparison between Noul and a four-level Score on the same set of candidates:

Candidate self-description (state)	Noul: Is the candidate strong in Python?	Score: four-level experience scale
My experience is in Java and Go. I have not used Python.	0.03	0.0（No experience）
I have used Python occasionally for small scripts alongside my main Java work.	0.14	1.0（Some familiarity）
I used Python every day for two years in my last job, mostly data pipelines.	0.81	2.05（Regular use in a job）
I have written Python daily for eight years, including maintaining a large Django codebase.	0.92	2.89（Deep expertise）

Noul judges only the single proposition "strong," and the value is the likelihood that it is true. You can define 0.3–0.7 in your code as "some experience," but the model cannot see your definition—the middle value could mean "moderate experience" or "hard to say," and the spacing between candidates is not the one you chose. Score checks candidates level by level against the descriptions you write; each candidate lands on or near a level you defined. If you disagree with a level, change the description and rerun.

The correct approach: for yes/no decisions, define the condition clearly and then ask—the official example sentence is Does the resume state that the candidate has used Python at work?; to measure a degree, use the defined levels of Score.

❌ Fine-tuning the threshold against a handful of test cases

Running five or six test messages repeatedly and tuning the threshold until every one is classified correctly—the officials explicitly advise against this: swap in another batch of cases and that "perfect threshold" often stops holding up. The correct approach: start from 0.7 / 0.3, adjust according to the cost rules in step 4, route middle values to humans, and after going live narrow or widen based on review volume and error rate.

For more general failure modes (literal interpretation, unreliable counting, putting date comparisons in code), see the Jev 1.13 pitfall handbook.

Advanced tips (optional)

Compute a "confidence" for Noul yourself. Noul has no confidence, but if you want to handle Noul and Choice / Score with the same threshold code, the officials provide a formula—take the distance from 0.5:

python

p = answer.noul
confidence = abs(2 * p - 1)  # p=0.5 时为 0，p=0 或 1 时为 1

It simply applies Choice's confidence formula to the two options "yes/no," so it is on the same scale as Choice's confidence. The derivation of the formula is in the next lesson, Confidence.

Try both interrogative and declarative sentences. A declarative sentence like The customer is requesting a refund works the same as a question—close to 1 means the statement is true. The officials suggest running both wordings on your own data and keeping whichever performs better.

Try it without writing code. In the console's Playground (https://console.typesafe.ai/playground), paste in the state and add a question to see the answer without any code.

Bypass the SDK and call the API directly. POST https://api.typesafe.ai/v1/systemone, and the raw JSON for a noul question looks like this (the is_urgent from the quick-start lesson):

json

{
  "state": "Our API integration started returning 500 errors on every request about 20 minutes ago, and we can't process any customer orders until this is fixed.",
  "model": "jev-latest",
  "questions": {
    "is_urgent": {
      "type": "noul",
      "instructions": "The message conveys urgency or time-sensitivity"
    }
  }
}

In the response, the answer for is_urgent is {"type": "noul", "noul": 0.99} (a recorded value from the official example). For the complete definition of the request field constraints, see API quick reference.

Lesson summary

✅ Jev Noul usage: one yes/no question at a time, write the full question in instructions (optionally use criteria to define the yes/no boundary), and Jev returns the probability that "the answer is yes."

✅ Reading the value: close to 1 is a strong yes, close to 0 is a strong no, close to 0.5 is a toss-up; there is no separate confidence—a single number describing the two-outcome distribution says it all.

✅ Thresholds: starting points yes ≥ 0.7, no ≤ 0.3; raise them if false "yes" is costly, lower them if missed true "yes" is costly; route 0.3–0.7 to a human; do not fine-tune against a few cases.

✅ Batching: for checklist-style judgments, put one condition per question into the same request; instructions can be written as an object, with code generating the questions in bulk.

✅ Choosing the type: clean yes/no where probability is useful → Noul; measuring a degree → Score; fixed options → Choice.

Key configuration fields at a glance:

Field	Recommended usage	Notes
`type`	`"noul"`	Fixed value
`instructions`	One yes/no question or a single statement; close to 1 = yes	For complex state, use backtick paths to name fields
`criteria`	Optional `NoulCriteria(true=…, false=…)`	Add when the boundary is subtle; most questions don't need it
Answer access	`response.answers["id"].noul`	also works `response.nouls["id"].noul`

To see how Jev Noul is used in real scenarios: Self-consistency (scoring insurance claims yes/no against a rubric of 14 items and measuring stability across multiple runs); Search and matching (reranking that directly sorts by noul probability, without going through thresholds); Cascaded extraction (structured instructions checking extraction results field by field).

Next lesson preview

Noul has no confidence, so how are Choice / Score confidence values computed from the probability distribution, and how should thresholds scale with risk? Next lesson: Confidence: what to do when the model says "I'm not sure".

You will learn:

The confidence formulas: Choice and Score each have their own, and both can be verified by hand from probabilities
Three-tier usage: high—execute automatically; medium—confirm or flag; low—escalate to a human
Scaling thresholds with risk: 0.5 as the fallback, and high-risk operations like approve_transfer need to be > 0.9
Alternative metrics when not using the official confidence: top probability, top-to-second ratio

Noul yes/no questions: get true probabilities from 0 to 1 ​

What you'll be able to do after this lesson ​

Your current predicament ​

When to use this technique ​

🎒 Preparation before starting ​

Core idea ​

Follow along ​

Step 1: Write a clean Jev Noul yes/no question (instructions and optional criteria) ​

Step 2: Send your first Noul request with typesafe-sdk ​

Step 3: Read noul values: what 0, 0.5, and 1 each mean ​

Step 4: Set noul probability thresholds to turn probabilities into code branches ​

Step 5: Send a batch of Nouls in one request: checklist batch verification and structured instructions ​

Checkpoint ✅ ​

Pitfall reminders ​

❌ Cramming two conditions into one Noul ​

❌ Asking in reverse, so near 1 becomes "no" ​

❌ Using noul as a degree scale: 0.5 is not "moderate" ​

❌ Fine-tuning the threshold against a handful of test cases ​

Advanced tips (optional) ​

Lesson summary ​

Next lesson preview ​

Noul yes/no questions: get true probabilities from 0 to 1

What you'll be able to do after this lesson

Your current predicament

When to use this technique

🎒 Preparation before starting

Core idea

Follow along

Step 1: Write a clean Jev Noul yes/no question (instructions and optional criteria)

Step 2: Send your first Noul request with typesafe-sdk

Step 3: Read noul values: what 0, 0.5, and 1 each mean

Step 4: Set noul probability thresholds to turn probabilities into code branches

Step 5: Send a batch of Nouls in one request: checklist batch verification and structured instructions

Checkpoint ✅

Pitfall reminders

❌ Cramming two conditions into one Noul

❌ Asking in reverse, so near 1 becomes "no"

❌ Using noul as a degree scale: 0.5 is not "moderate"

❌ Fine-tuning the threshold against a handful of test cases

Advanced tips (optional)

Lesson summary

Next lesson preview