Guides
How to build AI decisions you can trust
Technical, practical, and honest about limitations. Written for the engineer wiring up decision systems and for the operations lead deciding what to automate.
Using the Decisions API
How the API works, which question type to use, and how it compares with Structured Outputs.
Decisions API tutorial: requests, answers, and the parts that are easy to get wrong
Start here. The request and response shapes, how to read each answer type safely, asking several questions about one input, and what the API does not do.
9 min read
Predicate vs choice vs score: choosing the right decision type
Most decisions can be phrased three ways, and each returns different information. Which type fits which decision, with worked examples and common mistakes.
7 min read
Decisions API vs Structured Outputs: which to use for classification
Both can classify a ticket. They return different information. Where confidence comes from in each, and a design that uses both.
7 min read
Trusting a decision
What LLM confidence means, when to abstain, how to evaluate a classifier, and how to set thresholds from evidence.
Confidence thresholds: when to automate, review, or not decide
Probability, confidence, accuracy, calibration, and correctness are different things. How to set a review policy from the cost of each kind of mistake.
8 min read
Abstention: when an AI classifier should not decide
Abstaining is different from review and from refusal. The coverage trade-off, setting the threshold band by band, signals beyond confidence, and owning the queue.
8 min read
How to evaluate AI decisions before you automate them
Accuracy alone hides the mistakes that matter. How to build a test set, which numbers to report, how to read failures, and when an evaluation is good enough to act on.
9 min read
How to calibrate AI decisions with labeled examples
The procedure for checking whether confidence means what it says: how much data you need, what to compute, and how to read the result without fooling yourself.
9 min read
Measuring calibration: reliability diagrams, ECE, and the Brier score
Three ways to measure calibration, what each one hides, and which to use for which question, with code that computes them from labeled decisions.
8 min read
Running decisions in operations
Where human review fits, and how AI routing turns several judgments into one action.
Human-in-the-loop AI: when AI should decide and when it should ask a person
When should AI decide and when should it ask? Review patterns, what reviewers need to see, sizing the queue, and keeping oversight real.
8 min read
AI decision routing: from one-off judgment calls to an operational decision system
A routing decision is several smaller decisions plus a rule that combines them. How to design that rule, where people fit, and what to log.
8 min read