Developer resources
Templates and datasets as plain JSON
Everything on the templates and benchmarks pages, in machine-readable form. Use the templates in your own code; run the datasets through your own stack.
Decision templates
Format thethinkloop.template/v1: the question (named decision), the evidence checks as predicates, the input fields, two worked examples with expected answers, and a starting review policy. Free to copy and adapt.
import OpenAI from "openai";
const template = await fetch("https://www.thethinkloop.com/resources/templates/refund-eligibility.json")
.then((response) => response.json());
const client = new OpenAI(); // server-side only
const decision = await client.decisions.create({
model: "gpt-6-luna",
input: customerMessage,
questions: [
template.question,
...template.evidenceChecks.map(({ name, type, instructions }) => ({ name, type, instructions })),
],
});Benchmark datasets
Format thethinkloop.benchmark/v1: the exact question, the label set, a written labeling guide, and cases with an expected answer, a difficulty, and a note for hard cases. Synthetic messages only. Licensed CC BY 4.0. See the benchmark method.
| Dataset | Type | Cases | File |
|---|---|---|---|
| Refund eligibility v1.0.0 | Predicate | 26 | refund-eligibility.json |
| Team routing v1.0.0 | Choice | 28 | team-routing.json |
| Ticket urgency v1.0.0 | Score | 24 | ticket-urgency.json |
| Engineering escalation v1.0.0 | Predicate | 24 | engineering-escalation.json |