Skip to content
TheThinkLoop
Try the playground

Template - Support

Predicate

Engineering escalation

Whether a support issue needs engineering, or can be resolved by support.

A predicate decision that separates product defects from questions and account requests, so engineers only see tickets that need them.

Try it in the playgroundCopy the template

The decision

PredicateA probability from 0 to 1 that the condition holds.

Does this support issue need to go to engineering?

Evidence checks (asked in the same request)

  • Technical failure: The message reports an error, crash, or failed request.
  • Wide customer impact: Multiple users or production systems are affected.
  • Usage question: The customer is asking how to use an existing feature.

Expected output: A probability from 0 to 1 that the issue needs engineering, plus one probability per evidence check.

Input

The Decisions API reads one input. Combine these fields into it as labeled text so the model sees the facts the decision depends on.

Input fields
FieldTypeRequiredDescription
messagestringYesThe customer's message, verbatim. Include the subject line if there is one.
recent_releasesstringNoA short list of recent releases or known incidents, so the model can connect reports to changes.

Worked examples

  1. Reproducible API error after a release

    Since yesterday's release, the /v2/invoices endpoint returns a 500 error for any request with more than 100 line items. Request ID req_8f2a. Reproducible every time.

    A careful reviewer would say: Yes. A specific, reproducible server error tied to a release is a defect support cannot fix.

    Run this example

  2. Looking for a setting

    How do I add a second admin to our workspace? I can't find the setting.

    A careful reviewer would say: No. A usage question that support or documentation can answer.

    Run this example

Expected answers are human judgments written for this template, not model output.

How it works

Every unnecessary escalation interrupts an engineer; every missed one leaves a customer stuck. This predicate estimates which side of that line a ticket falls on.

Define "needs engineering" for your own organization. If support can restart jobs or change configuration, say so in the question, or tickets that support could handle will be escalated.

The evidence checks make the result reviewable. A high probability with no technical failure detected is worth a second look.

Human review and thresholds

Recommended review policy

Let confident "No" results stay with support automatically. Have a senior agent confirm "Yes" results before a ticket reaches engineering until you have measured how often the model is right.

Choosing thresholds

Weigh the cost of an engineer's interrupted hour against the cost of a customer waiting a day for an escalation that should have happened. Teams with a rotating on-call triager can afford a lower bar than teams where escalations go straight to feature developers.

Starting automation threshold
85%
Starting don’t-decide threshold
60%

Starting points for the playground, not recommendations. The right values depend on your costs and on how the model’s confidence tracks accuracy on your data. Calibrate them.

Limitations

  • The model cannot check logs, reproduce the issue, or know about an incident unless you include that context.
  • Vague reports ("it's broken") carry little evidence either way. Expect mid-range probabilities and plan for a clarifying question.
  • Known issues with documented workarounds should usually stay with support. Supply the known-issues list in the input.

Use it in your code

The template definition is plain JSON: the question, the evidence checks, the input fields, and the starting policy. The SDK call below sends the same question and checks to the Decisions API from your server.

engineering-escalation.jsonjson
{
  "$format": "thethinkloop.template/v1",
  "slug": "engineering-escalation",
  "title": "Engineering escalation",
  "decides": "Whether a support issue needs engineering, or can be resolved by support.",
  "question": {
    "name": "decision",
    "type": "predicate",
    "instructions": "Does this support issue need to go to engineering?"
  },
  "evidenceChecks": [
    {
      "name": "factor_1",
      "label": "Technical failure",
      "type": "predicate",
      "instructions": "The message reports an error, crash, or failed request."
    },
    {
      "name": "factor_2",
      "label": "Wide customer impact",
      "type": "predicate",
      "instructions": "Multiple users or production systems are affected."
    },
    {
      "name": "factor_3",
      "label": "Usage question",
      "type": "predicate",
      "instructions": "The customer is asking how to use an existing feature."
    }
  ],
  "inputs": [
    {
      "name": "message",
      "type": "string",
      "required": true,
      "description": "The customer's message, verbatim. Include the subject line if there is one."
    },
    {
      "name": "recent_releases",
      "type": "string",
      "required": false,
      "description": "A short list of recent releases or known incidents, so the model can connect reports to changes."
    }
  ],
  "expectedOutput": "A probability from 0 to 1 that the issue needs engineering, plus one probability per evidence check.",
  "examples": [
    {
      "title": "Reproducible API error after a release",
      "input": "Since yesterday's release, the /v2/invoices endpoint returns a 500 error for any request with more than 100 line items. Request ID req_8f2a. Reproducible every time.",
      "expected": "Yes"
    },
    {
      "title": "Looking for a setting",
      "input": "How do I add a second admin to our workspace? I can't find the setting.",
      "expected": "No"
    }
  ],
  "startingPolicy": {
    "automateAt": 0.85,
    "abstainBelow": 0.6,
    "note": "A starting point for exploration, not a recommendation. Calibrate against your own labeled examples."
  }
}
Equivalent SDK calljavascript
import OpenAI from "openai"; // openai >= 7.30.0

const client = new OpenAI(); // reads OPENAI_API_KEY on the server

const decision = await client.decisions.create({
  model: "gpt-6-luna",
  input: "Since yesterday's release, the /v2/invoices endpoint returns a 500 error for any request with more than 100 line items. Request ID req_8f2a. Reproducible every time.",
  questions: [
    {
      type: "predicate",
      name: "decision",
      instructions: "Does this support issue need to go to engineering?"
    },
    {
      type: "predicate",
      name: "factor_1",
      instructions: "The message reports an error, crash, or failed request."
    },
    {
      type: "predicate",
      name: "factor_2",
      instructions: "Multiple users or production systems are affected."
    },
    {
      type: "predicate",
      name: "factor_3",
      instructions: "The customer is asking how to use an existing feature."
    }
  ],
});

// Answers come back in question order. A question can also be refused.
const [main, ...checks] = decision.answers;
if (main.type === "refusal") throw new Error("The model declined to answer.");
if (main.type === "predicate") {
  console.log(main.probability); // 0–1 estimate that the condition is true
}

Download it as engineering-escalation.json, or browse all developer resources.

Need this decision tuned to your policy?

We are running a small number of pilots with B2B teams: your policy, your examples, your thresholds. Request early access.

Related