The short version of setting an AI support confidence threshold is that one global number does not work. You set a per-intent threshold, calibrate it against measured accuracy rather than the model's self-reported score, start conservative, loosen it as the audit trail builds trust, and keep a replayable escalation log so every handoff can be reviewed. A good threshold is a policy, not a dial.
Why bother being this precise? Because RLHF-trained models are miscalibrated: their stated confidence often overstates actual accuracy, and a claimed 90% can correspond to roughly 75% real accuracy. If the AI tells you it is 85% confident, trust the number only after you have measured whether 85 means 85.
What a confidence score actually is
A confidence score is the model's own estimate of how certain it is about a reply. Most support platforms expose it as a 0 to 1 or 0 to 100 number per answer. The routing rule reads it: when the score falls below a defined threshold, the system can route the interaction to a human agent, request clarification from the customer, or present the response with a disclaimer.
Two kinds of score exist in practice. One is the model's internal probability (logit-based); the other is a verbalised number the model writes out ("I am 80% sure"). Verbalised confidence is what most commercial APIs expose, and it is less reliable: verbalised LLM confidence can show ECE above 0.377, far worse than logit-based probabilities. ECE is Expected Calibration Error. Zero means stated confidence matches real accuracy exactly; below 0.05 on a labelled test set is a reasonable production target.
Why one global threshold fails
Picture a single AI support agent set to 0.80. On "what is your return window", the model answers correctly at 0.78 and gets forced into a handoff: a wasted human minute. On "cancel my subscription and refund today's charge", the model answers at 0.82 and the shopper is lost on a wrong answer: a customer-support failure. Same threshold, two bad outcomes.
The fix is to split by intent. A per-intent playbook recommends setting a higher threshold on refunds, billing, and cancellations, around 0.4 above the general line, and loosening on informational intents. The cost of being wrong on a refund is a refund; the cost of being wrong on "what are your opening hours" is a correction. Treat them differently.
Step 1: Group every intent by error cost
List the intents the agent handles. Score each one on two dimensions, 1 to 3:
- Reversibility of a wrong answer. 1 = a correction a human reads later fixes it (opening hours). 3 = money has moved or a decision has been locked (refund, cancellation, address change).
- Visibility of a wrong answer. 1 = the shopper might not notice. 3 = the shopper will complain or churn.
The product gives a 1 to 9 cost rank. Six or above gets a tight threshold. Three or below gets a loose one.
A typical ecommerce split:
| Category | Reversibility | Visibility | Cost rank | Threshold target | | --- | --- | --- | --- | --- | | Opening hours, FAQ | 1 | 1 | 1 | 0.55 | | Order status (WISMO) | 2 | 2 | 4 | 0.70 | | Returns eligibility | 2 | 3 | 6 | 0.80 | | Refund over threshold | 3 | 3 | 9 | 0.92 | | Account cancellation | 3 | 3 | 9 | 0.92 |
The threshold column is a starting point; the real values come from calibration.
Step 2: Calibrate against measured accuracy
The stated confidence is a starting point, not the truth. Build a labelled set of 100 to 300 conversations per intent, with the correct answer marked. Run the agent against it and record both the confidence score and whether the answer was right.
Bin the results by score bucket: 50 to 60, 60 to 70, and so on. For each bucket, compute the real accuracy. If 100 answers landed in the 80 to 90 bucket and 70 were correct, the stated 85 means a real 70. Customer service can tolerate effective confidence thresholds in the 80 to 85% range, versus 90 to 95% for financial services, defined in terms of measured accuracy, not stated confidence.
So the threshold is not "set it to 0.80". It is "find the stated score at which real accuracy crosses 80%, then use that as the handoff line". For many general-purpose models that stated score is 0.90 or higher.
Our note on auditing your AI agent's answers describes a weekly rubric that supplies the labelled data for this calibration without a separate project.
Step 3: Start conservative, loosen with evidence
Begin with a conservative threshold, watch for unnecessary handoffs, and relax the threshold as trust builds. This is counter-intuitive if you joined to save handoff cost, but a false handoff is cheap (an agent reads a reply the AI could have sent) while a false answer is expensive (money moves, trust drops).
A workable cadence: set the opening threshold at the stated score where measured accuracy equals your tolerance plus five percentage points. Review handoff reasons weekly. When a handoff bucket repeatedly turns out to be answers the AI could have given, loosen the threshold for that intent by 2 to 3 points. Never loosen without that evidence.
Step 4: Add signals that override confidence
A confidence rule alone is thin. Good designs combine confidence thresholds with sentiment signals, direct "talk to a human" requests, and compliance rules. These override the confidence score regardless of how high it is.
Three overrides worth every team's attention:
- Direct human request. The shopper writes "talk to a human", "agent please", or the equivalent in another language. Handoff immediately, no matter the AI's confidence. Direct human-handoff requests should be honored on the first message.
- Sentiment flag. Anger, frustration, or escalation vocabulary ("furious", "unacceptable", "my lawyer") routes to a human. Confidence is irrelevant here; the human handles the emotional conversation, the AI handles the next one.
- Compliance category. Fraud, legal disputes, account takeover, GDPR data requests, chargebacks. Always human. See when not to automate a ticket for the full list.
Step 5: Log every escalation so each one is replayable
Each escalation should be replayable: which signal fired, what the threshold was, and whether the human agreed with the handoff decision. Without this log you cannot tell whether your thresholds are right or wrong.
A minimum log entry per handoff:
- Intent classification.
- The AI's stated confidence.
- The threshold that fired.
- The rule name that fired (confidence rule, direct request, sentiment, compliance).
- The human's final answer, with a flag for whether it matched the AI's draft.
Review this log weekly. When the human agrees with the AI's draft on 95% of handoffs in a given intent, the threshold is too conservative. When the human disagrees on 20%, the threshold is too loose or the model needs a source refresh. Our note on designing the AI-to-human handoff walks through the full handoff log and the context the human needs on takeover.
Step 6: Watch for drift
Thresholds that were right in June can be wrong in October because sources changed, the model was updated, or the question mix shifted. Three cheap drift signals:
- Handoff rate per intent. A sudden rise means the model started refusing more often. A sudden drop can mean a source went stale and the model is answering confidently from old data.
- Reopen rate per intent. Rising reopens inside a confidence band means the threshold is now too low for that intent.
- Effective-accuracy gap. Rerun the labelled set monthly. When measured accuracy at the current threshold moves more than 2 percentage points from baseline, recalibrate.
Hallucination rates for grounded support agents should sit well under 1%; if the effective accuracy at your threshold starts drifting up on hallucination-prone intents, lower the threshold until the source is fixed.
How Keloa approaches confidence thresholds
Keloa's flow builder lets a team set per-intent thresholds and override rules side by side, with the agent's stated score, the measured-accuracy curve from the audit log, and the handoff reason visible on every conversation in the unified inbox. The AI agents ground every answer in your connected sources and cite them, so a measured-accuracy check becomes a filter in the inbox, not an export. When sources are incomplete for a request, the agent declines rather than guesses, which keeps the confidence rule honest.
For teams starting on this, our walk-throughs on reducing AI hallucinations and auditing AI agent answers cover the surrounding practice.
Frequently asked questions
Should the threshold be the same across languages? No. Measured accuracy varies by language, especially on long-tail terms. Calibrate separately for each language in production volume. Many teams end up with a slightly tighter threshold on secondary languages until the knowledge base coverage catches up.
Can we just use the model's self-reported confidence directly? Not safely. Treat the self-reported number as an input, map it to measured accuracy on your own labelled set, and set the threshold against the measured curve. This gap is why calibration exists.
How often should we recalibrate? Full recalibration quarterly, and lightweight checks weekly through the handoff log. Trigger an immediate recalibration after any model upgrade, major prompt change, or significant source refresh.
What is a sensible handoff rate target? There is no universal answer. For an ecommerce brand with mostly WISMO and returns, a 15 to 30% handoff rate is common. The right number is the one where your reopen rate stays flat and your CSAT per conversation holds. See CSAT for AI-handled tickets for the measurement setup.
Should confidence thresholds vary by customer tier? Yes if the business justifies it. VIP accounts tend to warrant a tighter threshold (more human attention). New or anonymous visitors tolerate a looser one. Encode the tier as part of the intent classification so the rule stays legible.
What does it look like when the threshold is wrong? Too tight: humans keep answering questions the AI's draft already got right, and the shopper waited for a slower reply with no benefit. Too loose: reopens rise and the handoff log shows the human disagreeing with the draft on refunds or address changes. Both show up in a week of handoff-log review.
Want a working threshold policy for your own tickets? Book a demo and we will walk through the audit log, calibration curve, and override rules against your own data.
Sources
- Decagon: What is a confidence score
- Tianpan: LLM confidence calibration in production
- Tianpan: Confidence-accuracy inversion in production LLMs
- Tianpan: LLM calibration production overconfidence
- Fini: AI support confidence threshold tuning playbook
- Fini: Bot-human escalation rules
- Eesel: AI chat escalation
- Aloware: AI-to-human handoff
- Suprmind: AI hallucination rates and benchmarks