Skip to main content
Confidence scores indicate how certain your agent is about each response. They’re calculated based on how well the user’s question matches your knowledge base content, helping you identify where your agent might need more training data.

​
How Confidence Works

When a user asks a question:
  1. Vector search — The question is compared against your knowledge base
  2. Match scoring — Each document gets a similarity score (0-100%)
  3. Confidence calculation — The highest match score becomes the confidence
  4. Response generation — The AI uses matched documents to answer

​
Score Ranges

​
Viewing Confidence Scores

​
In Conversations

  1. Go to Conversations
  2. Open a conversation
  3. Each assistant message shows its confidence score
Messages with low confidence are highlighted so you can easily spot potential issues.

​
In Analytics

  1. Go to Analytics
  2. View the Confidence section:
    • Average confidence over time
    • Distribution (how many responses at each level)
    • Trend (improving or declining)

​
On Agent Cards

The main agents list shows average confidence for each agent, giving you a quick health check.

​
Interpreting Scores

​
High Confidence (80-100%)

✅ What it means: Your knowledge base contains relevant information for this question. Example:
User: “What are your business hours?” Agent: “We’re open Monday to Friday, 9 AM to 6 PM.” (95% confidence)
The agent found a direct match in your documentation.

​
Medium Confidence (60-79%)

⚠️ What it means: The agent found related content but may be extrapolating. Example:
User: “Can I return an item after 60 days?” Agent: “Our return policy is 30 days from purchase.” (68% confidence)
The agent found return policy info but the specific 60-day scenario wasn’t documented.

​
Low Confidence (40-59%)

🟡 What it means: Limited relevant content found. The answer may be based on general knowledge. Example:
User: “Do you offer enterprise pricing?” Agent: “Please contact our sales team for enterprise options.” (45% confidence)
Consider adding enterprise pricing documentation.

​
Very Low Confidence (0-39%)

🔴 What it means: The agent is essentially guessing or using fallback responses. Example:
User: “What’s your CEO’s favorite color?” Agent: “I don’t have that information.” (12% confidence)
This is expected for off-topic questions.

​
Using Confidence for Automation

​
Email Channel

For the email channel, confidence scores drive automation: Configure the threshold in Channels → Email → Settings.

​
Improving Confidence

​
Add Missing Content

  1. Filter conversations by low confidence
  2. Identify common low-confidence questions
  3. Add relevant documentation to knowledge base

​
Review Low-Confidence Responses

Weekly routine:
  1. Go to Conversations
  2. Sort by confidence (low to high)
  3. Review responses
  4. Add missing info to knowledge base
In Analytics, monitor:
  • Is average confidence increasing over time?
  • Which topics have lowest confidence?
  • Are there sudden drops after knowledge base changes?

​
Confidence vs Accuracy

High confidence doesn’t guarantee accuracy. It means the agent found relevant content, not that the response is correct.

​
When Confidence Can Mislead

  1. Outdated content — High match to old, incorrect info
  2. Similar but wrong — Question matches related but different topic
  3. Partial information — Good match but incomplete answer

​
Best Practice

Combine confidence monitoring with:
  • User feedback (thumbs up/down)
  • Regular content audits
  • Human review of edge cases

​
API Access

Access confidence data programmatically:

​
Per-Message Confidence

Response includes confidenceScore for each assistant message.

​
Aggregate Analytics

​
Filtering Conversations

Find conversations that need attention:
  1. Go to Conversations
  2. Use filters:
    • Min Confidence — Show only above threshold
    • Max Confidence — Show only below threshold
  3. Review and improve knowledge base

​
Best Practices

Set a weekly calendar reminder to review low-confidence conversations. This is the fastest way to improve your agent.
  1. Start with 70% threshold — Adjust based on your needs
  2. Review weekly — Check low-confidence conversations
  3. Track trends — Monitor average confidence over time
  4. Combine with feedback — Use both metrics together
  5. Update regularly — Keep knowledge base current

​
Next Steps

Knowledge Base

Add content to improve confidence

Conversation History

Review past conversations