AI Agent + Human Agent Metrics in One Dashboard
You deployed an AI agent. It’s resolving tickets. Your team is handling the rest. And now you have two separate reporting surfaces—one for bots, one for humans—that don’t talk to each other.
That split creates blind spots. You can’t answer basic questions like: What percentage of our total support is handled by AI vs humans? Is AI actually reducing agent workload, or just intercepting easy tickets while the hard ones pile up?
A unified dashboard fixes this. Here’s what to track together and how to build it.
The reporting split problem
Zendesk reports AI agent performance in a dedicated dataset (AI agents – Essential or Advanced) and human agent performance in the Support – Tickets dataset. These datasets have different metrics, different time ranges, and different filters.
Out of the box, there’s no single view that shows:
- Total support volume (AI-resolved + human-resolved + dropped)
- Resolution time for AI vs human
- Whether AI is reducing human backlog or just handling incremental volume
- CSAT for AI-resolved vs human-resolved conversations
Without this unified view, AI investment is hard to evaluate and harder to optimize.
The metrics that matter together
1. Total resolution breakdown
The most important metric is the simplest: what percentage of total support interactions are resolved by AI vs humans?
| Segment | Source | Metric |
|---|---|---|
| AI-resolved | AI agents dataset | Automated resolutions |
| Human-resolved | Support – Tickets | Solved tickets |
| Dropped/abandoned | AI agents dataset | Dropped conversations |
| Escalated (AI → human) | AI agents dataset | Escalations |
Why it matters: If AI handles 40% of conversations but 25% of those are “dropped” (neither resolved nor escalated), your real AI resolution rate is lower than the headline number. Tracking all four segments gives you the honest picture.
2. Human workload trend
The promise of AI agents is fewer tickets for humans. Verify it.
Track ticket volume for human-handled tickets over time. If AI is working, this number should decline (or at least grow slower than total inbound volume). If it’s flat while total volume grows, AI is absorbing the growth—still valuable but different from “reducing agent workload.”
3. Escalation quality
When an AI agent hands off to a human, what happens? Track:
- Escalation rate — What percentage of AI conversations become human tickets?
- Escalated ticket FRT — How fast does a human pick up an AI-escalated ticket? Is there a routing gap?
- Escalated ticket resolution time — Are escalated tickets harder to resolve? (They often are, because easy ones were already deflected.)
If escalated tickets have significantly worse FRT or resolution time than organic tickets, there may be a routing problem: escalated tickets may sit in a queue while agents don’t realize they’re time-sensitive.
4. CSAT comparison
CSAT for AI-resolved conversations vs human-resolved tickets tells you whether customers are equally satisfied with both channels. In Zendesk:
- AI agents can send BSAT (bot satisfaction) surveys.
- Support tickets use the standard CSAT survey.
These are measured differently, so direct comparison requires care. But directionally, if BSAT is 70% and CSAT is 90%, there’s a quality gap in the AI experience worth investigating.
5. Cost per resolution
If you track cost per ticket, extend it to cost per resolution across both channels:
- Human cost per resolution = Agent cost (salary + tools) ÷ human-resolved tickets
- AI cost per resolution = AI agent subscription cost ÷ automated resolutions
The ratio between these two numbers is your AI ROI in a single metric. For most teams, AI cost per resolution is 5–20x cheaper than human—but only for the ticket types AI can actually handle.
How to build the unified dashboard
Option A: Side-by-side Explore tabs
The simplest approach in Zendesk Explore:
- Create a dashboard with two tabs: “AI Performance” and “Agent Performance.”
- Add a third tab “Combined View” with headline KPIs from both datasets: total volume (AI + human), AI resolution %, human ticket volume trend, and CSAT comparison.
- Use text widgets on the combined tab to manually pull in key numbers, since cross-dataset queries are limited in Explore.
Limitation: Explore can’t natively join the AI agents dataset and Support dataset in a single report. You’ll need separate reports side by side.
Option B: Export and combine
For a true merged view:
- Export AI agent metrics (automated resolutions, escalations, dropped conversations by week) from the AI agents Explore dataset.
- Export human ticket metrics (solved tickets, FRT, resolution time, CSAT by week) from the Support dataset.
- Combine in a spreadsheet or BI tool to create unified charts.
This is more work but produces the cleanest cross-channel analysis.
Option C: Use a third-party dashboard
Tools like TicketBoard pull data from Zendesk’s API and can display AI and human metrics in a single view without the dataset-joining limitation of Explore. If you’re already using a third-party dashboard, check whether it supports AI agent data.
Common mistakes
- Celebrating automated resolutions without checking drop-offs — A conversation where the user simply stopped responding isn’t a real resolution. Audit a sample of “automated resolutions” to verify quality.
- Comparing FRT across channels — AI agents respond in seconds; humans in minutes or hours. Comparing them directly isn’t useful. Instead, compare FRT for human tickets before vs after AI deployment to measure workload relief.
- Ignoring the escalation path — If AI escalations don’t include conversation context, agents waste time re-asking questions. Track escalated ticket handle time separately to catch this.
- Missing category-level analysis — AI might crush it on password resets but fail on refund requests. Aggregate numbers hide this. Break down AI resolution rate by contact reason or intent.
What to review weekly
Add three questions to your weekly support ops review:
- What % of total volume did AI resolve this week? Trend over the past 4 weeks. Is it growing, stable, or declining?
- Did human ticket volume decrease? If AI resolution is rising but human volume is flat, total demand is growing. You’re running in place, not gaining ground.
- Any BSAT or CSAT changes? A sudden drop in either metric signals a problem—new contact reasons AI can’t handle, or agents struggling with harder escalated tickets.
FAQ
We just launched AI agents. When should we build this dashboard? Give it 2–4 weeks to accumulate meaningful data, then build the combined view. Start simple: AI resolution % and human volume trend. Add CSAT comparison and escalation metrics once you have enough data points.
What if our AI resolution rate is low? Low AI resolution usually means content gaps (the AI doesn’t have answers for the top contact reasons) or configuration issues (the AI isn’t deployed on the right channels). Check the AI agent Insights dashboard for “no answer” intents and prioritize content for the top 5.
Should we set targets for AI resolution rate? Yes, but realistic ones. A brand-new AI agent might resolve 10–15% of conversations. A well-tuned one with strong help center coverage can hit 40–60%. Set quarterly targets and tie them to content improvements rather than arbitrary numbers.