Check that AI and agents actually finished the job.
An independent record of what each conversation actually did in your order, warehouse and payment systems, so “resolved” can’t overstate the work.
- O-8421 · cancelStop, cancel and refund all match$0.99Done
- O-8447 · address changeAddress matches the order$0.99Done
- O-8433 · cancelRefund never created, parcel shipped$108.99Reopen
The verified-completion ledger: every resolved conversation, what the record says and what it cost.
Sound familiar?
- The dashboard says resolved, and the refund never went out.
- Each vendor counts “resolved” its own way, and you’re billed on theirs.
- QA reads a sample of tickets and scores how polite they were.
You pay per resolution, plan staffing on it and report it upward, and nobody has checked it against the order.
What changes.
- Definition
- Resolved by a vendor’s ruleDone, checked in your own systems
- Coverage
- A sample read by a personEvery conversation joined to its record, with people settling disputes
- The AI judge
- A polite refund scores five starsJudged against what was actually done
- Savings
- Hours counted as dollarsHours counted, cash only when a decision changes
How it works. 6 steps, 4 stages.
- AI1
- Rules and automation3
- Your team2
- Approval
1Join
Rules and automation
Join each conversation to its episode
One customer need, across channels, matched to the order, shipment and payment records.
- Shopify
- Gorgias
- Zendesk
2Check
Rules and automation
Check the action happened
Refund created, warehouse stopped, replacement shipped, and no repeat contact in the window you chose.
AI
Judge the conversation
Scores the answer and flags where it disagrees with the record, calibrated on labeled examples.
3Review
Your teamApproval
Settle the disagreements
People adjudicate where the judge and the record disagree, and the judge learns from it.
4Fix
Your team
Fix the cause
Knowledge, policy, a tool or training, each with an owner.
Rules and automation
Report verified done by request type
A weekly record that doesn’t depend on any helpdesk’s definition.
Each defect becomes a rule, a test or a fix
What you get. Real work, every week.
| Conversation | AI quality score | What actually happened |
|---|---|---|
| O-8421 · cancel | 5 of 5 | Refund issued |
| O-8433 · cancel | 5 of 5 | Refund never created |
| O-8460 · return | 4 of 5 | Refund waiting on the return |
A calibrated judge is checked against the record. Where they disagree, a person decides.
Share of conversations marked resolved that the records confirm
Verified done by request type, from your own conversations. The weakest are where fixes start.
How you’ll know it’s working.
Verified-done share, by request type
Conversations marked resolved that the order, warehouse and payment records confirm.
False passes and false alarms, by intent
How often the judge and the record disagree, and which one was right.
Repeat contact after a resolved conversation
The simplest sign something wasn’t finished.
Baselines agreed before we build.
- Service episodes a month
- 10,000
- AI attempts
- 6,000
- Completed, then without a repeat contact
- 4,200, then 3,864
- Net hours released, after rework and QA
- 424
- Worth, at $30 an hour
- $12,720
- Added monthly cost
- $6,658
- Break-even realization
- 52%
An illustration, not a Fernway figure. If every released hour turns into a staffing or BPO decision, the month nets $6,062. If half do, it loses $298. The cost is $4,158 in per-resolution fees at $0.99, $1,500 to run and $1,000 of build, spread out. Hours become money only when a decision changes the invoice.
Works with what you have.
Support
- Gorgias
- Zendesk
- Intercom
- Observe.AI
Commerce
- Shopify
Fulfillment
- ShipBob
- We configure first
- As of October 2026, Zendesk reports contained and verified resolutions, and tools like Observe.AI score every conversation. We start from what you have, and check its verdicts against your records.
- We build when
- Vendor numbers disagree with your orders, or you need one record that doesn’t depend on any helpdesk’s definition.
What a first build includes.
Built in your accounts, around your team. Scope, fee and baseline agreed before we start.
- 1Design
- 2Build
- 3Run it, or hand it over
We build
- An episode record joining conversations to order, warehouse and payment
- Checks for the actions your AI and agents claim to finish
- A labeled test set to calibrate any AI judge
- A weekly verified-done report by request type
- A defect log with an owner for each fix
We start from
- Helpdesk, Shopify and warehouse access
- Payment or refund records
- A month of conversations and your current QA scores
From your team
- Owner
- CX lead, with finance
- Involves
- Customer service
- Finance
- Fulfillment
- Engineering
Questions teams ask.
We already have QA dashboards.
They score the conversation. This checks what happened after it, in the order, warehouse and payment records, and uses people to settle the disagreements. An AI judge can mark a polite refund that never went out as a success, so we calibrate it first.
Isn’t faster resolution better?
Not on its own. In a randomized Alibaba experiment, chats the AI was eligible to handle ran 16.8% faster. They were also rated 0.41 points lower on a five-point scale (a 2026 preprint reporting an August 2024 trial). We watch quality and repeat contact next to speed.
Will it show up as savings?
Only if a staffing or contract decision changes. The worked example shows why. We count the hours, then name the invoice or hire they’d change before calling anything cash.
Who owns it once it’s running?
You do. It’s built in your accounts, documented, with training for the people who run it. Ownership usually sits with your CX lead, with finance. We can keep running it with you, or step back.
How do we start?
Usually with a support review: two to four weeks, a fixed fee credited to the build, and a plan that’s yours. If the job is already clear, we can scope the build directly.
Part of your support engine. One of 12 use cases, in 6 parts.
See the whole support engineIn quality & proof
- Verified resolutionYou’re here
- Repeat-contact fixesTurn repeat contacts into fixes someone owns
Works with
Make “resolved” mean done. Start with a support review.
We reply within one business day.