AI Customer Service Metrics: Response Rate, Automation Rate and Resolution Rate

By Tina20 Nov,2026



A dashboard can report a high automation rate while buyers keep asking the same unresolved question. The problem is often the definition, not the arithmetic. “The bot sent something”, “the bot answered correctly” and “the issue was resolved without a person” are different events.

Use a small measurement set with explicit denominators. Keep platform response scores separate from internal AI outcomes, and retain enough conversation evidence to check what the numbers mean.

1. Write a measurement dictionary

Measure A practical internal definition Do not confuse it with
Reception completeness Expected messages available in the working system Agent response speed
Meaningful response time Time until a relevant response under your defined rule Instant acknowledgement time
AI response coverage Share of defined enquiries receiving an AI substantive answer Independent resolution
Answer accuracy Reviewed answers meeting factual and applicability criteria Polite tone
Independent resolution Defined issues completed without human handling under the agreed observation rule Buyer silence
Transfer pickup Time until a person accepts an exception Time the bot announces transfer
Repeat contact Return about the same issue within a defined period Every new message from the same buyer

These are proposed internal definitions, not universal vendor or marketplace formulas. Read each platform's current definition when interpreting Shopee, Lazada or TikTok Shop metrics. Do not copy your internal denominator into an official score.

2. Use a worked example with one cohort

Suppose a fictional shop receives 1,000 distinct service issues in a month. It classifies 800 as within the assistant's approved scope. AI gives substantive answers on 500. Review finds 460 of those answers factually correct and applicable. Under the shop's defined resolution and follow-up rule, 400 issues are independently resolved.

Calculation Result Interpretation
AI answer coverage: 500 / 1,000 50% Half of all issues received a substantive AI answer
In-scope coverage: 500 / 800 62.5% Coverage within the declared scope
Reviewed answer accuracy: 460 / 500 92% Correctness of the reviewed answered set
Overall independent resolution: 400 / 1,000 40% Share of all issues completed independently
In-scope independent resolution: 400 / 800 50% Completion within the approved scope

All figures are hypothetical. The same workload can legitimately produce several different percentages. The misleading move is publishing “50% resolution” without saying it excludes the 200 out-of-scope issues.

Correctness does not equal completion. Some of the 460 correct answers may still require clarification or a person to perform an action. Do not substitute accuracy for resolution.

3. Define the unit of work

Decide whether you count messages, conversations, tickets or distinct issues. One buyer may send five messages about one parcel. Another may raise a product question and a separate refund dispute in the same thread.

Choose a rule that your team can apply consistently. If a vendor bills per conversation while your operations report per issue, retain both definitions rather than forcing them into one number.

Set an observation window for reopened work. A conversation marked finished today may return tomorrow because the answer was wrong. Avoid counting the original closure as a permanent independent success without reviewing the defined follow-up period.

4. Review accuracy with meaningful criteria

An answer should use correct facts, apply to the identified item or order, respect conditions and avoid unsupported authority. Score these separately from readability and tone.

Include negative tests: missing knowledge, ambiguous variants, expired promotions and requests requiring approval. An appropriate transfer can be the correct result even though it does not increase independent resolution.

If reviewing a sample, record how it was selected and its size. A hand-picked set of easy product questions cannot represent the entire service queue. Include shifts, shops and issue types relevant to the claim.

5. Watch the exception queue

Track the oldest waiting handoff, not only the average. A few abandoned complaints can disappear inside a good overall response-time result.

Review transfers that ask the buyer to repeat details, contain inaccurate summaries or reach an unavailable owner. These are workflow failures that may not appear in the bot's answer score.

An increase in appropriate transfers after narrowing a risky topic may be a quality improvement. Judge it alongside unsupported promises, buyer effort and eventual completion rather than treating all human involvement as failure.

6. Build a weekly review that leads to action

Use three views: whether messages arrived and received ownership, whether answers were accurate, and whether issues reached a defensible outcome. Add total operating cost when the definitions are stable.

For each adverse change, inspect representative conversations before changing settings. A drop in resolution may reflect a new product with missing knowledge, a campaign generating unusual disputes or an unstaffed transfer queue. Each needs a different response.

Duoke can be evaluated against these definitions, but confirm the current dashboard and billing events rather than assuming its labels match your internal measures. A qualifying independent-resolution charge is not automatically identical to every operational resolution definition.

Publish fewer numbers with clearer meanings. A team that can explain its denominator and show the underlying service outcome has a stronger basis for improvement than one reporting a single unexplained “AI success rate”.
 

Further reading

Android Version
iOS Version
International Edition

Contact Us

Email:[email protected]

Address:6/F MANULIFE PLACE, 348 KWUN TONG ROAD, KOWLOON, Hong Kong, 999077

Service and Support

Facebook

WeChat Official Account

Rednote

TikTok

FairCloud Technology (HK) Limited