By Tina20 Nov,2026
A dashboard can report a high automation rate while buyers keep asking the same unresolved question. The problem is often the definition, not the arithmetic. “The bot sent something”, “the bot answered correctly” and “the issue was resolved without a person” are different events.Use a small measurement set with explicit denominators. Keep platform response scores separate from internal AI outcomes, and retain enough conversation evidence to check what the numbers mean.
1. Write a measurement dictionary
| Measure | A practical internal definition | Do not confuse it with |
|---|---|---|
| Reception completeness | Expected messages available in the working system | Agent response speed |
| Meaningful response time | Time until a relevant response under your defined rule | Instant acknowledgement time |
| AI response coverage | Share of defined enquiries receiving an AI substantive answer | Independent resolution |
| Answer accuracy | Reviewed answers meeting factual and applicability criteria | Polite tone |
| Independent resolution | Defined issues completed without human handling under the agreed observation rule | Buyer silence |
| Transfer pickup | Time until a person accepts an exception | Time the bot announces transfer |
| Repeat contact | Return about the same issue within a defined period | Every new message from the same buyer |
These are proposed internal definitions, not universal vendor or marketplace formulas. Read each platform's current definition when interpreting Shopee, Lazada or TikTok Shop metrics. Do not copy your internal denominator into an official score.
Suppose a fictional shop receives 1,000 distinct service issues in a month. It classifies 800 as within the assistant's approved scope. AI gives substantive answers on 500. Review finds 460 of those answers factually correct and applicable. Under the shop's defined resolution and follow-up rule, 400 issues are independently resolved.
| Calculation | Result | Interpretation |
|---|---|---|
| AI answer coverage: 500 / 1,000 | 50% | Half of all issues received a substantive AI answer |
| In-scope coverage: 500 / 800 | 62.5% | Coverage within the declared scope |
| Reviewed answer accuracy: 460 / 500 | 92% | Correctness of the reviewed answered set |
| Overall independent resolution: 400 / 1,000 | 40% | Share of all issues completed independently |
| In-scope independent resolution: 400 / 800 | 50% | Completion within the approved scope |
All figures are hypothetical. The same workload can legitimately produce several different percentages. The misleading move is publishing “50% resolution” without saying it excludes the 200 out-of-scope issues.
Correctness does not equal completion. Some of the 460 correct answers may still require clarification or a person to perform an action. Do not substitute accuracy for resolution.
Decide whether you count messages, conversations, tickets or distinct issues. One buyer may send five messages about one parcel. Another may raise a product question and a separate refund dispute in the same thread.
Choose a rule that your team can apply consistently. If a vendor bills per conversation while your operations report per issue, retain both definitions rather than forcing them into one number.
Set an observation window for reopened work. A conversation marked finished today may return tomorrow because the answer was wrong. Avoid counting the original closure as a permanent independent success without reviewing the defined follow-up period.
An answer should use correct facts, apply to the identified item or order, respect conditions and avoid unsupported authority. Score these separately from readability and tone.
Include negative tests: missing knowledge, ambiguous variants, expired promotions and requests requiring approval. An appropriate transfer can be the correct result even though it does not increase independent resolution.
If reviewing a sample, record how it was selected and its size. A hand-picked set of easy product questions cannot represent the entire service queue. Include shifts, shops and issue types relevant to the claim.
Track the oldest waiting handoff, not only the average. A few abandoned complaints can disappear inside a good overall response-time result.
Review transfers that ask the buyer to repeat details, contain inaccurate summaries or reach an unavailable owner. These are workflow failures that may not appear in the bot's answer score.
An increase in appropriate transfers after narrowing a risky topic may be a quality improvement. Judge it alongside unsupported promises, buyer effort and eventual completion rather than treating all human involvement as failure.
Use three views: whether messages arrived and received ownership, whether answers were accurate, and whether issues reached a defensible outcome. Add total operating cost when the definitions are stable.
For each adverse change, inspect representative conversations before changing settings. A drop in resolution may reflect a new product with missing knowledge, a campaign generating unusual disputes or an unstaffed transfer queue. Each needs a different response.
Duoke can be evaluated against these definitions, but confirm the current dashboard and billing events rather than assuming its labels match your internal measures. A qualifying independent-resolution charge is not automatically identical to every operational resolution definition.
Publish fewer numbers with clearer meanings. A team that can explain its denominator and show the underlying service outcome has a stronger basis for improvement than one reporting a single unexplained “AI success rate”.

Email:[email protected]
Address:6/F MANULIFE PLACE, 348 KWUN TONG ROAD, KOWLOON, Hong Kong, 999077