An assistant that answers confidently but unverifiably creates more work than it saves. What matters is not the answer, but where it came from.
A knowledge assistant in customer service rarely fails by answering too little. It fails by answering too convincingly – including when there is nothing behind the answer. The difference between a useful assistant and a dangerous one is not the answer itself, but whether it can be traced.
The problem is not ignorance, it is confidence at the wrong moment
An agent who does not know something looks it up or asks. An assistant that does not know may still produce a smooth, plausible answer. That smoothness is precisely the risk: it gives no signal that anything here needs checking.
On a call this turns into a promise nobody can keep. The error surfaces at the second contact – and then it costs twice, because the wrong answer has been documented in the meantime.
Source references as a required field, not an extra
Every answer should name where it came from: policy, work instruction, tariff description, with the section. Not so that every answer gets read, but so that it can be read when someone doubts it.
The side effect matters more than the main purpose: once sources are visible, it becomes obvious how often the same outdated passage is being cited. The assistant turns into an audit tool for the knowledge base.
“I don’t know” is a good answer
An assistant that stops when there is no basis and points to the responsible team is worth more in operation than one that always says something. That boundary has to be built in deliberately, though – it does not appear on its own.
It also has to be designed so that a refusal does not look like a fault. If the team believes the system is broken whenever it declines to answer, they will work around it.
The knowledge base is the actual product
No model compensates for two documents stating different deadlines. In almost every case where an assistant is considered unreliable, the source is ambiguous rather than the model poor.
The thankless work therefore comes before rollout: merging duplicate documents, setting validity dates, assigning ownership. That work pays off even if no assistant is introduced in the end.
Measuring the right thing
The obvious metric – satisfaction with the assistant – says little. What happens after the conversation is more telling. How often is the same case reopened? How often is an answer corrected later? How often does the team find the answer faster without the tool?
If the reopening rate falls, the assistant is working. If it stays flat while everyone is satisfied, you have a pleasant tool with no effect.
