all articles
ai · 5 min read

Customer-support chatbots: design, evaluation and human handoff

Design a support chatbot around approved knowledge, test answer quality and privacy, and define when customers should reach a human agent.

A support chatbot is a promise made in your company's name, at any hour, to people who are often already annoyed. The useful questions are narrow ones: what the bot is allowed to answer, where its answers come from, how you know they are right, and when it hands the customer to a person. This guide covers those four, in that order.

How a support bot produces an answer

Vendors blur three different mechanisms, and the difference decides how you maintain the bot.

Inference. A language model writes a reply from whatever text it is given at that moment: your instructions, the customer's message and any context you attach. Nothing is stored in the model by doing this.

Retrieval. Before the model replies, the system searches a set of documents you control (help articles, policies, product sheets) and passes the most relevant passages to the model. Most business support bots work this way. To change an answer, you change the document.

Retraining or fine-tuning. Someone deliberately prepares a dataset and runs a training job to change the model's behavior. It is a project with an owner, a dataset and a test, not something that happens in the background.

A common sales claim is that the bot "learns from every conversation." Treat that as a question for the vendor, not a feature. Ask whether customer chats are used to train any model, whether you can switch that off, and how long chats are kept. As one example of what the answer can look like, OpenAI's platform documentation says data sent to its API is not used to train its models unless the customer opts in, and that abuse-monitoring logs are kept for up to 30 days by default. Your vendor's terms may differ; get them in writing.

For your own planning, assume the bot knows what is in its approved sources today and nothing more. Improving it means improving those sources and your instructions.

Design around approved knowledge

Start with a scope document, one page long:

Answers: the topics the bot handles, each linked to the source document that backs it
Refuses or hands off: refunds, contract changes, complaints, anything involving health, legal or safety questions, and anything requiring access to a customer's account unless you have built that deliberately
Owner: the person who approves each source document and the date it was last reviewed
Tone: a few example replies written by your best support person

Then clean the sources. Contradictory articles produce contradictory answers. If your refund policy exists in three versions, the bot will find all three.

Instruct the bot to answer only from the retrieved sources and to say it does not know when they do not cover the question. NIST's Generative AI Profile lists confabulation, meaning confidently stated but false content, among the core risks of generative AI. Retrieval reduces it; it does not remove it.

The cost of getting this wrong is real. In Moffatt v. Air Canada (2024), a British Columbia tribunal held the airline responsible after its website chatbot gave a customer incorrect information about bereavement fares, and rejected the argument that the chatbot was responsible for its own statements. The amount was small. The point is that customers and tribunals treat the bot's words as yours.

Test answer quality before launch

Build a test set from real tickets, not invented ones. Pull 50 to 100 past questions across your common topics, plus the awkward ones.

Grade each answer on four criteria:

Correct: matches current policy.
Grounded: the claim appears in an approved source.
Safe: no promises the business has not made, no personal data repeated back unnecessarily.
Right route: handed off when it should have.

Add questions designed to break it: a policy that does not exist, a customer asking for an exception, a message in another language, someone pasting a card number. Write down the pass rate you will accept before you see results, and rerun the full set every time you change sources, instructions or the underlying model.

Privacy checks

Customers type things into chat boxes that they would never put in a form. Before launch:

Decide what the bot should never ask for, and what it should do if a customer volunteers it (card numbers, health details, passwords).
Check where chats are stored, who can read them and how long they are kept.
Confirm the vendor's training and retention terms, as above.
Update your privacy notice to describe the chatbot.
Tell people they are talking to an AI. The EU AI Act's transparency article requires that people be informed they are interacting with an AI system unless that is obvious, and the European Commission says the Act's transparency rules come into effect in August 2026. How this applies to you depends on your role and market; check with counsel.

Define the human handoff

The handoff is the part customers remember. Define the triggers in advance:

Trigger	What happens
Customer asks for a person	Hand off immediately, no retention attempt
Topic is on the refuse list	Explain briefly, hand off
Bot cannot find a source for the answer	Say so, offer a person
Same question asked twice, or clear frustration	Hand off
Anything involving money, cancellation or a complaint	Hand off
Outside staffed hours	Create a ticket, give a realistic response time

When the bot hands off, pass the whole conversation and what it already tried, so the customer does not repeat themselves. Make sure someone actually receives it: a handoff to an unmonitored inbox is worse than no bot.

After launch

Read a sample of conversations every week, especially the ones that ended in a handoff or with no reply from the customer. Each wrong or missing answer points to a source document to fix or a topic to add to the refuse list. Keep a short change log of source edits and instruction changes so you can trace why an answer changed.

A safer first step for many teams is a bot that drafts replies inside the helpdesk for an agent to approve. You get the test data and the source cleanup without putting unreviewed answers in front of customers.

Where automotif fits

This is AI integration work: an AI step inside your existing helpdesk or inbox, grounded in your approved documents, with a person approving before anything goes out until the test results justify more. If you are weighing a support bot, book a free call of 30 to 60 minutes and we can look at your tickets and sources together.