A chat bubble can be installed in an afternoon. Deciding what it should say when a customer asks for an exception takes longer, usually because the answer belongs to the business rather than the software.
For a small website or online store, a useful AI customer support assistant starts with a narrow set of questions, approved information, clear access rules and a reliable route to a person. A fluent answer is valuable only when it helps the customer act on information the business can stand behind.
This guide proposes a practical pilot for a US-facing business. It is not a report of a deployed RichFolds assistant, a comparison of paid chatbot plans or a claim that automation will reduce support costs. Documentation was checked on 17 October 2026. The examples below are fictional.
Choose the first job carefully
Start with something the assistant can explain from stable, public information. Product-care instructions, service preparation steps or the location of a published policy can make a more manageable pilot than order changes, refunds or account disputes.
Write a boundary sentence. For example: “The pilot explains approved care instructions for the current catalog and directs other questions to support.” That sentence is less glamorous than “an intelligent assistant for the entire customer journey,” but it gives the project a testable scope.
Review actual support themes with the people answering enquiries. Do not upload a collection of customer messages into a new service merely because they would make convenient examples. Use authorized, appropriately handled material or fictional questions that preserve the problem without exposing the person.
A good first question is repetitive enough to be worth helping with and bounded enough to verify. If nobody can agree on the correct answer, the first task is editorial work. The model should not become the temporary head of company policy.
Make a source register
Create a small register of the information the assistant is allowed to use. Each entry needs a title, owner, location, review date, intended audience and current status. Identify which version is authoritative when two pages disagree.
For a fictional clothing store, the register might contain product-care pages, a delivery explanation and a returns policy. An abandoned campaign page would stay out of the pilot even if its reassuring language made it easy to retrieve.
Separate public information from information that requires an authenticated customer or an internal role. A public assistant answering care questions does not need access to a customer database. Expanding its access “in case we need it later” makes the pilot harder to evaluate and maintain.
Keep the source register understandable to the content team. If changing a care instruction requires three people to decode an integration diagram, the assistant's information will eventually drift away from the website it is meant to explain.
Understand retrieval without treating it as a guarantee
Retrieval-augmented generation, often shortened to RAG, combines a search or retrieval step with a language model's response. The purpose is to provide relevant material from a chosen information collection rather than relying only on the model's general knowledge. Microsoft's RAG overview describes this pattern and the practical challenges around retrieval, content preparation and access.
For the business owner, the important questions are concrete. Did the system retrieve the right policy? Was it the current version? Did the generated answer preserve its conditions? Could the customer open a useful source link?
Retrieving a relevant paragraph does not prove that every sentence in the answer is correct. Google's grounding-check documentation distinguishes an answer candidate from the facts used to support it. That distinction is worth retaining even if your implementation uses a different provider.
Ask the developer to show the evidence path for one question: selected source, retrieved passage, generated answer and displayed citation. This is easier to inspect than a product demonstration in which every question conveniently has a perfect answer.
Design a useful answer contract
An answer contract describes what a satisfactory response must contain. Keep it short enough that a reviewer can apply it consistently.
For the clothing-store pilot, a proposed contract could require the assistant to identify the relevant product, summarize only the approved care advice, link to the source and acknowledge when the available information does not answer the question. It would prohibit inventing a product guarantee or approving a return.
The contract also needs a fallback. “I don't have an approved answer for that product; here is how to contact the team” can be a successful response. It is a better outcome than a confident guess delivered in a charming tone.
Decide how the assistant asks for clarification. If several products have similar names, it should request the missing product detail rather than silently choosing one. Ask only for information needed for the task, and explain why it is relevant.
Keep answering and taking action separate
Explaining a returns policy is different from issuing a refund. Explaining delivery options is different from changing an address on an order. A pilot can be useful without permission to perform either action.
Document the difference in the interface as well as in the implementation. A button labelled “Check my options” should not quietly become an authorization to modify an order. Any future action flow needs its own identity checks, permissions, confirmation rules and failure handling.
Access controls must be enforced by the application and retrieval layer. A friendly instruction telling a model not to reveal private information is not a substitute for restricting what it can retrieve. Microsoft's secure RAG architecture guidance discusses authorization filters in retrieval requests; the general implementation question is whether each request receives only the information that requester may access.
For an early public pilot, keep the data boundary simple. Use approved public material, avoid account-specific lookups and make the route to a person obvious. More capability can be added after the narrower system has evidence behind it.
Illustrative workflow, not a client result or completed test.
Build a question set before choosing the launch date
A useful pilot needs more than a handful of friendly prompts. Assemble questions covering routine answers, ambiguity, missing information and requests outside the assistant's authority.
| Question type | Fictional example | What the reviewer should inspect |
|---|---|---|
| Supported fact | How should I wash this cotton shirt? | Correct product and current source |
| Ambiguous request | Can I clean it normally? | Useful clarification rather than a guess |
| Missing information | Is this suitable for a process the care page does not mention? | Honest limitation and handoff |
| Policy exception | Can you approve a return after the published window? | No invented approval |
| Conflicting content | Two indexed pages give different advice | Detection or escalation of the conflict |
| Untrusted instruction | A retrieved page tells the assistant to ignore its rules | Source text treated as evidence, not authority |
This table is a proposed evaluation design, not a set of completed test results. Add the language and phrasing your audience actually uses. A support assistant that understands only the exact headings in your knowledge base is effectively asking customers to learn its filing system.
Score the behavior, not the confidence of the prose
Evaluate whether the answer is supported, relevant, appropriately limited and useful. Also inspect the source link and the next step offered to the customer. An elegant paragraph with an unrelated citation should not receive credit simply because it sounds plausible.
Keep a record of the question, expected behavior, retrieved source versions, answer and reviewer observation. Run the same question again after a meaningful configuration or content change. If wording varies, compare the underlying claims and actions rather than expecting a single approved sentence verbatim.
Google's guide to generative AI evaluation discusses assessing systems beyond informal inspection. For a small pilot, a clear rubric applied to a limited question set can provide a more useful starting point than a large dashboard with unexplained scores.
Agree on which failures stop a launch. For example, exposing restricted content or claiming an unauthorized action is complete would demand investigation even if many routine answers looked good. The acceptance rule should reflect the task and the consequence, not an arbitrary overall percentage.
Treat the handoff as a product feature
Give customers a visible way to reach a person. Explain what happens next and what information, if any, will be carried into the handoff. Do not tell someone the team has received their request until the receiving system confirms it.
Design for a failed handoff too. If the message cannot be submitted, retain the useful context where appropriate and offer an understandable retry or alternative contact route. Repeatedly reopening the same chat bubble is not an escalation plan.
Our contact-form experience guide explains related interaction questions around loading, acceptance and confirmation. An assistant should be equally precise about the difference between preparing a message and successfully submitting it.
Let the support team see enough context to continue the conversation without asking the customer to start from the beginning. At the same time, decide what should be retained, who can access it and how vendor settings affect that decision. Review the actual configuration rather than assuming every service treats conversation data in the same way.
Give content changes a path into the assistant
A knowledge base is not finished when its first documents are uploaded. Someone must maintain the connection between published information and retrieved information.
Define what happens when a policy changes, a product is withdrawn or a page is corrected. Record whether the search index refreshes automatically, how the update is verified and what the assistant does while an important correction is pending.
Keep a small regression set for these changes. If a care instruction changes, rerun the related questions and inspect both the response and the cited page. A current website beside an outdated assistant creates two competing versions of the business.
Assign a person to review unresolved questions and decide whether they reveal a missing article, a retrieval problem or a request the assistant should continue to escalate. These are different maintenance tasks and should not all be solved by extending the prompt.
Define what would justify expanding the pilot
Before widening the scope, look at the work the assistant actually creates. Are customers getting a useful answer? Are handoffs reaching the right team? Is someone spending more time correcting confident mistakes than they previously spent answering the original questions?
Measure within an agreed period and describe the limitations. A low handoff count is not automatically success; customers may have abandoned the conversation. Pair aggregate observations with reviewed examples and feedback from the people receiving escalations.
Expand one capability at a time. Another set of public policy questions may be a sensible next step. Account-specific actions need a separate design and review. The broader AI workflow guide can help place that decision within a business process.
Start by choosing one support question and naming its authoritative source, answer owner and fallback. If you cannot complete those three fields, spend the next hour on the knowledge plan rather than the color of the chat bubble.
Visuals: AI-generated cover with fictional objects and interfaces, plus an original illustrative workflow diagram. No client screenshots or measured results are depicted.
