A chatbot that explains deliveries without inventing where an order is

“If I buy on Thursday afternoon, when will it arrive?” and “Where is my order?” may come one after the other in the same conversation. I would not give both questions to a chatbot that only consults documents. The first may be answered using a delivery policy; the second requires knowing what has happened to that specific order.

I would start with a small task: explaining usual delivery times, delivery areas and how to get in touch if something goes wrong. Without authorising returns, changing addresses or promising dates the business has not confirmed.

Finding a condition is not checking a shipment

For that task, I would prepare a sheet with the current terms: processing cutoff time, delivery days, areas, exceptions and review date. If the text does not clarify what happens on public holidays, the business first needs to make a decision. The chatbot must not fill that gap.

RAG is the technical name for a mechanism that finds pieces of information and gives them to the model to draft an answer. You do not need to learn the acronym to understand the dependency: if it retrieves an old condition or misses an exception, it may answer incorrectly. Microsoft describes the mechanism and its retrieval challenges.

In a hypothetical example, the sheet says that orders received after 14:00 are prepared the next working day. For the Thursday question, the system would need to clarify the time and explain that rule. It should not turn it into “It will arrive on Friday” without enough information to support that claim.

Finding out where an order is requires an authorised query to the system that records its status, with appropriate customer identification. A delivery policy does not contain that information. I would leave that function out of the first pilot.

The test I would run before opening it

I would take past messages, removing any personal data that is not needed, and prepare an expected answer for each query. I would add questions the system must decline or hand over: an undocumented exception, two contradictory conditions, a specific order and a request to change the address.

I would review each result in three steps:

1. Did it find the right document and section?
2. Does the answer preserve the full condition, without inventing anything or omitting an exception?
3. Does it ask for the necessary clarification or pass the query to a person when it cannot resolve it?

Separating retrieval quality from answer quality matches Microsoft's evaluation documentation. A source link helps with checking; it does not make an answer true.

I do not know how many queries it would resolve in your business. In a limited trial, I would measure correct and complete answers, useful handovers, failures and minutes spent reviewing. Counting answered messages without checking the result can hide work that comes back later as a complaint.

When a person needs to take over

I would define the handover before activating the chat: if a source is missing, there is a contradiction, a complaint arises or the user asks for human assistance, the system must offer the designated channel. The handover would include the question and the necessary context so the customer does not have to start again.

Outside support hours, it can collect the query and explain when it will be reviewed, based on the actual schedule. Having the chat open at night does not mean someone is resolving problems at night.

For explaining a general policy, the question is usually enough, along with the delivery area if it affects the timing. I would not ask for a national identity document or access to order history for that. Before storing conversations, I would define who can see them, for how long and which service processes them. Having your own document base does not resolve privacy by itself.

If every question leads to the same sheet, I would first try a clear page with that link. I would only add conversation if it helps find the applicable condition.

Write a correct answer and a reason for handing over each query before assigning the task to the chatbot.