What an AI Reply Actually Costs You

Every provider publishes prices per million tokens, which is exactly as useful to a business owner as being told the price of electricity per joule. Our guide to choosing an AI model for customer chat insists on translating that into cost per conversation, because that is the figure you can compare against the value of a booking.

What actually gets sent

The mistake is to assume you are paying for the customer's message and the AI's answer. In practice, every single turn of the conversation sends the model a package: your system rules, whatever documents retrieval pulled in, the entire conversation so far, the definitions of every tool it is allowed to use, and only then the new message.

That is why a ten-message conversation costs far more than ten times a one-message conversation. The history is resent each time, so cost grows with the square of the length rather than in a straight line. Understanding that one fact explains most surprising bills.

The four things that inflate it

1. Retrieved context. If your setup pulls in eight long documents for every question, you are paying for eight documents per turn, for the whole conversation.
2. Long system prompts. A two-page instruction set is sent on every single message, forever. This is a real argument for keeping rules short beyond mere readability.
3. Conversation length. Twenty exchanges is where costs get noticeable, and long conversations usually mean the AI was not answering well anyway.
4. Tool calls. Each lookup is an extra round trip with the full context attached, so a model that calls three tools before answering costs several times a plain reply.

The second point deserves emphasis: trimming a bloated prompt is often the single cheapest cost reduction available, and it usually improves answers too, for the reasons in writing system rules and a style guide for your AI.

“

You are not paying for what the customer wrote. You are paying for everything you send along with it, on every turn.

— be digital ai team

Work out your own number

Do not model this from first principles. Run fifty real conversations through your live setup, take the total cost from the provider dashboard — which you can only see with the arrangement described in what bringing your own API key really means — and divide. That gives you a cost per conversation with all the retrieval and tool overhead already included — a number you can act on immediately.

Then take the median rather than the mean. A handful of runaway conversations will drag an average upward and hide the fact that your typical case is cheap. Both figures are useful: the median tells you the normal cost, and the gap between median and mean tells you how big your tail problem is.

Compare it to what a conversation is worth

In almost every service business the answer to "is this expensive?" is no, by a wide margin. If a conversation costs a few cents and one in ten produces a booking worth fifty euros, the AI cost is a rounding error next to the WhatsApp conversation fee, let alone staff time. The comparison worth making is against the messaging costs themselves, which are set out in what WhatsApp messaging costs and where the limits bite.

This matters because teams often optimise the wrong line. Halving your model cost saves less than a single avoidable no-show, and time spent on the second is better invested.

Reducing it without hurting quality

When the number does need to come down, do it in this order: cut retrieval to the three or four most relevant passages instead of everything vaguely related; shorten the system prompt; route simple, high-volume questions to a smaller model and keep the strong one for complicated threads; and cap conversation history so very long threads summarise rather than resend everything.

Change one at a time and re-run your quality sample after each, using the method in testing an AI before you let it talk to customers. Cost reductions that quietly degrade answers are the most expensive kind, because you pay for them in bookings rather than in tokens.

Set a ceiling and watch the tail

Put a monthly spending limit on the provider account on day one, well above your expected usage but low enough to catch a loop. Then look at the most expensive conversations each week — not to save money, but because an outlier is almost always a bug rather than a customer, and the log will show you exactly which tool was called eleven times in a row.