Essay · Draft
The Agent That Announces Without Doing
Language models will tell you the email was sent because that is the most likely next sentence. Here is why it happens, what did not work, and what did.
Kamel Dhakwani2026-09-08 · 2 min read
The most dangerous thing an AI agent does is not getting something wrong. It is telling you, politely and confidently, that it did something it did not do.
Our first version did this constantly. Asked to prepare a document, it would reply that the document was being prepared, and nothing would happen. People believe a sentence like that, especially when it is well written.
Why it happens
The mechanism is almost banal. In an agent loop, a message of plain text ends the turn. If the model writes "I will send that now" and stops, the conversation is over, and nothing was sent.
Lowering the reasoning effort to go faster makes it worse. So do instructions that are shouted or contradict each other. The model is predicting the most plausible next sentence, and "it is done" is a very plausible sentence.
What did not work
Our first reaction was instinctive: detect the announcement and force the agent to act. In August we added a guard that spotted these sentences and imposed a corrective turn with a mandatory tool call.
It backfired. It forced a tool on harmless phrases like "try it again", and it replaced answers that were perfectly good. Reading the model's words to judge the model's work turned out to be the same mistake in a new form.
What worked
The answer came from an old habit of mine. On a till, the screen does not decide whether a payment succeeded. The payment terminal does.
So we split the roles. The model understands and chooses. The code establishes what really happened. An agent may only say "sent" or "created" if the code has seen that action succeed, and an error is displayed as blocked, never as done.
We measured the same principle elsewhere. A card placed by the code on a confirmed fact appears 93 to 100% of the time. When we waited for the model to ask for it, it appeared twice in 21 attempts.
The objection
Someone will say that better models will solve this. They will reduce it, and I hope they do. But a company cannot run on a reduced risk of being told something false about its own operations.
Ask of any agent: when it says done, who checked?