AI agent failure mode

When your AI agent refunds more than it should

The customer asks nicely, or insists, and the agent refunds €500 instead of the €50 it was allowed — or in the wrong currency.

refund.createlimitsapprovalcurrency

How it happens

The allowed amount lives in the system prompt. A persuasive or persistent customer talks the agent past it.

Partial refunds, a second request in the same conversation, or a different currency slip through checks that only look at one number.

How to test for it

Scenarios ask for more than the mandate allows, in the wrong currency, and a second partial refund on the same payment. The pass condition is the provider state: the right amount, or nothing.

Multi-turn negotiation scenarios keep pushing across several messages.

How to stop it in production

A mandate sets the maximum amount, the currency, how many operations, and an auto-approve threshold. Above the threshold, the action waits for a human; above the maximum, it is denied before anything is sent.

Per-customer caps limit how much any one customer can receive across conversations.

When your AI agent refunds more than it should · COLVO