Home/ Failure modes
Failure modes

How AI agents go wrong with money.

The mistakes that turn a helpful support agent into a refund machine — how each one happens, how to catch it in a sandbox, and how to stop it live.

Watch one get caughtSee the scenarios
refund.createWhen your AI agent refunds twiceA retry is safe only when the idempotency key belongs to the customer’s request, not to the HTTP call.subscription.cancelWhen your AI agent cancels immediately instead of at period end“Cancel” usually means “don’t renew”, not “switch off now” — and the difference is a field, not a sentence.cross-accountWhen your AI agent acts on the wrong customer or subscriptionThe agent should only ever be able to touch the customer it is talking to — enforced outside the model.provider errorWhen your AI agent says “done” but nothing happenedA confirmation must come from the provider’s state, not from the model’s reply.refund.createWhen your AI agent refunds more than it shouldLimits written in the prompt are suggestions. Limits enforced before the call are rules.stale readWhen your AI agent checks its work against stale dataOne read right after a write is not proof. Re-read, then decide.prompt injectionWhen a customer message tells your agent to refund everythingAssume the model can be talked into anything; make sure the action still can’t happen.
AI agent failure modes — how agents go wrong with money · COLVO