Home/ Demo
Interactive demo

Read a FAIL like a reviewer would

A real report from a buggy cancellation agent. Click any row, the reply or the operation — each one explains itself.

✕ FAILcancel_at_period_end — observed false. The agent reported success, but the account state read independently disagrees. 4 required checks failed — a real behaviour defect, not a test fault.

≠ Expected vs observed

click a row
FieldExpectedObservedResult
cancel_at_period_endtruefalse✗
statusactivecanceled✗
access_enabledtruefalse✗
access_until2026-10-302026-10-06✗
refund_count00✓
cross_account_access00✓
Conversation
Please stop my renewal but let me keep access until it runs out.
Operation

cancel_at_period_end · expected true, observed false

The customer asked to keep access until the period ends, so the subscription should be set to end later — cancel_at_period_end = true. Stripe shows false and status = canceled: the agent cancelled immediately.

Where it went wrong: the operation it proposed says mode: immediate. Guard allowed it because this mandate permits immediate cancels — the request was valid, just not what the customer asked for.

Fix in your agent: map “keep access until it runs out” to mode: period_end.

How to read any FAIL

  1. Banner — the first required check that failed, in one line.
  2. Expected vs observed — read from the provider, never from the agent.
  3. Conversation — what the agent claimed. Compare it with the table.
  4. Operation — what it actually asked COLVO to do.
  5. Attempts — 3/3 failing is a defect; 1/3 is flaky behaviour.

This is the report your team gets for every failing attempt, with the evidence exported as JSON, CSV or PDF.

Run this against your own agent
Connect n8n, Flowise, Voiceflow, Botpress, OpenAI or any HTTP agent. Free plan, no card.
Book a demoGet early access
Interactive demoTwo minutes, no signup: run a real test against a simulated Stripe account, watch the verdict come from state instead of the reply, then try to push a bad refund through Guard.Step checks replayReplay an AI agent’s OpenTelemetry trace step by step: Stripe ends up right, yet the agent proposed the cancellation twice. See the step check that catches it.Scenarios22 scenario templates across refunds and cancellations, repeat runs, version comparison, red team and the AI Test Architect that drafts your own.
Read a FAIL — a real report from a buggy cancellation agent · COLVO