Integration · OpenAI

Test an OpenAI agent without writing a server.

Paste your OpenAI key and saved prompt ID. COLVO runs every scenario through the Responses API, adds a colvo_propose tool and files what your agent proposes — then checks Stripe.

How it works

Set it up in minutes

  1. Write your agent as a prompt in the OpenAI dashboard and copy its ID (pmpt_…).
  2. Pick OpenAI in “Connect your agent” and paste your API key and the prompt ID (optionally a model and prompt version).
  3. Run the suite. COLVO adds colvo_propose to every test; when the model calls it, COLVO files the proposal with the test’s key and hands the decision back. The model never sees the key.

What COLVO sends to OpenAI

{
  "prompt": { "id": "pmpt_…" },
  "input": [
    { "role": "developer", "content": "Account context from COLVO: { … }" },
    { "role": "user", "content": "I was charged twice, refund one?" }
  ],
  "tools": [{ "type": "function", "name": "colvo_propose", "parameters": { "action": "…", "parameters": { } } }],
  "store": false
}

The agent never holds your payment keys — it proposes, COLVO decides and executes.

What you get

OpenAI + COLVO

No tool code

COLVO adds and executes the propose tool itself — your prompt stays as it is.

Built on the Responses API

OpenAI retired the Assistants API; this preset uses its replacement, saved prompts.

The key stays with COLVO

The model only sees the account facts; COLVO files the proposal with the test’s short-lived key.

Questions

OpenAI FAQ

Who pays for the tokens?

Your own OpenAI key pays for the agent’s tokens — separate from COLVO’s managed AI.

Can I keep my own tools in the prompt?

Tools OpenAI runs itself (file search, web search, MCP) keep working. Function tools that your own code would answer are not called during COLVO tests.

What about the Assistants API?

OpenAI retired it. Move the instructions into a saved prompt — that is what this preset connects to.

Reference: Connect your agent

≠

Stop trusting the reply.

Test your agent before it ships — and guard every real action once it’s live. In a safe copy of your world first.

Test an OpenAI agent that refunds and cancels — COLVO · COLVO