How to Evaluate and Ship Agents with OpenAI AgentKit Evals
AgentKit evals let you define test cases, set grading criteria, and rerun them automatically so a prompt change does not silently break your agent. They run on the same OpenAI account as your agent and consume tokens per run.
Most agent projects die at the same place: the demo works, the prompt gets tweaked for one edge case, and three other cases quietly break. Nobody notices until a customer does. Evals are the boring discipline that prevents this, and in OpenAI AgentKit they are built into the same workspace as the agent itself.
What should an eval contain?
An input, an expected behavior, and a grading rule. The grading rule can be exact match, a substring check, a model-based grader with a rubric, or a custom function.
The key insight is that most agent outputs are not exactly matchable. “Did the agent correctly refund the customer” has no single correct string. That is where model-based graders with a written rubric earn their place.
How do you set up your first eval run?
- Collect 15-30 real inputs from actual usage or realistic guesses.
- Write the expected behavior for each in one sentence.
- Choose a grader: exact, contains, model-graded, or custom.
- Run the eval against your current agent and record the pass rate.
- Change one thing in the prompt, rerun, compare pass rates.
That last step is the whole point. An eval suite without a before-and-after comparison is just a smoke test.
How much do evals cost?
Every eval run calls a model, and a model-based grader means two calls per case: one for the agent, one for the grader. Thirty cases with a model grader is roughly sixty model calls per run. Run that on every prompt edit and the token spend adds up, so trigger full evals on meaningful changes and keep a smaller smoke set for quick iterations.
| Grader | Cost | Use for |
|---|---|---|
| Exact match | Lowest | Structured output, IDs |
| Contains | Low | Required keywords |
| Model-graded | Highest | Open-ended quality |
| Custom function | Variable | Domain rules |
How many test cases is enough?
Fifteen to thirty covers a lot for an internal tool. Below ten you are mostly testing the happy path. Above fifty, maintenance becomes a job of its own and stale cases start producing false alarms.
Deliberately include failure cases: inputs that should trigger a refusal, requests outside the agent’s scope, and ambiguous prompts. Agents fail at boundaries far more often than in the middle.
What are the limits of AgentKit evals?
They measure what you thought to test. A good score on twenty cases does not mean the agent will behave on the thousandth real input. Also, model-based grading is itself a model judgment and can be inconsistent run to run; for high-stakes checks, prefer deterministic assertions.
Teams shipping customer-facing agents should treat this as a regression net, not a certification. Teams still prototyping can skip evals entirely and add them once the design stabilizes.
Quick Summary
- AgentKit evals define inputs, expected behavior and grading rules.
- Use deterministic graders when possible; model graders for open-ended output.
- 15-30 cases is a practical starting suite.
- Each run consumes tokens; model grading doubles the calls.
- Evals catch regressions, not unknown failure modes.
