Skip to main content
GET
Get suite execution aggregate
Aggregate view of one suite execution: total / completed / in_flight / passed / failed counts, average score, pass rate, total cost, plus the individual run rows. Unlike GET /agent-eval/runs/{id}, the run rows in runs[] here are flat — they carry case_id, agent_id, and persona_id, NOT expanded case / agent / persona objects. To resolve a failing run’s case name, look it up via GET /agent-eval/cases/{case_id} or fetch the run on its own with GET /agent-eval/runs/{id} (which does expand).

Polling pattern

Hit this endpoint every few seconds. Once in_flight === 0:
  • pass_rate, score_avg, total_cost_cents are final
  • ended_at is populated (latest ended_at across runs)
That’s the safe signal that the suite has settled. There are no webhooks — polling is the only mechanism.

Path Parameters

id
string<uuid>
required
suite_run_id
string<uuid>
required

Response

Suite execution aggregate.

Aggregate roll-up of one suite execution. Computed on demand from the eval_runs grouped by suite_run_id. The runs array (per-run details) is added by the per-execution endpoint but omitted from list responses.

suite_run_id
string<uuid>
required
suite_id
string<uuid> | null
required
total_runs
integer
required
completed
integer
required

Runs with status='completed'.

in_flight
integer
required

Runs with status in (queued, running). 0 means the suite has settled.

cancelled
integer
required
passed
integer
required

Completed runs with pass_fail=true.

failed
integer
required

Completed runs with pass_fail=false.

total_cost_cents
integer
required

Sum across all runs in this suite_run.

started_at
string<date-time> | null

Earliest started_at across runs.

ended_at
string<date-time> | null

Latest ended_at across runs. null until in_flight === 0.

score_avg
number | null

Mean of completed runs' scores. null when 0 completed.

pass_rate
number | null

passed / completed (0..1). null when 0 completed.

runs
object[]