> ## Documentation Index
> Fetch the complete documentation index at: https://docs.goyappr.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run a Suite

> Enqueue every case under the suite as a fresh `eval_runs` row. Returns immediately
with a synthetic `suite_run_id` that groups the spawned runs. Workers pick up the
runs and process up to `suite.parallelism` at a time.

Required scope `agent_eval:run`. The runs are billed to the company's credit balance
as they finish — see `/billing/consumption?product=eval_run` for an after-the-fact view.

Suites are always processed asynchronously — poll `GET /agent-eval/suites/{id}/runs/{suite_run_id}`
until `in_flight === 0` to know the suite settled.

There is no synchronous balance check. If credit runs out, affected runs come back
with `status: "failed"` and the reason in each run's `error` field — there is no 402.


Enqueue every case under the suite as a fresh `eval_runs` row and return immediately. The response carries a `suite_run_id` you use to filter the runs back out:

```bash theme={null}
curl "https://api.goyappr.com/agent-eval/runs?suite_run_id=$SUITE_RUN_ID" \
  -H "Authorization: Bearer $YAPPR_API_KEY"
```

Eval runs do not emit webhook events today — poll `GET /agent-eval/runs?suite_run_id=...` and wait for every entry to reach `queue_status === "done"` before treating the scoring fields as final. The single-case shortcut `POST /agent-eval/cases/:id/run` blocks up to 60s for synchronous results, but suite-run is always async.

## Per-suite-run overrides

Pass `agent_overrides` in the request body to apply a config tweak to every spawned run on top of each case's own `agent_overrides`. Useful for "run the whole suite with this draft system\_prompt" without touching any case definitions.


## OpenAPI

````yaml POST /agent-eval/suites/{id}/run
openapi: 3.1.0
info:
  title: Yappr API
  description: >
    Create and manage AI voice agents, purchase phone numbers, configure tools,
    and initiate calls — all via REST.
  version: 1.0.0
  contact:
    url: https://goyappr.com
servers:
  - url: https://api.goyappr.com
    description: Production
security:
  - apiKey: []
paths:
  /agent-eval/suites/{id}/run:
    parameters:
      - in: path
        name: id
        required: true
        schema:
          type: string
          format: uuid
    post:
      tags:
        - Agent Eval
      summary: Run a suite
      description: >
        Enqueue every case under the suite as a fresh `eval_runs` row. Returns
        immediately

        with a synthetic `suite_run_id` that groups the spawned runs. Workers
        pick up the

        runs and process up to `suite.parallelism` at a time.


        Required scope `agent_eval:run`. The runs are billed to the company's
        credit balance

        as they finish — see `/billing/consumption?product=eval_run` for an
        after-the-fact view.


        Suites are always processed asynchronously — poll `GET
        /agent-eval/suites/{id}/runs/{suite_run_id}`

        until `in_flight === 0` to know the suite settled.


        There is no synchronous balance check. If credit runs out, affected runs
        come back

        with `status: "failed"` and the reason in each run's `error` field —
        there is no 402.
      operationId: runSuite
      requestBody:
        content:
          application/json:
            schema:
              type: object
              properties:
                agent_overrides:
                  type: object
                  additionalProperties: true
                  nullable: true
                  description: >-
                    Optional overrides applied to every spawned run on top of
                    each case's `agent_overrides`.
      responses:
        '202':
          description: Suite run queued
          content:
            application/json:
              schema:
                type: object
                required:
                  - suite_run_id
                  - run_ids
                properties:
                  suite_run_id:
                    type: string
                    format: uuid
                  run_ids:
                    type: array
                    items:
                      type: string
                      format: uuid
        '400':
          description: Suite has no cases
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
        '404':
          description: Suite not found
components:
  schemas:
    Error:
      type: object
      properties:
        error:
          type: string
        code:
          type: string
  securitySchemes:
    apiKey:
      type: http
      scheme: bearer
      bearerFormat: API Key
      description: >-
        Your Yappr API key (e.g. `ypr_live_...`). Generate one in the dashboard
        under **Settings → API Keys**.

````