Skip to main content
GET
/
agent-eval
/
suites
/
{id}
/
runs
/
{suite_run_id}
Get suite execution aggregate
curl --request GET \
  --url https://api.goyappr.com/agent-eval/suites/{id}/runs/{suite_run_id}
{
  "suite_run_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "suite_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "total_runs": 123,
  "completed": 123,
  "in_flight": 123,
  "cancelled": 123,
  "passed": 123,
  "failed": 123,
  "total_cost_cents": 123,
  "started_at": "2023-11-07T05:31:56Z",
  "ended_at": "2023-11-07T05:31:56Z",
  "score_avg": 123,
  "pass_rate": 123,
  "runs": [
    {
      "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "company_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "mode": "text",
      "created_at": "2023-11-07T05:31:56Z",
      "case_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "case": {
        "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "company_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "agent_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "persona_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "name": "Yes path — caller agrees on first ask",
        "scenario": "The persona is responding to a missed call from your business about their recent inquiry. They have time to talk for 5 minutes.",
        "success_criteria": [
          {
            "weight": 1,
            "description": "<string>",
            "pattern": "<string>",
            "tool_name": "<string>",
            "args_match": {},
            "node_id": "<string>",
            "rubric": "<string>"
          }
        ],
        "max_turns": 20,
        "pass_threshold": 80,
        "tool_policy": "mock",
        "created_at": "2023-11-07T05:31:56Z",
        "agent": {
          "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
          "name": "<string>",
          "type": "prompt",
          "flow_config": {
            "nodes": [
              {
                "id": "<string>",
                "name": "<string>",
                "position": {
                  "x": 123,
                  "y": 123
                },
                "agent_speaks_first": true,
                "greeting": "<string>",
                "is_literal": false,
                "next_step_id": "<string>",
                "auto_advance": true
              }
            ],
            "flow_config_version": "1",
            "metadata": {
              "custom_metadata_keys": [
                "<string>"
              ]
            }
          },
          "system_prompt": "<string>",
          "description": "<string>",
          "background_sound_volume": 0.3,
          "temperature": 1,
          "greeting_message": "<string>",
          "agent_speaks_first": true,
          "vad_stop_secs": 0.5,
          "vad_start_secs": 0.2,
          "vad_confidence": 0.7,
          "silence_timeout_secs": 60,
          "max_continuous_speech_secs": 120,
          "max_call_duration_secs": 600,
          "lead_memory_enabled": true,
          "is_active": true,
          "webhook_url": "<string>",
          "webhook_events": [],
          "extraction_parameters": [
            {
              "name": "customerName",
              "description": "The caller's full name as mentioned during the conversation"
            }
          ],
          "created_at": "2023-11-07T05:31:56Z",
          "updated_at": "2023-11-07T05:31:56Z"
        },
        "persona": {
          "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
          "company_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
          "name": "Frustrated tenant",
          "identity_prompt": "You are a 38-year-old tenant calling about a leaking pipe in your kitchen. You're frustrated because this is the third time you've reported it.",
          "language": "en",
          "created_at": "2023-11-07T05:31:56Z",
          "description": "<string>",
          "behavior_traits": {
            "patience": "low",
            "verbosity": "chatty",
            "cooperation": "cooperative",
            "interruption_tendency": "occasional",
            "goal": "Get a maintenance technician scheduled today"
          },
          "voice_config": {},
          "updated_at": "2023-11-07T05:31:56Z",
          "deleted_at": "2023-11-07T05:31:56Z"
        },
        "suite_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "description": "<string>",
        "agent_overrides": {},
        "tool_allowlist": [],
        "updated_at": "2023-11-07T05:31:56Z",
        "deleted_at": "2023-11-07T05:31:56Z"
      },
      "suite_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "suite_run_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "agent_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "persona_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "agent_model": "<string>",
      "persona_model": "<string>",
      "started_at": "2023-11-07T05:31:56Z",
      "ended_at": "2023-11-07T05:31:56Z",
      "duration_ms": 123,
      "score": 50,
      "pass_fail": true,
      "evaluation": {
        "score": 50,
        "pass_fail": true,
        "results": [
          {
            "assertion": {
              "weight": 1,
              "description": "<string>",
              "pattern": "<string>",
              "tool_name": "<string>",
              "args_match": {},
              "node_id": "<string>",
              "rubric": "<string>"
            },
            "passed": true,
            "weight": 123,
            "reason": "<string>"
          }
        ]
      },
      "agent_input_tokens": 0,
      "agent_output_tokens": 0,
      "persona_input_tokens": 0,
      "persona_output_tokens": 0,
      "agent_cost_cents": 0,
      "persona_cost_cents": 0,
      "total_cost_cents": 0,
      "error": "<string>",
      "agent_overrides": {}
    }
  ]
}

Documentation Index

Fetch the complete documentation index at: https://docs.goyappr.com/llms.txt

Use this file to discover all available pages before exploring further.

Aggregate view of one suite execution: total / completed / in_flight / passed / failed counts, average score, pass rate, total cost, plus the individual run rows with case (and nested agent + persona) expanded.

Polling pattern

Hit this endpoint every few seconds. Once in_flight === 0:
  • pass_rate, score_avg, total_cost_cents are final
  • ended_at is populated (latest ended_at across runs)
That’s the safe signal that the suite has settled. There are no webhooks — polling is the only mechanism.

Path Parameters

id
string<uuid>
required
suite_run_id
string<uuid>
required

Response

Suite execution aggregate.

Aggregate roll-up of one suite execution. Computed on demand from the eval_runs grouped by suite_run_id. The runs array (per-run details) is added by the per-execution endpoint but omitted from list responses.

suite_run_id
string<uuid>
required
suite_id
string<uuid> | null
required
total_runs
integer
required
completed
integer
required

Runs with status='completed'.

in_flight
integer
required

Runs with status in (queued, running). 0 means the suite has settled.

cancelled
integer
required
passed
integer
required

Completed runs with pass_fail=true.

failed
integer
required

Completed runs with pass_fail=false.

total_cost_cents
integer
required

Sum across all runs in this suite_run.

started_at
string<date-time> | null

Earliest started_at across runs.

ended_at
string<date-time> | null

Latest ended_at across runs. null until in_flight === 0.

score_avg
number | null

Mean of completed runs' scores. null when 0 completed.

pass_rate
number | null

passed / completed (0..1). null when 0 completed.

runs
object[]