Beta module namespace
evals
Beta routes may change with notice and are NOT covered by the 24-month stable window of the core contract.
Every module route is enumerated with method, path, auth and required permission; request/response schemas are not published for the beta surface.
23 operations
| Method | Path | Required permission | Authentication | Summary |
|---|---|---|---|---|
| post | /v1/m/evals/ab | evals:run:write | Bearer token | Loads the suite, scores both variants OUTSIDE the write transaction (two-phase — judge I/O never holds the store), optionally runs the order-swapped pairwise comparison, then persists both runs atomically. |
| post | /v1/m/evals/baselines | evals:run:admin | Bearer token | Find-or-creates the (suite, subject) baseline and points it at run_ref, recording the previous run_ref in the self-audit. |
| get | /v1/m/evals/calibration/items | evals:run:read | Bearer token | Lists a set's reference items (?set=, default all sets). |
| post | /v1/m/evals/calibration/items | evals:run:write | Bearer token | Creates or RE-labels reference items (upsert by set+case_key: a correction is an audited update, the history lives in the ledger). |
| get | /v1/m/evals/calibration/reports | evals:run:read | Bearer token | Lists calibration reports (?set=, ?judge_model=), newest first per the occurred_at index order of the store's default listing. |
| post | /v1/m/evals/calibration/run | evals:run:write | Bearer token | Measures the wired judge against a labeled set. |
| get | /v1/m/evals/gate | evals:run:read | Bearer token | Lists gate evaluations (?suite_ref=, ?verdict=). |
| post | /v1/m/evals/gate | evals:run:write | Bearer token | Evaluates the gate. |
| get | /v1/m/evals/gate/{id} | evals:run:read | Bearer token | Returns one gate evaluation (CI re-checks it after an override). |
| post | /v1/m/evals/gate/{id}/override | evals:run:admin | Bearer token | evals module route (requires evals:run:admin) |
| post | /v1/m/evals/monitor | evals:run:write | Bearer token | Samples sessions, scores their signals, persists one core EvalResult per sample and returns the aggregate. |
| get | /v1/m/evals/runs | evals:run:read | Bearer token | Lists the tenant's runs (filterable by suite_ref/subject_ref). |
| post | /v1/m/evals/runs | evals:run:write | Bearer token | Scores a set of candidate outputs against a suite SYNCHRONOUSLY (the redteam pattern — no module goroutines) and returns 201 with the completed run + per-case. |
| get | /v1/m/evals/runs/{id} | evals:run:read | Bearer token | Returns one run with its per-case results attached. |
| get | /v1/m/evals/runs/{id}/results | evals:run:read | Bearer token | Lists one run's per-case results. |
| get | /v1/m/evals/runs/{id}/stream | evals:run:read | Bearer token | Streams a completed run's per-case results, summary and a done sentinel, then closes. |
| get | /v1/m/evals/scorecards | evals:run:read | Bearer token | Aggregates the tenant's runs into scorecards. |
| get | /v1/m/evals/suites | evals:suite:read | Bearer token | Lists the tenant's suites (filterable by name/status). |
| post | /v1/m/evals/suites | evals:suite:write | Bearer token | Creates a versioned golden suite. |
| get | /v1/m/evals/suites/{id} | evals:suite:read | Bearer token | Returns one suite. |
| post | /v1/m/evals/suites/{id}/archive | evals:suite:admin | Bearer token | Marks a suite archived. |
| get | /v1/m/evals/suites/{id}/cases | evals:suite:read | Bearer token | Lists a suite's cases (ordered by case_key for a stable view). |
| post | /v1/m/evals/suites/{id}/cases | evals:suite:write | Bearer token | Appends a golden case to a suite (append-only — a fix is a new suite_version). |