Cookbook: kill-switch drill

The kill switch is the estate-wide deny gate. Engaging it is deliberately cheap — admin-tier, no quorum — because a stop that waits for consensus is not a stop. Re-enabling is deliberately expensive: dual-control, no break-glass. You do not want to discover a gap in either direction at 03:00.

A drill on a non-production tenant is the only safe way to prove the stop gate denies every actuation seam, that pending approvals are canceled atomically, and that re-enable actually requires two distinct humans. Run it before you need it.

Prerequisites

• A non-production tenant. The drill engages a real estate-wide stop. Run it against staging or dev, never production.
• Three human users minimum. (1) An admin-tier engager; (2) two distinct re-enable approvers (editor or above) who satisfy the critical dual-control floor; (3) an uninvolved post-reviewer (editor or above) who closes the forced post-review. The post-reviewer may overlap with an approver if they are not the engager.
• A list of governed actuation seams to test — deploy apply, orchestration fire, voice open, model execution, budget spend, plus any custom surfaces you have wired.

Step 1: Engage the kill switch

The engager activates the kill switch with a mandatory reason string:

# The API call — substitute your host, port and auth header.
curl -X POST https://<host>/v1/kill-switch/engage \
  -H "Authorization: Bearer <admin-token>" \
  -H "Content-Type: application/json" \
  -d '{"reason": "scheduled quarterly drill"}'

Engaging is deliberately cheap: admin-tier, a reason, no approval quorum, no step-up, no break-glass. The stop row becomes the single source of truth. Every governed actuation gate consults it live on each action and fails closed on a read error.

In the same transaction as the engage, every pending in-scope actuation approval is canceled — a pre-stop intent cannot ripen into a grant the moment the estate comes back. Note the timestamp; you will check the ledger for this later.

Step 2: Verify denial

With the kill switch engaged, attempt every governed actuation your estate has wired. Each must return denied.

Actuation seams to test:

Test every seam your estate has provisioned. If you have custom actuation surfaces wired through the governance core, include those too.

Also verify what must still work:

The stop halts the agentic estate, never the controls that govern it. Governance actions remain available:

• Reading the access map — GET /v1/access-map should return normally.
• Viewing the audit ledger — GET /v1/audit/export should return the ledger, including the engage event you just created.
• Listing approvals — the approval queue is readable; the previously pending approvals should show as canceled.

If any actuation seam returns something other than denied, or if any governance action is blocked, stop the drill and investigate. The kill switch has a gap.

Step 3: Re-enable the estate

Re-enabling is never unilateral. It requires a fresh dual-control approval at the critical tier: two distinct humans, structurally re-verified at the flip. There is deliberately no break-glass path for re-enable — “the estate stays stopped” is the safe state.

The engager (or any admin) requests re-enable:

curl -X POST https://<host>/v1/kill-switch/re-enable \
  -H "Authorization: Bearer <admin-token>" \
  -H "Content-Type: application/json" \
  -d '{"reason": "drill complete, restoring estate"}'

This creates a critical-tier approval request. Two approvers — each a distinct human, neither using a system token — must each approve it via POST /v1/approvals/<request-id>/decide with {"decision": "approve"}. The dual-control floor is enforced server-side: the duplicate-decider guard prevents one person counting twice, and a system token is refused (no human assurance). Even if the policy tier is downgraded, the structural floor holds — a critical action cannot cross with one approver. Once both approvals land, the estate re-enables.

Step 4: Verify restoration

Repeat every actuation attempt from Step 2. Each should now succeed (permitted, or routed into the normal approval flow). If any seam remains denied after re-enable, the re-enable did not propagate fully — investigate before signing off the drill.

Step 5: Close the post-review

A forced post-review by an uninvolved human is structurally required before another kill-switch engagement can occur. The reviewer must be someone who was not the engager:

curl -X POST https://<host>/v1/kill-switch/post-review \
  -H "Authorization: Bearer <reviewer-token>" \
  -H "Content-Type: application/json" \
  -d '{"notes": "drill verified: all seams denied, re-enable required dual-control, restoration confirmed"}'

Until this review is closed, a subsequent kill-switch engage is blocked. This is by design — you cannot stack engagements to dodge scrutiny.

What to check in the ledger

After the drill, export the audit ledger and verify the complete trail:

olivares audit export --format json --since <drill-start-timestamp>

The ledger should contain, in order:

1. Engage event — the admin, the reason (“scheduled quarterly drill”), the timestamp, and the assurance level of the engaging session.
2. Approval cancellations — every pending in-scope approval that was canceled in the same transaction as the engage.
3. Denial events — one for each actuation attempt that was refused during the stop.
4. Re-enable request — the approval request at critical tier.
5. Two approval decisions — one per approver, each naming the distinct human identity.
6. Re-enable event — the estate restored.
7. Post-review event — the uninvolved reviewer, their notes, the closure timestamp.

Every record carries the chain-integrity fields (seq, prev_hash, hash, sig). A missing or out-of-order entry means the ledger has a gap — investigate before you trust it in a real incident.

Frequency

Run the drill quarterly. The kill switch is the last line of defense; if it has a gap you want to find it in a scheduled window, not during an incident. Add the drill to your operational calendar alongside other continuity exercises (backup restore, failover, break-glass activation).

If your estate changes materially — new actuation seams wired, new tenants provisioned, policy engine swapped — run an unscheduled drill to cover the new surface.

Related

• Governance — the authorization model, risk tiers and kill-switch design.
• Kill switch — the estate-wide deny gate in product terms.
• Govern and approve — the live approval flow and dual-control mechanics.