cc-os/plugins/os-context/skills/audit-sessions/SKILL.md

---
description: Run the biweekly orchestration IRL audit - deterministic transcript stats precompute, then auditor fan-out judging flagged regions against the shipped `prompts/session-start/10-orchestration.md` rules, then a tiered tune-up report. Works from any directory. Trigger via the os-status due nudge or the recurring issue jared/ops#1 on Forgejo.
---

# Audit production sessions against the shipped orchestration rules

You are running the recurring (biweekly) IRL audit of the shipped delegation-economics
rules. The signal source is real production session transcripts — NEVER eval grid
scenarios or reserve sets (contaminated for wording tuning since 2026-07-08). This skill
works from any cwd; nothing here requires being inside the cc-os repo.

## Step 1 — Deterministic stats precompute (code, no judgment)

Run the driver at `<skill base dir>/../../audit/bin/audit-stats` (the skill base
directory is `<plugin root>/skills/audit-sessions`, the driver lives at
`<plugin root>/audit/bin/audit-stats`):

```bash
<plugin root>/audit/bin/audit-stats --exclude <CURRENT_SESSION_ID>
```

(If the current session id is unknown, omit `--exclude` — the 10-minute active-window
guard already defers the live session to the next run.)

- `--since` defaults to the last ledger row's date. It refuses to run without a prior row;
  a first-ever run needs an explicit `--since YYYY-MM-DD`.
- Output: `~/.local/state/os-context/audit/run-<date>/` containing `stats.tsv`
  (per-session deterministic columns), `flags.md` (heuristic missed-delegation regions),
  and `sheets/*.json` (full per-session fact-sheets). One row is appended to
  `~/.local/state/os-context/audit/metrics.tsv` — the trend ledger. Never edit the
  ledger by hand; never re-derive these numbers with a model.

## Step 2 — Auditor fan-out (judgment only where code can't judge)

Read `stats.tsv` and `flags.md` from the run dir. Sessions with zero spawns AND zero
flagged runs AND main_loop_share 1.0 at low turn counts usually need no auditor —
include them only in the aggregate. Batch the remaining sessions into ~3 parallel
auditor agents (5–8 sessions each), `model: "sonnet"`, background, one round each.

Each auditor prompt must include:
- The rubric: read `references/rubric.md` (sibling to this SKILL.md) and paste it in.
- The session's fact-sheet JSON path and flagged regions — auditors judge
  **mechanical vs judgment-dependent** for each flagged region by reading the transcript
  selectively around the cited jsonl lines (jq on line ranges, never the raw file).
- The self-report line: "State the exact model ID you are running as in the first line
  of your report." Cross-check it; flag any pin mismatch in the report.
- The exact per-session report format from the rubric.

## Step 3 — Synthesize and compare (main loop)

- Aggregate auditor findings into the tiered format: Tier 1 (HIGH, recurring),
  Tier 2 (MEDIUM), Tier 3 (solved surfaces — explicitly name what NOT to spend
  wording budget on).
- Compare this run's ledger row against prior rows in `metrics.tsv` — the previous
  run's Tier 1 items are hypotheses this run tests (e.g. did settled-design drift
  drop after the 2026-07-10 wording patch?). State each verdict explicitly.
- Write `report.md` into the run dir.

## Step 4 — Stop for the human

Report the tiers and trend verdicts, then STOP. Do not edit
`prompts/session-start/10-orchestration.md` without explicit approval — wording changes
are a human decision (injected-token budget; grid validation is unavailable). If approved
later: edit
`~/dev/cc-os/plugins/os-context/prompts/session-start/10-orchestration.md`, run `~/dev/cc-os/bin/refresh-plugins`,
and record the change in `~/dev/cc-os/docs/implementation-status.md`.

Finally, comment the audit date + report location on the recurring issue
`jared/ops#1` (Forgejo, via `tea`) and leave it open — recurring-convention
issues are never closed by the AI (ADR-0042).