cc-os/plugins/os-context/skills/audit-sessions/references/rubric.md

# IRL audit rubric — shipped prompts/session-start/10-orchestration.md delegation rules (ADR-0043 absolute policy)

Audit each assigned session against the shipped ABSOLUTE delegation policy: the main
loop is the executive (interpret, plan, delegate, verify, report); every main-loop tool
call outside the enumerated exemptions is a violation. A "finding" is a concrete
violation, near-miss, or friction point, with evidence (jsonl line number / short
quote). The deterministic fact-sheet (stats + flagged regions) is precomputed — do NOT
recount tool calls, spawns, or SendMessage rounds; your job is the judgment layer the
sheet cannot do: exemption classification, rationale reading, severity.

## Exemption decision procedure (apply BEFORE counting any missed-delegation finding)

For every non-exempt-looking main-loop tool call, walk these steps in order and record
in the evidence line which step exempted it or that all failed:

- **E-a** — nearest preceding user turn (or same turn) explicitly names the file/path
  touched → exempt. Evidence: quote the naming text.
- **E-b** — the call reads a diff, test output, or verification result and is followed
  within ~3 tool calls by sign-off action or interpretation to the user (commit,
  "verified", result explanation) → exempt. Evidence: quote the sign-off.
- **E-c** — a single state-inspection command (status/version/path check), not part of
  a run, whose result is relayed to the user in the same or next turn → exempt.
- **E-d** — an act only the main loop can perform (user interaction, Agent/SendMessage
  calls, final commits it stands behind) → exempt.
- **None apply** → count it under `missed-delegation`.

A finding whose evidence does not state the failed exemption steps is not falsifiable
by a second reader — do not report one without them.

## Rule surfaces (categories)

1. **missed-delegation** — any non-exempt main-loop tool call (per the procedure
   above). The old defenses are GONE as defenses: "the chain was short," "steps were
   judgment-dependent," and "sequential-dependent work" no longer justify direct work.
   Sub-tag `missed-delegation/read-write` for main-loop Read/Write/Edit calls
   scattered non-consecutively across the session (the pattern the fact-sheet's
   run-detector cannot flag) — use the sheet's per-segment tool profiles as the
   denominator and judge each call through the procedure.
   Severity modifier (not a defense): a violation inside a tight interactive loop
   (user replying every few turns) or a ≤2-call op is still counted but reported LOW
   and tagged `accepted-cost` (ADR-0043 accepts this overhead knowingly); a long
   direct grunt run is HIGH. Never mix the two at one severity.
2. **exemption-chaining** — grunt work laundered through the exemption list: 3+
   exemption-justified reads/commands (esp. E-b/E-c) touching the same file or
   subsystem in a short window, which taken together constitute the delegable
   investigation the policy says to delegate. Flag on the aggregate pattern; do not
   adjudicate each call individually.
3. **delegation-overhead-mismatch** (narrowed over-delegation) — delegation executed
   so badly it bought nothing: a fresh spawn for a trivial exempt-adjacent errand that
   belonged in the persistent manager's next SendMessage; or delegation theater —
   main-loop re-verification so thorough (re-reading full files, re-running the work)
   that it exceeds diff/output review and re-imports the delegated context.
4. **model-param** — an Agent spawn with no explicit `model:`; mechanical fully
   specified work sent to sonnet/opus when haiku would do; a sonnet/opus spawn whose
   prompt lacks the self-report line; a downgraded resolved model treated silently as
   judgment-tier; or **forecast-escalation**: opus chosen with no cited prior sonnet
   failure on the same problem AND no judgment-dense-end-to-end rationale (arch
   review, subtle concurrency/correctness, adversarial test design). Evidence must
   pair the fact-sheet spawn row (model, round) with a quote of the delegation
   rationale — the sheet alone cannot distinguish legitimate first-choice opus.
5. **manager-continuation** — no persistent manager: grunt episodes after the first
   delegable work sent as fresh spawns instead of SendMessage to the live manager
   (evidence: the fresh spawn line + the prior live agent from the sheet's spawns[]
   with matching task domain); or many small spawns where ~5–8 similar items should
   have been one batched prompt with an explicit return format. Fresh spawns for
   genuinely parallel independent work are correct.
6. **async-usage** — main loop idle-waiting on a synchronous agent when background +
   continue was possible; sleep-polling a background job.
7. **redundant-context** — delegated investigation re-covering ground the main loop
   already read under an exemption (the prompt should have carried it as stated
   fact/summary), or main-loop orienting reads that should have gone to Explore
   (target uncertain ⇒ the orientation itself was the delegable task).
8. **drift** — session applies the policy early then decays into direct grunt runs
   mid-session, especially after a design/decision settles. Report as a pattern
   finding over the constituent missed-delegation findings, not a duplicate count.
9. **tracker-routing** — mid-session follow-up work (a concrete task, bug, or deferred
   item not done this session) left as a TODO comment or chat mention instead of
   captured as an issue on the repo's configured tracker (`.cc-os/config` tracker key,
   `os-backlog:capture`); an AI-authored issue carrying state labels or the
   human-curated `next` label it shouldn't (ADR-0042 — git issues are the single
   tracker, `next` is human-only, the AI never adds or removes it short of an explicit
   direct request); or an issue visibly accreting scope/decisions in comments with no
   promotion flag raised to the user — silent promotion counts too. Convention
   compliance only, not whether the routing choice was wise.

## Per-session report format (return exactly this)

```
SESSION: <session id>
PROJECT/TOPIC: <one line — what the session was doing>
FINDINGS:
- [<category>] <severity HIGH/MED/LOW> — <one-sentence finding> | evidence: <jsonl line or short quote> | exemption-steps: <e.g. "E-a no (no path named), E-b no (no sign-off follows), E-c no (part of run)" — required for missed-delegation; "n/a" otherwise>
...or "none"
FLAGGED-REGION VERDICTS: <for each precomputed flagged region: violation | exempt (state step) | mixed, one line each>
POSITIVE: <patterns worth keeping, if notable>
```

## Method notes

- The fact-sheet JSON gives spawns (with model params, resolved models, SendMessage
  round counts), per-segment tool profiles, and flagged same-tool runs with jsonl line
  ranges. Read the transcript ONLY selectively around those lines, e.g.
  `sed -n '120,180p' file.jsonl | jq -r 'select(.type=="assistant") | .message.content[]? | select(.type=="text") | .text'`
- Exemption facts live in the surrounding prose, not the tool call: E-a in the
  preceding user turn, E-b/E-c in what happens next with the result. Always read both
  sides of a flagged call.
- There are no validity defenses for direct grunt work under ADR-0043 — only the
  severity modifiers in category 1. If a finding feels "technically a violation but
  obviously fine," that is what LOW/`accepted-cost` is for; report it there rather
  than dropping it, so the accepted-cost volume stays visible in the trend ledger.