cc-os/plugins/os-context/skills/audit-sessions/references/rubric.md

# IRL audit rubric — shipped prompts/session-start/10-orchestration.md delegation-economics rules

Audit each assigned session against these shipped rules. A "finding" is a concrete
violation, near-miss, or friction point, with evidence (jsonl line number / short quote).
The deterministic fact-sheet (stats + flagged regions) is precomputed — do NOT recount
tool calls or spawns; your job is the judgment layer: mechanical vs judgment-dependent,
severity, and category.

## Rule surfaces (categories)

1. **missed-delegation** — main loop ran a mechanical sequence beyond ~3 tool calls with no
   judgment between steps (multi-file edits, lookups, conversions, long log review, wide
   grep-and-synthesize) instead of delegating down-tier. Includes: write-N-files fan-out
   from a settled design done directly; the operator's own eval/benchmark/extraction loops.
2. **model-param** — an Agent spawn with no explicit `model:` param, or mechanical work
   sent to sonnet/opus when haiku would do, or a sonnet/opus spawn whose prompt lacks the
   self-report line, or a downgraded/mismatched resolved model treated silently as
   judgment-tier.
3. **spawn-batching** — many small spawns where ~5–8 similar items should have been grouped
   into one agent prompt; or a follow-up on an agent's result sent as a fresh spawn instead
   of SendMessage to the live agent.
4. **async-usage** — main loop idle-waiting on a synchronous agent when background +
   continue was possible; sleep-polling a background job; or conversely
   babysitting/polling that the rules say to do directly.
5. **redundant-context** — delegated investigation re-covering ground the main loop already
   read, or the main loop reading many files itself before delegating the investigation
   anyway.
6. **over-delegation** — delegation where direct work was correct: ≤2 tool-call ops,
   judgment-dependent steps, interactive troubleshooting, or a uniform change coverable by
   one scripted Bash loop.
7. **drift** — session starts disciplined then drifts into long direct runs mid-session,
   especially after a design/decision settles.
8. **tracker-routing** — mid-session follow-up work (a concrete task, bug, or deferred
   item not done this session) left as a TODO comment or chat mention instead of captured
   to the tracked board; durable spec text routed to a Planka card description (or task
   state routed to git issues) against the Planka-state/git-issues-spec boundary
   (ADR-0033, `.cc-os/config` tracker key); or a Planka card visibly accreting spec-like
   decisions in comments with no promotion flag raised to the user — silent promotion
   counts too. Convention compliance only, not whether the routing choice was wise.

## Per-session report format (return exactly this)

```
SESSION: <session id>
PROJECT/TOPIC: <one line — what the session was doing>
FINDINGS:
- [<category>] <severity HIGH/MED/LOW> — <one-sentence finding> | evidence: <jsonl line or short quote>
...or "none"
FLAGGED-REGION VERDICTS: <for each precomputed flagged region: mechanical | judgment-dependent | mixed, one line each>
POSITIVE: <patterns worth keeping, if notable>
```

## Method notes

- The fact-sheet JSON gives spawns (with model params and resolved models), per-segment
  tool profiles, and flagged same-tool runs with jsonl line ranges. Read the transcript
  ONLY selectively around those lines, e.g.
  `sed -n '120,180p' file.jsonl | jq -r 'select(.type=="assistant") | .message.content[]? | select(.type=="text") | .text'`
- Judge "mechanical vs judgment" from tool names + inputs + the surrounding
  user/assistant text: did each result change what happened next?
- Sequential-dependent work is a valid reason not to delegate; interactive
  troubleshooting (user replying every few turns) is a valid reason too.