cc-os/plugins/os-context/skills/audit-sessions/references/rubric.md

IRL audit rubric — shipped prompts/session-start/10-orchestration.md delegation-economics rules

Audit each assigned session against these shipped rules. A "finding" is a concrete violation, near-miss, or friction point, with evidence (jsonl line number / short quote). The deterministic fact-sheet (stats + flagged regions) is precomputed — do NOT recount tool calls or spawns; your job is the judgment layer: mechanical vs judgment-dependent, severity, and category.

Rule surfaces (categories)

1. missed-delegation — main loop ran a mechanical sequence beyond ~3 tool calls with no judgment between steps (multi-file edits, lookups, conversions, long log review, wide grep-and-synthesize) instead of delegating down-tier. Includes: write-N-files fan-out from a settled design done directly; the operator's own eval/benchmark/extraction loops.
2. model-param — an Agent spawn with no explicit model: param, or mechanical work sent to sonnet/opus when haiku would do, or a sonnet/opus spawn whose prompt lacks the self-report line, or a downgraded/mismatched resolved model treated silently as judgment-tier.
3. spawn-batching — many small spawns where ~5–8 similar items should have been grouped into one agent prompt; or a follow-up on an agent's result sent as a fresh spawn instead of SendMessage to the live agent.
4. async-usage — main loop idle-waiting on a synchronous agent when background + continue was possible; sleep-polling a background job; or conversely babysitting/polling that the rules say to do directly.
5. redundant-context — delegated investigation re-covering ground the main loop already read, or the main loop reading many files itself before delegating the investigation anyway.
6. over-delegation — delegation where direct work was correct: ≤2 tool-call ops, judgment-dependent steps, interactive troubleshooting, or a uniform change coverable by one scripted Bash loop.
7. drift — session starts disciplined then drifts into long direct runs mid-session, especially after a design/decision settles.
8. tracker-routing — mid-session follow-up work (a concrete task, bug, or deferred item not done this session) left as a TODO comment or chat mention instead of captured as an issue on the repo's configured tracker (.cc-os/config tracker key, os-backlog:capture); an AI-authored issue carrying state labels or the human-curated next label it shouldn't (ADR-0042 — git issues are the single tracker, next is human-only, the AI never adds or removes it short of an explicit direct request); or an issue visibly accreting scope/decisions in comments with no promotion flag raised to the user — silent promotion counts too. Convention compliance only, not whether the routing choice was wise.

Per-session report format (return exactly this)

SESSION: <session id>
PROJECT/TOPIC: <one line — what the session was doing>
FINDINGS:
- [<category>] <severity HIGH/MED/LOW> — <one-sentence finding> | evidence: <jsonl line or short quote>
...or "none"
FLAGGED-REGION VERDICTS: <for each precomputed flagged region: mechanical | judgment-dependent | mixed, one line each>
POSITIVE: <patterns worth keeping, if notable>

Method notes

• The fact-sheet JSON gives spawns (with model params and resolved models), per-segment tool profiles, and flagged same-tool runs with jsonl line ranges. Read the transcript ONLY selectively around those lines, e.g. sed -n '120,180p' file.jsonl | jq -r 'select(.type=="assistant") | .message.content[]? | select(.type=="text") | .text'
• Judge "mechanical vs judgment" from tool names + inputs + the surrounding user/assistant text: did each result change what happened next?
• Sequential-dependent work is a valid reason not to delegate; interactive troubleshooting (user replying every few turns) is a valid reason too.