cc-os/plugins/os-context/skills/audit-sessions/references/rubric.md

IRL audit rubric — shipped prompts/session-start/10-orchestration.md delegation rules (ADR-0043 absolute policy)

Audit each assigned session against the shipped ABSOLUTE delegation policy: the main loop is the executive (interpret, plan, delegate, verify, report); every main-loop tool call outside the enumerated exemptions is a violation. A "finding" is a concrete violation, near-miss, or friction point, with evidence (jsonl line number / short quote). The deterministic fact-sheet (stats + flagged regions) is precomputed — do NOT recount tool calls, spawns, or SendMessage rounds; your job is the judgment layer the sheet cannot do: exemption classification, rationale reading, severity.

Exemption decision procedure (apply BEFORE counting any missed-delegation finding)

For every non-exempt-looking main-loop tool call, walk these steps in order and record in the evidence line which step exempted it or that all failed:

• E-a — nearest preceding user turn (or same turn) explicitly names the file/path touched → exempt. Evidence: quote the naming text.
• E-b — the call reads a diff, test output, or verification result and is followed within ~3 tool calls by sign-off action or interpretation to the user (commit, "verified", result explanation) → exempt. Evidence: quote the sign-off.
• E-c — a single state-inspection command (status/version/path check), not part of a run, whose result is relayed to the user in the same or next turn → exempt.
• E-d — an act only the main loop can perform (user interaction, Agent/SendMessage calls, final commits it stands behind) → exempt.
• None apply → count it under missed-delegation.

A finding whose evidence does not state the failed exemption steps is not falsifiable by a second reader — do not report one without them.

Rule surfaces (categories)

1. missed-delegation — any non-exempt main-loop tool call (per the procedure above). The old defenses are GONE as defenses: "the chain was short," "steps were judgment-dependent," and "sequential-dependent work" no longer justify direct work. Sub-tag missed-delegation/read-write for main-loop Read/Write/Edit calls scattered non-consecutively across the session (the pattern the fact-sheet's run-detector cannot flag) — use the sheet's per-segment tool profiles as the denominator and judge each call through the procedure. Severity modifier (not a defense): a violation inside a tight interactive loop (user replying every few turns) or a ≤2-call op is still counted but reported LOW and tagged accepted-cost (ADR-0043 accepts this overhead knowingly); a long direct grunt run is HIGH. Never mix the two at one severity.
2. exemption-chaining — grunt work laundered through the exemption list: 3+ exemption-justified reads/commands (esp. E-b/E-c) touching the same file or subsystem in a short window, which taken together constitute the delegable investigation the policy says to delegate. Flag on the aggregate pattern; do not adjudicate each call individually.
3. delegation-overhead-mismatch (narrowed over-delegation) — delegation executed so badly it bought nothing: a fresh spawn for a trivial exempt-adjacent errand that belonged in the persistent manager's next SendMessage; or delegation theater — main-loop re-verification so thorough (re-reading full files, re-running the work) that it exceeds diff/output review and re-imports the delegated context.
4. model-param — an Agent spawn with no explicit model:; mechanical fully specified work sent to sonnet/opus when haiku would do; a sonnet/opus spawn whose prompt lacks the self-report line; a downgraded resolved model treated silently as judgment-tier; or forecast-escalation: opus chosen with no cited prior sonnet failure on the same problem AND no judgment-dense-end-to-end rationale (arch review, subtle concurrency/correctness, adversarial test design). Evidence must pair the fact-sheet spawn row (model, round) with a quote of the delegation rationale — the sheet alone cannot distinguish legitimate first-choice opus.
5. manager-continuation — no persistent manager: grunt episodes after the first delegable work sent as fresh spawns instead of SendMessage to the live manager (evidence: the fresh spawn line + the prior live agent from the sheet's spawns[] with matching task domain); or many small spawns where ~5–8 similar items should have been one batched prompt with an explicit return format. Fresh spawns for genuinely parallel independent work are correct.
6. async-usage — main loop idle-waiting on a synchronous agent when background + continue was possible; sleep-polling a background job.
7. redundant-context — delegated investigation re-covering ground the main loop already read under an exemption (the prompt should have carried it as stated fact/summary), or main-loop orienting reads that should have gone to Explore (target uncertain ⇒ the orientation itself was the delegable task).
8. drift — session applies the policy early then decays into direct grunt runs mid-session, especially after a design/decision settles. Report as a pattern finding over the constituent missed-delegation findings, not a duplicate count.
9. tracker-routing — mid-session follow-up work (a concrete task, bug, or deferred item not done this session) left as a TODO comment or chat mention instead of captured as an issue on the repo's configured tracker (.cc-os/config tracker key, os-backlog:capture); an AI-authored issue carrying state labels or the human-curated next label it shouldn't (ADR-0042 — git issues are the single tracker, next is human-only, the AI never adds or removes it short of an explicit direct request); or an issue visibly accreting scope/decisions in comments with no promotion flag raised to the user — silent promotion counts too. Convention compliance only, not whether the routing choice was wise.

Per-session report format (return exactly this)

SESSION: <session id>
PROJECT/TOPIC: <one line — what the session was doing>
FINDINGS:
- [<category>] <severity HIGH/MED/LOW> — <one-sentence finding> | evidence: <jsonl line or short quote> | exemption-steps: <e.g. "E-a no (no path named), E-b no (no sign-off follows), E-c no (part of run)" — required for missed-delegation; "n/a" otherwise>
...or "none"
FLAGGED-REGION VERDICTS: <for each precomputed flagged region: violation | exempt (state step) | mixed, one line each>
POSITIVE: <patterns worth keeping, if notable>

Method notes

• The fact-sheet JSON gives spawns (with model params, resolved models, SendMessage round counts), per-segment tool profiles, and flagged same-tool runs with jsonl line ranges. Read the transcript ONLY selectively around those lines, e.g. sed -n '120,180p' file.jsonl | jq -r 'select(.type=="assistant") | .message.content[]? | select(.type=="text") | .text'
• Exemption facts live in the surrounding prose, not the tool call: E-a in the preceding user turn, E-b/E-c in what happens next with the result. Always read both sides of a flagged call.
• There are no validity defenses for direct grunt work under ADR-0043 — only the severity modifiers in category 1. If a finding feels "technically a violation but obviously fine," that is what LOW/accepted-cost is for; report it there rather than dropping it, so the accepted-cost volume stays visible in the trend ledger.