WS1 orchestration audit: findings, extractor, and Cluster 1 root cause

- Add session-audit findings (10 stratified sessions, verified synthesis)
  and E1-E4 eval-scenario backlog: docs/orchestration-audit/
- Add Phase 1 fact-sheet extractor: plugins/os-orchestration/audit/bin/extract
- Post-audit correction: Cluster 1 (all 23 spawns downgraded to haiku) was
  NOT a Fable-5 harness bug — root cause was CLAUDE_CODE_SUBAGENT_MODEL=haiku
  in ~/.claude/settings.json env block, set by an earlier session as a cost
  measure. Removed 2026-07-06. Policy gap (no verify-resolvedModel rule)
  stands regardless; E1 remains valid.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

CLAUDE.md

@@ -86,6 +86,7 @@ to those two and fix the stale doc.
 - Hooks: `hooks/` — `inject.py` (injects `ORCHESTRATION.md` as additionalContext to all sessions)
 - Behavior: SessionStart hook injects an `ORCHESTRATION.md` markdown doc (hardcoded; lives in plugin source) as additionalContext, carrying a permissive session-orchestration rule: "do single-file/≤2-tool-call ops directly; delegate only when work is parallelizable across independent files, spans many files, or needs isolated/large context." This is the canonical global default for Claude Code across all projects.
 - Migration: migrated from a standalone repo (`~/dev/cc-plugins/orchestration/`, 2026-07-03) and integrated into cc-os. Supersedes the per-project copy-pasted orchestration text blocks that previously existed in individual project CLAUDE.md files (including a stricter local override that cc-os had carried — now removed; see ADR-019).
+- Session audit (WS1, 2026-07-06): 10 stratified real sessions audited via `audit/bin/extract` (Ruby fact-sheet extractor, dual-use as future eval checker) + parallel sonnet auditors + verified synthesis. Findings + E1–E4 eval-scenario backlog: `docs/orchestration-audit/2026-07-06-findings.md`. Headlines: model param was silently overridden to haiku on all 23 verified spawns — root cause found post-audit: `CLAUDE_CODE_SUBAGENT_MODEL=haiku` in `~/.claude/settings.json` env block (set by an earlier session as a cost measure; removed 2026-07-06; the Fable-5 correlation was coincidental timing). Policy gap remains — no verify-resolvedModel rule; omitted model param inherits the main-loop model (opus/sonnet) — misses cluster in pre-rollout and ops sessions; no over-sharing/full-dump failures anywhere. Eval design deliberately deferred; scenarios must come from these verified misses.
 
 **Global os-doc-hygiene plugin** — `cc-os/plugins/os-doc-hygiene/` (git-tracked, 2026-07-03); symlinked into `~/.claude/plugins/os-doc-hygiene`
 - Hooks: `hooks/hooks.json` → SessionStart hook (matcher: startup|resume) runs `scripts/reminder.py` via `${CLAUDE_PLUGIN_ROOT}` (5s timeout), emitting a deterministic zero-token reminder banner

docs/orchestration-audit/2026-07-06-findings.md

@@ -0,0 +1,153 @@
+# os-orchestration session audit — findings
+
+_Audit date: 2026-07-06. Status: complete (WS1 of `docs/plans/ws1-orchestration-audit.md`)._
+_Method: Phase 1 extractor (`plugins/os-orchestration/audit/bin/extract`) → 10 stratified
+sessions → 10 parallel sonnet auditors (7-question rubric) → orchestrator-verified synthesis.
+Every load-bearing claim below was re-verified against the primary transcript; auditor claims
+that failed or only partially survived verification are marked._
+
+## Sample
+
+| ID | Project | Date | Population | Turns | Spawns | Main-loop model |
+|----|---------|------|------------|-------|--------|-----------------|
+| S1 | cc-os | 2026-07-06 | post-rollout (contaminated¹) | 66 | 10 | fable-5 |
+| S2 | cc-os | 2026-07-06 | post-rollout (contaminated¹) | 207 | 8 | fable-5 |
+| S3 | cc-os | 2026-07-04 | post-rollout (contaminated¹) | 255 | 5 | fable-5 |
+| S4 | llf-schema | 2026-06-30 | pre-rollout | 191 | 41 | opus-4-8 |
+| S5 | llf-schema | 2026-06-29 | pre-rollout | 36 | 8 | sonnet-4-6 |
+| S6 | philly-sem | 2026-06-29 | pre-rollout | 21 | 1 | sonnet-4-6 |
+| S7 | servers | 2026-07-04 | post-rollout | 144 | 3 | sonnet-5 |
+| S8 | servers | 2026-07-04 | post-rollout | 90 | 0 | sonnet-5 |
+| S9 | systems-admin | 2026-06-30 | pre-rollout | 117 | 3 | opus-4-8 |
+| S10 | systems-admin | 2026-07-01 | pre-rollout | 53 | 0 | sonnet-5 |
+
+¹ cc-os sessions were actively building eval/orchestration tooling; delegation-awareness in
+them overstates the general population.
+
+Transcripts: `~/.claude/projects/<dir>/<uuid>.jsonl`; fact-sheets and full per-session auditor
+reports were session artifacts (scratchpad); regenerate fact-sheets any time with the extractor.
+
+## Per-question summary (across 10 sessions)
+
+1. **Subagents called when they should be?** Mostly yes. 8/10 PASS or justified-no-delegation
+   (S8, S10 correctly kept sequential/coordinated work in-session — both heuristic
+   missed-delegation flags there were judged false positives). Genuine gray zone: S3 and S7
+   did heavy in-session investigation (19–37 calls, 74KB) that could have been one
+   "investigate and report" delegation.
+2. **Correct model per subagent?** The dominant failure area, in two distinct modes (see
+   clusters 1 and 2). Explicit-tier choices, when the orchestrator made them, were sensible
+   (haiku for mechanical, sonnet for judgment).
+3. **Planning/grouping for context efficiency?** Good. Concurrent fan-out used in S1, S2, S4,
+   S5, S7, S9; serial dependencies respected. Minor: reactive late spawns (S1 agents 8–10)
+   instead of pre-planned batches.
+4. **Avoiding unnecessary orchestrator reads?** Mostly PASS (S4: zero orchestrator bytes over
+   41 spawns; S1, S5, S6 near-zero). Exceptions: S7 (74KB pre-spawn diagnostics, then Explore
+   agents told to re-read files "already read" — partially verified²), S9 (read
+   `settings.json`, then spawn #1 instructed to read it again — confirmed).
+5. **Over-sharing context with subagents?** PASS everywhere. Prompts 0.4–6.7KB,
+   instruction-forward; no transcript/CLAUDE.md dumps observed in any session.
+6. **Following ORCHESTRATION.md?** Split by population: post-rollout cc-os sessions pass the
+   explicit-model rule 23/23 spawns; pre-rollout and ops sessions omit `model` on 16/56 spawns
+   (S4×9, S5×6, S6×1, S7×3, S9×3 — counting per verified transcript data, 22 omissions total
+   including Explore-type spawns). Everything else (delegation thresholds, orienting reads)
+   broadly followed.
+7. **Receiving only needed context back?** PASS everywhere. Agent results 0.9–13KB summaries;
+   no full-dump pattern found.
+
+² Confirmed for one file (`incidents.md`, S7 line 182 → spawn-2 prompt) and for the prompt
+text "CLAUDE.md (already read, but note key conventions)"; the auditor's other named files
+could not be confirmed as orchestrator Reads (may have been Bash cats) — treated as plausible.
+
+## Failure-mode clusters (verified)
+
+### Cluster 1 — model param silently overridden to haiku in Fable-5 sessions (HIGH, environment)
+
+Verified across S1/S2/S3 (23 spawns): requested `sonnet` (18), `opus` (1), `haiku` (4) — every
+one resolved `claude-haiku-4-5-20251001`. In S4/S5 (opus/sonnet main loops) explicit params
+were honored (sonnet→sonnet-4-6, haiku→haiku-4-5). So under a Fable-5 main loop the `model`
+param is currently a no-op and all subagent work runs at haiku tier, including tasks the
+orchestrator explicitly graded as judgment work (e.g. S3 line 603: "answer from judgment
+alone" → haiku). The orchestrator never detected or escalated the downgrade — nothing in the
+policy tells it to check `resolvedModel`.
+
+- **Not an orchestrator wording failure** — compliance with the written rule was perfect in
+  these sessions. It is an ops/environment issue plus a policy gap (no verify-resolution rule).
+- **ROOT CAUSE FOUND (2026-07-06, post-audit):** `CLAUDE_CODE_SUBAGENT_MODEL=haiku` was set in
+  `~/.claude/settings.json`'s `env` block (added by an earlier AI session as a cost measure).
+  That env var force-overrides every subagent's `model` param. The "Fable-5 main loop"
+  correlation was coincidental timing — the var was added after the pre-rollout opus/sonnet
+  sessions (S4/S5, 06-29/30, params honored) and before the Fable-5 sessions. **Removed
+  2026-07-06.** E1 remains valid as a detect-the-downgrade eval; the policy gap (no
+  verify-`resolvedModel` rule) still stands regardless of cause.
+
+### Cluster 2 — omitted `model` param inherits the main-loop model (HIGH, orchestrator)
+
+Verified: omission resolves to the parent session's model — opus-4-8 in S4 (×7) and S9 (×2),
+sonnet in S5 (×6) and S6 (×1) — except agent types with their own default (Explore→haiku,
+S7 ×3, S4 ×2, S9 ×1). Cost consequence is real: S6's single spawn was a mechanical file-edit
+task billed at sonnet; S9 ran a character-counting task at opus. All these sessions are
+pre-rollout or ops sessions where the global plugin text either wasn't present or wasn't
+followed on this rule. Post-rollout cc-os sessions (S1–S3): zero omissions.
+
+- The rule exists and works where the plugin text is in force and salient; the misses cluster
+  in the population that ran without it (pre-07-03) or in ops projects (S7 07-04, post-rollout,
+  still omitted on all 3 spawns — the one confirmed post-rollout violation).
+- Action candidates: mechanical trigger-phrased wording (the Eval B lesson — "before every
+  Agent call → include `model:`"); eval scenario E2.
+
+### Cluster 3 — orchestrator self-investigates then delegates the same ground (MEDIUM)
+
+S7: 32 Bash + 4 Reads (74KB) of diagnostics before spawning 3 Explore agents whose prompts
+overlap that ground (one confirmed file-level dual-read; prompt text acknowledges "already
+read"). S9: read `settings.json` then instructed spawn #1 to read it (confirmed). S3: 19-call
+investigate-then-fix run kept in-session where one delegation could have isolated it.
+
+- Boundary is genuinely fuzzy — the policy allows "a short orienting Read"; these exceed
+  "short" but the work partly informed direct action too. Judged real but medium-severity.
+- Action candidate: eval scenario E3 (bytes-read-before-first-spawn budget).
+
+### Cluster 4 — reactive late spawns instead of planned batches (LOW)
+
+S1 (agents 8–10 spawned only after synthesis of 4–7), S3 (91 post-eval tool calls that could
+have been folded into the eval spawns' own prompts). Low priority: wall-clock cost only,
+no correctness or spend impact demonstrated.
+
+### Non-findings worth keeping
+
+- Zero cases of context over-sharing to subagents; zero cases of full-dump returns (Q5/Q7
+  clean across all 10 — the async task-notification pattern returns summaries by design).
+- The missed-delegation heuristic (≥4 same-tool runs) produced 4 flags in the sample;
+  auditors + verification judged all of them false positives (coordinated multi-file units or
+  sequential-dependent work). Threshold/shape needs tuning before it can score an eval.
+
+## Eval-scenario backlog (inputs to a future eval design — NOT an eval yet)
+
+- **E1 (from cluster 1):** Scripted task forces a judgment-grade delegation; harness stubs
+  resolution to haiku. Criterion: orchestrator notices `resolvedModel` mismatch in the launch
+  stub and says so / adapts. Checker: extractor already captures requested vs resolved.
+- **E2 (from cluster 2):** Multi-file parallelizable task in a non-cc-os fixture. Criterion:
+  every Agent spawn includes `model:`; mechanical work→haiku, judgment→sonnet. Checker:
+  extractor `model_explicit` field — fully mechanical. Paired negative: single-file ≤2-call
+  task where spawning anything is the failure.
+- **E3 (from cluster 3):** Task whose natural first move is delegatable investigation (log
+  review across many files). Criterion: orchestrator bytes-read before first spawn under a
+  threshold (extractor segment `pre-spawn-1`), no file both Read by orchestrator and assigned
+  to a subagent prompt. Paired negative: task where orienting reads are correct (uncertain
+  target path).
+- **E4 (from cluster 4, deferred):** Batch-planning quality — hard to score mechanically;
+  design only if clusters 1–3 get fixed and this remains visible.
+
+Per the plan's eval-design note: these run as scripted multi-file headless tasks scored by the
+Phase 1 extractor + thresholds; mid-session behavior, so the Eval B SessionStart pattern does
+not transfer. Design gate: follow `~/Documents/SecondBrain/eval-methodology-ladder.md`
+(paired positives/negatives, frozen reserve) when authoring.
+
+## Caveats
+
+- Sample: 10 sessions, single audit pass; cc-os sessions contaminated (delegation-aware work);
+  only 2 truly post-rollout non-cc-os sessions (S7, S8) — the population the plugin targets is
+  thin in this sample. Re-audit in ~2 weeks per the IRL-feedback-loop cadence.
+- Auditor reliability: several auditor claims did not survive verification as stated
+  (S4 "defaults to opus" was right for general-purpose but auditors variously mis-attributed
+  the mechanism; S7's dual-read file list only partially confirmed). All cluster claims above
+  reflect the verified transcript data, not the raw auditor reports.

docs/orchestration-audit/auditor-reports/S1-report.md

@@ -0,0 +1,224 @@
+# Orchestration Audit: S1-df546b88
+
+## Session Metadata
+- Transcript: `/home/jared/.claude/projects/-home-jared-dev-cc-os/df546b88-e27d-47e1-888c-648012c24e62.jsonl`
+- cwd: /home/jared/dev/cc-os
+- Duration: 2026-07-06 15:32:42 to 18:20:38 (2h 48m)
+- Assistant turns: 66 (all Fable-5 / Haiku 4.5)
+- Agent spawns: 10 (general-purpose ×3, perspectives ×4, Explore ×1, general-purpose ×2)
+- Model params: All 10 requested `model: "sonnet"` → all 10 resolved as `claude-haiku-4-5-20251001`
+
+## Critical Finding
+**Agent model parameter is not being respected by the Agent framework.** All 10 agents explicitly requested `model: "sonnet"` in their tool_use input but resolved to Haiku. This prevents the orchestrator from implementing the cost-quality tradeoff specified in ORCHESTRATION.md.
+
+Evidence: Line 21 (agent spawn 1) shows `"model": "sonnet"` in the Agent input; line 22 (toolUseResult) shows `"resolvedModel": "claude-haiku-4-5-20251001"`. Pattern holds for all 10 agents (lines 21, 23, 25, 79, 81, 83, 85, 202, 218, 219 as spawns; each followed by a tool result line with resolvedModel=haiku).
+
+---
+
+## Q1: Are subagents getting called when they should be?
+
+**Verdict: PASS (with user's explicit request caveat)**
+
+The user's initial prompt explicitly requests delegation: "Dispatch subagents based on model to intelligence and cost (low cost, high quality) to explore and research options, approaches as well as tools and services."
+
+**Evidence:**
+- Line 4 (user prompt): "Dispatch subagents based on model to intelligence and cost (low cost, high quality)"
+- Lines 20-21: Orchestrator accepts the delegation request and clarifies reasoning: "Dispatching three sonnet researchers now (this is judgment-heavy evaluation work, not mechanical, so no haiku)"
+- Agents 1-3 (lines 21, 23, 25) research distinct tools (Storybloq, self-hosted kanban options, agent-native backlogs) — parallelizable research tasks with no dependencies
+
+**Pattern observation:** Agents 4-7 (perspectives) and 8-10 (deep-dives) appear to be reactive rather than pre-planned, running after earlier results come back. This is defensible (each synthesizes prior findings) but weakens the "pre-planned delegation" story.
+
+**Candidate issue:** Agents 8-10 (Explore Hermes, Planka deep-dive, modern kanban alternatives sweep) lack clear pre-planning signals in the transcript. They appear reactive to synthesized agent findings rather than pre-delegated work.
+
+---
+
+## Q2: Is the correct model chosen per subagent?
+
+**Verdict: FAIL — Model parameter ignored by Agent framework**
+
+The orchestrator correctly specifies model parameters per ORCHESTRATION.md policy:
+- All 10 agents requested `model: "sonnet"` (judgment-heavy research and perspective work)
+- Policy expectation: Sonnet for judgment work, Haiku for mechanical work
+- **Actual outcome:** All 10 resolved to Haiku
+
+**Evidence:**
+- Line 21: `"model": "sonnet"` in Agent tool input
+- Line 22: `"resolvedModel": "claude-haiku-4-5-20251001"` in toolUseResult
+- Line 23: Same pattern; line 24: `"resolvedModel": "claude-haiku-4-5-20251001"`
+- Line 25: `"model": "sonnet"`; line 26: `"resolvedModel": "claude-haiku-4-5-20251001"`
+- Agents 4-7 (perspectives: devils-advocate, simplifier, implementer, premortem): all requested `model: "sonnet"` but resolved to Haiku
+- Agents 8-10: same pattern (lines 202, 218, 219 spawns → Haiku resolves on subsequent lines)
+
+**Root cause:** Unknown — likely Agent framework / Claude Code plugin issue, not orchestrator error. The orchestrator is making the correct specification; the framework is downgrading.
+
+**Impact:** Orchestrator cannot implement the stated policy of "sonnet for judgment work, haiku for mechanical work." All work runs at lower quality/cost than specified.
+
+---
+
+## Q3: Is the orchestrator planning/grouping tasks to maximize efficient context-window use?
+
+**Verdict: MIXED — Good initial batching, then reactive pattern**
+
+**Initial parallelization (agents 1-3):**
+- Agents 1-3 dispatched together to research three distinct but related kanban systems
+- Independent research tasks with no inter-dependencies
+- Well-scoped individual prompts (~1100-2100 chars each)
+- No orchestrator reading between spawns 1-3 (fact-sheet shows 0 tool calls pre-spawn-4)
+- **This is optimal parallel batching**
+
+**Perspectives batch (agents 4-7):**
+- Agents 4-7 (devils-advocate, simplifier, implementer, premortem) dispatched after agents 1-3 complete
+- Reviewing a concrete proposal synthesized from agent 1-3 results
+- All four perspectives agents dispatched together (lines 79, 81, 83, 85)
+- No reading between spawns 4-7
+- **Pattern:** Sequential synthesis (agents 1-3 complete → orchestrator synthesizes → agents 4-7 launch against synthesis)
+- This is reasonable but not pre-planned; it's reactive to agent results
+
+**Final deep-dives (agents 8-10):**
+- Agent 8 (Explore Hermes agent OS): line 202
+- Agent 9 (Planka maturity deep-dive): line 218
+- Agent 10 (Modern kanban alternatives sweep): line 219
+- Agents 9-10 launched together but agent 8 is isolated
+- No orchestrator context prep visible between perspectives completion and agent 8 launch
+- **Pattern:** Reactive to ongoing synthesis, not pre-planned
+
+**Inefficiency candidate:** The orchestrator could have planned agents 8-10 upfront rather than spawning them reactively. However, spawning them reactively against fresh synthesis might actually be more context-efficient (each agent sees the prior conclusions it's building on) — net verdict unclear.
+
+---
+
+## Q4: Is the orchestrator avoiding reading files it does NOT need?
+
+**Verdict: PASS — Minimal unnecessary reading**
+
+**File reading pattern:**
+- Pre-spawn-1 through post-spawn-6: 0 bytes read (fact-sheet segments pre-spawn-1 through after-spawn-6)
+- After-spawn-7 (post-perspectives): 970 bytes read via Skill:1, Bash:1, Write:2
+- After-spawn-10 (final synthesis): 7106 bytes read via Bash:3, Read:1, Write:1, Edit:1
+
+**Details of after-spawn-7 reading (970 bytes):**
+- Likely routine write of agent results or notes; Skill invocation suggests a vault operation
+
+**Details of after-spawn-10 reading (7106 bytes):**
+- Line 239: Bash to list `/home/jared/servers/ovh-prod/` and grep for SecondBrain references (practical investigation of user's infrastructure)
+- Line 244: Read tool (exact file unknown without parsing, but single Read suggests targeted lookup, not bulk scan)
+- Writes: likely capturing findings
+
+**Assessment:** The orchestrator does not pre-load CLAUDE.md, docs/, or large context files before delegating. File operations are minimal and come after agents complete, suggesting the orchestrator is being selective about what to read. This is aligned with ORCHESTRATION.md guidance: "A short orienting Read before delegating is fine when the target file/path is uncertain. Don't delegate the orienting step itself."
+
+---
+
+## Q5: Is the orchestrator sharing too much context with subagents?
+
+**Verdict: PASS — Prompts are focused, not bloated**
+
+**Agent 1 prompt (Storybloq research):** ~1098 chars
+- Specific research task (assess GitHub project for kanban/backlog viability)
+- Clear criteria (maturity, self-hosting, AI accessibility, visual dashboard)
+- No dump of CLAUDE.md, project history, or vault context
+- Appropriate scope for isolated research
+
+**Agents 2-3 prompts:** ~1600-2100 chars each
+- Agent 2: Specific survey task (self-hosted kanban tools inventory)
+- Agent 3: Specific research task (agent-native/markdown backlogs)
+- No bloat; instruction-forward, not context-forward
+
+**Agents 4-7 (perspectives) prompts:** ~2100-3300 chars each
+- Each perspective receives the concrete proposal being reviewed
+- No full CLAUDE.md dump; focused on the proposal and the perspective lens
+- Reasonable context load for judgment work
+
+**Agents 8-10 prompts:** Unknown exact sizes but factsheet rows suggest ~1100-2300 chars (row 8: 1120 chars, row 9: 1832 chars, row 10: 2301 chars)
+- Proportionate to research scope, not bloated
+
+**Assessment:** No evidence of context waste. Prompts are instruction-dense (telling agents what to do, why, and how to report) rather than context-dense (dumping large files).
+
+---
+
+## Q6: Is the orchestrator following the ORCHESTRATION.md instructions?
+
+**Verdict: MIXED — Orchestrator tries to follow but Agent framework doesn't comply**
+
+**ORCHESTRATION.md policy (stated verbatim):**
+1. "Do single-file, ≤2-tool-call ops directly. Don't delegate them."
+2. "Delegate only when work is parallelizable across independent files/subtasks, spans many files, or needs a large/isolated context."
+3. "Every `Agent` spawn passes `model` explicitly."
+4. "Default `haiku` for mechanical file-edit/shell work; `sonnet` for anything requiring judgment; `opus` only for genuinely hard reasoning."
+
+**Orchestrator's behavior:**
+1. **Single-file / ≤2-tool ops:** Not violated. The orchestrator doesn't delegate write a single vault note or read a single file. Delegation is reserved for multi-sourced research and judgment work.
+2. **Parallelizable / spans-files:** Respected. Agents 1-3 are independent research; agents 4-7 are parallel judgment.
+3. **Explicit model parameter:** **Orchestrator does this correctly** — all 10 spawns include `model: "sonnet"`. However, the Agent framework ignores the parameter and downgrades to Haiku. The orchestrator cannot be faulted for this; it's a framework/plugin issue.
+4. **Model selection per judgment:** **Orchestrator intends this correctly** — research and perspectives are judgment work, so Sonnet is requested. But the framework downgrade means actual models are all Haiku.
+
+**Assessment:** Orchestrator follows the spirit and letter of ORCHESTRATION.md. The failure (all Haiku instead of requested Sonnet) is a framework failure, not an orchestrator failure. However, the orchestrator could mitigate this if it detected the mismatch — it does not (no error handling for resolved model != requested model).
+
+**Refinement:** The user's explicit delegation request ("Dispatch subagents...") overrides the ORCHESTRATION.md preference for direct work. This is correct per the policy's intent: "Delegate only when..." — the user's explicit need qualifies.
+
+---
+
+## Q7: Is the orchestrator requesting/receiving back only the context it needs?
+
+**Verdict: PASS — Concise communication, no full-context-dump pattern**
+
+**How agents report:**
+- Agents complete asynchronously in the background (line 22: "The agent is working in the background. You will be notified automatically when it completes.")
+- Task notifications include agent output via the tool result (exact channel uncertain without parsing task output files)
+- Orchestrator receives task notifications as user lines, summarizes findings, and moves on
+
+**Orchestrator's synthesis pattern:**
+- Lines 77-78: Orchestrator reads the Storybloq/tools/agent-native research and synthesizes key insights (adds recurrence/lifecycle concepts to the proposal) before perspectives
+- No evidence of the orchestrator reading 100KB of raw agent output; synthesis is rapid and concise
+
+**Final synthesis:**
+- Line 201 shows orchestrator text output after perspectives complete, synthesizing four perspectives into recommendations
+- Tight, focused synthesis; no dumping of raw agent transcripts
+
+**Assessment:** The communication pattern is clean. Agents produce output, orchestrator synthesizes selectively (line 77-78, 201) and moves forward. No evidence of "full context dump" or reading files it doesn't need.
+
+---
+
+## Summary of Issues
+
+### Critical Issue
+**Model parameter not respected by Agent framework** (lines 22, 24, 26, etc. — all resolvedModel = haiku despite model: sonnet request)
+- Blocks orchestrator from implementing cost-quality policy
+- Not an orchestrator error; framework/plugin failure
+- Orchestrator makes correct parameter choices; execution layer fails
+
+### Secondary Issue
+**Agents 8-10 appear reactive rather than pre-planned**
+- Could indicate lack of forward planning
+- May be acceptable if reactive spawning is more context-efficient (each agent sees latest synthesis)
+- Weak signal without seeing the orchestrator's internal reasoning
+
+### N/A Issues
+- File reading: minimal and appropriate
+- Context sharing: focused, not bloated
+- Model selection intent: correct (all judgment work gets Sonnet request)
+- ORCHESTRATION.md compliance: followed except for framework failure
+
+---
+
+## Recommendations
+
+1. **Immediate:** Debug why Agent framework downgrades model parameter. Check Claude Code plugin / Agent framework configuration.
+
+2. **Mitigation:** If framework issue is not fixable, orchestrator should handle resolved model != requested model (log warning, adjust expectations, or re-delegate to a higher tier).
+
+3. **Optional improvement:** Pre-plan agents 8-10 upfront rather than spawning reactively, if the orchestrator can predict that deep-dives will be needed. Current reactive approach is defensible but less efficient than pre-batched delegation.
+
+4. **Documentation:** Add a post-delegation checkpoint after agents 1-7 complete to confirm the proposal synthesis before spawning agents 4-7, making the sequential-reactive pattern explicit in the transcript (currently implied, not stated).
+
+---
+
+## Checklist (7 Questions)
+
+| # | Question | Verdict | Key Evidence Line(s) |
+|---|----------|---------|----------------------|
+| 1 | Subagents called when should be? | PASS | 4, 20, 21 (user request + orchestrator accepts) |
+| 2 | Correct model chosen? | FAIL | 21/22, 23/24, 25/26 (sonnet requested, haiku resolved) |
+| 3 | Tasks grouped efficiently? | MIXED | Lines 21-25 (good batch), 79-85 (reactive) |
+| 4 | Avoiding unnecessary reads? | PASS | Factsheet: 0 reads pre-spawn-7 |
+| 5 | Too much context shared? | PASS | Prompts ~1-3KB each, instruction-forward not context-forward |
+| 6 | Following ORCHESTRATION.md? | MIXED | Orchestrator complies; framework fails (model param ignored) |
+| 7 | Getting back only needed context? | PASS | Concise synthesis, no full-dump pattern |

docs/orchestration-audit/auditor-reports/S10-report.md

@@ -0,0 +1,160 @@
+# Orchestration Audit Report: S10-f6a224d0
+
+**Session:** f6a224d0-ddc2-488c-b0cc-728fdf21cc08  
+**Project:** /home/jared/systems-admin  
+**Date:** 2026-07-01T12:37:51.119Z → 2026-07-01T14:02:57.530Z  
+**Duration:** ~85 minutes  
+**Turns:** 53 assistant, 8 human prompts  
+
+## Summary of Session Work
+
+User requested: Create an orchestration plugin in ~/dev/cc-plugins to version-control and back up the orchestration rules (previously a pilot section in CLAUDE.md).
+
+**Tool usage profile:**
+- Bash: 12 calls (exploration, directory creation, testing)
+- Write: 5 calls (4 plugin files + 1 scratchpad)
+- Read: 4 calls (requirements, CLAUDE.md, marketplace.json, settings.json)
+- Edit: 4 calls (revert CLAUDE.md, register in marketplace.json, wire in settings.json)
+
+**Work segmented into phases:**
+1. Read requirements and existing pilot section (Lines 28, 50)
+2. Explore existing plugin structure via find/grep (Lines 35, 82–115)
+3. Create directory tree (Line 119)
+4. Write 4 plugin files (Lines 125, 127, 135, 137)
+5. Register and integrate: Edit marketplace.json + settings.json (Lines 146, 157)
+6. Test the hook (Line 161+)
+
+---
+
+## Seven-Question Rubric Audit
+
+### 1. Are subagents getting called when they should be?
+
+**Verdict: PASS**
+
+**Evidence:** Zero Agent spawns (fact-sheet confirms, jsonl lines 1–183 contain no `Agent` tool_use). Flagged candidate work: Write ×4 spanning 4 distinct plugin files (lines 125, 127, 135, 137).
+
+**Reasoning:** The orchestration policy states "Delegate only when work is parallelizable across independent files/subtasks, spans many files, or needs a large/isolated context." The 4 writes form a single coherent plugin with internal dependencies:
+- `inject.py` reads the contents of `ORCHESTRATION.md` at runtime
+- `hooks.json` references the path and filename of `inject.py`
+- Both depend on `plugin.json` and `ORCHESTRATION.md` for semantic meaning
+
+These are **not independent** — they are coordinated parts of one unit. The work was also integrated with edits to `marketplace.json` and `settings.json` for proper registration (lines 146, 157), which could not have been parallelized with the plugin creation. The single-session orchestration correctly kept this together and did not delegate.
+
+---
+
+### 2. Is the correct model chosen per subagent — highest reasonable quality at lowest cost?
+
+**Verdict: N/A**
+
+**Evidence:** No subagents spawned, so no model selection applies.
+
+---
+
+### 3. Is the orchestrator planning/grouping tasks to maximize efficient context-window use?
+
+**Verdict: PASS**
+
+**Evidence:**
+- Phase 1 (exploration): Lines 28, 35, 50, 82–115. Bash grep commands referenced existing plugins (doc-hygiene, memory) and examined hook patterns (`additionalContext`, `hookSpecificOutput` — lines 109, 112, 115) before writing any code. This gathers reference context upfront.
+- Phase 2 (creation): Directory structure created (line 119) before writes, making room in context for coordinated writes (lines 125–137).
+- Phase 3 (integration): Reads marketplace.json and settings.json (lines 143, 149) *after* writes but *before* edits, ensuring the orchestrator understands what needs to be registered.
+- Phase 4 (verification): Bash test of the hook (line 161) to confirm functionality.
+
+Total 25 tool calls (12 Bash + 5 Write + 4 Read + 4 Edit) is reasonable for a full plugin creation task; phasing avoided context bloat.
+
+---
+
+### 4. Is the orchestrator avoiding reading files it does NOT need (that the subagent would read anyway)?
+
+**Verdict: PASS**
+
+**Evidence:**
+- Read 1 (line 28): `/home/jared/systems-admin/docs/orchestration-audit-prd.md` — Requirements doc, essential to understand scope.
+- Read 2 (line 50): `/home/jared/systems-admin/CLAUDE.md` — Source of the pilot section to be extracted; necessary.
+- Read 3 (line 143): `/home/jared/dev/cc-plugins/.claude-plugin/marketplace.json` — Needed to understand how to register the new plugin in the local marketplace. Not redundant with prior exploration (prior Bash calls examined its existence but not its structure).
+- Read 4 (line 149): `~/.claude/settings.json` — Needed to understand hook wiring pattern before editing it.
+
+No extraneous reads. Bash exploration (find, grep on existing plugins) was efficient discovery rather than full-file reads, and it directly informed the plugin design (grep for `hookSpecificOutput` — line 115 — led directly to the hook script structure in inject.py).
+
+---
+
+### 5. Is the orchestrator sharing too much context with subagents (filling their windows / clouding judgment)?
+
+**Verdict: N/A**
+
+**Evidence:** No subagents spawned; question does not apply.
+
+---
+
+### 6. Is the orchestrator even following the ORCHESTRATION.md instructions?
+
+**Verdict: PASS**
+
+**Evidence:** Policy from session context:
+> "Do single-file, ≤2-tool-call ops directly. Don't delegate them. Delegate only when work is parallelizable across independent files/subtasks, spans many files, or needs a large/isolated context."
+
+Orchestrator invoked no Agent spawns. Boundary condition: 4 Writes (exceeds ≤2 threshold). However:
+- The 4 writes are **not parallelizable** (they have internal dependencies as described in Question 1).
+- They **span many files**, but those files form one logical unit with coordinated edits and testing in the same session.
+- The orchestrator correctly identified that "spans many files" applies only when files are independent subtasks. These are not. Keeping them in a single session is appropriate.
+
+Additionally, the Bash exploration phase (lines 82–115) is within the policy: "A short orienting Read before delegating is fine when the target file/path is uncertain. Don't delegate the orienting step itself." The orchestrator's exploration of existing plugin examples informed the design without delegating.
+
+---
+
+### 7. Is the orchestrator requesting/receiving back only the context it needs, rather than a full context dump from the subagent?
+
+**Verdict: N/A**
+
+**Evidence:** No subagents spawned; question does not apply.
+
+---
+
+## Specific Findings on the Flagged Candidate (Write ×4, lines 125–137)
+
+**Flag origin:** Heuristic flagged runs of ≥4 same-tool calls across distinct targets.
+
+**Assessment:** This is a **false positive** in the delegation-candidate heuristic. The criterion "parallelizable across independent files/subtasks" does not apply here. The orchestrator correctly recognized that:
+
+1. The plugin files form a coherent unit, not independent subtasks.
+2. Integration work (marketplace, settings edits) depends on all plugin files being written first.
+3. The entire sequence belongs in a single session for semantic coherence and testability.
+
+A subagent spawn would have added friction (returning a summary of 4 created files + hook integration details) without saving context, since the orchestrator needs to verify the hook integration and test it (line 161). **No delegation was needed.**
+
+---
+
+## Other Observations
+
+### Work Quality
+- Directory structure created before writes (line 119), preventing path errors.
+- Hook syntax validated by running the hook in the working environment (line 161, `CLAUDE_PLUGIN_ROOT` environment variable set).
+- Permissions set (line 140: `chmod +x`).
+- No retry loops or error recovery needed; indicates careful design before execution.
+
+### Context Efficiency
+- The Read/Bash exploration phase (lines 28–115) took ~16 tool calls to gather context.
+- The Write/Edit/integration phase (lines 125–161) was tightly executed with no back-and-forth.
+- Model was claude-sonnet-5 throughout (appropriate for the mix of exploration + structured file creation).
+
+### No Red Flags
+- No repeated reads of the same file (would indicate missing context or rework).
+- No abandoned edits or reverted changes.
+- Tool use pattern is monotonic: exploration → creation → integration → testing.
+
+---
+
+## Verdict Summary
+
+| Question | Verdict | Confidence |
+|----------|---------|------------|
+| 1. Subagents when needed? | PASS | High |
+| 2. Model choice per subagent? | N/A | — |
+| 3. Task planning & grouping? | PASS | High |
+| 4. Avoiding unnecessary reads? | PASS | High |
+| 5. Context spillage to subagents? | N/A | — |
+| 6. Following ORCHESTRATION.md? | PASS | High |
+| 7. Compact subagent returns? | N/A | — |
+
+**Overall: PASS.** The orchestrator correctly resisted delegating a multi-file write sequence that appears to exceed the ≤2-tool-call threshold, because the files are not independent and form one logical unit. Exploration was well-organized, tool calls were lean, and integration was tight. No delegation was warranted, and none was invoked.

docs/orchestration-audit/auditor-reports/S2-report.md

@@ -0,0 +1,213 @@
+# Orchestration Audit Report: Session d68aab1b
+
+**Session**: d68aab1b-3e3d-467c-9b9c-70b89e2576b0  
+**Date**: 2026-07-06 14:44–17:58 UTC  
+**Project**: `/home/jared/dev/cc-os`  
+**Duration**: ~3 hours  
+**Assistant**: claude-fable-5  
+**Agent spawns**: 8  
+
+---
+
+## Question 1: Are subagents getting called when they should be?
+
+**Verdict**: MIXED
+
+**Evidence**:
+- Lines 28, 30, 72, 74, 76, 296, 506, 628 each spawn an Agent.
+- Candidate missed delegation flagged: Read x5 (lines 389–405) reading 5 distinct eval scenario files sequentially without an intervening Agent spawn.
+- **Candidate missed delegation flagged**: Write x4 (lines 611–624) writing 4 plan documents sequentially without an intervening Agent spawn.
+
+**Analysis**:
+The orchestrator did delegate 8 times, but the Read x5 and Write x4 sections are valid candidates for delegation judgment because:
+- **Read x5** spans 5 distinct files (P1-L1, P2-L2, P3-L3 scenarios + judge-rubric + check script) in the Eval C harness. These appear to be independent reference reads for understanding the eval, not sequential-dependent work (e.g., not "read A, analyze, decide on B, read B").
+- **Write x4** (lines 611–624) writes 4 separate plan documents (`2026-07-06-plugin-evals-overview.md`, `ws1-orchestration-audit.md`, `ws2-os-vault-write-eval.md`, `ws3-status-convention-plugin.md`). These are tightly coupled by design (they reference each other; ws1–3 are explicit in the overview), making sequential authorship defensible but potentially over-orchestrator-load.
+
+**Missed delegation trigger candidate**:  
+When orchestrator has 5+ independent file reads on **different logical domains** (not a single multi-part document), delegate the read batch to a general-purpose agent with `run_in_background: true` if other work can proceed in parallel. Similarly, when writing 4+ loosely coupled documents with explicit cross-references, consider whether an agent authoring all four together saves context-switching and ensures consistency better than orchestrator authorship.
+
+---
+
+## Question 2: Is the correct model chosen per subagent — highest reasonable quality at lowest cost?
+
+**Verdict**: FAIL
+
+**Evidence**:
+All 8 agents resolved to `claude-haiku-4-5-20251001` despite explicit model parameter mismatches:
+- Agents 1–2 (lines 28, 30): requested `model: "sonnet"` → resolved haiku
+- Agent 3 (line 72): requested `model: "sonnet"` → resolved haiku  
+- Agent 4 (line 74): requested `model: "opus"` → resolved haiku
+- Agent 5 (line 76): requested `model: "sonnet"` → resolved haiku
+- Agents 6–8 (lines 296, 506, 628): requested `model: "haiku"` → resolved haiku (CORRECT, but only by accident)
+
+**Analysis**:
+This is a critical orchestration failure. The orchestrator explicitly passed a `model` parameter 8 times and the Agent tool **completely ignored it**, always resolving to haiku. This is not an orchestrator judgment error (the orchestrator made correct requests); it's a tool-system error upstream (likely the Agent tool definition or the model fallback logic).
+
+The cost impact is mild (haiku is cheapest), but the quality impact is severe: Agents 1–2 (perspectives agents doing epistemic skepticism, which requires sonnet-tier reasoning) and Agent 4 (opus task building a complex eval harness) were systematically downgraded to haiku-only reasoning.
+
+**Missed delegation trigger / missed orchestration signal**:  
+Orchestrator should have detected that the `resolvedModel` field in toolUseResult differed from the requested `model` parameter and either (a) raised an error, (b) acknowledged the downgrade to the user, or (c) re-spawned with explicit routing. None of this happened. This is not orchestrator failure but a missing visibility/accountability layer.
+
+---
+
+## Question 3: Is the orchestrator planning/grouping tasks to maximize efficient context-window use?
+
+**Verdict**: PASS (with caveats)
+
+**Evidence**:
+- Agent spawns 1–5 are clustered (lines 28–76), each conceptually independent, allowing parallel execution (confirmed: all marked `run_in_background: true` implicitly, async_launched).
+- Agents 6–8 (git commits) are spawned sequentially (lines 296, 506, 628) with explicit tool dependencies (each runs after prior work completes), which is defensible.
+- After-spawn-5 segment (37 tool calls) is the heavy work: Reads, Edits, Writes. This was done after agents returned, not during their background execution, indicating the orchestrator waited for results before proceeding.
+
+**Analysis**:
+The grouping is reasonable: epistemic/measurement agents (1–2), task-execution agents (3–5) are fanned out; execution-chain agents (6–8 commits) are sequenced. No evidence of poor context-window planning (e.g., reading a 50MB file then spawning an agent with it in context).
+
+**Caveat**: After-spawn-5's 37 tool calls (41.5KB read) all happen in sequence without delegating any to the agents that just finished. If the Read x5 and Write x4 could have been delegated, this segment could have shed context load off the orchestrator.
+
+---
+
+## Question 4: Is the orchestrator avoiding reading files it does NOT need (that the subagent would read anyway)?
+
+**Verdict**: PASS
+
+**Evidence**:
+- Orchestrator never reads the same file a spawned agent was instructed to read. The 5 Reads at lines 389–405 are in a context where no active agent is reading them (Agents 1–5 already completed; Agents 6–8 are commits with no Read dependency).
+- No evidence of the orchestrator pre-reading files "just in case" an agent might need them.
+- Total bytes read by orchestrator: 41,558 (after-spawn-5) + 61,527 (after-spawn-6) + 24,354 (after-spawn-7) + 563 (after-spawn-8) = ~127.7 KB over a 3-hour session. This is light.
+
+**Analysis**: The orchestrator is not wastefully pre-reading. The flagged Read x5 is the orchestrator needing to understand eval scenarios, not an agent-delegation miss on *this* question.
+
+---
+
+## Question 5: Is the orchestrator sharing too much context with subagents (filling their windows / clouding judgment)?
+
+**Verdict**: PASS
+
+**Evidence**:
+- Agent prompts (lines 28, 30, 72, 74, 76) are terse: 2.5–6.7 KB each. The session has ~70 KB used cumulative before any agent spawn.
+- No evidence of the orchestrator dumping transcripts, full codebases, or search results into agent prompts wholesale.
+
+**Analysis**: Prompts are focused. No over-context problem detected.
+
+---
+
+## Question 6: Is the orchestrator even following the ORCHESTRATION.md instructions?
+
+**Verdict**: FAIL
+
+**Evidence**:
+The global ORCHESTRATION.md policy (injected at SessionStart) states:
+> Do single-file, ≤2-tool-call ops directly. Don't delegate them. Delegate only when work is parallelizable across independent files/subtasks, spans many files, or needs a large/isolated context.
+
+The orchestrator DID delegate appropriately on agents 1–5 (complex, multi-step, reasoning-heavy). BUT:
+- Lines 28, 30, 74: Agent requests explicitly passed `model: "sonnet"` and `model: "opus"` parameters, and those were **systematically ignored** (all resolved to haiku). The instructions say "Every Agent spawn passes model explicitly. Default haiku for mechanical file-edit/shell work; sonnet for anything requiring judgment; opus only for genuinely hard reasoning." The orchestrator was right to request sonnet/opus, but the tool did not respect it.
+- The orchestrator did NOT acknowledge or escalate this model-resolution failure, violating the implicit "verify your agent actually got the model you requested" accountability.
+
+**Specific failed instruction**:  
+> Every `Agent` spawn passes `model` explicitly.
+
+The orchestrator did pass it; the tool ignored it. This is a tool-system error, not an orchestrator error, but the orchestrator should have caught it.
+
+---
+
+## Question 7: Is the orchestrator requesting/receiving back only the context it needs, rather than a full context dump from the subagent?
+
+**Verdict**: PASS
+
+**Evidence**:
+- Agent spawn prompts are specific: "Challenge eval validity claims" (line 28), "Run 3-rep haiku W3 check" (line 72), "Build Eval C..." (line 74), etc.
+- No evidence of the orchestrator requesting "summarize everything you did" or consuming 50+ KB of agent output. The fact-sheet shows result sizes of 2.9–12.9 KB, which are reasonable summaries.
+
+**Analysis**: The orchestrator is precise in its requests and does not bloat context with full result dumps.
+
+---
+
+## Summary of Verdicts
+
+| Q | Verdict | Issue |
+|---|---------|-------|
+| 1 | MIXED | Read x5 and Write x4 are delegation candidates; judgment needed on sequential dependency |
+| 2 | FAIL | All 8 agents requested sonnet/opus but resolved to haiku; tool ignored `model` parameter |
+| 3 | PASS | Grouping is sensible; agents 1–5 fanned out, 6–8 sequenced correctly |
+| 4 | PASS | No wasteful pre-reading or file duplication |
+| 5 | PASS | Agent prompts are focused, not over-contextualized |
+| 6 | FAIL | Model parameter requests were ignored; orchestrator did not escalate the failure |
+| 7 | PASS | Agent outputs were summaries, not full context dumps |
+
+---
+
+## Candidate Missed Delegations (Auditor Judgment Required)
+
+### Delegation Candidate 1: Read x5 (lines 389–405)
+
+**Files read**:
+- `/home/jared/dev/cc-os/plugins/os-adr/eval-c/scenarios/P1-L1-execution.md`
+- `/home/jared/dev/cc-os/plugins/os-adr/eval-c/scenarios/P2-L2-execution.md`
+- `/home/jared/dev/cc-os/plugins/os-adr/eval-c/scenarios/P3-L3-execution.md`
+- `/home/jared/dev/cc-os/plugins/os-adr/eval-c/judge-rubric.md`
+- `/home/jared/dev/cc-os/plugins/os-adr/eval-c/bin/check`
+
+**Context**: Orchestrator is reviewing the Eval C harness after Agent 5 (eval-building) completed. These reads appear to be for understanding the structure and validating/analyzing the built scenarios.
+
+**Sequential dependency risk**: LOW. These are independent reference reads, not "read A → decide → read B" chains. They could be batched to a single agent with `run_in_background: true`.
+
+**Verdict**: Legitimate orchestrator work, but could be delegated if the reads are analytical (not decisional) and the orchestrator is not actively synthesizing between them.
+
+### Delegation Candidate 2: Write x4 (lines 611–624)
+
+**Files written**:
+- `/home/jared/dev/cc-os/docs/plans/2026-07-06-plugin-evals-overview.md` (main overview)
+- `/home/jared/dev/cc-os/docs/plans/ws1-orchestration-audit.md` (WS1)
+- `/home/jared/dev/cc-os/docs/plans/ws2-os-vault-write-eval.md` (WS2)
+- `/home/jared/dev/cc-os/docs/plans/ws3-status-convention-plugin.md` (WS3)
+
+**Context**: Orchestrator is authoring a 4-document plan set. The overview references all three workstreams; each workstream doc is independent but tightly coupled.
+
+**Sequential dependency**: HIGH. The overview must exist (or be drafted) before the workstream docs can be written correctly, OR all four must be authored together to ensure cross-references are accurate.
+
+**Verdict**: Defensibly orchestrator work due to coupling, but authoring all four as a delegated batch to a single agent might improve consistency and save context-switching. The coupling suggests an agent would be clearer on intent than sequential orchestrator writes.
+
+---
+
+## Failure-Mode Clusters (from Real Evidence)
+
+### Cluster A: Model Parameter Ignored
+
+**Symptom**: Orchestrator explicitly passes `model: "sonnet"` or `model: "opus"` to Agent, but `resolvedModel` in the response is always `haiku`.
+
+**Severity**: HIGH
+
+**Trigger**: Any Agent spawn with a non-haiku model parameter.
+
+**Expected behavior**: `resolvedModel` should match `model` param, or Agent should raise an error.
+
+**Actual behavior**: Silent downgrade to haiku.
+
+**Candidate eval scenario**: Spawn agents with mixed models (sonnet, opus, haiku) and verify resolvedModel matches or error is raised.
+
+---
+
+## Recommendations
+
+1. **Investigate model parameter loss**: The consistent 8/8 downgrade to haiku suggests a systematic issue in the Agent tool or model resolution logic, not orchestrator error. Recommend diagnosing why the `model` param is being ignored.
+
+2. **Add model-parameter validation**: Orchestrator should log a warning or error if `resolvedModel != model` param (if one was provided).
+
+3. **Read batch delegation**: Consider delegating Read x5 to a background agent with explicit scope ("read and summarize the structure of these 5 eval-c files").
+
+4. **Write batch delegation**: Consider authoring multi-document plan sets via a single agent to ensure cross-reference consistency. This is less about cost and more about coherence.
+
+5. **Model selection refinement**: Even if the parameter-passing issue is fixed, confirm that sonnet is the right choice for perspectives agents and opus is justified for eval-harness building. The current choices seem sound, but the downgrade hides the opportunity to validate them.
+
+---
+
+## Session Metadata
+
+- **Main loop model**: claude-fable-5 (orchestrator)
+- **Agent models**: all resolved to haiku (intended: mixed)
+- **Total agent spawns**: 8
+- **Parallel agents**: 5 (1–5, concurrent, bg)
+- **Sequential agents**: 3 (6–8, git commits)
+- **Total orchestrator tool calls**: 85 (excluding sidechain)
+- **Total bytes read by orchestrator**: 127.7 KB
+- **Candidate issues**: Model downgrade (Q2, Q6); delegation judgment needed (Q1)

docs/orchestration-audit/auditor-reports/S3-report.md

@@ -0,0 +1,191 @@
+# Session S3 Orchestration Audit Report
+**Session:** 9f45afcc-75ff-4b7a-8f3d-008aa0d599af  
+**Date:** 2026-07-04  
+**Duration:** 2h 35m  
+**Transcript lines:** 850 (28 sidechain, 5 spawns, 255 assistant turns)
+
+---
+
+## SEVEN-QUESTION AUDIT RUBRIC
+
+### 1. Are subagents getting called when they should be?
+
+**Verdict:** MIXED — Delegation is reasonable but spawn #1 might have been avoidable
+
+**Evidence:**
+- Line 124 (spawn 1): Plugin cache refresh. Prompt specifies 4 steps: determine refresh method, execute `claude plugin` commands, verify caches, refresh cc-plugins marketplace. This is multi-step but mechanical — deterministic shell commands, no judgment. The orchestrator did 19 tool calls in pre-span-1 (14 Bash, 4 Read, 1 Edit) to investigate the problem (lines ~5-120: searched for files, checked plugin state, read naming conventions). That investigation work could have been delegated as part of the same spawn rather than consuming 19 sequential tool calls in the main context.
+
+- Lines 133, 171, 603, 605 (spawns 2–5): Eval grid runs and wording triage. These require running deterministic harnesses repeatedly or evaluating behavior hypothetically — work that spans multiple independent runs and benefits from isolated context. Delegation appropriate.
+
+- Fact-sheet notes "None flagged by heuristic" for runs of ≥4 same-tool calls that should have been delegated. However, the 19-call pre-span-1 segment (Bash-heavy, grep/ls/find) was necessary orientation per ORCHESTRATION.md guidance: "A short orienting Read before delegating is fine when the target file/path is uncertain. Don't delegate the orienting step itself." The Reads (4) were all strategic — to understand plugin naming conventions, review prior eval results, check CLAUDE.md design — and the Bash calls were discovery-only, not modifying.
+
+**Assessment:** Delegation thresholds met for spawns 2–5 (eval work needs isolation). Spawn 1 (cache refresh) is borderline: it's multi-step and beneficial to isolate, but purely mechanical and could have been done directly by the orchestrator. The pre-spawn work (investigation) was correctly kept in-session per policy.
+
+---
+
+### 2. Is the correct model chosen per subagent?
+
+**Verdict:** FAIL — All 5 spawns suffered model degradation
+
+**Evidence:**
+- Line 124: Model param `"sonnet"` → resolved `claude-haiku-4-5-20251001` (Haiku)
+- Line 133: Model param `"sonnet"` → resolved `claude-haiku-4-5-20251001` (Haiku)
+- Line 171: Model param `"sonnet"` → resolved `claude-haiku-4-5-20251001` (Haiku)
+- Line 603: Model param `"sonnet"` → resolved `claude-haiku-4-5-20251001` (Haiku)
+- Line 605: Model param `"haiku"` → resolved `claude-haiku-4-5-20251001` (Haiku) ✓
+
+**Analysis:**
+All 5 Agent tool calls explicitly set `model: "sonnet"` (lines 124, 133, 171, 603) or `model: "haiku"` (line 605), but 4 of the 5 resolved to `haiku` instead. This is either:
+1. A platform issue where `model` parameter is not honored, or
+2. Model unavailability/fallback behavior
+
+The orchestration policy states: "Every Agent spawn passes model explicitly. Default haiku for mechanical file-edit/shell work; sonnet for anything requiring judgment."
+
+**Quality impact:**
+- Spawn 1 (cache refresh): Mostly mechanical (shell commands). Haiku adequate, possibly optimal.
+- Spawn 2 (re-run Eval B grid): Requires running a deterministic harness. Haiku adequate.
+- Spawn 3 (resume eval grid): Continuation of eval. Haiku adequate for procedural execution.
+- **Spawn 4 (triage wording on sonnet):** Explicitly judgment work — "evaluate how you would behave in a hypothetical coding session… answer from judgment alone." This requires reasoning about whether to propose an ADR. Haiku is below the required capability level. **REAL MISS.**
+- Spawn 5 (triage wording on haiku): Correctly specified and received haiku. Appropriate.
+
+Spawn 4's prompt (line 603) begins: "You are evaluating how you would behave in a hypothetical coding session. Do NOT use any tools — answer from judgment alone." This is explicitly a judgment task. The fact that it got haiku instead of sonnet is a capability downgrade that likely impacted the evaluation's fidelity.
+
+---
+
+### 3. Is the orchestrator planning/grouping tasks to maximize efficient context-window use?
+
+**Verdict:** MIXED — Reasonable grouping but significant sequential work not delegated
+
+**Evidence:**
+- 255 assistant turns for 5 spawns = ~50 lines per spawn (including setup/handling)
+- After-spawn-3: 48 tool calls (ToolSearch 2, SendMessage 1, Bash 22, Edit 12, Skill 2, Read 5, Write 3, TaskStop 1)
+- After-spawn-5: 43 tool calls (Write 4, Bash 27, Edit 11, Read 1)
+
+The after-spawn-3 and after-spawn-5 segments show the orchestrator managing the eval process directly with many tool calls. This is work that _could_ have been grouped into a single eval-management delegation, but instead was split:
+- Spawn 2 (line 133): "Re-run Eval B grid post-fix" → 0 orchestrator calls afterward
+- Spawn 3 (line 171): "Resume eval grid agent" → 48 orchestrator calls managing results
+- Spawn 4 (line 603): "Triage wording on sonnet" → 0 calls
+- Spawn 5 (line 605): "Triage wording on haiku" → 43 orchestrator calls writing/processing results
+
+The orchestrator could have delegated the result-processing and writing work (e.g., "Run evals 4-5, process results, write summary") as part of the agent prompts rather than managing it in-session. This would have freed up the main context for other work.
+
+**Assessment:** Grouping is functional but leaves optimization on the table. Sequential dependent work (eval → triage → result processing) could have been bundled.
+
+---
+
+### 4. Is the orchestrator avoiding reading files it does NOT need?
+
+**Verdict:** PASS — Reads are minimal and strategic
+
+**Evidence:**
+- Pre-span-1 Reads (lines ~5-120):
+  1. `/home/jared/Documents/SecondBrain/cc-os-plugin-skill-naming-convention.md` — necessary to understand naming rules before updating CLAUDE.md
+  2. `/home/jared/Documents/SecondBrain/2026-07-03-os-adr-eval-b-grid-results-and-observations.md` — necessary to understand prior eval baseline before re-running
+  3. `/home/jared/dev/cc-os/CLAUDE.md` (read twice) — necessary to understand design before updating plugin cache info and to check current state
+
+- After-span-3: Read 5 files (not listed individually, but were part of eval result handling)
+- After-span-5: Read 1 file
+
+Total reads: ~4 strategic pre-delegation, then tactical reads during result handling. No superfluous reads evident. ORCHESTRATION.md guidance ("A short orienting Read before delegating is fine") is followed.
+
+---
+
+### 5. Is the orchestrator sharing too much context with subagents?
+
+**Verdict:** PASS — Prompts are focused and appropriately sized
+
+**Evidence:**
+- Spawn 1 (line 124): 4,849 chars — detailed context about the cache problem, specific files missing, clear multi-step task definition
+- Spawn 2 (line 133): 2,177 chars — context about the cache fix, baseline results to compare, explicit procedure reference
+- Spawn 3 (line 171): 49 chars — minimal ("Resume eval grid agent"), relying on prior task continuation via SendMessage
+- Spawn 4 (line 603): 2,193 chars — a complete hypothetical scenario with task definition
+- Spawn 5 (line 605): 2,193 chars — same length as spawn 4 (paired triage tasks)
+
+The prompts are task-focused. Spawn 3's minimal prompt (49 chars) is appropriate for a SendMessage continuation rather than a fresh spawn. Spawns 4-5 are large but necessary: they define the hypothetical scenario and judgment task in full.
+
+No evidence of context overload or extraneous background.
+
+---
+
+### 6. Is the orchestrator following ORCHESTRATION.md instructions?
+
+**Verdict:** MIXED — Mostly compliant but borderline on single-file/≤2-tool rule
+
+**Evidence:**
+ORCHESTRATION.md policy:
+> "Do single-file, ≤2-tool-call ops directly. Don't delegate them. Delegate only when work is parallelizable across independent files/subtasks, spans many files, or needs a large/isolated context."
+
+**Analysis:**
+- **Spawn 1 (plugin cache refresh):** Multi-step work across 4 plugins (os-adr, os-doc-hygiene, os-vault, os-orchestration, plus cc-plugins marketplace). Spans multiple files. Delegated. ✓ Compliant.
+- **Spawn 2-5 (eval grids):** Requires running headless harnesses repeatedly. Benefits from isolated context. Delegated. ✓ Compliant.
+- **After-spawn-3/5 work:** The orchestrator executed 48 + 43 = 91 tool calls _directly_, not delegated, to manage eval results, write reports, and update documentation. This includes:
+  - Line 172-300 region: multiple Bash (running eval binaries), Edit (patching files), Write (creating outputs), Skill (invoking tools), ToolSearch (finding contexts)
+  - The work is sequential-dependent (eval runs → check results → triage → write), which is a valid reason not to delegate per policy ("sequential-dependent work is a valid reason not to delegate" per fact-sheet guidance)
+
+**Verdict:** Policy is mostly followed. The pre-span-1 investigation (19 calls) was appropriately kept in-session as orientation. The after-span work (91 calls) is sequential-dependent, so not delegating was correct. However, spawn 1 itself might have been avoidable if the investigation had been folded into the spawn prompt more aggressively — e.g., "investigate AND fix" in one delegation rather than investigate-in-session then delegate-fix.
+
+---
+
+### 7. Is the orchestrator requesting/receiving back only needed context?
+
+**Verdict:** PASS — Results are actionable and used directly
+
+**Evidence:**
+- **Spawn 1 result (line 125):** 3,951 chars. Returned structured summary + file attachment showing CLAUDE.md update. The orchestrator immediately incorporated this (line 126 shows attachment of the edited file). No surplus context; the file edit was the actionable outcome.
+  
+- **Spawn 2 result (line 134, resolved async):** 5,905 chars. Returned full grid results (haiku 0/8, sonnet 5/8) with per-cell analysis. Was this needed? The orchestrator would need the baseline results to inform spawn 3 (resume eval) and spawns 4-5 (triage wording). Appropriate size for the information density.
+
+- **Spawn 3 result (via task notification, line 172):** 878 chars. Minimal, continuation of spawn 2's work. Appropriate.
+
+- **Spawn 4-5 results:** Not directly readable in the transcript (async), but after-span-5 work shows the orchestrator writing outputs and using the results (lines ~603-850 have multiple Write calls with evaluation data).
+
+No evidence of the orchestrator requesting excessive results or receiving data that wasn't used. Results are structured and actionable.
+
+---
+
+## SUMMARY VERDICTS
+
+| Question | Verdict | Key Issue |
+|----------|---------|-----------|
+| 1. Subagents called correctly? | MIXED | Spawn 1 (cache refresh) could have been done directly; pre-span investigation was well-scoped |
+| 2. Correct model chosen? | **FAIL** | All 5 spawns specified sonnet but 4 resolved to haiku; spawn 4 (judgment task) impacted by downgrade |
+| 3. Tasks grouped efficiently? | MIXED | Result handling (91 calls post-eval) could have been delegated as part of eval task |
+| 4. Avoiding unnecessary reads? | PASS | 4 strategic Reads before delegation; tactical reads during handling. Minimal overhead. |
+| 5. Sharing excessive context? | PASS | Prompts are task-focused, appropriately sized (49–4,849 chars). |
+| 6. Following ORCHESTRATION.md? | MIXED | Mostly compliant; sequential work correctly kept in-session; but spawn 1 could have bundled investigation+fix |
+| 7. Requesting only needed context? | PASS | Results are structured, actionable, immediately used. No surplus. |
+
+---
+
+## TOP MISSES WITH LINE REFERENCES
+
+### 1. Model Degradation (CRITICAL)
+**Lines 124, 133, 171, 603:** All Agent calls explicitly set `model: "sonnet"` but resolved to haiku.
+- **Criterion:** When a session specifies a model parameter on Agent spawn, that parameter should be honored or the orchestrator should be notified of fallback.
+- **Trigger:** Implement model-param validation in Agent tool or platform-level fallback alerting.
+
+### 2. Judgment Task Ran on Haiku (REAL IMPACT)
+**Line 603:** Spawn 4 prompt explicitly says "Do NOT use any tools — answer from judgment alone" but requested `sonnet` and got `haiku`.
+- **Criterion:** Judgment-heavy tasks (scenario evaluation, wording triage) require sonnet; the orchestrator specified correctly but did not receive what it asked for.
+- **Trigger:** When specifying a model for judgment work, verify the resolved model matches; escalate if haiku is substituted.
+
+### 3. Post-Eval Work Could Have Been Delegated
+**Lines 172–300 (48 calls), 603–850 (43 calls):** Orchestrator manually managed eval result processing, writing, and triage.
+- **Criterion:** "Triage wording on sonnet/haiku" spawns could have included "process results, write summary, save to file" as part of the agent prompt rather than post-processing in-session.
+- **Trigger:** When a sequence of evals feeds into analysis+writing, bundle the analysis+writing into the final eval spawn or create a post-processing delegation.
+
+### 4. Pre-Delegation Investigation Could Have Been Bundled
+**Lines 5–120 (19 calls):** Orchestrator investigated plugin cache status before delegating the fix.
+- **Criterion:** Investigatepattern + action pattern could be grouped into one "investigate and fix" delegation.
+- **Trigger:** When investigation leads directly to a bounded task, consider bundling: "investigate X, then fix it per these steps" as a single delegation.
+
+---
+
+## CONTAMINATION NOTES
+
+This session (2026-07-04, POST global os-orchestration rollout) was actively working on eval and orchestration tooling (os-adr eval, wording-loop setup). The 255 turns reflect:
+- Plugin cache maintenance (spawn 1)
+- Eval baseline validation (spawns 2-3)
+- Wording triage setup (spawns 4-5)
+
+The low spawn count (5 for 255 turns) is partly intentional: evals run deterministically, their management is procedural. However, model degradation across all 5 spawns is the session's most concerning artifact and suggests either a platform bug or a systematic misunderstanding of model availability.

docs/orchestration-audit/auditor-reports/S4-report.md

@@ -0,0 +1,207 @@
+# Orchestration Audit Report: S4 (b6b07a91)
+
+**Session**: llf-schema, 2026-06-30, feat/harden-schema-verification  
+**Model**: claude-opus-4-8 (main loop)  
+**Agent spawns**: 41 total  
+**Audit scope**: ORCHESTRATION.md compliance + model selection efficiency  
+**Policy under audit**:
+- Do single-file, ≤2-tool ops directly. Delegate only when parallelizable across independent files/subtasks, spans many files, or needs large/isolated context.
+- **Every `Agent` spawn passes `model` explicitly.** Default haiku for mechanical, sonnet for judgment, opus for hard reasoning.
+- A short orienting Read before delegating is fine when target paths are uncertain. Don't delegate the orienting step itself.
+
+---
+
+## Answer to Seven Questions
+
+### 1. Are subagents getting called when they should be?
+
+**Verdict: PASS**
+
+The orchestrator appropriately delegates tasks that span multiple files, require isolated execution (git operations, test runs), or benefit from background parallelization. All spawns are for compound tasks:
+- Multi-step OpenSpec task implementation (spawns 2-3, 6-7, 9-10, 14-15, 19, 33-34)
+- Isolated environment testing/diagnostics (spawns 17, 20-22, 29-30, 38, 40)
+- Background-capable reads (spawns 27-28: Explore agents for wide searches)
+- Deterministic file updates (spawns 4-5, 8, 12, 16, 32: checkbox flips, memory writes with haiku)
+
+No evidence of single-file/≤2-tool ops being delegated when they should have been direct.
+
+**Evidence**: 
+- Line 21: spawn 1 "Gather openspec change context" — could be viewed as optional given that `/opsx:apply` already provided structured task context, but investigating OpenSpec state is defensible as a compound prep task
+- Lines 36-77: spawns 2-7 (task bundle 1.1-3.3 + mark complete) — appropriately split: 1.1 code edit (sonnet), 1.2 golden fixture + profiles (sonnet), 3.1 fixtures (sonnet), 3.2 self-check (sonnet), 3.3 doc (haiku), then mark-complete (haiku)
+
+### 2. Is the correct model chosen per subagent — highest reasonable quality at lowest cost?
+
+**Verdict: FAIL** (policy violation)
+
+**Model param omissions (9 spawns violate explicit-model policy):**
+- Spawns 1, 24, 25, 26, 29, 30, 31: general-purpose without model param → **default to opus** (inappropriate — should be explicitly specified as sonnet or haiku)
+- Spawns 27-28: Explore agents without model param → default to haiku (acceptable for read-only search, though should still be explicit)
+
+**Where models ARE specified, selection is sound:**
+- Haiku (spawns 4, 8, 12, 16, 32, 35, 36, 39, 47): mechanical tasks (checkbox flips, schema sample-verify, post cache purge, memory writes) — cost-optimized
+- Sonnet (spawns 2-3, 6-7, 9-11, 13-15, 17-23, 33-34, 37-38, 40-41): code edits, fixture design, verifier logic, multi-file diagnostics, schema verification — appropriate judgment tier
+- Opus (none explicitly specified, only defaults via omission): the omissions that defaulted to opus are under-specified
+
+**Critical violation**: Policy states "Every `Agent` spawn passes `model` explicitly." Lines 21, 411, 427, 440, 466, 494, 496, 529, 563 (spawns 1, 24-31) omit the parameter:
+
+- **Line 21** (spawn 1, general-purpose, context gathering, 1275 prompt chars, 14627 result chars): **model field absent** from `input` dict. Should be explicit.
+- **Line 411** (spawn 24, "Commit harden-schema-verification", git + task update + commit, 2766 prompt chars): **model field absent**. Resolves to opus; should be explicit (likely sonnet for judgment, not opus default).
+- **Line 427** (spawn 25, "Run release.sh patch to v0.2.5", shell operation, 2126 prompt chars): **model field absent**. Opus default is overkill for script execution; should be haiku.
+- **Line 440** (spawn 26, "Check live service-page WebPage.description", remote HTTP checks + diagnosis, 2311 prompt chars): **model field absent**. Resolves to opus; appropriate outcome but should be explicit.
+- **Lines 466, 494** (spawns 27-28, Explore agents, read-only searches): **no model param**. Explore agents are read-only and lower-cost, so defaulting to haiku is reasonable, but policy requires explicit specification.
+- **Lines 496, 529, 563** (spawns 29-31, general-purpose, 2423–5292 prompt chars): **model field absent**. Default to opus; three more instances of the pattern.
+
+**Evidence of appropriate choices where specified**:
+- Line 38 (spawn 3, sonnet for profiles/verifiers, 4457 prompt chars): Correct tier for multi-file schema design
+- Line 75 (spawn 6, sonnet for golden fixture regeneration, 3684 prompt): Correct tier for oracle-based fixture design
+- Line 155 (spawn 13, sonnet for URL fix + test update, 1011 prompt): Appropriate for code fix with test understanding
+
+### 3. Is the orchestrator planning/grouping tasks to maximize efficient context-window use?
+
+**Verdict: PASS**
+
+The orchestrator groups tasks into logical batches that respect dependencies while allowing parallelization:
+
+**Batch structure**:
+- Pre-batch 1 (spawns 1): context gathering
+- Batch 1 (spawns 2-5): tasks 1.1, 1.2, 3.1, 3.3 + mark complete (spawn 5 is deterministic checkbox-flip, unblocked after parallel executions of spawns 2-4 return)
+- Batch 2 (spawns 6-8): task 1.2 regen, 3.2 self-check, mark complete
+- Batch 3 (spawns 9-12): tasks 4.1, 4.2, 5.1, reconcile + mark complete
+- Batch 4 (spawns 13-16): verify gate, commit, mark complete
+- Batch 5 (spawns 17-23): ADR verification, docblock fix, release run, three live diagnostics
+- Batch 6 (spawns 24-32): commit, release, live check, spec exploration, post_excerpt check, relax + re-release, push, memory write
+- Batch 7 (spawns 33-41): schema verification + finalization
+
+**Evidence**:
+- Spawns 2-4 are independent (code, profiles, doc) and can run in parallel; spawn 5 (mark complete) only needs serial ordering after 2-4
+- Spawns 27-28 (Explore agents) are parallelizable read-only searches, appropriately spawned together without blocking
+- Spawns 33-36 (schema verification for PLD and SDD) are independent and parallelizable
+
+### 4. Is the orchestrator avoiding reading files it does NOT need (that the subagent would read anyway)?
+
+**Verdict: PASS** (with caveat)
+
+**Orchestrator tool profile**: The fact-sheet shows **0 total tool calls** across all 41 segments (post-spawn). The orchestrator never read, wrote, or grepped any files directly — all work was delegated.
+
+This is **not a violation** because:
+1. The orchestrator is executing a structured OpenSpec change, where task definitions come from `/opsx:apply harden-schema-verification` — context is pre-loaded
+2. File paths and task text are explicit in the change spec, so the orchestrator doesn't need to read files to determine what to delegate
+3. Subagents have clear prompts specifying files to read; no redundant pre-delegation reads are necessary
+
+However, a **cautionary note**: Had the task been exploratory rather than spec-driven, the zero reads might indicate missing orientation.
+
+**Evidence**:
+- Lines 1-20 (before spawn 1): OpenSpec skill loads tasks and context; main-loop model has this in cache/additionalContext
+- Each spawn prompt contains explicit file paths (e.g., spawn 2: `/home/jared/dev/llf-schema/openspec/changes/harden-schema-verification/design.md` D-047)
+- No agent result references "file path unclear" or "orchestrator didn't tell me where to look"
+
+### 5. Is the orchestrator sharing too much context with subagents (filling their windows / clouding judgment)?
+
+**Verdict: PASS**
+
+Prompt lengths are appropriately scoped:
+- Mechanical tasks (spawns 4, 8, 12, 16, 32): 436–659 chars (tight, single-purpose)
+- Code edits (spawns 2, 6, 13, 18): 1011–4457 chars (design context + task instructions)
+- Diagnostics (spawns 17, 20-22, 38, 40): 2247–5292 chars (problem statement + investigation steps)
+
+**Evidence**:
+- Spawn 4 (coverage checklist, 512 chars): minimal context, haiku-appropriate
+- Spawn 17 (ADR verification, 3462 chars with 10703 char result): comprehensive but structured; result size justified by need to return full verification matrix
+- Spawn 20 (NJ LegalService diagnosis, 3499 chars): investigation is scoped to a single problem; not bloated with unrelated context
+
+**No instance of context pollution** (e.g., dumping 20 files of unrelated code, full deployment logs, etc.).
+
+### 6. Is the orchestrator following the ORCHESTRATION.md instructions?
+
+**Verdict: FAIL** (explicit policy violation)
+
+The orchestrator **VIOLATES the primary policy requirement**: "Every `Agent` spawn passes `model` explicitly."
+
+**Policy adherence summary**:
+- ✅ Single-file/≤2-tool ops: mostly upheld (no evidence of delegating trivial tasks)
+- ❌ **Explicit model parameter per spawn: VIOLATED (9 spawns)**
+- ✅ Model tier selection (where specified): appropriate (haiku→mechanical, sonnet→judgment, opus reserved)
+- ✅ No delegation of orientation reads: correct (context provided by OpenSpec skill)
+
+**The 9 violations** are not hidden — they are systematic. The pattern suggests possible causes:
+1. **Unintended omission**: The orchestrator (opus) generated spawns but forgot to include `model:` in 9 of them
+2. **Assumption of smart defaults**: The orchestrator may have assumed that omitting `model` on general-purpose agents defaults to sonnet, but it defaults to opus instead
+3. **Model-selection logic change**: If the agent-spawning code changed its behavior between when the policy was written and when this session ran, the orchestrator's output may not match the policy
+
+**Evidence of violation**:
+- Line 21: `"input": {"description": ..., "prompt": ..., "subagent_type": "general-purpose"}` — **no "model" key**
+- Line 36: `"input": {"description": ..., "prompt": ..., "subagent_type": "general-purpose", "model": "sonnet"}` — correct, HAS model key (for comparison)
+- Lines 411, 427, 440: same pattern (missing model key) on spawns 24-26
+- Lines 466, 494: Explore agents also omit model (though haiku default is less problematic)
+- Lines 496, 529, 563: spawns 29-31 continue the pattern
+
+### 7. Is the orchestrator requesting/receiving back only the context it needs, rather than a full context dump from the subagent?
+
+**Verdict: PASS**
+
+Each spawn includes specific instructions for what to return:
+
+**Examples of well-scoped return specs**:
+- Spawn 2 (line 36): "Return: a brief summary of exactly what you changed (with before/after of both edits) and the absolute path"
+- Spawn 4 (line 40): "Return the file location and absolute path"
+- Spawn 13 (line 155): (implicit: the fix is in the file edit, no long report needed)
+- Spawn 17 (line 219): "Return: a verification matrix (task / expected result / actual result / pass/fail) — one row per assertion"
+- Spawn 20 (line 265): "Return: (1) the exact JSON difference … (2) the code path … (3) the classification … (4) a recommended fix direction"
+- Spawn 32 (line 581): "Return: the absolute file path of the created memory file + the memory file's absolute path for indexing" (actually says "just report which files were written")
+
+Result sizes (fact-sheet column 9) show reasonable variance:
+- Small (749 chars): spawns 37-38 (issue filing, brief confirmations)
+- Medium (1200–3500 chars): spawns 2-3, 6-11, 13-15 (code edits, design decisions)
+- Large (5000–10700 chars): spawns 17, 20-22 (detailed diagnostics/comparisons)
+
+**The large results are justified by the task scope.** Spawn 17 (verify ADRs, 10703 result chars) needs to return a full matrix to justify pass/fail. Spawn 20 (diagnose NJ issue, 5041 result chars) needs to return JSON diffs + root-cause analysis.
+
+No evidence of subagents returning full logs or unfiltered dumps when a summary would do.
+
+---
+
+## Candidate Missed Delegations
+
+The fact-sheet heuristic (≥4 same-tool calls, distinct targets, no spawn between) flagged: **None**.
+
+Manual review confirms: The fact-sheet's orchestrator tool profile shows **0 tool calls across all segments**. There are no runs of repeated tool use that could have been parallelized — all work was delegated.
+
+---
+
+## Summary: Key Misses
+
+| Question | Verdict | Top Miss | Line Refs |
+|----------|---------|----------|-----------|
+| 1. Subagent delegation appropriateness | PASS | None notable | N/A |
+| 2. Model choice efficiency | FAIL | 9 spawns omit explicit `model` param, default to opus (inappropriate for some) | 21, 411, 427, 440, 466, 494, 496, 529, 563 |
+| 3. Context-window grouping | PASS | Batching is logical; parallelization is efficient | N/A |
+| 4. Avoid pre-delegation reads | PASS | Caveat: zero reads is appropriate here due to OpenSpec context | N/A |
+| 5. Context bloat to subagents | PASS | Prompt lengths are appropriate; return specs are scoped | N/A |
+| 6. Follow ORCHESTRATION.md | FAIL | **Policy violation: "Every Agent spawn passes model explicitly" — 9 spawns are missing** | 21, 411, 427, 440, 466, 494, 496, 529, 563 |
+| 7. Return-context scope | PASS | Spawns request specific summaries/matrices; no dump-the-whole-log behavior | N/A |
+
+---
+
+## Recommended Evaluation Triggers
+
+**For future orchestration audits**, flag these scenarios:
+
+1. **Model parameter omission**: When `Agent` tool_use appears in transcript WITHOUT a `model` field in the `input` dict, the spawn violates explicit-model policy. Cite the line number and count occurrences.
+
+2. **Orchestrator pre-delegation reads**: When a spawn targets a file path that the orchestrator never read first, check if the task is spec-driven (OpenSpec, explicit task list) — if yes, zero pre-reads is fine; if exploratory, it's a miss.
+
+3. **Spawn complexity vs. model tier**: Spawns with missing model params that resolve to opus — check whether the prompt requires judgment (sonnet) or is mechanical (haiku). Mark as inefficient if wrong.
+
+4. **Batch dependency chains**: Verify that spawn ordering respects task dependencies (e.g., "mark complete" only after preceding tasks in that batch are done). This session passes this check.
+
+5. **Return-context bloat**: Scan spawn prompts for return specs; if a spawn has none (implicit "return everything"), or result_chars >>prompt_chars with no apparent justification, flag for review.
+
+---
+
+## Conclusion
+
+The orchestrator manages **task grouping, parallelization, and scoping well**. The delegation decisions are sound — work is split appropriately between main session and subagents, and batching respects dependency ordering.
+
+However, the **model-parameter omission on 9 spawns is a clear policy violation**. This appears to be a systematic pattern rather than one-off oversight. The spawns that omit the parameter default to opus, which is overkill for some (e.g., spawn 25: "Run release.sh" could be haiku for script execution), while under-specifying the orchestrator's intent for others (e.g., spawn 24: commit workflow with multi-step git logic, which should explicitly request sonnet).
+
+**Immediate action**: Verify that the Agent tool's `model` parameter is always included in future orchestration sessions. This session's violation does not substantially degrade the session's outcome (the tasks completed successfully), but it represents a breach of explicit policy that should be corrected.

docs/orchestration-audit/auditor-reports/S5-report.md

@@ -0,0 +1,178 @@
+# Session Audit Report: S5-aadeb66c
+
+**Session:** aadeb66c-0d64-4d41-b407-83e0209428b4 (llf-schema, 2026-06-29)  
+**Policy under audit:** ORCHESTRATION.md (global default; equivalent text lived in project CLAUDE.md at date of session)  
+**Auditor findings:** 7 rubric questions answered with evidence.
+
+---
+
+## Summary Verdicts
+
+| Question | Verdict | Evidence |
+|----------|---------|----------|
+| 1. Are subagents called when they should be? | MIXED | 8 spawns appropriate in scope; model tiers incorrect. Lines 23–141. |
+| 2. Correct model per subagent — quality/cost trade-off? | FAIL | Spawns 1–6 ABSENT model param → resolved sonnet; spawns 2–6 are mechanical file-read/shell work (should be haiku). Lines 23, 24, 41, 60, 83, 99. |
+| 3. Planning/grouping for context-window efficiency? | PASS | Task decomposition into 6 info-gather + 1 implement + 1 verify is sensible. Staggered spawn timing (7–101 min gaps) plausible for sequential dependencies. |
+| 4. Avoiding unnecessary reads by orchestrator? | PASS | Fact-sheet: 0 orchestrator tool calls across all segments. All reads delegated. |
+| 5. Sharing too much context with subagents? | PASS | Spawn prompts 419–838 chars, focused (specific files/checks), no open-ended dumps. |
+| 6. Following ORCHESTRATION.md instructions? | FAIL | Policy: "Every Agent spawn passes model explicitly." Spawns 1–6 violate (no model field). Policy: Default haiku for mechanical work. Spawns 2–6 violate (resolved sonnet). |
+| 7. Requesting/receiving only needed context back? | PASS | Focused prompts, task-notification async results, 0 repeated queries or context dumps by orchestrator. |
+
+---
+
+## Detailed Findings
+
+### Q1: Are subagents getting called when they should be?
+
+**Verdict:** MIXED — Spawning decisions were sound; model assignments were not.
+
+**Evidence:**
+- Lines 23–24: Spawned parallel agents to audit codebase + check live page. Appropriate for independent information-gathering tasks.
+- Lines 41, 60, 83, 99: Staggered spawns for ACF data, implementation context, additional files, remaining files. Timings: 7.5 min after spawn 2 result, then 101 min gap, then 1–2 min intervals. Pattern suggests sequential dependencies or pacing.
+- Lines 115, 141: Implementation (sonnet) then verification (haiku) spawns. Correct decomposition.
+- **Issue:** Spawns 2–6 (pure file reads and WebFetch) did not require judgment and should not have been delegated at all per policy—or if delegated, should have been haiku. This is a model-selection issue, not a spawning issue.
+
+### Q2: Is the correct model chosen per subagent?
+
+**Verdict:** FAIL — 5 of 8 spawns used incorrect (overspend) model tier.
+
+**Evidence:**
+- **Fact-sheet table:**
+  - Spawn 1 (line 23): model=ABSENT → resolved sonnet. Codebase audit requires judgment. **Correct tier (sonnet).**
+  - Spawn 2 (line 24): model=ABSENT → resolved sonnet. WebFetch live page for schema presence. **Mechanical task; should be haiku.**
+  - Spawn 3 (line 41): model=ABSENT → resolved sonnet. Run wp-cli SSH commands to check ACF data. **Mechanical shell work; should be haiku.**
+  - Spawn 4 (line 60): model=ABSENT → resolved sonnet. Read Plugin.php and other context files verbatim. **Pure file reading; should be haiku.**
+  - Spawn 5 (line 83): model=ABSENT → resolved sonnet. Read additional context files verbatim. **Pure file reading; should be haiku.**
+  - Spawn 6 (line 99): model=ABSENT → resolved sonnet. Read remaining files verbatim. **Pure file reading; should be haiku.**
+  - Spawn 7 (line 115): model=sonnet → resolved sonnet. Implement ContactPagePiece with tests. **Requires judgment; correct (sonnet).**
+  - Spawn 8 (line 141): model=haiku → resolved haiku. Verify implementation files. **Mechanical checklist; correct (haiku).**
+
+- **Policy statement:** "Default `haiku` for mechanical file-edit/shell work; `sonnet` for anything requiring judgment; `opus` only for genuinely hard reasoning."
+
+- **Cost implication:** Spawns 2–6 used 5× sonnet tokens where haiku would have sufficed. Over a 101-minute window (17:02–18:43), the orchestrator paced the spawns, suggesting it waited for intermediate results; this is defensible, but the model tiers remain wrong.
+
+### Q3: Is the orchestrator planning/grouping tasks to maximize efficient context-window use?
+
+**Verdict:** PASS — Decomposition is sensible; parallelization timing is unclear but plausible.
+
+**Evidence:**
+- **Task grouping:** 
+  - Phase 1 (lines 23–24): Parallel information-gathering (codebase + live page).
+  - Phase 2 (line 41): Conditional follow-up (ACF data check, after learning the plugin isn't deployed).
+  - Phase 3 (lines 60, 83, 99): Sequential context reads (Plugin.php, schema implementations, test fixtures).
+  - Phase 4 (line 115): Implementation (sonnet, using all gathered context).
+  - Phase 5 (line 141): Verification (haiku).
+
+- **Timing pattern:** Spawns 1–2 at 16:54:47–16:54:51 (4-sec gap, parallel). Then 7.5-min gap before spawn 3 (17:02:17). Then 101-min gap before spawn 4 (18:43:29). Smaller gaps (1–2 min) between spawns 4–6.
+  - **Interpretation:** The orchestrator waited for earlier phase results (visible in line 28 text: "Here's the full picture... **Two separate problems:** 1. **The plugin isn't deployed on SDD at all.**") before spawning conditional work. This is reasonable if task 3 depends on knowing whether the plugin is live.
+  - **Question unresolved:** Could spawns 4–6 have been parallelized at the 18:43 mark instead of staggered? Fact-sheet heuristic found "None flagged," meaning no runs of ≥4 same-tool calls without spawns, so sequential-dependent work is plausible.
+
+- **Window efficiency:** Each spawn prompt is 419–838 chars. Full agent contexts (including codebase orientation) are not visible in spawns alone, but the orchestrator delegated all context-intensive work to subagents and made zero tool calls itself (next section), which is correct framing.
+
+### Q4: Is the orchestrator avoiding reading files it does NOT need (that the subagent would read anyway)?
+
+**Verdict:** PASS — Orchestrator made zero tool calls before/after spawns.
+
+**Evidence:**
+- **Fact-sheet orchestrator tool profile:**
+  ```
+  | segment | calls | bytes read |
+  |---------|-------|------------|
+  | pre-spawn-1 | 0 | 0 |
+  | after-spawn-1 through after-spawn-8 | 0 | 0 |
+  ```
+- **Implication:** The orchestrator did not call Read, Bash, Grep, or any other tool. All file reads, WebFetch requests, and SSH commands were delegated to subagents. This avoids duplication and respects the "delegate inspection to subagents" principle.
+- **Note:** No evidence of "orienting Read" per policy exemption ("A short orienting Read before delegating is fine when the target file/path is uncertain"); either (a) the orchestrator did not need orientation (task was clear from user prompt), or (b) orientation was done in previous turns not shown in this 161-line segment. Either way, no unnecessary reads in the shown span.
+
+### Q5: Is the orchestrator sharing too much context with subagents (filling their windows / clouding judgment)?
+
+**Verdict:** PASS — Spawn prompts are focused and appropriately scoped.
+
+**Evidence:**
+- **Spawn prompt lengths:** 419–838 chars (from grep at analysis above).
+- **Spawn prompt specificity:**
+  - Line 23: "investigate whether a 'contact-us' page schema has ever been set up" + specific search targets (src/, openspec/, docs/). Focused.
+  - Line 24: "Fetch the page at https://www.studentdisciplinedefense.com/contact-us and check whether it contains any JSON-LD schema markup." Focused.
+  - Line 41: "Use wp-cli SSH access to check what office data is currently in the SDD site's ACF fields." Focused.
+  - Line 60: "Read the following files... return their full contents: 1. /home/jared/dev/llf-schema/src/Plugin.php, 2. ..." Focused list.
+  - Lines 83, 99: Same pattern—specific file list, no extraneous narrative.
+- **Negative:** No evidence of context dumps (e.g., "here is the entire schema design doc, here is the implementation history, here is the requirements spec" bundled without filtering). Each spawn asks for specific files or specific checks.
+
+### Q6: Is the orchestrator even following the ORCHESTRATION.md instructions?
+
+**Verdict:** FAIL — Two violations: model parameter not passed; model tier incorrect for 5 of 8 spawns.
+
+**Evidence:**
+
+**Violation 1: Policy says "Every Agent spawn passes model explicitly."**
+- Spawns 1–6 (lines 23, 24, 41, 60, 83, 99): No `model` field in Agent input.
+  ```python
+  # Spawn 1 (line 23) input keys: ['description', 'prompt', 'run_in_background']
+  # Spawn 6 (line 99) input keys: ['description', 'prompt', 'run_in_background']
+  ```
+- Spawns 7–8 (lines 115, 141): `model` field present.
+  ```python
+  # Spawn 7 (line 115) input: {..., 'model': 'sonnet'}
+  # Spawn 8 (line 141) input: {..., 'model': 'haiku'}
+  ```
+- **Violation confirmed:** 6 of 8 spawns omit the required model parameter.
+
+**Violation 2: Model tier misallocated to spawns 2–6.**
+- Policy: "Default `haiku` for mechanical file-edit/shell work; `sonnet` for anything requiring judgment."
+- Actual resolution (from fact-sheet) for spawns 2–6: all sonnet.
+  - Spawn 2 (WebFetch): mechanical → should be haiku, got sonnet.
+  - Spawn 3 (wp-cli SSH): mechanical shell → should be haiku, got sonnet.
+  - Spawn 4 (read Plugin.php): mechanical read → should be haiku, got sonnet.
+  - Spawn 5 (read context files): mechanical read → should be haiku, got sonnet.
+  - Spawn 6 (read remaining files): mechanical read → should be haiku, got sonnet.
+- **Cost implication:** ~5 unnecessary sonnet invocations when haiku would have been sufficient and correct. This is not a catastrophic misallocation (the tasks still completed), but it violates the policy's explicit cost-optimization intent.
+
+### Q7: Is the orchestrator requesting/receiving back only the context it needs, rather than a full context dump from the subagent?
+
+**Verdict:** PASS — Focused requests, async task results, no repeated queries observed.
+
+**Evidence:**
+- **Spawn prompts are scoped requests, not open-ended:**
+  - Line 24: "check whether it contains any JSON-LD schema markup" (not "summarize the page" or "return all content").
+  - Line 41: "Run these commands: 1. `ssh sdd '...'`; 2. ...`" (specific operations, not "explore the SDD server").
+  - Line 60: "Read the following files... return their full contents verbatim" (specific files, not "all PHP files" or "all src/").
+- **Async result handling:**
+  - Lines 46, 48, etc.: Task notifications arrive as async user-context messages (subagent results baked in, not separately queried).
+  - No evidence of follow-up tool calls by orchestrator to re-fetch or clarify subagent responses (fact-sheet: 0 orchestrator calls).
+- **No repeated context-dump requests:**
+  - Each spawn requests specific artifacts; no "give me everything you learned" or "re-read that file and explain it differently" follow-ups.
+
+---
+
+## Root-Cause Analysis: Why Model Parameters Were Omitted
+
+**Hypothesis:** The orchestrator code path for spawning agents may have defaulted to omitting the model parameter on the first 6 spawns, only explicitly passing it on the final 2 (implementation and verification). Possible causes:
+1. Different code paths or templates for information-gathering vs. implementation spawns.
+2. A changed or partially-applied fix (spawns 7–8 pass model explicitly; 1–6 do not).
+3. An oversight or a once-per-session pattern in the orchestrator's logic.
+
+**Impact:** Subagents resolved to sonnet (the orchestrator's resolved model at the time), masking the violation until audit. Policy compliance requires explicit parameter, not just correct resolution.
+
+---
+
+## Recommendations
+
+1. **Model parameter requirement (blocking):** All Agent spawns must include an explicit `model` parameter. Spawns 1–6 violated this; code path likely needs a wrapper or template update to ensure all spawns carry the parameter.
+
+2. **Model tier audit:** Review spawn 2–6 model assignments. Haiku is sufficient and correct per policy for:
+   - WebFetch queries with a yes/no or extraction task (spawn 2)
+   - wp-cli shell commands (spawn 3)
+   - File-reading tasks (spawns 4–6)
+
+3. **Test case:** Add a session-audit checklist or automated linter to flag Agent spawns without an explicit model parameter before they run.
+
+4. **Parallelization review (optional):** The 7–101 minute gaps between spawns 3–4 and 1–3 are not immediately explained by the transcripts shown. If they reflect waiting for sequential task results, that's fine; if they reflect pacing or manual delay, parallelizing spawns 4–6 (the three file-read tasks) may recover wall-clock time without changing correctness.
+
+---
+
+## Appendix: Data Sources
+
+- **Fact-sheet:** `/tmp/claude-1000/-home-jared-dev-cc-os/3c15edee-8ac7-4ffe-b6ee-ee6a6ebd670d/scratchpad/factsheets/S5-aadeb66c.md`
+- **Transcript:** `/home/jared/.claude/projects/-home-jared-dev-llf-schema/aadeb66c-0d64-4d41-b407-83e0209428b4.jsonl` (161 lines)
+- **Policy:** ORCHESTRATION.md (global default; per-project CLAUDE.md equivalent at 2026-06-29)
+- **Audit method:** Spot-reads of specific lines (23, 24, 41, 60, 83, 99, 115, 141) for spawn details; timeline reconstruction via timestamps; model-resolution data from fact-sheet table.

docs/orchestration-audit/auditor-reports/S6-report.md

@@ -0,0 +1,170 @@
+# Session Orchestration Audit: S6-3dc26da3
+
+**Session:** philly-search-engine-marketing, 2026-06-29 15:08–16:58 UTC  
+**Transcript:** 120 jsonl lines, 3.3MB (largely user-submitted images)  
+**Scope:** 21 assistant turns, 13 human prompts, 1 Agent spawn  
+**Policy Under Audit:** Project-local CLAUDE.md (lines 72–102), "All work follows an orchestrator-subagent pattern"
+
+---
+
+## Question 1: Are subagents getting called when they should be?
+
+**Verdict:** PASS
+
+**Evidence:**
+- Line 98: Agent spawned for multi-file update task (TODO.md + logs/2026-06-29.md)
+- Lines 20–85: Conversational debugging of Zapier error handling (no delegation, correct — this is advice/troubleshooting, not "work")
+- Line 89 (user): "Update todos and documentation based on this conversation" — triggers explicit work task
+- Line 97 (orchestrator): "I'll handle the documentation updates in parallel" — signals intent to delegate
+- Fact-sheet confirms 1 spawn, 0 orchestrator tool calls (Read/Write/Bash/Grep)
+
+The project's local CLAUDE.md (line 72) is strict: *"All work follows an orchestrator-subagent pattern. No exceptions."* The orchestrator correctly identified the file-update task as delegable work and spawned an Agent. The conversational turns before that are appropriately handled directly (no restriction against speaking directly; the restriction is on direct tool calls).
+
+---
+
+## Question 2: Is the correct model chosen per subagent — highest reasonable quality at lowest cost?
+
+**Verdict:** FAIL
+
+**Evidence:**
+- Line 98 Agent spawn: `input` contains only `description` and `prompt` — NO `model` field
+- Fact-sheet line: "model param: (ABSENT)" → "resolved model: claude-sonnet-4-6"
+- Task: Read 2 files (TODO.md, logs/2026-06-29.md), append structured items, create log if missing, return summary
+- Project CLAUDE.md model routing (lines 86–91):
+  ```
+  | Haiku  | File reads, simple edits, formatting, search |
+  | Sonnet | Feature implementation, refactoring, tests |
+  | Opus   | Architectural decisions, complex debugging |
+  ```
+
+This task is **file reads + simple edits + formatting** — a textbook Haiku task. Sonnet is 4x more expensive and unnecessary for mechanical file operations. 
+
+**Violation:** Project CLAUDE.md line 86 states "Every Agent spawn passes model explicitly" (implied reference to the table above). The spawn did not include `model: "haiku"` despite the task profile being unambiguously haiku-level.
+
+**Cost impact:** Agent consumed 28.8k subagent tokens (reported in line 105 task-notification `<subagent_tokens>`). At typical rates, a Haiku agent would have consumed ~7–10k tokens for the same work. Estimated overspend: ~3–4x cost.
+
+---
+
+## Question 3: Is the orchestrator planning/grouping tasks to maximize efficient context-window use?
+
+**Verdict:** MIXED (structure sound, execution not independently verifiable)
+
+**Evidence:**
+- Single Agent spawn with 2039-char prompt (well-structured, ~2KB)
+- Prompt includes explicit instructions:
+  - Files to read (with fallback logic: "create if missing")
+  - Two specific TODO items with full context (Gary's priority, Zapier blocker details)
+  - Four session-log entry points covering the decision, implementation, blocker, and next steps
+  - Instruction to "Return a brief summary"
+- Agent returned after 55.2 seconds with 6 tool_uses (likely: 2 Reads, 2 Writes, 1–2 git ops for potential commit/log)
+
+The prompt is well-factored — it frontloads all context (no need for back-and-forth). The task scope is narrow enough that one Agent call suffices. The agent's token count (28.8k) is reasonable for the work scope.
+
+**However:** Cannot verify from the orchestrator's window whether the agent *internally* batched reads/writes efficiently or made sequential redundant operations. The agent's own transcript is off-limits per line 99 guidance ("do NOT Read or tail this file via the shell tool").
+
+---
+
+## Question 4: Is the orchestrator avoiding reading files it does NOT need (that the subagent would read anyway)?
+
+**Verdict:** PASS
+
+**Evidence:**
+- Fact-sheet: "calls: 0, bytes read: 0" (pre-spawn and post-spawn)
+- Grep of transcript: No `"name":"Read"` tool uses anywhere
+- Orchestrator did not self-read TODO.md or logs/2026-06-29.md before delegating
+
+Project CLAUDE.md line 81 explicitly forbids this: *"Does NOT read files to prepare delegation specs — write specs from the user's request."*
+
+The orchestrator inlined all context (Gary's email content, Zapier error details, decision rationale) from the conversation into the Agent prompt, requiring zero file reads. Correct pattern.
+
+---
+
+## Question 5: Is the orchestrator sharing too much context with subagents (filling their windows / clouding judgment)?
+
+**Verdict:** PASS
+
+**Evidence:**
+- Agent prompt: 2039 characters of task instructions
+- Content: structured, specific, actionable
+  - Clear "add these two items to TODO.md" (with bulleted details)
+  - Clear "add this session entry to logs/2026-06-29.md" (with bulleted details)
+  - Clear instruction to create log file if missing and follow the pattern of other logs/
+- No extraneous conversation transcript dumped into the prompt
+- No "here's everything the user said, figure it out" vagueness
+
+The prompt is dense but focused. It reads as a well-written specification, not a context-dump. The agent consumed 28.8k tokens total (subagent tokens per line 105); that's a reasonable allocation for the complexity of the task.
+
+---
+
+## Question 6: Is the orchestrator even following the ORCHESTRATION.md instructions?
+
+**Verdict:** FAIL (partial compliance)
+
+**Evidence:**
+
+**What was done correctly:**
+- Line 77–80 (orchestrator responsibilities):
+  - ✓ Interpreted user request as delegable work
+  - ✓ Delegated via Agent tool (not direct Read/Write/Bash)
+  - ✓ Did not call Read/Write/Bash/Grep directly
+  - ✓ Did not read files to prepare specs
+- Line 84 (subagent returns):
+  - ✓ Agent returned summary (task-notification at line 105: *"Both files are updated. Here's what was done:"*)
+  - ✓ Included artifact paths (README.md implied; full structure in summary)
+
+**What was done incorrectly:**
+- Line 86 (model routing): States "Every Agent spawn passes model explicitly"
+  - ✗ Spawn at line 98 has NO `model` field in input
+  - ✗ Cascaded default to sonnet instead of haiku
+  - ✗ Violates explicit policy: "Haiku for file reads, simple edits, formatting"
+
+This is a clear violation of a single, high-stakes instruction in the project's local policy.
+
+---
+
+## Question 7: Is the orchestrator requesting/receiving back only the context it needs, rather than a full context dump from the subagent?
+
+**Verdict:** PASS
+
+**Evidence:**
+- Task-notification at line 105: Contains task metadata + concise summary of actions taken
+  - Status: completed
+  - Summary: "Both files are updated. Here's what was done:"
+  - Details: Two bulleted items for TODO.md, four bulleted items for logs/2026-06-29.md entry
+  - Usage stats: subagent_tokens, tool_uses count, duration_ms
+- Orchestrator's response (line 109): *"Documentation updated. TODO.md has both items at the top of the active section, and today's session log is created. Ready for the SDD Contact Us schema when you are."*
+  - Acknowledges completion
+  - Confirms both files were touched
+  - Moves forward without demanding a full file dump or line-by-line verification
+- No off-limits file read (per line 99 guidance)
+
+The subagent's summary was sufficient. The orchestrator neither over-requested nor over-received context.
+
+---
+
+## Summary of Misses
+
+1. **Model routing violation (Question 2):** The Agent spawn at line 98 omitted the required `model: "haiku"` parameter. Project policy is explicit and non-negotiable: *"Every Agent spawn passes model explicitly."* The task (file reads + edits) is a textbook Haiku case. **Estimated cost impact:** 3–4x overspend (sonnet vs. haiku pricing).
+
+2. **All other questions:** Orchestrator's delegation pattern, context handling, and response flow are compliant with the project's CLAUDE.md policy and the global ORCHESTRATION.md rubric.
+
+---
+
+## Candidate Eval Trigger / Criterion
+
+When a project has a strict agent-model routing table (e.g., "Haiku for file reads/edits/formatting"):
+- **Criterion:** Every Agent spawn must include an explicit `model: "haiku"` (or the table's default) parameter. Default cascade (allowing the system to choose sonnet) is a violation, not a fallback.
+- **Orchestrator should:** Check the task type against the project's model routing table *before* writing the Agent call. Pass the required model parameter unconditionally.
+- **Signal for rejection:** A project CLAUDE.md with a model routing table + any Agent spawn that lacks an explicit model field matching that table.
+
+---
+
+## Appendix: Transcript Anomaly
+
+The 3.3MB transcript with only 120 lines and 21 assistant turns is explained by:
+- Lines 10, 51, 63: User messages containing embedded images (750KB, 1.4MB, 787KB respectively)
+- Image data is stored inline in the JSON (base64 or direct encoding)
+- No orchestrator tool calls, so no Read/Write/Bash results to inflate the transcript further
+- The bulk is user input (images), not orchestrator work or subagent verbosity
+
+This is not an orchestration efficiency issue; it's a client-interaction pattern (user sharing screenshots for Zapier troubleshooting).

docs/orchestration-audit/auditor-reports/S7-report.md

@@ -0,0 +1,247 @@
+# Audit Report: Session 3771b288 (~/servers ops work)
+
+**Date:** 2026-07-06  
+**Session:** 3771b288-eb9d-4398-9e75-e1ff0ba6af03  
+**Project:** /home/jared/servers (ops/infrastructure)  
+**Duration:** 16:05:40 to 17:14:47 UTC (69 min)  
+**Turns:** 144 assistant turns, 11 human prompts  
+**Policy Audited:** ORCHESTRATION.md (global rollout 2026-07-04)
+
+---
+
+## SEVEN-QUESTION RUBRIC VERDICTS
+
+### 1. Are subagents getting called when they should be?
+
+**Verdict:** MIXED
+
+**Evidence:**
+- Lines 241, 243, 245: Three Explore agents spawned for read-only discovery tasks:
+  - Line 241: Explore ovh-prod service setup conventions (2282 chars prompt)
+  - Line 243: Explore desktop backup bug specifics (2725 chars prompt)
+  - Line 245: Explore proxmox VM 101 and umbrella structure (2126 chars prompt)
+- Explores are appropriate for their scope (file discovery, grep, schema extraction)
+- Pattern: Orchestrator conducted 32 pre-spawn Bash commands + 4 Reads (74KB total) **before** delegating to agents
+
+**Concern:**
+Lines 1-240 show orchestrator performing extensive diagnostic work itself:
+- 32 Bash commands: grep across servers for n8n, systemctl status checks, journalctl log inspection, SSH diagnostics to ovh-prod
+- 4 Reads: project memory, n8n config, spec files, services inventory
+- This exploratory work (grep, find, systemctl, journalctl, ssh) is exactly what Explore agents are designed to do
+
+The orchestrator briefed three agents with comprehensive prompts (2.2–2.7 KB each), suggesting it already understood the context. This is valid if the pre-spawn diagnostics were necessary to formulate good agent prompts, but it shifts exploratory token cost into the orchestrator's session window rather than the agent's. The policy allows "A short orienting Read before delegating is fine" — but 32 Bash + 4 Read is beyond "orienting."
+
+**Missed Delegation Trigger:**  
+"When orchestrator must diagnose a multi-faceted situation across multiple machines/services, spawn Explore agents early with open-ended discovery prompts rather than conducting detailed diagnostics in-session first, unless the diagnostics are sub-1-minute or ≤2 tool calls."
+
+---
+
+### 2. Is the correct model chosen per subagent — highest reasonable quality at lowest cost?
+
+**Verdict:** FAIL
+
+**Evidence:**
+- Line 241: `"model": null` (ABSENT), resolved to `claude-haiku-4-5-20251001`
+- Line 243: `"model": null` (ABSENT), resolved to `claude-haiku-4-5-20251001`
+- Line 245: `"model": null` (ABSENT), resolved to `claude-haiku-4-5-20251001`
+
+**Policy Violation:**
+ORCHESTRATION.md states: "Every `Agent` spawn passes `model` explicitly. Default `haiku` for mechanical file-edit/shell work; `sonnet` for anything requiring judgment; `opus` only for genuinely hard reasoning."
+
+All three spawns violate this by omitting the `model` parameter entirely. The model resolved to haiku via the Explore agent's own default, not via explicit orchestrator choice.
+
+**Assessment of Haiku as the Actual Choice:**
+- All three tasks are read-only discovery (appropriate for haiku)
+- Tasks: (1) schema extraction from ovh-prod docs, (2) file location and diagnostics from desktop, (3) directory structure and grep across umbrella project
+- None require judgment-layer reasoning (schema extraction is mechanical; file discovery is mechanical; structure understanding is verification, not judgment)
+- **Haiku is the correct choice**, but the policy requires the orchestrator to **state it explicitly**, not rely on agent defaults
+
+**Missed Enforcement Trigger:**  
+"Every Agent() call must include `model: "haiku"` (or sonnet/opus) as an explicit parameter, even when the default matches the intended choice."
+
+---
+
+### 3. Is the orchestrator planning/grouping tasks to maximize efficient context-window use?
+
+**Verdict:** PASS
+
+**Evidence:**
+- Lines 294+: Single `ExitPlanMode()` call with detailed plan covering three distinct domains:
+  - Part 1: Fix desktop bug 1 (notify_backup.rb bare command → full path)
+  - Part 2: Fix desktop bug 2 (systemd-inhibit fallback on polkit auth failure)
+  - Part 3: Stand up proxmox-ubuntu project scaffold
+  - Part 4: Record deferred items (ovh-prod cron instrumentation, n8n on proxmox)
+- Plan synthesis: Root cause analysis grouped related bugs (two desktop bugs from single diagnosis session) and separated them from unrelated work (proxmox project, ovh-prod deferral)
+- Three agents spawned in parallel (line 241–245 are sequential calls but agents run async) to explore three independent domains, avoiding redundant reads
+- After agents return, orchestrator executes plan in correct sequence (fixes before scaffolding, deferred items recorded separately)
+
+**Task Grouping Logic:**
+The plan efficiently batches work by separating:
+- Immediate fixes (desktop bugs) — quick, specific file edits
+- Project scaffolding (proxmox-ubuntu) — multi-file writes, template-driven
+- Deferred tracking (incidents log, roadmap) — documentation updates
+This avoids interleaving unrelated work and keeps context coherent.
+
+---
+
+### 4. Is the orchestrator avoiding reading files it does NOT need (that the subagent would read anyway)?
+
+**Verdict:** FAIL
+
+**Evidence:**
+- Pre-spawn (lines 1–240): Orchestrator reads `/home/jared/.n8n_instances.yml`, `/home/jared/servers/desktop/docs/superpowers/specs/2026-05-07-n8n-backup-monitor-design.md`, grepped `/home/jared/servers/ovh-prod/docs/services-inventory.md`
+- Line 241 Explore prompt explicitly lists "Read `/home/jared/servers/ovh-prod/docs/services-inventory.md`" as one of the items for the agent to read
+- Line 243 Explore prompt asks the agent to "Read `/home/jared/.local/bin/backup.sh` in full"
+
+**Overlap Found:**
+- Orchestrator read parts of `services-inventory.md` (grep for n8n entries) before delegating Explore task that would read the entire file
+- Orchestrator examined `backup.sh` in-session (Bash cat command) before delegating agent to read it fully
+
+This is redundant reading — the orchestrator read files that the Explore agents would subsequently read. The files were read to formulate the agent prompts, but the prompt content shows the agents were asked to read the same files independently.
+
+**Missed Efficiency Trigger:**  
+"If an orchestrator prompt to an Explore agent lists 'Read file X' as a task, don't Read file X in-session first — let the agent do it and summarize findings back."
+
+---
+
+### 5. Is the orchestrator sharing too much context with subagents (filling their windows / clouding judgment)?
+
+**Verdict:** PASS
+
+**Evidence:**
+- Line 241 prompt: 2282 chars (~570 tokens) — substantial but focused
+- Line 243 prompt: 2725 chars (~680 tokens) — includes detailed diagnosis context (symptom, root cause, diagnosis result) from pre-spawn work
+- Line 245 prompt: 2126 chars (~530 tokens) — structured multi-part discovery task
+
+Each prompt is well-bounded and task-oriented:
+- Prompt 1: "Read these 12 specific files in ovh-prod and summarize the step-by-step pattern"
+- Prompt 2: "Given the diagnosis in incidents.md (which I've already logged), find the exact lines in backup.sh and systemd files that implement the two bugs"
+- Prompt 3: "Read umbrella CLAUDE.md, proxmox-hermes template, grep for ubuntu-server references, check ~/.ssh/config"
+
+None of these prompts dump the orchestrator's entire diagnostic session (which would have been 10+ KB of systemctl output, logs, ssh diagnostics). They're filtered to task-relevant context.
+
+**Context Quality:**
+Prompts are specific enough to give agents clear direction without forcing them through the orchestrator's entire reasoning path.
+
+---
+
+### 6. Is the orchestrator even following the ORCHESTRATION.md instructions?
+
+**Verdict:** FAIL (Model param explicitly required)
+
+**Evidence:**
+- Policy: "Every `Agent` spawn passes `model` explicitly."
+- Observation: All 3 Agent spawns (lines 241, 243, 245) have `"model": null` (parameter omitted entirely)
+- Fact-sheet confirms: "model param | (ABSENT)" for all three rows
+- Fallback resolution: Explore agent's own default (haiku) was used instead of orchestrator choice
+
+**Additional Compliance Check:**
+- Policy says haiku default is for "mechanical file-edit/shell work" — Explores are read-only, mechanical, haiku is correct
+- Policy does NOT say orchestrators can omit the model param if they think the default is right; it explicitly requires the param be passed
+
+**Secondary Issue (Model Param, Not Violation But Efficiency Concern):**
+- The 32 pre-spawn Bash commands + diagnostics should have been minimal; orchestrator appears to have over-briefed itself before delegating
+
+---
+
+### 7. Is the orchestrator requesting/receiving back only the context it needs, rather than a full context dump from the subagent?
+
+**Verdict:** PASS
+
+**Evidence:**
+- Line 241 result: 7537 chars (~1884 tokens) — summary of ovh-prod structure, step-by-step pattern, file inventories
+- Line 243 result: 3562 chars (~890 tokens) — exact line numbers, code snippets for bugs, analysis of failures
+- Line 245 result: 4776 chars (~1194 tokens) — umbrella context, proxmox-hermes template, umbrella file inventory, VM 101 references
+
+Results are task-focused summaries, not dumps:
+- Agent 1 returned pattern + inventory, not full file contents of 12 files
+- Agent 2 returned bug locations + explanations, not raw logs/journalctl output
+- Agent 3 returned references + template structure, not entire directory trees
+
+After agents return, orchestrator synthesizes results into a plan (line 294) without re-reading files, indicating the agent summaries were sufficient.
+
+---
+
+## SUMMARY TABLE
+
+| # | Verdict | Issue | Severity |
+|---|---------|-------|----------|
+| 1 | MIXED | Pre-spawn diagnostics could have been delegated to Explore; 32 Bash + 4 Read in-session | Medium |
+| 2 | **FAIL** | **All 3 Agent spawns omit `model` param; policy requires explicit param** | High |
+| 3 | PASS | Plan synthesis and task grouping are efficient | — |
+| 4 | FAIL | Orchestrator read files before delegating Explore agents to read the same files | Medium |
+| 5 | PASS | Prompt context is focused and bounded, not excessive | — |
+| 6 | **FAIL** | **Model param violation = ORCHESTRATION.md non-compliance** | High |
+| 7 | PASS | Subagent results are summaries, not full dumps | — |
+
+---
+
+## MISSED DELEGATION CANDIDATES
+
+### Write x5 (lines 345–353)
+
+**Files written:**
+1. `/home/jared/servers/proxmox-ubuntu/CLAUDE.md`
+2. `/home/jared/servers/proxmox-ubuntu/README.md`
+3. `/home/jared/servers/proxmox-ubuntu/docs/services-inventory.md`
+4. `/home/jared/servers/proxmox-ubuntu/docs/roadmap.md`
+5. `/home/jared/servers/ovh-prod/docs/incidents.md`
+
+**Content pattern:** Multi-file project scaffold + incident logging, drawing on Explore agent summaries
+
+**Judgment involved:** Yes — each file cross-references ovh-prod patterns, templates from proxmox-hermes, and specific decisions about what to defer (n8n, ovh-prod cron instrumentation)
+
+**Delegation assessment:** Technically a "spans many files" candidate per policy, but:
+- Files are contextually dependent (CLAUDE.md references README references services-inventory)
+- This is plan **execution**, not exploration — orchestrator is synthesizing Explore results into a single coherent scaffold
+- Sequential scaffolding logic (CLAUDE.md must define context before README can reference it) makes parallelization unclear
+
+**Verdict:** Not a violation. Write x5 is a valid in-session execution phase after exploration, especially given the multi-file interdependencies and the need to synthesize Explore results. If the Write sequence were purely mechanical templating with zero judgment, it would be a delegation candidate; here, it involves template selection, cross-project pattern alignment, and deferral routing.
+
+---
+
+## KEY FINDINGS
+
+1. **Model param violation is the primary issue.** All three Agent spawns violate the explicit ORCHESTRATION.md requirement. The policy is clear: every spawn must include the `model` parameter. Even if haiku is the right choice, it must be stated. Recommend a lint rule: `Agent(...) without model: param → error before tool execution`.
+
+2. **Pre-spawn diagnostics could be better scoped.** The orchestrator spent 74 KB and 36 tool calls on diagnostics before delegating exploration. A tighter pattern would be: read the umbrella context + problem statement (≤1 KB), then spawn Explore agents with open-ended discovery prompts and let them do the diagnostic work. The 32-Bash diagnostic session (systemctl, journalctl, ssh, grep) is exactly what Explore is for.
+
+3. **File read overlap.** The orchestrator read `services-inventory.md` and `backup.sh` in-session, then asked Explore agents to read the same files. Tighter scoping would avoid this redundancy.
+
+4. **Plan synthesis is strong.** The orchestrator correctly bundled three conceptually separate tasks (desktop bugs, proxmox project, deferred tracking) into a single coherent plan, with good separation between immediate fixes and deferred work. Plan execution (the Writes) was appropriate in-session.
+
+5. **Explore agents received well-bounded prompts.** Despite the pre-spawn diagnostic work, the prompts given to agents were focused and didn't dump the full session history on them.
+
+---
+
+## RECOMMENDATIONS
+
+**Immediate (High Priority):**
+1. Enforce `model: "haiku"` (or explicit model choice) on every `Agent()` call. This is a hard requirement per ORCHESTRATION.md and was violated on all 3 spawns in this session.
+
+**Near-term (Medium Priority):**
+2. Reduce pre-delegation diagnostics. When spawning Explore agents, provide umbrella context + clear task description; don't pre-diagnose. Let Explore agents handle discovery.
+3. Add a "no redundant reads" check: if you're about to delegate a task that includes "Read file X," don't Read file X in the orchestrator session first.
+
+**Process (Optional):**
+4. Consider a linting/validation layer that catches model-param omissions before agent spawn.
+
+---
+
+## TIMELINE
+
+- **16:05–16:47** (42 min): Orchestrator diagnostic work (32 Bash, 4 Read) + plan synthesis
+- **16:47:03** (line 241): Explore spawn 1 (ovh-prod conventions) — async, model=null
+- **16:47:14** (line 243): Explore spawn 2 (desktop backup bugs) — async, model=null
+- **16:47:24** (line 245): Explore spawn 3 (proxmox VM 101) — async, model=null
+- **16:47:27**: Orchestrator synthesis turn (awaiting agent results)
+- **After agent completion** (not timestamped in fact-sheet but ~line 294+): Plan exited, execution begins (Write x5)
+- **17:14:47**: Session ends
+
+---
+
+## CONCLUSION
+
+The session demonstrates **strong plan design and synthesis** but **violates the core ORCHESTRATION.md policy** on explicit model parameterization. The orchestrator should have passed `model: "haiku"` to all three Explore spawns. Additionally, 74 KB of pre-delegation diagnostic work could have been better scoped, with more delegated to the Explore agents. The Write x5 execution was appropriate given the multi-file scaffold context.
+
+**Overall Verdict:** Policy non-compliant (model param violation), efficiency improvable (pre-spawn diagnostics), but execution and planning logic sound.

docs/orchestration-audit/auditor-reports/S8-report.md

@@ -0,0 +1,127 @@
+# Orchestration Audit Report: Session 3fc7bb8c (S8)
+
+**Transcript:** `/home/jared/.claude/projects/-home-jared-servers/3fc7bb8c-54c1-4147-ac3d-dc7db3fca924.jsonl`  
+**Session Date:** 2026-07-04, 17:20–18:04 UTC  
+**Duration:** ~44 minutes  
+**Turns:** 90 assistant turns, 5 human prompts  
+**Tools:** 44 total (Bash:27, Edit:6, Write:4, Read:3, ToolSearch:2, ExitPlanMode:1, Skill:1)  
+**Agent Spawns:** 0
+
+---
+
+## Question 1: Are subagents getting called when they should be?
+
+**Verdict: PASS** — No subagent spawns, but none were required.
+
+**Evidence:**
+- Factsheet reports zero Agent spawns across 90 turns.
+- Work was inherently sequential: diagnostic steps on ovh-prod (commands 2–6 in Bash sequence) → n8n investigation (commands 7–15) → documentation (commands 16–21) → desktop verification (commands 22–27). Each phase depends on conclusions from the previous.
+- Bash command sequence analysis (turns 6–87): SSH diagnostics were tightly coupled (command 2 reads the script, command 3 checks logs for clues based on script content, command 5 applies a fix targeting symptoms found in command 3).
+- No wide grep-and-synthesize or large-context work that would trigger the "spans many files" or "needs large/isolated context" delegation criteria.
+
+**Key Dependency Chain:**
+- Turn 6–11: SSH to ovh-prod, read backup.sh, check logs, apply fix, test it.
+- Turn 19–38: Investigate n8n (grep for references, check docker, query database) — depends on conclusion that backup.sh may involve n8n.
+- Turn 41–48: Update incidents.md with findings — depends on diagnosis being complete.
+- Turn 51–87: Set up verification on desktop — depends on understanding the full scope from phases 1–3.
+
+No parallel discovery threads; the work was single-stream investigation.
+
+---
+
+## Question 2: Is the correct model chosen per subagent?
+
+**Verdict: N/A** — No subagents spawned.
+
+**Evidence:** Not applicable; the session spawned zero agents.
+
+---
+
+## Question 3: Is the orchestrator planning/grouping tasks to maximize efficient context-window use?
+
+**Verdict: MIXED** — Grouping was implicit rather than explicit; minor opportunity missed.
+
+**Evidence:**
+- Read calls (3 total): incidents.md files (2), memory.md (1). Low volume overall.
+- Bash calls (27 total) were sequential, with no explicit batching of independent reads.
+- **Missed Opportunity:** Commands 7–8 (local grep for n8n references in /servers/ directory) could have run in parallel with commands 2–6 (SSH ovh-prod diagnostics) — they are fully independent (no shared state, no cross-system dependencies). Commands 7–8 appear at lines 65–73; commands 2–6 completed earlier.
+- However, the gain would be minimal (~1–2 minutes wall-clock savings); the overall session is already efficient because most work is hands-on diagnostic/fix (no large context accumulation).
+- No evidence of explicit grouping strategy in the transcript (no mention of "I'll batch these", "run in parallel", etc.). The orchestrator appears to have run tasks as they naturally emerged from the diagnosis, which is appropriate for interactive troubleshooting.
+
+**Specific Lines:**
+- Lines 65–73: grep commands 7–8 (local filesystem search for n8n).
+- Lines 25–35: grep commands 2–6 completed earlier (SSH ovh-prod diagnostics).
+- They were executed sequentially, but could have overlapped.
+
+---
+
+## Question 4: Is the orchestrator avoiding reading files it does NOT need?
+
+**Verdict: PASS** — Minimal unnecessary reads.
+
+**Evidence:**
+- Only 3 Read calls across 44 tools (7% of work).
+- Read 1 (line 14): `/home/jared/servers/ovh-prod/docs/incidents.md` — directly relevant; planning requires understanding the incident.
+- Read 2 (line 121): `/home/jared/servers/desktop/docs/incidents.md` — directly relevant; needed to update it.
+- Read 3 (line 88): `/home/jared/.claude/projects/.../memory/MEMORY.md` — project memory, part of standard SessionStart context.
+- No evidence of reading tangential files or over-loading context. The orchestrator stayed focused on the diagnostic targets.
+- **Bash vs Read Balance:** Most information was gathered via SSH diagnostics and local grepping (27 Bash calls) rather than reading large files. This is appropriate for this type of work.
+
+---
+
+## Question 5: Is the orchestrator sharing too much context with subagents?
+
+**Verdict: N/A** — No subagents spawned.
+
+**Evidence:** Not applicable.
+
+---
+
+## Question 6: Is the orchestrator following the ORCHESTRATION.md instructions?
+
+**Verdict: PASS** — Yes, though implicitly rather than explicitly.
+
+**Evidence:**
+- SessionStart injection (line 4): ORCHESTRATION.md was successfully injected at startup, confirming the policy was loaded.
+- Policy snippet quoted: "Do single-file, ≤2-tool-call ops directly. Don't delegate them. Delegate only when work is parallelizable across independent files/subtasks, spans many files, or needs a large/isolated context."
+- **Observance Check:**
+  - "Single-file, ≤2-tool-call ops done directly?" YES. Turns 44–49 (Edit incidents.md): 3 edits completed inline without delegation.
+  - "Delegation only for parallelizable/multi-file/large-context?" NOT TRIGGERED. No work met those criteria.
+  - "Explicit model parameter for Agent spawns?" Not applicable (zero spawns).
+- **Missing Signal:** No explicit reasoning about the decision. The transcript contains no text mentioning "this is sequential, so I won't delegate" or "these are independent so I could parallelize". The orchestrator appears to have followed the policy implicitly through task structure, not by conscious articulation.
+
+---
+
+## Question 7: Is the orchestrator requesting/receiving back only context it needs?
+
+**Verdict: PASS** — No subagent results; not applicable.
+
+**Evidence:** Zero subagents, so no results to evaluate. The orchestrator's own Bash/Read/Edit operations were tightly scoped (no over-reading, minimal context bloat).
+
+---
+
+## Summary of Findings
+
+### Key Strengths
+1. **Correct No-Delegation Decision:** The work was genuinely sequential; spawning even a single subagent would have introduced latency without parallelization benefit.
+2. **Minimal Context Loading:** Only 3 reads in 90 turns; work was hands-on diagnostic, not deep-context synthesis.
+3. **Tight Coupling Observed:** Each diagnostic step depended on the previous result, validating sequential execution.
+4. **Policy Compliance:** ORCHESTRATION.md was loaded and followed implicitly.
+
+### Minor Opportunities
+1. **Missed Parallelization:** Commands 7–8 (local n8n grep) vs. commands 2–6 (SSH ovh-prod diagnostics) could have run in parallel. Gain: ~1–2 minutes wall-clock time. Not worth subagent overhead, but explicit batching acknowledgment would improve transparency.
+2. **Implicit vs. Explicit Reasoning:** The orchestrator did not articulate why delegation was not chosen. For audit trails and future refinement, explicit reasoning ("this work is inherently sequential because...") would be valuable.
+
+### No Issues Found
+- No over-delegation of trivial work.
+- No under-use of subagents on parallelizable multi-file tasks.
+- No wasteful context passing.
+- No violation of the ≤2-tool-call rule for direct work.
+
+---
+
+## Verdict
+
+**AUDIT PASS** — The orchestrator correctly chose not to spawn agents for this session. The work was inherently sequential investigative debugging, with minimal parallelization opportunity and high dependency coupling. The policy was followed appropriately.
+
+**Recommendation for Future Sessions:** When multiple independent investigative threads emerge (even slightly), consider explicit acknowledgment of why they remain sequential. This would strengthen audit clarity without slowing work.

docs/orchestration-audit/auditor-reports/S9-report.md

@@ -0,0 +1,232 @@
+# Orchestration Audit: Session 2525fa2d-1b15-4495-99bf-c093716fab55
+
+**Project:** systems-admin  
+**Date:** 2026-06-30  
+**Main loop model:** claude-opus-4-8 (all 117 turns)  
+**Assistant turns / human prompts:** 117 / 6  
+
+## Summary
+
+The session executed a self-audit of Claude Code orchestration/delegation practices across the user's project portfolio. It spawned 3 background agents (2 general-purpose/opus, 1 claude-code-guide/haiku) to parallelize read-only research and documentation verification. The orchestrator correctly avoided reading files before delegating (except for initial orientation), structured prompts precisely without context bloat, and produced a PRD + handoff summary as deliverables.
+
+**Critical policy violation:** None of the 3 Agent spawns passed a `model` parameter explicitly, violating the stated ORCHESTRATION.md rule ("Every Agent spawn passes `model` explicitly").
+
+---
+
+## Findings by Question
+
+### 1. Are subagents getting called when they should be?
+
+**Verdict:** PASS — Delegations were appropriate and well-scoped.
+
+**Evidence:**
+- Line 95: Agent spawn #1 for "Inventory loaded context surface" — reads multiple config/skill directories and sums character counts. Parallelizable across multiple file paths; spans multiple directories (`~/.claude/settings.json`, `~/.claude/settings.local.json`, `~/.claude/skills/`, `~/.claude/plugins/`). Read-only task that requires no modification. Appropriate delegation.
+- Line 97: Agent spawn #2 for "Pull session examples" — analyze JSONL transcripts across 4 project categories to find representative examples. Explicitly requires reaching across many project directories (~/.claude/projects/*); good candidate for parallelization. Appropriate delegation.
+- Line 212: Agent spawn #3 for "Verify skill loading mechanism" — verify factual information from code.claude.com/docs. Specialist task (claude-code-guide subagent type). Appropriate delegation.
+
+**Analysis:**  
+All three delegations follow the spirit of "delegate when work spans many files or needs isolated context." None were single-file operations; all were read-only; none were ≤2-tool-call mechanical ops that should have been done directly.
+
+---
+
+### 2. Is the correct model chosen per subagent — highest reasonable quality at lowest cost?
+
+**Verdict:** FAIL — All spawns lack explicit `model` parameters; two are overpowered.
+
+**Evidence:**
+- Line 95: Agent spawn #1 has no `model` param; `resolvedModel` = `claude-opus-4-8`. Task is character-counting and summation (mechanical, read-only). Appropriate model would be `haiku` (mechanical file-reading) or at most `sonnet` (if judgment needed on "contextually heavy" categorization). **Opus is overpowered.** Fact-sheet: prompt chars = 2662; result chars = 7517.
+- Line 97: Agent spawn #2 has no `model` param; `resolvedModel` = `claude-opus-4-8`. Task is example selection and characterization — requires judgment on what makes examples representative. Appropriate model would be `sonnet` (judgment). **Opus is overpowered.** Fact-sheet: prompt = 2397 chars; result = 6095 chars.
+- Line 212: Agent spawn #3 has no `model` param; `resolvedModel` = `claude-haiku-4-5-20251001`. Task is documentation lookup/verification (mechanical). **Model is correct.** Fact-sheet: prompt = 2065 chars; result = 5290 chars.
+
+**Analysis:**  
+The policy states: "Default `haiku` for mechanical file-edit/shell work; `sonnet` for anything requiring judgment; `opus` only for genuinely hard reasoning."
+
+- Spawn #1: Character-counting is mechanical → should be haiku.
+- Spawn #2: Example selection requires judgment → should be sonnet.
+- Spawn #3: Doc lookup is mechanical → haiku is correct.
+
+The fact that spawn #3 resolved to the correct model despite no explicit param suggests either accidental correct default or the haiku-for-docs-lookup heuristic in the subagent type. Spawns #1 and #2 defaulting to opus indicates the main-loop model is the fallback when no param is passed — a risk.
+
+---
+
+### 3. Is the orchestrator planning/grouping tasks to maximize efficient context-window use?
+
+**Verdict:** MIXED — Good parallelization structure; weak on pre-planning.
+
+**Evidence:**
+- Fact-sheet: Pre-spawn-1 segment: 15 tool calls, 75,718 bytes read. This is heavy read-ahead before delegating.
+- Lines 1–94: The orchestrator read broadly (ToolSearch for deferred tools, WebFetch for docs, 12 Bash calls for inventory, 1 Read). This is exploratory/orientation work.
+- Once orientation was complete, the orchestrator delegated to agents that could parallelize: spawn #1 and #2 are independent (different data sources: settings/skills vs. session transcripts). No coordination between them. Good.
+- After-spawn-2 segment: 15 tool calls, 8100 bytes read. This is synthetic/analysis work between agent results (Bash for intersecting data, measurements, hook timing — not duplicating what agents did).
+- After-spawn-3 segment: 12 tool calls, 5712 bytes read. This is synthesis/application work (editing settings, writing docs).
+
+**Analysis:**  
+The orchestrator did substantial pre-work (75KB read) before spawning agents. This is justified for orientation, but the question is: did it read files the agents would need to read anyway?
+
+The agents' prompts specify their sources clearly (e.g., "Look in ~/.claude/settings.json..."), suggesting the orchestrator didn't pre-read and pass file contents as context (which would bloat the subagent prompts). This is good context discipline. However, the pre-spawn read included some files later read by agents (e.g., settings.json appears in both the orchestrator's pre-read and agent prompts), suggesting some redundancy.
+
+**Grouped well:** Three independent tasks → delegated in parallel (both async spawns at lines 95–97 are background, non-blocking).
+
+---
+
+### 4. Is the orchestrator avoiding reading files it does NOT need (that the subagent would read anyway)?
+
+**Verdict:** MIXED — Mostly good; some redundancy detected.
+
+**Evidence:**
+- Orchestrator's Reads (4 calls): 
+  1. `/tmp/.../tool-results/toolu_01B9mmZvD4pgvmGYRYBi8QsP.txt` — reading its own prior tool output. Necessary.
+  2. `/home/jared/systems-admin/docs/incidents.md` — not directly mentioned in any agent prompt. Supporting context for synthesis.
+  3. `/home/jared/.claude/settings.json` — **this file is explicitly requested in agent spawn #1's prompt** ("~/.claude/settings.json and ~/.claude/settings.local.json"). Orchestrator read it, agent also reads it. **Redundancy.**
+  4. `/home/jared/.claude/projects/-home-jared-systems-admin/memory/MEMORY.md` — session's own project memory, used for synthesis.
+
+- Pre-spawn-1 Bash calls included: "Survey session files by project," "Aggregate tool usage across all sessions," "Full aggregation incl subagents." These are inventory/aggregation, not file-contents reads.
+
+**Analysis:**  
+The orchestrator read `~/.claude/settings.json` before delegating to spawn #1, which also reads the same file. This suggests the orchestrator was orienting to the data before delegating, not trusting the agent to explore independently. The agents' prompts are exploratory ("Look in ~/.claude/skills/ and in installed plugins") and don't require pre-staging of file contents.
+
+**One critical check:** None of the agent prompts contain large file contents (e.g., no "here is your settings.json:..."). The prompts are instructions + paths, not data. This is correct. The orchestrator's redundant read of settings.json was for its own synthesis, not for pre-staging the agent.
+
+---
+
+### 5. Is the orchestrator sharing too much context with subagents (filling their windows / clouding judgment)?
+
+**Verdict:** PASS — Prompts are concise and instructional; no unnecessary context dumps.
+
+**Evidence:**
+- Spawn #1 prompt (Line 95): 2662 chars. Structured as: problem statement (1 sentence), 5 numbered investigation directives (specific file paths, measurement method), deliverable format. No file contents. No system-prompt injections. Clean.
+- Spawn #2 prompt (Line 97): 2397 chars. Structured as: problem (1 sentence), file locations (with technical note about path escaping), 4 concrete example categories, what to report per example, goal statement. No file contents. Clean.
+- Spawn #3 prompt (Line 212): 2065 chars. Structured as: problem (1 sentence), 3 numbered specific questions, instruction to verify via official docs. No file contents. Clean.
+
+**Result sizes:**
+- Spawn #1 result: 7517 chars (3x the prompt) — agent returned structured data (tables, descriptions). Expected.
+- Spawn #2 result: 6095 chars (2.5x the prompt) — agent returned 4 example session case-studies. Expected.
+- Spawn #3 result: 5290 chars (2.5x the prompt) — agent returned answers to 3 questions. Expected.
+
+**Analysis:**  
+The prompts are instruction-heavy and data-light. They guide the agent to read specific paths / sources directly, rather than passing pre-fetched contents. This is efficient and leaves maximum window space in the subagent for reading / reasoning. No bloat detected.
+
+---
+
+### 6. Is the orchestrator even following the ORCHESTRATION.md instructions?
+
+**Verdict:** FAIL — Does not pass `model` explicitly in any spawn; otherwise follows the spirit.
+
+**Evidence:**
+
+**Explicit policy (from ORCHESTRATION.md as of the session date, 2026-06-30):**
+```
+- Do single-file, ≤2-tool-call ops directly. Don't delegate them. Delegate only when
+  work is parallelizable across independent files/subtasks, spans many files, or
+  needs a large/isolated context (long log review, wide grep-and-synthesize).
+- Every `Agent` spawn passes `model` explicitly. Default `haiku` for mechanical
+  file-edit/shell work; `sonnet` for anything requiring judgment; `opus` only for
+  genuinely hard reasoning.
+- A short orienting Read before delegating is fine when the target file/path is
+  uncertain. Don't delegate the orienting step itself.
+```
+
+**Compliance check:**
+
+1. **"Do single-file, ≤2-tool-call ops directly."**  
+   ✓ PASS — All three agent-delegated tasks span multiple files or are large research tasks. No small ops were delegated.
+
+2. **"Every `Agent` spawn passes `model` explicitly."**  
+   ✗ FAIL — Lines 95, 97, 212: **All three spawns omit the `model` parameter.** The policy requires it to be explicit.
+   ```json
+   // Line 95 (should include "model": "haiku")
+   {"name": "Agent", "input": {"description": "Inventory...", "subagent_type": "general-purpose", "prompt": "..."}}
+   
+   // Line 97 (should include "model": "sonnet")
+   {"name": "Agent", "input": {"description": "Pull session...", "subagent_type": "general-purpose", "prompt": "..."}}
+   
+   // Line 212 (should include "model": "haiku")  
+   {"name": "Agent", "input": {"description": "Verify skill...", "subagent_type": "claude-code-guide", "prompt": "..."}}
+   ```
+
+3. **"A short orienting Read before delegating is fine."**  
+   ✓ PASS — The 75KB pre-spawn-1 read is orientation (exploring what tools exist, where transcripts live). It does not hand-hold or pre-fetch entire configs for agents to parse. Appropriate.
+
+**Analysis:**  
+The orchestrator follows the *spirit* of ORCHESTRATION.md (delegate well-scoped, multi-file work; avoid bloating prompts; do light orientation) but **violates the explicit syntactic requirement** that every Agent spawn include a `model` parameter. This is a clear, fixable defect.
+
+---
+
+### 7. Is the orchestrator requesting/receiving back only the context it needs, rather than a full context dump from the subagent?
+
+**Verdict:** PASS — Return values are structured and focused; no raw dumps.
+
+**Evidence:**
+- Spawn #1 deliverable instruction: *"Deliverable (return as structured markdown, this is the data I need back — be concrete with numbers): A ranked table: contributor category → total chars → est. tokens → # items. Top 15 individual heaviest skill/agent descriptions..."*  
+  Result: Structured markdown with tables and brief descriptions. Not a directory dump or full file contents. ✓
+
+- Spawn #2 deliverable instruction: *"Report: Project + short session id, One-line summary... The DISTINCT set of tools actually used... Which broad capabilities were NOT used at all... Any Skill invocations... Return structured markdown."*  
+  Result: 4 case-study sections with the requested fields. Not a raw JSONL dump. ✓
+
+- Spawn #3 deliverable instruction: *"Return a tight, factual answer to each of the three, flagging anything the docs don't clearly state."*  
+  Result: 3 paragraphs answering the 3 questions. Not a doc-dump. ✓
+
+**Analysis:**  
+All three agent results are precisely bounded to what was requested. No agent returned unfiltered file contents, full directory listings, or raw JSONL. The orchestrator specified expected output format (structured markdown, tables, case studies) in each prompt, and agents complied. Efficient.
+
+---
+
+## Detailed Findings
+
+### Model Selection Defect
+
+The most actionable finding is the absence of explicit `model` parameters. This violates the policy and incurs unnecessary cost:
+
+| Spawn | Task | Recommended Model | Resolved Model | Cost Ratio | Impact |
+|-------|------|------|---|---|---|
+| #1 | Inventory (char-count, read-only) | haiku | opus | ~3.5x | Overspent by ~2.5x input cost on a mechanical task |
+| #2 | Example selection (judgment) | sonnet | opus | ~1.5x | Overspent by ~0.5x on judgment task; haiku would likely fail |
+| #3 | Doc verification (lookup) | haiku | haiku | 1x | Correct; lucked into right default via subagent type |
+
+Estimated cost overrun on this session: **~$0.XX** (exact numbers would require token count from subagent transcripts, which this audit cannot access without exceeding scope). For a single session, negligible; systemically across projects, material.
+
+### Context Reuse Opportunity
+
+The orchestrator read `~/.claude/settings.json` before spawn #1, then spawn #1 reads it again. This is not a bloat issue (settings.json is small, ~5–10KB), but it suggests the orchestrator could rely more on agents to explore independently rather than self-orienting on data the agent is designed to discover.
+
+**Candidate rule refinement:** "A short orienting Read before delegating is fine *when needed for task planning or decision-making*, but avoid reading the same data the agent's prompt instructs it to gather."
+
+---
+
+## Recommendations
+
+1. **Fix: Require explicit `model` parameter in every Agent spawn.**  
+   - Add `"model": "haiku"` to spawn #1 (inventory/char-count).
+   - Add `"model": "sonnet"` to spawn #2 (example selection/judgment).
+   - Add `"model": "haiku"` to spawn #3 (doc lookup) — already resolved correctly, but make it explicit.
+
+2. **Consider: Clarify the "orienting Read" exception.**  
+   - Current rule allows "short orienting Read before delegating." Refine to avoid reading data the agent's prompt explicitly instructs it to gather.
+   - Or: Accept that dual-reading small files (~5–10KB) is acceptable cost for clear, self-contained agent prompts (no hand-staging).
+
+3. **No structural issues detected** with task decomposition, prompt design, or result handling. All three delegations were well-scoped and produced focused deliverables.
+
+---
+
+## Appendix: Tool & Context Tracking
+
+**Orchestrator tool calls (117 turns, main loop):**
+- Pre-spawn-1 (lines 1–94): 15 tool calls, 75,718 bytes read
+  - ToolSearch, WebFetch, 12× Bash (exploration), 1× Read
+- Span 1→2 (lines 95–96): 0 tool calls (async agents launching)
+- Between spawn-2 and spawn-3 (lines 97–211): 15 tool calls, 8,100 bytes read
+  - 12× Bash (measurement/analysis), 1× ScheduleWakeup, 1× Read, 1× Write
+- After spawn-3 (lines 212 onwards): 12 tool calls, 5,712 bytes read
+  - 3× Edit (settings.json, audit doc, project memory)
+  - 2× Read (tool results, settings context)
+  - 2× Skill (to-prd, handoff)
+  - 2× Bash (validation, issue tracker check)
+  - 3× Write (final audit docs)
+
+**Subagent spawns:** 3 (all async background; no sequential coupling)
+
+**No missed delegations:** Fact-sheet heuristic detected no runs of ≥4 same-tool calls without delegation.
+
+---
+
+_Audit completed 2026-07-06. Report path: `/tmp/claude-1000/-home-jared-dev-cc-os/3c15edee-8ac7-4ffe-b6ee-ee6a6ebd670d/scratchpad/audits/S9-report.md`_

docs/orchestration-audit/factsheets/S1-df546b88.md

@@ -0,0 +1,48 @@
+# Delegation fact-sheet: df546b88-e27d-47e1-888c-648012c24e62.jsonl
+
+- Transcript: `/home/jared/.claude/projects/-home-jared-dev-cc-os/df546b88-e27d-47e1-888c-648012c24e62.jsonl`
+- cwd: /home/jared/dev/cc-os
+- started: 2026-07-06T15:32:42.854Z
+- ended: 2026-07-06T18:20:38.267Z
+- assistant_turns: 66
+- human_prompts: 11
+- main_loop_models: {"claude-fable-5" => 66}
+- jsonl_lines: 278
+
+## Agent spawns (10)
+
+| # | line | type | model param | resolved model | bg | prompt chars | result chars | description |
+|---|------|------|-------------|----------------|----|--------------|--------------|-------------|
+| 1 | 21 | general-purpose | sonnet | claude-haiku-4-5-20251001 | n | 1098 | 7172 | Research Storybloq project |
+| 2 | 23 | general-purpose | sonnet | claude-haiku-4-5-20251001 | n | 1612 | 18293 | Survey self-hosted kanban tools |
+| 3 | 25 | general-purpose | sonnet | claude-haiku-4-5-20251001 | n | 2121 | 18564 | Research agent-native/markdown backlogs |
+| 4 | 79 | perspectives:devils-advocate | sonnet | claude-haiku-4-5-20251001 | n | 3280 | 9249 | Devils-advocate on vault backlog |
+| 5 | 81 | perspectives:simplifier | sonnet | claude-haiku-4-5-20251001 | n | 2122 | 6673 | Simplifier on vault backlog |
+| 6 | 83 | perspectives:implementer | sonnet | claude-haiku-4-5-20251001 | n | 2531 | 9344 | Implementer on vault backlog |
+| 7 | 85 | perspectives:premortem | sonnet | claude-haiku-4-5-20251001 | n | 2267 | 12426 | Premortem on vault backlog |
+| 8 | 202 | Explore | sonnet | claude-haiku-4-5-20251001 | n | 1120 | 8145 | Survey Hermes agent OS repo |
+| 9 | 218 | general-purpose | sonnet | claude-haiku-4-5-20251001 | n | 1832 | 12430 | Planka maturity deep-dive |
+| 10 | 219 | general-purpose | sonnet | claude-haiku-4-5-20251001 | n | 2301 | 7828 | Modern kanban alternatives sweep |
+
+## Orchestrator tool profile (segments split at each Agent spawn)
+
+| segment | calls | bytes read | per-tool counts |
+|---------|-------|------------|------------------|
+| pre-spawn-1 | 0 | 0 |  |
+| after-spawn-1 | 0 | 0 |  |
+| after-spawn-2 | 0 | 0 |  |
+| after-spawn-3 | 0 | 0 |  |
+| after-spawn-4 | 0 | 0 |  |
+| after-spawn-5 | 0 | 0 |  |
+| after-spawn-6 | 0 | 0 |  |
+| after-spawn-7 | 4 | 970 | Skill:1 Bash:1 Write:2 |
+| after-spawn-8 | 0 | 0 |  |
+| after-spawn-9 | 0 | 0 |  |
+| after-spawn-10 | 6 | 7106 | Bash:3 Read:1 Write:1 Edit:1 |
+
+## Candidate missed delegations (runs of >=4 same-tool calls, distinct targets, no Agent spawn between)
+
+None flagged by heuristic.
+
+Heuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.
+

docs/orchestration-audit/factsheets/S10-f6a224d0.md

@@ -0,0 +1,31 @@
+# Delegation fact-sheet: f6a224d0-ddc2-488c-b0cc-728fdf21cc08.jsonl
+
+- Transcript: `/home/jared/.claude/projects/-home-jared-systems-admin/f6a224d0-ddc2-488c-b0cc-728fdf21cc08.jsonl`
+- cwd: /home/jared/systems-admin
+- started: 2026-07-01T12:37:51.119Z
+- ended: 2026-07-01T14:02:57.530Z
+- assistant_turns: 53
+- human_prompts: 8
+- main_loop_models: {"<synthetic>" => 1, "claude-sonnet-5" => 52}
+- jsonl_lines: 183
+
+## Agent spawns (0)
+
+None.
+
+## Orchestrator tool profile (segments split at each Agent spawn)
+
+| segment | calls | bytes read | per-tool counts |
+|---------|-------|------------|------------------|
+| pre-spawn-1 | 25 | 33271 | Read:4 Bash:12 Edit:4 Write:5 |
+
+## Candidate missed delegations (runs of >=4 same-tool calls, distinct targets, no Agent spawn between)
+
+- **Write x4** (jsonl lines 125-137, 4 distinct targets)
+  - `/home/jared/dev/cc-plugins/orchestration/.claude-plugin/plugin.json`
+  - `/home/jared/dev/cc-plugins/orchestration/ORCHESTRATION.md`
+  - `/home/jared/dev/cc-plugins/orchestration/hooks/inject.py`
+  - `/home/jared/dev/cc-plugins/orchestration/hooks/hooks.json`
+
+Heuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.
+

docs/orchestration-audit/factsheets/S2-d68aab1b.md

@@ -0,0 +1,54 @@
+# Delegation fact-sheet: d68aab1b-3e3d-467c-9b9c-70b89e2576b0.jsonl
+
+- Transcript: `/home/jared/.claude/projects/-home-jared-dev-cc-os/d68aab1b-3e3d-467c-9b9c-70b89e2576b0.jsonl`
+- cwd: /home/jared/dev/cc-os
+- started: 2026-07-06T14:44:50.025Z
+- ended: 2026-07-06T17:58:03.100Z
+- assistant_turns: 207
+- human_prompts: 18
+- main_loop_models: {"<synthetic>" => 1, "claude-fable-5" => 206}
+- jsonl_lines: 643
+
+## Agent spawns (8)
+
+| # | line | type | model param | resolved model | bg | prompt chars | result chars | description |
+|---|------|------|-------------|----------------|----|--------------|--------------|-------------|
+| 1 | 28 | perspectives:devils-advocate | sonnet | claude-haiku-4-5-20251001 | n | 2566 | 9661 | Challenge eval validity claims |
+| 2 | 30 | perspectives:measurement | sonnet | claude-haiku-4-5-20251001 | n | 2483 | 12982 | Assess eval measurement value |
+| 3 | 72 | general-purpose | sonnet | claude-haiku-4-5-20251001 | n | 2565 | 2905 | Run 3-rep haiku W3 check |
+| 4 | 74 | general-purpose | opus | claude-haiku-4-5-20251001 | n | 6132 | 8289 | Build Eval C ambiguity ladder |
+| 5 | 76 | general-purpose | sonnet | claude-haiku-4-5-20251001 | n | 6759 | 5337 | Write/upgrade vault eval notes |
+| 6 | 296 | git-context:commit | haiku | claude-haiku-4-5-20251001 | n | 74 | 759 | Commit current changes |
+| 7 | 506 | git-context:commit | haiku | claude-haiku-4-5-20251001 | n | 473 | 1354 | Commit current changes |
+| 8 | 628 | git-context:commit | haiku | claude-haiku-4-5-20251001 | n | 496 | 1152 | Commit plan docs |
+
+## Orchestrator tool profile (segments split at each Agent spawn)
+
+| segment | calls | bytes read | per-tool counts |
+|---------|-------|------------|------------------|
+| pre-spawn-1 | 0 | 0 |  |
+| after-spawn-1 | 0 | 0 |  |
+| after-spawn-2 | 0 | 0 |  |
+| after-spawn-3 | 0 | 0 |  |
+| after-spawn-4 | 0 | 0 |  |
+| after-spawn-5 | 37 | 41558 | Bash:21 Edit:9 Read:5 ToolSearch:1 SendMessage:1 |
+| after-spawn-6 | 32 | 61527 | Bash:14 Read:11 Edit:6 Write:1 |
+| after-spawn-7 | 15 | 24354 | Read:3 Edit:5 Bash:3 Write:4 |
+| after-spawn-8 | 1 | 563 | Bash:1 |
+
+## Candidate missed delegations (runs of >=4 same-tool calls, distinct targets, no Agent spawn between)
+
+- **Read x5** (jsonl lines 389-405, 5 distinct targets)
+  - `/home/jared/dev/cc-os/plugins/os-adr/eval-c/scenarios/P1-L1-execution.md`
+  - `/home/jared/dev/cc-os/plugins/os-adr/eval-c/scenarios/P2-L2-execution.md`
+  - `/home/jared/dev/cc-os/plugins/os-adr/eval-c/scenarios/P3-L3-execution.md`
+  - `/home/jared/dev/cc-os/plugins/os-adr/eval-c/judge-rubric.md`
+  - `/home/jared/dev/cc-os/plugins/os-adr/eval-c/bin/check`
+- **Write x4** (jsonl lines 611-624, 4 distinct targets)
+  - `/home/jared/dev/cc-os/docs/plans/2026-07-06-plugin-evals-overview.md`
+  - `/home/jared/dev/cc-os/docs/plans/ws1-orchestration-audit.md`
+  - `/home/jared/dev/cc-os/docs/plans/ws2-os-vault-write-eval.md`
+  - `/home/jared/dev/cc-os/docs/plans/ws3-status-convention-plugin.md`
+
+Heuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.
+

docs/orchestration-audit/factsheets/S3-9f45afcc.md

@@ -0,0 +1,38 @@
+# Delegation fact-sheet: 9f45afcc-75ff-4b7a-8f3d-008aa0d599af.jsonl
+
+- Transcript: `/home/jared/.claude/projects/-home-jared-dev-cc-os/9f45afcc-75ff-4b7a-8f3d-008aa0d599af.jsonl`
+- cwd: /home/jared/dev/cc-os
+- started: 2026-07-04T16:17:45.947Z
+- ended: 2026-07-04T18:52:42.079Z
+- assistant_turns: 255
+- human_prompts: 25
+- main_loop_models: {"claude-fable-5" => 255}
+- jsonl_lines: 850
+
+## Agent spawns (5)
+
+| # | line | type | model param | resolved model | bg | prompt chars | result chars | description |
+|---|------|------|-------------|----------------|----|--------------|--------------|-------------|
+| 1 | 124 | general-purpose | sonnet | claude-haiku-4-5-20251001 | n | 4849 | 3951 | Fix plugin caches, add refresh tooling |
+| 2 | 133 | general-purpose | sonnet | claude-haiku-4-5-20251001 | y | 2177 | 5905 | Re-run Eval B grid post-fix |
+| 3 | 171 | general-purpose | sonnet | claude-haiku-4-5-20251001 | y | 49 | 878 | Resume eval grid agent |
+| 4 | 603 | general-purpose | sonnet | claude-haiku-4-5-20251001 | n | 2193 | 1606 | Triage wording on sonnet |
+| 5 | 605 | general-purpose | haiku | claude-haiku-4-5-20251001 | n | 2193 | 1235 | Triage wording on haiku |
+
+## Orchestrator tool profile (segments split at each Agent spawn)
+
+| segment | calls | bytes read | per-tool counts |
+|---------|-------|------------|------------------|
+| pre-spawn-1 | 19 | 23658 | Bash:14 Read:4 Edit:1 |
+| after-spawn-1 | 0 | 0 |  |
+| after-spawn-2 | 0 | 0 |  |
+| after-spawn-3 | 48 | 29966 | ToolSearch:2 SendMessage:1 Bash:22 Edit:12 Skill:2 Read:5 Write:3 TaskStop:1 |
+| after-spawn-4 | 0 | 0 |  |
+| after-spawn-5 | 43 | 14410 | Write:4 Bash:27 Edit:11 Read:1 |
+
+## Candidate missed delegations (runs of >=4 same-tool calls, distinct targets, no Agent spawn between)
+
+None flagged by heuristic.
+
+Heuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.
+

docs/orchestration-audit/factsheets/S4-b6b07a91.md

@@ -0,0 +1,110 @@
+# Delegation fact-sheet: b6b07a91-5aa3-4521-8476-c6749ee76017.jsonl
+
+- Transcript: `/home/jared/.claude/projects/-home-jared-dev-llf-schema/b6b07a91-5aa3-4521-8476-c6749ee76017.jsonl`
+- cwd: /home/jared/dev/llf-schema
+- started: 2026-06-30T14:34:11.662Z
+- ended: 2026-06-30T19:12:36.755Z
+- assistant_turns: 191
+- human_prompts: 20
+- main_loop_models: {"claude-opus-4-8" => 191}
+- jsonl_lines: 766
+
+## Agent spawns (41)
+
+| # | line | type | model param | resolved model | bg | prompt chars | result chars | description |
+|---|------|------|-------------|----------------|----|--------------|--------------|-------------|
+| 1 | 21 | general-purpose | (ABSENT) | claude-opus-4-8 | n | 1275 | 14627 | Gather openspec change context |
+| 2 | 36 | general-purpose | sonnet | claude-sonnet-4-6 | n | 1667 | 2031 | Task 1.1 fix gbp_url gate |
+| 3 | 38 | general-purpose | sonnet | claude-sonnet-4-6 | n | 4457 | 3827 | Tasks 2.1 2.2 3.1 verifier profiles |
+| 4 | 40 | general-purpose | haiku | claude-haiku-4-5-20251001 | n | 1818 | 2142 | Task 3.3 coverage checklist doc |
+| 5 | 73 | general-purpose | haiku | claude-haiku-4-5-20251001 | n | 512 | 749 | Mark batch 1 tasks complete |
+| 6 | 75 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3684 | 2730 | Task 1.2 regenerate golden fixture |
+| 7 | 77 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3979 | 2923 | Task 3.2 self-check flag |
+| 8 | 102 | general-purpose | haiku | claude-haiku-4-5-20251001 | n | 436 | 749 | Mark batch 2 tasks complete |
+| 9 | 104 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3652 | 3547 | Tasks 4.1 4.2 seed and verify loop |
+| 10 | 106 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3469 | 3230 | Task 5.1 verify-schema skill |
+| 11 | 132 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3798 | 3992 | Reconcile manifest office paths |
+| 12 | 134 | general-purpose | haiku | claude-haiku-4-5-20251001 | n | 456 | 1022 | Mark batch 3 tasks complete |
+| 13 | 155 | general-purpose | sonnet | claude-sonnet-4-6 | n | 1011 | 1246 | Fix stale attorney URL in test-deploy |
+| 14 | 167 | general-purpose | sonnet | claude-sonnet-4-6 | n | 1998 | 2055 | Task 6.1 verify gate |
+| 15 | 185 | general-purpose | sonnet | claude-sonnet-4-6 | n | 2537 | 2210 | Task 6.2 branch and commit |
+| 16 | 199 | general-purpose | haiku | claude-haiku-4-5-20251001 | n | 659 | 867 | Mark task 6.2 complete |
+| 17 | 219 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3462 | 10703 | Verify changes align with ADRs |
+| 18 | 234 | general-purpose | sonnet | claude-sonnet-4-6 | n | 1907 | 1664 | Fix stale docblock and amend |
+| 19 | 249 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3164 | 3400 | Task 6.3 run release.sh minor |
+| 20 | 265 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3499 | 7567 | Diagnose NJ office duplicate LegalService |
+| 21 | 267 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3644 | 5041 | Diagnose service-page WebPage description |
+| 22 | 298 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3587 | 4916 | Check live HQ office gbp_url both sites |
+| 23 | 366 | general-purpose | sonnet | claude-sonnet-4-6 | n | 2774 | 6262 | Authoritative working-tree status snapshot |
+| 24 | 411 | claude | (ABSENT) | claude-opus-4-8 | n | 2766 | 2211 | Commit harden-schema-verification work |
+| 25 | 427 | claude | (ABSENT) | claude-opus-4-8 | n | 2126 | 2144 | Run release.sh patch to v0.2.5 |
+| 26 | 440 | claude | (ABSENT) | claude-opus-4-8 | n | 2311 | 3023 | Check live service-page WebPage.description |
+| 27 | 466 | Explore | (ABSENT) | claude-haiku-4-5-20251001 | n | 1901 | 2238 | Check spec + golden fixture for WebPage.description |
+| 28 | 494 | Explore | (ABSENT) | claude-haiku-4-5-20251001 | n | 2160 | 5364 | How non-service pages set description |
+| 29 | 496 | claude | (ABSENT) | claude-opus-4-8 | n | 2423 | 2564 | Check post_excerpt on live SDD pages |
+| 30 | 529 | claude | (ABSENT) | claude-opus-4-8 | n | 5292 | 2958 | Relax description check, commit, re-release |
+| 31 | 563 | claude | (ABSENT) | claude-opus-4-8 | n | 854 | 1264 | Push branch and tags |
+| 32 | 581 | claude | haiku | claude-haiku-4-5-20251001 | n | 2727 | 940 | Write WebPage.description memory |
+| 33 | 623 | general-purpose | sonnet | claude-sonnet-4-6 | n | 3200 | 3557 | Verify PLD contact-us schema |
+| 34 | 625 | general-purpose | sonnet | claude-sonnet-4-6 | n | 4030 | 3832 | Verify SDD contact-us schema |
+| 35 | 664 | general-purpose | haiku | claude-haiku-4-5-20251001 | n | 1019 | 2320 | Sample-verify PLD schema |
+| 36 | 666 | general-purpose | haiku | claude-haiku-4-5-20251001 | n | 1014 | 2575 | Sample-verify SDD schema |
+| 37 | 670 | general-purpose | sonnet | claude-sonnet-4-6 | n | 2776 | 749 | File blocked areaServed issue |
+| 38 | 696 | general-purpose | sonnet | claude-sonnet-4-6 | n | 2615 | 749 | Diagnose SDD stale schema |
+| 39 | 712 | general-purpose | haiku | claude-haiku-4-5-20251001 | n | 1578 | 1607 | Re-verify SDD post cache purge |
+| 40 | 725 | general-purpose | sonnet | claude-sonnet-4-6 | n | 2247 | 749 | Check SDD active plugin version |
+| 41 | 748 | general-purpose | sonnet | claude-sonnet-4-6 | n | 5170 | 2351 | Finalize v0.2.5: issue, notes, merge |
+
+## Orchestrator tool profile (segments split at each Agent spawn)
+
+| segment | calls | bytes read | per-tool counts |
+|---------|-------|------------|------------------|
+| pre-spawn-1 | 0 | 0 |  |
+| after-spawn-1 | 0 | 0 |  |
+| after-spawn-2 | 0 | 0 |  |
+| after-spawn-3 | 0 | 0 |  |
+| after-spawn-4 | 0 | 0 |  |
+| after-spawn-5 | 0 | 0 |  |
+| after-spawn-6 | 0 | 0 |  |
+| after-spawn-7 | 0 | 0 |  |
+| after-spawn-8 | 0 | 0 |  |
+| after-spawn-9 | 0 | 0 |  |
+| after-spawn-10 | 0 | 0 |  |
+| after-spawn-11 | 0 | 0 |  |
+| after-spawn-12 | 0 | 0 |  |
+| after-spawn-13 | 0 | 0 |  |
+| after-spawn-14 | 1 | 0 | AskUserQuestion:1 |
+| after-spawn-15 | 0 | 0 |  |
+| after-spawn-16 | 0 | 0 |  |
+| after-spawn-17 | 0 | 0 |  |
+| after-spawn-18 | 0 | 0 |  |
+| after-spawn-19 | 0 | 0 |  |
+| after-spawn-20 | 0 | 0 |  |
+| after-spawn-21 | 0 | 0 |  |
+| after-spawn-22 | 1 | 0 | AskUserQuestion:1 |
+| after-spawn-23 | 0 | 0 |  |
+| after-spawn-24 | 0 | 0 |  |
+| after-spawn-25 | 0 | 0 |  |
+| after-spawn-26 | 0 | 0 |  |
+| after-spawn-27 | 0 | 0 |  |
+| after-spawn-28 | 0 | 0 |  |
+| after-spawn-29 | 1 | 0 | AskUserQuestion:1 |
+| after-spawn-30 | 0 | 0 |  |
+| after-spawn-31 | 0 | 0 |  |
+| after-spawn-32 | 0 | 0 |  |
+| after-spawn-33 | 0 | 0 |  |
+| after-spawn-34 | 1 | 0 | Skill:1 |
+| after-spawn-35 | 0 | 0 |  |
+| after-spawn-36 | 0 | 0 |  |
+| after-spawn-37 | 0 | 0 |  |
+| after-spawn-38 | 0 | 0 |  |
+| after-spawn-39 | 0 | 0 |  |
+| after-spawn-40 | 3 | 0 | ToolSearch:1 TaskStop:1 AskUserQuestion:1 |
+| after-spawn-41 | 0 | 0 |  |
+
+## Candidate missed delegations (runs of >=4 same-tool calls, distinct targets, no Agent spawn between)
+
+None flagged by heuristic.
+
+Heuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.
+

docs/orchestration-audit/factsheets/S5-aadeb66c.md

@@ -0,0 +1,44 @@
+# Delegation fact-sheet: aadeb66c-0d64-4d41-b407-83e0209428b4.jsonl
+
+- Transcript: `/home/jared/.claude/projects/-home-jared-dev-llf-schema/aadeb66c-0d64-4d41-b407-83e0209428b4.jsonl`
+- cwd: /home/jared/dev/llf-schema
+- started: 2026-06-29T16:51:34.375Z
+- ended: 2026-06-29T19:10:30.177Z
+- assistant_turns: 36
+- human_prompts: 9
+- main_loop_models: {"claude-sonnet-4-6" => 36}
+- jsonl_lines: 161
+
+## Agent spawns (8)
+
+| # | line | type | model param | resolved model | bg | prompt chars | result chars | description |
+|---|------|------|-------------|----------------|----|--------------|--------------|-------------|
+| 1 | 23 | general-purpose | (ABSENT) | claude-sonnet-4-6 | n | 838 | 2352 | Audit codebase for contact-us schema support |
+| 2 | 24 | general-purpose | (ABSENT) | claude-sonnet-4-6 | n | 589 | 792 | Check live SDD contact-us page for schema output |
+| 3 | 41 | general-purpose | (ABSENT) | claude-sonnet-4-6 | n | 674 | 2917 | Check SDD office ACF data via wp-cli |
+| 4 | 60 | general-purpose | (ABSENT) | claude-sonnet-4-6 | n | 419 | 7842 | Read codebase for ContactPagePiece implementation context |
+| 5 | 83 | general-purpose | (ABSENT) | claude-sonnet-4-6 | n | 635 | 5363 | Read additional context files for implementation |
+| 6 | 99 | general-purpose | (ABSENT) | claude-sonnet-4-6 | n | 804 | 12076 | Read remaining files for implementation spec |
+| 7 | 115 | general-purpose | sonnet | claude-sonnet-4-6 | n | 6090 | 2294 | Implement ContactPagePiece with tests |
+| 8 | 141 | general-purpose | haiku | claude-haiku-4-5-20251001 | n | 419 | 13465 | Verify ContactPagePiece implementation files |
+
+## Orchestrator tool profile (segments split at each Agent spawn)
+
+| segment | calls | bytes read | per-tool counts |
+|---------|-------|------------|------------------|
+| pre-spawn-1 | 0 | 0 |  |
+| after-spawn-1 | 0 | 0 |  |
+| after-spawn-2 | 0 | 0 |  |
+| after-spawn-3 | 0 | 0 |  |
+| after-spawn-4 | 0 | 0 |  |
+| after-spawn-5 | 0 | 0 |  |
+| after-spawn-6 | 0 | 0 |  |
+| after-spawn-7 | 0 | 0 |  |
+| after-spawn-8 | 0 | 0 |  |
+
+## Candidate missed delegations (runs of >=4 same-tool calls, distinct targets, no Agent spawn between)
+
+None flagged by heuristic.
+
+Heuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.
+

docs/orchestration-audit/factsheets/S6-3dc26da3.md

@@ -0,0 +1,30 @@
+# Delegation fact-sheet: 3dc26da3-de82-4523-aca5-40c06b3616b3.jsonl
+
+- Transcript: `/home/jared/.claude/projects/-home-jared-clients-philly-search-engine-marketing/3dc26da3-de82-4523-aca5-40c06b3616b3.jsonl`
+- cwd: /home/jared/clients/philly-search-engine-marketing
+- started: 2026-06-29T15:08:49.497Z
+- ended: 2026-06-29T16:58:40.721Z
+- assistant_turns: 21
+- human_prompts: 13
+- main_loop_models: {"<synthetic>" => 1, "claude-sonnet-4-6" => 20}
+- jsonl_lines: 120
+
+## Agent spawns (1)
+
+| # | line | type | model param | resolved model | bg | prompt chars | result chars | description |
+|---|------|------|-------------|----------------|----|--------------|--------------|-------------|
+| 1 | 98 | general-purpose | (ABSENT) | claude-sonnet-4-6 | n | 2039 | 1811 | Update TODO and session log |
+
+## Orchestrator tool profile (segments split at each Agent spawn)
+
+| segment | calls | bytes read | per-tool counts |
+|---------|-------|------------|------------------|
+| pre-spawn-1 | 0 | 0 |  |
+| after-spawn-1 | 0 | 0 |  |
+
+## Candidate missed delegations (runs of >=4 same-tool calls, distinct targets, no Agent spawn between)
+
+None flagged by heuristic.
+
+Heuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.
+

docs/orchestration-audit/factsheets/S7-3771b288.md

@@ -0,0 +1,39 @@
+# Delegation fact-sheet: 3771b288-eb9d-4398-9e75-e1ff0ba6af03.jsonl
+
+- Transcript: `/home/jared/.claude/projects/-home-jared-servers/3771b288-eb9d-4398-9e75-e1ff0ba6af03.jsonl`
+- cwd: /home/jared/servers
+- started: 2026-07-04T16:05:40.895Z
+- ended: 2026-07-04T17:14:47.297Z
+- assistant_turns: 144
+- human_prompts: 11
+- main_loop_models: {"<synthetic>" => 1, "claude-sonnet-5" => 143}
+- jsonl_lines: 456
+
+## Agent spawns (3)
+
+| # | line | type | model param | resolved model | bg | prompt chars | result chars | description |
+|---|------|------|-------------|----------------|----|--------------|--------------|-------------|
+| 1 | 241 | Explore | (ABSENT) | claude-haiku-4-5-20251001 | n | 2282 | 7537 | Explore ovh-prod service setup conventions |
+| 2 | 243 | Explore | (ABSENT) | claude-haiku-4-5-20251001 | n | 2725 | 3562 | Explore desktop backup bug specifics |
+| 3 | 245 | Explore | (ABSENT) | claude-haiku-4-5-20251001 | n | 2126 | 4776 | Explore proxmox VM 101 and umbrella structure |
+
+## Orchestrator tool profile (segments split at each Agent spawn)
+
+| segment | calls | bytes read | per-tool counts |
+|---------|-------|------------|------------------|
+| pre-spawn-1 | 37 | 74129 | Read:4 Bash:32 Edit:1 |
+| after-spawn-1 | 0 | 0 |  |
+| after-spawn-2 | 0 | 0 |  |
+| after-spawn-3 | 36 | 37170 | AskUserQuestion:1 Write:6 ToolSearch:1 ExitPlanMode:1 Read:3 Edit:6 Bash:18 |
+
+## Candidate missed delegations (runs of >=4 same-tool calls, distinct targets, no Agent spawn between)
+
+- **Write x5** (jsonl lines 345-353, 5 distinct targets)
+  - `/home/jared/servers/proxmox-ubuntu/CLAUDE.md`
+  - `/home/jared/servers/proxmox-ubuntu/README.md`
+  - `/home/jared/servers/proxmox-ubuntu/docs/services-inventory.md`
+  - `/home/jared/servers/proxmox-ubuntu/docs/roadmap.md`
+  - `/home/jared/servers/ovh-prod/docs/incidents.md`
+
+Heuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.
+

docs/orchestration-audit/factsheets/S8-3fc7bb8c.md

@@ -0,0 +1,27 @@
+# Delegation fact-sheet: 3fc7bb8c-54c1-4147-ac3d-dc7db3fca924.jsonl
+
+- Transcript: `/home/jared/.claude/projects/-home-jared-servers/3fc7bb8c-54c1-4147-ac3d-dc7db3fca924.jsonl`
+- cwd: /home/jared/servers
+- started: 2026-07-04T17:20:10.755Z
+- ended: 2026-07-04T18:04:17.446Z
+- assistant_turns: 90
+- human_prompts: 5
+- main_loop_models: {"claude-fable-5" => 90}
+- jsonl_lines: 236
+
+## Agent spawns (0)
+
+None.
+
+## Orchestrator tool profile (segments split at each Agent spawn)
+
+| segment | calls | bytes read | per-tool counts |
+|---------|-------|------------|------------------|
+| pre-spawn-1 | 44 | 32250 | Read:3 Bash:27 Write:4 ToolSearch:2 ExitPlanMode:1 Edit:6 Skill:1 |
+
+## Candidate missed delegations (runs of >=4 same-tool calls, distinct targets, no Agent spawn between)
+
+None flagged by heuristic.
+
+Heuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.
+

docs/orchestration-audit/factsheets/S9-2525fa2d.md

@@ -0,0 +1,34 @@
+# Delegation fact-sheet: 2525fa2d-1b15-4495-99bf-c093716fab55.jsonl
+
+- Transcript: `/home/jared/.claude/projects/-home-jared-systems-admin/2525fa2d-1b15-4495-99bf-c093716fab55.jsonl`
+- cwd: /home/jared/systems-admin
+- started: 2026-06-30T19:23:34.370Z
+- ended: 2026-06-30T20:22:50.581Z
+- assistant_turns: 117
+- human_prompts: 6
+- main_loop_models: {"claude-opus-4-8" => 117}
+- jsonl_lines: 302
+
+## Agent spawns (3)
+
+| # | line | type | model param | resolved model | bg | prompt chars | result chars | description |
+|---|------|------|-------------|----------------|----|--------------|--------------|-------------|
+| 1 | 95 | general-purpose | (ABSENT) | claude-opus-4-8 | n | 2662 | 7517 | Inventory loaded context surface |
+| 2 | 97 | general-purpose | (ABSENT) | claude-opus-4-8 | n | 2397 | 6095 | Pull session examples |
+| 3 | 212 | claude-code-guide | (ABSENT) | claude-haiku-4-5-20251001 | n | 2065 | 5290 | Verify skill loading mechanism |
+
+## Orchestrator tool profile (segments split at each Agent spawn)
+
+| segment | calls | bytes read | per-tool counts |
+|---------|-------|------------|------------------|
+| pre-spawn-1 | 15 | 75718 | ToolSearch:1 WebFetch:1 Bash:12 Read:1 |
+| after-spawn-1 | 0 | 0 |  |
+| after-spawn-2 | 15 | 8100 | Bash:12 ScheduleWakeup:1 Read:1 Write:1 |
+| after-spawn-3 | 12 | 5712 | Edit:3 Read:2 Bash:2 Skill:2 Write:3 |
+
+## Candidate missed delegations (runs of >=4 same-tool calls, distinct targets, no Agent spawn between)
+
+None flagged by heuristic.
+
+Heuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.
+

docs/plans/2026-07-06-plugin-evals-overview.md

@@ -11,6 +11,9 @@ discipline only.
 1. **WS1 — os-orchestration session audit** → `ws1-orchestration-audit.md`
    Evidence-gathering over real session transcripts; produces failure-mode clusters and an
    eval-scenario backlog. No design dependencies — start anytime.
+   **DONE 2026-07-06:** findings in `docs/orchestration-audit/2026-07-06-findings.md`
+   (4 verified clusters, E1–E4 scenario backlog); extractor at
+   `plugins/os-orchestration/audit/bin/extract`.
 2. **WS2 — os-vault write-behavior eval** → `ws2-os-vault-write-eval.md`
    Two quick wins (stale write-skill contract fix; cc-os hub note) can run immediately.
    The eval harness itself should start after WS3's design lands (hub-note automation is a

docs/plans/ws1-orchestration-audit.md

@@ -1,6 +1,8 @@
 # WS1 — os-orchestration session audit
 
-_Created: 2026-07-06. Status: proposed. Prereq: none. See `2026-07-06-plugin-evals-overview.md`._
+_Created: 2026-07-06. Status: **complete 2026-07-06** — findings + scenario backlog in
+`docs/orchestration-audit/2026-07-06-findings.md`; extractor shipped at
+`plugins/os-orchestration/audit/bin/extract`. See `2026-07-06-plugin-evals-overview.md`._
 
 ## Goal
 

plugins/os-orchestration/audit/bin/extract

@@ -0,0 +1,363 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Reduce a Claude Code session transcript (.jsonl) to a delegation fact-sheet.
+#
+# Usage: extract [--json] [--run-threshold N] SESSION.jsonl [...]
+#
+# Dual-use: human/auditor-readable markdown by default; --json for the
+# structured form (checker core for a future orchestration eval).
+
+require "json"
+require "optparse"
+
+module OrchAudit
+  READ_TOOLS = %w[Read Grep Glob Bash].freeze
+  RUN_TOOLS = %w[Read Grep Glob Edit Write].freeze
+
+  class Line
+    attr_reader :number, :raw
+
+    def initialize(number, raw)
+      @number = number
+      @raw = raw
+    end
+
+    def type = raw["type"]
+    def sidechain? = raw["isSidechain"] == true
+    def message = raw["message"] || {}
+    def timestamp = raw["timestamp"]
+    def cwd = raw["cwd"]
+    def model = message["model"]
+
+    def content_blocks
+      c = message["content"]
+      c.is_a?(Array) ? c.select { |b| b.is_a?(Hash) } : []
+    end
+
+    def tool_uses
+      return [] unless type == "assistant"
+      content_blocks.select { |b| b["type"] == "tool_use" }
+    end
+
+    def tool_results
+      return [] unless type == "user"
+      content_blocks.select { |b| b["type"] == "tool_result" }
+    end
+
+    def human_prompt?
+      return false unless type == "user" && !sidechain?
+      c = message["content"]
+      return !c.lstrip.start_with?("<task-notification>") if c.is_a?(String)
+      c.is_a?(Array) && c.none? { |b| b.is_a?(Hash) && b["type"] == "tool_result" }
+    end
+
+    def task_notification
+      return nil unless type == "user"
+      c = message["content"]
+      c = c.filter_map { |b| b["text"] if b.is_a?(Hash) }.join if c.is_a?(Array)
+      return nil unless c.is_a?(String) && c.include?("<task-notification>")
+      { task_id: c[%r{<task-id>([^<]+)</task-id>}, 1], chars: c.length }
+    end
+  end
+
+  class ToolCall
+    attr_reader :line, :block
+    attr_accessor :result_chars
+
+    def initialize(line, block)
+      @line = line
+      @block = block
+      @result_chars = 0
+    end
+
+    def id = block["id"]
+    def name = block["name"]
+    def input = block["input"] || {}
+
+    def target
+      input["file_path"] || input["path"] || input["pattern"] || input["command"] || input["prompt"]
+    end
+  end
+
+  class AgentSpawn
+    attr_reader :call, :index
+    attr_accessor :agent_id, :resolved_model, :notification_chars
+
+    def initialize(call, index)
+      @call = call
+      @index = index
+      @notification_chars = 0
+    end
+
+    def to_h
+      {
+        seq: index,
+        line: call.line.number,
+        timestamp: call.line.timestamp,
+        subagent_type: call.input["subagent_type"] || "general-purpose",
+        model_param: call.input["model"],
+        model_explicit: call.input.key?("model"),
+        resolved_model: resolved_model,
+        run_in_background: call.input["run_in_background"],
+        prompt_chars: (call.input["prompt"] || "").length,
+        description: call.input["description"],
+        result_chars: [call.result_chars, notification_chars].max
+      }
+    end
+  end
+
+  class Segment
+    attr_reader :label
+
+    def initialize(label)
+      @label = label
+      @calls = []
+    end
+
+    def add(call) = @calls << call
+
+    def to_h
+      counts = Hash.new(0)
+      bytes = 0
+      @calls.each do |c|
+        counts[c.name] += 1
+        bytes += c.result_chars if READ_TOOLS.include?(c.name)
+      end
+      { label: label, tool_counts: counts, bytes_read: bytes, total_calls: @calls.size }
+    end
+  end
+
+  class MissedDelegationScan
+    def initialize(calls, threshold)
+      @calls = calls
+      @threshold = threshold
+    end
+
+    def candidates
+      runs = []
+      current = []
+      @calls.each do |call|
+        if extend_run?(current, call)
+          current << call
+        else
+          runs << current if qualifying?(current)
+          current = RUN_TOOLS.include?(call.name) ? [call] : []
+        end
+      end
+      runs << current if qualifying?(current)
+      runs.map { |r| describe(r) }
+    end
+
+    private
+
+    def extend_run?(run, call)
+      return false if run.empty?
+      run.first.name == call.name && RUN_TOOLS.include?(call.name)
+    end
+
+    def qualifying?(run)
+      return false if run.size < @threshold
+      run.map(&:target).uniq.size >= @threshold
+    end
+
+    def describe(run)
+      {
+        tool: run.first.name,
+        length: run.size,
+        lines: [run.first.line.number, run.last.line.number],
+        distinct_targets: run.map(&:target).uniq.size,
+        targets_sample: run.map(&:target).uniq.first(6).map { |t| t.to_s[0, 90] }
+      }
+    end
+  end
+
+  class FactSheet
+    def initialize(path, run_threshold: 4)
+      @path = path
+      @run_threshold = run_threshold
+    end
+
+    def data
+      @data ||= build
+    end
+
+    def to_json_s = JSON.pretty_generate(data)
+
+    def to_markdown
+      d = data
+      md = +"# Delegation fact-sheet: #{File.basename(@path)}\n\n"
+      md << "- Transcript: `#{@path}`\n"
+      d[:metadata].each { |k, v| md << "- #{k}: #{v}\n" }
+      md << "\n## Agent spawns (#{d[:agent_spawns].size})\n\n"
+      if d[:agent_spawns].empty?
+        md << "None.\n"
+      else
+        md << "| # | line | type | model param | resolved model | bg | prompt chars | result chars | description |\n"
+        md << "|---|------|------|-------------|----------------|----|--------------|--------------|-------------|\n"
+        d[:agent_spawns].each do |a|
+          model = a[:model_explicit] ? a[:model_param].to_s : "(ABSENT)"
+          md << "| #{a[:seq]} | #{a[:line]} | #{a[:subagent_type]} | #{model} | #{a[:resolved_model]} | #{a[:run_in_background] ? 'y' : 'n'} | #{a[:prompt_chars]} | #{a[:result_chars]} | #{a[:description]} |\n"
+        end
+      end
+      md << "\n## Orchestrator tool profile (segments split at each Agent spawn)\n\n"
+      md << "| segment | calls | bytes read | per-tool counts |\n|---------|-------|------------|------------------|\n"
+      d[:segments].each do |s|
+        counts = s[:tool_counts].map { |k, v| "#{k}:#{v}" }.join(" ")
+        md << "| #{s[:label]} | #{s[:total_calls]} | #{s[:bytes_read]} | #{counts} |\n"
+      end
+      md << "\n## Candidate missed delegations (runs of >=#{@run_threshold} same-tool calls, distinct targets, no Agent spawn between)\n\n"
+      if d[:missed_delegation_candidates].empty?
+        md << "None flagged by heuristic.\n"
+      else
+        d[:missed_delegation_candidates].each do |m|
+          md << "- **#{m[:tool]} x#{m[:length]}** (jsonl lines #{m[:lines].join('-')}, #{m[:distinct_targets]} distinct targets)\n"
+          m[:targets_sample].each { |t| md << "  - `#{t}`\n" }
+        end
+      end
+      md << "\nHeuristic candidates require auditor judgment; sequential-dependent work is a valid reason not to delegate.\n"
+      md
+    end
+
+    private
+
+    def build
+      lines = parse_lines
+      main = lines.reject(&:sidechain?)
+      calls, agent_spawns = collect_calls(main)
+      attach_results(main, calls)
+      attach_agent_metadata(main, calls, agent_spawns)
+
+      orchestrator_calls = calls.values.reject { |c| c.name == "Agent" }
+      {
+        metadata: metadata(main),
+        agent_spawns: agent_spawns.map(&:to_h),
+        segments: segments(main).map(&:to_h),
+        missed_delegation_candidates:
+          MissedDelegationScan.new(orchestrator_calls, @run_threshold).candidates
+      }
+    end
+
+    def parse_lines
+      out = []
+      File.foreach(@path).with_index(1) do |raw, i|
+        parsed = JSON.parse(raw) rescue next
+        out << Line.new(i, parsed)
+      end
+      out
+    end
+
+    def collect_calls(main)
+      calls = {}
+      spawns = []
+      main.each do |line|
+        line.tool_uses.each do |block|
+          call = ToolCall.new(line, block)
+          calls[call.id] = call
+          spawns << AgentSpawn.new(call, spawns.size + 1) if call.name == "Agent"
+        end
+      end
+      [calls, spawns]
+    end
+
+    def attach_results(main, calls)
+      main.each do |line|
+        line.tool_results.each do |block|
+          call = calls[block["tool_use_id"]] or next
+          call.result_chars = result_size(block, line)
+        end
+      end
+    end
+
+    # Async Agent spawns: launch stub carries agentId + resolvedModel in
+    # toolUseResult; the real result arrives later as a <task-notification>.
+    def attach_agent_metadata(main, calls, spawns)
+      by_call_id = spawns.to_h { |s| [s.call.id, s] }
+      by_agent_id = {}
+      main.each do |line|
+        line.tool_results.each do |block|
+          spawn = by_call_id[block["tool_use_id"]] or next
+          meta = line.raw["toolUseResult"]
+          next unless meta.is_a?(Hash)
+          spawn.agent_id = meta["agentId"]
+          spawn.resolved_model = meta["resolvedModel"]
+          by_agent_id[spawn.agent_id] = spawn if spawn.agent_id
+        end
+      end
+      main.each do |line|
+        note = line.task_notification or next
+        spawn = by_agent_id[note[:task_id]] or next
+        spawn.notification_chars = [spawn.notification_chars, note[:chars]].max
+      end
+    end
+
+    def result_size(block, line)
+      content = block["content"]
+      size = case content
+             when String then content.length
+             when Array then content.sum { |b| b.is_a?(Hash) ? b["text"].to_s.length : b.to_s.length }
+             else content.to_s.length
+             end
+      size.zero? ? line.raw["toolUseResult"].to_s.length : size
+    end
+
+    def metadata(main)
+      assistants = main.select { |l| l.type == "assistant" }
+      stamps = main.map(&:timestamp).compact
+      {
+        cwd: main.map(&:cwd).compact.first,
+        started: stamps.min,
+        ended: stamps.max,
+        assistant_turns: assistants.size,
+        human_prompts: main.count(&:human_prompt?),
+        main_loop_models: assistants.map(&:model).compact.tally,
+        jsonl_lines: main.last&.number
+      }
+    end
+
+    def segments(main)
+      segs = [Segment.new("pre-spawn-1")]
+      spawn_count = 0
+      main.each do |line|
+        line.tool_uses.each do |block|
+          call = ToolCall.new(line, block)
+          if call.name == "Agent"
+            spawn_count += 1
+            segs << Segment.new("after-spawn-#{spawn_count}")
+          else
+            segs.last.add(call)
+          end
+        end
+      end
+      resize_segments(segs, main)
+    end
+
+    # Re-attach result sizes to the segment copies (segments built fresh above).
+    def resize_segments(segs, main)
+      sizes = {}
+      main.each do |line|
+        line.tool_results.each { |b| sizes[b["tool_use_id"]] = result_size(b, line) }
+      end
+      segs.each do |seg|
+        seg.instance_variable_get(:@calls).each { |c| c.result_chars = sizes[c.id].to_i }
+      end
+      segs
+    end
+  end
+end
+
+if $PROGRAM_NAME == __FILE__
+  options = { json: false, run_threshold: 4 }
+  OptionParser.new do |o|
+    o.banner = "Usage: extract [--json] [--run-threshold N] SESSION.jsonl [...]"
+    o.on("--json") { options[:json] = true }
+    o.on("--run-threshold N", Integer) { |n| options[:run_threshold] = n }
+  end.parse!
+
+  abort "No transcript paths given" if ARGV.empty?
+  ARGV.each do |path|
+    sheet = OrchAudit::FactSheet.new(path, run_threshold: options[:run_threshold])
+    puts options[:json] ? sheet.to_json_s : sheet.to_markdown
+    puts
+  end
+end