os-orchestration: baseline grid results + scripted-bulk-edit carve-out

Grid (36 reps + 2 canaries, all counted): negatives 18/18 both tiers, zero
over-triggering; positives 1/18 on delegate-at-all, but transcript
verification splits those FAILs into superior direct strategies (scripted
bulk edit, surgical greps - the audit's own justified-non-delegation shape)
vs genuine misses (haiku whole-file ingestion, per-file grind). Conditional
rules validated where engaged: 14/14 spawns explicit-model, self-report
downgrade flag worked. E2P/E3P criterion redesign pre-registered with
independently-anchored thresholds; reserve untouched. Analysis in vault:
os-orchestration-eval-baseline-grid-results.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

CLAUDE.md

@@ -89,6 +89,7 @@ to those two and fix the stale doc.
 - Session audit (WS1, 2026-07-06): 10 stratified real sessions audited via `audit/bin/extract` (Ruby fact-sheet extractor, dual-use as future eval checker) + parallel sonnet auditors + verified synthesis. Findings + E1–E4 eval-scenario backlog: `docs/orchestration-audit/2026-07-06-findings.md`. Headlines: model param was silently overridden to haiku on all 23 verified spawns — root cause found post-audit: `CLAUDE_CODE_SUBAGENT_MODEL=haiku` in `~/.claude/settings.json` env block (set by an earlier session as a cost measure; removed 2026-07-06; the Fable-5 correlation was coincidental timing). Omitted model param inherits the main-loop model (opus/sonnet) — misses cluster in pre-rollout and ops sessions; no over-sharing/full-dump failures anywhere.
 - Wording fixes shipped (2026-07-06, post-audit): ORCHESTRATION.md rewritten with trigger-conditioned when→then phrasing for the three verified clusters — explicit `model:` before every Agent call (cluster 2; a real cost exposure once the env override was removed), subagent model self-report on sonnet/opus spawns (cluster 1 — the E1 canary proved `resolvedModel` is NOT model-visible in the launch result; self-report is the only observable channel, and subagents do know their exact model ID), don't re-cover your own ground before delegating (cluster 3). Delegation thresholds unchanged. Cache refreshed via `bin/refresh-plugins`. Deterministic env-override detection (warn on `CLAUDE_CODE_SUBAGENT_MODEL`) is delegated to WS3's status-check design.
 - Eval harness E1–E3 (2026-07-06): `plugins/os-orchestration/eval/` — scripted headless orchestration-behavior eval scored by the audit extractor (extended with per-segment read targets). 6 run-set scenarios (E1P/N downgrade detection with `CLAUDE_CODE_SUBAGENT_MODEL=haiku` runner stub; E2P/N explicit-model fan-out vs no-delegation-on-trivial; E3P/N delegate-investigation vs orienting-reads-correct) + 6 frozen reserve twins; new Node.js "relaystation" fixture with deterministic sandbox-time log generation; deterministic-first checker (`eval/bin/check`, narrow frozen haiku-judge fallback for E1 language axes only, stubbable via `ORCH_EVAL_JUDGE_CMD`); model-free `bin/self-test` 21/21 green incl. shipped-instruction-compliant conformance transcripts. FAIL axes map only to verified audit misses; tier choice informational. **Scenario Task blocks are held-out — never run informally.** Procedure: `plugins/os-orchestration/eval/README.md`.
+- Baseline grid run (2026-07-06, run-set × sonnet/haiku × 3 reps + 2 counted canaries, all counted, zero harness errors): negatives 18/18 PASS both tiers (zero over-delegation, zero false downgrade claims); positives 1/18 on the delegate-at-all axis — but transcript verification shows most positive FAILs are *superior direct strategies* (E2P sonnet: one scripted bulk edit; E3P sonnet: surgical greps over 400KB logs), which the WS1 audit itself scored as justified non-delegation, vs genuine miss shapes (E3P haiku: whole 112KB logs into own context; E2P haiku: 20-call per-file grind). Conditional rules validated where engaged: 14/14 spawns explicit-model; the one delegating E1P rep flagged the stubbed downgrade via self-report. Criterion redesign for E2P/E3P A-axes is pre-registered (anchor thresholds independently, NOT to this grid) before any rescore/tuning; reserve stays frozen. ORCHESTRATION.md gained a scripted-bulk-edit carve-out (a uniform multi-file change one script covers is direct work). TSV: `eval/results/2026-07-06-baseline-grid.tsv`; full analysis: vault note [[os-orchestration-eval-baseline-grid-results]].
 
 **Global os-doc-hygiene plugin** — `cc-os/plugins/os-doc-hygiene/` (git-tracked, 2026-07-03); symlinked into `~/.claude/plugins/os-doc-hygiene`
 - Hooks: `hooks/hooks.json` → SessionStart hook (matcher: startup|resume) runs `scripts/reminder.py` via `${CLAUDE_PLUGIN_ROOT}` (5s timeout), emitting a deterministic zero-token reminder banner

plugins/os-orchestration/ORCHESTRATION.md

@@ -3,6 +3,8 @@
 - Do single-file, ≤2-tool-call ops directly. Don't delegate them. Delegate only when
   work is parallelizable across independent files/subtasks, spans many files, or
   needs a large/isolated context (long log review, wide grep-and-synthesize).
+  A uniform multi-file change that one scripted command covers is direct work too —
+  don't spawn agents for what a loop or script does in one coordinated operation.
 - Before every `Agent` call → set `model:` explicitly in that call. An omitted
   `model` silently bills the subagent at the main-loop model. Mechanical
   file-edit/shell work → `haiku`; anything requiring judgment → `sonnet`; genuinely

plugins/os-orchestration/eval/results/2026-07-06-baseline-grid.tsv

@@ -0,0 +1,39 @@
+E1N-pipeline-review	haiku	1	PASS	B:pass-mechanical	tiers=sonnet=3;spawns=3
+E1N-pipeline-review	haiku	2	PASS	B:pass-mechanical	tiers=sonnet=2;spawns=2
+E1N-pipeline-review	haiku	3	PASS	B:pass-judge	tiers=sonnet=3;spawns=3
+E1N-pipeline-review	sonnet	1	PASS	B:pass-mechanical	tiers=;spawns=0
+E1N-pipeline-review	sonnet	2	PASS	B:pass-mechanical	tiers=;spawns=0
+E1N-pipeline-review	sonnet	3	PASS	B:pass-mechanical	tiers=;spawns=0
+E1P-pipeline-review	haiku	1	FAIL	A:no-delegation	
+E1P-pipeline-review	haiku	2	FAIL	A:no-delegation	
+E1P-pipeline-review	haiku	3	FAIL	A:no-delegation	
+E1P-pipeline-review	sonnet	1	FAIL	A:no-delegation	
+E1P-pipeline-review	sonnet	2	PASS	A:pass;B:pass-mechanical	tiers=sonnet=6;mismatched=6
+E1P-pipeline-review	sonnet	3	FAIL	A:no-delegation	
+E2N-readme-typo	haiku	1	PASS	A:no-spawn	pre_bytes=1201
+E2N-readme-typo	haiku	2	PASS	A:no-spawn	pre_bytes=1201
+E2N-readme-typo	haiku	3	PASS	A:no-spawn	pre_bytes=1201
+E2N-readme-typo	sonnet	1	PASS	A:no-spawn	pre_bytes=483
+E2N-readme-typo	sonnet	2	PASS	A:no-spawn	pre_bytes=334
+E2N-readme-typo	sonnet	3	PASS	A:no-spawn	pre_bytes=398
+E2P-retry-policy	haiku	1	FAIL	A:no-delegation	
+E2P-retry-policy	haiku	2	FAIL	A:no-delegation	
+E2P-retry-policy	haiku	3	FAIL	A:no-delegation	
+E2P-retry-policy	sonnet	1	FAIL	A:no-delegation	
+E2P-retry-policy	sonnet	2	FAIL	A:no-delegation	
+E2P-retry-policy	sonnet	3	FAIL	A:no-delegation	
+E3N-brand-spelling	haiku	1	PASS	A:no-spawn	pre_bytes=2983;edited=yes
+E3N-brand-spelling	haiku	2	PASS	A:no-spawn	pre_bytes=2990;edited=yes
+E3N-brand-spelling	haiku	3	PASS	A:no-spawn	pre_bytes=2927;edited=yes
+E3N-brand-spelling	sonnet	1	PASS	A:no-spawn	pre_bytes=1967;edited=yes
+E3N-brand-spelling	sonnet	2	PASS	A:no-spawn	pre_bytes=790;edited=yes
+E3N-brand-spelling	sonnet	3	PASS	A:no-spawn	pre_bytes=1550;edited=yes
+E3P-dropped-events	haiku	1	FAIL	A:no-delegation	
+E3P-dropped-events	haiku	2	FAIL	A:no-delegation	
+E3P-dropped-events	haiku	3	FAIL	A:no-delegation	
+E3P-dropped-events	sonnet	1	FAIL	A:no-delegation	
+E3P-dropped-events	sonnet	2	FAIL	A:no-delegation	
+E3P-dropped-events	sonnet	3	FAIL	A:no-delegation	
+# canary reps (counted; wording v1 then v2):
+E1P-pipeline-review	sonnet	1	FAIL	A:pass;B:fail-mechanical	tiers=sonnet=3;mismatched=3
+E1P-pipeline-review	sonnet	1	FAIL	A:no-delegation