os-orchestration: WS4 IRL re-audit + Tier 1/2 wording patch

First production IRL audit (19 transcripts post-2026-07-08, 3 parallel
auditors; run artifacts in autoresearch/improve-260710-1057/,
gitignored). Solved: model-param discipline, over-delegation. Patched
into ORCHESTRATION.md: settled-design mid-session fan-out trigger,
never-sleep-poll rule, own eval/benchmark loops named as mechanical,
unfamiliar-tree Explore trigger. SendMessage example skipped (no
violations; token budget). One-shot patch — WS4 grids contaminated;
acceptance = next IRL audit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XknQRvihHDpYE47RTmUR4N
This commit is contained in:
jared 2026-07-10 11:24:59 -04:00
parent 7dae595526
commit 242ca84744
2 changed files with 28 additions and 6 deletions

View File

@ -256,6 +256,19 @@ into `~/.claude/plugins/os-vault`
`autoresearch/classic-260708-1039/`; vault note `autoresearch/classic-260708-1039/`; vault note
[[os-orchestration-ws4-econ-wording-results]]. IRL re-audit of ~5 real Fable sessions [[os-orchestration-ws4-econ-wording-results]]. IRL re-audit of ~5 real Fable sessions
scheduled ~2026-07-22. scheduled ~2026-07-22.
- **First IRL re-audit DONE (2026-07-10, ahead of schedule):** 19 production transcripts
post-2026-07-08 audited by 3 parallel agents against the shipped rules (run:
`autoresearch/improve-260710-1057/`, plan in `improvement-plan.md`). Solved surfaces:
model-param discipline (~48/49 spawns pinned), over-delegation guardrail. Tier 1 findings
→ ORCHESTRATION.md wording patch shipped same day (grid validation impossible —
run-set/reserve contaminated — so this is a one-shot production patch, not a wording
loop): (1) settled-design mid-session fan-out trigger, (2) never-sleep-poll rule, (3) own
eval/benchmark loops named as mechanical sequences, (4) unfamiliar-tree ~5-call Explore
trigger. SendMessage worked example deliberately skipped (no violations found;
injected-token budget). Acceptance check = next IRL audit (~23 weeks; baseline:
settled-design drift 8/19 sessions). Side finding needing separate repro: harness ignored
explicit `model: "sonnet"` pins on general-purpose agents (env override unset — distinct
from WS1 cause); the self-report line caught it.
**Global os-status plugin** — `cc-os/plugins/os-status/` (git-tracked, 2026-07-06); **Global os-status plugin** — `cc-os/plugins/os-status/` (git-tracked, 2026-07-06);
symlinked into `~/.claude/plugins/os-status` symlinked into `~/.claude/plugins/os-status`

View File

@ -13,10 +13,15 @@
spec, or plan (write-N-files fan-out is delegated work — the design decisions are spec, or plan (write-N-files fan-out is delegated work — the design decisions are
already made); an investigation spans many files or needs a large/isolated already made); an investigation spans many files or needs a large/isolated
context (long log review, wide grep-and-synthesize); or a mechanical multi-call context (long log review, wide grep-and-synthesize); or a mechanical multi-call
sequence (edits, lookups, conversions) needs no judgment between steps. This sequence (edits, lookups, conversions) needs no judgment between steps — this
holds for the whole session: after one round of spawns completes, the next includes your own eval/benchmark/extraction work: repeated near-identical
eligible chunk of work is delegated too — don't drift back into long direct runs parsers, headless grid runs, and CLI syntax hunts are mechanical sequences too;
mid-session. script once or delegate. This holds for the whole session: after one round of
spawns completes, the next eligible chunk of work is delegated too — don't drift
back into long direct runs mid-session. WHEN a design/decision settles
mid-session, stop before the first direct Write/Edit that implements it — the
resulting fan-out is a delegation event, even if the session started with
disciplined spawns.
- Work directly when: the op is single-file or ≤2 tool calls; steps are genuinely - Work directly when: the op is single-file or ≤2 tool calls; steps are genuinely
judgment-dependent (each result changes what you do next); the user is in the judgment-dependent (each result changes what you do next); the user is in the
loop every few turns (interactive troubleshooting — delegation overhead exceeds loop every few turns (interactive troubleshooting — delegation overhead exceeds
@ -30,7 +35,9 @@
new spawn re-pays the per-agent system-prompt tax. new spawn re-pays the per-agent system-prompt tax.
- Delegate async and keep working: launch independent subagents in the background - Delegate async and keep working: launch independent subagents in the background
and continue your own thread while they run; intervene only when a result shows and continue your own thread while they run; intervene only when a result shows
an agent off track. an agent off track. Never sleep-poll a background job from the main loop —
background it and rely on completion notification (or Monitor); a `sleep` loop is
main-loop idle time billed at the top rate.
- Before every `Agent` call → set `model:` explicitly in that call. An omitted - Before every `Agent` call → set `model:` explicitly in that call. An omitted
`model` silently bills the subagent at the main-loop model. Mechanical `model` silently bills the subagent at the main-loop model. Mechanical
file-edit/shell work → `haiku`; anything requiring judgment → `sonnet`; genuinely file-edit/shell work → `haiku`; anything requiring judgment → `sonnet`; genuinely
@ -45,6 +52,8 @@
already read goes into the subagent prompt as a stated fact or summary, not as an already read goes into the subagent prompt as a stated fact or summary, not as an
instruction to read it again. If an investigation will span many files, delegate instruction to read it again. If an investigation will span many files, delegate
it before reading them yourself — a short orienting Read is fine only when the it before reading them yourself — a short orienting Read is fine only when the
target file/path is uncertain. target file/path is uncertain. In an unfamiliar codebase, more than ~5 orienting
grep/read calls means the orientation itself is the delegable task — send it to
an Explore agent.
- Where a call exposes an effort dial (Workflow `agent()` opts), set it per stage: - Where a call exposes an effort dial (Workflow `agent()` opts), set it per stage:
mechanical stages `effort: low`; hard verify/judge stages `high`/`xhigh`. mechanical stages `effort: low`; hard verify/judge stages `high`/`xhigh`.