os-orchestration: WS4 IRL re-audit + Tier 1/2 wording patch

First production IRL audit (19 transcripts post-2026-07-08, 3 parallel
auditors; run artifacts in autoresearch/improve-260710-1057/,
gitignored). Solved: model-param discipline, over-delegation. Patched
into ORCHESTRATION.md: settled-design mid-session fan-out trigger,
never-sleep-poll rule, own eval/benchmark loops named as mechanical,
unfamiliar-tree Explore trigger. SendMessage example skipped (no
violations; token budget). One-shot patch — WS4 grids contaminated;
acceptance = next IRL audit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XknQRvihHDpYE47RTmUR4N
This commit is contained in:
jared 2026-07-10 11:24:59 -04:00
parent 7dae595526
commit 242ca84744
2 changed files with 28 additions and 6 deletions

View File

@ -256,6 +256,19 @@ into `~/.claude/plugins/os-vault`
`autoresearch/classic-260708-1039/`; vault note
[[os-orchestration-ws4-econ-wording-results]]. IRL re-audit of ~5 real Fable sessions
scheduled ~2026-07-22.
- **First IRL re-audit DONE (2026-07-10, ahead of schedule):** 19 production transcripts
post-2026-07-08 audited by 3 parallel agents against the shipped rules (run:
`autoresearch/improve-260710-1057/`, plan in `improvement-plan.md`). Solved surfaces:
model-param discipline (~48/49 spawns pinned), over-delegation guardrail. Tier 1 findings
→ ORCHESTRATION.md wording patch shipped same day (grid validation impossible —
run-set/reserve contaminated — so this is a one-shot production patch, not a wording
loop): (1) settled-design mid-session fan-out trigger, (2) never-sleep-poll rule, (3) own
eval/benchmark loops named as mechanical sequences, (4) unfamiliar-tree ~5-call Explore
trigger. SendMessage worked example deliberately skipped (no violations found;
injected-token budget). Acceptance check = next IRL audit (~23 weeks; baseline:
settled-design drift 8/19 sessions). Side finding needing separate repro: harness ignored
explicit `model: "sonnet"` pins on general-purpose agents (env override unset — distinct
from WS1 cause); the self-report line caught it.
**Global os-status plugin** — `cc-os/plugins/os-status/` (git-tracked, 2026-07-06);
symlinked into `~/.claude/plugins/os-status`

View File

@ -13,10 +13,15 @@
spec, or plan (write-N-files fan-out is delegated work — the design decisions are
already made); an investigation spans many files or needs a large/isolated
context (long log review, wide grep-and-synthesize); or a mechanical multi-call
sequence (edits, lookups, conversions) needs no judgment between steps. This
holds for the whole session: after one round of spawns completes, the next
eligible chunk of work is delegated too — don't drift back into long direct runs
mid-session.
sequence (edits, lookups, conversions) needs no judgment between steps — this
includes your own eval/benchmark/extraction work: repeated near-identical
parsers, headless grid runs, and CLI syntax hunts are mechanical sequences too;
script once or delegate. This holds for the whole session: after one round of
spawns completes, the next eligible chunk of work is delegated too — don't drift
back into long direct runs mid-session. WHEN a design/decision settles
mid-session, stop before the first direct Write/Edit that implements it — the
resulting fan-out is a delegation event, even if the session started with
disciplined spawns.
- Work directly when: the op is single-file or ≤2 tool calls; steps are genuinely
judgment-dependent (each result changes what you do next); the user is in the
loop every few turns (interactive troubleshooting — delegation overhead exceeds
@ -30,7 +35,9 @@
new spawn re-pays the per-agent system-prompt tax.
- Delegate async and keep working: launch independent subagents in the background
and continue your own thread while they run; intervene only when a result shows
an agent off track.
an agent off track. Never sleep-poll a background job from the main loop —
background it and rely on completion notification (or Monitor); a `sleep` loop is
main-loop idle time billed at the top rate.
- Before every `Agent` call → set `model:` explicitly in that call. An omitted
`model` silently bills the subagent at the main-loop model. Mechanical
file-edit/shell work → `haiku`; anything requiring judgment → `sonnet`; genuinely
@ -45,6 +52,8 @@
already read goes into the subagent prompt as a stated fact or summary, not as an
instruction to read it again. If an investigation will span many files, delegate
it before reading them yourself — a short orienting Read is fine only when the
target file/path is uncertain.
target file/path is uncertain. In an unfamiliar codebase, more than ~5 orienting
grep/read calls means the orientation itself is the delegable task — send it to
an Explore agent.
- Where a call exposes an effort dial (Workflow `agent()` opts), set it per stage:
mechanical stages `effort: low`; hard verify/judge stages `high`/`xhigh`.