os-orchestration: WS4 IRL re-audit + Tier 1/2 wording patch
First production IRL audit (19 transcripts post-2026-07-08, 3 parallel auditors; run artifacts in autoresearch/improve-260710-1057/, gitignored). Solved: model-param discipline, over-delegation. Patched into ORCHESTRATION.md: settled-design mid-session fan-out trigger, never-sleep-poll rule, own eval/benchmark loops named as mechanical, unfamiliar-tree Explore trigger. SendMessage example skipped (no violations; token budget). One-shot patch — WS4 grids contaminated; acceptance = next IRL audit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XknQRvihHDpYE47RTmUR4N
This commit is contained in:
parent
7dae595526
commit
242ca84744
|
|
@ -256,6 +256,19 @@ into `~/.claude/plugins/os-vault`
|
||||||
`autoresearch/classic-260708-1039/`; vault note
|
`autoresearch/classic-260708-1039/`; vault note
|
||||||
[[os-orchestration-ws4-econ-wording-results]]. IRL re-audit of ~5 real Fable sessions
|
[[os-orchestration-ws4-econ-wording-results]]. IRL re-audit of ~5 real Fable sessions
|
||||||
scheduled ~2026-07-22.
|
scheduled ~2026-07-22.
|
||||||
|
- **First IRL re-audit DONE (2026-07-10, ahead of schedule):** 19 production transcripts
|
||||||
|
post-2026-07-08 audited by 3 parallel agents against the shipped rules (run:
|
||||||
|
`autoresearch/improve-260710-1057/`, plan in `improvement-plan.md`). Solved surfaces:
|
||||||
|
model-param discipline (~48/49 spawns pinned), over-delegation guardrail. Tier 1 findings
|
||||||
|
→ ORCHESTRATION.md wording patch shipped same day (grid validation impossible —
|
||||||
|
run-set/reserve contaminated — so this is a one-shot production patch, not a wording
|
||||||
|
loop): (1) settled-design mid-session fan-out trigger, (2) never-sleep-poll rule, (3) own
|
||||||
|
eval/benchmark loops named as mechanical sequences, (4) unfamiliar-tree ~5-call Explore
|
||||||
|
trigger. SendMessage worked example deliberately skipped (no violations found;
|
||||||
|
injected-token budget). Acceptance check = next IRL audit (~2–3 weeks; baseline:
|
||||||
|
settled-design drift 8/19 sessions). Side finding needing separate repro: harness ignored
|
||||||
|
explicit `model: "sonnet"` pins on general-purpose agents (env override unset — distinct
|
||||||
|
from WS1 cause); the self-report line caught it.
|
||||||
|
|
||||||
**Global os-status plugin** — `cc-os/plugins/os-status/` (git-tracked, 2026-07-06);
|
**Global os-status plugin** — `cc-os/plugins/os-status/` (git-tracked, 2026-07-06);
|
||||||
symlinked into `~/.claude/plugins/os-status`
|
symlinked into `~/.claude/plugins/os-status`
|
||||||
|
|
|
||||||
|
|
@ -13,10 +13,15 @@
|
||||||
spec, or plan (write-N-files fan-out is delegated work — the design decisions are
|
spec, or plan (write-N-files fan-out is delegated work — the design decisions are
|
||||||
already made); an investigation spans many files or needs a large/isolated
|
already made); an investigation spans many files or needs a large/isolated
|
||||||
context (long log review, wide grep-and-synthesize); or a mechanical multi-call
|
context (long log review, wide grep-and-synthesize); or a mechanical multi-call
|
||||||
sequence (edits, lookups, conversions) needs no judgment between steps. This
|
sequence (edits, lookups, conversions) needs no judgment between steps — this
|
||||||
holds for the whole session: after one round of spawns completes, the next
|
includes your own eval/benchmark/extraction work: repeated near-identical
|
||||||
eligible chunk of work is delegated too — don't drift back into long direct runs
|
parsers, headless grid runs, and CLI syntax hunts are mechanical sequences too;
|
||||||
mid-session.
|
script once or delegate. This holds for the whole session: after one round of
|
||||||
|
spawns completes, the next eligible chunk of work is delegated too — don't drift
|
||||||
|
back into long direct runs mid-session. WHEN a design/decision settles
|
||||||
|
mid-session, stop before the first direct Write/Edit that implements it — the
|
||||||
|
resulting fan-out is a delegation event, even if the session started with
|
||||||
|
disciplined spawns.
|
||||||
- Work directly when: the op is single-file or ≤2 tool calls; steps are genuinely
|
- Work directly when: the op is single-file or ≤2 tool calls; steps are genuinely
|
||||||
judgment-dependent (each result changes what you do next); the user is in the
|
judgment-dependent (each result changes what you do next); the user is in the
|
||||||
loop every few turns (interactive troubleshooting — delegation overhead exceeds
|
loop every few turns (interactive troubleshooting — delegation overhead exceeds
|
||||||
|
|
@ -30,7 +35,9 @@
|
||||||
new spawn re-pays the per-agent system-prompt tax.
|
new spawn re-pays the per-agent system-prompt tax.
|
||||||
- Delegate async and keep working: launch independent subagents in the background
|
- Delegate async and keep working: launch independent subagents in the background
|
||||||
and continue your own thread while they run; intervene only when a result shows
|
and continue your own thread while they run; intervene only when a result shows
|
||||||
an agent off track.
|
an agent off track. Never sleep-poll a background job from the main loop —
|
||||||
|
background it and rely on completion notification (or Monitor); a `sleep` loop is
|
||||||
|
main-loop idle time billed at the top rate.
|
||||||
- Before every `Agent` call → set `model:` explicitly in that call. An omitted
|
- Before every `Agent` call → set `model:` explicitly in that call. An omitted
|
||||||
`model` silently bills the subagent at the main-loop model. Mechanical
|
`model` silently bills the subagent at the main-loop model. Mechanical
|
||||||
file-edit/shell work → `haiku`; anything requiring judgment → `sonnet`; genuinely
|
file-edit/shell work → `haiku`; anything requiring judgment → `sonnet`; genuinely
|
||||||
|
|
@ -45,6 +52,8 @@
|
||||||
already read goes into the subagent prompt as a stated fact or summary, not as an
|
already read goes into the subagent prompt as a stated fact or summary, not as an
|
||||||
instruction to read it again. If an investigation will span many files, delegate
|
instruction to read it again. If an investigation will span many files, delegate
|
||||||
it before reading them yourself — a short orienting Read is fine only when the
|
it before reading them yourself — a short orienting Read is fine only when the
|
||||||
target file/path is uncertain.
|
target file/path is uncertain. In an unfamiliar codebase, more than ~5 orienting
|
||||||
|
grep/read calls means the orientation itself is the delegable task — send it to
|
||||||
|
an Explore agent.
|
||||||
- Where a call exposes an effort dial (Workflow `agent()` opts), set it per stage:
|
- Where a call exposes an effort dial (Workflow `agent()` opts), set it per stage:
|
||||||
mechanical stages `effort: low`; hard verify/judge stages `high`/`xhigh`.
|
mechanical stages `effort: low`; hard verify/judge stages `high`/`xhigh`.
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue