cc-os/docs/orchestration-audit/2026-07-08-fable-econ-miniaudit.md

# Fable-5 main-loop orchestration economics — mini-audit (2026-07-08)

**Model**: this audit was performed by `claude-sonnet-5`. All audited transcripts carry
`"model":"claude-fable-5"` on non-sidechain assistant messages (i.e. Fable 5 is the main-loop
model in every session below).

## Method

1. Scanned all `~/.claude/projects/*/*.jsonl` files modified in the last 10 days
   (`scan.rb`), filtered to sessions with ≥20 non-sidechain assistant turns and at least one
   `claude-fable-5` main-loop message, excluded the live session
   (`ca17e6d2-6132-4681-8a02-8a7523cb5660`). 21 candidates found, almost all in `cc-os`; picked
   6 for project diversity: 3 `cc-os`, 2 `servers`, 1 `servers/desktop`, all ≥20 turns (24–255).
2. Ran the existing fact-sheet extractor
   (`plugins/os-orchestration/audit/bin/extract`) on each for spawn count/models,
   pre-/post-spawn tool segmentation, and its heuristic same-tool-run flags.
3. Wrote two throwaway Ruby scripts in scratchpad (`tokens.rb`, `bashcmds.rb`,
   `firstprompt.rb`) to sum `usage.output_tokens` per session (main-loop vs sidechain), track
   max/final context size, and find the longest unbroken run of main-loop tool calls with no
   `Agent` spawn in between, then classified each run by reading the actual tool inputs
   (Bash commands / file targets) in that range.

**Environment gotcha (noted for future audits):** in zsh, a variable named `path` is linked to
`$PATH`; `path=$(find ...)` silently corrupted `$PATH` mid-script and caused unrelated `ruby: not
found` (exit 127) failures. Renamed to `tpath`. Not a data issue, just a shell trap.

**Hard measurement limitation:** none of the 6 transcripts contain any `"isSidechain":true`
lines, even in the two sessions that used the `Agent` tool (9f45afcc, 5 spawns; df12b180, 1
spawn). Subagent execution in this environment/version is logged only as a `tool_use`/
`toolUseResult` pair (with `agentId`/`resolvedModel` metadata) plus an eventual
`<task-notification>` result — no subagent transcript is persisted into the parent `.jsonl`. So
**subagent output-token spend is not recoverable from these files at all.** The only proxy
available is the char length of the `tool_result`/notification payload returned to the main
loop, converted at ~4 chars/token. All "% main-loop" figures below use that proxy where spawns
occurred, and are explicitly not real subagent token counts (real subagent spend is very likely
much higher than the proxy, since it only reflects what was returned, not what the subagent
itself burned getting there).

> **Addendum (2026-07-08, WS4 Step 2):** the limitation above is narrower than stated —
> subagent spend is absent from the *parent* transcript, but each subagent's full transcript
> (with per-message `usage` blocks) persists separately at
> `~/.claude/projects/<flattened-cwd>/<session-id>/subagents/agent-<id>.jsonl` (verified for
> both interactive and headless sessions). Real main-loop-share measurement is therefore
> possible; the eval harness A-econ axis uses it. The proxy figures below were NOT recomputed.
> See vault note [[claude-code-subagent-transcripts-and-token-accounting]].

## Per-session detail

### 1. `9f45afcc` — cc-os — cache-refresh tooling + WS2 autoresearch prep
- 255 turns, 25 human prompts, 850 jsonl lines, 2026-07-04 16:17–18:52
- 5 Agent spawns, all `general-purpose`, `model` param `sonnet`×4/`haiku`×1, **resolved model
  `claude-haiku-4-5` in all 5 cases** (param mismatch: 4 of 5 requested sonnet, all resolved
  haiku)
- main-loop output tokens: **237,679** (255 msgs); proxy subagent output: 13,575 chars via
  result/notification payloads (≈3.4K tokens) → **main-loop ≈98.6% of proxy-measured output**
- pre-spawn segment: 19 calls / 23,658 bytes read (Bash 14, Read 4, Edit 1)
- longest direct run: **48 calls** (after spawn 3: ToolSearch 2, SendMessage 1, Bash 22, Edit 12,
  Skill 2, Read 5, Write 3, TaskStop 1) — mixed: driving/monitoring the background eval-grid
  agent (SendMessage/TaskStop/ToolSearch) interleaved with direct mechanical plugin-cache-refresh
  edits. Not a clean single-purpose run.

### 2. `eba26343` — cc-os — WS2 os-vault wording loop (`/autoresearch`)
- 223 turns, 2 human prompts (127 total `type:user` lines incl. tool results/notifications), 583
  jsonl lines, 2026-07-07 16:48–18:01
- **0 Agent spawns**
- main-loop output tokens: **216,019** (223 msgs) — 100% of measured output (no spawns to split
  against)
- whole-session tool profile: 119 calls / 119,587 bytes (Read 17, Bash 64, Skill 1, ToolSearch 1,
  TaskCreate 5, TaskUpdate 9, Edit 20, Write 2)
- longest direct run: **119 calls = the entire session** (no spawn boundary at all). Sampled the
  64 Bash commands directly: this is a scripted headless-runner loop — `bin/run P1/P2/P3...` /
  `bin/check` invocations against `eval/scenarios/`, `sleep N; wc -l results.tsv`
  polling of background task output, `awk`/`sort` aggregation of `results.tsv`, and git
  commit/push at each iteration boundary. **Classify as justified scripted-bulk-op / polling
  work**, not a delegation miss — matches the ORCHESTRATION.md scripted-bulk-edit carve-out; a
  subagent adds no value polling a script's own stdout.

### 3. `5f53e0c0` — cc-os — `/opsx:apply add-os-adr-plugin`
- 222 turns, 4 human prompts, 506 jsonl lines, 2026-07-03 17:15–17:43
- **0 Agent spawns**
- main-loop output tokens: **314,899** (222 msgs, highest of the 6) — 100% of measured output
- whole-session tool profile: 128 calls / 129,924 bytes (Bash 39, Read 16, Write 35, Edit 38)
- longest direct run: **128 calls = the entire session**. Contains three flagged sub-runs from
  the extractor: Read×10 (design docs, lines 27–52 — reasonable orientation), **Write×13 (lines
  96–137, 13 distinct new files: `lib/adr.rb`, `lib/adr/record.rb`, `repository.rb`,
  `index.rb`, `template.rb`, `bin/adr-new`, …)**, and Edit×10 (lines 326–353, fixing 5 files
  post-review). **Genuine miss candidate**: the Write×13 run is scaffolding independent Ruby
  class files against an *already-fixed* `design.md` (read in the first 10 calls) — a
  parallelizable implementation task that ran entirely in the main loop and is a large share of
  this session's 315K output tokens.

### 4. `3fc7bb8c` — servers — backup verification + vault write
- 90 turns, 5 human prompts, 236 jsonl lines, 2026-07-04 17:20–18:04
- **0 Agent spawns**
- main-loop output tokens: **63,607**
- whole-session tool profile: 44 calls / 32,250 bytes (Read 3, Bash 27, Write 4, ToolSearch 2,
  ExitPlanMode 1, Edit 6, Skill 1)
- longest direct run: **44 calls = whole session** — interactive backup-system investigation
  ("Are we confident backups will work as expected now?") ending in a vault write. Judgment-
  heavy, sequential-dependent (each Bash result determines the next check). **Justified direct
  work**, no clean miss shape.

### 5. `7bc0dda4` — servers — Proxmox HAOS install troubleshooting
- 82 turns, 15 human prompts (highest interaction density of the 6), 249 jsonl lines,
  2026-07-03 19:49–20:42
- **0 Agent spawns**
- main-loop output tokens: **39,513**
- whole-session tool profile: 39 calls / 6,124 bytes (Bash 27, Edit 8, Read 2, Skill 1, Write 1)
- longest direct run: **39 calls = whole session** — live interactive install/debug loop (disk
  space checks, image download, phase-2 error triage against user-supplied screenshots).
  **Justified direct work**: 15 human turns in 82 total means the human is in the loop almost
  every other turn; delegation overhead would exceed any savings.

### 6. `df12b180` — servers/desktop — Remote Control OAuth scope troubleshooting
- 24 turns, 6 human prompts, 90 jsonl lines, 2026-07-06 23:13–23:35
- 1 Agent spawn (`claude-code-guide`, `model` param `sonnet`, resolved `claude-sonnet-5`,
  foreground, prompt 1,008 chars → result 5,689 chars)
- main-loop output tokens: **24,051**; proxy subagent output ≈1,422 tokens (5,689 chars/4) →
  **main-loop ≈94.4% of proxy-measured output**
- pre-spawn: 4 Bash calls / 3,154 bytes (credential/scope investigation)
- post-spawn: 3 calls (Read 1, Write 2 — writing up the answer)
- longest direct run: 4 calls (the pre-spawn investigation). **Good delegation example**: a
  bounded doc-lookup ("Remote Control usage") was correctly routed to the docs-specialist
  subagent instead of the main loop reading source/docs itself.

## Summary table

| session | project | turns | spawns | main-loop out tok | % main-loop (proxy, see caveat) | pre-spawn calls/bytes | longest direct run | classification |
|---|---|---|---|---|---|---|---|---|
| 9f45afcc | cc-os | 255 | 5 | 237,679 | ~98.6% | 19 / 23,658 | 48 | mixed: agent-monitoring + mechanical cache-refresh edits |
| eba26343 | cc-os | 223 | 0 | 216,019 | 100% | 119 / 119,587 (whole session) | 119 (whole session) | scripted headless-eval polling loop — justified |
| 5f53e0c0 | cc-os | 222 | 0 | 314,899 | 100% | 128 / 129,924 (whole session) | 128 (whole session) | OpenSpec apply — Write×13 independent files = **miss candidate** |
| 3fc7bb8c | servers | 90 | 0 | 63,607 | 100% | 44 / 32,250 (whole session) | 44 (whole session) | interactive investigation — justified |
| 7bc0dda4 | servers | 82 | 0 | 39,513 | 100% | 39 / 6,124 (whole session) | 39 (whole session) | interactive troubleshooting (15 human turns) — justified |
| df12b180 | servers/desktop | 24 | 1 | 24,051 | ~94.4% | 4 / 3,154 | 4 | pre-spawn investigation, correct delegation of doc lookup |

## Caveats

- 4 of the 6 sessions are from `cc-os` (matches the actual population: 19 of 21 candidate
  sessions in the last 10 days were `cc-os`) — the sample is not representative of all-client
  usage, just of what exists in the last 10 days of transcripts.
- "% main-loop" is a **proxy** built only from what subagents returned to the main loop
  (tool_result/notification char count ÷4), not measured subagent token spend — real subagent
  cost is invisible in every transcript examined (0/6 have `isSidechain:true` lines). Any
  economics conclusion about spawn cost-effectiveness needs a different data source (e.g. Task
  output files under `tasks/`, if retained) — not attempted here.
- "Longest direct run" for 4 of 6 sessions equals the *entire session* because zero spawns
  occurred; this collapses "pre-spawn segment" and "longest run" into the same number for those
  rows — noted in the table rather than hidden.
- Miss/justified classification is a single-auditor read of tool-call sequences, not a rubric
  score; treat as a directional signal for the eval plan, not a verdict.

## Miss-shape summary

- **The only clean miss candidate found is `5f53e0c0`'s Write×13 run**: scaffolding independent
  Ruby class files against an already-fixed design doc, done entirely in the (most expensive)
  main loop across a 222-turn, 314,899-output-token session with zero spawns.
- **Scripted/polling loops are not misses even when very long** (`eba26343`, 119 consecutive
  direct calls): driving `bin/run`/`sleep`/`wc -l results.tsv` from the main loop is the correct
  shape per the existing scripted-bulk-op carve-out — a subagent adds no value polling a script's
  own output.
- **Interactive/judgment-heavy sessions correctly stayed direct** (`3fc7bb8c`, `7bc0dda4`): both
  have no spawns and no flagged miss runs; `7bc0dda4`'s 15 human prompts in 82 turns make
  delegation overhead a net loss.
- **Subagent economics are structurally unmeasurable from transcripts as configured**: 0 of 6
  sessions logged sidechain entries even where Agent spawned; only a lossy char-count proxy
  exists via task-notification payloads. This is itself an eval-plan-relevant finding — any eval
  claiming to measure delegated-vs-direct token spend needs an instrumentation change (capture
  Task output, or read `tasks/*.output` files) before it can trust a token-economics number.
- **Where spawns did happen, the main loop still did substantial direct work afterward**
  (`9f45afcc`: 43–48-call runs after spawns 3 and 5) rather than delegating a second time —
  worth checking in future audits whether that's justified (synthesis/judgment) or another miss
  shape.
- **No over-delegation observed anywhere** — zero ≤2-call trivial ops routed to a subagent in any
  of the 6 sessions.