cc-os

Commit Graph

Author	SHA1	Message	Date
jared	ab258d5ba1	os-orchestration eval: key E3P root-cause concepts by scenario id The reserve twin E3P-retry-storm targets gen-logs' other planted incident (2026-07-02 auth-token-expiry 401 storm) and was authored before axis B existed; the single concept table would have failed a compliant model. Fixed pre-reserve-measurement without reading reserve files (incident chain comes from gen-logs). Run-set scoring byte-identical; self-test 34/34 incl. keying proof both directions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 11:05:03 -04:00
jared	b226601bea	os-orchestration: WS4 eval extension — econ axes, E5 batching pair, fable column, pre-registered E2P/E3P criterion redesign - extract: main-loop/sidechain output-token accounting (subagent transcripts persist at <projects>/<flat>/<session-id>/subagents/agent-*.jsonl — the mini-audit's instrumentation gap, now solved) + SendMessage round counting - check: econ info on every row (mltok/sctok/mlshare/prebytes/spawns/maxrounds); E2P v2 (scripted-direct branch + services task-complete); E3P v2 (74KB S7 whole-session ingestion anchor + mechanical root-cause concepts, delegation no longer required); E5P/E5N arms (grouped-delegation vs parallel-fanout, ## Expected files coverage convention) - run: fable → claude-fable-5 alias - scenarios: E5P-handler-rollout + E5N-limit-rebalance (+ frozen reserve twins, authored unread by the orchestrating session) + handler-spec/probe-spec fixture docs - self-test 32/32 model-free; ORCHESTRATION.v3-draft.md parked (NOT shipped — Step-3 baseline must run against v2); H3 threshold pre-registered in plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 09:49:44 -04:00
jared	fe83fdb290	os-orchestration: E1-E3 eval harness + cluster-1 wording v2 (self-report) Harness: 6 run-set + 6 frozen-reserve paired scenarios from the WS1 verified misses, Node relaystation fixture (deterministic sandbox-time logs), headless runner with CLAUDE_CODE_SUBAGENT_MODEL stub for E1P, deterministic-first checker driving audit/bin/extract (extended with per-segment read targets), narrow frozen haiku-judge fallback for E1 language axes, model-free self-test 21/21 incl. shipped-instruction-compliant conformance transcripts. Wording v2: the E1 canary (counted) proved resolvedModel is not model-visible in the launch result, so the launch-stub-comparison rule was unactionable; replaced with subagent model self-report on sonnet/opus spawns (subagents know their exact model ID - verified live). Scenarios/checker/rubric untouched. Deterministic env-override detection routed to WS3 (plan updated). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-06 18:04:15 -04:00

Author

SHA1

Message

Date

jared

ab258d5ba1

os-orchestration eval: key E3P root-cause concepts by scenario id

The reserve twin E3P-retry-storm targets gen-logs' other planted incident
(2026-07-02 auth-token-expiry 401 storm) and was authored before axis B
existed; the single concept table would have failed a compliant model.
Fixed pre-reserve-measurement without reading reserve files (incident chain
comes from gen-logs). Run-set scoring byte-identical; self-test 34/34 incl.
keying proof both directions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-08 11:05:03 -04:00

jared

b226601bea

os-orchestration: WS4 eval extension — econ axes, E5 batching pair, fable column, pre-registered E2P/E3P criterion redesign

- extract: main-loop/sidechain output-token accounting (subagent transcripts
  persist at <projects>/<flat>/<session-id>/subagents/agent-*.jsonl — the
  mini-audit's instrumentation gap, now solved) + SendMessage round counting
- check: econ info on every row (mltok/sctok/mlshare/prebytes/spawns/maxrounds);
  E2P v2 (scripted-direct branch + services task-complete); E3P v2 (74KB S7
  whole-session ingestion anchor + mechanical root-cause concepts, delegation
  no longer required); E5P/E5N arms (grouped-delegation vs parallel-fanout,
  ## Expected files coverage convention)
- run: fable → claude-fable-5 alias
- scenarios: E5P-handler-rollout + E5N-limit-rebalance (+ frozen reserve twins,
  authored unread by the orchestrating session) + handler-spec/probe-spec fixture docs
- self-test 32/32 model-free; ORCHESTRATION.v3-draft.md parked (NOT shipped —
  Step-3 baseline must run against v2); H3 threshold pre-registered in plan

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-08 09:49:44 -04:00

jared

fe83fdb290

os-orchestration: E1-E3 eval harness + cluster-1 wording v2 (self-report)

Harness: 6 run-set + 6 frozen-reserve paired scenarios from the WS1 verified
misses, Node relaystation fixture (deterministic sandbox-time logs), headless
runner with CLAUDE_CODE_SUBAGENT_MODEL stub for E1P, deterministic-first
checker driving audit/bin/extract (extended with per-segment read targets),
narrow frozen haiku-judge fallback for E1 language axes, model-free self-test
21/21 incl. shipped-instruction-compliant conformance transcripts.

Wording v2: the E1 canary (counted) proved resolvedModel is not model-visible
in the launch result, so the launch-stub-comparison rule was unactionable;
replaced with subagent model self-report on sonnet/opus spawns (subagents know
their exact model ID - verified live). Scenarios/checker/rubric untouched.
Deterministic env-override detection routed to WS3 (plan updated).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-06 18:04:15 -04:00

3 Commits