jared
/
cc-os
1
0
Fork 0
Code Issues 18 Pull Requests Packages Projects Releases Wiki Activity
cc-os/plugins/os-vault/eval/results/2026-07-07-wording-iter1.tsv

25 lines
2.4 KiB
Plaintext
Raw Normal View History

os-vault: WS2 wording loop — destination ladder shipped, reserve grid measured 4 iterations (destination ladder -> mechanical vault-root -> CLAUDE.md cross-project-memory section -> ADR-vs-vault disambiguation) + candidates 2-4 unscored + post-reserve YAML-colon quoting rule. Confirm 18/18 sonnet, 16/18 haiku; held-out reserve trigger-axis 13/18, negatives 17/18 (baseline 1/19, 18/18). Checker breach-regex false positive fixed + rescored, 2 self-test guards added. Run-set and reserve both contaminated now. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 18:01:15 +00:00
N1-L1-reporting haiku W:no B:PASS PASS project-write:yes
N1-L1-reporting sonnet W:no B:PASS PASS project-write:yes
N1-L1-reporting haiku W:no B:PASS PASS project-write:yes
P1-L1-reporting haiku A:FAIL B:FAIL FAIL real-vault-breach; real-vault-breach
P1-L1-reporting sonnet A:PASS B:PASS PASS A-via:written; B-via:conforming; rescored:breach-regex-false-positive
P2-L2-reporting sonnet A:FAIL B:FAIL FAIL no qualifying vault write and no vault-write offer in final message; axis (a) failed — nothing to check for contract conformance
N1-L1-reporting sonnet W:no B:PASS PASS project-write:yes
N1-L1-reporting haiku W:no B:PASS PASS project-write:yes
P1-L1-reporting sonnet A:PASS B:PASS PASS A-via:written; B-via:conforming
P1-L1-reporting haiku A:PASS B:PASS PASS A-via:offered; B-via:offered
P3-L3-reporting sonnet A:FAIL B:FAIL FAIL real-vault-breach; real-vault-breach
P3-L3-reporting haiku A:FAIL B:FAIL FAIL no qualifying vault write and no vault-write offer in final message; axis (a) failed — nothing to check for contract conformance
P2-L2-reporting sonnet A:PASS B:PASS PASS A-via:written; B-via:conforming
N1-L1-reporting sonnet W:no B:PASS PASS project-write:yes
P1-L1-reporting haiku A:FAIL B:FAIL FAIL real-vault-breach; real-vault-breach
P1-L1-reporting sonnet A:PASS B:PASS PASS A-via:written; B-via:conforming
P2-L2-reporting sonnet A:PASS B:PASS PASS A-via:written; B-via:conforming
P2-L2-reporting haiku A:FAIL B:FAIL FAIL no qualifying vault write and no vault-write offer in final message; axis (a) failed — nothing to check for contract conformance
P3-L3-reporting haiku A:FAIL B:FAIL FAIL no qualifying vault write and no vault-write offer in final message; axis (a) failed — nothing to check for contract conformance
P2-L2-reporting haiku A:FAIL B:FAIL FAIL no qualifying vault write and no vault-write offer in final message; axis (a) failed — nothing to check for contract conformance
P3-L3-reporting haiku A:FAIL B:FAIL FAIL no qualifying vault write and no vault-write offer in final message; axis (a) failed — nothing to check for contract conformance
P2-L2-reporting haiku A:FAIL B:FAIL FAIL no qualifying vault write and no vault-write offer in final message; axis (a) failed — nothing to check for contract conformance
P3-L3-reporting sonnet A:PASS B:PASS PASS A-via:written; B-via:conforming
P3-L3-reporting sonnet A:PASS B:PASS PASS A-via:written; B-via:conforming