cc-os/plugins/os-adr/eval/fixture/project/docs/adr
jared 00e22f1676 Add os-adr evaluation harnesses (Eval A and Eval B)
Build comprehensive evaluation framework for the os-adr plugin (ADR-021).

Eval A: prompted skill-execution evaluation across haiku/sonnet models.
Six ADR lifecycle scenarios, deterministic Ruby checker, sandbox runner,
/autoresearch loop for wording optimization.

Eval B: held-out unprompted-behavior evaluation. Seven scenarios
(W1–W3 write-trigger, R1–R4 retrieval), webhook fixture with 6-ADR history
(Superseded pair + distractors), headless runner (isolated SessionStart
context), two-axis deterministic-first checker (consultation + citation
accuracy, AI judge fallback for new-file writes).

Both harnesses self-tested model-free. Grid runs deferred per locked
rollout order. Updated docs/specs to freeze scenario shapes and document
methodologies.
2026-07-03 16:44:42 -04:00
..
0001-use-sqlite-for-local-persistence.md Add os-adr evaluation harnesses (Eval A and Eval B) 2026-07-03 16:44:42 -04:00
0002-use-json-for-config-exports.md Add os-adr evaluation harnesses (Eval A and Eval B) 2026-07-03 16:44:42 -04:00
0003-use-yaml-for-config-exports.md Add os-adr evaluation harnesses (Eval A and Eval B) 2026-07-03 16:44:42 -04:00
0004-retry-outbound-sync-with-exponential-backoff.md Add os-adr evaluation harnesses (Eval A and Eval B) 2026-07-03 16:44:42 -04:00
README.md Add os-adr evaluation harnesses (Eval A and Eval B) 2026-07-03 16:44:42 -04:00

README.md

Architecture Decision Records

One file per decision, NNNN-kebab-title.md, created via /os-adr:create.

ID Title Status Date
0001 Use SQLite for local persistence Accepted 2026-05-01
0002 Use JSON for config exports Superseded 2026-05-10
0003 Use YAML for config exports Accepted 2026-05-20
0004 Retry outbound sync with exponential backoff Accepted 2026-06-02