cc-os/docs/memory-system
jared 00e22f1676 Add os-adr evaluation harnesses (Eval A and Eval B)
Build comprehensive evaluation framework for the os-adr plugin (ADR-021).

Eval A: prompted skill-execution evaluation across haiku/sonnet models.
Six ADR lifecycle scenarios, deterministic Ruby checker, sandbox runner,
/autoresearch loop for wording optimization.

Eval B: held-out unprompted-behavior evaluation. Seven scenarios
(W1–W3 write-trigger, R1–R4 retrieval), webhook fixture with 6-ADR history
(Superseded pair + distractors), headless runner (isolated SessionStart
context), two-axis deterministic-first checker (consultation + citation
accuracy, AI judge fallback for new-file writes).

Both harnesses self-tested model-free. Grid runs deferred per locked
rollout order. Updated docs/specs to freeze scenario shapes and document
methodologies.
2026-07-03 16:44:42 -04:00
..
benchmark Complete graphify-ollama-setup OpenSpec change 2026-06-05 11:44:40 -04:00
00-README.md Remove handoff doc and pivot to Graphify knowledge layer 2026-06-04 07:57:39 -04:00
01-video-synthesis.md Remove handoff doc and pivot to Graphify knowledge layer 2026-06-04 07:57:39 -04:00
02-system-design.md Update design docs and add implementation runbook per ADR-011/012 2026-06-04 09:35:43 -04:00
03-architecture-decisions.md Add os-adr evaluation harnesses (Eval A and Eval B) 2026-07-03 16:44:42 -04:00
04-build-plan.md Rename memory plugin to os-vault; fix marketplace registration 2026-07-03 09:43:18 -04:00
05-implementation-process.md Rename memory plugin to os-vault; fix marketplace registration 2026-07-03 09:43:18 -04:00
06-graphify-evaluation.md Remove handoff doc and pivot to Graphify knowledge layer 2026-06-04 07:57:39 -04:00
07-graph-connectivity-findings.md Complete memory plugin implementation — hooks, skills, and graph-connectivity findings 2026-06-05 16:39:49 -04:00
08-sb-active-use-plan.md Rename memory plugin to os-vault; fix marketplace registration 2026-07-03 09:43:18 -04:00
09-sb-content-plan.md Rename memory plugin to os-vault; fix marketplace registration 2026-07-03 09:43:18 -04:00
10-sb-findability-plan.md Rename memory plugin to os-vault; fix marketplace registration 2026-07-03 09:43:18 -04:00