jared
/
cc-os
1
0
Fork 0
Code Issues 18 Pull Requests Packages Projects Releases Wiki Activity
cc-os/openspec/specs
History
jared 00e22f1676 Add os-adr evaluation harnesses (Eval A and Eval B) 
Build comprehensive evaluation framework for the os-adr plugin (ADR-021).

Eval A: prompted skill-execution evaluation across haiku/sonnet models.
Six ADR lifecycle scenarios, deterministic Ruby checker, sandbox runner,
/autoresearch loop for wording optimization.

Eval B: held-out unprompted-behavior evaluation. Seven scenarios
(W1–W3 write-trigger, R1–R4 retrieval), webhook fixture with 6-ADR history
(Superseded pair + distractors), headless runner (isolated SessionStart
context), two-axis deterministic-first checker (consultation + citation
accuracy, AI judge fallback for new-file writes).

Both harnesses self-tested model-free. Grid runs deferred per locked
rollout order. Updated docs/specs to freeze scenario shapes and document
methodologies.
 		2026-07-03 16:44:42 -04:00
..
adr-authoring Build os-adr plugin: ADR authoring, session hook, migration, retrieval 2026-07-03 14:24:19 -04:00
adr-eval-b-harness Add os-adr evaluation harnesses (Eval A and Eval B) 2026-07-03 16:44:42 -04:00
adr-migration Build os-adr plugin: ADR authoring, session hook, migration, retrieval 2026-07-03 14:24:19 -04:00
adr-plugin-packaging Build os-adr plugin: ADR authoring, session hook, migration, retrieval 2026-07-03 14:24:19 -04:00
adr-retrieval Build os-adr plugin: ADR authoring, session hook, migration, retrieval 2026-07-03 14:24:19 -04:00
adr-session-awareness Build os-adr plugin: ADR authoring, session hook, migration, retrieval 2026-07-03 14:24:19 -04:00
hook-python-port Source memory plugin in git; port hooks to Python; archive change 2026-06-12 12:42:05 -04:00
incremental-migration Add incremental-migration + reference-benchmark change (ADR-013) 2026-06-04 11:17:32 -04:00
local-model-selection Archive graphify-ollama-setup and sync new main specs 2026-06-05 11:48:02 -04:00
memory-plugin-hooks Source memory plugin in git; port hooks to Python; archive change 2026-06-12 12:42:05 -04:00
memory-plugin-skills Complete memory plugin implementation — hooks, skills, and graph-connectivity findings 2026-06-05 16:39:49 -04:00
memsearch-episodic Archive memsearch-episodic-layer change 2026-06-09 14:35:50 -04:00
memsearch-sync Source memory plugin in git; port hooks to Python; archive change 2026-06-12 12:42:05 -04:00
plugin-source Source memory plugin in git; port hooks to Python; archive change 2026-06-12 12:42:05 -04:00
project-graph-lifecycle Source memory plugin in git; port hooks to Python; archive change 2026-06-12 12:42:05 -04:00
reference-extraction-benchmark Add incremental-migration + reference-benchmark change (ADR-013) 2026-06-04 11:17:32 -04:00
vault-graph-build Archive graphify-ollama-setup and sync new main specs 2026-06-05 11:48:02 -04:00