jared
/
cc-os
1
0
Fork 0
Code Issues 18 Pull Requests Packages Projects Releases Wiki Activity
cc-os/docs/memory-system/benchmark/reference-outputs
History
jared 77febc8aca Add incremental-migration + reference-benchmark change (ADR-013) 
Invert the build order to build-first / migrate-incrementally and
redesign Step 2c as a Claude reference-set benchmark.

- ADR-013 records the build-order inversion; CLAUDE.md locked-decisions
  pointer updated
- New benchmark deliverable under docs/memory-system/benchmark/:
  shared extraction spec + copy/paste dispatch prompt + reference-outputs
- Runbook (05): Step 1 = fixture selection with bulk migration deferred;
  Step 2c = Claude-tiers-only reference set, quality-only metrics, Opus
  as gold-standard rubric (produces references, not a model choice)
- 04-build-plan reconciled to the new build order and benchmark design
- Sync incremental-migration + reference-extraction-benchmark specs into
  openspec/specs/; archive the completed change

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 		2026-06-04 11:17:32 -04:00
..
README.md Add incremental-migration + reference-benchmark change (ADR-013) 2026-06-04 11:17:32 -04:00

README.md

Reference Outputs

Last updated: 2026-06-04

This directory holds the per-model Graphify-shaped extraction fragments generated by the Claude-tier reference-set benchmark (see ../dispatch-prompt.md).

File naming convention

<note-slug>.<tier>.md

Where:

  • <note-slug> is the kebab-case identifier for the fixture note (assigned in Step 1c of the build runbook)
  • <tier> is one of: haiku, sonnet, opus

Contents

Each file contains the YAML extraction fragment for one fixture note as produced by one Claude tier. Files are generated by running the dispatch prompt — they do not exist until that step is executed.

claude-opus-4-8's .opus.md files are the gold-standard rubric against which local Ollama doc-extraction models are scored in a later step.