jared
/
cc-os
1
0
Fork 0
Code Issues 18 Pull Requests Packages Projects Releases Wiki Activity
48 Commits 2 Branches 0 Tags 3.1 MiB
Commit Graph

5 Commits

Author SHA1 Message Date
jared c645faae59 Fix skill namespacing (drop name: frontmatter), add refresh-plugins, wire eval howto 
- Remove explicit name: frontmatter from os-adr and os-doc-hygiene SKILL.mds
  so slash commands register namespaced (/os-adr:find, not bare /find)
- Add bin/refresh-plugins to refresh stale local plugin caches
- CLAUDE.md: cache-refresh procedure, naming-convention pointer, mandate
  reading the vault autoresearch-eval howto before the next eval

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-04 14:01:58 -04:00
jared 5b399d5614 Record Eval A and Eval B grid results in status docs 
Eval A grid ran clean across all scenarios/tiers. Eval B grid (1 rep/cell)
shows a real gap: haiku never unprompted-consults the ADR system in any
scenario, while sonnet passes 5/8 (misses W3 write-trigger and R1 direct-
conflict retrieval). Updates CLAUDE.md component status and the eval-b
README so the next session picks up from here instead of re-running blind.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-07-03 17:36:37 -04:00
jared 00e22f1676 Add os-adr evaluation harnesses (Eval A and Eval B) 
Build comprehensive evaluation framework for the os-adr plugin (ADR-021).

Eval A: prompted skill-execution evaluation across haiku/sonnet models.
Six ADR lifecycle scenarios, deterministic Ruby checker, sandbox runner,
/autoresearch loop for wording optimization.

Eval B: held-out unprompted-behavior evaluation. Seven scenarios
(W1–W3 write-trigger, R1–R4 retrieval), webhook fixture with 6-ADR history
(Superseded pair + distractors), headless runner (isolated SessionStart
context), two-axis deterministic-first checker (consultation + citation
accuracy, AI judge fallback for new-file writes).

Both harnesses self-tested model-free. Grid runs deferred per locked
rollout order. Updated docs/specs to freeze scenario shapes and document
methodologies.
 2026-07-03 16:44:42 -04:00
jared 23a7efa6b3 Remove committed __pycache__ and gitignore Python bytecode 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-03 14:24:26 -04:00
jared 36abd43107 Build os-adr plugin: ADR authoring, session hook, migration, retrieval 
Implements OpenSpec change add-os-adr-plugin (archived as
2026-07-03-add-os-adr-plugin; delta specs synced to openspec/specs/).

- plugins/os-adr/: Ruby OO core (Record, Repository, Index, Template,
  Detector, Migrator, MigrationReport, Finder) behind thin bin/ CLIs;
  Python SessionStart hook via plugin-relative hooks.json; four skills
  (create, init, migrate, find); invariants.md; 47 Ruby + 15 Python
  model-free tests, all passing.
- Migration pilot passed (threshold 25%): viking 0%, delta-refinery 0%,
  llf-schema 8.3%. Retrieval verified against the real llf-schema graph;
  resolved that ADRs need no Graphify tag/edge convention (layer 3
  expands query paths via node source_files).
- docs/adr-system/: requirements (04), PRD (05), eval sketches (06).
- ADR-020 recorded; CLAUDE.md Implemented Components updated with the
  locked rollout order (pilot projects -> cc-os retrofit -> wider).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-03 14:24:19 -04:00