jared
/
cc-os
1
0
Fork 0
Code Issues 18 Pull Requests Packages Projects Releases Wiki Activity
cc-os/plugins/os-adr/eval-c/fixture/project
History
jared d3be4c5ece Add os-adr Eval C: ambiguity-ladder discrimination eval 
Eval B wording reached 8/8 (sonnet) / 7/8 (haiku) on its run-set
after five iterations; subsequent W3 stability testing confirmed
intermittent flicker but acceptable performance. As Eval B is now
contaminated by iteration, we need a held-out measurement set to
assess generalization to new projects and cues.

Eval C measures whether learned behavior generalizes across
decreasing cue explicitness (explicit → moderate → conceptual
framing) and, critically, whether the model correctly avoids
false positives (over-triggering) when no Accepted ADR is in play.
Six paired scenarios (positive/negative at each level) run against
a job-execution domain fixture; a frozen reserve-set (notifications
domain) becomes the measurement set if anyone tunes wording against
the run-set.

New: `plugins/os-adr/eval-c/` — bin/ (run/check/self-test/sandbox
scripts), fixture/ (taskq async job queue, 6 ADRs, trigger-phrased
CLAUDE.md), scenarios/ (6 run-set), scenarios-reserve/ (6 reserve-
set), README.md (measurement discipline + design), judge-rubric.md.
Model-free self-test passing. First real grid run is a pending
decision.
 		2026-07-06 12:02:58 -04:00
..
docs/adr Add os-adr Eval C: ambiguity-ladder discrimination eval 2026-07-06 12:02:58 -04:00
taskq Add os-adr Eval C: ambiguity-ladder discrimination eval 2026-07-06 12:02:58 -04:00
.gitignore Add os-adr Eval C: ambiguity-ladder discrimination eval 2026-07-06 12:02:58 -04:00
.graphifyignore Add os-adr Eval C: ambiguity-ladder discrimination eval 2026-07-06 12:02:58 -04:00
CLAUDE.md Add os-adr Eval C: ambiguity-ladder discrimination eval 2026-07-06 12:02:58 -04:00
README.md Add os-adr Eval C: ambiguity-ladder discrimination eval 2026-07-06 12:02:58 -04:00

README.md

Task Queue — async job queue system

A small Python async job queue for reliable job processing.

Architecture decisions

This project records architecture decisions as ADRs in docs/adr/ (index: docs/adr/README.md).

  • When your task involves making an architecture-level choice, or changing/replacing an approach this codebase already uses (job execution, persistence, notifications) → first run /os-adr:find to check whether a recorded decision already covers it.
  • Mechanical rule: before your first edit to any existing source or config file, run /os-adr:find on the paths you are about to touch — one cheap, deterministic CLI call. This applies to additions too.
  • When you make such a choice → record it with /os-adr:create.
  • If an Accepted ADR covers the approach you are changing, you are reversing a recorded decision. A task that reverses a recorded decision is not complete until the superseding ADR is created — implementing the change and stopping leaves docs/adr/ asserting the opposite of what the code does.