title: [Evaluation name, e.g. "os-adr Eval B (unprompted write-trigger & retrieval)"]
summary: [What was measured and the key finding — 1-2 sentences, e.g. "Haiku never self-triggers the ADR system without explicit prompting; sonnet passes 5/8 with two near-miss failures."]
tags:
- type/eval-results
- domain/llm-evaluation
- tool/[tool-under-test] # the skill/plugin/feature being measured
- project/[project] # project that contains the eval harness
scope: global
last_updated: YYYY-MM-DD
date: YYYY-MM-DD # creation date — set once, never updated
related:
- [note-slug] # cross-links to follow-up evals, methodology notes, or design docs
source: [project name] # project that contains the eval harness
---
# [Evaluation Title]
## Results Grid/Threshold
<!-- The raw measurement: grid table with pass/fail per cell, OR hypothesis→verdict mapping, or threshold scores. Include rep counts per cell and evaluation date. Include a brief summary line above the table/mapping explaining what passing looks like. -->
**Passing criterion:** [Define what PASS means for this eval — e.g., all scenarios across both tiers, or per-tier threshold, or axis-level aggregate.]
| [Column] | [Column] |
|---|---|
| [Cell] | [Cell] |
**Summary:** [1–2 sentences recapping which tiers/cells passed/failed and the baseline numbers. This is what the reader scans first to know whether results are in the expected ballpark.]
## Measurement Setup
**Hypotheses / Scenarios Tested**
<!-- If hypothesis-driven: name each hypothesis and its test scenarios. If scenario-driven: name each scenario group (e.g., W1–W3 for write-triggers, R1–R4 for retrieval). One paragraph or brief list per hypothesis/group, clarifying what correctness looks like. -->
**Fixture and Sampling**
<!-- Which fixture(s) were used (real project vs synthetic, language/domain, size, ADR/decision history if relevant). Fixture generalization risk. Reps per cell. Rationale for rep count (e.g., 1 rep/cell for baseline discovery, 3 reps for wording-loop stability). -->
**Experimental Control (Frozen Surfaces)**
<!-- What was held constant during this eval and why. For skill-wording loops: checker, fixtures, scenarios, rubric frozen — only wording moves. For harness-design evals: [list what was frozen]. Explicitly name anything intentionally NOT frozen, and why (e.g., model tiers vary to establish tier-specific performance; rep count chosen for discovery vs confirmation). -->
## Validity and Limitations
**How to Interpret These Results**
<!-- Training-set vs held-out framing — if this eval was optimized against (wording loop, rubric tuning), state that explicitly. What the numbers do and do not support (e.g., "8/8 on a training-set grid means wording direction is sound, not that the behavior generalizes"). Confidence caveats: variance at low reps, fixture-specific behavior, scenarios-only measurement (not live observability). When to read a result as significant (e.g., majority of reps, control cells hold). When to read a result as noise (e.g., single-rep flips between cells, below the rep threshold for stability). -->
**Weaknesses of This Eval (Its Ladder Level)**
<!-- What this eval can't see or didn't test. Examples: single fixture generalization (would a second fixture in a different language / domain change the results?), 1 rep/cell variance (high noise floor), no ambiguity axis (scenarios are clear-cue vs no clear boundary-case testing), limited distractor count, no longer held-out (wording was tuned against this grid — it's now training-set), model-specific failure modes, ablation surfaces that were never tested (e.g., "channel ablation not run — don't assume hook redundancy"). Open questions the eval can't resolve. Open questions the eval raises. -->
<!-- 2–5 bullets MAX. Each bullet is a rule someone designing a DIFFERENT eval could apply, phrased as a generalizable statement with a one-line why. NOT a recap of this eval's results — only things that transfer (process mistakes to avoid, harness-design gotchas, phrasing mechanisms that worked). If a lesson is recurring across evals, it belongs in a methodology note, not here — see the promotion prompt below. -->
- [Generalizable rule — why it matters]
**Promotion check:** <!-- Required before finishing this note: does any lesson above belong in a methodology note ([[eval-methodology-ladder]], [[running-autoresearch-skill-evals]], or a new one)? If yes, update that note NOW and wikilink it here; the lesson stays here as a bullet pointing at the methodology note. Eval-results notes are episodic records — methodology notes are where compounding knowledge accretes. State the outcome: "promoted X to [[note]]" or "nothing recurring yet." -->
<!-- Explicit criterion under which these results justify deployment, adoption, or real-world rollout. Examples: "Sonnet 8/8 passes for pilot rollout on projects with Rust / Go codebases; haiku pending 3-rep confirmation on the W3 edge case (running today, 2026-07-06)." Or: "5/8 passes the 'prompting issue, not capability gap' threshold; spin up a wording loop before production rollout." Tier-specific status. Conditions that would change the call. -->
<!-- If these results motivate changing anything (skill wording, hook text, CLAUDE.md sections, harness criteria), write the CONCRETE candidate text here — the actual proposed wording in clear language, not a description of it. This is the handoff artifact to the tuning loop or follow-up session: a reader (human or AI) should see exactly what is proposed to change and why, and later diff it against what actually shipped. Tag each candidate with its target surface (which file/section) and which measured gap it addresses. If nothing is proposed, say "none." -->
<!-- If there is a follow-up eval harness (e.g., Eval C at a higher ladder level), point to it. Otherwise, point to the ladder-approach methodology note explaining the progression. One paragraph or brief list of what gets tested next and why (e.g., "Eval C will add ambiguity-ladder discrimination scenarios to test whether the model mistakes clear-cue for ambiguous-cue; if it does, wording tuning stops and the feature is capability-limited"). -->
## Related
<!-- Wikilinks to: prior baselines or follow-up evals, the methodology note(s) (ladder approach, autoresearch procedure), the skills/design docs being evaluated, and hub notes if any. -->
