# Classifier golden examples

Layer-2 reversion-protection fixtures for the **classifier** (design D7,
`openspec/changes/add-check/design.md`). Each case is a small **static** fixture
doc-tree (`input/`, stable bytes → stable sha256) paired with `expected.json`, a
full schema-valid machine report representing the expected classification of that
tree. These are **distinct** from the schema-shape fixtures one level up
(`../valid_report.json` / `../invalid_report.json`), which only exercise
`validate_report.py`.

Adding or changing a classifier golden is **human-gated** per the META-RULE in
`invariants.md`.

## Cases

| Dir | class / subtype | op_type | exact_edit.kind | tier | scanner signal |
|-----|-----------------|---------|-----------------|------|----------------|
| `1-orphaned` | stale / orphaned | deterministic | `delete-range` | confirm | `broken_reference` |
| `2-superseded` | stale / superseded | deterministic | `move-to-archive` | auto | `stale_name_location` |
| `3-completed-in-place` | stale / completed-in-place | deterministic | `insert-frontmatter` | auto | `archive_to_live_ratio` |
| `4-duplicated` | stale / duplicated | deterministic | `dedupe` | auto | `stale_name_location` (see note) |
| `5-distill` | bloat / distill | generative | (none) | confirm | `archive_to_live_ratio` |

Each case directory holds:

- `input/` — the fixture doc-tree (the bytes the scanner and the model see).
- `proposals.json` — the hand-authored model-judgment proposal(s) used to
  *generate* `expected.json` (kept so the computed fields are reproducible).
- `expected.json` — the full schema-valid machine report (the frozen expectation).
  Computed fields (`expected_sha256`, `raw_tokens`, derived `safety_tier`) are
  **real**: generated by piping a real `scanner.py` artifact + `proposals.json`
  through `report_builder.py`. `scan.project_root` is normalized to `<input>` for
  portability; `generated_at` timestamps are wall-clock and intentionally not
  asserted on.
- `notes.md` — the one taxonomy edge + scanner signal the case pins.

> **Note (case 4):** the scanner has no dedicated duplication signal. Case 4 rides
> a co-occurring `stale_name_location` ("legacy") as its deterministic trigger; the
> *duplicated* judgment itself is the model's. See `4-duplicated/notes.md`.

## Two harnesses

### Hermetic unit harness (in pytest — `tests/test_classifier_golden.py`)

Runs with **NO network and NO model**. For each case it asserts only the
deterministic / stable parts: scanner signals on the expected paths,
`validate_report.py` accepts each `expected.json`, and internal consistency of the
stable classification fields (closed enums, op_type, `exact_edit.kind`, the derived
`safety_tier` recomputed, and `expected_sha256` re-verified against the fixture
bytes). This is the regression gate that runs in CI.

### Live model-classification regression (OUTSIDE pytest — manual / agent-invoked)

This is the **model-dependent** check; it is intentionally NOT part of the unit
suite because it calls a live model. Run it manually (or have an agent run it) when
you want to confirm the classifier still reaches the expected judgment.

For each case directory:

1. Run an actual classification over `input/` — either `/os-doc-hygiene:check
   --scope <case>/input` or by invoking the `check` skill against that
   tree. This produces a fresh machine report.

   > **Confirm the invocation once the check command lands.** `--scope` is a
   > filter on a scan rooted at the resolved project root, not a re-root — to
   > classify a case in isolation you want the check *rooted at*
   > `<case>/input` (as the hermetic suite roots the scanner). Verify the exact
   > flag/rooting against the shipped `skills/check/SKILL.md` before relying on it.
2. Diff the produced classification's **stable** fields against `expected.json`,
   per entry:
   - `category.class`
   - `category.subtype`
   - `op_type`
   - the derived `safety_tier`
   - `exact_edit.kind`

   A mismatch on any of these is a **regression** — investigate before accepting.

3. **Advisory only** (flag for human review, do **not** auto-fail):
   - `op` prose wording.
   - exact anchor line numbers (`exact_edit.anchor.start_line` / `end_line`) and
     `reducible_range`.

   These are model-authored and may drift slightly without indicating a real
   classification regression; surface them for a human to eyeball rather than
   gating on them.

Because `expected.json` encodes a captured model judgment, any *intended* change to
the expected classification is a golden change → **human-gated** per the META-RULE.

## Regenerating an `expected.json` (after an approved fixture change)

```
python3 scripts/scanner.py --root examples/golden/classifier/<case>/input > /tmp/scan.json
python3 scripts/report_builder.py \
    --scan /tmp/scan.json \
    --proposals examples/golden/classifier/<case>/proposals.json \
    --root examples/golden/classifier/<case>/input \
    --out-json examples/golden/classifier/<case>/expected.json
# then normalize scan.project_root to "<input>" and re-run the hermetic suite.
```

`raw_tokens` reflect whichever token-estimator backend was active at generation
time (the zero-dependency heuristic here, since no tiktoken vocab is vendored).
The hermetic suite does **not** assert on `raw_tokens`, so regenerating on a
tiktoken-equipped machine may shift those counts harmlessly — don't mistake that
diff for a regression.