cc-os/plugins/os-doc-hygiene/openspec/changes/archive/2026-07-15-calibrate-assessment-inventory/design.md

# Design: calibrate-assessment-inventory

## Context

Wayfinder map #49 produced a fully locked design, recorded in
`plugins/os-doc-hygiene/lifecycle-spec.md` (§1 extract index, §2 nominations
memory + canonical ordering, §8 protocol steps 3/6 + test coverage) and the
2026-07-15 amendments to ADR-0038/0039, committed as `9f11df3`. This change
implements it. There are no open design questions — where this document
states a choice, it is restating the locked decision and its rationale, not
making a new one.

Current state: `:calibrate` persists only confirmed rules; keep verdicts,
human declines, and open consults evaporate with the run's scratch dir and
are re-derived every pass. `calibrate_helpers.py` has helper functions but no
intake filter and no single serialization path for `.dochygiene-rules.json`.
The `:clean` extract-then-delete op writes to the vault but leaves no in-repo
pointer.

## Goals / Non-Goals

**Goals:**

- Persist every settled judge verdict: answers as plain rules (keep included),
  "no" as `nominations.rejected`, "not yet" as `nominations.consults`.
- Deterministically block exact-repeat re-nominations and hand the judge its
  history (related rejections + open consults) as context.
- One canonical writer for `.dochygiene-rules.json` so every write is
  ordering-stable and reviewable.
- Leave a discoverable pointer (`extracted.md`) when an extraction physically
  leaves the repo for the vault.

**Non-Goals:**

- No changes to `rulebook.py` semantics — it stays nomination-unaware
  (contract: "which rule governs this path"). Only `calibrate_helpers` reads
  the `nominations` key.
- No consult resurfacing in `:check`/`:clean` (ADR-0039 boundary — calibrate
  only).
- No automated revisit of rejections or singleton keep rules — both exit by
  hand-deletion only.
- No new eval harness for the judge-context / consult-loop behavior (#59) —
  worked examples + production IRL audits instead.
- No post-write hook for ordering enforcement (rejected in #53).

## Decisions

All decided on map #49; restated with rationale:

1. **Keep verdicts are plain `lifetime: keep` rules, not a separate store**
   (#51). Matched = managed removes them from the calibrate pool and the glob
   protects future files for free. Corollary: the class-never-path test is
   relaxed for the keep tier only — a singleton keep merely protects, so
   exact-path globs are allowed there and remain forbidden for
   `temporary`/`delete-once-served`.
2. **`nominations` holds only the two non-rule residues** (#52): `rejected`
   ("the answer is no") and `consults` ("no answer yet"). It never touches the
   file filter. Entry schemas per lifecycle-spec §2; consults deliberately
   carry no lifetime; presence in `consults` = open, no status field.
3. **Rejections block exact glob+lifetime repeats only** (#52). Variants flow
   to the judge annotated with the related rejection — memory, not veto.
   Relatedness is deterministic: the two globs' match sets intersect on the
   current shortlist.
4. **The intake filter is deterministic code, not a model step** (#52,
   invariant #6): `NominationIntakeFilter` in `calibrate_helpers.py`, run as
   pipeline Step 3.5 between haiku nomination and the judge; drops are logged
   in the run summary.
5. **Canonical ordering is writer-enforced, no hook** (#53). Grouping:
   `rules` by tier (delete-once-served, temporary, keep), glob-sorted within
   group; `nominations` after `rules`; `consults` before `rejected` (pending
   -action queue reads first), each glob-sorted. Idempotent; hand edits
   re-canonicalize on the next write. Unknown fields warn and round-trip
   (never dropped).
6. **Consult exits** (#56): (a) human answer settles purpose → normal rule
   persisted, consult entry deleted; (b) not rule-worthy → rewritten into
   `rejected` with `rejected_by: "human"`; (c) defer → stays, resurfaces next
   run. New rejections/consults appear in the Step 5 report but are not
   individually gated — they are memory, not deletion authority.
7. **`extracted.md` append is owned by the extract op** (#57): distill →
   `/os-vault:write` → append pointer line → `git rm`, all inside the same
   per-file transaction and single hygiene commit — no window where the doc is
   gone but undiscoverable. Vault extractions only; repo-durable residue
   (ADR/CLAUDE.md) gets no index line. Self-protection is the already-shipped
   global `**/extracted.md → keep` rule.
8. **Test coverage** (#59): unit tests only, for the two deterministic seams
   (intake filter; writer). The two model-facing behaviors get worked examples
   (already landed in judge.md / SKILL.md with the design commit) and IRL
   audits as the next signal.

## Implementation shape

- `scripts/calibrate_helpers.py`: add `NominationIntakeFilter` (small
  single-responsibility class, injected shortlist/rules-file data, returns
  `(survivors_with_annotations, dropped_log)`) and a `RulesFileWriter` (or
  extend the existing write path) that owns *all* serialization of
  `.dochygiene-rules.json`, including nominations read/validate/warn.
- `skills/calibrate/SKILL.md` + workflow files: wire Step 3.5 (run the filter,
  feed its annotations + open consults into the judge prompt's existing
  "Nominations memory" section), Step 5 ("Open consults" report section,
  consult exits a/b/c), Step 6 (persist keep verdicts / rejections / consults
  through the canonical writer, dedupe consults by glob at write time). The
  SKILL.md consult-loop worked example loses its "design-level pending
  wiring" marker.
- `scripts/patch_applier.py`: extend the `extract-then-delete` path — when the
  extraction destination is the vault, append the pointer entry (format per
  lifecycle-spec §1) to `<dir>/extracted.md` (create if absent) and stage it
  in the same hygiene commit; on extraction failure, neither append nor
  delete applies (existing skip semantics).
- `tests/`: extend `test_calibrate_helpers.py` (or add
  `test_nomination_intake.py` / writer tests) covering exact-repeat drops,
  the two #52 worked relatedness cases, round-trip canonical ordering, and
  unknown-field warnings; patch-applier tests cover the append-then-delete
  atomicity and the no-append-on-repo-durable-extraction case.

## Risks / Trade-offs

- [Nominations grow without bound — stale rejections accumulate] → accepted
  by design: exits are hand-deletion only (removals stay HITL); the canonical
  ordering keeps the file reviewable, and entries are small.
- [Judge treats a related rejection as a veto and under-nominates] → the
  judge.md "Nominations memory" section states variants-are-not-vetoes with a
  worked example; IRL session audits are the detection path (#59), and IRL
  evidence triggers harness design per eval discipline.
- [Writer round-trips unknown fields but a hand-edit introduces malformed
  JSON] → existing skip-and-warn loader discipline applies; the writer only
  runs on an already-parsed structure.
- [extracted.md append fails after a successful vault write] → the per-file
  transaction skips the delete too, so the doc is never gone-but-unindexed;
  the vault note is orphaned-but-harmless and the next run retries.

## Migration Plan

No data migration: `nominations` is additive to `schema_version` 1 and the v1
loader ignores unknown top-level keys. Existing `.dochygiene-rules.json`
files re-canonicalize on their first post-change write. Rollback is code-only
(revert the change; `nominations` keys left in project files are inert).
After implementation: full test suite green (407 existing tests + new), then
`bin/refresh-plugins`.

## Open Questions

None — the design is locked (map #49); ambiguities found during
implementation go back to `lifecycle-spec.md` as the source of truth.