cc-os/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/design.md

Design: calibrate-assessment-inventory

Context

Wayfinder map #49 produced a fully locked design, recorded in plugins/os-doc-hygiene/lifecycle-spec.md (§1 extract index, §2 nominations memory + canonical ordering, §8 protocol steps 3/6 + test coverage) and the 2026-07-15 amendments to ADR-0038/0039, committed as 9f11df3. This change implements it. There are no open design questions — where this document states a choice, it is restating the locked decision and its rationale, not making a new one.

Current state: :calibrate persists only confirmed rules; keep verdicts, human declines, and open consults evaporate with the run's scratch dir and are re-derived every pass. calibrate_helpers.py has helper functions but no intake filter and no single serialization path for .dochygiene-rules.json. The :clean extract-then-delete op writes to the vault but leaves no in-repo pointer.

Goals / Non-Goals

Goals:

• Persist every settled judge verdict: answers as plain rules (keep included), "no" as nominations.rejected, "not yet" as nominations.consults.
• Deterministically block exact-repeat re-nominations and hand the judge its history (related rejections + open consults) as context.
• One canonical writer for .dochygiene-rules.json so every write is ordering-stable and reviewable.
• Leave a discoverable pointer (extracted.md) when an extraction physically leaves the repo for the vault.

Non-Goals:

• No changes to rulebook.py semantics — it stays nomination-unaware (contract: "which rule governs this path"). Only calibrate_helpers reads the nominations key.
• No consult resurfacing in :check/:clean (ADR-0039 boundary — calibrate only).
• No automated revisit of rejections or singleton keep rules — both exit by hand-deletion only.
• No new eval harness for the judge-context / consult-loop behavior (#59) — worked examples + production IRL audits instead.
• No post-write hook for ordering enforcement (rejected in #53).

Decisions

All decided on map #49; restated with rationale:

1. Keep verdicts are plain lifetime: keep rules, not a separate store (#51). Matched = managed removes them from the calibrate pool and the glob protects future files for free. Corollary: the class-never-path test is relaxed for the keep tier only — a singleton keep merely protects, so exact-path globs are allowed there and remain forbidden for temporary/delete-once-served.
2. nominations holds only the two non-rule residues (#52): rejected ("the answer is no") and consults ("no answer yet"). It never touches the file filter. Entry schemas per lifecycle-spec §2; consults deliberately carry no lifetime; presence in consults = open, no status field.
3. Rejections block exact glob+lifetime repeats only (#52). Variants flow to the judge annotated with the related rejection — memory, not veto. Relatedness is deterministic: the two globs' match sets intersect on the current shortlist.
4. The intake filter is deterministic code, not a model step (#52, invariant #6): NominationIntakeFilter in calibrate_helpers.py, run as pipeline Step 3.5 between haiku nomination and the judge; drops are logged in the run summary.
5. Canonical ordering is writer-enforced, no hook (#53). Grouping: rules by tier (delete-once-served, temporary, keep), glob-sorted within group; nominations after rules; consults before rejected (pending -action queue reads first), each glob-sorted. Idempotent; hand edits re-canonicalize on the next write. Unknown fields warn and round-trip (never dropped).
6. Consult exits (#56): (a) human answer settles purpose → normal rule persisted, consult entry deleted; (b) not rule-worthy → rewritten into rejected with rejected_by: "human"; (c) defer → stays, resurfaces next run. New rejections/consults appear in the Step 5 report but are not individually gated — they are memory, not deletion authority.
7. extracted.md append is owned by the extract op (#57): distill → /os-vault:write → append pointer line → git rm, all inside the same per-file transaction and single hygiene commit — no window where the doc is gone but undiscoverable. Vault extractions only; repo-durable residue (ADR/CLAUDE.md) gets no index line. Self-protection is the already-shipped global **/extracted.md → keep rule.
8. Test coverage (#59): unit tests only, for the two deterministic seams (intake filter; writer). The two model-facing behaviors get worked examples (already landed in judge.md / SKILL.md with the design commit) and IRL audits as the next signal.

Implementation shape

• scripts/calibrate_helpers.py: add NominationIntakeFilter (small single-responsibility class, injected shortlist/rules-file data, returns (survivors_with_annotations, dropped_log)) and a RulesFileWriter (or extend the existing write path) that owns all serialization of .dochygiene-rules.json, including nominations read/validate/warn.
• skills/calibrate/SKILL.md + workflow files: wire Step 3.5 (run the filter, feed its annotations + open consults into the judge prompt's existing "Nominations memory" section), Step 5 ("Open consults" report section, consult exits a/b/c), Step 6 (persist keep verdicts / rejections / consults through the canonical writer, dedupe consults by glob at write time). The SKILL.md consult-loop worked example loses its "design-level pending wiring" marker.
• scripts/patch_applier.py: extend the extract-then-delete path — when the extraction destination is the vault, append the pointer entry (format per lifecycle-spec §1) to <dir>/extracted.md (create if absent) and stage it in the same hygiene commit; on extraction failure, neither append nor delete applies (existing skip semantics).
• tests/: extend test_calibrate_helpers.py (or add test_nomination_intake.py / writer tests) covering exact-repeat drops, the two #52 worked relatedness cases, round-trip canonical ordering, and unknown-field warnings; patch-applier tests cover the append-then-delete atomicity and the no-append-on-repo-durable-extraction case.

Risks / Trade-offs

• [Nominations grow without bound — stale rejections accumulate] → accepted by design: exits are hand-deletion only (removals stay HITL); the canonical ordering keeps the file reviewable, and entries are small.
• [Judge treats a related rejection as a veto and under-nominates] → the judge.md "Nominations memory" section states variants-are-not-vetoes with a worked example; IRL session audits are the detection path (#59), and IRL evidence triggers harness design per eval discipline.
• [Writer round-trips unknown fields but a hand-edit introduces malformed JSON] → existing skip-and-warn loader discipline applies; the writer only runs on an already-parsed structure.
• [extracted.md append fails after a successful vault write] → the per-file transaction skips the delete too, so the doc is never gone-but-unindexed; the vault note is orphaned-but-harmless and the next run retries.

Migration Plan

No data migration: nominations is additive to schema_version 1 and the v1 loader ignores unknown top-level keys. Existing .dochygiene-rules.json files re-canonicalize on their first post-change write. Rollback is code-only (revert the change; nominations keys left in project files are inert). After implementation: full test suite green (407 existing tests + new), then bin/refresh-plugins.

Open Questions

None — the design is locked (map #49); ambiguities found during implementation go back to lifecycle-spec.md as the source of truth.