diff --git a/docs/implementation-status/os-doc-hygiene.md b/docs/implementation-status/os-doc-hygiene.md index 52dc09d..7b0c539 100644 --- a/docs/implementation-status/os-doc-hygiene.md +++ b/docs/implementation-status/os-doc-hygiene.md @@ -43,3 +43,17 @@ _Leaf file of [../implementation-status.md](../implementation-status.md). Read o all 4 mandatory rows; rule persistence to `.dochygiene-rules.json` is pending human approval. Results: `plugins/os-doc-hygiene/openspec/changes/lifecycle-aware-doc-hygiene/calibration-pass-1-results.md`. Design: `plugins/os-doc-hygiene/lifecycle-spec.md`. Suite: 407 passed. +- **Calibrate assessment inventory implemented (2026-07-15, map #49 / ADR-0038-0039 + amendments)** — the locked nominations-memory design is now code: `RulesFileWriter` + (single canonical serialization path for `.dochygiene-rules.json`; tier-grouped, + glob-sorted, idempotent, unknown fields warn-and-round-trip) and `NominationIntakeFilter` + (deterministic Step 3.5 — exact glob+lifetime repeats dropped, variants annotated via + shortlist match-set intersection, open consults always forwarded) in + `calibrate_helpers.py`; calibrate SKILL.md Steps 3.5/5/6 wired (Open-consults report + section with the three exits; keep verdicts, human declines, and consults all persist + through the writer); `patch_applier.py` extract-then-delete appends the `extracted.md` + vault pointer (keyed off `extraction_target: "cross-repo"` + a skill-supplied + `extraction_pointer`, fail-closed: no pointer → no delete); clean SKILL.md Step 6.5 + supplies the pointer metadata. OpenSpec change: + `plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/`. Suite: 434 + passed (26 new unit tests across writer/intake-filter/applier seams). diff --git a/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/.openspec.yaml b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/.openspec.yaml new file mode 100644 index 0000000..4f63482 --- /dev/null +++ b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-07-15 diff --git a/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/design.md b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/design.md new file mode 100644 index 0000000..f17a8c7 --- /dev/null +++ b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/design.md @@ -0,0 +1,143 @@ +# Design: calibrate-assessment-inventory + +## Context + +Wayfinder map #49 produced a fully locked design, recorded in +`plugins/os-doc-hygiene/lifecycle-spec.md` (§1 extract index, §2 nominations +memory + canonical ordering, §8 protocol steps 3/6 + test coverage) and the +2026-07-15 amendments to ADR-0038/0039, committed as `9f11df3`. This change +implements it. There are no open design questions — where this document +states a choice, it is restating the locked decision and its rationale, not +making a new one. + +Current state: `:calibrate` persists only confirmed rules; keep verdicts, +human declines, and open consults evaporate with the run's scratch dir and +are re-derived every pass. `calibrate_helpers.py` has helper functions but no +intake filter and no single serialization path for `.dochygiene-rules.json`. +The `:clean` extract-then-delete op writes to the vault but leaves no in-repo +pointer. + +## Goals / Non-Goals + +**Goals:** + +- Persist every settled judge verdict: answers as plain rules (keep included), + "no" as `nominations.rejected`, "not yet" as `nominations.consults`. +- Deterministically block exact-repeat re-nominations and hand the judge its + history (related rejections + open consults) as context. +- One canonical writer for `.dochygiene-rules.json` so every write is + ordering-stable and reviewable. +- Leave a discoverable pointer (`extracted.md`) when an extraction physically + leaves the repo for the vault. + +**Non-Goals:** + +- No changes to `rulebook.py` semantics — it stays nomination-unaware + (contract: "which rule governs this path"). Only `calibrate_helpers` reads + the `nominations` key. +- No consult resurfacing in `:check`/`:clean` (ADR-0039 boundary — calibrate + only). +- No automated revisit of rejections or singleton keep rules — both exit by + hand-deletion only. +- No new eval harness for the judge-context / consult-loop behavior (#59) — + worked examples + production IRL audits instead. +- No post-write hook for ordering enforcement (rejected in #53). + +## Decisions + +All decided on map #49; restated with rationale: + +1. **Keep verdicts are plain `lifetime: keep` rules, not a separate store** + (#51). Matched = managed removes them from the calibrate pool and the glob + protects future files for free. Corollary: the class-never-path test is + relaxed for the keep tier only — a singleton keep merely protects, so + exact-path globs are allowed there and remain forbidden for + `temporary`/`delete-once-served`. +2. **`nominations` holds only the two non-rule residues** (#52): `rejected` + ("the answer is no") and `consults` ("no answer yet"). It never touches the + file filter. Entry schemas per lifecycle-spec §2; consults deliberately + carry no lifetime; presence in `consults` = open, no status field. +3. **Rejections block exact glob+lifetime repeats only** (#52). Variants flow + to the judge annotated with the related rejection — memory, not veto. + Relatedness is deterministic: the two globs' match sets intersect on the + current shortlist. +4. **The intake filter is deterministic code, not a model step** (#52, + invariant #6): `NominationIntakeFilter` in `calibrate_helpers.py`, run as + pipeline Step 3.5 between haiku nomination and the judge; drops are logged + in the run summary. +5. **Canonical ordering is writer-enforced, no hook** (#53). Grouping: + `rules` by tier (delete-once-served, temporary, keep), glob-sorted within + group; `nominations` after `rules`; `consults` before `rejected` (pending + -action queue reads first), each glob-sorted. Idempotent; hand edits + re-canonicalize on the next write. Unknown fields warn and round-trip + (never dropped). +6. **Consult exits** (#56): (a) human answer settles purpose → normal rule + persisted, consult entry deleted; (b) not rule-worthy → rewritten into + `rejected` with `rejected_by: "human"`; (c) defer → stays, resurfaces next + run. New rejections/consults appear in the Step 5 report but are not + individually gated — they are memory, not deletion authority. +7. **`extracted.md` append is owned by the extract op** (#57): distill → + `/os-vault:write` → append pointer line → `git rm`, all inside the same + per-file transaction and single hygiene commit — no window where the doc is + gone but undiscoverable. Vault extractions only; repo-durable residue + (ADR/CLAUDE.md) gets no index line. Self-protection is the already-shipped + global `**/extracted.md → keep` rule. +8. **Test coverage** (#59): unit tests only, for the two deterministic seams + (intake filter; writer). The two model-facing behaviors get worked examples + (already landed in judge.md / SKILL.md with the design commit) and IRL + audits as the next signal. + +## Implementation shape + +- `scripts/calibrate_helpers.py`: add `NominationIntakeFilter` (small + single-responsibility class, injected shortlist/rules-file data, returns + `(survivors_with_annotations, dropped_log)`) and a `RulesFileWriter` (or + extend the existing write path) that owns *all* serialization of + `.dochygiene-rules.json`, including nominations read/validate/warn. +- `skills/calibrate/SKILL.md` + workflow files: wire Step 3.5 (run the filter, + feed its annotations + open consults into the judge prompt's existing + "Nominations memory" section), Step 5 ("Open consults" report section, + consult exits a/b/c), Step 6 (persist keep verdicts / rejections / consults + through the canonical writer, dedupe consults by glob at write time). The + SKILL.md consult-loop worked example loses its "design-level pending + wiring" marker. +- `scripts/patch_applier.py`: extend the `extract-then-delete` path — when the + extraction destination is the vault, append the pointer entry (format per + lifecycle-spec §1) to `/extracted.md` (create if absent) and stage it + in the same hygiene commit; on extraction failure, neither append nor + delete applies (existing skip semantics). +- `tests/`: extend `test_calibrate_helpers.py` (or add + `test_nomination_intake.py` / writer tests) covering exact-repeat drops, + the two #52 worked relatedness cases, round-trip canonical ordering, and + unknown-field warnings; patch-applier tests cover the append-then-delete + atomicity and the no-append-on-repo-durable-extraction case. + +## Risks / Trade-offs + +- [Nominations grow without bound — stale rejections accumulate] → accepted + by design: exits are hand-deletion only (removals stay HITL); the canonical + ordering keeps the file reviewable, and entries are small. +- [Judge treats a related rejection as a veto and under-nominates] → the + judge.md "Nominations memory" section states variants-are-not-vetoes with a + worked example; IRL session audits are the detection path (#59), and IRL + evidence triggers harness design per eval discipline. +- [Writer round-trips unknown fields but a hand-edit introduces malformed + JSON] → existing skip-and-warn loader discipline applies; the writer only + runs on an already-parsed structure. +- [extracted.md append fails after a successful vault write] → the per-file + transaction skips the delete too, so the doc is never gone-but-unindexed; + the vault note is orphaned-but-harmless and the next run retries. + +## Migration Plan + +No data migration: `nominations` is additive to `schema_version` 1 and the v1 +loader ignores unknown top-level keys. Existing `.dochygiene-rules.json` +files re-canonicalize on their first post-change write. Rollback is code-only +(revert the change; `nominations` keys left in project files are inert). +After implementation: full test suite green (407 existing tests + new), then +`bin/refresh-plugins`. + +## Open Questions + +None — the design is locked (map #49); ambiguities found during +implementation go back to `lifecycle-spec.md` as the source of truth. diff --git a/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/proposal.md b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/proposal.md new file mode 100644 index 0000000..26fc427 --- /dev/null +++ b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/proposal.md @@ -0,0 +1,34 @@ +# Proposal: calibrate-assessment-inventory + +## Why + +Wayfinder map #49 locked the design (in `lifecycle-spec.md`, committed 2026-07-15) for persisting the *full* `:calibrate` assessment inventory — today only confirmed rules survive a calibration pass, so keep verdicts, human declines, and open consults evaporate and get re-litigated every pass. The design also fixed two judge-quality defects observed in calibration pass #1 (widening to container globs; no memory of prior rejections). This change implements that locked design; every decision is already made — see tickets #51–#57, #59 on the map. + +## What Changes + +- **Nominations memory**: the project rules file (`.dochygiene-rules.json`) gains a top-level `nominations` key with `consults` and `rejected` sub-keys (additive to `schema_version` 1). `rulebook.py` stays nomination-unaware. +- **`NominationIntakeFilter`** (new class in `calibrate_helpers.py`): deterministic Step 3.5 of the calibrate pipeline — drops exact glob+lifetime repeats of recorded rejections, annotates related (match-set-intersecting) history for the judge. +- **Canonical-writer extension** (`calibrate_helpers.py`): all rules-file writes go through one writer enforcing canonical ordering (rules grouped delete-once-served / temporary / keep, glob-sorted; `nominations` after `rules`, `consults` before `rejected`; unknown fields warned, round-tripped). No hook enforcement. +- **Pipeline wiring**: Step 3.5 (intake filter) before the judge; Step 5 report gains an "Open consults" section; Step 6 persistence extended — keep verdicts persist as plain keep rules (incl. exact-path singletons, per the keep-tier relaxation), human declines persist as `rejected` entries, consult verdicts dedupe into `consults`. +- **Extract-op append**: `:clean`'s extract-then-delete op appends a pointer entry to the per-directory `extracted.md` index, atomically within the single hygiene commit (the global `**/extracted.md → keep` rule is already in `rulebook.json`). +- **Two unit-test suites** (no new eval harnesses, per ticket #59): intake filter (exact-repeat drops, relatedness annotations incl. the two #52 worked cases) and writer (round-trip ordering, unknown-field warnings). + +## Capabilities + +### New Capabilities + +(none — every change extends an existing capability) + +### Modified Capabilities + +- `lifecycle-rulebook`: envelope gains the optional project-file-only `nominations` key (consults/rejected schemas); writes are writer-enforced into canonical ordering; loader remains nomination-unaware and add-only merge is unchanged. +- `calibrate`: new deterministic intake-filter requirement (Step 3.5); Persistence Rules by Scope extended to keep verdicts / rejections / consults; judge intake gains nominations-memory context (rejections are variants-not-vetoes, open consults must resurface); Rule-Quality Class-Never-Path gains the keep-tier relaxation and the enumerate-siblings-never-widen-to-container test. +- `doc-clean`: the extract-then-delete op additionally appends the `extracted.md` pointer entry in the same per-file transaction / single hygiene commit. + +## Impact + +- **Code**: `scripts/calibrate_helpers.py` (NominationIntakeFilter + canonical writer), `skills/calibrate/SKILL.md` + `workflows/*.md` (Step 3.5/5/6 wiring — judge.md context sections already landed with the design commit), `scripts/patch_applier.py` (extract-op append). +- **Tests**: `tests/test_calibrate_helpers.py` extended (or split) for the two suites; existing 407-test suite must stay green. +- **Data**: `.dochygiene-rules.json` files may now carry `nominations`; older readers unaffected (additive, `rulebook.py` ignores it). +- **Source of truth**: `plugins/os-doc-hygiene/lifecycle-spec.md` §1 (extract index), §2 (nominations memory, canonical ordering), §8 (steps 3/6, test coverage); ADR-0038/0039 amendments 2026-07-15. +- **Post-implementation**: run `bin/refresh-plugins`. diff --git a/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/specs/calibrate/spec.md b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/specs/calibrate/spec.md new file mode 100644 index 0000000..63de5bb --- /dev/null +++ b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/specs/calibrate/spec.md @@ -0,0 +1,151 @@ +# Delta: calibrate (calibrate-assessment-inventory) + +## ADDED Requirements + +### Requirement: Deterministic Nomination Intake Filter + +Between cheap-model nomination and strong-model judgment, `:calibrate` SHALL +run a deterministic `NominationIntakeFilter` (in `calibrate_helpers.py`, no +model — invariant #6). A nomination whose glob+lifetime exactly equals a +`rejected` entry SHALL be dropped before the judge and logged in the run +summary. Surviving nominations SHALL be annotated with every related +rejection, where related means the two globs' match sets intersect on the +current shortlist (deterministic, computed from the scan). The annotations +plus all open consults SHALL enter the judge prompt as its "Nominations +memory" input section; related rejections are context for the judge, never a +veto. + +#### Scenario: Exact glob+lifetime repeat is dropped before the judge + +- **WHEN** haiku nominates `docs/research/** -> temporary` and `nominations.rejected` contains an entry with glob `docs/research/**` and lifetime `temporary` +- **THEN** the nomination is dropped at intake, never reaches the judge, and the drop is logged in the run summary + +#### Scenario: A variant flows through annotated, not blocked + +- **WHEN** haiku nominates `docs/research/drafts/** -> temporary` and `nominations.rejected` contains `docs/research/** -> temporary`, and the two globs' match sets intersect on the current shortlist +- **THEN** the nomination proceeds to the judge carrying the related rejection (its why and consider_instead) as context, and the judge may still confirm it + +#### Scenario: Open consults always reach the judge prompt + +- **WHEN** `nominations.consults` is non-empty at intake time +- **THEN** every open consult is included in the judge prompt's "Nominations memory" section, regardless of what haiku nominated this round + +### Requirement: Consult Persistence and Resurfacing + +Open `consult` verdicts SHALL persist to `nominations.consults` (deduped by +glob at write time) rather than dying with the run. Consults SHALL resurface +in `:calibrate` only — `:check` and `:clean` are unchanged — appearing in the +judge prompt and as an "Open consults" section of the rule report. A consult +SHALL exit in exactly one of three ways: (a) a human answer settles the +purpose — a normal rule is persisted and the consult entry deleted; (b) the +human deems it not rule-worthy — the entry is rewritten into `rejected` with +`rejected_by: "human"` and the human's why; (c) the human defers — the entry +stays and resurfaces next run. New rejections and consults SHALL appear in +the rule report but are not individually gated — they are memory, not +deletion authority. + +#### Scenario: A consult survives the run and resurfaces + +- **WHEN** a judge verdict is `consult` and the run ends without a human answer +- **THEN** the consult is written to `nominations.consults`, and the next `:calibrate` run surfaces it in both the judge prompt and the report's "Open consults" section + +#### Scenario: An answered consult becomes a rule and disappears + +- **WHEN** the human answers an open consult in a way that settles the artifact's purpose +- **THEN** a normal rule is persisted through the standard report flow and the consult entry is deleted in the same write + +#### Scenario: A declined consult becomes a human rejection + +- **WHEN** the human answers that an open consult's artifact class is not rule-worthy +- **THEN** the consult entry is rewritten into `nominations.rejected` with `rejected_by: "human"` and the stated reason + +#### Scenario: Consults never surface outside calibrate + +- **WHEN** `:check` or `:clean` runs against a project with open consults +- **THEN** their behavior is unchanged — consults neither appear in output nor affect any classification + +## MODIFIED Requirements + +### Requirement: Persistence Rules by Scope + +Project-rulebook writes SHALL land on judge confirmation once the human has +reviewed the rule report. Global-rulebook writes (writing into +`plugins/os-doc-hygiene/rulebook.json`) SHALL additionally require explicit +human gating, distinct from project-rule confirmation, since it is a +cross-repo write into cc-os. Rule removals SHALL be HITL-only in all cases, +with recorded reasoning, regardless of scope. Every settled verdict SHALL +persist: judge `keep` verdicts become ordinary `lifetime: keep` rules in +`rules` (including exact-path singletons, per the keep-tier relaxation); +human declines at the rule report persist as `rejected` entries with +`rejected_by: "human"`; open `consult` verdicts persist to +`nominations.consults`, deduped by glob at write time. All persistence SHALL +go through the canonical rules-file writer. + +#### Scenario: Project rule persists on judge confirmation plus report review + +- **WHEN** the judge verdict is `confirm` for a project-scoped rule and the human has reviewed its rule report +- **THEN** the rule is written to the project's `.dochygiene-rules.json` + +#### Scenario: Global rulebook writes require an additional explicit gate + +- **WHEN** a proposed rule would be written to the global `rulebook.json` +- **THEN** a distinct human confirmation for the cross-repo write is required, beyond the project-rule confirmation step + +#### Scenario: Rule removal is always HITL-only + +- **WHEN** any rule (project or global) is proposed for removal +- **THEN** the removal happens only via explicit human instruction, with the reasoning recorded, never as an automatic side effect of a calibration pass + +#### Scenario: A keep verdict persists as a plain keep rule + +- **WHEN** the judge's settled verdict for a cluster is that the artifacts must be retained +- **THEN** an ordinary `lifetime: keep` rule is persisted to `rules` — removing the cluster from future calibrate pools and protecting future matches — even when the glob is an exact-path singleton + +#### Scenario: A human decline persists as a rejection + +- **WHEN** the human declines a judge-confirmed rule at the rule report +- **THEN** a `rejected` entry with `rejected_by: "human"` is written, so a later haiku round cannot re-nominate the identical glob+lifetime without the judge knowing + +### Requirement: Rule-Quality Test — Class Never Path + +A proposed rule's glob SHALL name a recurring class of artifact, never an +identifier unique to a single instance. A glob that hardcodes a name +recurring by convention (e.g. `PRD.md`, `HANDOFF-*.md`, +`migration-report.md`) is acceptable. A glob that hardcodes a run-id, hash, +or bare timestamp unique to one instance is not acceptable. A rule that +currently matches only one file is acceptable; a rule that can, by +construction, only ever match one file is a failed generalization and SHALL +be flagged loudly rather than silently persisted. **Keep-tier relaxation:** +exact-path/instance globs ARE acceptable for `lifetime: keep` entries only — +this test exists to prevent bad deletion rules, and a singleton keep merely +protects; instance globs remain forbidden for `temporary` and +`delete-once-served`. When the near-miss boundary check reveals sibling +artifacts a glob misses, the fix SHALL be to enumerate the conventional +prefixes as separate rule entries, never to widen the glob to the containing +directory; container-claiming globs are justified ONLY when the directory is +wholly machine-owned (e.g. `plugins/*/.pytest_cache/`). + +#### Scenario: A convention-recurring name is acceptable + +- **WHEN** a proposed rule's glob is `HANDOFF-*.md` +- **THEN** it passes the class-never-path test, since `HANDOFF-*` is a recurring naming convention, not a single instance + +#### Scenario: An instance-unique identifier fails the test + +- **WHEN** a proposed rule's glob hardcodes a specific run-id or hash string that can only ever identify one artifact +- **THEN** the rule fails the class-never-path test and is flagged loudly, not silently persisted + +#### Scenario: One current match is fine; one-EVER match is not + +- **WHEN** a proposed rule currently matches exactly one file +- **THEN** it is acceptable if the glob's structure could match future similarly-named files; it is flagged as a failed generalization if the glob's structure can never match any file but the one it names today + +#### Scenario: A singleton keep passes under the keep-tier relaxation + +- **WHEN** a proposed rule is `docs/research/clutter-pattern-inventory.md -> keep` +- **THEN** it passes despite being an exact-path instance glob, because the keep tier only protects; the same glob with lifetime `temporary` or `delete-once-served` fails + +#### Scenario: Missed siblings are enumerated, never widened to the container + +- **WHEN** the boundary check shows `autoresearch/classic-*/` misses sibling runs under `autoresearch/improve-*/` +- **THEN** the fix is a second rule entry `autoresearch/improve-*/` (same cluster), never a widening to `autoresearch/*/` or `autoresearch/**`, so keep-worthy content can still live in the container without a counter-rule diff --git a/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/specs/doc-clean/spec.md b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/specs/doc-clean/spec.md new file mode 100644 index 0000000..ee12862 --- /dev/null +++ b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/specs/doc-clean/spec.md @@ -0,0 +1,66 @@ +# Delta: doc-clean (calibrate-assessment-inventory) + +## MODIFIED Requirements + +### Requirement: Applier Applies Delete and Extract-Then-Delete Under the Tier Matrix + +The patch applier SHALL support two new op kinds, `delete` and +`extract-then-delete`. For both, immediately before applying, it SHALL +re-run `git ls-files ` and a dirty check against that specific path — +never trusting the cached report tier or rule claim — and SHALL apply the +tier matrix from the `lifecycle-deletion` spec to decide whether the entry +may proceed as `auto` or must be treated as `confirm` (already gated +upstream by the clean skill). `delete` SHALL perform a `git rm` (recursive +for a directory-rule aggregate entry) staged into the run's single hygiene +commit. `extract-then-delete` SHALL first complete its generative extraction +step (repo-durable via the existing live-read Sonnet distillation path +writing into an ADR/CLAUDE.md/docs target, or cross-repo via +`/os-vault:write`) and SHALL only perform the `git rm` once extraction has +succeeded; both steps SHALL land in the same single hygiene commit, or, on +extraction failure, neither SHALL be applied for that entry (skip, not a +run-level hard failure, unless the failure matches an existing hard-failure +trigger). When the extraction destination is the vault (the content +physically leaves the repo), the op SHALL additionally append a pointer +entry to the deleted file's per-directory `extracted.md` index (creating the +file if absent), in the same atomic sequence — distill → `/os-vault:write` → +append the pointer line → `git rm` — all staged into the same single hygiene +commit, so there is no window where the doc is gone but undiscoverable. The +pointer entry SHALL name the vault note, state why a future reader would +follow it, and record the source filename and date. Repo-durable extraction +targets (ADR, CLAUDE.md, docs) SHALL NOT produce an index entry — they are +already discoverable in-repo. + +#### Scenario: delete performs a true git rm at apply time + +- **WHEN** the applier applies a `delete` entry +- **THEN** it re-verifies tracked/clean status via `git ls-files` and a dirty check, then performs a `git rm` staged into the single hygiene commit + +#### Scenario: extract-then-delete only deletes after extraction succeeds + +- **WHEN** the applier applies an `extract-then-delete` entry +- **THEN** it completes the extraction write (ADR/CLAUDE.md/docs or vault) first, and performs the `git rm` only after that write succeeds + +#### Scenario: A failed extraction skips the delete for that entry + +- **WHEN** the extraction step of an `extract-then-delete` entry fails +- **THEN** the delete is not applied for that entry, the entry is reported as skipped, and the run is not treated as a hard failure unless the failure matches an existing hard-failure trigger (applier exit 2, write error) + +#### Scenario: Directory-rule aggregate delete removes the whole directory + +- **WHEN** the applier applies a `delete` entry whose path is a directory-rule aggregate entry +- **THEN** it performs a recursive `git rm` removing the entire matched directory in one operation staged into the single hygiene commit + +#### Scenario: A vault extraction leaves an extracted.md pointer in the same commit + +- **WHEN** an `extract-then-delete` entry extracts to the vault via `/os-vault:write` +- **THEN** a pointer entry naming the vault note, the reason to follow it, the source filename, and the date is appended to the deleted file's directory `extracted.md` (created if absent), and the append, the deletion, and the index file are all staged into the same single hygiene commit + +#### Scenario: A repo-durable extraction leaves no index entry + +- **WHEN** an `extract-then-delete` entry extracts into an ADR, CLAUDE.md, or docs target inside the repo +- **THEN** no `extracted.md` entry is written — the residue is already discoverable in-repo + +#### Scenario: A failed pointer append skips the delete + +- **WHEN** the vault write succeeds but appending the `extracted.md` pointer fails +- **THEN** the `git rm` is not applied for that entry and it is reported as skipped, preserving the invariant that a doc is never gone but undiscoverable diff --git a/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/specs/lifecycle-rulebook/spec.md b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/specs/lifecycle-rulebook/spec.md new file mode 100644 index 0000000..7c8bf3f --- /dev/null +++ b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/specs/lifecycle-rulebook/spec.md @@ -0,0 +1,99 @@ +# Delta: lifecycle-rulebook (calibrate-assessment-inventory) + +## MODIFIED Requirements + +### Requirement: Rulebook Locations and Envelope + +The plugin SHALL ship a global rulebook at +`plugins/os-doc-hygiene/rulebook.json`, resolved relative to plugin scripts, +present in every project. A project MAY additionally provide a committed +repo-root `.dochygiene-rules.json` override. Both files SHALL use the +envelope `{"schema_version": 1, "rules": [...]}`. The project file MAY +additionally carry a top-level `nominations` key (see the Nominations Memory +requirement); the loader SHALL ignore unknown top-level keys, so the key is +additive to `schema_version` 1. The per-project override SHALL NOT live under +gitignored `.cc-os/` — it SHALL be a committed, reviewable dotfile. + +#### Scenario: Global rulebook is always present + +- **WHEN** the rulebook loader runs in any project +- **THEN** it loads `plugins/os-doc-hygiene/rulebook.json` resolved relative to the plugin scripts directory + +#### Scenario: Per-project override is optional and committed + +- **WHEN** a project has no `.dochygiene-rules.json` at its repo root +- **THEN** the loader proceeds using only the global rulebook, and when the file is present it is read as a committed, reviewable file, never from `.cc-os/` + +#### Scenario: Both files share one envelope shape + +- **WHEN** either the global rulebook or a project override is loaded +- **THEN** it is validated against the envelope `{"schema_version": 1, "rules": [...]}` + +#### Scenario: A nominations key does not disturb the v1 loader + +- **WHEN** a project `.dochygiene-rules.json` carries a top-level `nominations` key alongside `rules` +- **THEN** `rulebook.py` loads the `rules` array exactly as before, ignoring the unknown top-level key without warning or error + +## ADDED Requirements + +### Requirement: Nominations Memory Lives in the Project Rules File + +The project `.dochygiene-rules.json` MAY carry a top-level `nominations` key +holding exactly two lists: `consults` (open questions — entries with `glob`, +`question`, `evidence`, `cluster_key`, `asked_on`, and deliberately NO +lifetime) and `rejected` (settled "no" answers — entries with `glob`, +`lifetime`, `why`, optional `consider_instead`, `rejected_by` (`"judge"` or +`"human"`), `judged_on`). The `nominations` key SHALL never affect which +files the rulebook governs — only entries in `rules` decide that. +`rulebook.py` SHALL remain nomination-unaware; only the calibrate helpers +read the key, and the calibrate reader SHALL warn on unrecognized nomination +fields, mirroring the rules array's unknown-field discipline. Rejected +entries and exact-path singleton keep rules SHALL exit only by hand-deletion +(removals stay HITL with recorded reasoning); no automated revisit path +SHALL exist. + +#### Scenario: Nominations never filter files + +- **WHEN** the scanner or rulebook resolves the governing rule for a path that only a `nominations` entry's glob matches +- **THEN** the path is treated as unmatched/unmanaged — nominations carry no lifecycle authority + +#### Scenario: Consult entries carry no lifetime + +- **WHEN** a consult entry is written to `nominations.consults` +- **THEN** it records `glob`, `question`, `evidence`, `cluster_key`, `asked_on` and no lifetime field — presence in the list means open, with no status field + +#### Scenario: Unrecognized nomination fields warn in the calibrate reader + +- **WHEN** the calibrate helpers read a nominations entry containing an unknown field +- **THEN** a warning is emitted and the field is preserved, never silently dropped + +#### Scenario: A rejection leaves only by hand-deletion + +- **WHEN** a calibration pass runs against a rules file containing a stale rejection +- **THEN** no automated path removes or expires the entry; it is removed only by explicit human edit + +### Requirement: Canonical Writer-Enforced Ordering + +Every code path that serializes `.dochygiene-rules.json` SHALL write through +one canonical writer that emits: `rules` grouped by lifetime tier in the +order delete-once-served, temporary, keep, glob-sorted within each group; +`nominations` after `rules`; `consults` before `rejected`, each glob-sorted. +The writer SHALL be idempotent (canonicalizing an already-canonical file is a +no-op) and SHALL round-trip unknown fields with a warning rather than +dropping them. Ordering SHALL NOT be enforced by any hook; hand edits +re-canonicalize on the next write. + +#### Scenario: Writes are grouped and sorted canonically + +- **WHEN** the writer serializes a rules file containing rules of all three tiers plus nominations +- **THEN** the output orders rules delete-once-served → temporary → keep with globs sorted within each group, and nominations follows rules with consults before rejected, each list glob-sorted + +#### Scenario: Canonicalization is idempotent + +- **WHEN** the writer serializes a file it previously wrote, unchanged +- **THEN** the output is byte-identical + +#### Scenario: A hand-edited file re-canonicalizes on the next write + +- **WHEN** a human appends a rule out of tier order and a later calibrate run persists a new entry +- **THEN** the whole file is rewritten in canonical order in that write, with no hook involved in the interim diff --git a/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/tasks.md b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/tasks.md new file mode 100644 index 0000000..d45d741 --- /dev/null +++ b/plugins/os-doc-hygiene/openspec/changes/calibrate-assessment-inventory/tasks.md @@ -0,0 +1,33 @@ +# Tasks: calibrate-assessment-inventory + +TDD (red-green-refactor) throughout; run the full suite (407 existing tests) +after each group. Source of truth for behavior: `lifecycle-spec.md` §1/§2/§8. + +## 1. Canonical rules-file writer (calibrate_helpers.py) + +- [x] 1.1 Write failing tests: canonical ordering (rules grouped delete-once-served → temporary → keep, glob-sorted; nominations after rules; consults before rejected, glob-sorted), idempotent round-trip (byte-identical rewrite), hand-edit re-canonicalization, unknown-field warn-and-round-trip (rules and nomination entries) +- [x] 1.2 Implement the writer as the single serialization path for `.dochygiene-rules.json`, including the nominations read/validate/warn path (`rulebook.py` untouched) +- [x] 1.3 Route every existing calibrate persistence write through the writer + +## 2. NominationIntakeFilter (calibrate_helpers.py) + +- [x] 2.1 Write failing tests: exact glob+lifetime repeat is dropped and logged; the two #52 worked cases (exact repeat of `docs/research/** -> temporary` dropped; variant `docs/research/drafts/** -> temporary` survives annotated); match-set-intersection relatedness computed from an injected shortlist; open consults always passed through to the judge input +- [x] 2.2 Implement `NominationIntakeFilter` (deterministic, injected rules-file data + shortlist; returns survivors-with-annotations + dropped log) + +## 3. Calibrate skill wiring (SKILL.md + workflows) + +- [x] 3.1 Wire Step 3.5: run the intake filter between nomination and judgment; feed annotations + open consults into the judge prompt's "Nominations memory" section; surface drops in the run summary +- [x] 3.2 Wire Step 5: add the "Open consults" report section with the three exits (answer → rule + consult deleted; decline → rejected with `rejected_by: "human"`; defer → stays); new rejections/consults shown but not individually gated +- [x] 3.3 Wire Step 6: persist keep verdicts as plain keep rules (exact-path singletons allowed — keep-tier relaxation), human declines as rejections, consult verdicts into `nominations.consults` deduped by glob — all through the canonical writer +- [x] 3.4 Remove the "design-level pending pipeline wiring" marker from the SKILL.md consult-loop worked example; verify judge.md needs no further edits (its map-#49 sections landed with the design commit) + +## 4. Extract-op extracted.md append (patch_applier.py) + +- [x] 4.1 Write failing tests: vault extraction appends the pointer entry (correct format: vault note name, why, source filename, date) to `/extracted.md`, creating it if absent, staged in the same hygiene commit as the `git rm`; repo-durable extraction (ADR/CLAUDE.md/docs) writes no entry; failed vault write skips append and delete; failed append skips the delete +- [x] 4.2 Implement the append in the extract-then-delete path (distill → vault write → append → git rm, one per-file transaction) + +## 5. Verification and rollout + +- [x] 5.1 Full test suite green (existing 407 + new); no informal runs of held-out eval scenarios +- [x] 5.2 Run `bin/refresh-plugins` and update the SKILL.md/workflow cache-sensitive docs if the refresh surfaces anything +- [x] 5.3 Record completion per repo convention: build-plan/status index untouched unless design changed; note in `docs/implementation-status/` leaf if warranted diff --git a/plugins/os-doc-hygiene/scripts/calibrate_helpers.py b/plugins/os-doc-hygiene/scripts/calibrate_helpers.py index 0e72e21..d609858 100644 --- a/plugins/os-doc-hygiene/scripts/calibrate_helpers.py +++ b/plugins/os-doc-hygiene/scripts/calibrate_helpers.py @@ -26,11 +26,14 @@ purely on lists of strings (paths) handed in by the orchestrating skill. from __future__ import annotations import fnmatch +import json import re from dataclasses import dataclass, field -from pathlib import PurePosixPath +from pathlib import Path, PurePosixPath from typing import Optional +import rulebook as _rulebook + _DEFAULT_SAMPLE_CAP = 5 # --- shape-class helpers --------------------------------------------------- @@ -330,3 +333,212 @@ class RuleQualityChecker: "narrower": narrower, "match_counts": {glob_a: count_a, glob_b: count_b}, } + + +# --- canonical rules-file writer -------------------------------------------- + +# Known fields for a rule entry — reuses rulebook.py's schema so the writer +# and the loader never drift (rulebook.py itself stays nomination-unaware; +# this is a read-only reference to its field set, design.md Non-Goal 1). +_RULE_KNOWN_FIELDS = _rulebook._KNOWN_FIELDS + +# Nominations entry schemas (lifecycle-spec.md §2). Consult entries +# deliberately carry no lifetime; rejected entries deliberately carry no +# status field. +_CONSULT_KNOWN_FIELDS = {"glob", "question", "evidence", "cluster_key", "asked_on"} +_REJECTED_KNOWN_FIELDS = { + "glob", + "lifetime", + "why", + "consider_instead", + "rejected_by", + "judged_on", +} + +# Canonical tier order for `rules`: delete-once-served -> temporary -> keep. +# Anything else (e.g. a missing/unrecognized lifetime) sorts last. +_LIFETIME_TIER_ORDER = { + "delete-once-served": 0, + "temporary": 1, + "keep": 2, +} + + +def _warn_unknown_fields(entry: dict, known_fields: set, label: str, warnings: list) -> None: + unknown = set(entry.keys()) - known_fields + if unknown: + glob_pattern = entry.get("glob", "") + warnings.append( + f"{label} {glob_pattern!r}: unrecognized field(s) {sorted(unknown)} " + "— preserved, not dropped" + ) + + +def _sort_by_glob(entries: list) -> list: + return sorted(entries, key=lambda e: e.get("glob", "")) + + +class RulesFileWriter: + """The single canonical serialization path for `.dochygiene-rules.json` + (design.md Implementation shape). Owns: + + - canonical ordering: `rules` grouped by lifetime tier (delete-once- + served -> temporary -> keep), glob-sorted within each group; + `nominations` after `rules`, with `consults` before `rejected`, each + glob-sorted. + - idempotency: canonicalizing an already-canonical structure is a no-op. + - unknown-field discipline: unrecognized fields on rules AND nomination + entries are warned about and round-tripped, never dropped. + - the nominations read/validate/warn path — `rulebook.py` stays + nomination-unaware; this is the only code that interprets the + `nominations` key. + + No knowledge of git, classification, or judge/report logic — purely a + load/canonicalize/write seam over the parsed JSON structure. + """ + + def canonicalize(self, data: dict) -> "tuple[dict, list]": + """Pure in-memory transform: returns (canonical_dict, warnings). + Does no I/O.""" + warnings: list = [] + + rules_in = data.get("rules", []) or [] + rules_out = [] + for raw in rules_in: + rule = dict(raw) + _warn_unknown_fields(rule, _RULE_KNOWN_FIELDS, "rule", warnings) + rules_out.append(rule) + + rules_out = sorted( + rules_out, + key=lambda r: ( + _LIFETIME_TIER_ORDER.get(r.get("lifetime"), 99), + r.get("glob", ""), + ), + ) + + canonical: dict = { + "schema_version": data.get("schema_version", 1), + "rules": rules_out, + } + + nominations_in = data.get("nominations") or {} + consults_in = nominations_in.get("consults", []) or [] + rejected_in = nominations_in.get("rejected", []) or [] + + consults_out = [] + for raw in consults_in: + entry = dict(raw) + _warn_unknown_fields(entry, _CONSULT_KNOWN_FIELDS, "consult entry", warnings) + consults_out.append(entry) + + rejected_out = [] + for raw in rejected_in: + entry = dict(raw) + _warn_unknown_fields(entry, _REJECTED_KNOWN_FIELDS, "rejected entry", warnings) + rejected_out.append(entry) + + consults_out = _sort_by_glob(consults_out) + rejected_out = _sort_by_glob(rejected_out) + + if consults_out or rejected_out: + canonical["nominations"] = { + "consults": consults_out, + "rejected": rejected_out, + } + + return canonical, warnings + + def load(self, path: Path) -> "tuple[dict, list]": + """Reads a rules file from disk and returns (canonical_dict, + warnings). A missing file yields the empty envelope, matching + rulebook.py's own missing-file behavior.""" + path = Path(path) + try: + text = path.read_text(encoding="utf-8") + except FileNotFoundError: + return self.canonicalize({"schema_version": 1, "rules": []}) + + data = json.loads(text) + return self.canonicalize(data) + + def write(self, path: Path, data: dict) -> list: + """Canonicalizes *data* and writes it to *path* as pretty JSON with + a trailing newline. Returns the warnings collected during + canonicalization.""" + path = Path(path) + canonical, warnings = self.canonicalize(data) + text = json.dumps(canonical, indent=2) + "\n" + path.write_text(text, encoding="utf-8") + return warnings + + +# --- nomination intake filter ------------------------------------------------ + + +class NominationIntakeFilter: + """Deterministic, no-model filter run between haiku nomination and the + strong-model judge (lifecycle-spec.md §8 step 3; calibrate spec's + "Deterministic Nomination Intake Filter" requirement). + + Injected inputs: the rules-file's `rejected` entries and open `consults` + (both read via `RulesFileWriter`/the calibrate skill, never by this + class), plus, per call, the haiku nominations and the current scanned + shortlist. No model, no I/O — pure set/string logic over injected data. + """ + + def __init__(self, rejected: list, consults: list) -> None: + self._rejected = list(rejected) + self._consults = list(consults) + + def _matches(self, glob_pattern: str, shortlist: list) -> set: + return {p for p in shortlist if _glob_matches(glob_pattern, p)} + + def filter(self, nominations: list, shortlist: list) -> dict: + """Returns: + { + "survivors": [ {**nomination, "related_rejections": [...]}, ... ], + "dropped": [ {**nomination, "reason": "..."}, ... ], + "consults": , + } + """ + survivors = [] + dropped = [] + + for nomination in nominations: + glob_pattern = nomination.get("glob") + lifetime = nomination.get("lifetime") + + exact_match = next( + ( + r + for r in self._rejected + if r.get("glob") == glob_pattern and r.get("lifetime") == lifetime + ), + None, + ) + if exact_match is not None: + dropped.append( + { + **nomination, + "reason": ( + "exact glob+lifetime repeat of a rejected nomination" + ), + } + ) + continue + + related_rejections = [] + nomination_matches = self._matches(glob_pattern, shortlist) + for rejection in self._rejected: + rejection_matches = self._matches(rejection.get("glob", ""), shortlist) + if nomination_matches & rejection_matches: + related_rejections.append(dict(rejection)) + + survivors.append({**nomination, "related_rejections": related_rejections}) + + return { + "survivors": survivors, + "dropped": dropped, + "consults": list(self._consults), + } diff --git a/plugins/os-doc-hygiene/scripts/patch_applier.py b/plugins/os-doc-hygiene/scripts/patch_applier.py index 2d59f6a..80485c1 100644 --- a/plugins/os-doc-hygiene/scripts/patch_applier.py +++ b/plugins/os-doc-hygiene/scripts/patch_applier.py @@ -296,6 +296,52 @@ def _ranges_overlap(a_start: int, a_end: int, b_start: int, b_end: int) -> bool: # atomic per-path operation, mirroring move-to-archive's special-casing. _LIFECYCLE_DELETE_KINDS = {"delete", "extract-then-delete"} +# exact_edit.extraction_target value ("cross-repo") that means the content +# physically left the repo (vault write via /os-vault:write) and therefore +# needs a discoverable extracted.md pointer. The other valid value, +# "repo-durable" (ADR/CLAUDE.md/docs residue), is already discoverable +# in-repo and gets no index entry (lifecycle-spec §1). This field is part +# of the frozen machine-report schema (validate_report.py KIND_TABLE +# required_fields for extract-then-delete) — reused here rather than +# duplicated under a new name. +_VAULT_EXTRACTION_TARGET = "cross-repo" + +_POINTER_REQUIRED_FIELDS = ("vault_note", "reason", "date") + + +def _extracted_md_relpath(path: str) -> str: + """Return the `extracted.md` path in the same directory as *path*.""" + parent = os.path.dirname(path) + return f"{parent}/extracted.md" if parent else "extracted.md" + + +def _valid_extraction_pointer(pointer: Any) -> bool: + """True iff *pointer* has non-empty vault_note/reason/date strings.""" + if not isinstance(pointer, dict): + return False + for key in _POINTER_REQUIRED_FIELDS: + value = pointer.get(key) + if not isinstance(value, str) or not value.strip(): + return False + return True + + +def _format_extracted_pointer(pointer: dict, source_filename: str) -> str: + """Format the extracted.md pointer line per lifecycle-spec §1: + + - [[vault: tool/graphify-clustering-behavior]] — why HDBSCAN + over-merges doc clusters; read before tuning any graphify + clustering params. (extracted from `graphify-clustering-notes.md`, + 2026-07-15) + """ + vault_note = pointer["vault_note"] + reason = pointer["reason"] + date = pointer["date"] + return ( + f"- [[vault: {vault_note}]] — {reason} " + f"(extracted from `{source_filename}`, {date})" + ) + # --------------------------------------------------------------------------- # PatchApplier @@ -668,6 +714,17 @@ class PatchApplier: extraction step complete via `extraction_complete: true` on the entry. This applier never performs the generative extraction itself. + When `exact_edit.extraction_target == "cross-repo"` (the vault case — + content physically left the repo via `/os-vault:write`), the git rm + additionally requires a valid top-level `extraction_pointer` dict + (`vault_note`, `reason`, `date`, all non-empty strings), and a pointer + line is appended to the deleted file's directory `extracted.md` + (creating it if absent) BEFORE the git rm, staged into the same + commit. `extraction_target == "repo-durable"` (ADR/CLAUDE.md/docs + residue) writes no index entry — it is already discoverable in-repo. + A failed append (or missing/invalid pointer metadata) skips the + delete, same fail-closed semantics as an unconfirmed extraction. + Directory-rule aggregate entries (`exact_edit.is_directory: true`) bypass the content-hash guard entirely (no single file to hash) but NEVER bypass the git-state re-verification. @@ -704,6 +761,53 @@ class PatchApplier: )) return applied, skipped_out, failed_out, staged + # extract-then-delete, vault destination: append a discoverable + # pointer to the deleted file's directory `extracted.md` BEFORE the + # git rm, so there is never a window where the doc is gone but + # undiscoverable (lifecycle-spec §1 / design decision #7). Repo- + # durable destinations (ADR/CLAUDE.md/docs, or no destination at all) + # are already discoverable in-repo and get no index entry. + if ( + kind == "extract-then-delete" + and ee.get("extraction_target") == _VAULT_EXTRACTION_TARGET + ): + pointer = entry.get("extraction_pointer") + if not _valid_extraction_pointer(pointer): + skipped_out.append(_skipped( + path, kind, idx, + "extraction-pointer-invalid", + "extraction_target is \"cross-repo\" (vault) but " + "extraction_pointer is missing or incomplete (needs " + "vault_note, reason, date); the delete is withheld so the " + "doc is never gone-but-unindexed.", + )) + return applied, skipped_out, failed_out, staged + + extracted_rel = _extracted_md_relpath(path) + extracted_abs = self._root / extracted_rel + pointer_line = _format_extracted_pointer(pointer, Path(path).name) + + try: + existing = ( + self._fs.read_bytes(extracted_abs) + if self._fs.exists(extracted_abs) else b"" + ) + if existing and not existing.endswith(b"\n"): + existing += b"\n" + self._fs.write_bytes( + extracted_abs, existing + pointer_line.encode("utf-8") + b"\n" + ) + except Exception as exc: # noqa: BLE001 + skipped_out.append(_skipped( + path, kind, idx, + "extracted-md-append-failed", + f"Could not append pointer to {extracted_rel}: {exc}; the " + "delete is withheld so the doc is never gone-but-unindexed.", + )) + return applied, skipped_out, failed_out, staged + + staged.append(extracted_rel) + try: self._git.rm(abs_path, self._root, recursive=is_directory) except Exception as exc: # noqa: BLE001 diff --git a/plugins/os-doc-hygiene/skills/calibrate/SKILL.md b/plugins/os-doc-hygiene/skills/calibrate/SKILL.md index a076231..1077604 100644 --- a/plugins/os-doc-hygiene/skills/calibrate/SKILL.md +++ b/plugins/os-doc-hygiene/skills/calibrate/SKILL.md @@ -5,10 +5,12 @@ description: Learn new lifecycle rules for a project by clustering unmatched fil # Hygiene Calibrate Skill Orchestrates the learn-new-rules loop (lifecycle-spec.md §8): **cluster-and- -sample → cheap-model nominate → strong-model judge → rule report (human) → -persist → retest**. It runs over the **unmatched pool** (unmatched = unmanaged -= not governed by any existing rulebook rule, per `rulebook.py`), and is the -only new skill this change adds — `check`/`clean` are unchanged in structure +sample → cheap-model nominate → deterministic intake filter (drop repeat +rejections, carry related rejections + open consults forward) → strong- +model judge → rule report (human, including open consults) → persist → +retest**. It runs over the **unmatched pool** (unmatched = unmanaged = not +governed by any existing rulebook rule, per `rulebook.py`), and is the only +new skill this change adds — `check`/`clean` are unchanged in structure (ADR-0039/-0041, `lifecycle-spec.md` §7). All scripts live under `${CLAUDE_PLUGIN_ROOT}/scripts/`. Run them with @@ -161,15 +163,71 @@ haiku at face value. --- +### Step 3.5 — (D) Nomination intake filter — `calibrate_helpers.NominationIntakeFilter` + +Deterministic, no model (lifecycle-spec.md §8 step 3; +`NominationIntakeFilter` requirement). Reads the project rules file's +`nominations` key via `RulesFileWriter.load`, then drops any nomination that +exactly repeats a `rejected` glob+lifetime, annotates survivors with every +*related* rejection (match-set intersection on the current shortlist), and +passes ALL open consults through unconditionally — this is the input the +judge prompt's "Nominations memory" section (Step 4) consumes. + +```bash +python3 -c ' +import json, os, sys +from pathlib import Path +sys.path.insert(0, os.environ["CLAUDE_PLUGIN_ROOT"] + "/scripts") +from calibrate_helpers import RulesFileWriter, NominationIntakeFilter + +scan = json.loads(Path(os.environ["SCRATCH"] + "/scan.json").read_text()) +project_rules = Path(scan["project_root"]) / ".dochygiene-rules.json" +nominations = json.loads(Path(os.environ["SCRATCH"] + "/nominations.json").read_text()) +shortlist = scan.get("shortlist", []) + +writer = RulesFileWriter() +data, load_warnings = writer.load(project_rules) +memory = data.get("nominations", {}) + +result = NominationIntakeFilter( + rejected=memory.get("rejected", []), + consults=memory.get("consults", []), +).filter(nominations, shortlist) + +Path(os.environ["SCRATCH"] + "/intake.json").write_text(json.dumps(result, indent=2)) +print(f"{len(result[\"survivors\"])} survivors, {len(result[\"dropped\"])} dropped, {len(result[\"consults\"])} open consults") +if result["dropped"]: + for d in result["dropped"]: + print(f" dropped: {d[\"glob\"]} -> {d[\"lifetime\"]} ({d[\"reason\"]})") +' +``` + +- **Surface every drop in the run summary** shown to the human alongside the + Step 5 report — a dropped nomination never reaches the judge, so this is + the only place it is visible. +- `intake.json`'s `survivors` (each nomination plus its `related_rejections` + annotation) and `consults` (all open consults, unconditionally) are what + gets embedded in Step 4's judge prompt as the "Nominations memory" input + section — feed the whole `survivors` array (not the raw + `nominations.json`) forward into Step 4, and include `consults` even when + empty (an empty array is a valid, meaningful "no open consults" signal). + +--- + ### Step 4 — (M) Strong-model batched judgment — **ONE Opus/Fable subagent** Dispatch a **single batched** strong-model subagent (`model: opus`, or the -project's configured Fable-tier model) to judge ALL nominations from Step 3 +project's configured Fable-tier model) to judge ALL nominations from Step 3.5 in one call (LOOP-GUARD: point it at `workflows/judge.md`, never this SKILL.md). The judge gathers its OWN evidence — re-reads matched paths against the live tree, checks near-miss boundaries — rather than trusting the haiku nomination's claims. +Judge on `intake.json`'s `survivors` (each nomination annotated with its +`related_rejections`), never the raw `nominations.json` — the dropped +exact-repeats never reach this step. `intake.json`'s `consults` is the +"Open consults" input regardless of what haiku nominated this round. + ``` Agent tool parameters: - subagent_type: "general-purpose" @@ -181,8 +239,11 @@ Agent tool parameters: Project root: - Nominations to judge (verbatim): - + Nominations to judge (verbatim, each with related_rejections): + + + Nominations memory — open consults (unconditional, may be empty): + Seed intake: @@ -256,29 +317,156 @@ report rather than silently dropping or persisting it; ask the human whether to have the judge re-amend it (loop back to Step 4 for that one rule) or drop it from this round. +#### Open consults section + +List every entry from `$SCRATCH/intake.json`'s `consults` — the open +consults from prior runs (plus any new `consult` verdict this run) — one per +entry: + +``` +Open consult: + Asked: Cluster: + Question: + Evidence: +``` + +Each open consult has exactly three exits, all human, none of which is +gated the way a rule confirmation is: + +- **(a) Answer settles the purpose** — the human's answer determines a + lifetime; persist a normal rule through this same report/Step 6 flow AND + delete the consult entry, in the same write (the rule supersedes it). +- **(b) Not rule-worthy** — the human decides the artifact class isn't a + rule; rewrite the entry into `nominations.rejected` with + `rejected_by: "human"` and the human's stated why. +- **(c) Defer** — no answer this round; the entry stays as-is and resurfaces + in the next `:calibrate` run's Step 3.5/4/5. + +New rejections and consults produced this run (from judge `consult` +verdicts, or from exit (b) above) appear in this report for visibility but +are **not individually gated** — unlike proposed rules, they carry no +deletion authority, only memory. Only proposed rules go through the "Persist +these N project rules?" gate below. + **No rule is written anywhere until the human has seen this report and responded.** Ask: "Persist these N project rules? (yes / no / a subset by -number)". Only proceed to Step 6 for the rules the human approves. +number)". Separately, resolve each open consult per its three exits above. +Only proceed to Step 6 for the rules the human approves and the consults the +human has settled or declined this round. --- -### Step 6 — (D) Persistence +### Step 6 — (D) Persistence — canonical writer only + +Every settled verdict persists through `calibrate_helpers.RulesFileWriter` +(load → mutate the parsed dict → write) — no code path serializes +`.dochygiene-rules.json` by hand. Before running this, translate the +human's Step 5 responses into four in-memory lists (the orchestrating skill's +own bookkeeping — there is no scratch file for these, since they come from +the live conversation, not a subagent): + +- `approved_rules` — the judge `confirm`/`amend` rule objects the human said + yes to (including `keep`-lifetime rules and Step-5-exit-(a) rules that + answer an open consult). +- `declined` — `(rule, why)` pairs for judge-proposed rules the human said + no to at the report. +- `settled_consult_globs` — globs of open consults the human resolved this + run, either via exit (a) (answered, folded into `approved_rules` above) or + exit (b) (declined, folded into `new_human_rejections` below); these are + removed from `nominations.consults` in the same write. +- `new_human_rejections` — `(glob, lifetime, why)` for exit-(b) consults + ("not rule-worthy"), written with `rejected_by: "human"`. +- `still_open_consults` — judge `consult` verdicts from THIS run + (`{glob, question, evidence, cluster_key}`) that remain unresolved after + Step 5 (exit (c), or simply not reached this round). + +```bash +python3 -c ' +import json, os, sys +from datetime import date +from pathlib import Path +sys.path.insert(0, os.environ["CLAUDE_PLUGIN_ROOT"] + "/scripts") +from calibrate_helpers import RulesFileWriter + +scan = json.loads(Path(os.environ["SCRATCH"] + "/scan.json").read_text()) +project_rules = Path(scan["project_root"]) / ".dochygiene-rules.json" +today = date.today().isoformat() + +# approved_rules / declined / settled_consult_globs / new_human_rejections / +# still_open_consults come from the human'"'"'s Step 5 responses (see above). + +writer = RulesFileWriter() +data, load_warnings = writer.load(project_rules) +nominations = data.setdefault("nominations", {"consults": [], "rejected": []}) + +# 1. Judge confirm/amend verdicts the human approved -> plain rules +# (keep verdicts and exact-path keep singletons allowed -- the keep-tier +# relaxation). confirmed_by/confirmed_on are set here, never by the judge. +for rule in approved_rules: + rule["confirmed_by"] = "human" + rule["confirmed_on"] = today + data["rules"].append(rule) + +# 2. Human declines at the rule report -> rejected entries, so a later +# haiku round cannot re-nominate the identical glob+lifetime without the +# judge knowing. +for rule, why in declined: + nominations["rejected"].append({ + "glob": rule["glob"], "lifetime": rule["lifetime"], + "why": why, "rejected_by": "human", "judged_on": today, + }) + +# 3. Consults the human declined this run (Step 5 exit b) -> rejected +# entries too, same rejected_by/judged_on convention. +for glob_pattern, lifetime, why in new_human_rejections: + nominations["rejected"].append({ + "glob": glob_pattern, "lifetime": lifetime, + "why": why, "rejected_by": "human", "judged_on": today, + }) + +# 4. Still-open consults from this run -> nominations.consults, deduped by +# glob (an existing entry with the same glob wins -- never duplicated). +existing_consult_globs = {c["glob"] for c in nominations["consults"]} +for consult in still_open_consults: + if consult["glob"] not in existing_consult_globs: + nominations["consults"].append({**consult, "asked_on": today}) + +# 5. Consults the human just settled this run (answered or declined) leave +# nominations.consults -- the rule (step 1) or rejection (step 3) +# supersedes them. +nominations["consults"] = [ + c for c in nominations["consults"] if c["glob"] not in settled_consult_globs +] + +write_warnings = writer.write(project_rules, data) +' +``` - **Project rules** (the common case): land in `/.dochygiene- - rules.json` on judge `confirm`/`amend` PLUS this step's human approval. - Read-modify-write the envelope (`{"schema_version": 1, "rules": [...]}`, - creating the file if absent), appending only — never mutating or removing - an existing rule here. + rules.json` on judge `confirm`/`amend` PLUS this step's human approval — + including plain `lifetime: keep` rules for judge `keep`-purpose verdicts, + exact-path singletons allowed under the keep-tier relaxation. +- **Human declines** at the rule report persist as `rejected` entries with + `rejected_by: "human"` (never silently dropped) — see item 2 above. +- **Open `consult` verdicts** persist to `nominations.consults`, deduped by + glob at write time — see item 4 above. An answered consult (Step 5 exit a) + is deleted from `consults` in the same write that persists its superseding + rule; a declined consult (exit b) is deleted from `consults` in the same + write that adds its `rejected` entry; a deferred consult (exit c) is left + untouched and resurfaces next run. - **Global rulebook writes** (`plugins/os-doc-hygiene/rulebook.json`) require a SEPARATE, EXPLICIT confirmation beyond the project-rule approval above — this is a cross-repo write into cc-os itself. Ask distinctly: "This rule looks like it belongs in the GLOBAL rulebook (applies to every project), not just this one. Write it to the global rulebook instead/as well? (yes/ - no)". Only write on explicit "yes" to THIS question. + no)". Only write on explicit "yes" to THIS question. (Rejections/consults + are always project-scoped memory — never written to the global rulebook.) - **Removals are HITL-only, always**, regardless of scope: only remove a rule when the human explicitly asks to, with the reasoning recorded in the rule's own `note` field (or a comment in the calibration run's summary) — - never as an automatic side effect of a calibration pass. + never as an automatic side effect of a calibration pass. New rejections + and consults are memory, not deletion authority, and are never gated the + way a rule persist/remove is (Step 5). Each persisted rule gets `confirmed_by` (the human's decision, not the judge's — a model-proposed rule may never set `confirm: true` on itself, @@ -289,13 +477,12 @@ it may only ask) and `confirmed_on` (today's date) per the rulebook schema ### Consult loop — worked example (map #49, #56/#59) -Design-level example of consult persistence (`lifecycle-spec.md` §2 -"Nominations memory"); the pipeline wiring lands with the nominations-memory -implementation. Run 1: the judge returns `consult` on -`docs/orchestration-audit/*.md` ("retained audit trail, or disposable once -the tune-up lands?"). Step 6's writer persists it to `nominations.consults` -(glob, question, evidence, cluster_key, asked_on — deliberately no -lifetime). Run 2, weeks later: the deterministic intake filter injects the +Worked example of consult persistence (`lifecycle-spec.md` §2 "Nominations +memory"), wired through Steps 3.5/5/6 above. Run 1: the judge returns +`consult` on `docs/orchestration-audit/*.md` ("retained audit trail, or +disposable once the tune-up lands?"). Step 6's writer persists it to +`nominations.consults` (glob, question, evidence, cluster_key, asked_on — +deliberately no lifetime). Run 2, weeks later: the deterministic intake filter injects the still-open consult into the judge prompt's "Nominations memory" section, AND the Step 5 report renders it under "Open consults". Three exits, all human: (a) the human answers "audit trail" → a `lifetime: keep` rule is persisted @@ -309,7 +496,8 @@ own. ### Step 7 — (D) Retest loop -Re-run Steps 1-6 against the shrunk unmatched pool. Stop when: +Re-run Steps 1-6 (including 3.5) against the shrunk unmatched pool. Stop +when: - a round yields **fewer than 2 new persisted rules**, OR - the unmatched pool shrank by **less than 10%** since the previous round, @@ -356,8 +544,12 @@ When run is explicitly cc-os pass #1: ## Invariants -- Steps 1, 2, 5, 6 are deterministic scripts/logic — **no model**. +- Steps 1, 2, 3.5, 5, 6 are deterministic scripts/logic — **no model**. - Step 3 = **haiku** (cheap, per-cluster nomination, patterns only). +- Step 3.5 = the deterministic `NominationIntakeFilter` — exact glob+lifetime + repeats of a `rejected` entry are dropped before the judge ever sees them; + survivors carry `related_rejections`; all open consults pass through + unconditionally. - Step 4 = **ONE batched Opus/Fable** judge call, never per-cluster. - **No rule is persisted before the Step 5 report has been shown to the human** (hard invariant — never skip Step 5, never merge it with Step 6). diff --git a/plugins/os-doc-hygiene/skills/calibrate/workflows/judge.md b/plugins/os-doc-hygiene/skills/calibrate/workflows/judge.md index d893110..4c538d9 100644 --- a/plugins/os-doc-hygiene/skills/calibrate/workflows/judge.md +++ b/plugins/os-doc-hygiene/skills/calibrate/workflows/judge.md @@ -150,6 +150,9 @@ nomination (nominations with a `null` glob need no entry): "cluster_key": "docs::status-#", "verdict": "consult", "rule": null, + "glob": "docs/orchestration-audit/*.md", + "question": "Are the dated audit tune-up reports a retained audit trail, or disposable once the tune-up lands?", + "evidence": "4 files, one per audit run; docs/implementation-status.md references only the latest", "reasoning": "Cannot determine from content or naming whether these status snapshots are meant to be retained as an audit trail or are disposable scratch notes -- purpose is genuinely unclear." } ] @@ -163,6 +166,15 @@ nomination (nominations with a `null` glob need no entry): `delete-once-served`). Do NOT include `confirmed_by`/`confirmed_on` — you are proposing, not confirming; **you may never set `confirm: true` on a rule yourself, only the human can.** +- **For `consult`, ALSO include `glob` (echoed from the nomination you're + judging — the persisted consult entry needs it and `rule` is `null`), + `question` (the specific ambiguity, one sentence, phrased for a human to + answer), and `evidence` (what you found — sample size, what references + exist, what a representative file's content told you). These three are + what `nominations.consults` persists (lifecycle-spec.md §2) — `reasoning` + alone is not enough downstream, since the Step 5 "Open consults" section + and the persisted entry both need `question` and `evidence` as distinct + fields, not one prose blob. - `reasoning` is shown to the human in the Step 5 rule report's "why" field — write it in plain language, not JSON-schema-speak. diff --git a/plugins/os-doc-hygiene/skills/clean/SKILL.md b/plugins/os-doc-hygiene/skills/clean/SKILL.md index 05d8fd0..df92edc 100644 --- a/plugins/os-doc-hygiene/skills/clean/SKILL.md +++ b/plugins/os-doc-hygiene/skills/clean/SKILL.md @@ -337,7 +337,7 @@ For each `approved_det` entry with `exact_edit.kind == extraction against a missing file. 2. **Live-read the file contents now** (same freshness rule as Step 8's generative path — never trust the report's cached text). -3. **Branch on `exact_edit.extraction_dest`:** +3. **Branch on `exact_edit.extraction_target`:** - **`"repo-durable"`** — dispatch the SAME generative Sonnet-subagent path `doc-clean` already uses (LOOP-GUARD: point it at `skills/clean/workflows/extract.md`, NEVER this SKILL.md). Give it the @@ -358,9 +358,15 @@ For each `approved_det` entry with `exact_edit.kind == source path, why it's evergreen) for it to choose vault frontmatter. `/os-vault:write` owns its own destination — this step supplies content and context only, per spec §1 ("no new destinations" for cross-repo - extraction). + extraction). On success, capture the pointer metadata the applier needs + for the `extracted.md` index entry (lifecycle-spec §1): the vault note + name `/os-vault:write` reports, a one-line reason a future reader would + follow the link, and today's date (`YYYY-MM-DD`). 4. **On extraction success:** record the entry's original report index in - `extraction_confirmed_indices` for Step 7 below. + `extraction_confirmed_indices` for Step 7 below; for `"cross-repo"` + entries also record the captured pointer metadata in + `extraction_pointers[idx]`. Without it the applier fails that entry + closed (`extraction-pointer-invalid`) and skips the deletion. 5. **On extraction failure** (subagent errors, or `/os-vault:write` fails): do NOT add the index to `extraction_confirmed_indices` — leave the entry unconfirmed. This is a per-entry fails-closed skip (patch_applier.py will @@ -379,6 +385,9 @@ from pathlib import Path report = json.loads(Path(REPORT_PATH).read_text()) for idx in extraction_confirmed_indices: report["entries"][idx]["extraction_complete"] = True +for idx, pointer in extraction_pointers.items(): + # cross-repo entries only: {"vault_note": ..., "reason": ..., "date": ...} + report["entries"][idx]["extraction_pointer"] = pointer Path(SCRATCH, "report_with_extraction.json").write_text(json.dumps(report)) ``` diff --git a/plugins/os-doc-hygiene/tests/test_nomination_intake.py b/plugins/os-doc-hygiene/tests/test_nomination_intake.py new file mode 100644 index 0000000..19def92 --- /dev/null +++ b/plugins/os-doc-hygiene/tests/test_nomination_intake.py @@ -0,0 +1,144 @@ +""" +Tests for `NominationIntakeFilter` in scripts/calibrate_helpers.py. + +Covers tasks 2.1/2.2 of the calibrate-assessment-inventory change: the +deterministic, no-model filter that runs between haiku nomination and the +strong-model judge (lifecycle-spec.md §8 step 3, calibrate spec's +"Deterministic Nomination Intake Filter" requirement). + +sys.path is patched here (no shared conftest), matching the existing +tests/test_calibrate_helpers.py pattern. +""" + +from __future__ import annotations + +import sys +from pathlib import Path + +_SCRIPTS_DIR = Path(__file__).parent.parent / "scripts" +if str(_SCRIPTS_DIR) not in sys.path: + sys.path.insert(0, str(_SCRIPTS_DIR)) + +from calibrate_helpers import NominationIntakeFilter # noqa: E402 + + +def _rejected(glob, lifetime, why="why", consider_instead=None): + d = { + "glob": glob, + "lifetime": lifetime, + "why": why, + "rejected_by": "judge", + "judged_on": "2026-07-15", + } + if consider_instead is not None: + d["consider_instead"] = consider_instead + return d + + +def _consult(glob, question="q?"): + return { + "glob": glob, + "question": question, + "evidence": "e", + "cluster_key": "k", + "asked_on": "2026-07-15", + } + + +class TestExactRepeatDrop: + def test_exact_glob_and_lifetime_match_is_dropped_and_logged(self): + rejected = [_rejected("docs/research/**", "temporary")] + filt = NominationIntakeFilter(rejected=rejected, consults=[]) + + nominations = [{"glob": "docs/research/**", "lifetime": "temporary"}] + shortlist = ["docs/research/notes.md"] + + result = filt.filter(nominations, shortlist) + + assert result["survivors"] == [] + assert len(result["dropped"]) == 1 + assert result["dropped"][0]["glob"] == "docs/research/**" + assert result["dropped"][0]["lifetime"] == "temporary" + + def test_same_glob_different_lifetime_is_not_dropped(self): + rejected = [_rejected("docs/research/**", "temporary")] + filt = NominationIntakeFilter(rejected=rejected, consults=[]) + + nominations = [{"glob": "docs/research/**", "lifetime": "keep"}] + shortlist = ["docs/research/notes.md"] + + result = filt.filter(nominations, shortlist) + + assert len(result["survivors"]) == 1 + assert result["dropped"] == [] + + +class TestVariantAnnotation: + def test_variant_survives_annotated_with_related_rejection(self): + # #52 worked example: docs/research/** -> temporary was rejected; + # docs/research/drafts/** -> temporary is a narrower variant whose + # match set intersects the rejected glob's match set on the + # shortlist -- it survives, annotated with the rejection. + rejected = [ + _rejected( + "docs/research/**", + "temporary", + why="findings docs other artifacts link to", + consider_instead="an extract-to-vault rule", + ) + ] + filt = NominationIntakeFilter(rejected=rejected, consults=[]) + + nominations = [{"glob": "docs/research/drafts/**", "lifetime": "temporary"}] + shortlist = [ + "docs/research/drafts/outline.md", + "docs/research/other-notes.md", + ] + + result = filt.filter(nominations, shortlist) + + assert len(result["survivors"]) == 1 + survivor = result["survivors"][0] + assert survivor["glob"] == "docs/research/drafts/**" + assert len(survivor["related_rejections"]) == 1 + related = survivor["related_rejections"][0] + assert related["glob"] == "docs/research/**" + assert related["why"] == "findings docs other artifacts link to" + assert related["consider_instead"] == "an extract-to-vault rule" + assert result["dropped"] == [] + + def test_no_annotation_when_match_sets_do_not_intersect(self): + rejected = [_rejected("docs/research/**", "temporary")] + filt = NominationIntakeFilter(rejected=rejected, consults=[]) + + nominations = [{"glob": "docs/other-area/**", "lifetime": "temporary"}] + # shortlist has no path that both globs could match in common + shortlist = ["docs/research/notes.md", "docs/other-area/thing.md"] + + result = filt.filter(nominations, shortlist) + + assert len(result["survivors"]) == 1 + survivor = result["survivors"][0] + assert survivor["related_rejections"] == [] + + +class TestOpenConsultsPassThrough: + def test_open_consults_pass_through_even_with_no_nominations(self): + consults = [_consult("docs/orchestration-audit/*.md")] + filt = NominationIntakeFilter(rejected=[], consults=consults) + + result = filt.filter([], []) + + assert result["consults"] == consults + assert result["survivors"] == [] + assert result["dropped"] == [] + + def test_open_consults_pass_through_alongside_survivors(self): + consults = [_consult("docs/orchestration-audit/*.md")] + filt = NominationIntakeFilter(rejected=[], consults=consults) + + nominations = [{"glob": "a/*.md", "lifetime": "keep"}] + result = filt.filter(nominations, ["a/x.md"]) + + assert result["consults"] == consults + assert len(result["survivors"]) == 1 diff --git a/plugins/os-doc-hygiene/tests/test_patch_applier.py b/plugins/os-doc-hygiene/tests/test_patch_applier.py index 938c103..4012dbd 100644 --- a/plugins/os-doc-hygiene/tests/test_patch_applier.py +++ b/plugins/os-doc-hygiene/tests/test_patch_applier.py @@ -873,14 +873,21 @@ def _lifecycle_entry( is_directory: bool = False, extraction_complete: bool | None = None, lifecycle: dict | None = None, + extraction_target: str | None = None, + extraction_pointer: dict | None = None, ) -> dict: """Build a minimal `delete` / `extract-then-delete` report entry. Deliberately independent of KIND_TABLE (see module docstring above). + `extraction_target` (when given) mirrors the pre-existing frozen-schema + field validate_report.py already requires on extract-then-delete entries + (`"repo-durable"` or `"cross-repo"`, the vault case). """ exact_edit: dict[str, Any] = {"kind": kind} if is_directory: exact_edit["is_directory"] = True + if extraction_target is not None: + exact_edit["extraction_target"] = extraction_target if extraction_complete is not None: exact_edit["extraction_complete"] = extraction_complete entry_extraction_complete = extraction_complete @@ -899,6 +906,8 @@ def _lifecycle_entry( } if extraction_complete is not None: entry["extraction_complete"] = extraction_complete + if extraction_pointer is not None: + entry["extraction_pointer"] = extraction_pointer return entry @@ -1173,6 +1182,211 @@ class TestExtractThenDeleteFailsClosed: assert git.rm_calls == [] +class WriteFailsFS(MemFS): + """MemFS that raises on write_bytes for a chosen path (simulates an + unwritable extracted.md — e.g. its directory was removed).""" + + def __init__(self, files: dict[str, bytes] | None = None, fail_path: str | None = None) -> None: + super().__init__(files) + self._fail_path = fail_path + + def write_bytes(self, path: Path, data: bytes) -> None: + if self._fail_path is not None and str(path) == self._fail_path: + raise OSError("simulated write failure: directory removed") + super().write_bytes(path, data) + + +class TestExtractThenDeleteVaultPointerAppend: + """Task 4.2: vault-destination extractions append an extracted.md pointer, + staged into the same transaction as the git rm; repo-durable destinations + get no index entry; a failed append (or missing/invalid pointer + metadata) skips the delete (fail closed, never gone-but-unindexed).""" + + def _pointer(self, **overrides) -> dict: + base = { + "vault_note": "tool/graphify-clustering-behavior", + "reason": "why HDBSCAN over-merges doc clusters; read before " + "tuning any graphify clustering params", + "date": "2026-07-15", + } + base.update(overrides) + return base + + def test_vault_extraction_appends_pointer_and_creates_extracted_md(self, tmp_path): + fs = MemFS() + git = RecordingGitRm() + git_state = RecordingGitState(tracked=True, dirty=False) + entry = _lifecycle_entry( + "docs/graphify-clustering-notes.md", "extract-then-delete", + safety_tier="auto", extraction_complete=True, + extraction_target="cross-repo", extraction_pointer=self._pointer(), + ) + + result = _applier_ls(tmp_path, fs=fs, git=git, git_state=git_state).apply([entry]) + + assert result["applied"][0]["kind"] == "extract-then-delete" + assert result["skipped"] == [] + assert len(git.rm_calls) == 1 + + extracted_path = str(tmp_path / "docs" / "extracted.md") + assert extracted_path in fs.writes + content = fs.writes[extracted_path].decode("utf-8") + expected_line = ( + "- [[vault: tool/graphify-clustering-behavior]] — why HDBSCAN " + "over-merges doc clusters; read before tuning any graphify " + "clustering params (extracted from `graphify-clustering-notes.md`, " + "2026-07-15)" + ) + assert expected_line in content + assert "docs/extracted.md" in result["staged_paths"] + assert "docs/graphify-clustering-notes.md" in result["staged_paths"] + + def test_vault_extraction_appends_to_existing_extracted_md(self, tmp_path): + existing_path = str(tmp_path / "docs" / "extracted.md") + existing_content = ( + "- [[vault: tool/old-note]] — prior entry " + "(extracted from `old.md`, 2026-07-01)\n" + ) + fs = MemFS({existing_path: existing_content.encode("utf-8")}) + git = RecordingGitRm() + git_state = RecordingGitState(tracked=True, dirty=False) + entry = _lifecycle_entry( + "docs/graphify-clustering-notes.md", "extract-then-delete", + safety_tier="auto", extraction_complete=True, + extraction_target="cross-repo", extraction_pointer=self._pointer(), + ) + + result = _applier_ls(tmp_path, fs=fs, git=git, git_state=git_state).apply([entry]) + + assert result["applied"][0]["kind"] == "extract-then-delete" + content = fs.writes[existing_path].decode("utf-8") + assert "tool/old-note" in content + assert "tool/graphify-clustering-behavior" in content + # Prior entry preserved before the newly appended one. + assert content.index("tool/old-note") < content.index("tool/graphify-clustering-behavior") + + def test_repo_durable_extraction_writes_no_extracted_md(self, tmp_path): + fs = MemFS() + git = RecordingGitRm() + git_state = RecordingGitState(tracked=True, dirty=False) + entry = _lifecycle_entry( + "docs/old-guide.md", "extract-then-delete", + safety_tier="auto", extraction_complete=True, + extraction_target="repo-durable", + ) + + result = _applier_ls(tmp_path, fs=fs, git=git, git_state=git_state).apply([entry]) + + assert result["applied"][0]["kind"] == "extract-then-delete" + assert fs.writes == {} + assert result["staged_paths"] == ["docs/old-guide.md"] + + def test_no_extraction_target_writes_no_extracted_md(self, tmp_path): + """Absence of extraction_target (or any value other than + "cross-repo") is treated as repo-durable — no pointer append.""" + fs = MemFS() + git = RecordingGitRm() + git_state = RecordingGitState(tracked=True, dirty=False) + entry = _lifecycle_entry( + "docs/old-guide.md", "extract-then-delete", + safety_tier="auto", extraction_complete=True, + ) + + result = _applier_ls(tmp_path, fs=fs, git=git, git_state=git_state).apply([entry]) + + assert result["applied"][0]["kind"] == "extract-then-delete" + assert fs.writes == {} + + def test_failed_append_skips_the_delete(self, tmp_path): + extracted_path = str(tmp_path / "docs" / "extracted.md") + fs = WriteFailsFS(fail_path=extracted_path) + git = RecordingGitRm() + git_state = RecordingGitState(tracked=True, dirty=False) + entry = _lifecycle_entry( + "docs/graphify-clustering-notes.md", "extract-then-delete", + safety_tier="auto", extraction_complete=True, + extraction_target="cross-repo", extraction_pointer=self._pointer(), + ) + + result = _applier_ls(tmp_path, fs=fs, git=git, git_state=git_state).apply([entry]) + + assert result["applied"] == [] + assert git.rm_calls == [] + assert result["skipped"][0]["reason"] == "extracted-md-append-failed" + + def test_missing_pointer_metadata_skips_the_delete(self, tmp_path): + fs = MemFS() + git = RecordingGitRm() + git_state = RecordingGitState(tracked=True, dirty=False) + entry = _lifecycle_entry( + "docs/graphify-clustering-notes.md", "extract-then-delete", + safety_tier="auto", extraction_complete=True, + extraction_target="cross-repo", + # No extraction_pointer at all. + ) + + result = _applier_ls(tmp_path, fs=fs, git=git, git_state=git_state).apply([entry]) + + assert result["applied"] == [] + assert git.rm_calls == [] + assert result["skipped"][0]["reason"] == "extraction-pointer-invalid" + assert fs.writes == {} + + def test_incomplete_pointer_metadata_skips_the_delete(self, tmp_path): + fs = MemFS() + git = RecordingGitRm() + git_state = RecordingGitState(tracked=True, dirty=False) + entry = _lifecycle_entry( + "docs/graphify-clustering-notes.md", "extract-then-delete", + safety_tier="auto", extraction_complete=True, + extraction_target="cross-repo", + extraction_pointer={"vault_note": "tool/x"}, # missing reason/date + ) + + result = _applier_ls(tmp_path, fs=fs, git=git, git_state=git_state).apply([entry]) + + assert result["applied"] == [] + assert git.rm_calls == [] + assert result["skipped"][0]["reason"] == "extraction-pointer-invalid" + assert fs.writes == {} + + def test_extraction_complete_false_skips_before_any_pointer_logic(self, tmp_path): + """extraction_complete: false still skips everything, even with a + fully-formed vault pointer present.""" + fs = MemFS() + git = RecordingGitRm() + git_state = RecordingGitState(tracked=True, dirty=False) + entry = _lifecycle_entry( + "docs/graphify-clustering-notes.md", "extract-then-delete", + safety_tier="auto", extraction_complete=False, + extraction_target="cross-repo", extraction_pointer=self._pointer(), + ) + + result = _applier_ls(tmp_path, fs=fs, git=git, git_state=git_state).apply([entry]) + + assert result["applied"] == [] + assert git.rm_calls == [] + assert result["skipped"][0]["reason"] == "extraction-not-confirmed" + assert fs.writes == {} + + def test_root_level_file_uses_root_extracted_md(self, tmp_path): + fs = MemFS() + git = RecordingGitRm() + git_state = RecordingGitState(tracked=True, dirty=False) + entry = _lifecycle_entry( + "graphify-clustering-notes.md", "extract-then-delete", + safety_tier="auto", extraction_complete=True, + extraction_target="cross-repo", extraction_pointer=self._pointer(), + ) + + result = _applier_ls(tmp_path, fs=fs, git=git, git_state=git_state).apply([entry]) + + assert result["applied"][0]["kind"] == "extract-then-delete" + extracted_path = str(tmp_path / "extracted.md") + assert extracted_path in fs.writes + assert "extracted.md" in result["staged_paths"] + + class TestLifecycleConfirmTierGating: """Task 4.1(d): confirm-tier lifecycle entries never auto-apply silently.""" diff --git a/plugins/os-doc-hygiene/tests/test_rules_file_writer.py b/plugins/os-doc-hygiene/tests/test_rules_file_writer.py new file mode 100644 index 0000000..dc8c77e --- /dev/null +++ b/plugins/os-doc-hygiene/tests/test_rules_file_writer.py @@ -0,0 +1,303 @@ +""" +Tests for `RulesFileWriter` in scripts/calibrate_helpers.py. + +Covers tasks 1.1/1.2 of the calibrate-assessment-inventory change: the +single canonical serialization path for `.dochygiene-rules.json` — +- canonical ordering: rules grouped delete-once-served -> temporary -> keep, + glob-sorted within each group; nominations after rules; consults before + rejected, each glob-sorted +- idempotent round-trip (byte-identical rewrite of an already-canonical file) +- hand-edit re-canonicalization (out-of-order input is rewritten canonical) +- unknown-field warn-and-round-trip, for both rules and nomination entries + +sys.path is patched here (no shared conftest), matching the existing +tests/test_calibrate_helpers.py pattern. +""" + +from __future__ import annotations + +import json +import sys +from pathlib import Path + +_SCRIPTS_DIR = Path(__file__).parent.parent / "scripts" +if str(_SCRIPTS_DIR) not in sys.path: + sys.path.insert(0, str(_SCRIPTS_DIR)) + +from calibrate_helpers import RulesFileWriter # noqa: E402 + + +def _rule(glob, lifetime, **extra): + d = {"glob": glob, "lifetime": lifetime, "confirmed_by": "human", "confirmed_on": "2026-07-15"} + d.update(extra) + return d + + +class TestCanonicalOrdering: + def test_rules_grouped_by_tier_in_order_and_glob_sorted(self): + data = { + "schema_version": 1, + "rules": [ + _rule("z/keep.md", "keep"), + _rule("a/keep.md", "keep"), + _rule("b/temp/*.md", "temporary"), + _rule("a/temp/*.md", "temporary"), + _rule("z/served.md", "delete-once-served"), + _rule("a/served.md", "delete-once-served"), + ], + } + writer = RulesFileWriter() + canonical, _warnings = writer.canonicalize(data) + globs = [r["glob"] for r in canonical["rules"]] + assert globs == [ + "a/served.md", + "z/served.md", + "a/temp/*.md", + "b/temp/*.md", + "a/keep.md", + "z/keep.md", + ] + + def test_nominations_after_rules_consults_before_rejected_sorted(self): + data = { + "schema_version": 1, + "rules": [_rule("a.md", "keep")], + "nominations": { + "rejected": [ + { + "glob": "z/reject.md", + "lifetime": "temporary", + "why": "why z", + "rejected_by": "judge", + "judged_on": "2026-07-15", + }, + { + "glob": "a/reject.md", + "lifetime": "temporary", + "why": "why a", + "rejected_by": "judge", + "judged_on": "2026-07-15", + }, + ], + "consults": [ + { + "glob": "z/consult.md", + "question": "q z", + "evidence": "e", + "cluster_key": "k", + "asked_on": "2026-07-15", + }, + { + "glob": "a/consult.md", + "question": "q a", + "evidence": "e", + "cluster_key": "k", + "asked_on": "2026-07-15", + }, + ], + }, + } + writer = RulesFileWriter() + canonical, _warnings = writer.canonicalize(data) + + keys = list(canonical.keys()) + assert keys.index("rules") < keys.index("nominations") + + nomination_keys = list(canonical["nominations"].keys()) + assert nomination_keys.index("consults") < nomination_keys.index("rejected") + + assert [c["glob"] for c in canonical["nominations"]["consults"]] == [ + "a/consult.md", + "z/consult.md", + ] + assert [r["glob"] for r in canonical["nominations"]["rejected"]] == [ + "a/reject.md", + "z/reject.md", + ] + + def test_nominations_omitted_when_absent(self): + data = {"schema_version": 1, "rules": [_rule("a.md", "keep")]} + writer = RulesFileWriter() + canonical, _warnings = writer.canonicalize(data) + assert "nominations" not in canonical + + def test_nominations_omitted_when_present_but_empty(self): + data = { + "schema_version": 1, + "rules": [_rule("a.md", "keep")], + "nominations": {"consults": [], "rejected": []}, + } + writer = RulesFileWriter() + canonical, _warnings = writer.canonicalize(data) + assert "nominations" not in canonical + + +class TestIdempotentRoundTrip: + def test_writing_an_already_canonical_file_is_byte_identical(self, tmp_path): + data = { + "schema_version": 1, + "rules": [ + _rule("a/served.md", "delete-once-served"), + _rule("a/temp/*.md", "temporary"), + _rule("a/keep.md", "keep"), + ], + "nominations": { + "consults": [ + { + "glob": "a/consult.md", + "question": "q", + "evidence": "e", + "cluster_key": "k", + "asked_on": "2026-07-15", + } + ], + "rejected": [ + { + "glob": "a/reject.md", + "lifetime": "temporary", + "why": "why", + "rejected_by": "judge", + "judged_on": "2026-07-15", + } + ], + }, + } + writer = RulesFileWriter() + path = tmp_path / ".dochygiene-rules.json" + writer.write(path, data) + first_bytes = path.read_bytes() + + loaded, _warnings = writer.load(path) + writer.write(path, loaded) + second_bytes = path.read_bytes() + + assert first_bytes == second_bytes + + +class TestHandEditRecanonicalization: + def test_hand_edited_out_of_order_file_recanonicalizes_on_next_write(self, tmp_path): + # Hand-authored file: keep rule listed before temporary/served rules, + # rejected before consults -- out of canonical order. + hand_edited = { + "schema_version": 1, + "rules": [ + _rule("a/keep.md", "keep"), + _rule("a/served.md", "delete-once-served"), + _rule("a/temp/*.md", "temporary"), + ], + "nominations": { + "rejected": [ + { + "glob": "a/reject.md", + "lifetime": "temporary", + "why": "why", + "rejected_by": "judge", + "judged_on": "2026-07-15", + } + ], + "consults": [ + { + "glob": "a/consult.md", + "question": "q", + "evidence": "e", + "cluster_key": "k", + "asked_on": "2026-07-15", + } + ], + }, + } + path = tmp_path / ".dochygiene-rules.json" + path.write_text(json.dumps(hand_edited, indent=2), encoding="utf-8") + + writer = RulesFileWriter() + loaded, _warnings = writer.load(path) + writer.write(path, loaded) + + on_disk = json.loads(path.read_text(encoding="utf-8")) + assert [r["glob"] for r in on_disk["rules"]] == [ + "a/served.md", + "a/temp/*.md", + "a/keep.md", + ] + nomination_keys = list(on_disk["nominations"].keys()) + assert nomination_keys.index("consults") < nomination_keys.index("rejected") + + +class TestUnknownFieldsWarnAndRoundtrip: + def test_unknown_rule_field_warns_and_is_preserved(self): + data = { + "schema_version": 1, + "rules": [_rule("a.md", "keep", mystery_field="surprise")], + } + writer = RulesFileWriter() + canonical, warnings = writer.canonicalize(data) + assert canonical["rules"][0]["mystery_field"] == "surprise" + assert any("mystery_field" in w for w in warnings) + + def test_unknown_consult_field_warns_and_is_preserved(self): + data = { + "schema_version": 1, + "rules": [], + "nominations": { + "consults": [ + { + "glob": "a/consult.md", + "question": "q", + "evidence": "e", + "cluster_key": "k", + "asked_on": "2026-07-15", + "extra_field": "unexpected", + } + ], + "rejected": [], + }, + } + writer = RulesFileWriter() + canonical, warnings = writer.canonicalize(data) + assert canonical["nominations"]["consults"][0]["extra_field"] == "unexpected" + assert any("extra_field" in w for w in warnings) + + def test_unknown_rejected_field_warns_and_is_preserved(self): + data = { + "schema_version": 1, + "rules": [], + "nominations": { + "consults": [], + "rejected": [ + { + "glob": "a/reject.md", + "lifetime": "temporary", + "why": "why", + "rejected_by": "judge", + "judged_on": "2026-07-15", + "surprise_field": "oops", + } + ], + }, + } + writer = RulesFileWriter() + canonical, warnings = writer.canonicalize(data) + assert canonical["nominations"]["rejected"][0]["surprise_field"] == "oops" + assert any("surprise_field" in w for w in warnings) + + +class TestLoad: + def test_load_reads_file_and_returns_canonical_data(self, tmp_path): + data = { + "schema_version": 1, + "rules": [_rule("a/keep.md", "keep")], + } + path = tmp_path / ".dochygiene-rules.json" + path.write_text(json.dumps(data), encoding="utf-8") + writer = RulesFileWriter() + loaded, warnings = writer.load(path) + assert loaded["rules"][0]["glob"] == "a/keep.md" + assert warnings == [] + + def test_load_missing_file_returns_empty_envelope(self, tmp_path): + path = tmp_path / "does-not-exist.json" + writer = RulesFileWriter() + loaded, warnings = writer.load(path) + assert loaded["rules"] == [] + assert "nominations" not in loaded + assert warnings == []