cc-os/plugins/os-doc-hygiene/openspec/changes/archive/2026-06-24-add-clean/design.md

Design: Clean

Context

The check pipeline (build step #3) is in place: scanner.py + report_builder.py + validate_report.py + state_store.py together produce a schema-valid .dochygiene/report.json and report.md, then stamp last_check. The applier is also in place: patch_applier.py exists, has 263 passing tests, and knows how to apply every kind in the KIND_TABLE via per-file transactions with a content-hash guard. What is missing is the orchestration skill that loads that report, gates dangerous entries, invokes the applier, handles generative entries, and produces a single reviewable git commit.

Constraints that shape this design:

• Invariant #4 (no accumulating artifacts): keep only the latest report pair. The clean skill must not write new state files; last_clean is the only new state entry.
• Invariant #5 (git-safe single commit): clean/committed tree required or an auto WIP checkpoint; the cleanup output is exactly one commit. NEVER two commits.
• Invariant #6 (deterministic-first): apply, stage, and commit are scripts. Generative distillation is the only model pass during clean.
• Invariant #7 (safety-tier gating): auto ops apply without prompt; confirm ops escalate. Gating does NOT relax under sweep.
• Invariant #8 (content guard): the applier reads expected_sha256 from the report; if a file changed since check, the entire file batch is skipped (not just the conflicting entry). The guard is a content hash, not mtime — but per the META-RULE, invariant #8's wording ("mtime guard") must not change without human approval.

Goals / Non-Goals

Goals:

• A hygiene-clean skill that loads the report, gates confirms, runs the applier, handles generative entries via Sonnet, stages precisely, commits once, and stamps last_clean.
• A workflows/distill.md Sonnet workflow for generative entries (LOOP-GUARD via subagent pointer).
• Live routing in commands/hygiene.md for clean and sweep (replacing Phase 3 stubs).
• Integration tests covering the invariant #5 and #7 GAPs identified in the design.

Non-Goals:

• token_estimate weighting (injection_frequency, weighted_tokens, rollup) — v2 bonus (PRD phase 5).
• Inbound-link rewriting after move-to-archive — v1 move only; next check flags new orphans.
• The live model-classification regression harness — separate, manually invoked.

Decisions

D1. Per-file transaction with descending-anchor application — the content-hash guard lives in patch_applier.py

The core correctness invariant: applying edits one-at-a-time to a file is wrong because each write shifts line numbers, causing subsequent anchors to land on the wrong lines and potentially tripping the guard on stale content. The resolution:

1. Read file bytes once.
2. Verify sha256(bytes) == expected_sha256 for every anchored entry on that file. On any mismatch, skip the entire file batch (reason: content-changed-since-check, recommendation: re-analysis) and continue other files.
3. Apply anchored edits in memory, descending by anchor.start_line — edits to later lines first so earlier-line anchors remain valid.
4. Apply insert-frontmatter last (it prepends, shifting all lines; doing it last means descending-anchor edits see the correct line numbers).
5. Write once; stage once.

The guard is a content hash only — no mtime pre-filter. Per-entry generated_at in the report records the file's hash instant (distinct from the envelope/last_check stamp); the applier does not use generated_at, only expected_sha256.

Alternative considered: apply edits sequentially with line-offset tracking — rejected: complex, error-prone across multiple edit kinds, and untestable in isolation.

D2. insert-frontmatter: re-derive at apply time (resolves the deferred hole from add-check)

The add-check design recorded a deferred hole: insert-frontmatter has has_anchor = False and therefore carries no expected_sha256, so invariant #8 cannot protect it at apply time via the normal content-hash path.

Resolution (implemented in patch_applier.py): at apply time, re-read the file and parse its frontmatter fresh:

• Key already present with the target value → idempotent no-op (success).
• Key already present with a different value → skip with reason frontmatter-key-conflict; NEVER overwrite.
• Key absent → insert key: value (creating a --- block if needed) as the last in-memory step (after all descending-anchor edits).

This re-derivation replaces the missing hash guard for frontmatter; it is idempotent and does not trust any cached state. This resolves the deferred hole noted in add-check and in CLAUDE.md.

D3. Incompatible-ops detection — skip the file, never compose

Per file, before any in-memory editing, the applier checks:

• Does move-to-archive co-occur with any other op on the same file?
• Do any anchor ranges overlap?

If either is true, skip the entire file with reason incompatible-ops-on-file and record a recommendation for re-analysis. V1 never attempts to compose or sequence these — the interaction is undefined and the risk of silent corruption is too high.

Alternative considered: attempt partial application (apply non-conflicting ops) — rejected: partial application of overlapping anchors produces unpredictable results and leaves the file in a state that cannot be re-analyzed cleanly.

D4. Safety-tier gating — one batch-confirm before any mutation

Invariant #7 requires auto ops to apply without prompt and confirm ops to escalate. The skill reads safety_tier from the report (derived by report_builder via derive_safety_tier) — never recomputes or trusts a model.

Gating sequence:

1. Partition entries into auto+deterministic → mechanical; confirm+deterministic (i.e., delete-range) → approval; generative (always confirm) → approval.
2. Gate confirm-tier FIRST, before any mutation: present a single batch-confirm list (path · category · op · tokens · why) with per-entry opt-out. Visually distinguish delete-range (irreversible) from reversible auto ops.
3. Approved set = all auto entries + approved confirm entries.
4. Apply in that order.

Under sweep (/hygiene sweep = check-then-clean), the confirm gate runs identically — the convenience path does NOT bypass invariant #7.

Audit of approvals is folded into the commit body (no separate .dochygiene/decisions file — avoids new artifact, respects invariant #4).

D5. Git safety — WIP checkpoint, single commit, precise staging, rollback

Pre-flight:

1. Resolve project_root via StateStore.
2. git status --porcelain. Clean → baseline = HEAD. Dirty → auto-create a WIP checkpoint commit of user work FIRST (per invariant #5 "or auto WIP checkpoint"); baseline = that checkpoint. The cleanup commit is still exactly one.
3. Tracked-files-only: untracked candidate docs are skipped and reported (they have no git history to roll back to).

Staging (precise):

• Non-move ops: applier writes the file; skill calls git add <staged_paths>.
• move-to-archive: applier calls git mv (stages both source removal and dest add); skill must NOT git add the dest again (double-add corrupts the index).
• NEVER git add -A or git add . (the report lives in gitignored .dochygiene/ and must never enter the commit).

Single commit via git-context commit-apply --message-stdin (gem 0.4.0). The commit message format:

docs: hygiene cleanup (N edits across M files)

Auto: <count> (move-to-archive ×a, replace-text ×b, dedupe ×c, freeze ×d)
Confirmed: <count> (delete-range ×e, distill ×f, split ×g)
Skipped (re-analysis recommended): <count>

No Co-Authored-By, no AI attribution, no emoji, no trailers (house rule confirmed by user). Do NOT delegate to the full commit skill (its grouping/secret-scan logic may split into multiple commits, violating invariant #5).

Failure handling:

• Hard failure (applier exit 2, git mv fail, write error) → git restore/reset to baseline; abort with a structured error.
• Partial success (applier exit 1, some file batches guard-skipped) is NOT a rollback — commit what applied, report skipped files with re-analysis recommended.

Mid-run last_clean stamp: stamped only after the commit succeeds, to the commit instant (not the run-start instant, so last_clean always points at an actual commit).

D6. Generative entries — live-read freshness, Sonnet subagent, skill writes

Generative entries (distill, split, and any unknown op_type) always require confirm gating. After approval, the skill (not the applier) handles them:

1. Confirm the file still exists (a move-to-archive for the same file earlier in the run would make it gone).
2. Read the live file contents now (not from the report cache — generative entries have no expected_sha256, so the only freshness guarantee is a live read).
3. Dispatch to a Sonnet subagent pointing at workflows/distill.md (LOOP-GUARD — the subagent reads that workflow doc, not SKILL.md, to avoid recursion).
4. Subagent returns new prose (or new-primary + archived-section for split).
5. Skill writes + stages the result.

A file with BOTH a generative entry AND a deterministic entry is treated as having incompatible-ops-on-file (see D3) — v1 never composes; re-analysis recommended.

D7. sweep is command-level sequencing, not a third skill

/hygiene sweep → invoke hygiene-check skill, then invoke hygiene-clean skill, passing --scope/--category to both. This is wired in commands/hygiene.md, not in a third skill or by having clean call check internally.

No double-commit: check writes only to the gitignored .dochygiene/; the single clean commit is the only git commit produced by the sweep.

Alternative considered: a hygiene-sweep skill — rejected: sweep is purely check ; clean with shared flags; a dedicated skill adds ceremony with no new logic.

Risks / Trade-offs

• [delete-range is irreversible] → Confirm-gated (invariant #7); the batch-confirm display MUST visually distinguish deletes from reversible auto ops. Mitigated by the WIP checkpoint (user can always git reset --hard <checkpoint>) but the content is gone from that commit. Documented.
• [move-to-archive creates link orphans] → V1 move only; the next /hygiene check flags new orphans via the broken_reference signal. Documented in the clean summary as a follow-up action.
• [generative second-guard hole] → Generative entries have no expected_sha256; freshness is guaranteed only via the live read at dispatch time. Do not cache the span between the report and the Sonnet call. If the file changes between skill start and Sonnet dispatch, the live read picks up the change (safe); do not add a pre-write hash check (would need a second read, adding complexity with minimal benefit in practice).
• [file with both generative + deterministic entries] → Incompatible-ops-on-file; entire file is skipped, re-analysis recommended. V1 never composes.
• [insert-frontmatter key conflict] → Preserved (skip + frontmatter-key-conflict reason), never overwritten. The idempotent re-derivation in D2 covers this.
• [tracked-files-only] → Untracked candidate docs are skipped and reported. A user who adds a doc without git add-ing it won't see it cleaned. Documented.
• [invariant #8 naming vs. mechanism] → Invariant #8 is titled "mtime guard" but the enforcement is a whole-file content sha256. The applier docstring notes this; the invariant wording is not changed without human approval (META-RULE).

Migration Plan

Additive only — no existing behavior changes. Deploy order:

1. workflows/distill.md (no dependencies beyond existing scripts).
2. skills/hygiene-clean/SKILL.md (depends on distill.md and the already-built patch_applier.py).
3. Integration tests (tests/test_clean_integration.py).
4. commands/hygiene.md update (stub replacement only — replaces the "Phase 4" stubs; no structural change to the command file).
5. CONTEXT.md updates (scripts/CONTEXT.md, skills/hygiene-clean/CONTEXT.md).

Rollback: remove skills/hygiene-clean/SKILL.md, workflows/distill.md, and the integration tests; revert commands/hygiene.md to Phase 3 stubs. The deterministic core, report-builder, state-store, and patch_applier.py are untouched.

Open Questions (resolved — recorded as decisions)

1. Insert-frontmatter at apply time → re-derive at apply time (D2); the deferred hole from add-check is now resolved.
2. Guard mechanism → content hash only (expected_sha256), no mtime pre-filter. Invariant #8 wording preserved (META-RULE).
3. Commit attribution → NONE (git-context house rule, user-confirmed). No Co-Authored-By, no AI attribution, no emoji.
4. Generative freshness → live read at dispatch time is the only guard; no cached span (D6).
5. Sweep is command-level → wired in commands/hygiene.md, not a third skill (D7).
6. Partial success → NOT a rollback; commit what applied, report skipped (D5).
7. Audit trail → folded into the commit body; no .dochygiene/decisions file (respects invariant #4).
8. git mv double-staging → the applier stages git mv both sides; skill must NOT git add the dest path again (D5).