# Lifecycle-aware doc hygiene — spec _Status: Locked design (wayfinder map [#31](https://forgejo.swansoncloud.com/jared/cc-os/issues/31)); assembled 2026-07-14 from decision tickets #32–#48. Extended 2026-07-15 by wayfinder map [#49](https://forgejo.swansoncloud.com/jared/cc-os/issues/49) (assessment-inventory persistence + rulebook refinements; tickets #51–#57, #59)._ _Last updated: 2026-07-15_ Extends os-doc-hygiene from stale/bloat monitoring to lifecycle management: a rulebook assigns every managed file a **lifetime**, `:check`/`:clean` gain delete and extract-then-delete behavior, and a new `:calibrate` skill learns rules per project. Validated against calibration project #1 (cc-os) per the criteria in [Calibration](#calibration-pass-1-cc-os). ADR set: [ADR-0038](../../docs/adr/) (rulebook location; amended 2026-07-15 — rules-file scope includes nominations memory + the extract-index keep rule), ADR-0039 (deletion autonomy tiers; amended 2026-07-15 — map #49 tier interactions), ADR-0040 (no ignore-surface propagation), ADR-0041 (determinism-promotion principle). ## 1. Lifetime taxonomy (#33) Three lifetimes, plus one modifier: - **keep** — indefinite; scanned and reported, never deleted. - **temporary** — age-triggered deletion (see [Temporary tier](#4-temporary-tier-43-48)). - **delete-once-served** — purpose-triggered deletion (see [Served signals](#5-delete-once-served-served-signal-split-43)). - **extract** (modifier on deletion, not a fourth lifetime) — distill durable content before deleting. Extraction reuses existing knowledge routing (#36): repo-durable residue → ADR/CLAUDE.md/docs; cross-repo lessons → SecondBrain via `/os-vault:write`. No new destinations, no "retired specs" pile. ### Extract index (`extracted.md`) — map #49 [#57](https://forgejo.swansoncloud.com/jared/cc-os/issues/57) When an extract-then-delete sends content to the **vault** (the case where the insight physically leaves the repo), the deletion leaves a pointer behind in a per-directory **`extracted.md`** — the index stays where a future reader is already looking. Repo-durable residue (an ADR, a CLAUDE.md line) is already discoverable in-repo and gets **no** index line. Entry format: ```markdown - [[vault: tool/graphify-clustering-behavior]] — why HDBSCAN over-merges doc clusters; read before tuning any graphify clustering params. (extracted from `graphify-clustering-notes.md`, 2026-07-15) ``` - **The `:clean` extract op owns the entry.** The op runs: distill → `/os-vault:write` (returns the note name) → append the pointer line → delete the original — atomically, in `:clean`'s single reviewable commit, so there is no window where the doc is gone but undiscoverable. `os-vault:write` stays repo-agnostic. - **Self-protection is one global rule**: `{ "glob": "**/extracted.md", "lifetime": "keep" }` in the global rulebook — the self-describing filename means zero per-project bookkeeping and the index can never become a deletion candidate anywhere. (A generic `index.md` name was rejected: it collides with existing index/README conventions, and a global `**/index.md → keep` is far too broad.) - **Extract-worthy vs plain-keep — three tests, in order:** 1. **Cited → KEEP.** Anything repo artifacts link to or derive from stays (deleting breaks the provenance chain). 2. **Generalizes AND standalone → EXTRACT.** The insight would change behavior in a *different* repo (tool behavior, methodology — the vault routing test) and nothing in-repo links to it. 3. **Unclear → consult.** The existing mandatory verdict; never guess an extract. - The convention is spec-defined and universal — part of the extract op itself, **not** a `conventions.json` entry (that catalog holds completion-conventions a project opts into via served-signal graduation; the extract index is neither). Distinct from all lifetimes: an **IGNORE surface** — paths the scanner never walks at all (vs `keep` = walked and reported, never deleted). Seed members: `graphify-out/**`, `.dochygiene/**`. The ignore surface is an explicit list, never inferred from `.gitignore` (gitignored ≠ deletable AND gitignored ≠ keepable — #43). ## 2. Rulebook (#34, #38, #40, #44) ### Locations - **Global:** `plugins/os-doc-hygiene/rulebook.json`, resolved relative to plugin scripts. Ships everywhere. - **Per-project override:** repo-root `.dochygiene-rules.json`, committed (matches the `.dochygiene-ignore` precedent; deliberately NOT in gitignored `.cc-os/` — ADR-0038). Both use envelope `{"schema_version": 1, "rules": [...]}`. The project file may additionally carry a top-level `nominations` key (see [Nominations memory](#nominations-memory--map-49-5256) below); the v1 loader ignores unknown top-level keys, so the key is additive to `schema_version` 1. ### Glob dialect gitignore-style via stdlib `glob.translate(recursive=True, include_hidden=True)` (Python ≥ 3.13); patterns repo-root-relative, real `**`; compiled once at load. Directory rules = patterns covering a subtree. ### Precedence (two-axis, source first) project file-rule > project directory-rule > global file-rule > global directory-rule; ties broken by longest pattern, then last-defined. File-level rules override directory rules (#38). The merge is **add-only**: a project neutralizes a global rule by shadowing it with `lifetime: "keep"` — there is no rule-removal mechanism. ### Per-rule fields ```json { "glob": "autoresearch/*/**", "lifetime": "keep|temporary|delete-once-served", "extract": true, "served_when": "free text — hint consumed by the LLM classifier, no trigger DSL", "served_when_path": "openspec/changes/archive/{id}/", "retain_recent": 3, "max_age_days": 3, "confirm": true, "confirmed_by": "human | ", "confirmed_on": "2026-07-14", "source": "clutter-inventory #41", "note": "free-text rationale" } ``` - `retain_recent` default 3; `max_age_days` default **3** (not null-means-forever) — #43 amendments to the #40 schema. - `served_when_path` is the deterministic sibling of `served_when` (#43). - `confirm: true` is an optional always-confirm escape hatch — **human-settable only**; a model-proposed rule may never set it, only ask a human to (#42/#43). - **There is no `propagate_ignore` field.** It was reserved in #40 and dropped by #44: a lifecycle rule never writes into another tool's ignore/config surface (ADR-0040). - Unmatched files get NO lifetime — they flow through existing signals unchanged and become `:calibrate` candidates (**unmatched = unmanaged**). ### Scanner consumption New stdlib `rulebook.py` loader. A directory-rule match **prunes the walk** (files beneath are never opened; one aggregate entry) — this also implements the IGNORE surface for free. A file-rule match attaches a **lifecycle signal** (rule ref + lifetime + served_when) consumed by the classifier as a new signal class (#37). ### Validation Skip-and-warn per invalid/unconfirmed rule — a rule missing `confirmed_by` never acts (#39) but doesn't take scanning down. Hard-fail on unparseable JSON or unknown `schema_version`. `.dochygiene-ignore` remains a hand-authored, human-only escape hatch, unmanaged by the rulebook (#44). ### Nominations memory (`nominations` key) — map #49 (#52/#56) Every judge verdict with an actual answer (keep / temporary / delete-once-served / ignore) persists as a plain rule in `rules` (#51 — keep verdicts are ordinary `lifetime: keep` rules; matched = managed removes them from the calibrate pool, and the glob covers future files for free). The `nominations` key holds only the two residues that cannot be rules: "the answer is no" (`rejected`) and "no answer yet" (`consults`). It **never touches the file filter** — rules alone decide which files are governed. ```json "nominations": { "consults": [ { "glob": "docs/orchestration-audit/*.md", "question": "Are the dated audit tune-up reports a retained audit trail, or disposable once the tune-up lands?", "evidence": "4 files, one per audit run; docs/implementation-status.md references only the latest", "cluster_key": "docs/orchestration-audit::report-#", "asked_on": "2026-07-15" } ], "rejected": [ { "glob": "docs/research/**", "lifetime": "temporary", "why": "findings docs other artifacts link to, not disposable output", "consider_instead": "an extract-to-vault rule for standalone findings docs", "rejected_by": "judge", "judged_on": "2026-07-15" } ] } ``` - **Rejection entry:** `glob`, `lifetime`, `why`, optional `consider_instead` (the judge's amend pointer), `rejected_by: "judge" | "human"`, `judged_on`. Human declines at the §8 rule report also persist as rejections (`rejected_by: "human"`) — otherwise haiku can re-nominate what the human personally declined and the judge won't know. - **Consult entry:** `glob`, `question`, `evidence`, `cluster_key`, `asked_on`. Deliberately **no lifetime** — the uncertainty about purpose is the point. Presence in `consults` = open; no status field. This fixes consult evaporation (verdicts previously died with the run's scratch dir; the same question was re-derived every run). - **Consult exits:** (a) a human answer settles purpose → a normal rule is persisted and the consult entry deleted (the rule supersedes it); (b) "not rule-worthy" → rewritten into `rejected` with the human's why; (c) defer → stays, resurfaces next `:calibrate` run. Consults resurface in `:calibrate` **only** (`:check`/`:clean` unchanged — ADR-0039 boundary) and never filter anything. - **What a rejection blocks — exact glob+lifetime repeats only.** A variant (different lifetime, narrower glob) flows to the judge normally, carrying the past rejection as context. Stale rejections leave by hand-deletion (removals stay HITL, with recorded reasoning). - Exact-path singleton **keep** rules (#51) likewise leave only by hand-deletion — no automated revisit path: an automated revisit would be the one place a machine argues a keep back toward deletion, the exact failure mode the keep tier prevents. - `rulebook.py` stays **nomination-unaware** — its contract is "which rule governs this path"; only `calibrate_helpers` reads the key. The calibrate reader warns on unrecognized nomination fields, mirroring the rules array's unknown-field discipline. - Writes land via the same canonical writer as rules (next subsection); new rejections/consults appear in the §8 rule report but are not individually gated — they are memory, not deletion authority; hand-editing is the escape hatch. ### Canonical ordering — map #49 (#53) **Writer-enforced, no hook.** Every code path that serializes `.dochygiene-rules.json` writes it grouped by lifetime tier (delete-once-served, temporary, keep), sorted by glob within each group; `nominations` after `rules`, `consults` before `rejected` (the pending-action queue reads first), each list glob-sorted. Idempotent; no drift window between a command finishing and a hook firing; hand edits re-canonicalize on the next write. (A post-command hook was considered and rejected as more moving parts for the same result.) ## 3. Deletion semantics and autonomy tiers (#35, #43 — ADR-0039) Delete = **true git deletion in a dedicated hygiene commit**; git history is the archive. No `archive/` dirs or graveyard branches (relocated clutter still distracts AI/search). Rule-backed deletes are **auto** — a rule confirmed per #39 *is* the standing consent; no per-run prompt. The auto/confirm line is drawn on **evidence quality + recoverability**, not file type: | Case | Behavior | |---|---| | IGNORE surface (`graphify-out/**`, `.dochygiene/**`) | never walked | | lifetime `keep` | scanned + reported, never deleted | | tracked + delete rule + clean worktree | **auto** | | tracked + delete rule + DIRTY | **confirm** (an uncommitted diff dies with the file) | | untracked + delete rule | **confirm** (no history to recover from) | | no rule match | unmanaged; existing signals only, never deleted | `clean` verifies tracked+clean **at runtime** (`git ls-files` + dirty check); it never trusts the rule's word for it. No `recoverable_via` field, no "regenerate" class. ## 4. Temporary tier (#43, #48) **retain-recent-N + age, not age alone.** Defaults `retain_recent: 3`, `max_age_days: 3` (both per-rule overridable). The newest 3 entries matching a rule are always kept regardless of age (they show current trajectory); an entry ranked 4th or older is deleted once it exceeds `max_age_days`. 90-day windows were rejected as far too slow. **Retention unit = the rule's match entry**: a file for file rules, a run *directory* for directory rules ("3 most recent autoresearch runs", not "3 most recent files inside one run"). **Age = git commit time, falling back to filesystem mtime for untracked files.** mtime-alone was rejected (clone/branch-switch resets every mtime, putting the rulebook to sleep on fresh checkouts); the objection doesn't apply to the fallback, since untracked files don't exist in a fresh clone. No per-rule `age_source` field (#48). **Untracked directory entries** use the directory inode's own mtime (one stat), NOT a recursive max-mtime walk — the tier's failure mode is self-healing (a spuriously bumped mtime merely delays deletion one round), so the cheap signal suffices (#48). ## 5. delete-once-served: served-signal split (#43) Two slots, split by evidence quality: ```json { "glob": "openspec/changes/*/", "lifetime": "delete-once-served", "served_when_path": "openspec/changes/archive/{id}/" } ``` Scanner **proves** the served condition → may delete silently under the tier matrix. ```json { "glob": "docs/plans/*.md", "lifetime": "delete-once-served", "served_when": "the effort this plan describes has shipped" } ``` Classifier **judges** the condition → **always forced to confirm**, regardless of tracked status. The LLM may propose; it may never silently destroy on a hunch. Example of the boundary: `autoresearch/*/` can be auto (a concluded run is provable from the filesystem); `PRD.md` cannot (purpose-triggered — "did this ship?" is a judgment; and it is NOT temporary, since age-keying would delete the PRD of a feature not yet built — #45 Addendum 2). ## 6. Determinism promotion (#43 item 7, #47 — ADR-0041) **Design principle: hygiene drives projects toward structurally-obvious maintenance.** When a rule's served signal is subjective (classifier-judged), the tool does not merely downgrade it to confirm — it names the subjectivity and recommends a concrete structural convention that would graduate the rule to `served_when_path` and make it silent. Confirm-fatigue is the incentive to fix the convention. ### Completion-conventions catalog `plugins/os-doc-hygiene/conventions.json` — global-only, machine-readable so the deterministic pipeline can emit nudges without an LLM. No per-project override: the catalog only recommends; adoption lands in the project's own rulebook. Each entry: name, what it proves, the `served_when_path` / frontmatter template a rule graduates to, and a one-line human pitch. v1 contents — exactly two conventions: - **archive-bucket** — "done" = the file moved into a sibling `archive/` dir (`docs/plans/x.md` → `docs/plans/archive/x.md`); graduates a rule to `served_when_path: /archive/{name}`. Precedent: openspec changes. - **status-frontmatter** — "done" = a `status: shipped|done` frontmatter key; file stays put; scanner reads frontmatter. Successor-artifact checks stay in the fog until a calibration pass demands one. ### Nudge surfacing (split by capability) - **`:check`** names promotion candidates in every report (deterministic, recurring — the report gains a promotion-candidates section). - **`:calibrate`** may go further and DRAFT the adoption — the graduated rule plus the file moves — for human approval. It proposes, never applies unasked. ## 7. Pipeline integration (#37) Lifecycle categorization is a **new signal class** in the existing scanner/classifier pipeline; `delete` and `extract-then-delete` are **new op types** in the clean report schema. The only new skill is **`os-doc-hygiene:calibrate`**. ## 8. The :calibrate protocol (#42, #39, #45) The learn-new-rules loop, run per project: 1. **Cluster-and-sample** over unmatched files (unmatched = unmanaged = the candidate pool). Clustering exists precisely so rules are written over the cluster, not one member. 2. **Nomination (cheap model):** haiku nominates a bare glob + lifetime per cluster — constrained to produce *patterns*, never exact-instance globs. 3. **Nomination intake filter (deterministic — `NominationIntakeFilter` in `calibrate_helpers.py`):** a nomination whose glob+lifetime exactly equals a `rejected` entry (§2 Nominations memory) is dropped before the judge, logged in the run summary. Survivors are annotated with every *related* rejection — related = the two globs' match sets intersect on the current shortlist (deterministic, from the scan). Annotations plus all open consults enter the judge prompt as a "Nominations memory" input section. 4. **Judgment (strong model):** one batched Opus/Fable judge gathers its own evidence and authors final rule entries (#39: weak-model discoveries need strong-model confirmation). Verdicts: **confirm / reject / amend / consult** — consult is mandatory when an artifact's purpose is unclear (regenerable ≠ removable). 5. **Rule report to the human — before any rule is persisted.** Per proposed rule, the report shows: 1. the **glob verbatim**, exactly as it would be persisted; 2. every path it currently matches (or a capped sample + total count); 3. **the boundary — near-miss paths it does NOT match** (this caught a real bug during #45: `autoresearch/classic-*/` silently missing `autoresearch/improve-260710-1057/`); 4. lifetime + behavior tier (auto vs confirm); 5. a plain-language why — what the artifact is and why it's clutter. The human reviews patterns and examples, not JSON schema. 6. **Persistence:** project rules land on judge confirmation; **global-rulebook writes are human-gated** (a cross-repo write into cc-os). Rule removals are HITL-only, with recorded reasoning. Every settled verdict persists as a plain rule — judge `keep` verdicts become ordinary `lifetime: keep` rules, including exact-path singletons (#51). Human declines persist as `rejected` entries (`rejected_by: "human"`); open `consult` verdicts persist to `nominations.consults`, deduped by glob at write time (§2 Nominations memory). 7. **Retest loop:** stop at <2 new rules OR <10% unmatched shrink; hard cap 3 rounds. Seed intake: the #41 clutter-inventory seed candidates enter at judge intake — **full intake for every run after calibration pass #1** (see the one-off carve-out below). ### Rule-quality tests the report enforces - **The rule is the CLASS, never the PATH.** A glob may hardcode a name that *recurs by convention* (`PRD.md`, `HANDOFF-*.md`, `migration-report.md`); it may NOT hardcode an identifier *unique to one instance* (a run-id, hash, or bare timestamp). A rule matching one file today is fine; a rule that can only EVER match one file is a failed generalization — flag loudly, never silently persist. **Keep-tier relaxation (map #49 #51):** exact-path/instance globs ARE allowed for `lifetime: keep` entries only — this test exists to prevent bad DELETION rules, and a singleton keep (e.g. `docs/research/clutter-pattern-inventory.md`) merely protects. Instance globs stay forbidden for `temporary`/`delete-once-served`. - **Glob-breadth tie-breaker: prefer the NARROWER glob.** Too-narrow fails safe (leaves clutter; the recurring pass catches it next round — self- healing). Too-broad fails dangerous (deletes a keeper — not self-healing). Readability beats cleverness when the cost is one missed round. - **Enumerate siblings — never widen to the container.** When the near-miss boundary check reveals sibling artifacts a glob misses, the fix is to ENUMERATE the conventional prefixes as separate rule entries (`autoresearch/classic-*/` + `autoresearch/improve-*/`), never to widen to the container (`autoresearch/*/`, `autoresearch/**`). A container-claiming glob mortgages the directory's entire future: nothing keep-worthy can ever live there without a counter-rule (a future `autoresearch/methodology-notes.md` would be claimed by a deletion rule). A sibling flavor that can't be named yet fails safe — it gets its own entry next pass, the same self-healing property the narrower-glob tie-breaker relies on. Container globs are justified ONLY when the directory is wholly machine-owned (`plugins/*/.pytest_cache/`), where nothing keep-worthy can appear inside. ### Test coverage for the map-#49 additions (#59) - `NominationIntakeFilter` and the canonical-writer extension get **unit tests only** (pure deterministic logic — invariant #6). Fixtures assert exact-repeat drops, match-set-intersection relatedness annotations, and round-trip ordering + unknown-field warnings. - The judge's "Nominations memory" context and the consult resurfacing loop get **no scenario harness** — both are human-gated downstream, so no silent-failure path exists a harness would uniquely catch. Mitigation is one worked example each (judge.md input section; calibrate SKILL.md), with production IRL session audits as the next signal; IRL evidence of the judge ignoring rejection context is what triggers harness design (reading `~/Documents/SecondBrain/howto/running-autoresearch-skill-evals.md` first, per eval discipline). ## 9. Calibration pass #1: cc-os (#45) Project: **cc-os** — proves the protocol *works* before testing whether it *generalizes*; the self-referential risk is accepted and paid for by the validation criteria carrying the weight. - **Precision hard gate:** the pass FAILS if any rule persisted to the rulebook has a glob matching a protected path — regardless of behavior tier (a confirm gate is a safety property of the human sitting there, not of the protocol). Exploration-time consult verdicts on protected paths are FREE. Protected set (fixed before the pass, human-edited, never revised after): eval `scenarios/`/`scenarios-reserve/`/`fixture/`/judge-rubric.md; `openspec/specs/`; `docs/adr/**`; mirrored `.claude/`/`.codex/`/`.pi/` skill dirs; `CLAUDE.md`; plugin source. - **Recall floor: 8 of the 10 cc-os rows of the #41 inventory**, with 4 mandatory (missing any fails the pass): `autoresearch//`, `HANDOFF-*.md`, `docs/adr/migration-report.md`, `.dochygiene/report.{json,md}`. `graphify-out/` is **void, not a miss** (it is IGNORE surface per #43). The recall floor is a grading bar, not runtime behavior — err-toward-keeping comes from the hard gate + consult. - **Seed hold-out (one-off for pass #1 ONLY):** the cc-os rows of #41 are the sealed answer key and are withheld from judge intake — otherwise the pass is an open-book exam. This deliberately deviates from #42; **every later run uses full seed intake.** Do not mistake the carve-out for a permanent property. - Novel matches beyond the answer key are expected and human-spot-checked — a wrong novel match triggers rule adjustment + a retest round, not failure. - A do-nothing pass cannot pass: the recall floor makes the pass falsifiable in the finding direction. ## Out of scope for this design Shipping the recurring cross-project categorize-and-learn skill (charted as fog on map #31). Ignore-surface propagation into other tools' config (rejected — ADR-0040); if disposable files polluting the knowledge graph later proves painful, the fix belongs to graphify or os-vault:onboard-project.