cc-os/plugins/os-doc-hygiene/lifecycle-spec.md

Lifecycle-aware doc hygiene — spec

Status: Locked design (wayfinder map #31); assembled 2026-07-14 from decision tickets #32–#48. Extended 2026-07-15 by wayfinder map #49 (assessment-inventory persistence + rulebook refinements; tickets #51–#57, #59). Last updated: 2026-07-15

Extends os-doc-hygiene from stale/bloat monitoring to lifecycle management: a rulebook assigns every managed file a lifetime, :check/:clean gain delete and extract-then-delete behavior, and a new :calibrate skill learns rules per project. Validated against calibration project #1 (cc-os) per the criteria in Calibration.

ADR set: ADR-0038 (rulebook location; amended 2026-07-15 — rules-file scope includes nominations memory + the extract-index keep rule), ADR-0039 (deletion autonomy tiers; amended 2026-07-15 — map #49 tier interactions), ADR-0040 (no ignore-surface propagation), ADR-0041 (determinism-promotion principle).

1. Lifetime taxonomy (#33)

Three lifetimes, plus one modifier:

• keep — indefinite; scanned and reported, never deleted.
• temporary — age-triggered deletion (see Temporary tier).
• delete-once-served — purpose-triggered deletion (see Served signals).
• extract (modifier on deletion, not a fourth lifetime) — distill durable content before deleting. Extraction reuses existing knowledge routing (#36): repo-durable residue → ADR/CLAUDE.md/docs; cross-repo lessons → SecondBrain via /os-vault:write. No new destinations, no "retired specs" pile.

Extract index (extracted.md) — map #49 #57

When an extract-then-delete sends content to the vault (the case where the insight physically leaves the repo), the deletion leaves a pointer behind in a per-directory extracted.md — the index stays where a future reader is already looking. Repo-durable residue (an ADR, a CLAUDE.md line) is already discoverable in-repo and gets no index line. Entry format:

- [[vault: tool/graphify-clustering-behavior]] — why HDBSCAN over-merges doc
  clusters; read before tuning any graphify clustering params.
  (extracted from `graphify-clustering-notes.md`, 2026-07-15)

• The :clean extract op owns the entry. The op runs: distill → /os-vault:write (returns the note name) → append the pointer line → delete the original — atomically, in :clean's single reviewable commit, so there is no window where the doc is gone but undiscoverable. os-vault:write stays repo-agnostic.
• Self-protection is one global rule: { "glob": "**/extracted.md", "lifetime": "keep" } in the global rulebook — the self-describing filename means zero per-project bookkeeping and the index can never become a deletion candidate anywhere. (A generic index.md name was rejected: it collides with existing index/README conventions, and a global **/index.md → keep is far too broad.)
• Extract-worthy vs plain-keep — three tests, in order:
  1. Cited → KEEP. Anything repo artifacts link to or derive from stays (deleting breaks the provenance chain).
  2. Generalizes AND standalone → EXTRACT. The insight would change behavior in a different repo (tool behavior, methodology — the vault routing test) and nothing in-repo links to it.
  3. Unclear → consult. The existing mandatory verdict; never guess an extract.
• The convention is spec-defined and universal — part of the extract op itself, not a conventions.json entry (that catalog holds completion-conventions a project opts into via served-signal graduation; the extract index is neither).

Distinct from all lifetimes: an IGNORE surface — paths the scanner never walks at all (vs keep = walked and reported, never deleted). Seed members: graphify-out/**, .dochygiene/**. The ignore surface is an explicit list, never inferred from .gitignore (gitignored ≠ deletable AND gitignored ≠ keepable — #43).

2. Rulebook (#34, #38, #40, #44)

Locations

• Global: plugins/os-doc-hygiene/rulebook.json, resolved relative to plugin scripts. Ships everywhere.
• Per-project override: repo-root .dochygiene-rules.json, committed (matches the .dochygiene-ignore precedent; deliberately NOT in gitignored .cc-os/ — ADR-0038).

Both use envelope {"schema_version": 1, "rules": [...]}. The project file may additionally carry a top-level nominations key (see Nominations memory below); the v1 loader ignores unknown top-level keys, so the key is additive to schema_version 1.

Glob dialect

gitignore-style via stdlib glob.translate(recursive=True, include_hidden=True) (Python ≥ 3.13); patterns repo-root-relative, real **; compiled once at load. Directory rules = patterns covering a subtree.

Precedence (two-axis, source first)

project file-rule > project directory-rule > global file-rule > global directory-rule; ties broken by longest pattern, then last-defined. File-level rules override directory rules (#38). The merge is add-only: a project neutralizes a global rule by shadowing it with lifetime: "keep" — there is no rule-removal mechanism.

Per-rule fields

{
  "glob": "autoresearch/*/**",
  "lifetime": "keep|temporary|delete-once-served",
  "extract": true,
  "served_when": "free text — hint consumed by the LLM classifier, no trigger DSL",
  "served_when_path": "openspec/changes/archive/{id}/",
  "retain_recent": 3,
  "max_age_days": 3,
  "confirm": true,
  "confirmed_by": "human | <strong-model-id>",
  "confirmed_on": "2026-07-14",
  "source": "clutter-inventory #41",
  "note": "free-text rationale"
}

• retain_recent default 3; max_age_days default 3 (not null-means-forever) — #43 amendments to the #40 schema.
• served_when_path is the deterministic sibling of served_when (#43).
• confirm: true is an optional always-confirm escape hatch — human-settable only; a model-proposed rule may never set it, only ask a human to (#42/#43).
• There is no propagate_ignore field. It was reserved in #40 and dropped by #44: a lifecycle rule never writes into another tool's ignore/config surface (ADR-0040).
• Unmatched files get NO lifetime — they flow through existing signals unchanged and become :calibrate candidates (unmatched = unmanaged).

Scanner consumption

New stdlib rulebook.py loader. A directory-rule match prunes the walk (files beneath are never opened; one aggregate entry) — this also implements the IGNORE surface for free. A file-rule match attaches a lifecycle signal (rule ref + lifetime + served_when) consumed by the classifier as a new signal class (#37).

Validation

Skip-and-warn per invalid/unconfirmed rule — a rule missing confirmed_by never acts (#39) but doesn't take scanning down. Hard-fail on unparseable JSON or unknown schema_version.

.dochygiene-ignore remains a hand-authored, human-only escape hatch, unmanaged by the rulebook (#44).

Nominations memory (nominations key) — map #49 (#52/#56)

Every judge verdict with an actual answer (keep / temporary / delete-once-served / ignore) persists as a plain rule in rules (#51 — keep verdicts are ordinary lifetime: keep rules; matched = managed removes them from the calibrate pool, and the glob covers future files for free). The nominations key holds only the two residues that cannot be rules: "the answer is no" (rejected) and "no answer yet" (consults). It never touches the file filter — rules alone decide which files are governed.

"nominations": {
  "consults": [
    {
      "glob": "docs/orchestration-audit/*.md",
      "question": "Are the dated audit tune-up reports a retained audit trail, or disposable once the tune-up lands?",
      "evidence": "4 files, one per audit run; docs/implementation-status.md references only the latest",
      "cluster_key": "docs/orchestration-audit::report-#",
      "asked_on": "2026-07-15"
    }
  ],
  "rejected": [
    {
      "glob": "docs/research/**",
      "lifetime": "temporary",
      "why": "findings docs other artifacts link to, not disposable output",
      "consider_instead": "an extract-to-vault rule for standalone findings docs",
      "rejected_by": "judge",
      "judged_on": "2026-07-15"
    }
  ]
}

• Rejection entry: glob, lifetime, why, optional consider_instead (the judge's amend pointer), rejected_by: "judge" | "human", judged_on. Human declines at the §8 rule report also persist as rejections (rejected_by: "human") — otherwise haiku can re-nominate what the human personally declined and the judge won't know.
• Consult entry: glob, question, evidence, cluster_key, asked_on. Deliberately no lifetime — the uncertainty about purpose is the point. Presence in consults = open; no status field. This fixes consult evaporation (verdicts previously died with the run's scratch dir; the same question was re-derived every run).
• Consult exits: (a) a human answer settles purpose → a normal rule is persisted and the consult entry deleted (the rule supersedes it); (b) "not rule-worthy" → rewritten into rejected with the human's why; (c) defer → stays, resurfaces next :calibrate run. Consults resurface in :calibrate only (:check/:clean unchanged — ADR-0039 boundary) and never filter anything.
• What a rejection blocks — exact glob+lifetime repeats only. A variant (different lifetime, narrower glob) flows to the judge normally, carrying the past rejection as context. Stale rejections leave by hand-deletion (removals stay HITL, with recorded reasoning).
• Exact-path singleton keep rules (#51) likewise leave only by hand-deletion — no automated revisit path: an automated revisit would be the one place a machine argues a keep back toward deletion, the exact failure mode the keep tier prevents.
• rulebook.py stays nomination-unaware — its contract is "which rule governs this path"; only calibrate_helpers reads the key. The calibrate reader warns on unrecognized nomination fields, mirroring the rules array's unknown-field discipline.
• Writes land via the same canonical writer as rules (next subsection); new rejections/consults appear in the §8 rule report but are not individually gated — they are memory, not deletion authority; hand-editing is the escape hatch.

Canonical ordering — map #49 (#53)

Writer-enforced, no hook. Every code path that serializes .dochygiene-rules.json writes it grouped by lifetime tier (delete-once-served, temporary, keep), sorted by glob within each group; nominations after rules, consults before rejected (the pending-action queue reads first), each list glob-sorted. Idempotent; no drift window between a command finishing and a hook firing; hand edits re-canonicalize on the next write. (A post-command hook was considered and rejected as more moving parts for the same result.)

3. Deletion semantics and autonomy tiers (#35, #43 — ADR-0039)

Delete = true git deletion in a dedicated hygiene commit; git history is the archive. No archive/ dirs or graveyard branches (relocated clutter still distracts AI/search).

Rule-backed deletes are auto — a rule confirmed per #39 is the standing consent; no per-run prompt. The auto/confirm line is drawn on evidence quality + recoverability, not file type:

Case	Behavior
IGNORE surface (graphify-out/**, .dochygiene/**)	never walked
lifetime keep	scanned + reported, never deleted
tracked + delete rule + clean worktree	auto
tracked + delete rule + DIRTY	confirm (an uncommitted diff dies with the file)
untracked + delete rule	confirm (no history to recover from)
no rule match	unmanaged; existing signals only, never deleted

clean verifies tracked+clean at runtime (git ls-files + dirty check); it never trusts the rule's word for it. No recoverable_via field, no "regenerate" class.

4. Temporary tier (#43, #48)

retain-recent-N + age, not age alone. Defaults retain_recent: 3, max_age_days: 3 (both per-rule overridable). The newest 3 entries matching a rule are always kept regardless of age (they show current trajectory); an entry ranked 4th or older is deleted once it exceeds max_age_days. 90-day windows were rejected as far too slow.

Retention unit = the rule's match entry: a file for file rules, a run directory for directory rules ("3 most recent autoresearch runs", not "3 most recent files inside one run").

Age = git commit time, falling back to filesystem mtime for untracked files. mtime-alone was rejected (clone/branch-switch resets every mtime, putting the rulebook to sleep on fresh checkouts); the objection doesn't apply to the fallback, since untracked files don't exist in a fresh clone. No per-rule age_source field (#48).

Untracked directory entries use the directory inode's own mtime (one stat), NOT a recursive max-mtime walk — the tier's failure mode is self-healing (a spuriously bumped mtime merely delays deletion one round), so the cheap signal suffices (#48).

5. delete-once-served: served-signal split (#43)

Two slots, split by evidence quality:

{ "glob": "openspec/changes/*/", "lifetime": "delete-once-served",
  "served_when_path": "openspec/changes/archive/{id}/" }

Scanner proves the served condition → may delete silently under the tier matrix.

{ "glob": "docs/plans/*.md", "lifetime": "delete-once-served",
  "served_when": "the effort this plan describes has shipped" }

Classifier judges the condition → always forced to confirm, regardless of tracked status. The LLM may propose; it may never silently destroy on a hunch.

Example of the boundary: autoresearch/*/ can be auto (a concluded run is provable from the filesystem); PRD.md cannot (purpose-triggered — "did this ship?" is a judgment; and it is NOT temporary, since age-keying would delete the PRD of a feature not yet built — #45 Addendum 2).

6. Determinism promotion (#43 item 7, #47 — ADR-0041)

Design principle: hygiene drives projects toward structurally-obvious maintenance. When a rule's served signal is subjective (classifier-judged), the tool does not merely downgrade it to confirm — it names the subjectivity and recommends a concrete structural convention that would graduate the rule to served_when_path and make it silent. Confirm-fatigue is the incentive to fix the convention.

Completion-conventions catalog

plugins/os-doc-hygiene/conventions.json — global-only, machine-readable so the deterministic pipeline can emit nudges without an LLM. No per-project override: the catalog only recommends; adoption lands in the project's own rulebook. Each entry: name, what it proves, the served_when_path / frontmatter template a rule graduates to, and a one-line human pitch.

v1 contents — exactly two conventions:

• archive-bucket — "done" = the file moved into a sibling archive/ dir (docs/plans/x.md → docs/plans/archive/x.md); graduates a rule to served_when_path: <dir>/archive/{name}. Precedent: openspec changes.
• status-frontmatter — "done" = a status: shipped|done frontmatter key; file stays put; scanner reads frontmatter.

Successor-artifact checks stay in the fog until a calibration pass demands one.

Nudge surfacing (split by capability)

• :check names promotion candidates in every report (deterministic, recurring — the report gains a promotion-candidates section).
• :calibrate may go further and DRAFT the adoption — the graduated rule plus the file moves — for human approval. It proposes, never applies unasked.

7. Pipeline integration (#37)

Lifecycle categorization is a new signal class in the existing scanner/classifier pipeline; delete and extract-then-delete are new op types in the clean report schema. The only new skill is os-doc-hygiene:calibrate.

8. The :calibrate protocol (#42, #39, #45)

The learn-new-rules loop, run per project:

1. Cluster-and-sample over unmatched files (unmatched = unmanaged = the candidate pool). Clustering exists precisely so rules are written over the cluster, not one member.
2. Nomination (cheap model): haiku nominates a bare glob + lifetime per cluster — constrained to produce patterns, never exact-instance globs.
3. Nomination intake filter (deterministic — NominationIntakeFilter in calibrate_helpers.py): a nomination whose glob+lifetime exactly equals a rejected entry (§2 Nominations memory) is dropped before the judge, logged in the run summary. Survivors are annotated with every related rejection — related = the two globs' match sets intersect on the current shortlist (deterministic, from the scan). Annotations plus all open consults enter the judge prompt as a "Nominations memory" input section.
4. Judgment (strong model): one batched Opus/Fable judge gathers its own evidence and authors final rule entries (#39: weak-model discoveries need strong-model confirmation). Verdicts: confirm / reject / amend / consult — consult is mandatory when an artifact's purpose is unclear (regenerable ≠ removable).
5. Rule report to the human — before any rule is persisted. Per proposed rule, the report shows:
   1. the glob verbatim, exactly as it would be persisted;
   2. every path it currently matches (or a capped sample + total count);
   3. the boundary — near-miss paths it does NOT match (this caught a real bug during #45: autoresearch/classic-*/ silently missing autoresearch/improve-260710-1057/);
   4. lifetime + behavior tier (auto vs confirm);
   5. a plain-language why — what the artifact is and why it's clutter. The human reviews patterns and examples, not JSON schema.
6. Persistence: project rules land on judge confirmation; global-rulebook writes are human-gated (a cross-repo write into cc-os). Rule removals are HITL-only, with recorded reasoning. Every settled verdict persists as a plain rule — judge keep verdicts become ordinary lifetime: keep rules, including exact-path singletons (#51). Human declines persist as rejected entries (rejected_by: "human"); open consult verdicts persist to nominations.consults, deduped by glob at write time (§2 Nominations memory).
7. Retest loop: stop at <2 new rules OR <10% unmatched shrink; hard cap 3 rounds.

Seed intake: the #41 clutter-inventory seed candidates enter at judge intake — full intake for every run after calibration pass #1 (see the one-off carve-out below).

Rule-quality tests the report enforces

• The rule is the CLASS, never the PATH. A glob may hardcode a name that recurs by convention (PRD.md, HANDOFF-*.md, migration-report.md); it may NOT hardcode an identifier unique to one instance (a run-id, hash, or bare timestamp). A rule matching one file today is fine; a rule that can only EVER match one file is a failed generalization — flag loudly, never silently persist. Keep-tier relaxation (map #49 #51): exact-path/instance globs ARE allowed for lifetime: keep entries only — this test exists to prevent bad DELETION rules, and a singleton keep (e.g. docs/research/clutter-pattern-inventory.md) merely protects. Instance globs stay forbidden for temporary/delete-once-served.
• Glob-breadth tie-breaker: prefer the NARROWER glob. Too-narrow fails safe (leaves clutter; the recurring pass catches it next round — self- healing). Too-broad fails dangerous (deletes a keeper — not self-healing). Readability beats cleverness when the cost is one missed round.
• Enumerate siblings — never widen to the container. When the near-miss boundary check reveals sibling artifacts a glob misses, the fix is to ENUMERATE the conventional prefixes as separate rule entries (autoresearch/classic-*/ + autoresearch/improve-*/), never to widen to the container (autoresearch/*/, autoresearch/**). A container-claiming glob mortgages the directory's entire future: nothing keep-worthy can ever live there without a counter-rule (a future autoresearch/methodology-notes.md would be claimed by a deletion rule). A sibling flavor that can't be named yet fails safe — it gets its own entry next pass, the same self-healing property the narrower-glob tie-breaker relies on. Container globs are justified ONLY when the directory is wholly machine-owned (plugins/*/.pytest_cache/), where nothing keep-worthy can appear inside.

Test coverage for the map-#49 additions (#59)

• NominationIntakeFilter and the canonical-writer extension get unit tests only (pure deterministic logic — invariant #6). Fixtures assert exact-repeat drops, match-set-intersection relatedness annotations, and round-trip ordering + unknown-field warnings.
• The judge's "Nominations memory" context and the consult resurfacing loop get no scenario harness — both are human-gated downstream, so no silent-failure path exists a harness would uniquely catch. Mitigation is one worked example each (judge.md input section; calibrate SKILL.md), with production IRL session audits as the next signal; IRL evidence of the judge ignoring rejection context is what triggers harness design (reading ~/Documents/SecondBrain/howto/running-autoresearch-skill-evals.md first, per eval discipline).

9. Calibration pass #1: cc-os (#45)

Project: cc-os — proves the protocol works before testing whether it generalizes; the self-referential risk is accepted and paid for by the validation criteria carrying the weight.

• Precision hard gate: the pass FAILS if any rule persisted to the rulebook has a glob matching a protected path — regardless of behavior tier (a confirm gate is a safety property of the human sitting there, not of the protocol). Exploration-time consult verdicts on protected paths are FREE. Protected set (fixed before the pass, human-edited, never revised after): eval scenarios//scenarios-reserve//fixture//judge-rubric.md; openspec/specs/; docs/adr/**; mirrored .claude//.codex//.pi/ skill dirs; CLAUDE.md; plugin source.
• Recall floor: 8 of the 10 cc-os rows of the #41 inventory, with 4 mandatory (missing any fails the pass): autoresearch/<run-id>/, HANDOFF-*.md, docs/adr/migration-report.md, .dochygiene/report.{json,md}. graphify-out/ is void, not a miss (it is IGNORE surface per #43). The recall floor is a grading bar, not runtime behavior — err-toward-keeping comes from the hard gate + consult.
• Seed hold-out (one-off for pass #1 ONLY): the cc-os rows of #41 are the sealed answer key and are withheld from judge intake — otherwise the pass is an open-book exam. This deliberately deviates from #42; every later run uses full seed intake. Do not mistake the carve-out for a permanent property.
• Novel matches beyond the answer key are expected and human-spot-checked — a wrong novel match triggers rule adjustment + a retest round, not failure.
• A do-nothing pass cannot pass: the recall floor makes the pass falsifiable in the finding direction.

Out of scope for this design

Shipping the recurring cross-project categorize-and-learn skill (charted as fog on map #31). Ignore-surface propagation into other tools' config (rejected — ADR-0040); if disposable files polluting the knowledge graph later proves painful, the fix belongs to graphify or os-vault:onboard-project.