14 KiB
Lifecycle-aware doc hygiene — spec
Status: Locked design (wayfinder map #31); assembled 2026-07-14 from decision tickets #32–#48. Last updated: 2026-07-14
Extends os-doc-hygiene from stale/bloat monitoring to lifecycle management: a
rulebook assigns every managed file a lifetime, :check/:clean gain
delete and extract-then-delete behavior, and a new :calibrate skill learns
rules per project. Validated against calibration project #1 (cc-os) per the
criteria in Calibration.
ADR set: ADR-0038 (rulebook location), ADR-0039 (deletion autonomy tiers), ADR-0040 (no ignore-surface propagation), ADR-0041 (determinism-promotion principle).
1. Lifetime taxonomy (#33)
Three lifetimes, plus one modifier:
- keep — indefinite; scanned and reported, never deleted.
- temporary — age-triggered deletion (see Temporary tier).
- delete-once-served — purpose-triggered deletion (see Served signals).
- extract (modifier on deletion, not a fourth lifetime) — distill durable
content before deleting. Extraction reuses existing knowledge routing (#36):
repo-durable residue → ADR/CLAUDE.md/docs; cross-repo lessons → SecondBrain
via
/os-vault:write. No new destinations, no "retired specs" pile.
Distinct from all lifetimes: an IGNORE surface — paths the scanner never
walks at all (vs keep = walked and reported, never deleted). Seed members:
graphify-out/**, .dochygiene/**. The ignore surface is an explicit list,
never inferred from .gitignore (gitignored ≠ deletable AND gitignored ≠
keepable — #43).
2. Rulebook (#34, #38, #40, #44)
Locations
- Global:
plugins/os-doc-hygiene/rulebook.json, resolved relative to plugin scripts. Ships everywhere. - Per-project override: repo-root
.dochygiene-rules.json, committed (matches the.dochygiene-ignoreprecedent; deliberately NOT in gitignored.cc-os/— ADR-0038).
Both use envelope {"schema_version": 1, "rules": [...]}.
Glob dialect
gitignore-style via stdlib glob.translate(recursive=True, include_hidden=True) (Python ≥ 3.13); patterns repo-root-relative, real **;
compiled once at load. Directory rules = patterns covering a subtree.
Precedence (two-axis, source first)
project file-rule > project directory-rule > global file-rule > global
directory-rule; ties broken by longest pattern, then last-defined. File-level
rules override directory rules (#38). The merge is add-only: a project
neutralizes a global rule by shadowing it with lifetime: "keep" — there is
no rule-removal mechanism.
Per-rule fields
{
"glob": "autoresearch/*/**",
"lifetime": "keep|temporary|delete-once-served",
"extract": true,
"served_when": "free text — hint consumed by the LLM classifier, no trigger DSL",
"served_when_path": "openspec/changes/archive/{id}/",
"retain_recent": 3,
"max_age_days": 3,
"confirm": true,
"confirmed_by": "human | <strong-model-id>",
"confirmed_on": "2026-07-14",
"source": "clutter-inventory #41",
"note": "free-text rationale"
}
retain_recentdefault 3;max_age_daysdefault 3 (not null-means-forever) — #43 amendments to the #40 schema.served_when_pathis the deterministic sibling ofserved_when(#43).confirm: trueis an optional always-confirm escape hatch — human-settable only; a model-proposed rule may never set it, only ask a human to (#42/#43).- There is no
propagate_ignorefield. It was reserved in #40 and dropped by #44: a lifecycle rule never writes into another tool's ignore/config surface (ADR-0040). - Unmatched files get NO lifetime — they flow through existing signals
unchanged and become
:calibratecandidates (unmatched = unmanaged).
Scanner consumption
New stdlib rulebook.py loader. A directory-rule match prunes the walk
(files beneath are never opened; one aggregate entry) — this also implements
the IGNORE surface for free. A file-rule match attaches a lifecycle signal
(rule ref + lifetime + served_when) consumed by the classifier as a new signal
class (#37).
Validation
Skip-and-warn per invalid/unconfirmed rule — a rule missing confirmed_by
never acts (#39) but doesn't take scanning down. Hard-fail on unparseable JSON
or unknown schema_version.
.dochygiene-ignore remains a hand-authored, human-only escape hatch,
unmanaged by the rulebook (#44).
3. Deletion semantics and autonomy tiers (#35, #43 — ADR-0039)
Delete = true git deletion in a dedicated hygiene commit; git history is
the archive. No archive/ dirs or graveyard branches (relocated clutter still
distracts AI/search).
Rule-backed deletes are auto — a rule confirmed per #39 is the standing consent; no per-run prompt. The auto/confirm line is drawn on evidence quality + recoverability, not file type:
| Case | Behavior |
|---|---|
IGNORE surface (graphify-out/**, .dochygiene/**) |
never walked |
lifetime keep |
scanned + reported, never deleted |
| tracked + delete rule + clean worktree | auto |
| tracked + delete rule + DIRTY | confirm (an uncommitted diff dies with the file) |
| untracked + delete rule | confirm (no history to recover from) |
| no rule match | unmanaged; existing signals only, never deleted |
clean verifies tracked+clean at runtime (git ls-files + dirty check);
it never trusts the rule's word for it. No recoverable_via field, no
"regenerate" class.
4. Temporary tier (#43, #48)
retain-recent-N + age, not age alone. Defaults retain_recent: 3,
max_age_days: 3 (both per-rule overridable). The newest 3 entries matching a
rule are always kept regardless of age (they show current trajectory); an
entry ranked 4th or older is deleted once it exceeds max_age_days. 90-day
windows were rejected as far too slow.
Retention unit = the rule's match entry: a file for file rules, a run directory for directory rules ("3 most recent autoresearch runs", not "3 most recent files inside one run").
Age = git commit time, falling back to filesystem mtime for untracked
files. mtime-alone was rejected (clone/branch-switch resets every mtime,
putting the rulebook to sleep on fresh checkouts); the objection doesn't apply
to the fallback, since untracked files don't exist in a fresh clone. No
per-rule age_source field (#48).
Untracked directory entries use the directory inode's own mtime (one stat), NOT a recursive max-mtime walk — the tier's failure mode is self-healing (a spuriously bumped mtime merely delays deletion one round), so the cheap signal suffices (#48).
5. delete-once-served: served-signal split (#43)
Two slots, split by evidence quality:
{ "glob": "openspec/changes/*/", "lifetime": "delete-once-served",
"served_when_path": "openspec/changes/archive/{id}/" }
Scanner proves the served condition → may delete silently under the tier matrix.
{ "glob": "docs/plans/*.md", "lifetime": "delete-once-served",
"served_when": "the effort this plan describes has shipped" }
Classifier judges the condition → always forced to confirm, regardless of tracked status. The LLM may propose; it may never silently destroy on a hunch.
Example of the boundary: autoresearch/*/ can be auto (a concluded run is
provable from the filesystem); PRD.md cannot (purpose-triggered — "did this
ship?" is a judgment; and it is NOT temporary, since age-keying would delete
the PRD of a feature not yet built — #45 Addendum 2).
6. Determinism promotion (#43 item 7, #47 — ADR-0041)
Design principle: hygiene drives projects toward structurally-obvious
maintenance. When a rule's served signal is subjective (classifier-judged),
the tool does not merely downgrade it to confirm — it names the subjectivity
and recommends a concrete structural convention that would graduate the rule
to served_when_path and make it silent. Confirm-fatigue is the incentive to
fix the convention.
Completion-conventions catalog
plugins/os-doc-hygiene/conventions.json — global-only, machine-readable so
the deterministic pipeline can emit nudges without an LLM. No per-project
override: the catalog only recommends; adoption lands in the project's own
rulebook. Each entry: name, what it proves, the served_when_path /
frontmatter template a rule graduates to, and a one-line human pitch.
v1 contents — exactly two conventions:
- archive-bucket — "done" = the file moved into a sibling
archive/dir (docs/plans/x.md→docs/plans/archive/x.md); graduates a rule toserved_when_path: <dir>/archive/{name}. Precedent: openspec changes. - status-frontmatter — "done" = a
status: shipped|donefrontmatter key; file stays put; scanner reads frontmatter.
Successor-artifact checks stay in the fog until a calibration pass demands one.
Nudge surfacing (split by capability)
:checknames promotion candidates in every report (deterministic, recurring — the report gains a promotion-candidates section).:calibratemay go further and DRAFT the adoption — the graduated rule plus the file moves — for human approval. It proposes, never applies unasked.
7. Pipeline integration (#37)
Lifecycle categorization is a new signal class in the existing
scanner/classifier pipeline; delete and extract-then-delete are new op
types in the clean report schema. The only new skill is
os-doc-hygiene:calibrate.
8. The :calibrate protocol (#42, #39, #45)
The learn-new-rules loop, run per project:
- Cluster-and-sample over unmatched files (unmatched = unmanaged = the candidate pool). Clustering exists precisely so rules are written over the cluster, not one member.
- Nomination (cheap model): haiku nominates a bare glob + lifetime per cluster — constrained to produce patterns, never exact-instance globs.
- Judgment (strong model): one batched Opus/Fable judge gathers its own evidence and authors final rule entries (#39: weak-model discoveries need strong-model confirmation). Verdicts: confirm / reject / amend / consult — consult is mandatory when an artifact's purpose is unclear (regenerable ≠ removable).
- Rule report to the human — before any rule is persisted. Per proposed
rule, the report shows:
- the glob verbatim, exactly as it would be persisted;
- every path it currently matches (or a capped sample + total count);
- the boundary — near-miss paths it does NOT match (this caught a
real bug during #45:
autoresearch/classic-*/silently missingautoresearch/improve-260710-1057/); - lifetime + behavior tier (auto vs confirm);
- a plain-language why — what the artifact is and why it's clutter. The human reviews patterns and examples, not JSON schema.
- Persistence: project rules land on judge confirmation; global-rulebook writes are human-gated (a cross-repo write into cc-os). Rule removals are HITL-only, with recorded reasoning.
- Retest loop: stop at <2 new rules OR <10% unmatched shrink; hard cap 3 rounds.
Seed intake: the #41 clutter-inventory seed candidates enter at judge intake — full intake for every run after calibration pass #1 (see the one-off carve-out below).
Rule-quality tests the report enforces
- The rule is the CLASS, never the PATH. A glob may hardcode a name that
recurs by convention (
PRD.md,HANDOFF-*.md,migration-report.md); it may NOT hardcode an identifier unique to one instance (a run-id, hash, or bare timestamp). A rule matching one file today is fine; a rule that can only EVER match one file is a failed generalization — flag loudly, never silently persist. - Glob-breadth tie-breaker: prefer the NARROWER glob. Too-narrow fails safe (leaves clutter; the recurring pass catches it next round — self- healing). Too-broad fails dangerous (deletes a keeper — not self-healing). Readability beats cleverness when the cost is one missed round.
9. Calibration pass #1: cc-os (#45)
Project: cc-os — proves the protocol works before testing whether it generalizes; the self-referential risk is accepted and paid for by the validation criteria carrying the weight.
- Precision hard gate: the pass FAILS if any rule persisted to the
rulebook has a glob matching a protected path — regardless of behavior tier
(a confirm gate is a safety property of the human sitting there, not of the
protocol). Exploration-time consult verdicts on protected paths are FREE.
Protected set (fixed before the pass, human-edited, never revised after):
eval
scenarios//scenarios-reserve//fixture//judge-rubric.md;openspec/specs/;docs/adr/**; mirrored.claude//.codex//.pi/skill dirs;CLAUDE.md; plugin source. - Recall floor: 8 of the 10 cc-os rows of the #41 inventory, with 4
mandatory (missing any fails the pass):
autoresearch/<run-id>/,HANDOFF-*.md,docs/adr/migration-report.md,.dochygiene/report.{json,md}.graphify-out/is void, not a miss (it is IGNORE surface per #43). The recall floor is a grading bar, not runtime behavior — err-toward-keeping comes from the hard gate + consult. - Seed hold-out (one-off for pass #1 ONLY): the cc-os rows of #41 are the sealed answer key and are withheld from judge intake — otherwise the pass is an open-book exam. This deliberately deviates from #42; every later run uses full seed intake. Do not mistake the carve-out for a permanent property.
- Novel matches beyond the answer key are expected and human-spot-checked — a wrong novel match triggers rule adjustment + a retest round, not failure.
- A do-nothing pass cannot pass: the recall floor makes the pass falsifiable in the finding direction.
Out of scope for this design
Shipping the recurring cross-project categorize-and-learn skill (charted as fog on map #31). Ignore-surface propagation into other tools' config (rejected — ADR-0040); if disposable files polluting the knowledge graph later proves painful, the fix belongs to graphify or os-vault:onboard-project.