301 lines
14 KiB
Markdown
301 lines
14 KiB
Markdown
# Lifecycle-aware doc hygiene — spec
|
||
|
||
_Status: Locked design (wayfinder map [#31](https://forgejo.swansoncloud.com/jared/cc-os/issues/31)); assembled 2026-07-14 from decision tickets #32–#48._
|
||
_Last updated: 2026-07-14_
|
||
|
||
Extends os-doc-hygiene from stale/bloat monitoring to lifecycle management: a
|
||
rulebook assigns every managed file a **lifetime**, `:check`/`:clean` gain
|
||
delete and extract-then-delete behavior, and a new `:calibrate` skill learns
|
||
rules per project. Validated against calibration project #1 (cc-os) per the
|
||
criteria in [Calibration](#calibration-pass-1-cc-os).
|
||
|
||
ADR set: [ADR-0038](../../docs/adr/) (rulebook location), ADR-0039 (deletion
|
||
autonomy tiers), ADR-0040 (no ignore-surface propagation), ADR-0041
|
||
(determinism-promotion principle).
|
||
|
||
## 1. Lifetime taxonomy (#33)
|
||
|
||
Three lifetimes, plus one modifier:
|
||
|
||
- **keep** — indefinite; scanned and reported, never deleted.
|
||
- **temporary** — age-triggered deletion (see [Temporary tier](#4-temporary-tier-43-48)).
|
||
- **delete-once-served** — purpose-triggered deletion (see [Served signals](#5-delete-once-served-served-signal-split-43)).
|
||
- **extract** (modifier on deletion, not a fourth lifetime) — distill durable
|
||
content before deleting. Extraction reuses existing knowledge routing (#36):
|
||
repo-durable residue → ADR/CLAUDE.md/docs; cross-repo lessons → SecondBrain
|
||
via `/os-vault:write`. No new destinations, no "retired specs" pile.
|
||
|
||
Distinct from all lifetimes: an **IGNORE surface** — paths the scanner never
|
||
walks at all (vs `keep` = walked and reported, never deleted). Seed members:
|
||
`graphify-out/**`, `.dochygiene/**`. The ignore surface is an explicit list,
|
||
never inferred from `.gitignore` (gitignored ≠ deletable AND gitignored ≠
|
||
keepable — #43).
|
||
|
||
## 2. Rulebook (#34, #38, #40, #44)
|
||
|
||
### Locations
|
||
|
||
- **Global:** `plugins/os-doc-hygiene/rulebook.json`, resolved relative to
|
||
plugin scripts. Ships everywhere.
|
||
- **Per-project override:** repo-root `.dochygiene-rules.json`, committed
|
||
(matches the `.dochygiene-ignore` precedent; deliberately NOT in gitignored
|
||
`.cc-os/` — ADR-0038).
|
||
|
||
Both use envelope `{"schema_version": 1, "rules": [...]}`.
|
||
|
||
### Glob dialect
|
||
|
||
gitignore-style via stdlib `glob.translate(recursive=True,
|
||
include_hidden=True)` (Python ≥ 3.13); patterns repo-root-relative, real `**`;
|
||
compiled once at load. Directory rules = patterns covering a subtree.
|
||
|
||
### Precedence (two-axis, source first)
|
||
|
||
project file-rule > project directory-rule > global file-rule > global
|
||
directory-rule; ties broken by longest pattern, then last-defined. File-level
|
||
rules override directory rules (#38). The merge is **add-only**: a project
|
||
neutralizes a global rule by shadowing it with `lifetime: "keep"` — there is
|
||
no rule-removal mechanism.
|
||
|
||
### Per-rule fields
|
||
|
||
```json
|
||
{
|
||
"glob": "autoresearch/*/**",
|
||
"lifetime": "keep|temporary|delete-once-served",
|
||
"extract": true,
|
||
"served_when": "free text — hint consumed by the LLM classifier, no trigger DSL",
|
||
"served_when_path": "openspec/changes/archive/{id}/",
|
||
"retain_recent": 3,
|
||
"max_age_days": 3,
|
||
"confirm": true,
|
||
"confirmed_by": "human | <strong-model-id>",
|
||
"confirmed_on": "2026-07-14",
|
||
"source": "clutter-inventory #41",
|
||
"note": "free-text rationale"
|
||
}
|
||
```
|
||
|
||
- `retain_recent` default 3; `max_age_days` default **3** (not
|
||
null-means-forever) — #43 amendments to the #40 schema.
|
||
- `served_when_path` is the deterministic sibling of `served_when` (#43).
|
||
- `confirm: true` is an optional always-confirm escape hatch — **human-settable
|
||
only**; a model-proposed rule may never set it, only ask a human to (#42/#43).
|
||
- **There is no `propagate_ignore` field.** It was reserved in #40 and dropped
|
||
by #44: a lifecycle rule never writes into another tool's ignore/config
|
||
surface (ADR-0040).
|
||
- Unmatched files get NO lifetime — they flow through existing signals
|
||
unchanged and become `:calibrate` candidates (**unmatched = unmanaged**).
|
||
|
||
### Scanner consumption
|
||
|
||
New stdlib `rulebook.py` loader. A directory-rule match **prunes the walk**
|
||
(files beneath are never opened; one aggregate entry) — this also implements
|
||
the IGNORE surface for free. A file-rule match attaches a **lifecycle signal**
|
||
(rule ref + lifetime + served_when) consumed by the classifier as a new signal
|
||
class (#37).
|
||
|
||
### Validation
|
||
|
||
Skip-and-warn per invalid/unconfirmed rule — a rule missing `confirmed_by`
|
||
never acts (#39) but doesn't take scanning down. Hard-fail on unparseable JSON
|
||
or unknown `schema_version`.
|
||
|
||
`.dochygiene-ignore` remains a hand-authored, human-only escape hatch,
|
||
unmanaged by the rulebook (#44).
|
||
|
||
## 3. Deletion semantics and autonomy tiers (#35, #43 — ADR-0039)
|
||
|
||
Delete = **true git deletion in a dedicated hygiene commit**; git history is
|
||
the archive. No `archive/` dirs or graveyard branches (relocated clutter still
|
||
distracts AI/search).
|
||
|
||
Rule-backed deletes are **auto** — a rule confirmed per #39 *is* the standing
|
||
consent; no per-run prompt. The auto/confirm line is drawn on **evidence
|
||
quality + recoverability**, not file type:
|
||
|
||
| Case | Behavior |
|
||
|---|---|
|
||
| IGNORE surface (`graphify-out/**`, `.dochygiene/**`) | never walked |
|
||
| lifetime `keep` | scanned + reported, never deleted |
|
||
| tracked + delete rule + clean worktree | **auto** |
|
||
| tracked + delete rule + DIRTY | **confirm** (an uncommitted diff dies with the file) |
|
||
| untracked + delete rule | **confirm** (no history to recover from) |
|
||
| no rule match | unmanaged; existing signals only, never deleted |
|
||
|
||
`clean` verifies tracked+clean **at runtime** (`git ls-files` + dirty check);
|
||
it never trusts the rule's word for it. No `recoverable_via` field, no
|
||
"regenerate" class.
|
||
|
||
## 4. Temporary tier (#43, #48)
|
||
|
||
**retain-recent-N + age, not age alone.** Defaults `retain_recent: 3`,
|
||
`max_age_days: 3` (both per-rule overridable). The newest 3 entries matching a
|
||
rule are always kept regardless of age (they show current trajectory); an
|
||
entry ranked 4th or older is deleted once it exceeds `max_age_days`. 90-day
|
||
windows were rejected as far too slow.
|
||
|
||
**Retention unit = the rule's match entry**: a file for file rules, a run
|
||
*directory* for directory rules ("3 most recent autoresearch runs", not "3
|
||
most recent files inside one run").
|
||
|
||
**Age = git commit time, falling back to filesystem mtime for untracked
|
||
files.** mtime-alone was rejected (clone/branch-switch resets every mtime,
|
||
putting the rulebook to sleep on fresh checkouts); the objection doesn't apply
|
||
to the fallback, since untracked files don't exist in a fresh clone. No
|
||
per-rule `age_source` field (#48).
|
||
|
||
**Untracked directory entries** use the directory inode's own mtime (one
|
||
stat), NOT a recursive max-mtime walk — the tier's failure mode is
|
||
self-healing (a spuriously bumped mtime merely delays deletion one round), so
|
||
the cheap signal suffices (#48).
|
||
|
||
## 5. delete-once-served: served-signal split (#43)
|
||
|
||
Two slots, split by evidence quality:
|
||
|
||
```json
|
||
{ "glob": "openspec/changes/*/", "lifetime": "delete-once-served",
|
||
"served_when_path": "openspec/changes/archive/{id}/" }
|
||
```
|
||
Scanner **proves** the served condition → may delete silently under the tier
|
||
matrix.
|
||
|
||
```json
|
||
{ "glob": "docs/plans/*.md", "lifetime": "delete-once-served",
|
||
"served_when": "the effort this plan describes has shipped" }
|
||
```
|
||
Classifier **judges** the condition → **always forced to confirm**, regardless
|
||
of tracked status. The LLM may propose; it may never silently destroy on a
|
||
hunch.
|
||
|
||
Example of the boundary: `autoresearch/*/` can be auto (a concluded run is
|
||
provable from the filesystem); `PRD.md` cannot (purpose-triggered — "did this
|
||
ship?" is a judgment; and it is NOT temporary, since age-keying would delete
|
||
the PRD of a feature not yet built — #45 Addendum 2).
|
||
|
||
## 6. Determinism promotion (#43 item 7, #47 — ADR-0041)
|
||
|
||
**Design principle: hygiene drives projects toward structurally-obvious
|
||
maintenance.** When a rule's served signal is subjective (classifier-judged),
|
||
the tool does not merely downgrade it to confirm — it names the subjectivity
|
||
and recommends a concrete structural convention that would graduate the rule
|
||
to `served_when_path` and make it silent. Confirm-fatigue is the incentive to
|
||
fix the convention.
|
||
|
||
### Completion-conventions catalog
|
||
|
||
`plugins/os-doc-hygiene/conventions.json` — global-only, machine-readable so
|
||
the deterministic pipeline can emit nudges without an LLM. No per-project
|
||
override: the catalog only recommends; adoption lands in the project's own
|
||
rulebook. Each entry: name, what it proves, the `served_when_path` /
|
||
frontmatter template a rule graduates to, and a one-line human pitch.
|
||
|
||
v1 contents — exactly two conventions:
|
||
|
||
- **archive-bucket** — "done" = the file moved into a sibling `archive/` dir
|
||
(`docs/plans/x.md` → `docs/plans/archive/x.md`); graduates a rule to
|
||
`served_when_path: <dir>/archive/{name}`. Precedent: openspec changes.
|
||
- **status-frontmatter** — "done" = a `status: shipped|done` frontmatter key;
|
||
file stays put; scanner reads frontmatter.
|
||
|
||
Successor-artifact checks stay in the fog until a calibration pass demands one.
|
||
|
||
### Nudge surfacing (split by capability)
|
||
|
||
- **`:check`** names promotion candidates in every report (deterministic,
|
||
recurring — the report gains a promotion-candidates section).
|
||
- **`:calibrate`** may go further and DRAFT the adoption — the graduated rule
|
||
plus the file moves — for human approval. It proposes, never applies unasked.
|
||
|
||
## 7. Pipeline integration (#37)
|
||
|
||
Lifecycle categorization is a **new signal class** in the existing
|
||
scanner/classifier pipeline; `delete` and `extract-then-delete` are **new op
|
||
types** in the clean report schema. The only new skill is
|
||
**`os-doc-hygiene:calibrate`**.
|
||
|
||
## 8. The :calibrate protocol (#42, #39, #45)
|
||
|
||
The learn-new-rules loop, run per project:
|
||
|
||
1. **Cluster-and-sample** over unmatched files (unmatched = unmanaged = the
|
||
candidate pool). Clustering exists precisely so rules are written over the
|
||
cluster, not one member.
|
||
2. **Nomination (cheap model):** haiku nominates a bare glob + lifetime per
|
||
cluster — constrained to produce *patterns*, never exact-instance globs.
|
||
3. **Judgment (strong model):** one batched Opus/Fable judge gathers its own
|
||
evidence and authors final rule entries (#39: weak-model discoveries need
|
||
strong-model confirmation). Verdicts: **confirm / reject / amend /
|
||
consult** — consult is mandatory when an artifact's purpose is unclear
|
||
(regenerable ≠ removable).
|
||
4. **Rule report to the human — before any rule is persisted.** Per proposed
|
||
rule, the report shows:
|
||
1. the **glob verbatim**, exactly as it would be persisted;
|
||
2. every path it currently matches (or a capped sample + total count);
|
||
3. **the boundary — near-miss paths it does NOT match** (this caught a
|
||
real bug during #45: `autoresearch/classic-*/` silently missing
|
||
`autoresearch/improve-260710-1057/`);
|
||
4. lifetime + behavior tier (auto vs confirm);
|
||
5. a plain-language why — what the artifact is and why it's clutter.
|
||
The human reviews patterns and examples, not JSON schema.
|
||
5. **Persistence:** project rules land on judge confirmation;
|
||
**global-rulebook writes are human-gated** (a cross-repo write into cc-os).
|
||
Rule removals are HITL-only, with recorded reasoning.
|
||
6. **Retest loop:** stop at <2 new rules OR <10% unmatched shrink; hard cap 3
|
||
rounds.
|
||
|
||
Seed intake: the #41 clutter-inventory seed candidates enter at judge intake —
|
||
**full intake for every run after calibration pass #1** (see the one-off
|
||
carve-out below).
|
||
|
||
### Rule-quality tests the report enforces
|
||
|
||
- **The rule is the CLASS, never the PATH.** A glob may hardcode a name that
|
||
*recurs by convention* (`PRD.md`, `HANDOFF-*.md`, `migration-report.md`); it
|
||
may NOT hardcode an identifier *unique to one instance* (a run-id, hash, or
|
||
bare timestamp). A rule matching one file today is fine; a rule that can
|
||
only EVER match one file is a failed generalization — flag loudly, never
|
||
silently persist.
|
||
- **Glob-breadth tie-breaker: prefer the NARROWER glob.** Too-narrow fails
|
||
safe (leaves clutter; the recurring pass catches it next round — self-
|
||
healing). Too-broad fails dangerous (deletes a keeper — not self-healing).
|
||
Readability beats cleverness when the cost is one missed round.
|
||
|
||
## 9. Calibration pass #1: cc-os (#45)
|
||
|
||
Project: **cc-os** — proves the protocol *works* before testing whether it
|
||
*generalizes*; the self-referential risk is accepted and paid for by the
|
||
validation criteria carrying the weight.
|
||
|
||
- **Precision hard gate:** the pass FAILS if any rule persisted to the
|
||
rulebook has a glob matching a protected path — regardless of behavior tier
|
||
(a confirm gate is a safety property of the human sitting there, not of the
|
||
protocol). Exploration-time consult verdicts on protected paths are FREE.
|
||
Protected set (fixed before the pass, human-edited, never revised after):
|
||
eval `scenarios/`/`scenarios-reserve/`/`fixture/`/judge-rubric.md;
|
||
`openspec/specs/`; `docs/adr/**`; mirrored `.claude/`/`.codex/`/`.pi/` skill
|
||
dirs; `CLAUDE.md`; plugin source.
|
||
- **Recall floor: 8 of the 10 cc-os rows of the #41 inventory**, with 4
|
||
mandatory (missing any fails the pass): `autoresearch/<run-id>/`,
|
||
`HANDOFF-*.md`, `docs/adr/migration-report.md`,
|
||
`.dochygiene/report.{json,md}`. `graphify-out/` is **void, not a miss** (it
|
||
is IGNORE surface per #43). The recall floor is a grading bar, not runtime
|
||
behavior — err-toward-keeping comes from the hard gate + consult.
|
||
- **Seed hold-out (one-off for pass #1 ONLY):** the cc-os rows of #41 are the
|
||
sealed answer key and are withheld from judge intake — otherwise the pass is
|
||
an open-book exam. This deliberately deviates from #42; **every later run
|
||
uses full seed intake.** Do not mistake the carve-out for a permanent
|
||
property.
|
||
- Novel matches beyond the answer key are expected and human-spot-checked —
|
||
a wrong novel match triggers rule adjustment + a retest round, not failure.
|
||
- A do-nothing pass cannot pass: the recall floor makes the pass falsifiable
|
||
in the finding direction.
|
||
|
||
## Out of scope for this design
|
||
|
||
Shipping the recurring cross-project categorize-and-learn skill (charted as
|
||
fog on map #31). Ignore-surface propagation into other tools' config
|
||
(rejected — ADR-0040); if disposable files polluting the knowledge graph later
|
||
proves painful, the fix belongs to graphify or os-vault:onboard-project.
|