274 lines
16 KiB
Markdown
274 lines
16 KiB
Markdown
# Design: Check
|
||
|
||
## Context
|
||
|
||
The deterministic core (build step #2) and the token estimator are in place:
|
||
`scanner.py` emits the intermediate `{ project_root, scope_globs, excluded_dirs,
|
||
files_scanned, shortlist, signals }` artifact; `state_store.py` owns lifecycle
|
||
timestamps, atomic writes, and report rollover; `validate_report.py` owns the frozen
|
||
schema validation, including the single-source `derive_safety_tier(...)` function and
|
||
`KIND_TABLE`; `token_estimator.py` exposes `default_estimator().estimate_for_report(text)`.
|
||
This change builds the first AI pass on top of them — the `check` skill — plus the
|
||
`/hygiene` command that the `SessionStart` reminder already advertises.
|
||
|
||
Constraints that shape this design:
|
||
|
||
- **Invariant #6 (deterministic-first):** scan, state, patch-apply, and
|
||
token-estimate are scripts. The model does only classification and prose
|
||
distillation. `raw_tokens` comes from the local estimator, never a model.
|
||
- **Invariant #10 (`safety_tier` is derived):** `safety_tier` is computed only by
|
||
`derive_safety_tier(op_type, is_destructive, is_reversible)` — never model-assigned.
|
||
- **Invariant #11 (`op_type` is a property of the chosen op):** `op_type` is
|
||
`deterministic` iff the op carries an `exact_edit`; a single subtype may map to
|
||
either op-type depending on the chosen op.
|
||
- **Invariant #4 (report rollover):** exactly one `report.json` + `report.md` pair
|
||
survives each write; `write_report` deletes the prior pair before writing the new
|
||
one.
|
||
- **Invariant #8 (mtime/content guard):** the cleaner verifies a file's current hash
|
||
matches the entry's `expected_sha256` before applying a cached edit.
|
||
|
||
## Goals / Non-Goals
|
||
|
||
**Goals:**
|
||
|
||
- A `/hygiene` command surface: `check` (scan + classify + report) and `status`
|
||
(read-only timestamps), with `clean` / `sweep` reserved.
|
||
- A `hygiene-check` skill orchestrating scan → Sonnet classify → deterministic
|
||
finalize → validate → write + stamp `last_check`.
|
||
- A standalone, model-free `report_builder.py` finalize pass that owns the four
|
||
fields the model must not author (`expected_sha256`, `safety_tier`,
|
||
`is_destructive`/`is_reversible` for deterministic ops, `raw_tokens`).
|
||
- Hermetic classifier golden examples + a unit harness that never calls a live model.
|
||
|
||
**Non-Goals:**
|
||
|
||
- The `clean` skill, patch-applier, mtime-guard application, and `sweep` (Phase 4).
|
||
- `token_estimate` weighting (`injection_frequency`, `weighted_tokens`, rollup) — v2
|
||
bonus; this change populates only `raw_tokens`.
|
||
- The live model-classification regression harness (separate, manually invoked).
|
||
|
||
## Decisions
|
||
|
||
### D1. A deterministic finalize pass between classify and write — `report_builder.py`
|
||
|
||
This is the decision that shapes everything. Four per-entry fields cannot be
|
||
model-authored without violating a frozen invariant or being physically impossible:
|
||
|
||
- `exact_edit.expected_sha256` — the sha256 of real file bytes; the model never sees
|
||
the bytes, and inventing a hash is meaningless.
|
||
- `safety_tier` — invariant #10: derived only by `derive_safety_tier(...)`, never
|
||
model-assigned.
|
||
- `is_destructive` / `is_reversible` for deterministic ops — fixed by
|
||
`exact_edit.kind` via `KIND_TABLE` in `validate_report.py`.
|
||
- `token_estimate.raw_tokens` — invariant #6: from the local token estimator, never a
|
||
model.
|
||
|
||
Therefore check MUST include a deterministic step **between** model classification and
|
||
`StateStore.write_report`. A new standalone script `scripts/report_builder.py` is that
|
||
step: a model-free assembler. **Input:** the scanner artifact plus the model's slim
|
||
per-file classification proposals. **Output:** a full schema-valid machine report plus
|
||
a deterministic human-report skeleton. For each proposal it reads the anchored span,
|
||
computes `expected_sha256`, looks up `(is_destructive, is_reversible)` from `kind`,
|
||
calls `derive_safety_tier(...)` **imported** from `validate_report.py` (single source
|
||
of truth — no re-implementation), calls
|
||
`token_estimator.default_estimator().estimate_for_report(span_text)`, stamps per-entry
|
||
`generated_at` at each file's hash instant, and assembles `scan` / `shortlist` /
|
||
`entries`. **Alternative considered:** have the skill assemble the report inline in
|
||
the SKILL.md workflow — rejected: the four guardrail fields then have no enforced,
|
||
unit-testable home, and the model could leak into a field it must not author.
|
||
|
||
### D2. Command surface — a single `commands/hygiene.md` dispatching on `$ARGUMENTS`
|
||
|
||
One command file. Frontmatter is minimal (`name: hygiene`; a description naming
|
||
`/hygiene check` to scan and `/hygiene status` for timestamps). Routes:
|
||
|
||
- `check [--scope <glob-or-path>] [--category <class|subtype>]` → invoke the
|
||
`hygiene-check` skill.
|
||
- `status` → read-only: `StateStore.get_last_check` / `get_last_clean` /
|
||
`get_last_reminded` plus whether a report exists. No scan, no model. `status` lives
|
||
**inline** in the command (a few `python3` calls), not in a separate skill.
|
||
- `clean` / `sweep` → reserved: "not yet implemented (Phase 4)".
|
||
- no/unknown args → usage + current status.
|
||
|
||
**Alternative considered:** a separate `status` skill — rejected: `status` is a few
|
||
read-only timestamp reads, not judgment; a skill would be ceremony.
|
||
|
||
### D3. Check skill flow — scan, classify, finalize, validate, write, stamp
|
||
|
||
`skills/hygiene-check/SKILL.md` frontmatter mirrors the `commit` skill (`name:
|
||
hygiene-check`; a description). Like the `commit` skill, the SKILL.md workflow runs
|
||
deterministic scripts via Bash, dispatches a Sonnet subagent for judgment-only
|
||
classification, then runs deterministic scripts to assemble/validate/write — the
|
||
sanctioned mechanism inside a skill workflow (D9). Flow (**D** = deterministic script,
|
||
**M** = model):
|
||
|
||
1. **(D) Scan** — `python3 scripts/scanner.py` (auto-resolves root, default
|
||
`**/*.md`, default excludes incl. `.dochygiene/`). `--scope` applies a
|
||
`--globs`/path filter. Capture the artifact `{ project_root, scope_globs,
|
||
excluded_dirs, files_scanned, shortlist, signals }`.
|
||
2. **(D) Select candidates** — real candidates = signal-bearing paths (the keys of
|
||
`signals`). Zero-signal shortlisted files are **presumptively cleared**: they stay
|
||
in `shortlist`, produce no entries, and are **not read by the model**. `--category`
|
||
filters only at the entry stage.
|
||
3. **(M) Classify** each signal-bearing candidate (Sonnet subagent). The subagent
|
||
reads each candidate file plus its scanner signals and returns a **slim proposal**
|
||
per file (judgment only, no computed fields):
|
||
- `category` `{ class, subtype }` from the closed enum
|
||
(`STALE_SUBTYPES`/`BLOAT_SUBTYPES`) justified by cited signals (PRD taxonomy).
|
||
- `signals` passed through **verbatim** (scanner names: `broken_reference`,
|
||
`version_skew`, `edit_recency_vs_churn`, `stale_name_location`,
|
||
`archive_to_live_ratio`, `frontmatter_marker`) with an optional one-line gloss in
|
||
`detail`.
|
||
- `op` (a human sentence); `op_type` `deterministic` | `generative` (a property of
|
||
the chosen op, invariant #11).
|
||
- If `deterministic`: an `exact_edit` **skeleton** — `kind` (closed enum) +
|
||
`anchor.{start_line,end_line}` where required + kind-specific fields
|
||
(`dest_path` / `key,value` / `match,replacement` / `canonical_ref`). The model
|
||
does **not** supply `expected_sha256`, `is_destructive`, `is_reversible`, or
|
||
`safety_tier`.
|
||
- If `generative`: no `exact_edit`; instead a **non-persisted** `reducible_range`
|
||
`{start_line,end_line}` so the assembler counts `raw_tokens` on the real text.
|
||
- Plus `confidence`.
|
||
|
||
**Decision rules** (subtype → kind → tier): destructive deletion of unique content
|
||
→ `delete-range` (→`confirm`); content-preserving relocation → `move-to-archive`
|
||
(→`auto`); freeze a completed doc → `insert-frontmatter` (→`auto`); exact duplicate
|
||
preserved elsewhere → `dedupe` with `canonical_ref` (→`auto`); known-target link fix
|
||
→ `replace-text` (→`auto`); prose condensation/splitting → `generative`
|
||
(→`confirm`).
|
||
4. **(D) Finalize** via `report_builder.py` — per proposal: read the file; compute
|
||
`expected_sha256` over current bytes (anchor-bearing kinds only); set
|
||
`(is_destructive, is_reversible)` from `KIND_TABLE[kind]`; `safety_tier =
|
||
derive_safety_tier(op_type, is_destructive, is_reversible)` imported from
|
||
`validate_report.py`; extract the span (anchor range / `reducible_range` / whole
|
||
file for `move-to-archive`) and `token_estimate = estimator.estimate_for_report(span_text)`
|
||
(v1 `raw_tokens` only, weighting null); assemble the envelope `schema_version`
|
||
`"1.0"`, `tool_version` from `plugin.json`, per-entry `generated_at` = that file's
|
||
hash instant, envelope `generated_at` = the run instant (which also becomes
|
||
`last_check`). Also emit the deterministic human-report skeleton (the mechanical
|
||
parts); the optional per-entry "why" gloss is model-written.
|
||
5. **(D) Validate BEFORE writing** — run `validate_report.py` on a **scratch** path
|
||
(scratchpad, NOT `.dochygiene/`). `write_report` deletes the prior pair first, so
|
||
validating after the write would destroy the last good report (invariant #4). Only
|
||
on exit 0 proceed.
|
||
6. **(D) Write + rollover** — `StateStore.write_report(json_blob, md_blob)` (atomic,
|
||
keeps exactly one report pair, invariant #4).
|
||
7. **(D) Stamp** — `StateStore.set_last_check(generated_at)` using the same run
|
||
instant from step 4.
|
||
8. Surface the human-report summary plus the two report paths.
|
||
|
||
**Model routing:** classification = Sonnet; escalate a single file to Opus only on
|
||
low confidence for hard distinctions (stale-vs-bloat; `delete-range`/destructive vs
|
||
generative rewrite of the same contradicted/superseded content). Steps 1, 2, 4, 5, 6,
|
||
7 use no model (invariant #6).
|
||
|
||
### D4. Failure handling
|
||
|
||
- **Validation fail (exit 1):** do NOT `write_report` (the prior pair is preserved).
|
||
Map violations to `entries[i]`, re-prompt the subagent to fix only the offending
|
||
proposals or drop an unfixable entry and re-validate. Never write an invalid report.
|
||
- **Validator usage error (exit 2):** an internal bug; stop.
|
||
- **Empty shortlist / no signal-bearing files:** write a report with empty `entries`
|
||
(still valid) and still `set_last_check`.
|
||
- **Malformed proposal JSON:** the assembler rejects it before computing; re-prompt.
|
||
|
||
### D5. Scoping
|
||
|
||
`--scope` narrows the scanner (`--globs` or a path-prefix filter on the shortlist).
|
||
`--category` filters which **entries** are produced — the scanner is category-agnostic,
|
||
so the filter is applied after classification. Both are recorded in the human-report
|
||
header.
|
||
|
||
### D6. Reports — fixed paths, one pair, human skeleton
|
||
|
||
`StateStore` hard-codes the paths: machine `.dochygiene/report.json`, human
|
||
`.dochygiene/report.md`, both written via `StateStore.write_report`; rollover keeps
|
||
exactly one pair (invariant #4); `.dochygiene/` is gitignored. The human report
|
||
skeleton groups by Stale / Bloat / Cleared with per-entry path, category, op, tier,
|
||
~tokens, signal; the header shows timestamp, scope, files scanned, and
|
||
candidate/cleared counts. The structural parts are script-built; only the optional
|
||
per-entry "why" gloss is model-written.
|
||
|
||
### D7. Classifier golden examples — distinct from schema fixtures, hermetic harness
|
||
|
||
Classifier goldens are **distinct** from `examples/golden/valid_report.json` /
|
||
`invalid_report.json` (those are schema-shape fixtures for `validate_report.py` only).
|
||
Classifier goldens are the Layer-2 reversion-protection layer: input doc-tree →
|
||
expected classification. Layout `examples/golden/classifier/<n>-<name>/`, each with
|
||
`input/` (a small static fixture tree, stable sha256s), `expected.json` (a full
|
||
schema-valid machine report = the expected classification), and an optional
|
||
`notes.md`. 3–5 cases, each mapping a subtype to a **distinct** `exact_edit.kind` to
|
||
cover the kind table + tier derivation:
|
||
|
||
- `orphaned` → `stale`/`orphaned`/`deterministic`/`delete-range`/`confirm`
|
||
- `superseded` → `stale`/`superseded`/`deterministic`/`move-to-archive`/`auto`
|
||
- `completed-in-place` → `stale`/`completed-in-place`/`deterministic`/`insert-frontmatter`/`auto`
|
||
- `duplicated` → `stale`/`duplicated`/`deterministic`/`dedupe`/`auto`
|
||
- `distill` → `bloat`/`distill`/`generative`/(none)/`confirm`
|
||
|
||
The harness MUST be **hermetic** (no live model call in pytest):
|
||
`tests/test_classifier_golden.py` asserts only deterministic/stable parts — (1)
|
||
`scanner.py` on `input/` emits the expected signals on the right paths; (2)
|
||
`validate_report.py` on each `expected.json` → exit 0; (3) a stable-field match against
|
||
a **captured/committed** check output (`category.class`, `category.subtype`,
|
||
`op_type`, the derived `safety_tier`, `exact_edit.kind`). The **live**
|
||
model-classification regression (running an actual check against `input/` and diffing)
|
||
is a **separate, manually/agent-invoked** harness — NOT part of the unit suite.
|
||
Op-prose and exact anchor line numbers are advisory (flag a mismatch for human review,
|
||
not a hard fail). Adding or changing classifier goldens is **human-gated** per the
|
||
META-RULE.
|
||
|
||
## Risks / Trade-offs
|
||
|
||
- **[`insert-frontmatter` has no content guard — DEFERRED hole]** →
|
||
`insert-frontmatter` has `has_anchor = False`, so it carries **no**
|
||
`expected_sha256`; invariant #8's content guard cannot protect it at clean time.
|
||
This is a known deferred hole — Phase 4 (clean) MUST handle `insert-frontmatter`
|
||
application explicitly (e.g. re-derive frontmatter presence at apply time rather
|
||
than trusting a cached hash). Recorded here and in CLAUDE.md so it is not silently
|
||
inherited.
|
||
- **[Guard is a content hash despite the "mtime guard" naming]** → The guard compares
|
||
`expected_sha256`, not mtime. Per-entry `generated_at` is **that file's hash
|
||
instant** (distinct from the envelope `generated_at` / `last_check` stamp). The
|
||
naming is historical; the mechanism is content-hash.
|
||
- **[Zero-signal files are unread]** → Zero-signal shortlisted files are presumptively
|
||
cleared and never read by the model (a cost decision). This is a known v1 recall
|
||
limit — a file with a real problem but no scanner signal is missed. Documented.
|
||
- **[`tool_version` source mismatch]** → `tool_version` is read from `plugin.json`
|
||
(currently `0.0.1`), while `valid_report.json` shows `0.1.0`. Read from
|
||
`plugin.json` and flag the mismatch; do not hardcode.
|
||
- **[Generative `raw_tokens` needs a span to count]** → A generative op has no
|
||
`exact_edit`, so the model returns a **non-persisted** `reducible_range`; the
|
||
assembler counts `raw_tokens` over that span. The range is not written to the report.
|
||
|
||
## Migration Plan
|
||
|
||
Additive only — no existing behavior changes. Deploy order follows the task order:
|
||
`report_builder.py` + its tests first (it is the guardrail all entries flow through),
|
||
`commands/hygiene.md` in parallel, then the `hygiene-check` skill (depends on the
|
||
builder and the command), then the classifier goldens + hermetic harness, then the
|
||
CONTEXT.md updates. Rollback is removing the new script, command, skill, and fixtures;
|
||
the deterministic core and estimator are untouched.
|
||
|
||
## Open Questions (resolved — recorded as decisions)
|
||
|
||
1. **`report_builder.py` is standalone**, importing `derive_safety_tier` + `KIND_TABLE`
|
||
from `validate_report.py` (no re-implementation).
|
||
2. **Entry signals = scanner signals verbatim** + an optional model gloss in `detail`
|
||
(the validator does not constrain signal names; verbatim pass-through is a
|
||
convention for trust).
|
||
3. **`insert-frontmatter` has `has_anchor = False`** → no `expected_sha256` → no #8
|
||
content guard at clean time: a **deferred hole**, Phase 4 must handle it.
|
||
4. **The guard is a content hash (`expected_sha256`)** despite the "mtime guard"
|
||
naming; per-entry `generated_at` = that file's hash instant (distinct from the
|
||
envelope/`last_check` stamp).
|
||
5. **Zero-signal files are unread** (a cost decision) — a known v1 recall limit,
|
||
documented.
|
||
6. **Generative `raw_tokens`:** the model returns a non-persisted `reducible_range`;
|
||
the assembler counts that span.
|
||
7. **Validate-before-rollover** is a hard sequencing constraint (validate on a scratch
|
||
path before `write_report` deletes the prior pair).
|
||
8. **`tool_version` from `plugin.json`** (currently `0.0.1`; `valid_report.json` shows
|
||
`0.1.0` — read from `plugin.json`, flag the mismatch).
|
||
9. **A subagent + Bash inside a skill is sanctioned** per the `commit`-skill precedent.
|