257 lines
14 KiB
Markdown
257 lines
14 KiB
Markdown
# doc-check Specification
|
|
|
|
## Purpose
|
|
|
|
Defines the `/os-doc-hygiene:check` command and the `check` skill it dispatches to:
|
|
the deterministic scan-classify-finalize-validate pipeline that turns a project's
|
|
docs into a schema-valid machine + human report pair, including rulebook-driven
|
|
scanning, judgment-only Sonnet classification, the model-free finalize pass that
|
|
authors safety-critical fields, and validation-gated report writes.
|
|
|
|
## Requirements
|
|
### Requirement: `/hygiene` Command Surface
|
|
|
|
The plugin SHALL provide a single `/hygiene` command that dispatches on its
|
|
arguments. `/os-doc-hygiene:check [--scope <glob-or-path>] [--category <class|subtype>]`
|
|
SHALL invoke the `check` skill. `/os-doc-hygiene:status` SHALL read and report the
|
|
lifecycle timestamps (`last_check`, `last_clean`, `last_reminded`) and whether a
|
|
report exists, using no scan and no model. `/os-doc-hygiene:clean` and `/os-doc-hygiene:sweep`
|
|
SHALL be reserved and SHALL report that they are not yet implemented (Phase 4). No
|
|
arguments or unknown arguments SHALL print usage plus the current status.
|
|
|
|
#### Scenario: Check dispatches to the skill
|
|
|
|
- **WHEN** the user runs `/os-doc-hygiene:check`
|
|
- **THEN** the command invokes the `check` skill, passing through any `--scope` or `--category` flag
|
|
|
|
#### Scenario: Status is read-only
|
|
|
|
- **WHEN** the user runs `/os-doc-hygiene:status`
|
|
- **THEN** the command reports `last_check`, `last_clean`, `last_reminded`, and whether a report exists, without running a scan or any model
|
|
|
|
#### Scenario: Clean and sweep are reserved
|
|
|
|
- **WHEN** the user runs `/os-doc-hygiene:clean` or `/os-doc-hygiene:sweep`
|
|
- **THEN** the command reports that the subcommand is not yet implemented (Phase 4) and does not mutate anything
|
|
|
|
#### Scenario: Unknown arguments print usage
|
|
|
|
- **WHEN** the user runs `/hygiene` with no arguments or an unrecognized subcommand
|
|
- **THEN** the command prints usage and the current status
|
|
|
|
### Requirement: Check Skill Orchestrates Scan, Classification, and Report Writing
|
|
|
|
The `check` skill SHALL orchestrate the check pipeline: load the lifecycle
|
|
rulebook (global plus any project override), run the deterministic scanner
|
|
(consuming the rulebook so directory-rule matches prune the walk and
|
|
lifecycle signals are attached per the `lifecycle-rulebook` spec), dispatch
|
|
a Sonnet subagent for judgment-only classification of the signal-bearing
|
|
candidates, run the deterministic finalize pass (which also computes
|
|
`promotion_candidates` from `conventions.json`), validate, write the report
|
|
pair, and stamp `last_check`. The skill SHALL run all non-judgment steps as
|
|
deterministic scripts with no model (invariant #6). Zero-signal shortlisted
|
|
files SHALL be treated as presumptively cleared: they SHALL remain in the
|
|
shortlist, produce no entries, and SHALL NOT be read by the model. A
|
|
`--scope` argument SHALL narrow the scanner; a `--category` argument SHALL
|
|
filter which entries are produced after classification; both SHALL be
|
|
recorded in the human-report header.
|
|
|
|
#### Scenario: Skill runs the full pipeline
|
|
|
|
- **WHEN** the `check` skill runs
|
|
- **THEN** it loads the rulebook, scans (deterministic, rulebook-aware), classifies signal-bearing candidates (Sonnet), finalizes (deterministic, including promotion candidates), validates (deterministic), writes the report pair (deterministic), and stamps `last_check` (deterministic)
|
|
|
|
#### Scenario: Zero-signal files are not read by the model
|
|
|
|
- **WHEN** a shortlisted file carries no scanner signals
|
|
- **THEN** it remains in the shortlist, produces no entry, and is not read by the classification model
|
|
|
|
#### Scenario: Scope and category are recorded and applied
|
|
|
|
- **WHEN** the user passes `--scope docs/**/*.md` and `--category bloat`
|
|
- **THEN** the scanner is narrowed by the scope, only `bloat` entries are produced after classification, and both the scope and the category are recorded in the human-report header
|
|
|
|
#### Scenario: Rulebook load failure is a hard failure, not a silent skip
|
|
|
|
- **WHEN** the rulebook loader hard-fails (unparseable JSON or unknown `schema_version` in either rulebook file)
|
|
- **THEN** the check skill stops and reports the rulebook error before running the scanner, rather than proceeding with lifecycle signals silently disabled
|
|
|
|
### Requirement: Scanner Consumes the Rulebook for Pruning and Lifecycle Signals
|
|
|
|
The deterministic scanner SHALL consult the loaded rulebook during its walk.
|
|
A directory-rule match (including IGNORE-surface entries) SHALL prune the
|
|
walk beneath that directory per the `lifecycle-rulebook` spec. A file-rule
|
|
match SHALL attach a lifecycle signal to that file's shortlist entry. These
|
|
lifecycle signals SHALL flow into the classification subagent as a new
|
|
signal class alongside the pre-existing stale/bloat signals, and MAY drive
|
|
`op`/`op_type` selection toward `delete` or `extract-then-delete` per the
|
|
`lifecycle-deletion` spec.
|
|
|
|
#### Scenario: A directory-rule prune is reflected in the scan artifact
|
|
|
|
- **WHEN** the scanner encounters a directory matching a directory rule
|
|
- **THEN** the scan artifact reflects the prune (no files beneath it are in `files_scanned`), and, for non-IGNORE directory rules, exactly one aggregate shortlist entry appears for that directory
|
|
|
|
#### Scenario: A file-rule lifecycle signal reaches the classifier
|
|
|
|
- **WHEN** a file matches a file-rule with `lifetime: delete-once-served`
|
|
- **THEN** the classification subagent receives the lifecycle signal (rule reference, lifetime, served_when/served_when_path) as part of that file's signals, verbatim, per the existing "signals are passed through verbatim" contract
|
|
|
|
### Requirement: Report Gains a Promotion-Candidates Section
|
|
|
|
The machine and human reports produced by `:check` SHALL include a
|
|
`promotion_candidates` section (top-level, sibling to `entries`), populated
|
|
deterministically by the finalize pass from `conventions.json` for every
|
|
classifier-judged lifecycle entry with an applicable, not-yet-adopted
|
|
convention. This section SHALL be present (possibly empty) on every run,
|
|
including runs with no lifecycle entries.
|
|
|
|
#### Scenario: A run with an applicable convention names it in both reports
|
|
|
|
- **WHEN** a classifier-judged entry has an applicable, unadopted convention
|
|
- **THEN** both the machine report's `promotion_candidates` array and the human report show the candidate with its one-line pitch
|
|
|
|
#### Scenario: A run with no applicable conventions still has the section, empty
|
|
|
|
- **WHEN** no classifier-judged entry has an applicable unadopted convention
|
|
- **THEN** `promotion_candidates` is present as an empty array/section rather than omitted
|
|
|
|
### Requirement: Classification Subagent Returns Judgment-Only Proposals
|
|
|
|
The Sonnet classification subagent SHALL return, per signal-bearing candidate, a slim
|
|
proposal containing only judgment fields: `category` (`class` and `subtype` from the
|
|
closed enum, justified by cited signals), the scanner `signals` passed through
|
|
verbatim with an optional one-line gloss in `detail`, `op` (a human sentence),
|
|
`op_type` (`deterministic` or `generative`, a property of the chosen op per invariant
|
|
#11), and `confidence`. When `op_type` is `deterministic`, the proposal SHALL carry an
|
|
`exact_edit` skeleton (`kind` plus the kind's required sub-fields and `anchor` where
|
|
required) and SHALL NOT carry `expected_sha256`, `is_destructive`, `is_reversible`, or
|
|
`safety_tier`. When `op_type` is `generative`, the proposal SHALL carry no
|
|
`exact_edit` and instead a non-persisted `reducible_range` so the finalize pass can
|
|
count `raw_tokens` over the real span. Low-confidence hard distinctions
|
|
(stale-vs-bloat; destructive-deletion-vs-generative-rewrite) MAY be escalated to Opus.
|
|
|
|
#### Scenario: Deterministic proposal carries only the exact-edit skeleton
|
|
|
|
- **WHEN** the subagent classifies a file as a deterministic op
|
|
- **THEN** the proposal includes the `exact_edit` skeleton (`kind`, required sub-fields, `anchor` where required) and omits `expected_sha256`, `is_destructive`, `is_reversible`, and `safety_tier`
|
|
|
|
#### Scenario: Generative proposal carries a reducible range, not an exact edit
|
|
|
|
- **WHEN** the subagent classifies a file as a generative op
|
|
- **THEN** the proposal carries no `exact_edit` and instead a non-persisted `reducible_range` for the finalize pass to count tokens over
|
|
|
|
#### Scenario: Signals are passed through verbatim
|
|
|
|
- **WHEN** the subagent emits a proposal
|
|
- **THEN** its `signals` are the scanner's signal names verbatim, with any added wording confined to the optional `detail` gloss
|
|
|
|
### Requirement: Deterministic Finalize Pass Owns the Non-Model Fields
|
|
|
|
A standalone, model-free finalize pass (`report_builder.py`) SHALL sit between model
|
|
classification and the report write, and SHALL author the four per-entry fields that
|
|
the model must not author. For each proposal it SHALL compute
|
|
`exact_edit.expected_sha256` over the file's current bytes for anchor-bearing kinds,
|
|
SHALL set `(is_destructive, is_reversible)` from `KIND_TABLE[kind]`, SHALL compute
|
|
`safety_tier` by calling `derive_safety_tier(op_type, is_destructive, is_reversible)`
|
|
imported from `validate_report.py` (the single source of truth, invariant #10), and
|
|
SHALL source `token_estimate.raw_tokens` from the local token estimator
|
|
(`default_estimator().estimate_for_report(span_text)`, invariant #6). It SHALL stamp
|
|
each entry's `generated_at` at that file's hash instant and set the envelope
|
|
`generated_at` to the run instant. The model SHALL NOT supply any of these four
|
|
fields.
|
|
|
|
#### Scenario: The finalize pass computes the content hash and derives the tier
|
|
|
|
- **WHEN** the finalize pass processes a deterministic proposal with `kind` = `move-to-archive`
|
|
- **THEN** it computes `expected_sha256` over the file's current bytes, sets `is_destructive` = false and `is_reversible` = true from `KIND_TABLE`, and derives `safety_tier` = `auto` via `derive_safety_tier`
|
|
|
|
#### Scenario: raw_tokens comes from the local estimator, never the model
|
|
|
|
- **WHEN** the finalize pass sets a `token_estimate`
|
|
- **THEN** `raw_tokens` is the local estimator's count of the span (no model, no API call), with the weighting fields null in v1
|
|
|
|
#### Scenario: The model cannot author the derived fields
|
|
|
|
- **WHEN** a proposal arrives from the classification subagent
|
|
- **THEN** `expected_sha256`, `safety_tier`, and (for deterministic ops) `is_destructive`/`is_reversible` are absent from the proposal and are authored only by the finalize pass
|
|
|
|
### Requirement: Validate Before Rollover
|
|
|
|
The check SHALL validate the assembled report with `validate_report.py` on a scratch
|
|
path (not under `.dochygiene/`) and SHALL write the report pair only on validator exit
|
|
0. Because `StateStore.write_report` deletes the prior report pair before writing the
|
|
new one (invariant #4), validation SHALL NOT run against `.dochygiene/`, so a
|
|
validation failure never destroys the last good report. On a validation failure (exit
|
|
1) the check SHALL NOT write the report; on an empty shortlist or no signal-bearing
|
|
files the check SHALL still write a valid empty-`entries` report and stamp
|
|
`last_check`.
|
|
|
|
#### Scenario: Invalid report is never written
|
|
|
|
- **WHEN** the assembled report fails validation (exit 1)
|
|
- **THEN** the check does not call `write_report`, the prior report pair is preserved, and the offending entries are re-prompted or dropped before re-validating
|
|
|
|
#### Scenario: Validation runs on a scratch path
|
|
|
|
- **WHEN** the check validates the assembled report
|
|
- **THEN** validation runs against a scratch path outside `.dochygiene/`, so the last good report in `.dochygiene/` is never deleted by a failed run
|
|
|
|
#### Scenario: Empty shortlist still produces a valid report and stamp
|
|
|
|
- **WHEN** the scanner returns no signal-bearing files
|
|
- **THEN** the check writes a valid report with empty `entries` and stamps `last_check`
|
|
|
|
### Requirement: Report Pair Is Written and last_check Stamped
|
|
|
|
On a successful check, the skill SHALL write exactly one machine report
|
|
(`.dochygiene/report.json`) and one human report (`.dochygiene/report.md`) via
|
|
`StateStore.write_report` (atomic, rollover-bounded to one pair per invariant #4), and
|
|
SHALL stamp `last_check` to the same run instant used as the envelope `generated_at`.
|
|
The human report SHALL be a deterministic skeleton grouping entries by Stale, Bloat,
|
|
and Cleared with per-entry path, category, op, tier, token count, and signal, and a
|
|
header showing the timestamp, scope, files scanned, and candidate/cleared counts; only
|
|
an optional per-entry "why" gloss MAY be model-written.
|
|
|
|
#### Scenario: One report pair survives the write
|
|
|
|
- **WHEN** the check completes successfully
|
|
- **THEN** exactly one `report.json` and one `report.md` exist in `.dochygiene/`, and any prior pair has been rolled over
|
|
|
|
#### Scenario: last_check matches the envelope timestamp
|
|
|
|
- **WHEN** the check writes the report
|
|
- **THEN** `last_check` is stamped to the same run instant recorded as the envelope `generated_at`
|
|
|
|
### Requirement: Classifier Golden Examples Are Hermetic and Human-Gated
|
|
|
|
Classifier golden examples SHALL live under `examples/golden/classifier/<n>-<name>/`
|
|
(an `input/` fixture tree with stable hashes, an `expected.json` schema-valid report,
|
|
and an optional `notes.md`), distinct from the schema-shape fixtures
|
|
(`valid_report.json` / `invalid_report.json`). The golden unit harness SHALL be
|
|
hermetic — it SHALL NOT call a live model — and SHALL assert only deterministic,
|
|
stable parts: that the scanner emits the expected signals on the right paths, that
|
|
each `expected.json` validates (exit 0), and that the stable fields (`category.class`,
|
|
`category.subtype`, `op_type`, derived `safety_tier`, `exact_edit.kind`) match a
|
|
captured/committed check output. Op-prose and exact anchor line numbers SHALL be
|
|
advisory (flagged for review, not hard-failed). The live model-classification
|
|
regression SHALL be a separate, manually or agent-invoked harness, not part of the
|
|
unit suite. Adding or changing classifier goldens SHALL be human-gated per the
|
|
META-RULE.
|
|
|
|
#### Scenario: Golden harness makes no live model call
|
|
|
|
- **WHEN** the classifier golden unit tests run
|
|
- **THEN** they assert scanner signals, `expected.json` validity, and stable-field matches against a committed capture, with no live model invocation
|
|
|
|
#### Scenario: Classifier goldens are distinct from schema fixtures
|
|
|
|
- **WHEN** a contributor looks for the schema-shape fixtures versus the classifier goldens
|
|
- **THEN** the schema fixtures (`valid_report.json` / `invalid_report.json`) and the classifier goldens (`examples/golden/classifier/`) are separate, and `examples/golden/CONTEXT.md` documents the distinction
|
|
|
|
#### Scenario: Changing a golden requires human approval
|
|
|
|
- **WHEN** a contributor adds or changes a classifier golden example
|
|
- **THEN** the change is gated on explicit human approval per the META-RULE before it takes effect
|
|
|