--- description: Learn new lifecycle rules for a project by clustering unmatched files, nominating candidate globs (cheap model), and having a strong model judge and confirm/reject/amend them, with a mandatory rule report to the human before any persistence. Invoked by `/os-doc-hygiene:calibrate`. --- # Hygiene Calibrate Skill Orchestrates the learn-new-rules loop (lifecycle-spec.md §8): **cluster-and- sample → cheap-model nominate → strong-model judge → rule report (human) → persist → retest**. It runs over the **unmatched pool** (unmatched = unmanaged = not governed by any existing rulebook rule, per `rulebook.py`), and is the only new skill this change adds — `check`/`clean` are unchanged in structure (ADR-0039/-0041, `lifecycle-spec.md` §7). All scripts live under `${CLAUDE_PLUGIN_ROOT}/scripts/`. Run them with `python3` from the user's project directory (`cwd`). Use the session scratchpad directory for all intermediate artifacts. > **Precondition:** requires `CLAUDE_PLUGIN_ROOT`. Every script path resolves > against it; abort rather than guessing a path if it is unset. > Pick a scratch dir once and reuse it: `SCRATCH="$(mktemp -d)"`. ## What this skill NEVER does - It never applies a rule without the human having seen the Step 4 rule report first (spec: "no rule shall be persisted before this report has been shown"). - It never removes a rule automatically — removals are HITL-only in all cases, with recorded reasoning (spec §5). - It never writes to the global `plugins/os-doc-hygiene/rulebook.json` without an explicit, distinct confirmation beyond project-rule confirmation (cross-repo write into cc-os). - It never applies a drafted convention adoption (§6 below) without explicit human confirmation. --- ## Workflow ### Step 1 — (D) Load the rulebook and scan for the unmatched pool Same rulebook-load pattern as `check`'s Step 0.5, but here the candidate pool is the **unmatched** paths — files the current rulebook leaves ungoverned — not the signal-bearing shortlist. ```bash export SCRATCH python3 -c ' import json, os, sys from pathlib import Path sys.path.insert(0, os.environ["CLAUDE_PLUGIN_ROOT"] + "/scripts") from rulebook import load_rulebook, RulebookLoadError from scanner import Scanner, _resolve_project_root, _git_log_real, _git_commit_time_real root = _resolve_project_root(Path.cwd()) project_rules = root / ".dochygiene-rules.json" try: rulebook = load_rulebook(project_path=project_rules if project_rules.is_file() else None) except RulebookLoadError as exc: print(json.dumps({"error": "rulebook-load-failed", "detail": str(exc)})) sys.exit(2) scanner = Scanner( root=root, rulebook=rulebook, git_log_fn=_git_log_real, git_commit_time_fn=_git_commit_time_real, ) artifact = scanner.run() # The unmatched pool is every file the scan encountered that carries no # lifecycle signal AND is not itself an IGNORE-pruned/directory-rule # aggregate entry -- i.e. shortlist entries with no rulebook governance. signals = artifact.get("signals", {}) unmatched = [p for p in artifact.get("shortlist", []) if p not in signals] Path(os.environ["SCRATCH"] + "/scan.json").write_text(json.dumps(artifact, indent=2)) Path(os.environ["SCRATCH"] + "/unmatched.json").write_text(json.dumps(unmatched, indent=2)) print(f"unmatched pool: {len(unmatched)} paths") ' ``` - Exit `2` / rulebook load failure → hard STOP, same as `check` Step 0.5. Do not proceed with a silently-empty rulebook. - Note: `signals` here means ANY signal (stale/bloat/lifecycle) — a file with a stale/bloat signal but no lifecycle rule match is still "unmatched" with respect to the rulebook, and belongs in the pool. Filter precisely on `lifecycle`-named signals if the project has files carrying only non-lifecycle signals that should stay in the pool: ```python unmatched = [ p for p in artifact["shortlist"] if not any(s.get("name") == "lifecycle" for s in artifact.get("signals", {}).get(p, [])) ] ``` Use this refined filter, not the simpler one above, when `signals` may carry non-lifecycle entries for shortlisted paths. If `unmatched` is empty → report "Nothing to calibrate — every shortlisted file is already governed by a rulebook rule." **STOP.** --- ### Step 2 — (D) Cluster and sample — `calibrate_helpers.ClusterSampler` Deterministic, no model. Groups unmatched paths by path-shape (directory prefix + filename shape class — digit runs collapse to `#`, hex-looking runs collapse to `~`) and samples representatives per cluster, capped. ```bash python3 -c ' import json, os, sys from pathlib import Path sys.path.insert(0, os.environ["CLAUDE_PLUGIN_ROOT"] + "/scripts") from calibrate_helpers import ClusterSampler unmatched = json.loads(Path(os.environ["SCRATCH"] + "/unmatched.json").read_text()) clusters = ClusterSampler().cluster_to_dicts(unmatched) Path(os.environ["SCRATCH"] + "/clusters.json").write_text(json.dumps(clusters, indent=2)) print(f"{len(clusters)} clusters") ' ``` Each cluster is `{key, dir_prefix, shape, total, sample}`. Rules are always proposed against a cluster, never a single instance in isolation (spec: "the nomination is derived from a cluster of similar unmatched paths"). --- ### Step 3 — (M) Cheap-model nomination — **haiku subagent, one per cluster** For each cluster, dispatch a **haiku** subagent (LOOP-GUARD: point it at `workflows/nominate.md`, never this SKILL.md) constrained to nominate a bare glob pattern + candidate lifetime — patterns only, never an exact-instance glob (a run-id, hash, or bare timestamp hardcoded into the glob). ``` Agent tool parameters: - subagent_type: "general-purpose" - model: haiku - description: "Nominate lifecycle rule for cluster: " - prompt: | Read and follow the workflow at: ${CLAUDE_PLUGIN_ROOT}/skills/calibrate/workflows/nominate.md Cluster: / shape Total matching paths: Sample paths: Return ONLY the JSON object specified in the workflow. ``` Collect all nominations into `"$SCRATCH/nominations.json"` (array, one per cluster, tagged with the originating `cluster.key`). **Do not trust a haiku nomination as final.** The "class, never path" rule-quality test is enforced by the strong-model judge (Step 4) plus the deterministic `RuleQualityChecker` (Step 5's report), never accepted from haiku at face value. --- ### Step 4 — (M) Strong-model batched judgment — **ONE Opus/Fable subagent** Dispatch a **single batched** strong-model subagent (`model: opus`, or the project's configured Fable-tier model) to judge ALL nominations from Step 3 in one call (LOOP-GUARD: point it at `workflows/judge.md`, never this SKILL.md). The judge gathers its OWN evidence — re-reads matched paths against the live tree, checks near-miss boundaries — rather than trusting the haiku nomination's claims. ``` Agent tool parameters: - subagent_type: "general-purpose" - model: opus - description: "Judge doc-hygiene calibration nominations" - prompt: | Read and follow the workflow at: ${CLAUDE_PLUGIN_ROOT}/skills/calibrate/workflows/judge.md Project root: Nominations to judge (verbatim): Seed intake: Return ONLY the JSON array of verdicts specified in the workflow. ``` **Verdicts** are exactly one of `confirm` / `reject` / `amend` / `consult`. `consult` is MANDATORY whenever the judge cannot determine if an artifact is regenerable or must be retained — never resolved to `confirm` or `reject` in that case. Write the judge's verdict array to `"$SCRATCH/verdicts.json"`. **Seed intake:** the #41 clutter-inventory seed candidates enter at THIS step (judge intake), for every calibration run **except** cc-os calibration pass #1, which withholds them as a sealed answer key (one-off carve-out, see `lifecycle-spec.md` §9). If this run IS cc-os pass #1, do NOT include seed candidates in the judge prompt. Every other run (including later cc-os runs) includes full seed intake. --- ### Step 5 — (D) Rule report to the human — BEFORE any persistence Deterministic, no model — `calibrate_helpers.RuleReportBuilder` plus `RuleQualityChecker`. For every judge verdict of `confirm` or `amend` (never for `reject`/`consult` — those are not proposed for persistence), assemble the 5-element report and run the quality lints: ```bash python3 -c ' import json, os, sys from pathlib import Path sys.path.insert(0, os.environ["CLAUDE_PLUGIN_ROOT"] + "/scripts") from calibrate_helpers import RuleReportBuilder, RuleQualityChecker scan = json.loads(Path(os.environ["SCRATCH"] + "/scan.json").read_text()) verdicts = json.loads(Path(os.environ["SCRATCH"] + "/verdicts.json").read_text()) all_paths = scan.get("shortlist", []) proposed = [v["rule"] for v in verdicts if v["verdict"] in ("confirm", "amend")] builder = RuleReportBuilder() checker = RuleQualityChecker() report = [] for rule in proposed: entry = builder.build(rule, all_paths).to_dict() entry["quality"] = { "class_not_path": checker.class_not_path(rule["glob"], all_paths).to_dict(), } report.append(entry) Path(os.environ["SCRATCH"] + "/rule_report.json").write_text(json.dumps(report, indent=2)) print(json.dumps(report, indent=2)) ' ``` Render this to the human as **patterns and examples, not JSON schema** — per rule: ``` Proposed rule: Lifetime: Tier: Matches (): [+ N more] Near-miss (does NOT match, but looks similar): Why: Quality check: ``` **If `class_not_path` failed** (a glob that can, by construction, only ever match one file — a failed generalization), flag it LOUDLY in the rendered report rather than silently dropping or persisting it; ask the human whether to have the judge re-amend it (loop back to Step 4 for that one rule) or drop it from this round. **No rule is written anywhere until the human has seen this report and responded.** Ask: "Persist these N project rules? (yes / no / a subset by number)". Only proceed to Step 6 for the rules the human approves. --- ### Step 6 — (D) Persistence - **Project rules** (the common case): land in `/.dochygiene- rules.json` on judge `confirm`/`amend` PLUS this step's human approval. Read-modify-write the envelope (`{"schema_version": 1, "rules": [...]}`, creating the file if absent), appending only — never mutating or removing an existing rule here. - **Global rulebook writes** (`plugins/os-doc-hygiene/rulebook.json`) require a SEPARATE, EXPLICIT confirmation beyond the project-rule approval above — this is a cross-repo write into cc-os itself. Ask distinctly: "This rule looks like it belongs in the GLOBAL rulebook (applies to every project), not just this one. Write it to the global rulebook instead/as well? (yes/ no)". Only write on explicit "yes" to THIS question. - **Removals are HITL-only, always**, regardless of scope: only remove a rule when the human explicitly asks to, with the reasoning recorded in the rule's own `note` field (or a comment in the calibration run's summary) — never as an automatic side effect of a calibration pass. Each persisted rule gets `confirmed_by` (the human's decision, not the judge's — a model-proposed rule may never set `confirm: true` on itself, it may only ask) and `confirmed_on` (today's date) per the rulebook schema (`rulebook.py`'s `_KNOWN_FIELDS`). --- ### Consult loop — worked example (map #49, #56/#59) Design-level example of consult persistence (`lifecycle-spec.md` §2 "Nominations memory"); the pipeline wiring lands with the nominations-memory implementation. Run 1: the judge returns `consult` on `docs/orchestration-audit/*.md` ("retained audit trail, or disposable once the tune-up lands?"). Step 6's writer persists it to `nominations.consults` (glob, question, evidence, cluster_key, asked_on — deliberately no lifetime). Run 2, weeks later: the deterministic intake filter injects the still-open consult into the judge prompt's "Nominations memory" section, AND the Step 5 report renders it under "Open consults". Three exits, all human: (a) the human answers "audit trail" → a `lifetime: keep` rule is persisted and the consult entry deleted (the rule supersedes it); (b) "not rule-worthy" → the entry is rewritten into `nominations.rejected` with the human's why (`rejected_by: "human"`); (c) no answer → the entry stays and resurfaces on run 3. A consult never filters files and never expires on its own. --- ### Step 7 — (D) Retest loop Re-run Steps 1-6 against the shrunk unmatched pool. Stop when: - a round yields **fewer than 2 new persisted rules**, OR - the unmatched pool shrank by **less than 10%** since the previous round, - **hard cap: 3 rounds**, regardless of shrink rate. Track round count and the unmatched-pool size at the start of each round in the scratch dir (`$SCRATCH/round_N_unmatched_count.txt`) to compute shrink %. --- ## §6 (design.md) — draft convention adoption, never apply unasked While reviewing the unmatched pool, `:calibrate` MAY notice a pattern that would benefit from a `conventions.json` convention (`archive-bucket` or `status-frontmatter`) rather than a plain rule. If so, it MAY draft the adoption — the graduated rule (e.g. `served_when_path: /archive/{name}`) PLUS the concrete file moves or frontmatter additions the convention implies — and present it to the human alongside the Step 5 rule report, for approval. **It never applies a drafted adoption without explicit confirmation** — no rulebook write and no file move happens until the human confirms. --- ## Calibration pass #1 (cc-os) — special-case reminders See `lifecycle-spec.md` §9 and openspec task group 6 for the full protocol. When run is explicitly cc-os pass #1: - Withhold the #41 seed candidates from judge intake (Step 4) — do NOT paste them into the judge prompt. - The **protected set is a hard gate**: if ANY rule proposed for persistence (Step 5/6) has a glob matching a path in the protected set (eval `scenarios/`/`scenarios-reserve/`/`fixture/`/`judge-rubric.md`; `openspec/specs/`; `docs/adr/**`; mirrored `.claude/`/`.codex/`/`.pi/` skill dirs; `CLAUDE.md`; plugin source), REFUSE to persist that rule and flag it loudly — regardless of the tier the judge assigned it. A `consult` verdict touching a protected path during exploration is free (does not fail the pass) as long as it is never persisted. - Do not treat this carve-out as a permanent behavior — every later run (including future cc-os runs) uses full seed intake. --- ## Invariants - Steps 1, 2, 5, 6 are deterministic scripts/logic — **no model**. - Step 3 = **haiku** (cheap, per-cluster nomination, patterns only). - Step 4 = **ONE batched Opus/Fable** judge call, never per-cluster. - **No rule is persisted before the Step 5 report has been shown to the human** (hard invariant — never skip Step 5, never merge it with Step 6). - **Global-rulebook writes require a SEPARATE explicit confirmation** beyond project-rule approval. - **Removals are HITL-only in all cases**, with recorded reasoning. - **A model-proposed rule may never self-set `confirm: true`** — only the human's Step 5/6 response does. - Retest loop stops at <2 new rules OR <10% shrink; hard cap 3 rounds. - **LOOP GUARD:** the nominate subagent prompt MUST point to `workflows/nominate.md`; the judge subagent prompt MUST point to `workflows/judge.md`. Neither ever points to this SKILL.md. - **SUBAGENT AUTHORIZATION:** both subagents are executors — authorization is terminal. Neither re-asks for approval; if either objects, REPORT-AND-EXIT and let the orchestrator (this skill, or ultimately the human at Step 5/6) adjudicate.