569 lines
25 KiB
Markdown
569 lines
25 KiB
Markdown
---
|
|
description: Learn new lifecycle rules for a project by clustering unmatched files, nominating candidate globs (cheap model), and having a strong model judge and confirm/reject/amend them, with a mandatory rule report to the human before any persistence. Invoked by `/os-doc-hygiene:calibrate`.
|
|
---
|
|
|
|
# Hygiene Calibrate Skill
|
|
|
|
Orchestrates the learn-new-rules loop (lifecycle-spec.md §8): **cluster-and-
|
|
sample → cheap-model nominate → deterministic intake filter (drop repeat
|
|
rejections, carry related rejections + open consults forward) → strong-
|
|
model judge → rule report (human, including open consults) → persist →
|
|
retest**. It runs over the **unmatched pool** (unmatched = unmanaged = not
|
|
governed by any existing rulebook rule, per `rulebook.py`), and is the only
|
|
new skill this change adds — `check`/`clean` are unchanged in structure
|
|
(ADR-0039/-0041, `lifecycle-spec.md` §7).
|
|
|
|
All scripts live under `${CLAUDE_PLUGIN_ROOT}/scripts/`. Run them with
|
|
`python3` from the user's project directory (`cwd`). Use the session
|
|
scratchpad directory for all intermediate artifacts.
|
|
|
|
> **Precondition:** requires `CLAUDE_PLUGIN_ROOT`. Every script path resolves
|
|
> against it; abort rather than guessing a path if it is unset.
|
|
|
|
> Pick a scratch dir once and reuse it: `SCRATCH="$(mktemp -d)"`.
|
|
|
|
## What this skill NEVER does
|
|
|
|
- It never applies a rule without the human having seen the Step 4 rule
|
|
report first (spec: "no rule shall be persisted before this report has
|
|
been shown").
|
|
- It never removes a rule automatically — removals are HITL-only in all
|
|
cases, with recorded reasoning (spec §5).
|
|
- It never writes to the global `plugins/os-doc-hygiene/rulebook.json`
|
|
without an explicit, distinct confirmation beyond project-rule
|
|
confirmation (cross-repo write into cc-os).
|
|
- It never applies a drafted convention adoption (§6 below) without explicit
|
|
human confirmation.
|
|
|
|
---
|
|
|
|
## Workflow
|
|
|
|
### Step 1 — (D) Load the rulebook and scan for the unmatched pool
|
|
|
|
Same rulebook-load pattern as `check`'s Step 0.5, but here the candidate pool
|
|
is the **unmatched** paths — files the current rulebook leaves ungoverned —
|
|
not the signal-bearing shortlist.
|
|
|
|
```bash
|
|
export SCRATCH
|
|
python3 -c '
|
|
import json, os, sys
|
|
from pathlib import Path
|
|
sys.path.insert(0, os.environ["CLAUDE_PLUGIN_ROOT"] + "/scripts")
|
|
from rulebook import load_rulebook, RulebookLoadError
|
|
from scanner import Scanner, _resolve_project_root, _git_log_real, _git_commit_time_real
|
|
|
|
root = _resolve_project_root(Path.cwd())
|
|
project_rules = root / ".dochygiene-rules.json"
|
|
try:
|
|
rulebook = load_rulebook(project_path=project_rules if project_rules.is_file() else None)
|
|
except RulebookLoadError as exc:
|
|
print(json.dumps({"error": "rulebook-load-failed", "detail": str(exc)}))
|
|
sys.exit(2)
|
|
|
|
scanner = Scanner(
|
|
root=root, rulebook=rulebook,
|
|
git_log_fn=_git_log_real, git_commit_time_fn=_git_commit_time_real,
|
|
)
|
|
artifact = scanner.run()
|
|
|
|
# The unmatched pool is every file the scan encountered that carries no
|
|
# lifecycle signal AND is not itself an IGNORE-pruned/directory-rule
|
|
# aggregate entry -- i.e. shortlist entries with no rulebook governance.
|
|
signals = artifact.get("signals", {})
|
|
unmatched = [p for p in artifact.get("shortlist", []) if p not in signals]
|
|
|
|
Path(os.environ["SCRATCH"] + "/scan.json").write_text(json.dumps(artifact, indent=2))
|
|
Path(os.environ["SCRATCH"] + "/unmatched.json").write_text(json.dumps(unmatched, indent=2))
|
|
print(f"unmatched pool: {len(unmatched)} paths")
|
|
'
|
|
```
|
|
|
|
- Exit `2` / rulebook load failure → hard STOP, same as `check` Step 0.5. Do
|
|
not proceed with a silently-empty rulebook.
|
|
- Note: `signals` here means ANY signal (stale/bloat/lifecycle) — a file with
|
|
a stale/bloat signal but no lifecycle rule match is still "unmatched" with
|
|
respect to the rulebook, and belongs in the pool. Filter precisely on
|
|
`lifecycle`-named signals if the project has files carrying only
|
|
non-lifecycle signals that should stay in the pool:
|
|
|
|
```python
|
|
unmatched = [
|
|
p for p in artifact["shortlist"]
|
|
if not any(s.get("name") == "lifecycle" for s in artifact.get("signals", {}).get(p, []))
|
|
]
|
|
```
|
|
|
|
Use this refined filter, not the simpler one above, when `signals` may carry
|
|
non-lifecycle entries for shortlisted paths.
|
|
|
|
If `unmatched` is empty → report "Nothing to calibrate — every shortlisted
|
|
file is already governed by a rulebook rule." **STOP.**
|
|
|
|
---
|
|
|
|
### Step 2 — (D) Cluster and sample — `calibrate_helpers.ClusterSampler`
|
|
|
|
Deterministic, no model. Groups unmatched paths by path-shape (directory
|
|
prefix + filename shape class — digit runs collapse to `#`, hex-looking runs
|
|
collapse to `~`) and samples representatives per cluster, capped.
|
|
|
|
```bash
|
|
python3 -c '
|
|
import json, os, sys
|
|
from pathlib import Path
|
|
sys.path.insert(0, os.environ["CLAUDE_PLUGIN_ROOT"] + "/scripts")
|
|
from calibrate_helpers import ClusterSampler
|
|
|
|
unmatched = json.loads(Path(os.environ["SCRATCH"] + "/unmatched.json").read_text())
|
|
clusters = ClusterSampler().cluster_to_dicts(unmatched)
|
|
Path(os.environ["SCRATCH"] + "/clusters.json").write_text(json.dumps(clusters, indent=2))
|
|
print(f"{len(clusters)} clusters")
|
|
'
|
|
```
|
|
|
|
Each cluster is `{key, dir_prefix, shape, total, sample}`. Rules are always
|
|
proposed against a cluster, never a single instance in isolation (spec: "the
|
|
nomination is derived from a cluster of similar unmatched paths").
|
|
|
|
---
|
|
|
|
### Step 3 — (M) Cheap-model nomination — **haiku subagent, one per cluster**
|
|
|
|
For each cluster, dispatch a **haiku** subagent (LOOP-GUARD: point it at
|
|
`workflows/nominate.md`, never this SKILL.md) constrained to nominate a bare
|
|
glob pattern + candidate lifetime — patterns only, never an exact-instance
|
|
glob (a run-id, hash, or bare timestamp hardcoded into the glob).
|
|
|
|
```
|
|
Agent tool parameters:
|
|
- subagent_type: "general-purpose"
|
|
- model: haiku
|
|
- description: "Nominate lifecycle rule for cluster: <cluster.key>"
|
|
- prompt: |
|
|
Read and follow the workflow at:
|
|
${CLAUDE_PLUGIN_ROOT}/skills/calibrate/workflows/nominate.md
|
|
|
|
Cluster: <cluster.dir_prefix> / shape <cluster.shape>
|
|
Total matching paths: <cluster.total>
|
|
Sample paths:
|
|
<cluster.sample, one per line>
|
|
|
|
Return ONLY the JSON object specified in the workflow.
|
|
```
|
|
|
|
Collect all nominations into `"$SCRATCH/nominations.json"` (array, one per
|
|
cluster, tagged with the originating `cluster.key`).
|
|
|
|
**Do not trust a haiku nomination as final.** The "class, never path"
|
|
rule-quality test is enforced by the strong-model judge (Step 4) plus the
|
|
deterministic `RuleQualityChecker` (Step 5's report), never accepted from
|
|
haiku at face value.
|
|
|
|
---
|
|
|
|
### Step 3.5 — (D) Nomination intake filter — `calibrate_helpers.NominationIntakeFilter`
|
|
|
|
Deterministic, no model (lifecycle-spec.md §8 step 3;
|
|
`NominationIntakeFilter` requirement). Reads the project rules file's
|
|
`nominations` key via `RulesFileWriter.load`, then drops any nomination that
|
|
exactly repeats a `rejected` glob+lifetime, annotates survivors with every
|
|
*related* rejection (match-set intersection on the current shortlist), and
|
|
passes ALL open consults through unconditionally — this is the input the
|
|
judge prompt's "Nominations memory" section (Step 4) consumes.
|
|
|
|
```bash
|
|
python3 -c '
|
|
import json, os, sys
|
|
from pathlib import Path
|
|
sys.path.insert(0, os.environ["CLAUDE_PLUGIN_ROOT"] + "/scripts")
|
|
from calibrate_helpers import RulesFileWriter, NominationIntakeFilter
|
|
|
|
scan = json.loads(Path(os.environ["SCRATCH"] + "/scan.json").read_text())
|
|
project_rules = Path(scan["project_root"]) / ".dochygiene-rules.json"
|
|
nominations = json.loads(Path(os.environ["SCRATCH"] + "/nominations.json").read_text())
|
|
shortlist = scan.get("shortlist", [])
|
|
|
|
writer = RulesFileWriter()
|
|
data, load_warnings = writer.load(project_rules)
|
|
memory = data.get("nominations", {})
|
|
|
|
result = NominationIntakeFilter(
|
|
rejected=memory.get("rejected", []),
|
|
consults=memory.get("consults", []),
|
|
).filter(nominations, shortlist)
|
|
|
|
Path(os.environ["SCRATCH"] + "/intake.json").write_text(json.dumps(result, indent=2))
|
|
print(f"{len(result[\"survivors\"])} survivors, {len(result[\"dropped\"])} dropped, {len(result[\"consults\"])} open consults")
|
|
if result["dropped"]:
|
|
for d in result["dropped"]:
|
|
print(f" dropped: {d[\"glob\"]} -> {d[\"lifetime\"]} ({d[\"reason\"]})")
|
|
'
|
|
```
|
|
|
|
- **Surface every drop in the run summary** shown to the human alongside the
|
|
Step 5 report — a dropped nomination never reaches the judge, so this is
|
|
the only place it is visible.
|
|
- `intake.json`'s `survivors` (each nomination plus its `related_rejections`
|
|
annotation) and `consults` (all open consults, unconditionally) are what
|
|
gets embedded in Step 4's judge prompt as the "Nominations memory" input
|
|
section — feed the whole `survivors` array (not the raw
|
|
`nominations.json`) forward into Step 4, and include `consults` even when
|
|
empty (an empty array is a valid, meaningful "no open consults" signal).
|
|
|
|
---
|
|
|
|
### Step 4 — (M) Strong-model batched judgment — **ONE Opus/Fable subagent**
|
|
|
|
Dispatch a **single batched** strong-model subagent (`model: opus`, or the
|
|
project's configured Fable-tier model) to judge ALL nominations from Step 3.5
|
|
in one call (LOOP-GUARD: point it at `workflows/judge.md`, never this
|
|
SKILL.md). The judge gathers its OWN evidence — re-reads matched paths
|
|
against the live tree, checks near-miss boundaries — rather than trusting
|
|
the haiku nomination's claims.
|
|
|
|
Judge on `intake.json`'s `survivors` (each nomination annotated with its
|
|
`related_rejections`), never the raw `nominations.json` — the dropped
|
|
exact-repeats never reach this step. `intake.json`'s `consults` is the
|
|
"Open consults" input regardless of what haiku nominated this round.
|
|
|
|
```
|
|
Agent tool parameters:
|
|
- subagent_type: "general-purpose"
|
|
- model: opus
|
|
- description: "Judge doc-hygiene calibration nominations"
|
|
- prompt: |
|
|
Read and follow the workflow at:
|
|
${CLAUDE_PLUGIN_ROOT}/skills/calibrate/workflows/judge.md
|
|
|
|
Project root: <scan.project_root>
|
|
|
|
Nominations to judge (verbatim, each with related_rejections):
|
|
<contents of $SCRATCH/intake.json's "survivors">
|
|
|
|
Nominations memory — open consults (unconditional, may be empty):
|
|
<contents of $SCRATCH/intake.json's "consults">
|
|
|
|
Seed intake: <see "Seed intake" below — include or omit per pass>
|
|
|
|
Return ONLY the JSON array of verdicts specified in the workflow.
|
|
```
|
|
|
|
**Verdicts** are exactly one of `confirm` / `reject` / `amend` / `consult`.
|
|
`consult` is MANDATORY whenever the judge cannot determine if an artifact is
|
|
regenerable or must be retained — never resolved to `confirm` or `reject` in
|
|
that case. Write the judge's verdict array to `"$SCRATCH/verdicts.json"`.
|
|
|
|
**Seed intake:** the #41 clutter-inventory seed candidates enter at THIS
|
|
step (judge intake), for every calibration run **except** cc-os calibration
|
|
pass #1, which withholds them as a sealed answer key (one-off carve-out, see
|
|
`lifecycle-spec.md` §9). If this run IS cc-os pass #1, do NOT include seed
|
|
candidates in the judge prompt. Every other run (including later cc-os runs)
|
|
includes full seed intake.
|
|
|
|
---
|
|
|
|
### Step 5 — (D) Rule report to the human — BEFORE any persistence
|
|
|
|
Deterministic, no model — `calibrate_helpers.RuleReportBuilder` plus
|
|
`RuleQualityChecker`. For every judge verdict of `confirm` or `amend` (never
|
|
for `reject`/`consult` — those are not proposed for persistence), assemble
|
|
the 5-element report and run the quality lints:
|
|
|
|
```bash
|
|
python3 -c '
|
|
import json, os, sys
|
|
from pathlib import Path
|
|
sys.path.insert(0, os.environ["CLAUDE_PLUGIN_ROOT"] + "/scripts")
|
|
from calibrate_helpers import RuleReportBuilder, RuleQualityChecker
|
|
|
|
scan = json.loads(Path(os.environ["SCRATCH"] + "/scan.json").read_text())
|
|
verdicts = json.loads(Path(os.environ["SCRATCH"] + "/verdicts.json").read_text())
|
|
all_paths = scan.get("shortlist", [])
|
|
|
|
proposed = [v["rule"] for v in verdicts if v["verdict"] in ("confirm", "amend")]
|
|
builder = RuleReportBuilder()
|
|
checker = RuleQualityChecker()
|
|
|
|
report = []
|
|
for rule in proposed:
|
|
entry = builder.build(rule, all_paths).to_dict()
|
|
entry["quality"] = {
|
|
"class_not_path": checker.class_not_path(rule["glob"], all_paths).to_dict(),
|
|
}
|
|
report.append(entry)
|
|
|
|
Path(os.environ["SCRATCH"] + "/rule_report.json").write_text(json.dumps(report, indent=2))
|
|
print(json.dumps(report, indent=2))
|
|
'
|
|
```
|
|
|
|
Render this to the human as **patterns and examples, not JSON schema** — per
|
|
rule:
|
|
|
|
```
|
|
Proposed rule: <glob verbatim>
|
|
Lifetime: <lifetime> Tier: <auto|confirm>
|
|
Matches (<total>): <sample paths, one per line> [+ N more]
|
|
Near-miss (does NOT match, but looks similar): <near-miss paths, or "(none)">
|
|
Why: <plain-language "what this is and why it's clutter">
|
|
Quality check: <PASS, or the flagged reason if class_not_path failed>
|
|
```
|
|
|
|
**If `class_not_path` failed** (a glob that can, by construction, only ever
|
|
match one file — a failed generalization), flag it LOUDLY in the rendered
|
|
report rather than silently dropping or persisting it; ask the human whether
|
|
to have the judge re-amend it (loop back to Step 4 for that one rule) or
|
|
drop it from this round.
|
|
|
|
#### Open consults section
|
|
|
|
List every entry from `$SCRATCH/intake.json`'s `consults` — the open
|
|
consults from prior runs (plus any new `consult` verdict this run) — one per
|
|
entry:
|
|
|
|
```
|
|
Open consult: <glob>
|
|
Asked: <asked_on> Cluster: <cluster_key>
|
|
Question: <question>
|
|
Evidence: <evidence>
|
|
```
|
|
|
|
Each open consult has exactly three exits, all human, none of which is
|
|
gated the way a rule confirmation is:
|
|
|
|
- **(a) Answer settles the purpose** — the human's answer determines a
|
|
lifetime; persist a normal rule through this same report/Step 6 flow AND
|
|
delete the consult entry, in the same write (the rule supersedes it).
|
|
- **(b) Not rule-worthy** — the human decides the artifact class isn't a
|
|
rule; rewrite the entry into `nominations.rejected` with
|
|
`rejected_by: "human"` and the human's stated why.
|
|
- **(c) Defer** — no answer this round; the entry stays as-is and resurfaces
|
|
in the next `:calibrate` run's Step 3.5/4/5.
|
|
|
|
New rejections and consults produced this run (from judge `consult`
|
|
verdicts, or from exit (b) above) appear in this report for visibility but
|
|
are **not individually gated** — unlike proposed rules, they carry no
|
|
deletion authority, only memory. Only proposed rules go through the "Persist
|
|
these N project rules?" gate below.
|
|
|
|
**No rule is written anywhere until the human has seen this report and
|
|
responded.** Ask: "Persist these N project rules? (yes / no / a subset by
|
|
number)". Separately, resolve each open consult per its three exits above.
|
|
Only proceed to Step 6 for the rules the human approves and the consults the
|
|
human has settled or declined this round.
|
|
|
|
---
|
|
|
|
### Step 6 — (D) Persistence — canonical writer only
|
|
|
|
Every settled verdict persists through `calibrate_helpers.RulesFileWriter`
|
|
(load → mutate the parsed dict → write) — no code path serializes
|
|
`.dochygiene-rules.json` by hand. Before running this, translate the
|
|
human's Step 5 responses into four in-memory lists (the orchestrating skill's
|
|
own bookkeeping — there is no scratch file for these, since they come from
|
|
the live conversation, not a subagent):
|
|
|
|
- `approved_rules` — the judge `confirm`/`amend` rule objects the human said
|
|
yes to (including `keep`-lifetime rules and Step-5-exit-(a) rules that
|
|
answer an open consult).
|
|
- `declined` — `(rule, why)` pairs for judge-proposed rules the human said
|
|
no to at the report.
|
|
- `settled_consult_globs` — globs of open consults the human resolved this
|
|
run, either via exit (a) (answered, folded into `approved_rules` above) or
|
|
exit (b) (declined, folded into `new_human_rejections` below); these are
|
|
removed from `nominations.consults` in the same write.
|
|
- `new_human_rejections` — `(glob, lifetime, why)` for exit-(b) consults
|
|
("not rule-worthy"), written with `rejected_by: "human"`.
|
|
- `still_open_consults` — judge `consult` verdicts from THIS run
|
|
(`{glob, question, evidence, cluster_key}`) that remain unresolved after
|
|
Step 5 (exit (c), or simply not reached this round).
|
|
|
|
```bash
|
|
python3 -c '
|
|
import json, os, sys
|
|
from datetime import date
|
|
from pathlib import Path
|
|
sys.path.insert(0, os.environ["CLAUDE_PLUGIN_ROOT"] + "/scripts")
|
|
from calibrate_helpers import RulesFileWriter
|
|
|
|
scan = json.loads(Path(os.environ["SCRATCH"] + "/scan.json").read_text())
|
|
project_rules = Path(scan["project_root"]) / ".dochygiene-rules.json"
|
|
today = date.today().isoformat()
|
|
|
|
# approved_rules / declined / settled_consult_globs / new_human_rejections /
|
|
# still_open_consults come from the human'"'"'s Step 5 responses (see above).
|
|
|
|
writer = RulesFileWriter()
|
|
data, load_warnings = writer.load(project_rules)
|
|
nominations = data.setdefault("nominations", {"consults": [], "rejected": []})
|
|
|
|
# 1. Judge confirm/amend verdicts the human approved -> plain rules
|
|
# (keep verdicts and exact-path keep singletons allowed -- the keep-tier
|
|
# relaxation). confirmed_by/confirmed_on are set here, never by the judge.
|
|
for rule in approved_rules:
|
|
rule["confirmed_by"] = "human"
|
|
rule["confirmed_on"] = today
|
|
data["rules"].append(rule)
|
|
|
|
# 2. Human declines at the rule report -> rejected entries, so a later
|
|
# haiku round cannot re-nominate the identical glob+lifetime without the
|
|
# judge knowing.
|
|
for rule, why in declined:
|
|
nominations["rejected"].append({
|
|
"glob": rule["glob"], "lifetime": rule["lifetime"],
|
|
"why": why, "rejected_by": "human", "judged_on": today,
|
|
})
|
|
|
|
# 3. Consults the human declined this run (Step 5 exit b) -> rejected
|
|
# entries too, same rejected_by/judged_on convention.
|
|
for glob_pattern, lifetime, why in new_human_rejections:
|
|
nominations["rejected"].append({
|
|
"glob": glob_pattern, "lifetime": lifetime,
|
|
"why": why, "rejected_by": "human", "judged_on": today,
|
|
})
|
|
|
|
# 4. Still-open consults from this run -> nominations.consults, deduped by
|
|
# glob (an existing entry with the same glob wins -- never duplicated).
|
|
existing_consult_globs = {c["glob"] for c in nominations["consults"]}
|
|
for consult in still_open_consults:
|
|
if consult["glob"] not in existing_consult_globs:
|
|
nominations["consults"].append({**consult, "asked_on": today})
|
|
|
|
# 5. Consults the human just settled this run (answered or declined) leave
|
|
# nominations.consults -- the rule (step 1) or rejection (step 3)
|
|
# supersedes them.
|
|
nominations["consults"] = [
|
|
c for c in nominations["consults"] if c["glob"] not in settled_consult_globs
|
|
]
|
|
|
|
write_warnings = writer.write(project_rules, data)
|
|
'
|
|
```
|
|
|
|
- **Project rules** (the common case): land in `<project-root>/.dochygiene-
|
|
rules.json` on judge `confirm`/`amend` PLUS this step's human approval —
|
|
including plain `lifetime: keep` rules for judge `keep`-purpose verdicts,
|
|
exact-path singletons allowed under the keep-tier relaxation.
|
|
- **Human declines** at the rule report persist as `rejected` entries with
|
|
`rejected_by: "human"` (never silently dropped) — see item 2 above.
|
|
- **Open `consult` verdicts** persist to `nominations.consults`, deduped by
|
|
glob at write time — see item 4 above. An answered consult (Step 5 exit a)
|
|
is deleted from `consults` in the same write that persists its superseding
|
|
rule; a declined consult (exit b) is deleted from `consults` in the same
|
|
write that adds its `rejected` entry; a deferred consult (exit c) is left
|
|
untouched and resurfaces next run.
|
|
- **Global rulebook writes** (`plugins/os-doc-hygiene/rulebook.json`) require
|
|
a SEPARATE, EXPLICIT confirmation beyond the project-rule approval above —
|
|
this is a cross-repo write into cc-os itself. Ask distinctly: "This rule
|
|
looks like it belongs in the GLOBAL rulebook (applies to every project),
|
|
not just this one. Write it to the global rulebook instead/as well? (yes/
|
|
no)". Only write on explicit "yes" to THIS question. (Rejections/consults
|
|
are always project-scoped memory — never written to the global rulebook.)
|
|
- **Removals are HITL-only, always**, regardless of scope: only remove a
|
|
rule when the human explicitly asks to, with the reasoning recorded in the
|
|
rule's own `note` field (or a comment in the calibration run's summary) —
|
|
never as an automatic side effect of a calibration pass. New rejections
|
|
and consults are memory, not deletion authority, and are never gated the
|
|
way a rule persist/remove is (Step 5).
|
|
|
|
Each persisted rule gets `confirmed_by` (the human's decision, not the
|
|
judge's — a model-proposed rule may never set `confirm: true` on itself,
|
|
it may only ask) and `confirmed_on` (today's date) per the rulebook schema
|
|
(`rulebook.py`'s `_KNOWN_FIELDS`).
|
|
|
|
---
|
|
|
|
### Consult loop — worked example (map #49, #56/#59)
|
|
|
|
Worked example of consult persistence (`lifecycle-spec.md` §2 "Nominations
|
|
memory"), wired through Steps 3.5/5/6 above. Run 1: the judge returns
|
|
`consult` on `docs/orchestration-audit/*.md` ("retained audit trail, or
|
|
disposable once the tune-up lands?"). Step 6's writer persists it to
|
|
`nominations.consults` (glob, question, evidence, cluster_key, asked_on —
|
|
deliberately no lifetime). Run 2, weeks later: the deterministic intake filter injects the
|
|
still-open consult into the judge prompt's "Nominations memory" section, AND
|
|
the Step 5 report renders it under "Open consults". Three exits, all human:
|
|
(a) the human answers "audit trail" → a `lifetime: keep` rule is persisted
|
|
and the consult entry deleted (the rule supersedes it); (b) "not
|
|
rule-worthy" → the entry is rewritten into `nominations.rejected` with the
|
|
human's why (`rejected_by: "human"`); (c) no answer → the entry stays and
|
|
resurfaces on run 3. A consult never filters files and never expires on its
|
|
own.
|
|
|
|
---
|
|
|
|
### Step 7 — (D) Retest loop
|
|
|
|
Re-run Steps 1-6 (including 3.5) against the shrunk unmatched pool. Stop
|
|
when:
|
|
|
|
- a round yields **fewer than 2 new persisted rules**, OR
|
|
- the unmatched pool shrank by **less than 10%** since the previous round,
|
|
- **hard cap: 3 rounds**, regardless of shrink rate.
|
|
|
|
Track round count and the unmatched-pool size at the start of each round in
|
|
the scratch dir (`$SCRATCH/round_N_unmatched_count.txt`) to compute shrink %.
|
|
|
|
---
|
|
|
|
## §6 (design.md) — draft convention adoption, never apply unasked
|
|
|
|
While reviewing the unmatched pool, `:calibrate` MAY notice a pattern that
|
|
would benefit from a `conventions.json` convention (`archive-bucket` or
|
|
`status-frontmatter`) rather than a plain rule. If so, it MAY draft the
|
|
adoption — the graduated rule (e.g. `served_when_path: <dir>/archive/{name}`)
|
|
PLUS the concrete file moves or frontmatter additions the convention
|
|
implies — and present it to the human alongside the Step 5 rule report, for
|
|
approval. **It never applies a drafted adoption without explicit
|
|
confirmation** — no rulebook write and no file move happens until the human
|
|
confirms.
|
|
|
|
---
|
|
|
|
## Calibration pass #1 (cc-os) — special-case reminders
|
|
|
|
See `lifecycle-spec.md` §9 and openspec task group 6 for the full protocol.
|
|
When run is explicitly cc-os pass #1:
|
|
|
|
- Withhold the #41 seed candidates from judge intake (Step 4) — do NOT paste
|
|
them into the judge prompt.
|
|
- The **protected set is a hard gate**: if ANY rule proposed for persistence
|
|
(Step 5/6) has a glob matching a path in the protected set (eval
|
|
`scenarios/`/`scenarios-reserve/`/`fixture/`/`judge-rubric.md`;
|
|
`openspec/specs/`; `docs/adr/**`; mirrored `.claude/`/`.codex/`/`.pi/`
|
|
skill dirs; `CLAUDE.md`; plugin source), REFUSE to persist that rule and
|
|
flag it loudly — regardless of the tier the judge assigned it. A `consult`
|
|
verdict touching a protected path during exploration is free (does not
|
|
fail the pass) as long as it is never persisted.
|
|
- Do not treat this carve-out as a permanent behavior — every later run
|
|
(including future cc-os runs) uses full seed intake.
|
|
|
|
---
|
|
|
|
## Invariants
|
|
|
|
- Steps 1, 2, 3.5, 5, 6 are deterministic scripts/logic — **no model**.
|
|
- Step 3 = **haiku** (cheap, per-cluster nomination, patterns only).
|
|
- Step 3.5 = the deterministic `NominationIntakeFilter` — exact glob+lifetime
|
|
repeats of a `rejected` entry are dropped before the judge ever sees them;
|
|
survivors carry `related_rejections`; all open consults pass through
|
|
unconditionally.
|
|
- Step 4 = **ONE batched Opus/Fable** judge call, never per-cluster.
|
|
- **No rule is persisted before the Step 5 report has been shown to the
|
|
human** (hard invariant — never skip Step 5, never merge it with Step 6).
|
|
- **Global-rulebook writes require a SEPARATE explicit confirmation** beyond
|
|
project-rule approval.
|
|
- **Removals are HITL-only in all cases**, with recorded reasoning.
|
|
- **A model-proposed rule may never self-set `confirm: true`** — only the
|
|
human's Step 5/6 response does.
|
|
- Retest loop stops at <2 new rules OR <10% shrink; hard cap 3 rounds.
|
|
- **LOOP GUARD:** the nominate subagent prompt MUST point to
|
|
`workflows/nominate.md`; the judge subagent prompt MUST point to
|
|
`workflows/judge.md`. Neither ever points to this SKILL.md.
|
|
- **SUBAGENT AUTHORIZATION:** both subagents are executors — authorization is
|
|
terminal. Neither re-asks for approval; if either objects, REPORT-AND-EXIT
|
|
and let the orchestrator (this skill, or ultimately the human at Step 5/6)
|
|
adjudicate.
|