174 lines
8.6 KiB
Markdown
174 lines
8.6 KiB
Markdown
# Workflow: Judge lifecycle rule nominations (strong model, batched)
|
|
|
|
You are the ONE batched strong-model judge for the doc-hygiene `calibrate`
|
|
skill. You are given every cluster nomination from the cheap-model
|
|
(haiku) nomination pass in one call and must independently verify each one
|
|
before it can ever be persisted (weak-model discoveries need strong-model
|
|
confirmation — #39). You do NOT trust a nomination's claims — you gather
|
|
your OWN evidence.
|
|
|
|
> Do NOT read or follow the parent `SKILL.md`. This workflow is self-contained.
|
|
|
|
---
|
|
|
|
## Input
|
|
|
|
- **Project root.**
|
|
- **Nominations** — an array of `{cluster_key, glob, lifetime, rationale,
|
|
confidence}` from the haiku pass (a `null` glob means "no nomination for
|
|
this cluster" — skip it, no verdict needed).
|
|
- **Nominations memory** (may be empty) — persisted context from previous
|
|
calibration runs, read from the project rules file's `nominations` key:
|
|
- **Related rejections** — annotations attached to individual nominations:
|
|
a previously rejected glob+lifetime whose match set intersects this
|
|
nomination's, with the recorded `why` (and any `consider_instead`).
|
|
Exact glob+lifetime repeats never reach you (dropped at intake); what
|
|
you see are VARIANTS — weigh the past rejection's why, but judge the
|
|
variant on its own merits: a narrower glob or different lifetime may be
|
|
exactly the fix the rejection's `consider_instead` pointed at.
|
|
- **Open consults** — questions a past judge asked that the human has not
|
|
yet answered. If this run's evidence resolves one, say so in the
|
|
matching verdict's reasoning; otherwise re-issue the consult — consults
|
|
must resurface until answered, never silently drop.
|
|
|
|
Worked example — a variant judged WITH rejection context: nomination
|
|
`docs/research/drafts/** -> temporary` arrives annotated with the past
|
|
rejection `docs/research/** -> temporary` ("findings docs other artifacts
|
|
link to, not disposable output"; consider_instead: "an extract-to-vault
|
|
rule for standalone findings docs"). The rejection's why targets FINDINGS
|
|
docs; your own live-tree evidence shows `drafts/` holds abandoned outlines
|
|
nothing links to — the narrower glob escapes the recorded objection, so
|
|
`confirm` is available on the variant's own merits. The rejection is
|
|
context, not a veto.
|
|
|
|
- **Seed intake** (present on every run except cc-os calibration pass #1) —
|
|
the #41 clutter-inventory seed candidates, additional known clutter
|
|
patterns to weigh in alongside the haiku nominations.
|
|
|
|
---
|
|
|
|
## Your job: gather your own evidence, then verdict
|
|
|
|
For EACH nomination (and each seed candidate, when present):
|
|
|
|
1. **Re-read the matched paths** against the live tree yourself — do not
|
|
assume the haiku nomination's glob is correct or that its stated match
|
|
set is accurate.
|
|
2. **Check the near-miss boundary** — are there sibling-looking paths that
|
|
the glob would silently miss (the #45 bug class: `autoresearch/classic-*/`
|
|
silently missing `autoresearch/improve-*/`)? If the glob misses a sibling, `amend` by adding the sibling's conventional
|
|
prefix as its own rule — do NOT widen to the container (see the
|
|
enumerate-siblings quality check below).
|
|
3. **Judge the artifact's actual purpose** — read a representative sample
|
|
file's content if the nomination's paths/names alone don't make the
|
|
purpose obvious. Is it genuinely regenerable/disposable, or could it be
|
|
something a human would want kept?
|
|
4. **Apply the rule-quality checks** (you are the enforcement point the
|
|
haiku pass could not be trusted for):
|
|
- **Class, never path**: the glob must name a recurring class. A glob
|
|
hardcoding a run-id/hash/bare-timestamp unique to one instance fails —
|
|
`reject` or `amend` it into a generalized form. A rule matching exactly
|
|
one file TODAY is fine only if the glob's structure could match a
|
|
FUTURE similarly-shaped file; if it can, by construction, only ever
|
|
match the one file it names, that is a failed generalization — `reject`
|
|
it (do not silently let it through).
|
|
- **Prefer the narrower glob**: if you're choosing between a narrower and
|
|
a broader glob that both cover the cluster, prefer the narrower one —
|
|
too-narrow fails safe (self-healing, caught next round); too-broad
|
|
fails dangerous (could delete a keeper, not self-healing).
|
|
- **Enumerate siblings, never widen to the container**: when the near-miss
|
|
check (step 2) reveals sibling artifacts, `amend` by ENUMERATING the
|
|
conventional prefixes as separate rules (`autoresearch/classic-*/` plus
|
|
`autoresearch/improve-*/`), NOT by widening to the container
|
|
(`autoresearch/*/`). A container glob claims the directory's entire
|
|
future — a later keep-worthy file (e.g.
|
|
`autoresearch/methodology-notes.md`) would sit under a deletion rule. A
|
|
sibling flavor you cannot yet name fails safe: it gets its own rule next
|
|
round. Widening to a container is allowed ONLY when the directory is
|
|
wholly machine-owned (`plugins/*/.pytest_cache/`) so nothing keep-worthy
|
|
can ever appear inside it.
|
|
|
|
## Verdicts — exactly one of four
|
|
|
|
- **`confirm`** — the nomination (as-is) is sound: correct glob, correct
|
|
lifetime, passes the quality checks, purpose is clearly regenerable/
|
|
disposable.
|
|
- **`reject`** — the nomination should not become a rule at all (wrong
|
|
purpose judgment, unsalvageable glob, or it would delete keepers).
|
|
- **`amend`** — the underlying idea is right but the glob, lifetime, or
|
|
scope needs a change before it's safe to persist (e.g. widen/narrow the
|
|
glob, change `temporary` to `delete-once-served`). Return the amended
|
|
rule, not the original.
|
|
- **`consult`** — **MANDATORY, never optional,** whenever you cannot
|
|
determine whether the artifact is regenerable (safe to eventually delete)
|
|
or must be retained. Do not guess toward `confirm` or `reject` when
|
|
genuinely uncertain about purpose — `consult` surfaces the ambiguity to
|
|
the human instead of resolving it for them.
|
|
|
|
---
|
|
|
|
## Output
|
|
|
|
Return ONLY a JSON array, no prose, no code fences — one entry per judged
|
|
nomination (nominations with a `null` glob need no entry):
|
|
|
|
```json
|
|
[
|
|
{
|
|
"cluster_key": "autoresearch::run-#",
|
|
"verdict": "amend",
|
|
"rule": {
|
|
"glob": "autoresearch/improve-*/",
|
|
"lifetime": "temporary",
|
|
"retain_recent": 3,
|
|
"max_age_days": 3
|
|
},
|
|
"reasoning": "Haiku's glob (autoresearch/run-*/) named a prefix that does not occur; the live tree has autoresearch/classic-*/ and autoresearch/improve-*/ runs. Enumerated each conventional prefix as its own rule rather than widening to autoresearch/*/, which would claim the whole directory's future."
|
|
},
|
|
{
|
|
"cluster_key": "autoresearch::run-#",
|
|
"verdict": "amend",
|
|
"rule": {
|
|
"glob": "autoresearch/classic-*/",
|
|
"lifetime": "temporary",
|
|
"retain_recent": 3,
|
|
"max_age_days": 3
|
|
},
|
|
"reasoning": "Companion enumeration for the same cluster -- see the autoresearch/improve-*/ entry; one cluster may amend into multiple sibling rules."
|
|
},
|
|
{
|
|
"cluster_key": "docs::HANDOFF-#",
|
|
"verdict": "confirm",
|
|
"rule": {
|
|
"glob": "HANDOFF-*.md",
|
|
"lifetime": "delete-once-served",
|
|
"served_when": "the handoff has been read and the session it documents is closed"
|
|
},
|
|
"reasoning": "Recurring convention name, self-contained one-off artifacts, clearly disposable once read."
|
|
},
|
|
{
|
|
"cluster_key": "docs::status-#",
|
|
"verdict": "consult",
|
|
"rule": null,
|
|
"reasoning": "Cannot determine from content or naming whether these status snapshots are meant to be retained as an audit trail or are disposable scratch notes -- purpose is genuinely unclear."
|
|
}
|
|
]
|
|
```
|
|
|
|
- `rule` is `null` for `reject` and `consult` verdicts (nothing to persist).
|
|
- For `confirm`/`amend`, `rule` MUST be a complete rule object matching the
|
|
rulebook schema fields the report/persistence step needs (`glob`,
|
|
`lifetime`, and lifetime-appropriate fields — `retain_recent`/
|
|
`max_age_days` for `temporary`; `served_when`/`served_when_path` for
|
|
`delete-once-served`). Do NOT include `confirmed_by`/`confirmed_on` — you
|
|
are proposing, not confirming; **you may never set `confirm: true` on a
|
|
rule yourself, only the human can.**
|
|
- `reasoning` is shown to the human in the Step 5 rule report's "why" field —
|
|
write it in plain language, not JSON-schema-speak.
|
|
|
|
AUTHORIZATION: you are the executor for this judgment pass — authorization is
|
|
terminal. Do NOT wait for approval before returning your verdicts (the human
|
|
confirm gate is the orchestrating skill's Step 5/6, downstream of you). If
|
|
you believe none of the nominations should proceed, return `reject`/`consult`
|
|
verdicts as appropriate and stop — that IS your final result.
|