os-doc-hygiene: calibrate assessment-inventory design locked — spec deltas + ADR amendments (map #49, ticket #58)
Persist the full :calibrate assessment inventory + rulebook refinements per wayfinder map #49 (tickets #51-#57, #59): nominations memory (consults/rejected) in the rules file, deterministic nomination intake filter, writer-enforced canonical ordering, keep-verdict persistence with keep-tier relaxation, enumerate-siblings rule-quality test, extracted.md extract-index convention + global keep rule, judge.md corrective edits, and the test-coverage decisions. Implementation hands off to an OpenSpec change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LjFRroqLsvkL2WrMJtRBrK
This commit is contained in:
parent
807dcc8d90
commit
9f11df3e2a
|
|
@ -2,14 +2,24 @@
|
||||||
"schema_version": 1,
|
"schema_version": 1,
|
||||||
"rules": [
|
"rules": [
|
||||||
{
|
{
|
||||||
"glob": "autoresearch/*/",
|
"glob": "autoresearch/classic-*/",
|
||||||
"lifetime": "temporary",
|
"lifetime": "temporary",
|
||||||
"retain_recent": 3,
|
"retain_recent": 3,
|
||||||
"max_age_days": 30,
|
"max_age_days": 30,
|
||||||
"confirmed_by": "human",
|
"confirmed_by": "human",
|
||||||
"confirmed_on": "2026-07-15",
|
"confirmed_on": "2026-07-15",
|
||||||
"source": "calibration pass #1 (lifecycle-aware-doc-hygiene)",
|
"source": "calibration pass #1 (lifecycle-aware-doc-hygiene); narrowed from autoresearch/*/ 2026-07-15",
|
||||||
"note": "Autoresearch run directories are disposable run output; keep the 3 newest, age out the rest."
|
"note": "Autoresearch run directories are disposable run output; keep the 3 newest, age out the rest. Narrowed to conventional prefixes so autoresearch/ can hold keep-worthy content."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"glob": "autoresearch/improve-*/",
|
||||||
|
"lifetime": "temporary",
|
||||||
|
"retain_recent": 3,
|
||||||
|
"max_age_days": 30,
|
||||||
|
"confirmed_by": "human",
|
||||||
|
"confirmed_on": "2026-07-15",
|
||||||
|
"source": "calibration pass #1 (lifecycle-aware-doc-hygiene); narrowed from autoresearch/*/ 2026-07-15",
|
||||||
|
"note": "Autoresearch run directories are disposable run output; keep the 3 newest, age out the rest. Narrowed to conventional prefixes so autoresearch/ can hold keep-worthy content."
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"glob": "docs/orchestration-audit/factsheets/*.md",
|
"glob": "docs/orchestration-audit/factsheets/*.md",
|
||||||
|
|
|
||||||
|
|
@ -25,3 +25,7 @@ Rule overrides are visible in review and travel with the repo — a rule that ca
|
||||||
## Alternatives rejected
|
## Alternatives rejected
|
||||||
|
|
||||||
(1) .cc-os/dochygiene-rules.json — rejected: .cc-os/ is gitignored local config; deletion-authorizing rules must be committed, reviewable, and shared, not per-machine. (2) Global rulebook only, no override — rejected: repos where specs/plans are the product need exceptions (ticket #34). (3) YAML/TOML with comments — rejected in #40: JSON parses with stdlib in both the Python pipeline and any tooling; a per-rule note field substitutes for comments.
|
(1) .cc-os/dochygiene-rules.json — rejected: .cc-os/ is gitignored local config; deletion-authorizing rules must be committed, reviewable, and shared, not per-machine. (2) Global rulebook only, no override — rejected: repos where specs/plans are the product need exceptions (ticket #34). (3) YAML/TOML with comments — rejected in #40: JSON parses with stdlib in both the Python pipeline and any tooling; a per-rule note field substitutes for comments.
|
||||||
|
|
||||||
|
## Amendment (2026-07-15 — wayfinder map #49)
|
||||||
|
|
||||||
|
The rules file's scope now includes **nominations memory**: an optional second top-level key `nominations` with two sub-keys — `consults` (open judge questions that resurface each `:calibrate` run until answered) and `rejected` (declined nominations, judge- or human-declined, that block exact glob+lifetime repeats at nomination intake and travel to the judge as context for variants). The key is additive to schema_version 1 (the v1 loader ignores unknown top-level keys); `rulebook.py` stays nomination-unaware — only the calibrate pipeline reads it. Serialization of the whole file is writer-enforced canonical ordering (rules grouped delete-once-served → temporary → keep, glob-sorted within groups; `nominations` after `rules`, `consults` before `rejected`) — no hook. The global rulebook additionally carries a `**/extracted.md → keep` rule, protecting the per-directory extract-index convention (map #49 ticket #57) in every project with zero per-project bookkeeping. Detail: plugins/os-doc-hygiene/lifecycle-spec.md (§1 extract index; §2 nominations memory + canonical ordering); decision tickets #51–#53, #56, #57.
|
||||||
|
|
|
||||||
|
|
@ -25,3 +25,7 @@ Routine hygiene runs unattended on the safe population while every non-recoverab
|
||||||
## Alternatives rejected
|
## Alternatives rejected
|
||||||
|
|
||||||
(1) Confirm everything — rejected: rulebook becomes advisory; confirm-fatigue makes the human rubber-stamp. (2) Tier by file type or a recoverable_via/regenerate class — rejected: recoverability is a property of git state at runtime, not of the rule's claim. (3) 90-day retention windows — rejected as far too slow in the age of AI. (4) mtime-only age — rejected: clone/branch-switch resets every mtime, silently putting the whole rulebook to sleep. (5) Model-settable confirm:true — rejected: used too liberally, everything drifts back to always-confirm.
|
(1) Confirm everything — rejected: rulebook becomes advisory; confirm-fatigue makes the human rubber-stamp. (2) Tier by file type or a recoverable_via/regenerate class — rejected: recoverability is a property of git state at runtime, not of the rule's claim. (3) 90-day retention windows — rejected as far too slow in the age of AI. (4) mtime-only age — rejected: clone/branch-switch resets every mtime, silently putting the whole rulebook to sleep. (5) Model-settable confirm:true — rejected: used too liberally, everything drifts back to always-confirm.
|
||||||
|
|
||||||
|
## Amendment (2026-07-15 — wayfinder map #49)
|
||||||
|
|
||||||
|
Tier interactions decided on map #49: (1) the "class, never path" rule-quality test is relaxed for the **keep tier only** — exact-path singleton keep rules are allowed (the test exists to prevent bad deletion rules; a singleton keep merely protects) and leave the rulebook only by hand-deletion, never an automated revisit: an automated revisit would be the one place a machine argues a keep back toward deletion, the exact failure mode the keep tier prevents. Instance globs stay forbidden for temporary/delete-once-served. (2) Nominations memory (ADR-0038 amendment) is memory, not deletion authority: rejections and consults never filter files and are not individually gated at persistence; rules alone authorize deletion. (3) Consults resurface in `:calibrate` only — `:check`/`:clean` are unchanged. Detail: plugins/os-doc-hygiene/lifecycle-spec.md §2/§8; decision tickets #51, #52, #56.
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
# Lifecycle-aware doc hygiene — spec
|
# Lifecycle-aware doc hygiene — spec
|
||||||
|
|
||||||
_Status: Locked design (wayfinder map [#31](https://forgejo.swansoncloud.com/jared/cc-os/issues/31)); assembled 2026-07-14 from decision tickets #32–#48._
|
_Status: Locked design (wayfinder map [#31](https://forgejo.swansoncloud.com/jared/cc-os/issues/31)); assembled 2026-07-14 from decision tickets #32–#48. Extended 2026-07-15 by wayfinder map [#49](https://forgejo.swansoncloud.com/jared/cc-os/issues/49) (assessment-inventory persistence + rulebook refinements; tickets #51–#57, #59)._
|
||||||
_Last updated: 2026-07-14_
|
_Last updated: 2026-07-15_
|
||||||
|
|
||||||
Extends os-doc-hygiene from stale/bloat monitoring to lifecycle management: a
|
Extends os-doc-hygiene from stale/bloat monitoring to lifecycle management: a
|
||||||
rulebook assigns every managed file a **lifetime**, `:check`/`:clean` gain
|
rulebook assigns every managed file a **lifetime**, `:check`/`:clean` gain
|
||||||
|
|
@ -9,8 +9,10 @@ delete and extract-then-delete behavior, and a new `:calibrate` skill learns
|
||||||
rules per project. Validated against calibration project #1 (cc-os) per the
|
rules per project. Validated against calibration project #1 (cc-os) per the
|
||||||
criteria in [Calibration](#calibration-pass-1-cc-os).
|
criteria in [Calibration](#calibration-pass-1-cc-os).
|
||||||
|
|
||||||
ADR set: [ADR-0038](../../docs/adr/) (rulebook location), ADR-0039 (deletion
|
ADR set: [ADR-0038](../../docs/adr/) (rulebook location; amended 2026-07-15 —
|
||||||
autonomy tiers), ADR-0040 (no ignore-surface propagation), ADR-0041
|
rules-file scope includes nominations memory + the extract-index keep rule),
|
||||||
|
ADR-0039 (deletion autonomy tiers; amended 2026-07-15 — map #49 tier
|
||||||
|
interactions), ADR-0040 (no ignore-surface propagation), ADR-0041
|
||||||
(determinism-promotion principle).
|
(determinism-promotion principle).
|
||||||
|
|
||||||
## 1. Lifetime taxonomy (#33)
|
## 1. Lifetime taxonomy (#33)
|
||||||
|
|
@ -25,6 +27,44 @@ Three lifetimes, plus one modifier:
|
||||||
repo-durable residue → ADR/CLAUDE.md/docs; cross-repo lessons → SecondBrain
|
repo-durable residue → ADR/CLAUDE.md/docs; cross-repo lessons → SecondBrain
|
||||||
via `/os-vault:write`. No new destinations, no "retired specs" pile.
|
via `/os-vault:write`. No new destinations, no "retired specs" pile.
|
||||||
|
|
||||||
|
### Extract index (`extracted.md`) — map #49 [#57](https://forgejo.swansoncloud.com/jared/cc-os/issues/57)
|
||||||
|
|
||||||
|
When an extract-then-delete sends content to the **vault** (the case where the
|
||||||
|
insight physically leaves the repo), the deletion leaves a pointer behind in a
|
||||||
|
per-directory **`extracted.md`** — the index stays where a future reader is
|
||||||
|
already looking. Repo-durable residue (an ADR, a CLAUDE.md line) is already
|
||||||
|
discoverable in-repo and gets **no** index line. Entry format:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
- [[vault: tool/graphify-clustering-behavior]] — why HDBSCAN over-merges doc
|
||||||
|
clusters; read before tuning any graphify clustering params.
|
||||||
|
(extracted from `graphify-clustering-notes.md`, 2026-07-15)
|
||||||
|
```
|
||||||
|
|
||||||
|
- **The `:clean` extract op owns the entry.** The op runs: distill →
|
||||||
|
`/os-vault:write` (returns the note name) → append the pointer line →
|
||||||
|
delete the original — atomically, in `:clean`'s single reviewable commit,
|
||||||
|
so there is no window where the doc is gone but undiscoverable.
|
||||||
|
`os-vault:write` stays repo-agnostic.
|
||||||
|
- **Self-protection is one global rule**: `{ "glob": "**/extracted.md",
|
||||||
|
"lifetime": "keep" }` in the global rulebook — the self-describing filename
|
||||||
|
means zero per-project bookkeeping and the index can never become a
|
||||||
|
deletion candidate anywhere. (A generic `index.md` name was rejected: it
|
||||||
|
collides with existing index/README conventions, and a global
|
||||||
|
`**/index.md → keep` is far too broad.)
|
||||||
|
- **Extract-worthy vs plain-keep — three tests, in order:**
|
||||||
|
1. **Cited → KEEP.** Anything repo artifacts link to or derive from stays
|
||||||
|
(deleting breaks the provenance chain).
|
||||||
|
2. **Generalizes AND standalone → EXTRACT.** The insight would change
|
||||||
|
behavior in a *different* repo (tool behavior, methodology — the vault
|
||||||
|
routing test) and nothing in-repo links to it.
|
||||||
|
3. **Unclear → consult.** The existing mandatory verdict; never guess an
|
||||||
|
extract.
|
||||||
|
- The convention is spec-defined and universal — part of the extract op
|
||||||
|
itself, **not** a `conventions.json` entry (that catalog holds
|
||||||
|
completion-conventions a project opts into via served-signal graduation;
|
||||||
|
the extract index is neither).
|
||||||
|
|
||||||
Distinct from all lifetimes: an **IGNORE surface** — paths the scanner never
|
Distinct from all lifetimes: an **IGNORE surface** — paths the scanner never
|
||||||
walks at all (vs `keep` = walked and reported, never deleted). Seed members:
|
walks at all (vs `keep` = walked and reported, never deleted). Seed members:
|
||||||
`graphify-out/**`, `.dochygiene/**`. The ignore surface is an explicit list,
|
`graphify-out/**`, `.dochygiene/**`. The ignore surface is an explicit list,
|
||||||
|
|
@ -41,7 +81,10 @@ keepable — #43).
|
||||||
(matches the `.dochygiene-ignore` precedent; deliberately NOT in gitignored
|
(matches the `.dochygiene-ignore` precedent; deliberately NOT in gitignored
|
||||||
`.cc-os/` — ADR-0038).
|
`.cc-os/` — ADR-0038).
|
||||||
|
|
||||||
Both use envelope `{"schema_version": 1, "rules": [...]}`.
|
Both use envelope `{"schema_version": 1, "rules": [...]}`. The project file
|
||||||
|
may additionally carry a top-level `nominations` key (see [Nominations
|
||||||
|
memory](#nominations-memory--map-49-5256) below); the v1 loader ignores
|
||||||
|
unknown top-level keys, so the key is additive to `schema_version` 1.
|
||||||
|
|
||||||
### Glob dialect
|
### Glob dialect
|
||||||
|
|
||||||
|
|
@ -104,6 +147,84 @@ or unknown `schema_version`.
|
||||||
`.dochygiene-ignore` remains a hand-authored, human-only escape hatch,
|
`.dochygiene-ignore` remains a hand-authored, human-only escape hatch,
|
||||||
unmanaged by the rulebook (#44).
|
unmanaged by the rulebook (#44).
|
||||||
|
|
||||||
|
### Nominations memory (`nominations` key) — map #49 (#52/#56)
|
||||||
|
|
||||||
|
Every judge verdict with an actual answer (keep / temporary /
|
||||||
|
delete-once-served / ignore) persists as a plain rule in `rules` (#51 — keep
|
||||||
|
verdicts are ordinary `lifetime: keep` rules; matched = managed removes them
|
||||||
|
from the calibrate pool, and the glob covers future files for free). The
|
||||||
|
`nominations` key holds only the two residues that cannot be rules: "the
|
||||||
|
answer is no" (`rejected`) and "no answer yet" (`consults`). It **never
|
||||||
|
touches the file filter** — rules alone decide which files are governed.
|
||||||
|
|
||||||
|
```json
|
||||||
|
"nominations": {
|
||||||
|
"consults": [
|
||||||
|
{
|
||||||
|
"glob": "docs/orchestration-audit/*.md",
|
||||||
|
"question": "Are the dated audit tune-up reports a retained audit trail, or disposable once the tune-up lands?",
|
||||||
|
"evidence": "4 files, one per audit run; docs/implementation-status.md references only the latest",
|
||||||
|
"cluster_key": "docs/orchestration-audit::report-#",
|
||||||
|
"asked_on": "2026-07-15"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"rejected": [
|
||||||
|
{
|
||||||
|
"glob": "docs/research/**",
|
||||||
|
"lifetime": "temporary",
|
||||||
|
"why": "findings docs other artifacts link to, not disposable output",
|
||||||
|
"consider_instead": "an extract-to-vault rule for standalone findings docs",
|
||||||
|
"rejected_by": "judge",
|
||||||
|
"judged_on": "2026-07-15"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Rejection entry:** `glob`, `lifetime`, `why`, optional `consider_instead`
|
||||||
|
(the judge's amend pointer), `rejected_by: "judge" | "human"`, `judged_on`.
|
||||||
|
Human declines at the §8 rule report also persist as rejections
|
||||||
|
(`rejected_by: "human"`) — otherwise haiku can re-nominate what the human
|
||||||
|
personally declined and the judge won't know.
|
||||||
|
- **Consult entry:** `glob`, `question`, `evidence`, `cluster_key`,
|
||||||
|
`asked_on`. Deliberately **no lifetime** — the uncertainty about purpose is
|
||||||
|
the point. Presence in `consults` = open; no status field. This fixes
|
||||||
|
consult evaporation (verdicts previously died with the run's scratch dir;
|
||||||
|
the same question was re-derived every run).
|
||||||
|
- **Consult exits:** (a) a human answer settles purpose → a normal rule is
|
||||||
|
persisted and the consult entry deleted (the rule supersedes it); (b) "not
|
||||||
|
rule-worthy" → rewritten into `rejected` with the human's why; (c) defer →
|
||||||
|
stays, resurfaces next `:calibrate` run. Consults resurface in `:calibrate`
|
||||||
|
**only** (`:check`/`:clean` unchanged — ADR-0039 boundary) and never
|
||||||
|
filter anything.
|
||||||
|
- **What a rejection blocks — exact glob+lifetime repeats only.** A variant
|
||||||
|
(different lifetime, narrower glob) flows to the judge normally, carrying
|
||||||
|
the past rejection as context. Stale rejections leave by hand-deletion
|
||||||
|
(removals stay HITL, with recorded reasoning).
|
||||||
|
- Exact-path singleton **keep** rules (#51) likewise leave only by
|
||||||
|
hand-deletion — no automated revisit path: an automated revisit would be
|
||||||
|
the one place a machine argues a keep back toward deletion, the exact
|
||||||
|
failure mode the keep tier prevents.
|
||||||
|
- `rulebook.py` stays **nomination-unaware** — its contract is "which rule
|
||||||
|
governs this path"; only `calibrate_helpers` reads the key. The calibrate
|
||||||
|
reader warns on unrecognized nomination fields, mirroring the rules
|
||||||
|
array's unknown-field discipline.
|
||||||
|
- Writes land via the same canonical writer as rules (next subsection); new
|
||||||
|
rejections/consults appear in the §8 rule report but are not individually
|
||||||
|
gated — they are memory, not deletion authority; hand-editing is the
|
||||||
|
escape hatch.
|
||||||
|
|
||||||
|
### Canonical ordering — map #49 (#53)
|
||||||
|
|
||||||
|
**Writer-enforced, no hook.** Every code path that serializes
|
||||||
|
`.dochygiene-rules.json` writes it grouped by lifetime tier
|
||||||
|
(delete-once-served, temporary, keep), sorted by glob within each group;
|
||||||
|
`nominations` after `rules`, `consults` before `rejected` (the
|
||||||
|
pending-action queue reads first), each list glob-sorted. Idempotent; no
|
||||||
|
drift window between a command finishing and a hook firing; hand edits
|
||||||
|
re-canonicalize on the next write. (A post-command hook was considered and
|
||||||
|
rejected as more moving parts for the same result.)
|
||||||
|
|
||||||
## 3. Deletion semantics and autonomy tiers (#35, #43 — ADR-0039)
|
## 3. Deletion semantics and autonomy tiers (#35, #43 — ADR-0039)
|
||||||
|
|
||||||
Delete = **true git deletion in a dedicated hygiene commit**; git history is
|
Delete = **true git deletion in a dedicated hygiene commit**; git history is
|
||||||
|
|
@ -224,12 +345,20 @@ The learn-new-rules loop, run per project:
|
||||||
cluster, not one member.
|
cluster, not one member.
|
||||||
2. **Nomination (cheap model):** haiku nominates a bare glob + lifetime per
|
2. **Nomination (cheap model):** haiku nominates a bare glob + lifetime per
|
||||||
cluster — constrained to produce *patterns*, never exact-instance globs.
|
cluster — constrained to produce *patterns*, never exact-instance globs.
|
||||||
3. **Judgment (strong model):** one batched Opus/Fable judge gathers its own
|
3. **Nomination intake filter (deterministic —
|
||||||
|
`NominationIntakeFilter` in `calibrate_helpers.py`):** a nomination whose
|
||||||
|
glob+lifetime exactly equals a `rejected` entry (§2 Nominations memory)
|
||||||
|
is dropped before the judge, logged in the run summary. Survivors are
|
||||||
|
annotated with every *related* rejection — related = the two globs'
|
||||||
|
match sets intersect on the current shortlist (deterministic, from the
|
||||||
|
scan). Annotations plus all open consults enter the judge prompt as a
|
||||||
|
"Nominations memory" input section.
|
||||||
|
4. **Judgment (strong model):** one batched Opus/Fable judge gathers its own
|
||||||
evidence and authors final rule entries (#39: weak-model discoveries need
|
evidence and authors final rule entries (#39: weak-model discoveries need
|
||||||
strong-model confirmation). Verdicts: **confirm / reject / amend /
|
strong-model confirmation). Verdicts: **confirm / reject / amend /
|
||||||
consult** — consult is mandatory when an artifact's purpose is unclear
|
consult** — consult is mandatory when an artifact's purpose is unclear
|
||||||
(regenerable ≠ removable).
|
(regenerable ≠ removable).
|
||||||
4. **Rule report to the human — before any rule is persisted.** Per proposed
|
5. **Rule report to the human — before any rule is persisted.** Per proposed
|
||||||
rule, the report shows:
|
rule, the report shows:
|
||||||
1. the **glob verbatim**, exactly as it would be persisted;
|
1. the **glob verbatim**, exactly as it would be persisted;
|
||||||
2. every path it currently matches (or a capped sample + total count);
|
2. every path it currently matches (or a capped sample + total count);
|
||||||
|
|
@ -239,10 +368,15 @@ The learn-new-rules loop, run per project:
|
||||||
4. lifetime + behavior tier (auto vs confirm);
|
4. lifetime + behavior tier (auto vs confirm);
|
||||||
5. a plain-language why — what the artifact is and why it's clutter.
|
5. a plain-language why — what the artifact is and why it's clutter.
|
||||||
The human reviews patterns and examples, not JSON schema.
|
The human reviews patterns and examples, not JSON schema.
|
||||||
5. **Persistence:** project rules land on judge confirmation;
|
6. **Persistence:** project rules land on judge confirmation;
|
||||||
**global-rulebook writes are human-gated** (a cross-repo write into cc-os).
|
**global-rulebook writes are human-gated** (a cross-repo write into cc-os).
|
||||||
Rule removals are HITL-only, with recorded reasoning.
|
Rule removals are HITL-only, with recorded reasoning. Every settled
|
||||||
6. **Retest loop:** stop at <2 new rules OR <10% unmatched shrink; hard cap 3
|
verdict persists as a plain rule — judge `keep` verdicts become ordinary
|
||||||
|
`lifetime: keep` rules, including exact-path singletons (#51). Human
|
||||||
|
declines persist as `rejected` entries (`rejected_by: "human"`); open
|
||||||
|
`consult` verdicts persist to `nominations.consults`, deduped by glob at
|
||||||
|
write time (§2 Nominations memory).
|
||||||
|
7. **Retest loop:** stop at <2 new rules OR <10% unmatched shrink; hard cap 3
|
||||||
rounds.
|
rounds.
|
||||||
|
|
||||||
Seed intake: the #41 clutter-inventory seed candidates enter at judge intake —
|
Seed intake: the #41 clutter-inventory seed candidates enter at judge intake —
|
||||||
|
|
@ -257,10 +391,43 @@ carve-out below).
|
||||||
bare timestamp). A rule matching one file today is fine; a rule that can
|
bare timestamp). A rule matching one file today is fine; a rule that can
|
||||||
only EVER match one file is a failed generalization — flag loudly, never
|
only EVER match one file is a failed generalization — flag loudly, never
|
||||||
silently persist.
|
silently persist.
|
||||||
|
**Keep-tier relaxation (map #49 #51):** exact-path/instance globs ARE
|
||||||
|
allowed for `lifetime: keep` entries only — this test exists to prevent
|
||||||
|
bad DELETION rules, and a singleton keep (e.g.
|
||||||
|
`docs/research/clutter-pattern-inventory.md`) merely protects. Instance
|
||||||
|
globs stay forbidden for `temporary`/`delete-once-served`.
|
||||||
- **Glob-breadth tie-breaker: prefer the NARROWER glob.** Too-narrow fails
|
- **Glob-breadth tie-breaker: prefer the NARROWER glob.** Too-narrow fails
|
||||||
safe (leaves clutter; the recurring pass catches it next round — self-
|
safe (leaves clutter; the recurring pass catches it next round — self-
|
||||||
healing). Too-broad fails dangerous (deletes a keeper — not self-healing).
|
healing). Too-broad fails dangerous (deletes a keeper — not self-healing).
|
||||||
Readability beats cleverness when the cost is one missed round.
|
Readability beats cleverness when the cost is one missed round.
|
||||||
|
- **Enumerate siblings — never widen to the container.** When the near-miss
|
||||||
|
boundary check reveals sibling artifacts a glob misses, the fix is to
|
||||||
|
ENUMERATE the conventional prefixes as separate rule entries
|
||||||
|
(`autoresearch/classic-*/` + `autoresearch/improve-*/`), never to widen to
|
||||||
|
the container (`autoresearch/*/`, `autoresearch/**`). A container-claiming
|
||||||
|
glob mortgages the directory's entire future: nothing keep-worthy can ever
|
||||||
|
live there without a counter-rule (a future
|
||||||
|
`autoresearch/methodology-notes.md` would be claimed by a deletion rule). A
|
||||||
|
sibling flavor that can't be named yet fails safe — it gets its own entry
|
||||||
|
next pass, the same self-healing property the narrower-glob tie-breaker
|
||||||
|
relies on. Container globs are justified ONLY when the directory is wholly
|
||||||
|
machine-owned (`plugins/*/.pytest_cache/`), where nothing keep-worthy can
|
||||||
|
appear inside.
|
||||||
|
|
||||||
|
### Test coverage for the map-#49 additions (#59)
|
||||||
|
|
||||||
|
- `NominationIntakeFilter` and the canonical-writer extension get **unit
|
||||||
|
tests only** (pure deterministic logic — invariant #6). Fixtures assert
|
||||||
|
exact-repeat drops, match-set-intersection relatedness annotations, and
|
||||||
|
round-trip ordering + unknown-field warnings.
|
||||||
|
- The judge's "Nominations memory" context and the consult resurfacing loop
|
||||||
|
get **no scenario harness** — both are human-gated downstream, so no
|
||||||
|
silent-failure path exists a harness would uniquely catch. Mitigation is
|
||||||
|
one worked example each (judge.md input section; calibrate SKILL.md), with
|
||||||
|
production IRL session audits as the next signal; IRL evidence of the
|
||||||
|
judge ignoring rejection context is what triggers harness design (reading
|
||||||
|
`~/Documents/SecondBrain/howto/running-autoresearch-skill-evals.md` first,
|
||||||
|
per eval discipline).
|
||||||
|
|
||||||
## 9. Calibration pass #1: cc-os (#45)
|
## 9. Calibration pass #1: cc-os (#45)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -14,6 +14,14 @@
|
||||||
"confirmed_on": "2026-07-14",
|
"confirmed_on": "2026-07-14",
|
||||||
"source": "lifecycle-spec #43",
|
"source": "lifecycle-spec #43",
|
||||||
"note": "IGNORE surface: legacy pre-ADR-0027 state directory, never walked. .cc-os/** (the current state dir) is already covered by scanner.py's pre-existing hardcoded self-exclusion (design.md §7 state-dir wrinkle)."
|
"note": "IGNORE surface: legacy pre-ADR-0027 state directory, never walked. .cc-os/** (the current state dir) is already covered by scanner.py's pre-existing hardcoded self-exclusion (design.md §7 state-dir wrinkle)."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"glob": "**/extracted.md",
|
||||||
|
"lifetime": "keep",
|
||||||
|
"confirmed_by": "human",
|
||||||
|
"confirmed_on": "2026-07-15",
|
||||||
|
"source": "map #49 ticket #57 (extract-index convention)",
|
||||||
|
"note": "Per-directory extract index: pointer lines left behind by :clean's extract-then-delete op for vault extractions. The self-describing filename lets one global keep rule protect every project's index; the index can never become a deletion candidate."
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -287,6 +287,26 @@ it may only ask) and `confirmed_on` (today's date) per the rulebook schema
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
### Consult loop — worked example (map #49, #56/#59)
|
||||||
|
|
||||||
|
Design-level example of consult persistence (`lifecycle-spec.md` §2
|
||||||
|
"Nominations memory"); the pipeline wiring lands with the nominations-memory
|
||||||
|
implementation. Run 1: the judge returns `consult` on
|
||||||
|
`docs/orchestration-audit/*.md` ("retained audit trail, or disposable once
|
||||||
|
the tune-up lands?"). Step 6's writer persists it to `nominations.consults`
|
||||||
|
(glob, question, evidence, cluster_key, asked_on — deliberately no
|
||||||
|
lifetime). Run 2, weeks later: the deterministic intake filter injects the
|
||||||
|
still-open consult into the judge prompt's "Nominations memory" section, AND
|
||||||
|
the Step 5 report renders it under "Open consults". Three exits, all human:
|
||||||
|
(a) the human answers "audit trail" → a `lifetime: keep` rule is persisted
|
||||||
|
and the consult entry deleted (the rule supersedes it); (b) "not
|
||||||
|
rule-worthy" → the entry is rewritten into `nominations.rejected` with the
|
||||||
|
human's why (`rejected_by: "human"`); (c) no answer → the entry stays and
|
||||||
|
resurfaces on run 3. A consult never filters files and never expires on its
|
||||||
|
own.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
### Step 7 — (D) Retest loop
|
### Step 7 — (D) Retest loop
|
||||||
|
|
||||||
Re-run Steps 1-6 against the shrunk unmatched pool. Stop when:
|
Re-run Steps 1-6 against the shrunk unmatched pool. Stop when:
|
||||||
|
|
|
||||||
|
|
@ -17,6 +17,30 @@ your OWN evidence.
|
||||||
- **Nominations** — an array of `{cluster_key, glob, lifetime, rationale,
|
- **Nominations** — an array of `{cluster_key, glob, lifetime, rationale,
|
||||||
confidence}` from the haiku pass (a `null` glob means "no nomination for
|
confidence}` from the haiku pass (a `null` glob means "no nomination for
|
||||||
this cluster" — skip it, no verdict needed).
|
this cluster" — skip it, no verdict needed).
|
||||||
|
- **Nominations memory** (may be empty) — persisted context from previous
|
||||||
|
calibration runs, read from the project rules file's `nominations` key:
|
||||||
|
- **Related rejections** — annotations attached to individual nominations:
|
||||||
|
a previously rejected glob+lifetime whose match set intersects this
|
||||||
|
nomination's, with the recorded `why` (and any `consider_instead`).
|
||||||
|
Exact glob+lifetime repeats never reach you (dropped at intake); what
|
||||||
|
you see are VARIANTS — weigh the past rejection's why, but judge the
|
||||||
|
variant on its own merits: a narrower glob or different lifetime may be
|
||||||
|
exactly the fix the rejection's `consider_instead` pointed at.
|
||||||
|
- **Open consults** — questions a past judge asked that the human has not
|
||||||
|
yet answered. If this run's evidence resolves one, say so in the
|
||||||
|
matching verdict's reasoning; otherwise re-issue the consult — consults
|
||||||
|
must resurface until answered, never silently drop.
|
||||||
|
|
||||||
|
Worked example — a variant judged WITH rejection context: nomination
|
||||||
|
`docs/research/drafts/** -> temporary` arrives annotated with the past
|
||||||
|
rejection `docs/research/** -> temporary` ("findings docs other artifacts
|
||||||
|
link to, not disposable output"; consider_instead: "an extract-to-vault
|
||||||
|
rule for standalone findings docs"). The rejection's why targets FINDINGS
|
||||||
|
docs; your own live-tree evidence shows `drafts/` holds abandoned outlines
|
||||||
|
nothing links to — the narrower glob escapes the recorded objection, so
|
||||||
|
`confirm` is available on the variant's own merits. The rejection is
|
||||||
|
context, not a veto.
|
||||||
|
|
||||||
- **Seed intake** (present on every run except cc-os calibration pass #1) —
|
- **Seed intake** (present on every run except cc-os calibration pass #1) —
|
||||||
the #41 clutter-inventory seed candidates, additional known clutter
|
the #41 clutter-inventory seed candidates, additional known clutter
|
||||||
patterns to weigh in alongside the haiku nominations.
|
patterns to weigh in alongside the haiku nominations.
|
||||||
|
|
@ -32,9 +56,9 @@ For EACH nomination (and each seed candidate, when present):
|
||||||
set is accurate.
|
set is accurate.
|
||||||
2. **Check the near-miss boundary** — are there sibling-looking paths that
|
2. **Check the near-miss boundary** — are there sibling-looking paths that
|
||||||
the glob would silently miss (the #45 bug class: `autoresearch/classic-*/`
|
the glob would silently miss (the #45 bug class: `autoresearch/classic-*/`
|
||||||
silently missing `autoresearch/improve-*/`)? If the glob is too narrow,
|
silently missing `autoresearch/improve-*/`)? If the glob misses a sibling, `amend` by adding the sibling's conventional
|
||||||
consider `amend`ing it to a broader (but still class-not-path-safe) glob
|
prefix as its own rule — do NOT widen to the container (see the
|
||||||
that catches the sibling too.
|
enumerate-siblings quality check below).
|
||||||
3. **Judge the artifact's actual purpose** — read a representative sample
|
3. **Judge the artifact's actual purpose** — read a representative sample
|
||||||
file's content if the nomination's paths/names alone don't make the
|
file's content if the nomination's paths/names alone don't make the
|
||||||
purpose obvious. Is it genuinely regenerable/disposable, or could it be
|
purpose obvious. Is it genuinely regenerable/disposable, or could it be
|
||||||
|
|
@ -52,6 +76,17 @@ For EACH nomination (and each seed candidate, when present):
|
||||||
a broader glob that both cover the cluster, prefer the narrower one —
|
a broader glob that both cover the cluster, prefer the narrower one —
|
||||||
too-narrow fails safe (self-healing, caught next round); too-broad
|
too-narrow fails safe (self-healing, caught next round); too-broad
|
||||||
fails dangerous (could delete a keeper, not self-healing).
|
fails dangerous (could delete a keeper, not self-healing).
|
||||||
|
- **Enumerate siblings, never widen to the container**: when the near-miss
|
||||||
|
check (step 2) reveals sibling artifacts, `amend` by ENUMERATING the
|
||||||
|
conventional prefixes as separate rules (`autoresearch/classic-*/` plus
|
||||||
|
`autoresearch/improve-*/`), NOT by widening to the container
|
||||||
|
(`autoresearch/*/`). A container glob claims the directory's entire
|
||||||
|
future — a later keep-worthy file (e.g.
|
||||||
|
`autoresearch/methodology-notes.md`) would sit under a deletion rule. A
|
||||||
|
sibling flavor you cannot yet name fails safe: it gets its own rule next
|
||||||
|
round. Widening to a container is allowed ONLY when the directory is
|
||||||
|
wholly machine-owned (`plugins/*/.pytest_cache/`) so nothing keep-worthy
|
||||||
|
can ever appear inside it.
|
||||||
|
|
||||||
## Verdicts — exactly one of four
|
## Verdicts — exactly one of four
|
||||||
|
|
||||||
|
|
@ -83,12 +118,23 @@ nomination (nominations with a `null` glob need no entry):
|
||||||
"cluster_key": "autoresearch::run-#",
|
"cluster_key": "autoresearch::run-#",
|
||||||
"verdict": "amend",
|
"verdict": "amend",
|
||||||
"rule": {
|
"rule": {
|
||||||
"glob": "autoresearch/*/",
|
"glob": "autoresearch/improve-*/",
|
||||||
"lifetime": "temporary",
|
"lifetime": "temporary",
|
||||||
"retain_recent": 3,
|
"retain_recent": 3,
|
||||||
"max_age_days": 3
|
"max_age_days": 3
|
||||||
},
|
},
|
||||||
"reasoning": "Haiku's glob (autoresearch/run-*/) missed autoresearch/improve-*/ and autoresearch/classic-*/ siblings observed in the live tree; widened to autoresearch/*/ which still excludes non-run files at that level."
|
"reasoning": "Haiku's glob (autoresearch/run-*/) named a prefix that does not occur; the live tree has autoresearch/classic-*/ and autoresearch/improve-*/ runs. Enumerated each conventional prefix as its own rule rather than widening to autoresearch/*/, which would claim the whole directory's future."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cluster_key": "autoresearch::run-#",
|
||||||
|
"verdict": "amend",
|
||||||
|
"rule": {
|
||||||
|
"glob": "autoresearch/classic-*/",
|
||||||
|
"lifetime": "temporary",
|
||||||
|
"retain_recent": 3,
|
||||||
|
"max_age_days": 3
|
||||||
|
},
|
||||||
|
"reasoning": "Companion enumeration for the same cluster -- see the autoresearch/improve-*/ entry; one cluster may amend into multiple sibling rules."
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cluster_key": "docs::HANDOFF-#",
|
"cluster_key": "docs::HANDOFF-#",
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue