Design the nominations key schema and intake filter #56

New Issue

Closed

opened 2026-07-15 16:01:44 +00:00 by jared · 1 comment

jared commented

2026-07-15 16:01:44 +00:00

Owner

Map: #49

Question

Design the nominations key schema and the intake filter. Already decided (see map #49 decisions): same file as rules; rejects hard-block only the exact glob+lifetime pair; variants pass through with the past rejection shown as judge context; rejects may carry a consider-instead pointer; consults are a resurfacing queue, never a filter.

To design here:

Entry shape for a rejection — e.g. { "glob": "docs/research/**", "lifetime": "temporary", "verdict": "rejected", "why": "findings docs other artifacts link to, not disposable output", "consider_instead": "an extract-to-vault rule for standalone findings docs", "judged_on": "2026-07-15" } — confirm/adjust fields.
Entry shape for an open consult — what the judge asked, what evidence it gathered, status; how :calibrate re-presents it to the human next run and how it graduates (to a rule) or terminates (to a rejection).
Where the exact-repeat filter lands in the pipeline (nomination intake, before the judge) and where past-rejection context is injected (judge evidence).
Boundary question: do exact-path singleton keep rules ever get revisited, or is hand-deletion (HITL, recorded reasoning) the only path back? Charting lean: hand-deletion only — the discuss-and-edit loop is the escape hatch.
Canonical ordering of the nominations key (writer-enforced, after rules).

Map: #49 ## Question Design the `nominations` key schema and the intake filter. Already decided (see map #49 decisions): same file as rules; rejects hard-block only the exact glob+lifetime pair; variants pass through with the past rejection shown as judge context; rejects may carry a consider-instead pointer; consults are a resurfacing queue, never a filter. To design here: 1. **Entry shape for a rejection** — e.g. `{ "glob": "docs/research/**", "lifetime": "temporary", "verdict": "rejected", "why": "findings docs other artifacts link to, not disposable output", "consider_instead": "an extract-to-vault rule for standalone findings docs", "judged_on": "2026-07-15" }` — confirm/adjust fields. 2. **Entry shape for an open consult** — what the judge asked, what evidence it gathered, status; how :calibrate re-presents it to the human next run and how it graduates (to a rule) or terminates (to a rejection). 3. **Where the exact-repeat filter lands** in the pipeline (nomination intake, before the judge) and where past-rejection context is injected (judge evidence). 4. **Boundary question:** do exact-path singleton keep rules ever get revisited, or is hand-deletion (HITL, recorded reasoning) the only path back? Charting lean: hand-deletion only — the discuss-and-edit loop is the escape hatch. 5. Canonical ordering of the `nominations` key (writer-enforced, after `rules`).

jared added the

wayfinder:grilling

label 2026-07-15 16:01:44 +00:00

jared referenced this issue

2026-07-15 16:04:52 +00:00

Assemble the spec deltas + ADR amendments #58

jared referenced this issue

2026-07-15 16:05:21 +00:00

Residual assessments live in a nominations key in .dochygiene-rules.json; rejects block exact repeats only #52

jared self-assigned this 2026-07-15 16:33:23 +00:00

jared commented

2026-07-15 16:43:01 +00:00

Author

Owner

Resolution — design approved (human sign-off 2026-07-15). KISS boundary reaffirmed first: every verdict with an actual answer (keep/temporary/delete-once-served/ignore) persists as a plain rule in the existing rules array (#51). The nominations key holds only the two residues that cannot be rules: "the answer is no" (rejected) and "no answer yet" (consults).

Schema — second top-level key, two sub-keys (approved over flat array w/ verdict field)

{
  "schema_version": 1,
  "rules": [ "…unchanged…" ],
  "nominations": {
    "consults": [
      {
        "glob": "docs/orchestration-audit/*.md",
        "question": "Are the dated audit tune-up reports a retained audit trail, or disposable once the tune-up lands?",
        "evidence": "4 files, one per audit run; docs/implementation-status.md references only the latest",
        "cluster_key": "docs/orchestration-audit::report-#",
        "asked_on": "2026-07-15"
      }
    ],
    "rejected": [
      {
        "glob": "docs/research/**",
        "lifetime": "temporary",
        "why": "findings docs other artifacts link to, not disposable output",
        "consider_instead": "an extract-to-vault rule for standalone findings docs",
        "rejected_by": "judge",
        "judged_on": "2026-07-15"
      }
    ]
  }
}

Rejection entry: glob, lifetime, why, optional consider_instead, rejected_by: "judge" | "human", judged_on. No verdict field — the sub-key says it. Human declines at Step 5 also persist as rejections (rejected_by: "human"), approved — otherwise haiku can re-nominate what the human personally declined and the judge won't know.
Consult entry: glob, question, evidence, cluster_key, asked_on. Deliberately no lifetime — the uncertainty about purpose is the point. Presence in consults = open; no status field. Solves today's evaporation problem (consult verdicts die with the run's scratch dir; same question re-derived every run).
Consult exits: (a) human answer settles purpose → normal rule persisted, consult entry deleted (rule supersedes); (b) "not rule-worthy" → rewritten into rejected with the human's why; (c) defer → stays, resurfaces next run. Consults resurface in :calibrate only (:check/:clean unchanged, ADR-0039 boundary); they never filter anything (#52).

Pipeline placement

New deterministic Step 3.5 "Nomination intake filter" (NominationIntakeFilter in calibrate_helpers.py), between nomination collection (Step 3) and judge dispatch (Step 4):

Hard-drop: nomination whose glob+lifetime exactly equals a rejected entry is dropped pre-judge, logged in the run summary.
Context injection: survivors are annotated with every related rejection — related = the two globs' match sets intersect on the current shortlist (deterministic, from the scan). Annotations + all open consults enter the judge prompt as a new "Nominations memory" input section; consult verdicts dedupe by glob at write time.
rulebook.py stays nomination-unaware — its contract is "which rule governs this path"; only calibrate_helpers reads the key. Additive to schema_version 1 (v1 loader ignores extra top-level keys).
Writes land at Step 6 via the same canonical writer as rules; new rejections/consults are shown in the Step 5 report but not individually gated — memory, not deletion authority; hand-editing is the escape hatch (#52).

Boundary (sub-decision 4, approved)

Exact-path singleton keep rules leave only by hand-deletion (HITL, recorded reasoning) — no automated revisit path. Composes with the removals-are-HITL-only invariant; an automated revisit would be the one place a machine argues a keep back toward deletion, the exact failure mode the keep tier prevents.

Canonical ordering (sub-decision 5)

nominations after rules; consults before rejected (pending-action queue reads first); each list glob-sorted; all writer-enforced by the same Step 6 writer that orders rules (#53 — no hook). Calibrate reader warns on unrecognized nomination fields, mirroring the rules array's unknown-field discipline.

Folds into the assemble ticket #58; nothing edited in the working tree by this ticket.

**Resolution — design approved (human sign-off 2026-07-15).** KISS boundary reaffirmed first: every verdict with an actual answer (keep/temporary/delete-once-served/ignore) persists as a plain rule in the existing `rules` array (#51). The `nominations` key holds only the two residues that cannot be rules: "the answer is no" (rejected) and "no answer yet" (consults). ## Schema — second top-level key, two sub-keys (approved over flat array w/ verdict field) ```json { "schema_version": 1, "rules": [ "…unchanged…" ], "nominations": { "consults": [ { "glob": "docs/orchestration-audit/*.md", "question": "Are the dated audit tune-up reports a retained audit trail, or disposable once the tune-up lands?", "evidence": "4 files, one per audit run; docs/implementation-status.md references only the latest", "cluster_key": "docs/orchestration-audit::report-#", "asked_on": "2026-07-15" } ], "rejected": [ { "glob": "docs/research/**", "lifetime": "temporary", "why": "findings docs other artifacts link to, not disposable output", "consider_instead": "an extract-to-vault rule for standalone findings docs", "rejected_by": "judge", "judged_on": "2026-07-15" } ] } } ``` - **Rejection entry:** `glob`, `lifetime`, `why`, optional `consider_instead`, `rejected_by: "judge" | "human"`, `judged_on`. No `verdict` field — the sub-key says it. **Human declines at Step 5 also persist as rejections** (`rejected_by: "human"`), approved — otherwise haiku can re-nominate what the human personally declined and the judge won't know. - **Consult entry:** `glob`, `question`, `evidence`, `cluster_key`, `asked_on`. Deliberately **no lifetime** — the uncertainty about purpose is the point. Presence in `consults` = open; no status field. Solves today's evaporation problem (consult verdicts die with the run's scratch dir; same question re-derived every run). - Consult exits: (a) human answer settles purpose → normal rule persisted, consult entry deleted (rule supersedes); (b) "not rule-worthy" → rewritten into `rejected` with the human's why; (c) defer → stays, resurfaces next run. Consults resurface in `:calibrate` only (`:check`/`:clean` unchanged, ADR-0039 boundary); they never filter anything (#52). ## Pipeline placement New deterministic **Step 3.5 "Nomination intake filter"** (`NominationIntakeFilter` in `calibrate_helpers.py`), between nomination collection (Step 3) and judge dispatch (Step 4): - **Hard-drop:** nomination whose glob+lifetime exactly equals a `rejected` entry is dropped pre-judge, logged in the run summary. - **Context injection:** survivors are annotated with every *related* rejection — related = the two globs' match sets intersect on the current shortlist (deterministic, from the scan). Annotations + all open consults enter the judge prompt as a new "Nominations memory" input section; consult verdicts dedupe by glob at write time. - `rulebook.py` stays nomination-unaware — its contract is "which rule governs this path"; only `calibrate_helpers` reads the key. Additive to schema_version 1 (v1 loader ignores extra top-level keys). - Writes land at Step 6 via the same canonical writer as rules; new rejections/consults are shown in the Step 5 report but not individually gated — memory, not deletion authority; hand-editing is the escape hatch (#52). ## Boundary (sub-decision 4, approved) Exact-path singleton keep rules leave only by **hand-deletion** (HITL, recorded reasoning) — no automated revisit path. Composes with the removals-are-HITL-only invariant; an automated revisit would be the one place a machine argues a keep back toward deletion, the exact failure mode the keep tier prevents. ## Canonical ordering (sub-decision 5) `nominations` after `rules`; `consults` before `rejected` (pending-action queue reads first); each list glob-sorted; all writer-enforced by the same Step 6 writer that orders `rules` (#53 — no hook). Calibrate reader warns on unrecognized nomination fields, mirroring the rules array's unknown-field discipline. Folds into the assemble ticket #58; nothing edited in the working tree by this ticket.

jared closed this issue

2026-07-15 16:43:01 +00:00

jared referenced this issue

2026-07-15 16:43:23 +00:00

Decide eval/test coverage for the nominations intake filter and consult loop #59

jared referenced this issue

2026-07-15 16:43:43 +00:00

Wayfinder map: calibrate assessment inventory & rulebook refinements (os-doc-hygiene) #49