Check the researcher's stated cause before any design runs #980

New issue

Open

opened 2026-10-10 12:44:06 +00:00 by jared · 2 comments

jared commented

2026-10-10 12:44:06 +00:00

Owner

Problem

ADR-0285 made the researcher state cause {mechanism, defect_class} (plugins/os-sdlc/eval/ticket-takedown/harness/cards/researcher.md, last bullet; harness/schemas/research.json).
In live mode the grader grades every design against this cause (lib/takedown/query/live_grade_inputs.rb:13-16).
Designers also see the cause in the research reply (lib/takedown/presenter/run_prompts.rb:57).
The grader may not judge the cause (harness/cards/grader.md:11).
So one unchecked cause steers the designers and grades them, and nothing detects a shallow or wrong cause in live mode.
This is the root cause challenger: dig past the first instinct to a more systemic origin, so the fix is more resilient.
Neither guard may change harness/cards/grader.md or its schema, because that forces recalibration.
Source: Codex review.
Related: the sibling ticket "Measure how often the researcher's cause is shallower than the ruled cause, and flag runs where the critic names a deeper cause" (its result decides whether part (b) is needed now). Parent work: #974.

Reproduce

n/a: design ticket, nothing to reproduce.

Approach

(a) Research#verify! (lib/takedown/model/research.rb:11-20) rejects a blank mechanism or defect_class, and requires a path:line in mechanism.
(b) One independent, blind cause check runs before design. When it disagrees with the researcher, the run escalates or replaces research.cause before any design or grade.
Part (a) is cheap. Part (b) may become an eval first.

Required work:

Reject a blank cause mechanism in Research#verify! — check: present mechanism in plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/research.rb

Acceptance criteria

Given a research reply with a blank mechanism or defect_class, when Research#verify! runs, then it rejects the reply.
Given a mechanism with no path:line, when Research#verify! runs, then it rejects the reply.
Given a blind cause check that disagrees with the researcher, when the run reaches the design step, then the run escalates or replaces research.cause before any design or grade.

Origin

Trigger: Codex review of ADR-0285 cause flow, requested by Jared 2026-10-10
Improvised this session: none
Chain: unchecked cause steers and grades designs ← ADR-0285 added cause with no check (docs/adr, ADR-0285) ← DESIGN (ADR-0285)
Root candidate: none
Where: plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/research.rb
Session: ace499f5-e9c6-470b-9571-70c97b32ef50
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-prism-architect/ace499f5-e9c6-470b-9571-70c97b32ef50.jsonl
Skeptic: VERDICT: CREATE

## Problem ADR-0285 made the researcher state `cause {mechanism, defect_class}` (`plugins/os-sdlc/eval/ticket-takedown/harness/cards/researcher.md`, last bullet; `harness/schemas/research.json`). In live mode the grader grades every design against this cause (`lib/takedown/query/live_grade_inputs.rb:13-16`). Designers also see the cause in the research reply (`lib/takedown/presenter/run_prompts.rb:57`). The grader may not judge the cause (`harness/cards/grader.md:11`). So one unchecked cause steers the designers and grades them, and nothing detects a shallow or wrong cause in live mode. This is the root cause challenger: dig past the first instinct to a more systemic origin, so the fix is more resilient. Neither guard may change `harness/cards/grader.md` or its schema, because that forces recalibration. Source: Codex review. Related: the sibling ticket "Measure how often the researcher's cause is shallower than the ruled cause, and flag runs where the critic names a deeper cause" (its result decides whether part (b) is needed now). Parent work: #974. ## Reproduce n/a: design ticket, nothing to reproduce. ## Approach (a) `Research#verify!` (`lib/takedown/model/research.rb:11-20`) rejects a blank `mechanism` or `defect_class`, and requires a `path:line` in `mechanism`. (b) One independent, blind cause check runs before design. When it disagrees with the researcher, the run escalates or replaces `research.cause` before any design or grade. Part (a) is cheap. Part (b) may become an eval first. Required work: 1. Reject a blank cause mechanism in `Research#verify!` — check: present `mechanism` in `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/research.rb` ## Acceptance criteria 1. Given a research reply with a blank `mechanism` or `defect_class`, when `Research#verify!` runs, then it rejects the reply. 2. Given a `mechanism` with no `path:line`, when `Research#verify!` runs, then it rejects the reply. 3. Given a blind cause check that disagrees with the researcher, when the run reaches the design step, then the run escalates or replaces `research.cause` before any design or grade. ## Origin - Trigger: Codex review of ADR-0285 cause flow, requested by Jared 2026-10-10 - Improvised this session: none - Chain: unchecked cause steers and grades designs ← ADR-0285 added `cause` with no check (docs/adr, ADR-0285) ← DESIGN (ADR-0285) - Root candidate: none - Where: plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/research.rb - Session: ace499f5-e9c6-470b-9571-70c97b32ef50 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-prism-architect/ace499f5-e9c6-470b-9571-70c97b32ef50.jsonl - Skeptic: VERDICT: CREATE

jared added the

plugin/os-sdlc

filed-by/user

labels

2026-10-10 12:44:06 +00:00

jared referenced this issue

2026-10-10 13:41:51 +00:00

Grade every takedown design against the stated cause, and compare each revision with the previous design #974

jared added the

category/umbrella

label

2026-10-10 13:50:38 +00:00

jared commented

2026-10-10 14:21:08 +00:00

Author

Owner

Commit e4d0fcd2 (ADR-0288) changes Research#verify! but does not close any acceptance criterion of this ticket.

What it adds to verify!: still_relevant must be true; relevance_evidence needs one entry with a ref and a finding; reproduction must be either a run (ran true, command, output_excerpt) or a not-runnable location (ran false, location matching Research::LOCATION = /\A\S+:\d+/, reason_not_run).

Overlap with this ticket: the path:line rule exists now, but only for reproduction.location, not for cause.mechanism. AC1 (blank mechanism/defect_class) and AC2 (path:line in mechanism) stay open; AC2 can reuse Research::LOCATION and add one entry to Research::CHECKS. AC3 (blind cause check) is not touched.

Commit e4d0fcd2 (ADR-0288) changes `Research#verify!` but does not close any acceptance criterion of this ticket. What it adds to `verify!`: `still_relevant` must be true; `relevance_evidence` needs one entry with a ref and a finding; `reproduction` must be either a run (`ran` true, `command`, `output_excerpt`) or a not-runnable location (`ran` false, `location` matching `Research::LOCATION` = `/\A\S+:\d+/`, `reason_not_run`). Overlap with this ticket: the path:line rule exists now, but only for `reproduction.location`, not for `cause.mechanism`. AC1 (blank `mechanism`/`defect_class`) and AC2 (path:line in `mechanism`) stay open; AC2 can reuse `Research::LOCATION` and add one entry to `Research::CHECKS`. AC3 (blind cause check) is not touched.

jared referenced this issue from a commit

2026-10-12 14:02:16 +00:00

feat(takedown): researcher reproduces the defect in a sandbox and checks relevance; research-only mode

jared referenced this issue

2026-10-12 14:33:29 +00:00

Classify researcher cause vs ruled cause over training cases #1000

jared referenced this issue

2026-10-12 14:33:29 +00:00

Reject blank or location-free cause in Research#verify! #1001

jared commented

2026-10-12 14:33:58 +00:00

Author

Owner

Decomposed 2026-10-12. Child: #1001 (Research#verify! shape check, AC 1-2). AC 3 (blind cause check) has no child: the skeptic dropped a placeholder ticket because it waits on the counts in #1000. Reopen it as a child only if those counts show shallow or wrong causes are common. Umbrella stays open.