Test-reviewer passes a stub that cannot run the code under test; diff-test stops at the first red step #903

New issue

Open

opened 2026-10-02 14:19:12 +00:00 by jared · 1 comment

jared commented

2026-10-02 14:19:12 +00:00

Owner

Problem

The test-reviewer passed a test whose stub cannot run the code under test. The diff-test gate then stops at the first red step, so later failures stay hidden. This is the same pattern as #890 (#878). Parent: #859.

Observed

In #880 implementation 2, tests/presenter/brief_drafted_map_test.rb used Facts = Struct.new(:reports) with no #fact. Brief#compose raised NoMethodError. The test-reviewer (dispatch 7) passed it.
Three programmer-repair dispatches could not fix it, because programmers may not edit tests. They never saw 3 further failures in tests/command/implementation/open_map_name_test.rb, because the diff-test gate stops at the first red step.

Reproduce

n/a: decompose decides the reproduction. The evidence is implementation 2 of #880 (hand-fixed in c585e62c).

Approach

Decide in decompose how the test-reviewer detects a stub that cannot run the code under test, and whether the diff-test gate reports every red step.

Origin

Trigger: three failed programmer-repair dispatches in #880 implementation 2.
Improvised this session: hand-fixed the stub in c585e62c.
Chain: escalation ← stub lacks #fact ← test-reviewer passed the test. Stops at DESIGN (test-reviewer charter and diff-test gate; see #890).
Root candidate: #890
Where: tests/presenter/brief_drafted_map_test.rb, test-reviewer, diff-test gate
Session: 616e6f92-69d9-4a00-bf30-33a77404deb3
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-ticket-880/616e6f92-69d9-4a00-bf30-33a77404deb3.jsonl

## Problem The test-reviewer passed a test whose stub cannot run the code under test. The diff-test gate then stops at the first red step, so later failures stay hidden. This is the same pattern as #890 (#878). Parent: #859. ## Observed In #880 implementation 2, `tests/presenter/brief_drafted_map_test.rb` used `Facts = Struct.new(:reports)` with no `#fact`. `Brief#compose` raised NoMethodError. The test-reviewer (dispatch 7) passed it. Three programmer-repair dispatches could not fix it, because programmers may not edit tests. They never saw 3 further failures in `tests/command/implementation/open_map_name_test.rb`, because the diff-test gate stops at the first red step. ## Reproduce n/a: decompose decides the reproduction. The evidence is implementation 2 of #880 (hand-fixed in c585e62c). ## Approach Decide in decompose how the test-reviewer detects a stub that cannot run the code under test, and whether the diff-test gate reports every red step. ## Origin - Trigger: three failed programmer-repair dispatches in #880 implementation 2. - Improvised this session: hand-fixed the stub in c585e62c. - Chain: escalation ← stub lacks #fact ← test-reviewer passed the test. Stops at DESIGN (test-reviewer charter and diff-test gate; see #890). - Root candidate: #890 - Where: tests/presenter/brief_drafted_map_test.rb, test-reviewer, diff-test gate - Session: 616e6f92-69d9-4a00-bf30-33a77404deb3 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-ticket-880/616e6f92-69d9-4a00-bf30-33a77404deb3.jsonl

jared added the

plugin/os-sdlc

filed-by/user

labels

2026-10-02 14:19:12 +00:00

jared referenced this issue

2026-10-02 14:34:55 +00:00

Resolve drafted maps from maps/drafts/ through Map::Catalog #880

jared referenced this issue

2026-10-02 19:49:45 +00:00

Let the programmer escalate a test defect from a gate-sourced node #890

jared commented

2026-10-02 19:50:17 +00:00

Author

Owner

Duplicate check against #890: NOT a duplicate. Leaving open.

What #890 covers: the stub defect. #890's revised approach lets the programmer escalate a defective test on gate-sourced nodes, so the repair-bound burn this ticket describes stops.

What remains (not covered by #890):

The premise "diff-test stops at the first red step" is false per the run transcript: every diff-test run reported 32 runs, ... 1 errors; open_map_name_test.rb was never in the selection.
Real defect: test selection. Project::Changes#test_paths (query/project/changes.rb:15-16) maps tests by name only, and DiffTestPlan#widened? (model/diff_test_plan.rb:24-25) widens to the suite only when no test maps at all. So the unmapped model/map.rb and model/map/headers.rb did not widen the run.
Check this against ADR-0202 ("an unmapped code file widens to the suite"): the code may contradict it.

Suggested re-scope: retitle to the diff-test selection gap, with ADR-0202 as the reference.

Duplicate check against #890: NOT a duplicate. Leaving open. What #890 covers: the stub defect. #890's revised approach lets the programmer escalate a defective test on gate-sourced nodes, so the repair-bound burn this ticket describes stops. What remains (not covered by #890): - The premise "diff-test stops at the first red step" is false per the run transcript: every diff-test run reported `32 runs, ... 1 errors`; `open_map_name_test.rb` was never in the selection. - Real defect: test selection. `Project::Changes#test_paths` (query/project/changes.rb:15-16) maps tests by name only, and `DiffTestPlan#widened?` (model/diff_test_plan.rb:24-25) widens to the suite only when no test maps at all. So the unmapped `model/map.rb` and `model/map/headers.rb` did not widen the run. - Check this against ADR-0202 ("an unmapped code file widens to the suite"): the code may contradict it. Suggested re-scope: retitle to the diff-test selection gap, with ADR-0202 as the reference.

jared added the

category/owner-ruling

label

2026-10-10 13:31:51 +00:00