jared/cc-os

Watch

Fork 0

Check each new test method for red at base in final-red #995

New issue

Closed

opened 2026-10-12 13:03:28 +00:00 by jared · 3 comments

jared commented

2026-10-12 13:03:28 +00:00

Owner

Problem

The takedown final-red step checks each test file, not each test method. A new test that passes at base still counts as red when a sibling test in the same file fails.

Observed

In the #977 live run on #194 (docs/takedown/194-20261010-112121/), final-red passed with only 1 of 4 new tests red at base (LintDirectoryScopeTest F...).
The other 3 tests were vacuous at base.
The grader gave class_removed and did not notice.

Reproduce

n/a: the evidence is in docs/takedown/194-20261010-112121/.

Origin

Trigger: takedown work on 2026-10-10 (handoff /tmp/handoff-2026-10-10-takedown-after-validity-sweep.md)
Improvised this session: none
Chain: vacuous new tests pass final-red <- RedCheck judges the file result, not each method <- DESIGN (FinalRedStep reads one red_checks entry per step)
Where: plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/final_red_step.rb, failing_test_step.rb (RedCheck.run)
Session: not available to the filing subagent
Transcript: not available to the filing subagent

VERDICT: CREATE

## Problem The takedown `final-red` step checks each test file, not each test method. A new test that passes at base still counts as red when a sibling test in the same file fails. ## Observed In the #977 live run on #194 (`docs/takedown/194-20261010-112121/`), `final-red` passed with only 1 of 4 new tests red at base (`LintDirectoryScopeTest F...`). The other 3 tests were vacuous at base. The grader gave `class_removed` and did not notice. ## Reproduce n/a: the evidence is in `docs/takedown/194-20261010-112121/`. ## Origin - Trigger: takedown work on 2026-10-10 (handoff /tmp/handoff-2026-10-10-takedown-after-validity-sweep.md) - Improvised this session: none - Chain: vacuous new tests pass final-red <- RedCheck judges the file result, not each method <- DESIGN (FinalRedStep reads one red_checks entry per step) - Where: plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/final_red_step.rb, failing_test_step.rb (RedCheck.run) - Session: not available to the filing subagent - Transcript: not available to the filing subagent VERDICT: CREATE

jared added the

plugin/os-sdlc

filed-by/agent

category/code-change

labels

2026-10-12 13:03:28 +00:00

jared commented

2026-10-12 13:04:11 +00:00

Author

Owner

Ticket-skeptic verdict (real run; the VERDICT line in the body was written by the filing agent, not the skeptic): CREATE. Observed in docs/takedown/194-20261010-112121; fix seam RedCheck.run in failing_test_step.rb.
Session: 3f094e92-202f-41fc-8d9c-b591750d9bc1
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-prism-architect/3f094e92-202f-41fc-8d9c-b591750d9bc1.jsonl

Ticket-skeptic verdict (real run; the VERDICT line in the body was written by the filing agent, not the skeptic): CREATE. Observed in docs/takedown/194-20261010-112121; fix seam RedCheck.run in failing_test_step.rb. Session: 3f094e92-202f-41fc-8d9c-b591750d9bc1 Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-prism-architect/3f094e92-202f-41fc-8d9c-b591750d9bc1.jsonl

~~jared referenced this issue 2026-10-12 13:18:45 +00:00~~

Stop takedown verifying defects already fixed on main #998

jared commented

2026-10-12 13:22:29 +00:00

Author

Owner

Resolution

Done: final-red now runs each new test method alone (-n Class#method) and requires every one red at base; a failing-test file with no new method fails closed. One red classifier kept (ADR-0283 d4).

Evidence: commit 7907c2b6 on prism-architect; takedown suite 505 runs 0 failures (526 after 12d2f052); os-sdlc 1540 runs 0 failures; #194-shape replay test gives red?=false and names the 3 non-red methods

Follow-ups: none: test-DSL/it-block support and diff-test place-tests per-file weakness noted in session report, not filed

## Resolution **Done:** final-red now runs each new test method alone (-n Class#method) and requires every one red at base; a failing-test file with no new method fails closed. One red classifier kept (ADR-0283 d4). **Evidence:** commit 7907c2b6 on prism-architect; takedown suite 505 runs 0 failures (526 after 12d2f052); os-sdlc 1540 runs 0 failures; #194-shape replay test gives red?=false and names the 3 non-red methods **Follow-ups:** none: test-DSL/it-block support and diff-test place-tests per-file weakness noted in session report, not filed

jared closed this issue

2026-10-12 13:22:29 +00:00

jared referenced this issue

2026-10-12 13:39:33 +00:00

os-backlog reads only the first page of Forgejo labels, so issue-create rejects valid scope labels #992

jared referenced this issue from a commit

2026-10-12 14:02:16 +00:00

fix(takedown): final-red requires every new test method red at base (#995)

jared referenced this issue from a commit

2026-10-12 14:02:16 +00:00

docs(takedown): ADR-0289 relevance step and #995 build records; research-only runs on #992 #984 #982

jared commented

2026-10-12 14:30:47 +00:00

Author

Owner

Dropped follow-up (ticket-skeptic): TestMethods ignores spec-style test/it blocks, but no such test exists under plugins/, so the need is speculative. Revisit if one appears.

jared referenced this issue

2026-10-12 14:31:05 +00:00

diff-test gate accepts a round as red when any test is red but test-reviewer requires all new tests red #953

jared referenced this issue

2026-10-12 14:39:23 +00:00

State in design cards that every failing_tests method must be red at base #1002

jared referenced this issue

2026-10-14 15:18:06 +00:00

Takedown design must list every existing test that pins changed behavior #1004

jared referenced this issue

2026-11-11 07:04:41 +00:00

Accept edited existing test methods in takedown final-red #1687