jared/cc-os

Watch

Fork 0

Takedown: give the reviser the green-suite failure, and name the test in a polish rejection #1521

New issue

Closed

opened 2026-11-03 00:03:06 +00:00 by jared · 2 comments

jared commented

2026-11-03 00:03:06 +00:00

Owner

Problem

The takedown pipeline cannot recover from a failed green-suite step. A failed green-suite step sends the run back to a revise round. The revise prompt does not contain the suite output, and the reviser cannot run commands. Thus the reviser cannot see the error that caused the revise round. Also, a rejected polish round does not name the test that is not red at base.

Observed

Run docs/takedown/1511-20261102-181654/ failed at the last green-suite step. The revise-2 prompt did not contain the revise-1 suite failure.

green-suite  failed  final design revise-2: breaks existing tests: plugins/os-sdlc/tests/cops/domain_effect_in_edge_test.rb; lint offenses: ...

The revise-2 failure was KeyError: key not found: "EffectConstants" (19-green-suite.json:12).

Run docs/takedown/1511-20261102-175027/ failed at polish-1. The rejection text does not name the test.

12 polish-1  failed  polish round polish-1 rejected: the failing tests are not red at base

The polish prompt gave the polisher a Metrics/AbcSize offense on the test file (12-polish-1.prompt.md:22). The lesson for that offense says to split the test. The polisher split test_file_form_constants_are_a_setting_with_the_old_default into two tests. One of the two tests passes at base. The polisher card says to keep the tests as they are (harness/cards/polisher.md:10).

Reproduce

n/a: each failure needs a live takedown run (about $2 and 25 min); the run directories above hold the full evidence.

Approach

The suite record of a design is a hash in RunState#suite_checks, keyed by step name. GreenSuiteStep stores SuiteCheck#to_h there. The hash has the keys green, note, targets, and offenses. Each targets entry has passed and output. The output is empty when the target passed. The record has no tests_green key. Flow#suite_tests_green? derives it: every target has passed true. The revise prompt must do the same.

Place the new section outside the text from ## Best design so far to ## Objections, because test/command/pipeline_test.rb:323 slices that text. When no suite check exists for the step, the section is empty; read it with a guarded lookup, as run_state.rb:123 does. The latest suite check is the last value of suite_checks. In a revise round, current_step still names the design that failed, so its check is the last one.

Required work:

Add a private method suite_section to RunPrompts. It returns an empty string when @run.suite_checks is empty or when every target of the latest check has passed true. Else it returns a section "Suite check" with the check note and the output of each target that has passed false. Add #{suite_section} to revise after the "Accepted decisions named as the cause" section. — check: present def suite_section in plugins/os-sdlc/eval/ticket-takedown/lib/takedown/presenter/run_prompts.rb
In PolishStep#problem, keep the RedCheck result and return its note when it is not red. Replace the fixed text "the failing tests are not red at base" with that note. Remove still_red?. — check: present red_check.note in plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/polish_step.rb
Remove the fixed rejection text from PolishStep. — check: absent the failing tests are not red at base in plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/polish_step.rb
Add tests for both changes to the existing test files test/presenter/run_prompts_test.rb and test/command/polish_test.rb. — check: present Suite check in plugins/os-sdlc/eval/ticket-takedown/test/presenter/run_prompts_test.rb

Acceptance criteria

Given a run whose latest suite check has a target with passed false, when the revise prompt is built, then the prompt has a "Suite check" section with the check note and the failing target output.
Given a run whose latest suite check has every target passed, when the revise prompt is built, then the prompt has no "Suite check" section.
Given a run with no suite check, when the revise prompt is built, then the prompt has no "Suite check" section.
Given a run with two suite checks, when the revise prompt is built, then the section holds the output of the later check only.
Given a polish round whose failing tests are not red at base, when the round is rejected, then the failure note names each test that is not red at base.
Given a polish round that breaks the suite, when the round is rejected, then the failure note is still the suite check note.

Owner sketch

module Takedown
  class RunPrompts
    def revise  # adds suite_section to the prompt
    private
    def suite_section  # -> String; "" unless the latest suite check has a target with passed false
  end

  class PolishStep
    def problem(run, design)  # -> String or nil; returns RedCheck#note when the check is not red
  end
end

Non-goals

Whether the polisher may edit test files. #1082 asks polish to fix lint offenses in test files; this ticket does not change that.

Origin

Trigger: takedown-run --live 1511, two runs on 2026-11-02 (docs/takedown/1511-20261102-175027, docs/takedown/1511-20261102-181654)
Improvised this session: none; #1511 goes to /os-sdlc:implement without a takedown design (ruling 2026-11-02)
Chain: revise-2 repeats a suite failure ← the revise prompt omits suite_checks (presenter/run_prompts.rb:36-42) ← the reviser cannot run commands (harness/cards/reviser.md:17) ← DESIGN plugins/os-sdlc/eval/ticket-takedown/lib/takedown/step_profile.rb:8,12,31,57
Root candidate: none found; searched takedown, revise prompt, suite_checks, green-suite, polish, red at base, RedCheck, run_prompts, reviser (#1082 is related, not a match)
Where: plugins/os-sdlc/eval/ticket-takedown/lib/takedown/presenter/run_prompts.rb
Session: cb530987-16c7-4de2-aacd-d43080b77d35
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/cb530987-16c7-4de2-aacd-d43080b77d35.jsonl

VERDICT: CREATE — no open ticket covers the revise prompt that omits the green-suite failure, or the polish rejection that does not name the test; #1082 is related.

## Problem The takedown pipeline cannot recover from a failed green-suite step. A failed green-suite step sends the run back to a revise round. The revise prompt does not contain the suite output, and the reviser cannot run commands. Thus the reviser cannot see the error that caused the revise round. Also, a rejected polish round does not name the test that is not red at base. ## Observed Run `docs/takedown/1511-20261102-181654/` failed at the last green-suite step. The revise-2 prompt did not contain the revise-1 suite failure. ``` green-suite failed final design revise-2: breaks existing tests: plugins/os-sdlc/tests/cops/domain_effect_in_edge_test.rb; lint offenses: ... ``` The revise-2 failure was `KeyError: key not found: "EffectConstants"` (`19-green-suite.json:12`). Run `docs/takedown/1511-20261102-175027/` failed at polish-1. The rejection text does not name the test. ``` 12 polish-1 failed polish round polish-1 rejected: the failing tests are not red at base ``` The polish prompt gave the polisher a `Metrics/AbcSize` offense on the test file (`12-polish-1.prompt.md:22`). The lesson for that offense says to split the test. The polisher split `test_file_form_constants_are_a_setting_with_the_old_default` into two tests. One of the two tests passes at base. The polisher card says to keep the tests as they are (`harness/cards/polisher.md:10`). ## Reproduce n/a: each failure needs a live takedown run (about $2 and 25 min); the run directories above hold the full evidence. ## Approach The suite record of a design is a hash in `RunState#suite_checks`, keyed by step name. `GreenSuiteStep` stores `SuiteCheck#to_h` there. The hash has the keys `green`, `note`, `targets`, and `offenses`. Each `targets` entry has `passed` and `output`. The `output` is empty when the target passed. The record has no `tests_green` key. `Flow#suite_tests_green?` derives it: every target has `passed` true. The revise prompt must do the same. Place the new section outside the text from `## Best design so far` to `## Objections`, because `test/command/pipeline_test.rb:323` slices that text. When no suite check exists for the step, the section is empty; read it with a guarded lookup, as `run_state.rb:123` does. The latest suite check is the last value of `suite_checks`. In a revise round, `current_step` still names the design that failed, so its check is the last one. Required work: 1. Add a private method `suite_section` to `RunPrompts`. It returns an empty string when `@run.suite_checks` is empty or when every target of the latest check has `passed` true. Else it returns a section "Suite check" with the check `note` and the `output` of each target that has `passed` false. Add `#{suite_section}` to `revise` after the "Accepted decisions named as the cause" section. — check: present `def suite_section` in `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/presenter/run_prompts.rb` 2. In `PolishStep#problem`, keep the `RedCheck` result and return its `note` when it is not red. Replace the fixed text "the failing tests are not red at base" with that note. Remove `still_red?`. — check: present `red_check.note` in `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/polish_step.rb` 3. Remove the fixed rejection text from `PolishStep`. — check: absent `the failing tests are not red at base` in `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/polish_step.rb` 4. Add tests for both changes to the existing test files `test/presenter/run_prompts_test.rb` and `test/command/polish_test.rb`. — check: present `Suite check` in `plugins/os-sdlc/eval/ticket-takedown/test/presenter/run_prompts_test.rb` ## Acceptance criteria 1. Given a run whose latest suite check has a target with `passed` false, when the revise prompt is built, then the prompt has a "Suite check" section with the check note and the failing target output. 2. Given a run whose latest suite check has every target passed, when the revise prompt is built, then the prompt has no "Suite check" section. 3. Given a run with no suite check, when the revise prompt is built, then the prompt has no "Suite check" section. 4. Given a run with two suite checks, when the revise prompt is built, then the section holds the output of the later check only. 5. Given a polish round whose failing tests are not red at base, when the round is rejected, then the failure note names each test that is not red at base. 6. Given a polish round that breaks the suite, when the round is rejected, then the failure note is still the suite check note. ## Owner sketch ```ruby module Takedown class RunPrompts def revise # adds suite_section to the prompt private def suite_section # -> String; "" unless the latest suite check has a target with passed false end class PolishStep def problem(run, design) # -> String or nil; returns RedCheck#note when the check is not red end end ``` ## Non-goals Whether the polisher may edit test files. #1082 asks polish to fix lint offenses in test files; this ticket does not change that. ## Origin - Trigger: `takedown-run --live 1511`, two runs on 2026-11-02 (`docs/takedown/1511-20261102-175027`, `docs/takedown/1511-20261102-181654`) - Improvised this session: none; #1511 goes to `/os-sdlc:implement` without a takedown design (ruling 2026-11-02) - Chain: revise-2 repeats a suite failure ← the revise prompt omits `suite_checks` (`presenter/run_prompts.rb:36-42`) ← the reviser cannot run commands (`harness/cards/reviser.md:17`) ← DESIGN `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/step_profile.rb:8,12,31,57` - Root candidate: none found; searched takedown, revise prompt, suite_checks, green-suite, polish, red at base, RedCheck, run_prompts, reviser (#1082 is related, not a match) - Where: plugins/os-sdlc/eval/ticket-takedown/lib/takedown/presenter/run_prompts.rb - Session: cb530987-16c7-4de2-aacd-d43080b77d35 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/cb530987-16c7-4de2-aacd-d43080b77d35.jsonl VERDICT: CREATE — no open ticket covers the revise prompt that omits the green-suite failure, or the polish rejection that does not name the test; #1082 is related.

jared added the

labels

2026-11-03 00:03:06 +00:00

jared referenced this issue

2026-11-03 13:06:43 +00:00

os-sdlc cop Sdlc/Structural/DomainOperandInEdge: an edge compares with our literal, a limit it computes, or a second-source set (A19 R2) #1510

jared commented

2026-11-03 13:07:18 +00:00

Author

Owner

Work started on branch ticket-1521 via /os-sdlc:implement (session a5c3e206).

jared referenced this issue from a commit

2026-11-03 13:20:35 +00:00

feat(takedown): give the reviser the green-suite failure; name the test in a polish rejection (#1521)

jared commented

2026-11-03 13:20:41 +00:00

Author

Owner

Resolution

Done: Revise prompt carries a Suite check section (latest failing check note plus failing target output; absent when no check or all targets pass). PolishStep#problem returns the RedCheck note, naming each test not red at base; fixed rejection text removed.

Evidence: main 8ab2bfc1; takedown tests 613 runs 0 failures; os-sdlc suite 3243 runs 0 failures; rubocop clean; contract audit pass (opus). Hand-finished after escalation 100: AC5 fixture named no test method, fixture corrected.

Follow-ups: Dropped: dedupe the passed-rule between Flow#suite_tests_green? and RunPrompts#failing_targets (ticket Approach chose derivation; one-line rule). Dropped: AC6 test gap, existing test_a_rejected_round_is_flagged asserts the suite note. #1511 next.

## Resolution **Done:** Revise prompt carries a Suite check section (latest failing check note plus failing target output; absent when no check or all targets pass). PolishStep#problem returns the RedCheck note, naming each test not red at base; fixed rejection text removed. **Evidence:** main 8ab2bfc1; takedown tests 613 runs 0 failures; os-sdlc suite 3243 runs 0 failures; rubocop clean; contract audit pass (opus). Hand-finished after escalation 100: AC5 fixture named no test method, fixture corrected. **Follow-ups:** Dropped: dedupe the passed-rule between Flow#suite_tests_green? and RunPrompts#failing_targets (ticket Approach chose derivation; one-line rule). Dropped: AC6 test gap, existing test_a_rejected_round_is_flagged asserts the suite note. #1511 next.

jared closed this issue

2026-11-03 13:20:41 +00:00