jared/cc-os

Watch

Fork 0

Keep failure blocks in takedown green-suite recorded output #1425

New issue

Closed

opened 2026-10-27 16:54:35 +00:00 by jared · 3 comments

jared commented

2026-10-27 16:54:35 +00:00

Owner

Problem

The ticket-takedown green-suite step records a failing test path but drops the failure text. The takedown fails with no evidence to debug from.

Observed

In run docs/takedown/1416-20261027-123627, files 14-green-suite.json and 20-green-suite.json list plugins/os-sdlc/tests/crc/tickets_bin_test.rb in targets[0].failing_paths.
targets[0].output holds 14,480 characters of repeated sqlite fork_safety.rb:43 warnings.
It holds no failure line, no backtrace, and no Minitest summary.
Takedown::SuiteRun#failing_paths parses the full output, but Takedown::SuiteRun#entry keeps only output.lines.last(TAIL) with TAIL = 40.
Warnings printed after the failure block fill the last 40 lines, so the failure block falls outside the kept window.
The same test file passes 3/3 alone and in the full suite on main.

Reproduce

The fork warnings no longer appear on main bb9f1f88 (#1421 removed InProcessBin#forked). Reproduce with any failing test that prints more than 40 lines of noise after its Minitest failure block:

SuiteRun.new(target: "t", command: "c", passed: false, output: failure_block + ("warning\n" * 100)).entry(repo)["output"]

The result contains no Failure: line.

Approach

Keep the Minitest failure blocks in the recorded output, not only the last 40 lines.

Required work:

Build the recorded output from the failure blocks that failing_paths already finds, plus the final summary — check: absent output.lines.last(TAIL) in plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/suite_run.rb

Acceptance criteria

Given a failed run whose output has a failure block followed by more than 40 noise lines, when the entry is built, then output contains the failure block.
Given a passed run, when the entry is built, then output is empty.

Owner sketch

class Takedown::SuiteRun
  def entry(repo)
end

Origin

Trigger: takedown run docs/takedown/1416-20261027-123627, steps 14 and 20
Improvised this session: none
Chain: failure with no evidence <- entry keeps only the last 40 lines (plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/suite_run.rb:18) <- DESIGN (tail window chosen without regard to trailing noise)
Root candidate: none found; searched SuiteRun, suite_run, TAIL, green-suite, failure block, recorded output, takedown
Where: plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/suite_run.rb:18
Session: c843bd27-b3bf-4e36-9243-df1697457146
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/c843bd27-b3bf-4e36-9243-df1697457146.jsonl

VERDICT: CREATE

## Problem The ticket-takedown `green-suite` step records a failing test path but drops the failure text. The takedown fails with no evidence to debug from. ## Observed In run `docs/takedown/1416-20261027-123627`, files `14-green-suite.json` and `20-green-suite.json` list `plugins/os-sdlc/tests/crc/tickets_bin_test.rb` in `targets[0].failing_paths`. `targets[0].output` holds 14,480 characters of repeated sqlite `fork_safety.rb:43` warnings. It holds no failure line, no backtrace, and no Minitest summary. `Takedown::SuiteRun#failing_paths` parses the full output, but `Takedown::SuiteRun#entry` keeps only `output.lines.last(TAIL)` with `TAIL = 40`. Warnings printed after the failure block fill the last 40 lines, so the failure block falls outside the kept window. The same test file passes 3/3 alone and in the full suite on main. ## Reproduce The fork warnings no longer appear on main bb9f1f88 (#1421 removed `InProcessBin#forked`). Reproduce with any failing test that prints more than 40 lines of noise after its Minitest failure block: ``` SuiteRun.new(target: "t", command: "c", passed: false, output: failure_block + ("warning\n" * 100)).entry(repo)["output"] ``` The result contains no `Failure:` line. ## Approach Keep the Minitest failure blocks in the recorded output, not only the last 40 lines. Required work: 1. Build the recorded output from the failure blocks that `failing_paths` already finds, plus the final summary — check: absent `output.lines.last(TAIL)` in `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/suite_run.rb` ## Acceptance criteria 1. Given a failed run whose output has a failure block followed by more than 40 noise lines, when the entry is built, then `output` contains the failure block. 2. Given a passed run, when the entry is built, then `output` is empty. ## Owner sketch ```ruby class Takedown::SuiteRun def entry(repo) end ``` ## Origin - Trigger: takedown run `docs/takedown/1416-20261027-123627`, steps 14 and 20 - Improvised this session: none - Chain: failure with no evidence <- `entry` keeps only the last 40 lines (plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/suite_run.rb:18) <- DESIGN (tail window chosen without regard to trailing noise) - Root candidate: none found; searched SuiteRun, suite_run, TAIL, green-suite, failure block, recorded output, takedown - Where: plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/suite_run.rb:18 - Session: c843bd27-b3bf-4e36-9243-df1697457146 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/c843bd27-b3bf-4e36-9243-df1697457146.jsonl VERDICT: CREATE

jared added the

plugin/os-sdlc

filed-by/user

category/code-change

labels

2026-10-27 16:54:35 +00:00

jared referenced this issue

2026-10-27 17:01:12 +00:00

Guard-map test-reviewer always settles error for missing red_tests #1416

jared commented

2026-10-27 17:08:50 +00:00

Author

Owner

Diagnosis of the recurring plugins/os-sdlc/tests/crc/tickets_bin_test.rb green-suite failure (runs 1416-20261027-123627, 1415-20261027-123629, 1415-20261027-125443 and earlier). This is a separate defect from the truncation in this ticket. The 40-line tail hid it.

Environment of the step. ScratchRepo#build (plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/scratch_repo.rb) runs git archive of the base commit, strips plugins/os-sdlc/eval and docs/handoffs, then git init and one commit named "base". SuiteRunner runs sh -c "ruby plugins/os-sdlc/tests/all.rb" with cwd = that repo. The scratch repo has one commit.

Reproduction (same recipe, main 6a0acac0, no design applied): 2776 runs, 1 failure.

1) Failure:
CrcTicketsBinTest#test_the_os_sdlc_ledger_at_88a6bd35_emits_the_crc_rows_as_a_last_batch_of_eight [plugins/os-sdlc/tests/crc/tickets_bin_test.rb:118]:
Expected #<Process::Status: pid 3719802 exit 128> to be success?.

Cause. Helper ledger_at (tickets_bin_test.rb:115-120) runs git -C <repo root> show 88a6bd35:docs/crc/os-sdlc/ledger.tsv. It needs historic commit 88a6bd35. In the scratch repo git says fatal: invalid object name '88a6bd35' (exit 128). The test passes on main because main has that history. It fails for every design, with no design applied.

Smallest fix. Commit the ledger blob as a fixture (for example plugins/os-sdlc/tests/fixtures/crc/os-sdlc-ledger-88a6bd35.tsv) and read it with File.read in ledger_at. This removes the history dependency.

Follow-ups. The sqlite3 fork_safety warning repeats in the output and floods the tail; filter it when fixing #1425. Keep failure blocks, not the log tail.

Diagnosis of the recurring `plugins/os-sdlc/tests/crc/tickets_bin_test.rb` green-suite failure (runs 1416-20261027-123627, 1415-20261027-123629, 1415-20261027-125443 and earlier). This is a separate defect from the truncation in this ticket. The 40-line tail hid it. **Environment of the step.** `ScratchRepo#build` (plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/scratch_repo.rb) runs `git archive` of the base commit, strips `plugins/os-sdlc/eval` and `docs/handoffs`, then `git init` and one commit named "base". `SuiteRunner` runs `sh -c "ruby plugins/os-sdlc/tests/all.rb"` with cwd = that repo. The scratch repo has one commit. **Reproduction** (same recipe, main 6a0acac0, no design applied): 2776 runs, 1 failure. ``` 1) Failure: CrcTicketsBinTest#test_the_os_sdlc_ledger_at_88a6bd35_emits_the_crc_rows_as_a_last_batch_of_eight [plugins/os-sdlc/tests/crc/tickets_bin_test.rb:118]: Expected #<Process::Status: pid 3719802 exit 128> to be success?. ``` **Cause.** Helper `ledger_at` (tickets_bin_test.rb:115-120) runs `git -C <repo root> show 88a6bd35:docs/crc/os-sdlc/ledger.tsv`. It needs historic commit 88a6bd35. In the scratch repo git says `fatal: invalid object name '88a6bd35'` (exit 128). The test passes on main because main has that history. It fails for every design, with no design applied. **Smallest fix.** Commit the ledger blob as a fixture (for example plugins/os-sdlc/tests/fixtures/crc/os-sdlc-ledger-88a6bd35.tsv) and read it with File.read in `ledger_at`. This removes the history dependency. **Follow-ups.** The sqlite3 fork_safety warning repeats in the output and floods the tail; filter it when fixing #1425. Keep failure blocks, not the log tail.

jared referenced this issue

2026-10-27 17:10:55 +00:00

Guard-map test-reviewer always settles error for missing red_tests #1416

jared referenced this issue from a commit

2026-10-27 17:22:39 +00:00

test(os-sdlc): read the 88a6bd35 ledger from a fixture, not git history

jared commented

2026-10-27 17:46:09 +00:00

Author

Owner

Work started: worktree .claude/worktrees/ticket-1425, branch ticket-1425.

jared referenced this issue from a commit

2026-10-27 17:49:08 +00:00

fix(os-sdlc): keep failure blocks in takedown green-suite recorded output (#1425)

jared referenced this issue from a commit

2026-10-27 17:49:08 +00:00

fix(os-sdlc): keep crash output tail when no failure block (#1425)

jared referenced this issue from a commit

2026-10-27 17:49:08 +00:00

fix(os-sdlc): split recorded_output to satisfy AbcSize (#1425)

jared referenced this issue

2026-10-27 17:50:10 +00:00

Takedown RedCheck keeps only the last 40 lines of output #1432

jared commented

2026-10-27 17:50:52 +00:00

Author

Owner

Resolution

Done: SuiteRun#recorded_output records each failure block (capped at 30 lines) plus the Minitest summary; falls back to the last 40 lines when the output has no failure block (crash before Minitest runs).

Evidence: main d6190950 (commits 81c92611, 19883ddc, e3532310). test/model/suite_run_test.rb: 3 tests. Takedown suite 624 runs, 0 failures; rubocop clean. Built by hand in a worktree, test first, not via /os-sdlc:implement.

Follow-ups: #1432 RedCheck 40-line tail (skeptic VERDICT: CREATE, no root candidate); shared block parser for failing_paths and recorded_output: dropped, no defect.

## Resolution **Done:** SuiteRun#recorded_output records each failure block (capped at 30 lines) plus the Minitest summary; falls back to the last 40 lines when the output has no failure block (crash before Minitest runs). **Evidence:** main d6190950 (commits 81c92611, 19883ddc, e3532310). test/model/suite_run_test.rb: 3 tests. Takedown suite 624 runs, 0 failures; rubocop clean. Built by hand in a worktree, test first, not via /os-sdlc:implement. **Follow-ups:** #1432 RedCheck 40-line tail (skeptic VERDICT: CREATE, no root candidate); shared block parser for failing_paths and recorded_output: dropped, no defect.

jared closed this issue

2026-10-27 17:50:53 +00:00

jared referenced this issue from a commit

2026-10-27 17:50:57 +00:00

docs(os-sdlc): History rows for #1425 and #1426