os-sdlc: green-assert harness-errors when suite errors lack a parseable location #278

New issue

Open

opened 2026-08-05 18:33:26 +00:00 by jared · 2 comments

jared commented

2026-08-05 18:33:26 +00:00

Owner

Context

Ticket-276 run: green-assert exited 3 ('no parseable failing file location') even though the suite output contains two real errors (TransitionTableIntegrityTest, KeyError 'ac-lint', with full backtraces). Suspect the interleaved Ruby 4 frozen-string-literal warnings and/or error-vs-failure format break the parser, so a legitimately red suite becomes a harness error and stalls the pipeline instead of routing a fix round.

## Context Ticket-276 run: green-assert exited 3 ('no parseable failing file location') even though the suite output contains two real errors (TransitionTableIntegrityTest, KeyError 'ac-lint', with full backtraces). Suspect the interleaved Ruby 4 frozen-string-literal warnings and/or error-vs-failure format break the parser, so a legitimately red suite becomes a harness error and stalls the pipeline instead of routing a fix round.

jared added the

label

2026-08-05 18:33:26 +00:00

jared commented

2026-08-06 20:01:10 +00:00

Author

Owner

Spec (verified still-present 2026-08-06; ready-for-agent):

Root cause: lib/os_sdlc/green_assert_failure_sidecar.rb parses failing locations with FAILURE_LOCATION = /\[([^\]:]+\.rb):\d+\]/ (line 11). That bracketed [file.rb:NN] form only appears in minitest Failure blocks. Error blocks carry the location in backtrace lines ( path/file.rb:12:in 'method') with no brackets, so an errors-only red suite yields no match -> HarnessError -> exit 3 ("no parseable failing file location") and the pipeline stalls instead of routing a fix round.

Seam: GreenAssertFailureSidecar#failing_path / #failure_class in lib/os_sdlc/green_assert_failure_sidecar.rb; tests in tests/green_assert_failure_sidecar_test.rb (all current fixtures use only the Failure bracket format).

Tasks (TDD, red-green):

Add failing fixtures/tests: (a) real-shaped minitest output containing only Error blocks with backtraces (mirror the ticket-276 case: TransitionTableIntegrityTest, KeyError); (b) output with interleaved Ruby frozen-string-literal warnings preceding a normal Failure block.
Extend the sidecar: when the bracket pattern misses, fall back to Error-block backtrace parsing — take the first backtrace frame whose path is a test file (under the target's test dir / matches _test.rb), returning that path.
Keep the garbage-input path intact: truly unparseable output still raises HarnessError (exit 3).

Acceptance criteria:

Errors-only suite output resolves failing_path to the failing test file; no HarnessError.
Warning-noise-interleaved Failure output parses as before.
Existing sidecar tests stay green; garbage input still exits 3.
Full suite green: ruby tests/all.rb from plugins/os-sdlc.

Spec (verified still-present 2026-08-06; ready-for-agent): **Root cause:** `lib/os_sdlc/green_assert_failure_sidecar.rb` parses failing locations with `FAILURE_LOCATION = /\[([^\]:]+\.rb):\d+\]/` (line 11). That bracketed `[file.rb:NN]` form only appears in minitest **Failure** blocks. **Error** blocks carry the location in backtrace lines (` path/file.rb:12:in 'method'`) with no brackets, so an errors-only red suite yields no match -> `HarnessError` -> exit 3 ("no parseable failing file location") and the pipeline stalls instead of routing a fix round. **Seam:** `GreenAssertFailureSidecar#failing_path` / `#failure_class` in `lib/os_sdlc/green_assert_failure_sidecar.rb`; tests in `tests/green_assert_failure_sidecar_test.rb` (all current fixtures use only the Failure bracket format). **Tasks (TDD, red-green):** 1. Add failing fixtures/tests: (a) real-shaped minitest output containing only Error blocks with backtraces (mirror the ticket-276 case: TransitionTableIntegrityTest, KeyError); (b) output with interleaved Ruby frozen-string-literal warnings preceding a normal Failure block. 2. Extend the sidecar: when the bracket pattern misses, fall back to Error-block backtrace parsing — take the first backtrace frame whose path is a test file (under the target's test dir / matches `_test.rb`), returning that path. 3. Keep the garbage-input path intact: truly unparseable output still raises HarnessError (exit 3). **Acceptance criteria:** - Errors-only suite output resolves `failing_path` to the failing test file; no HarnessError. - Warning-noise-interleaved Failure output parses as before. - Existing sidecar tests stay green; garbage input still exits 3. - Full suite green: `ruby tests/all.rb` from plugins/os-sdlc.

jared added the

ready-for-agent

label

2026-08-06 20:01:13 +00:00

jared added the

plugin/os-sdlc

label

2026-08-13 18:12:36 +00:00

jared added the

frozen

label

2026-08-16 14:25:05 +00:00