Verifier brief carries stale failed gate record instead of dispatch's final green gate run #551

New issue

Open

opened 2026-09-10 18:09:49 +00:00 by jared · 0 comments

jared commented

2026-09-10 18:09:49 +00:00

Owner

Context

During implementation 9 of ticket #528, the behavior-verifier dispatch received a gate block from the brief/aggregate builder that reflected an earlier failed gate run instead of the final green gate run. This occurred in the os-sdlc pipeline.

Observed

Programmer dispatch 141 ran the gate three times. The step_results rows recorded id 157 step "gate" exit 1 at 14:03:17 with output OsSdlc::HarnessError: os-sdlc: no test file found for plugins/os-sdlc/lib/os_sdlc/runner/gate_step_outcome.rb; tried: plugins/os-sdlc/tests/os_sdlc/runner/gate_step_outcome_test.rb, .... Id 158 step "gate" exit 1 at 14:05:27 repeated the same HarnessError. Then id 159 step "diff-test" exit 0 at 14:05:37 with output 38 runs, 59 assertions, 0 failures, 0 errors, 0 skips and id 160 step "lint" exit 0 at 14:05:37 completed successfully.

The Runner recorded dispatch 141 verdict pass and routed to behavior-verifier (dispatch 142). The verifier's brief showed "gate": {"verdict":"fail", ...} with the HarnessError text. The verifier's brief field was null. The verifier refused the red gate per its rules and wrote verdict: error. The subagent-stop then returned {"type":"implementation_failed","reason":"verdict:error"} and closed the implementation. The test file plugins/os-sdlc/tests/runner/gate_step_outcome_test.rb existed on disk at the time.

Reproduce

Open an implementation with the poodr map on a ticket whose programmer creates a new lib file, runs the gate before the matching test file exists (HarnessError), then adds the test and reruns the gate to green. Dispatch behavior-verifier and inspect its brief: the gate block shows the earlier failed run.

Expected

The verifier brief's gate block reflects the same gate outcome the Runner used to route (the latest complete gate run for that dispatch). A stale HarnessError row must not appear as the current gate verdict. The brief field must not be null. The system must select the most recent successful gate result for the dispatch, or handle HarnessError steps specially to ensure they do not pollute the gate verdict in later dispatches.

Origin

Trigger: implementation 9 failure in os-sdlc pipeline dispatch 141 → 142
Improvised this session: none
Chain: verifier brief shows stale HarnessError ← brief/aggregate builder picks wrong gate row ← DESIGN (ADR-0168, BriefFacts/aggregate code picks first or wrong-ordered gate result)
Root candidate: this ticket (also determine whether HarnessError should count as a gate result at all)
Where: plugins/os-sdlc/lib/os_sdlc/runner (aggregate or BriefFacts code from ADR-0168; exact class UNKNOWN)
Session: d9e1984d-93c8-4649-9712-a825b7d0af36
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/d9e1984d-93c8-4649-9712-a825b7d0af36.jsonl

Skeptic verdict: Complete pipeline run hit real design defect (aggregate/BriefFacts picks stale HarnessError gate row over latest green run), root points at ADR-0168 code with a concrete fix location; no existing mechanism covers HarnessError-vs-verdict selection.

Parent ticket: #528

## Context During implementation 9 of ticket #528, the behavior-verifier dispatch received a gate block from the brief/aggregate builder that reflected an earlier failed gate run instead of the final green gate run. This occurred in the os-sdlc pipeline. ## Observed Programmer dispatch 141 ran the gate three times. The step_results rows recorded id 157 step "gate" exit 1 at 14:03:17 with output `OsSdlc::HarnessError: os-sdlc: no test file found for plugins/os-sdlc/lib/os_sdlc/runner/gate_step_outcome.rb; tried: plugins/os-sdlc/tests/os_sdlc/runner/gate_step_outcome_test.rb, ...`. Id 158 step "gate" exit 1 at 14:05:27 repeated the same HarnessError. Then id 159 step "diff-test" exit 0 at 14:05:37 with output `38 runs, 59 assertions, 0 failures, 0 errors, 0 skips` and id 160 step "lint" exit 0 at 14:05:37 completed successfully. The Runner recorded dispatch 141 verdict pass and routed to behavior-verifier (dispatch 142). The verifier's brief showed `"gate": {"verdict":"fail", ...}` with the HarnessError text. The verifier's `brief` field was `null`. The verifier refused the red gate per its rules and wrote `verdict: error`. The subagent-stop then returned `{"type":"implementation_failed","reason":"verdict:error"}` and closed the implementation. The test file `plugins/os-sdlc/tests/runner/gate_step_outcome_test.rb` existed on disk at the time. ## Reproduce Open an implementation with the poodr map on a ticket whose programmer creates a new lib file, runs the gate before the matching test file exists (HarnessError), then adds the test and reruns the gate to green. Dispatch behavior-verifier and inspect its brief: the gate block shows the earlier failed run. ## Expected The verifier brief's gate block reflects the same gate outcome the Runner used to route (the latest complete gate run for that dispatch). A stale HarnessError row must not appear as the current gate verdict. The `brief` field must not be `null`. The system must select the most recent successful gate result for the dispatch, or handle HarnessError steps specially to ensure they do not pollute the gate verdict in later dispatches. ## Origin - Trigger: implementation 9 failure in os-sdlc pipeline dispatch 141 → 142 - Improvised this session: none - Chain: verifier brief shows stale HarnessError ← brief/aggregate builder picks wrong gate row ← DESIGN (ADR-0168, BriefFacts/aggregate code picks first or wrong-ordered gate result) - Root candidate: this ticket (also determine whether HarnessError should count as a gate result at all) - Where: plugins/os-sdlc/lib/os_sdlc/runner (aggregate or BriefFacts code from ADR-0168; exact class UNKNOWN) - Session: d9e1984d-93c8-4649-9712-a825b7d0af36 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/d9e1984d-93c8-4649-9712-a825b7d0af36.jsonl **Skeptic verdict:** Complete pipeline run hit real design defect (aggregate/BriefFacts picks stale HarnessError gate row over latest green run), root points at ADR-0168 code with a concrete fix location; no existing mechanism covers HarnessError-vs-verdict selection. **Parent ticket:** #528

jared added the

plugin/os-sdlc

filed-by/agent

labels

2026-09-10 18:09:49 +00:00

jared referenced this issue

2026-09-10 18:11:35 +00:00

Gate calls take 45 s; per-step wall time is not recorded #528

jared referenced this issue

2026-09-10 18:39:17 +00:00

Gate calls take 45 s; per-step wall time is not recorded #528

jared referenced this issue