jared/cc-os

Watch

Fork 0

Select the presenter by outcome; escalate replies must not ask for repair #826

New issue

Closed

opened 2026-09-29 15:55:04 +00:00 by jared · 4 comments

jared commented

2026-09-29 15:55:04 +00:00

Owner

Problem

On an escalate verdict, the agent gets the wrong instruction. Gate::Reply#message (plugins/os-sdlc/lib/os_sdlc/presenter/gate/reply.rb:52-56) tells the agent to "correct the failures above, then rerun" for every verdict that is not PASS. An escalation has no failures to correct. Two classifications escalate (Gate::ESCALATING_CLASSIFICATIONS, model/gate.rb:6):

unexpected_pass: only from the test-writer node's diff-test gate (expect: red). The agent reads "correct the failures", but no test failed; the new test passed. agents/test-writer.md:111-120 tells it to "fix the tests" until red, which invites weakening the assertion.
green_report_nonzero_exit: from any expect-green test gate (programmer, refactorer, suite-check). Suite-check never reruns (agents/suite-check.md:59).

The escalate verdict ends the run only if the agent stops while the verdict is still escalate. A rerun with a correct red result overwrites the verdict with pass. ADR-0158 routing does not change.

The cause is structural. Each caller hard-codes one presenter: Gate::Run#record (command/gate/run.rb:77) always uses Gate::Reply, and Pipeline::NextStep (line 62) always uses Implementation::TerminalPayload. Each presenter then branches inside itself. FailureDirective has one fixed text for every escalation reason.

User direction (2026-09-29): a presenter owns each communication that the AI or the user sees. The code triggers the reply and selects the presenter that fits the outcome.

Reproduce

n/a: design ticket. The wrong text is visible in the Reply#message branch for any non-PASS verdict.

Approach

Select the presenter by verdict and classification at the call site (Gate::Run#record), so each presenter has one message and no branch on verdict. Replies per classification:

unexpected_pass (test-writer):

diff-test: unexpected_pass. Your new test passed before any production change.
Compare the test with the behavior the ticket adds:
- The test does not use the missing behavior: rewrite it so it fails for that
  reason, then rerun `os-sdlc-gate <id>`.
- The behavior already exists: stop and report. The run escalates to the Orchestrator.

green_report_nonzero_exit: stop and report; the reply names the classification and points to the known causes in references/escalation-procedure.md. It does not tell the agent to correct failures or rerun.

agents/test-writer.md:111-120: separate unexpected_pass (diagnose, then repair or report, as above) from broken / test-lint (repair loop, unchanged).

Terminal side: FailureDirective takes the escalation reason and names the matching section of references/escalation-procedure.md (the cause text stays only in that file). TerminalPayload (presenter/implementation/terminal_payload.rb:34) passes the reason; OpenPayload:19 calls it with no reason and keeps today's text.

Required work: none beyond the acceptance criteria.

Owner sketch

Gate::Run#record (command/gate/run.rb:77) owns the choice: it selects the reply presenter from the verdict's classification instead of always building Reply.
One presenter per outcome under presenter/gate/, each with one fixed message and no branch on verdict:
- Reply: pass and fail (text unchanged).
- a reply for unexpected_pass (repair-or-report text in Approach).
- a reply for green_report_nonzero_exit (stop-and-report text, points to references/escalation-procedure.md).
  The shared payload shape (verdict, step, classification, passed_steps, output) stays the same for all of them.
FailureDirective owns the directive text: FailureDirective.for(reason:) returns the section-specific text for a known reason prefix and today's text for nil or an unknown prefix. TerminalPayload passes @implementation.failed_reason; OpenPayload passes no reason.
agents/test-writer.md owns the agent-side instruction for unexpected_pass.

Acceptance criteria

Given a test-writer gate with classification unexpected_pass, when the agent receives the gate reply, then the reply does not say "failures" and tells the agent to repair the test or, if the behavior already exists, to stop and report.
Given a gate with classification green_report_nonzero_exit, when the agent receives the gate reply, then the reply tells the agent to stop and report, and does not tell it to correct failures.
Given an escalation with reason prefix green_report_nonzero_exit:, when the Orchestrator receives the terminal payload, then the directive names the matching section of references/escalation-procedure.md.
Given a gate verdict of fail, when the agent receives the gate reply, then the reply is unchanged ("correct the failures above, then rerun ...").

Origin

Trigger: #813 seam analysis (presenter inventory), 2026-09-29
Improvised this session: none; known causes were put in references/escalation-procedure.md instead
Chain: wrong repair instruction on escalate ← Gate::Reply branches only on PASS ← callers hard-code one presenter per call site ← DESIGN (presenter/gate/reply.rb, pipeline/next_step.rb:62)
Root candidate: this ticket
Where: plugins/os-sdlc/lib/os_sdlc/presenter/gate/reply.rb; presenter/implementation/terminal_payload.rb; presenter/failure_directive.rb
Session: 0aa46819-aa36-4e80-9438-9ef8b7314bf2
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-ticket-takedown/0aa46819-aa36-4e80-9438-9ef8b7314bf2.jsonl
Approach revised 2026-09-30 (user ruling, option A; see comment)

## Problem On an escalate verdict, the agent gets the wrong instruction. `Gate::Reply#message` (plugins/os-sdlc/lib/os_sdlc/presenter/gate/reply.rb:52-56) tells the agent to "correct the <step> failures above, then rerun" for every verdict that is not PASS. An escalation has no failures to correct. Two classifications escalate (`Gate::ESCALATING_CLASSIFICATIONS`, model/gate.rb:6): - `unexpected_pass`: only from the test-writer node's `diff-test` gate (`expect: red`). The agent reads "correct the failures", but no test failed; the new test passed. `agents/test-writer.md:111-120` tells it to "fix the tests" until red, which invites weakening the assertion. - `green_report_nonzero_exit`: from any expect-green test gate (programmer, refactorer, suite-check). Suite-check never reruns (`agents/suite-check.md:59`). The escalate verdict ends the run only if the agent stops while the verdict is still escalate. A rerun with a correct red result overwrites the verdict with pass. ADR-0158 routing does not change. The cause is structural. Each caller hard-codes one presenter: `Gate::Run#record` (command/gate/run.rb:77) always uses `Gate::Reply`, and `Pipeline::NextStep` (line 62) always uses `Implementation::TerminalPayload`. Each presenter then branches inside itself. `FailureDirective` has one fixed text for every escalation reason. User direction (2026-09-29): a presenter owns each communication that the AI or the user sees. The code triggers the reply and selects the presenter that fits the outcome. ## Reproduce n/a: design ticket. The wrong text is visible in the `Reply#message` branch for any non-PASS verdict. ## Approach Select the presenter by verdict and classification at the call site (`Gate::Run#record`), so each presenter has one message and no branch on verdict. Replies per classification: `unexpected_pass` (test-writer): ``` diff-test: unexpected_pass. Your new test passed before any production change. Compare the test with the behavior the ticket adds: - The test does not use the missing behavior: rewrite it so it fails for that reason, then rerun `os-sdlc-gate <id>`. - The behavior already exists: stop and report. The run escalates to the Orchestrator. ``` `green_report_nonzero_exit`: stop and report; the reply names the classification and points to the known causes in `references/escalation-procedure.md`. It does not tell the agent to correct failures or rerun. `agents/test-writer.md:111-120`: separate `unexpected_pass` (diagnose, then repair or report, as above) from `broken` / `test-lint` (repair loop, unchanged). Terminal side: `FailureDirective` takes the escalation reason and names the matching section of `references/escalation-procedure.md` (the cause text stays only in that file). `TerminalPayload` (presenter/implementation/terminal_payload.rb:34) passes the reason; `OpenPayload:19` calls it with no reason and keeps today's text. Required work: none beyond the acceptance criteria. ## Owner sketch - `Gate::Run#record` (command/gate/run.rb:77) owns the choice: it selects the reply presenter from the verdict's classification instead of always building `Reply`. - One presenter per outcome under presenter/gate/, each with one fixed message and no branch on verdict: - `Reply`: pass and fail (text unchanged). - a reply for `unexpected_pass` (repair-or-report text in Approach). - a reply for `green_report_nonzero_exit` (stop-and-report text, points to references/escalation-procedure.md). The shared payload shape (`verdict`, `step`, `classification`, `passed_steps`, `output`) stays the same for all of them. - `FailureDirective` owns the directive text: `FailureDirective.for(reason:)` returns the section-specific text for a known reason prefix and today's text for `nil` or an unknown prefix. `TerminalPayload` passes `@implementation.failed_reason`; `OpenPayload` passes no reason. - `agents/test-writer.md` owns the agent-side instruction for `unexpected_pass`. ## Acceptance criteria 1. Given a test-writer gate with classification `unexpected_pass`, when the agent receives the gate reply, then the reply does not say "failures" and tells the agent to repair the test or, if the behavior already exists, to stop and report. 2. Given a gate with classification `green_report_nonzero_exit`, when the agent receives the gate reply, then the reply tells the agent to stop and report, and does not tell it to correct failures. 3. Given an escalation with reason prefix `green_report_nonzero_exit:`, when the Orchestrator receives the terminal payload, then the directive names the matching section of `references/escalation-procedure.md`. 4. Given a gate verdict of fail, when the agent receives the gate reply, then the reply is unchanged ("correct the <step> failures above, then rerun ..."). ## Origin - Trigger: #813 seam analysis (presenter inventory), 2026-09-29 - Improvised this session: none; known causes were put in references/escalation-procedure.md instead - Chain: wrong repair instruction on escalate ← `Gate::Reply` branches only on PASS ← callers hard-code one presenter per call site ← DESIGN (presenter/gate/reply.rb, pipeline/next_step.rb:62) - Root candidate: this ticket - Where: plugins/os-sdlc/lib/os_sdlc/presenter/gate/reply.rb; presenter/implementation/terminal_payload.rb; presenter/failure_directive.rb - Session: 0aa46819-aa36-4e80-9438-9ef8b7314bf2 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-ticket-takedown/0aa46819-aa36-4e80-9438-9ef8b7314bf2.jsonl - Approach revised 2026-09-30 (user ruling, option A; see comment)

jared added the

plugin/os-sdlc

filed-by/agent

labels

2026-09-29 15:55:04 +00:00

jared referenced this issue

2026-09-29 15:55:17 +00:00

Suite gate fails green suites when the machine is under load #813

jared commented

2026-09-30 11:19:35 +00:00

Author

Owner

Approach update (2026-09-30, user ruling)

Corrections to the ticket text:

Current paths: presenter/gate/reply.rb:52-56, command/gate/run.rb:77 (Reply.new), pipeline/next_step.rb:62, presenter/failure_directive.rb.
unexpected_pass comes only from the test-writer node's diff-test gate (expect: red), not from suite-check. green_report_nonzero_exit comes from any expect-green test gate (programmer, refactorer, suite-check).
The escalate verdict ends the run only if the agent stops while the verdict is still escalate. A rerun that gives a correct red result overwrites the verdict with pass (ADR-0158 routing does not change).

Chosen reply per classification (option A):

unexpected_pass (test-writer):

diff-test: unexpected_pass. Your new test passed before any production change.
Compare the test with the behavior the ticket adds:
- The test does not use the missing behavior: rewrite it so it fails for that
  reason, then rerun `os-sdlc-gate <id>`.
- The behavior already exists: stop and report. The run escalates to the Orchestrator.

green_report_nonzero_exit (suite-check never reruns, agents/suite-check.md:59): stop and report; the reply names the classification and the known causes in references/escalation-procedure.md.

Added scope: agents/test-writer.md:111-120 separates unexpected_pass (diagnose, then repair or report) from broken / test-lint (repair loop).

Terminal side: FailureDirective takes the escalation reason and names the matching section of references/escalation-procedure.md; the cause text stays only in that file. OpenPayload:19 calls it with no reason.

Revised acceptance criteria:

Given a test-writer gate with classification unexpected_pass, when the agent receives the gate reply, then the reply does not say "failures" and tells the agent to repair the test or, if the behavior already exists, to stop and report.
Given a gate with classification green_report_nonzero_exit, when the agent receives the gate reply, then the reply tells the agent to stop and report, and does not tell it to correct failures.
Given an escalation with reason prefix green_report_nonzero_exit:, when the Orchestrator receives the terminal payload, then the directive names the matching section of references/escalation-procedure.md.

## Approach update (2026-09-30, user ruling) Corrections to the ticket text: - Current paths: `presenter/gate/reply.rb:52-56`, `command/gate/run.rb:77` (`Reply.new`), `pipeline/next_step.rb:62`, `presenter/failure_directive.rb`. - `unexpected_pass` comes only from the test-writer node's `diff-test` gate (`expect: red`), not from suite-check. `green_report_nonzero_exit` comes from any expect-green test gate (programmer, refactorer, suite-check). - The escalate verdict ends the run only if the agent stops while the verdict is still escalate. A rerun that gives a correct red result overwrites the verdict with pass (ADR-0158 routing does not change). Chosen reply per classification (option A): `unexpected_pass` (test-writer): ``` diff-test: unexpected_pass. Your new test passed before any production change. Compare the test with the behavior the ticket adds: - The test does not use the missing behavior: rewrite it so it fails for that reason, then rerun `os-sdlc-gate <id>`. - The behavior already exists: stop and report. The run escalates to the Orchestrator. ``` `green_report_nonzero_exit` (suite-check never reruns, `agents/suite-check.md:59`): stop and report; the reply names the classification and the known causes in `references/escalation-procedure.md`. Added scope: `agents/test-writer.md:111-120` separates `unexpected_pass` (diagnose, then repair or report) from `broken` / `test-lint` (repair loop). Terminal side: `FailureDirective` takes the escalation reason and names the matching section of `references/escalation-procedure.md`; the cause text stays only in that file. `OpenPayload:19` calls it with no reason. Revised acceptance criteria: 1. Given a test-writer gate with classification `unexpected_pass`, when the agent receives the gate reply, then the reply does not say "failures" and tells the agent to repair the test or, if the behavior already exists, to stop and report. 2. Given a gate with classification `green_report_nonzero_exit`, when the agent receives the gate reply, then the reply tells the agent to stop and report, and does not tell it to correct failures. 3. Given an escalation with reason prefix `green_report_nonzero_exit:`, when the Orchestrator receives the terminal payload, then the directive names the matching section of `references/escalation-procedure.md`.

jared commented

2026-09-30 11:21:41 +00:00

Author

Owner

Work started: os-sdlc pipeline on branch ticket-826 (worktree .claude/worktrees/ticket-826).

jared commented

2026-09-30 11:28:52 +00:00

Author

Owner

Resolution

Done: Gate::Run#record selects the reply presenter by classification: UnexpectedPassReply (repair-or-report), GreenReportNonzeroExitReply (stop and report), Reply for pass/fail (unchanged). FailureDirective.for(reason:) names the matching escalation-procedure.md section; TerminalPayload passes the reason, OpenPayload passes none. agents/test-writer.md separates unexpected_pass from broken/test-lint.

Evidence: Commit 2e9b4d99 on branch ticket-826 (os-sdlc implementation 3): tests run_reply_test.rb and terminal_escalation_directive_test.rb red then green; behavior-verifier passed all 4 acceptance criteria; suite-check gate passed; contract-auditor passed.

Follow-ups: none

## Resolution **Done:** Gate::Run#record selects the reply presenter by classification: UnexpectedPassReply (repair-or-report), GreenReportNonzeroExitReply (stop and report), Reply for pass/fail (unchanged). FailureDirective.for(reason:) names the matching escalation-procedure.md section; TerminalPayload passes the reason, OpenPayload passes none. agents/test-writer.md separates unexpected_pass from broken/test-lint. **Evidence:** Commit 2e9b4d99 on branch ticket-826 (os-sdlc implementation 3): tests run_reply_test.rb and terminal_escalation_directive_test.rb red then green; behavior-verifier passed all 4 acceptance criteria; suite-check gate passed; contract-auditor passed. **Follow-ups:** none

jared closed this issue

2026-09-30 11:28:53 +00:00

jared commented

2026-09-30 11:30:21 +00:00

Author

Owner

Follow-up dropped by ticket-skeptic: os-sdlc-implementation-report refusing after implementation_complete is already open as #836.

jared referenced this issue

2026-09-30 11:30:22 +00:00

os-sdlc-implementation-report refuses after implementation_complete #836

jared referenced this issue from a commit

2026-10-01 18:24:32 +00:00