jared/cc-os

Watch

Fork 0

Let the programmer escalate a test defect from a gate-sourced node #890

New issue

Closed

opened 2026-10-01 20:19:09 +00:00 by jared · 4 comments

jared commented

2026-10-01 20:19:09 +00:00

Owner

Problem

A test defect that shows only after production code exists burns the programmer-repair bound. Programmers may not edit tests, and a gate-sourced node ignores any agent verdict, so the programmer can only retry until the bound is spent.

Observed

In #873 implementation 5, every test first raised KeyError: key not found: "implementation-map-update" at controller/boot/commands.rb:54. That was the intended missing seam, so test-reviewer passed correctly. After the programmer built the command, run_cli_ok in tests/support/subagent_start_cli.rb lacked stdin:. Dispatches 34-36 spent the repair bound and escalated bound_exhausted:programmer-repair/fail.
In #880 implementation 2, test-reviewer saw Errno::ENOENT for the drafted map, which was also the intended missing behavior. After production code, Brief#compose raised NoMethodError: undefined method 'fact' for an instance of OsSdlc::BriefDraftedMapTest::Facts (presenter/brief.rb:51). Three programmer-repair dispatches could not fix it.
In both runs the defect could not show at test-review time. programmer.md says "If a test appears wrong, disclose that fact but leave the test unchanged". CompletionIntake#verdict_outcome ignores agent verdicts on gate-sourced nodes, so that disclosure routes nowhere.

Reproduce

n/a: process defect in past runs; see #873 implementation 5 and #880 implementation 2.

Approach

Let the programmer end its repair loop on a test defect with the existing escalate verdict (ADR-0258 decision 2: downstream agents escalate). On a gate-sourced node, CompletionIntake#verdict_outcome takes escalate from the handoff content when present. Any other agent verdict on a gate-sourced node is ignored, so an agent cannot pass a red gate. The run goes to needs-orchestrator, and the orchestrator hand-fixes the test under the escalation procedure. The test-reviewer brief does not change: both triggers were red for the intended reason at review time. HandoffContent already accepts an optional verdict when the verdict is not required.

Required work:

Tell the programmer to escalate on a test defect — check: present verdict: escalate in plugins/os-sdlc/agents/programmer.md
Remove the disclose-only instruction — check: absent If a test appears wrong, disclose that fact but leave the test unchanged in plugins/os-sdlc/agents/programmer.md
Remove the no-verdict instruction at programmer.md:113 — check: absent field. The gates determine the result. in plugins/os-sdlc/agents/programmer.md
New ADR amending 0141/0134/0135, created by the orchestrator with /os-adr:create before implementation-open, with this decision text: "On a node whose verdict source is gate, the handoff may carry verdict: escalate, and only that value. When it is present, CompletionIntake records escalate as the dispatch verdict, whatever the gate recorded and whether or not the gate ran. The run then follows the node's escalate edge. Any other handoff verdict on a gate-sourced node is ignored. Agents still cannot author pass, fail or error on gated nodes. The gate step results stay the record of the gate classification (ADR-0248)." — check: present Agents still cannot author pass, fail or error on gated nodes in docs/adr/*.md
Let the handoff format accept verdict: escalate on gate-sourced nodes in /home/jared/dev/cc-os/plugins/os-sdlc/reference/handoff-artifact-format.md (line 31) — check: absent never expects a in plugins/os-sdlc/reference/handoff-artifact-format.md
Name a programmer escalation on a test defect as an escalation cause in /home/jared/dev/cc-os/plugins/os-sdlc/references/escalation-procedure.md (line 3) — check: present test defect in plugins/os-sdlc/references/escalation-procedure.md

Acceptance criteria

Given a run at programmer-repair whose gate recorded fail, when the programmer submits a handoff with verdict: escalate and a disclosure that names the test defect, then the dispatch verdict is escalate and the run's position is needs-orchestrator.
Given a run at programmer whose gate was never run, when the programmer submits a handoff with verdict: escalate, then the dispatch verdict is escalate (not a gate_skip error), no gate_skip step result is recorded, and the run's position is needs-orchestrator.
Regression guard (passes today): Given a run at programmer whose gate recorded fail, when the programmer submits a handoff with verdict: pass, then the dispatch verdict is fail and the run routes to programmer-repair.
Regression guard (passes today): Given a run at programmer whose gate recorded pass, when the programmer submits a handoff with no verdict, then the dispatch verdict is pass and the run routes to behavior-verifier.

Owner sketch

class CompletionIntake
  def verdict_outcome
    return agent_verdict_outcome if node.verdict_source == "agent"
    return agent_escalation_outcome if agent_escalated?

    gate_verdict_outcome
  end
end

Origin

Trigger: implementation 5 of #873; implementation 2 of #880 (see #903)
Improvised this session: hand-fixed run_cli_ok stdin:, reopened as implementation 6
Chain: escalation after a spent repair bound ← programmer cannot route a test defect ← gate-sourced nodes ignore agent verdicts ← DESIGN (CompletionIntake verdict source)
Root candidate: none
Where: plugins/os-sdlc/lib/os_sdlc/command/completion_intake.rb, plugins/os-sdlc/agents/programmer.md
Session: n/a (filed by a subagent of the #873 session)
Transcript: n/a

Found during #873. Approach revised 2026-10-02 after the predicted-ruling review (see comment).

## Problem A test defect that shows only after production code exists burns the programmer-repair bound. Programmers may not edit tests, and a gate-sourced node ignores any agent verdict, so the programmer can only retry until the bound is spent. ## Observed In #873 implementation 5, every test first raised `KeyError: key not found: "implementation-map-update"` at `controller/boot/commands.rb:54`. That was the intended missing seam, so test-reviewer passed correctly. After the programmer built the command, `run_cli_ok` in `tests/support/subagent_start_cli.rb` lacked `stdin:`. Dispatches 34-36 spent the repair bound and escalated `bound_exhausted:programmer-repair/fail`. In #880 implementation 2, test-reviewer saw `Errno::ENOENT` for the drafted map, which was also the intended missing behavior. After production code, `Brief#compose` raised `NoMethodError: undefined method 'fact' for an instance of OsSdlc::BriefDraftedMapTest::Facts` (presenter/brief.rb:51). Three programmer-repair dispatches could not fix it. In both runs the defect could not show at test-review time. `programmer.md` says "If a test appears wrong, disclose that fact but leave the test unchanged". `CompletionIntake#verdict_outcome` ignores agent verdicts on gate-sourced nodes, so that disclosure routes nowhere. ## Reproduce n/a: process defect in past runs; see #873 implementation 5 and #880 implementation 2. ## Approach Let the programmer end its repair loop on a test defect with the existing `escalate` verdict (ADR-0258 decision 2: downstream agents escalate). On a gate-sourced node, `CompletionIntake#verdict_outcome` takes `escalate` from the handoff content when present. Any other agent verdict on a gate-sourced node is ignored, so an agent cannot pass a red gate. The run goes to `needs-orchestrator`, and the orchestrator hand-fixes the test under the escalation procedure. The test-reviewer brief does not change: both triggers were red for the intended reason at review time. `HandoffContent` already accepts an optional verdict when the verdict is not required. Required work: 1. Tell the programmer to escalate on a test defect — check: present `verdict: escalate` in `plugins/os-sdlc/agents/programmer.md` 2. Remove the disclose-only instruction — check: absent `If a test appears wrong, disclose that fact but leave the test unchanged` in `plugins/os-sdlc/agents/programmer.md` 3. Remove the no-verdict instruction at programmer.md:113 — check: absent `field. The gates determine the result.` in `plugins/os-sdlc/agents/programmer.md` 4. New ADR amending 0141/0134/0135, created by the orchestrator with /os-adr:create before implementation-open, with this decision text: "On a node whose verdict source is `gate`, the handoff may carry `verdict: escalate`, and only that value. When it is present, CompletionIntake records `escalate` as the dispatch verdict, whatever the gate recorded and whether or not the gate ran. The run then follows the node's `escalate` edge. Any other handoff verdict on a gate-sourced node is ignored. Agents still cannot author pass, fail or error on gated nodes. The gate step results stay the record of the gate classification (ADR-0248)." — check: present `Agents still cannot author pass, fail or error on gated nodes` in `docs/adr/*.md` 5. Let the handoff format accept `verdict: escalate` on gate-sourced nodes in `/home/jared/dev/cc-os/plugins/os-sdlc/reference/handoff-artifact-format.md` (line 31) — check: absent `never expects a` in `plugins/os-sdlc/reference/handoff-artifact-format.md` 6. Name a programmer escalation on a test defect as an escalation cause in `/home/jared/dev/cc-os/plugins/os-sdlc/references/escalation-procedure.md` (line 3) — check: present `test defect` in `plugins/os-sdlc/references/escalation-procedure.md` ## Acceptance criteria 1. Given a run at programmer-repair whose gate recorded fail, when the programmer submits a handoff with `verdict: escalate` and a disclosure that names the test defect, then the dispatch verdict is `escalate` and the run's position is `needs-orchestrator`. 2. Given a run at programmer whose gate was never run, when the programmer submits a handoff with `verdict: escalate`, then the dispatch verdict is `escalate` (not a `gate_skip` error), no `gate_skip` step result is recorded, and the run's position is `needs-orchestrator`. 3. Regression guard (passes today): Given a run at programmer whose gate recorded fail, when the programmer submits a handoff with `verdict: pass`, then the dispatch verdict is `fail` and the run routes to programmer-repair. 4. Regression guard (passes today): Given a run at programmer whose gate recorded pass, when the programmer submits a handoff with no verdict, then the dispatch verdict is `pass` and the run routes to behavior-verifier. ## Owner sketch ```ruby class CompletionIntake def verdict_outcome return agent_verdict_outcome if node.verdict_source == "agent" return agent_escalation_outcome if agent_escalated? gate_verdict_outcome end end ``` ## Origin - Trigger: implementation 5 of #873; implementation 2 of #880 (see #903) - Improvised this session: hand-fixed `run_cli_ok` `stdin:`, reopened as implementation 6 - Chain: escalation after a spent repair bound ← programmer cannot route a test defect ← gate-sourced nodes ignore agent verdicts ← DESIGN (CompletionIntake verdict source) - Root candidate: none - Where: plugins/os-sdlc/lib/os_sdlc/command/completion_intake.rb, plugins/os-sdlc/agents/programmer.md - Session: n/a (filed by a subagent of the #873 session) - Transcript: n/a Found during #873. Approach revised 2026-10-02 after the predicted-ruling review (see comment).

jared added the

plugin/os-sdlc

filed-by/agent

labels

2026-10-01 20:19:09 +00:00

jared referenced this issue

2026-10-01 20:19:13 +00:00

Add the reroute gate and the reroute landing with the REROUTE block #873

jared referenced this issue

2026-10-01 20:25:09 +00:00

no pipeline path for guard/characterization tests (unexpected_pass escalation) #859

jared commented

2026-10-01 21:15:28 +00:00

Author

Owner

More evidence (2026-10-01, #878 implementation 11): test-reviewer passed plugins/os-sdlc/tests/controller/implementation/change_auditor_reroute_test.rb on round 2 (dispatch 74) although its helper stop_change_auditor sent subagent-stop without a dispatch_id and never called next session-1, so no implementation could route the run. The reviewer prompt told it to run the file and check helper validity. Two programmer dispatches (75, 76) were spent before the programmer named the defect. Hand-fixed in 5125c6b5; reopened as implementation 12, which completed (a99c06fb).

More evidence (2026-10-01, #878 implementation 11): test-reviewer passed plugins/os-sdlc/tests/controller/implementation/change_auditor_reroute_test.rb on round 2 (dispatch 74) although its helper stop_change_auditor sent subagent-stop without a dispatch_id and never called next session-1, so no implementation could route the run. The reviewer prompt told it to run the file and check helper validity. Two programmer dispatches (75, 76) were spent before the programmer named the defect. Hand-fixed in 5125c6b5; reopened as implementation 12, which completed (a99c06fb).

jared referenced this issue

2026-10-01 21:15:37 +00:00

Accept a reroute verdict from change-auditor and route it to map-router #878

jared referenced this issue

2026-10-01 21:23:06 +00:00

no pipeline path for guard/characterization tests (unexpected_pass escalation) #859

jared referenced this issue

2026-10-02 14:19:12 +00:00

Test-reviewer passes a stub that cannot run the code under test; diff-test stops at the first red step #903

jared referenced this issue from a commit

2026-10-02 17:43:17 +00:00

test(os-sdlc): #878 hand-fix: stop_change_auditor calls next after subagent-stop so the run routes (implementation 11 test defect, #890)

jared commented

2026-10-02 19:49:44 +00:00

Author

Owner

PREDICTED RULING: REVISE

a. No. The brief edit would not be accepted.

b. Reasons:

feedback-lens-request-is-a-design-symptom, lens L2: challenge the code first. A charter clause is a rule edit.
feedback-remove-habits-dont-backfill-heuristics: test-reviewer.md lines 9, 19 and 21 already require "red for the intended reason".
feedback-no-speculative-hardening-in-artifacts: the clause has no mechanism. The reviewer may "Judge only from the supplied facts", and red_tests drops backtrace frames (parser/test_report.rb:5).

c. The user would call it lazy because the clause does not stop its own trigger. In #873 the KeyError came from controller/boot/commands.rb:54, the intended missing seam. In #880 the reviewer saw Errno::ENOENT, the intended behavior. Both defects showed only after production code existed. Then the programmer, which may not edit tests and whose agent verdict CompletionIntake#verdict_outcome ignores, used up programmer-repair. Research done: TestReport, CompletionIntake, programmer.md, the maps, ADR-0211 and ADR-0258, and the #873 and #880 transcripts.

d. Approach: on gate-sourced nodes, CompletionIntake accepts only the existing escalate verdict from the handoff. programmer.md tells the programmer to escalate on a test defect, and the orchestrator hand-fixes the test (escalation procedure step 4). This stays inside ADR-0258 decision 2: downstream agents escalate. The body is edited to match.

e. Not read: the tests that pin how gate nodes treat agent verdicts, and ADR-0133. ADR-0135 says handoffs carry a verdict "where the node's verdict source is agent". HandoffContent already accepts an optional verdict. If the user reads ADR-0135 as exclusive, it needs a clarifying amendment.

PREDICTED RULING: REVISE a. No. The brief edit would not be accepted. b. Reasons: - feedback-lens-request-is-a-design-symptom, lens L2: challenge the code first. A charter clause is a rule edit. - feedback-remove-habits-dont-backfill-heuristics: test-reviewer.md lines 9, 19 and 21 already require "red for the intended reason". - feedback-no-speculative-hardening-in-artifacts: the clause has no mechanism. The reviewer may "Judge only from the supplied facts", and red_tests drops backtrace frames (parser/test_report.rb:5). c. The user would call it lazy because the clause does not stop its own trigger. In #873 the KeyError came from controller/boot/commands.rb:54, the intended missing seam. In #880 the reviewer saw Errno::ENOENT, the intended behavior. Both defects showed only after production code existed. Then the programmer, which may not edit tests and whose agent verdict CompletionIntake#verdict_outcome ignores, used up programmer-repair. Research done: TestReport, CompletionIntake, programmer.md, the maps, ADR-0211 and ADR-0258, and the #873 and #880 transcripts. d. Approach: on gate-sourced nodes, CompletionIntake accepts only the existing `escalate` verdict from the handoff. programmer.md tells the programmer to escalate on a test defect, and the orchestrator hand-fixes the test (escalation procedure step 4). This stays inside ADR-0258 decision 2: downstream agents escalate. The body is edited to match. e. Not read: the tests that pin how gate nodes treat agent verdicts, and ADR-0133. ADR-0135 says handoffs carry a verdict "where the node's verdict source is agent". HandoffContent already accepts an optional verdict. If the user reads ADR-0135 as exclusive, it needs a clarifying amendment.

jared referenced this issue

2026-10-02 19:50:17 +00:00

Test-reviewer passes a stub that cannot run the code under test; diff-test stops at the first red step #903

jared changed title from ~~Require test-reviewer to confirm tests reach production code~~ to Let the programmer escalate a test defect from a gate-sourced node

2026-10-02 19:53:28 +00:00

jared commented

2026-10-02 20:02:48 +00:00

Author

Owner

Work started on branch ticket-890 through /os-sdlc:implement (poodr-ticket-implementation map). ADR-0269 (amends ADR-0141, ADR-0134, ADR-0135) committed on the branch.

jared commented

2026-10-02 20:13:00 +00:00

Author

Owner

Resolution

Done: On a gate-sourced node a handoff verdict: escalate now becomes the dispatch verdict (gate ran or not) and the run follows the escalate edge; other agent verdicts stay ignored. programmer.md tells the programmer to escalate a defective test. HandoffContent verdict set now comes from Verdict::ALL. handoff-artifact-format.md and escalation-procedure.md updated. ADR-0269 amends ADR-0141/0134/0135; decided autonomously in an AFK run, awaiting the user's ruling.

Evidence: Implementation 23 complete, 13 dispatches, one test-reviewer and one contract-auditor fail each fixed in-run, suite gate exit 0 twice. New test tests/pipeline/cli_subagent_stop_gate_escalation_test.rb covers gate-fail escalate, ungated escalate, and two regression guards. Commits 5f0ffe85 (ADR-0269), 25d69c3c, 4dc28101 on main.

Follow-ups: none

## Resolution **Done:** On a gate-sourced node a handoff verdict: escalate now becomes the dispatch verdict (gate ran or not) and the run follows the escalate edge; other agent verdicts stay ignored. programmer.md tells the programmer to escalate a defective test. HandoffContent verdict set now comes from Verdict::ALL. handoff-artifact-format.md and escalation-procedure.md updated. ADR-0269 amends ADR-0141/0134/0135; decided autonomously in an AFK run, awaiting the user's ruling. **Evidence:** Implementation 23 complete, 13 dispatches, one test-reviewer and one contract-auditor fail each fixed in-run, suite gate exit 0 twice. New test tests/pipeline/cli_subagent_stop_gate_escalation_test.rb covers gate-fail escalate, ungated escalate, and two regression guards. Commits 5f0ffe85 (ADR-0269), 25d69c3c, 4dc28101 on main. **Follow-ups:** none

jared closed this issue

2026-10-02 20:13:00 +00:00

jared referenced this issue

2026-10-02 20:20:31 +00:00

os-sdlc project-config --help treats --help as a project name and writes .sdlc/project.yaml #860

jared referenced this issue from a commit

2026-10-03 17:55:50 +00:00

docs(adr): ADR-0269 programmer may escalate from a gate-sourced node (#890)

jared referenced this issue from a commit

2026-10-03 17:55:50 +00:00

feat(os-sdlc): Audited POODR boundaries for #890; CompletionIntake#verdict_outcome owns the gate-node escalate branch; R1-R6 present; contracts protected. (#890)

jared referenced this issue from a commit