jared/cc-os

Watch

Fork 0

behavior-verifier brief omits test_changes, producing a wrong fail and an unexpected_pass escalation #1111

New issue

Closed

opened 2026-10-18 16:47:53 +00:00 by jared · 5 comments

jared commented

2026-10-18 16:47:53 +00:00

Owner

Problem

The behavior-verifier brief carries no test_changes. The verifier cannot see tests that already exist in the working tree. It marks work as outstanding when that work is done.

Observed

Implementation 65 of #1105, target os-backlog.
The programmer went green in dispatch 582.
The behavior-verifier brief in dispatch 583 carried no test_changes.
The verifier marked the AC2 nonzero-exit case and the AC4 ten-name pins as outstanding. Both tests were already in the working tree and green.
The Runner sent the run to test-writer in dispatch 584. Its diff-test returned unexpected_pass.
The run escalated to needs-orchestrator (escalation 38). The orchestrator hand-finished the work.

Reproduce

n/a: the evidence is implementation 65 of #1105 (dispatches 582-584, escalation 38).

Approach

Enrich test_changes (and red_tests, if the verifier needs them) at the behavior-verifier node, or have Brief::Facts supply them to that node.

Required work:

Supply test_changes to the behavior-verifier node — check: present test_changes in plugins/os-sdlc/maps/poodr-ticket-implementation.yaml
Add a test that the verifier brief for a green increment contains the round's test diff.

Acceptance criteria

Given a green increment with placed tests, when the behavior-verifier brief is built, then it contains the round's test diff in test_changes.
Given tests already in the working tree and green, when the verifier runs, then it does not mark them outstanding.

Owner sketch

class Brief::Facts
  def test_changes
end

Origin

Trigger: implementation 65 of #1105, dispatches 582-584
Improvised this session: orchestrator hand-finished the run
Chain: unexpected_pass escalation 38 ← verifier marked done tests outstanding ← brief 583 omitted test_changes ← DESIGN (plugins/os-sdlc/maps/poodr-ticket-implementation.yaml)
Root candidate: none
Where: plugins/os-sdlc/maps/poodr-ticket-implementation.yaml, Brief::Facts
Session: 26783407-6911-48d5-a818-55fbfbc70108
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/26783407-6911-48d5-a818-55fbfbc70108.jsonl

VERDICT: CREATE

## Problem The behavior-verifier brief carries no `test_changes`. The verifier cannot see tests that already exist in the working tree. It marks work as outstanding when that work is done. ## Observed Implementation 65 of #1105, target os-backlog. The programmer went green in dispatch 582. The behavior-verifier brief in dispatch 583 carried no `test_changes`. The verifier marked the AC2 nonzero-exit case and the AC4 ten-name pins as outstanding. Both tests were already in the working tree and green. The Runner sent the run to test-writer in dispatch 584. Its diff-test returned `unexpected_pass`. The run escalated to needs-orchestrator (escalation 38). The orchestrator hand-finished the work. ## Reproduce n/a: the evidence is implementation 65 of #1105 (dispatches 582-584, escalation 38). ## Approach Enrich `test_changes` (and `red_tests`, if the verifier needs them) at the behavior-verifier node, or have `Brief::Facts` supply them to that node. Required work: 1. Supply `test_changes` to the behavior-verifier node — check: present `test_changes` in `plugins/os-sdlc/maps/poodr-ticket-implementation.yaml` 2. Add a test that the verifier brief for a green increment contains the round's test diff. ## Acceptance criteria 1. Given a green increment with placed tests, when the behavior-verifier brief is built, then it contains the round's test diff in `test_changes`. 2. Given tests already in the working tree and green, when the verifier runs, then it does not mark them outstanding. ## Owner sketch ```ruby class Brief::Facts def test_changes end ``` ## Origin - Trigger: implementation 65 of #1105, dispatches 582-584 - Improvised this session: orchestrator hand-finished the run - Chain: unexpected_pass escalation 38 ← verifier marked done tests outstanding ← brief 583 omitted test_changes ← DESIGN (plugins/os-sdlc/maps/poodr-ticket-implementation.yaml) - Root candidate: none - Where: plugins/os-sdlc/maps/poodr-ticket-implementation.yaml, Brief::Facts - Session: 26783407-6911-48d5-a818-55fbfbc70108 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/26783407-6911-48d5-a818-55fbfbc70108.jsonl VERDICT: CREATE

jared added the

plugin/os-sdlc

filed-by/agent

category/code-change

labels

2026-10-18 16:47:53 +00:00

jared referenced this issue

2026-10-18 16:52:59 +00:00

Add category/structure and an os-backlog categorize gate #1105

jared referenced this issue

2026-10-18 18:58:39 +00:00

Runner commit message on implementation_complete is the contract-auditor summary #987

jared commented

2026-10-18 18:58:40 +00:00

Author

Owner

Resolution

Done: Mechanism: Brief::Facts code_changes/test_changes do not show the real increment. Occurrence: #1114 "Add test_changes to the behavior-verifier brief".

Evidence: #1114 adds evidence that every behavior-verifier dispatch in run 64 (#1104) reported test_changes absent; it locates the cause in the behavior-verifier node enrich list of maps/poodr-ticket-implementation.yaml.

Follow-ups: none

## Resolution **Done:** Mechanism: Brief::Facts code_changes/test_changes do not show the real increment. Occurrence: #1114 "Add test_changes to the behavior-verifier brief". **Evidence:** #1114 adds evidence that every behavior-verifier dispatch in run 64 (#1104) reported test_changes absent; it locates the cause in the behavior-verifier node enrich list of maps/poodr-ticket-implementation.yaml. **Follow-ups:** none

jared referenced this issue

2026-10-18 18:59:18 +00:00

Add test_changes to the behavior-verifier brief #1114

jared referenced this issue

2026-10-21 21:47:58 +00:00

Brief builder gives no test_changes when the project root resolves outside the worktree #1227

jared commented

2026-10-23 16:01:41 +00:00

Author

Owner

Occurrence: Give behavior-verifier every changed file of the green diff
Seen: plugins/os-sdlc/lib/os_sdlc/dispatch/brief/facts.rb
Run: session b0da7c4b-f4b3-4f3b-96cd-0b77e82451d4, implementation 118 (ticket #1239), dispatches 979 (programmer), 982 (behavior-verifier), 983 (test-writer), escalation 64
Adds: non-test files under tests/ (tests/all.rb) and files under docs/ are missing from code_changes; the brief had no test_changes. The verifier marked AC1 outstanding; test-writer gated green (2452 runs, 0 failures) and escalated unexpected_pass. Orchestrator hand-finished.

Occurrence: Give behavior-verifier every changed file of the green diff Seen: plugins/os-sdlc/lib/os_sdlc/dispatch/brief/facts.rb Run: session b0da7c4b-f4b3-4f3b-96cd-0b77e82451d4, implementation 118 (ticket #1239), dispatches 979 (programmer), 982 (behavior-verifier), 983 (test-writer), escalation 64 Adds: non-test files under tests/ (tests/all.rb) and files under docs/ are missing from code_changes; the brief had no test_changes. The verifier marked AC1 outstanding; test-writer gated green (2452 runs, 0 failures) and escalated unexpected_pass. Orchestrator hand-finished.

jared referenced this issue

2026-10-23 16:01:42 +00:00

Stop the 60s suite budget from failing bin/fast-gate on a loaded host #1239

jared commented

2026-10-26 14:24:23 +00:00

Author

Owner

Occurrence: Let behavior-verifier judge a ticket whose change is only test-support code
Seen: plugins/os-sdlc/maps/poodr-ticket-implementation.yaml (behavior-verifier enrich) and Brief::Facts#code_changes
Run: Session: 0bd25a72-f8ca-4331-a047-9cefdb9b0e3b; implementation 168 (ticket #1394), dispatch 1465
Adds: An all-test diff gives code_changes null, and the verifier settles error with MISSING: code_changes. Add a criterion: code_changes null with a populated test_changes must not settle error.

Occurrence: Let behavior-verifier judge a ticket whose change is only test-support code Seen: plugins/os-sdlc/maps/poodr-ticket-implementation.yaml (behavior-verifier enrich) and Brief::Facts#code_changes Run: Session: 0bd25a72-f8ca-4331-a047-9cefdb9b0e3b; implementation 168 (ticket #1394), dispatch 1465 Adds: An all-test diff gives code_changes null, and the verifier settles error with MISSING: code_changes. Add a criterion: code_changes null with a populated test_changes must not settle error.

jared referenced this issue

2026-10-26 14:25:32 +00:00

Trace the handoff complete journey in plugins/os and build its site #1394

jared commented

2026-10-26 15:47:04 +00:00

Author

Owner

Occurrence: behavior-verifier brief has no test_changes after a reroute
Seen: plugins/os-sdlc/lib/os_sdlc/dispatch/brief/facts.rb (#test_changes) and the poodr-ticket-implementation verifier node
Run: implementation 169 (#1393), session 4453856e-c594-48a3-8871-c95b3f21ff40, dispatch 1478, handoff 1508, escalation 78
Adds: test edits were uncommitted in the worktree (orchestrator hand-fix of 48 test files plus one test, before reroute to programmer), so a working-tree diff against HEAD should have shown them; the verifier still reported 'test_changes is absent from the dispatch brief' and failed AC 1, then test-writer hit unexpected_pass. Points to the verifier node's brief not carrying the fact, not to an empty diff.

Occurrence: behavior-verifier brief has no test_changes after a reroute Seen: plugins/os-sdlc/lib/os_sdlc/dispatch/brief/facts.rb (#test_changes) and the poodr-ticket-implementation verifier node Run: implementation 169 (#1393), session 4453856e-c594-48a3-8871-c95b3f21ff40, dispatch 1478, handoff 1508, escalation 78 Adds: test edits were uncommitted in the worktree (orchestrator hand-fix of 48 test files plus one test, before reroute to programmer), so a working-tree diff against HEAD should have shown them; the verifier still reported 'test_changes is absent from the dispatch brief' and failed AC 1, then test-writer hit unexpected_pass. Points to the verifier node's brief not carrying the fact, not to an empty diff.

jared commented

2026-10-27 15:43:37 +00:00

Author

Owner

Work started on branch ticket-1111 through /os-sdlc:implement (map poodr-ticket-implementation).

jared closed this issue

2026-10-27 15:54:33 +00:00

jared referenced this issue

2026-10-27 15:55:39 +00:00

Trace the os-backlog comment journey in plugins/os-backlog and build its site #1395

jared referenced this issue from a commit

2026-10-27 15:55:40 +00:00