behavior-verifier gets a truncated code_changes brief fact and returns verdict:error, which fails the implementation; truncation must be disclosed and error must route to repair, not terminal

jared commented

2026-09-18 15:29:36 +00:00

Owner

VERDICT: CREATE (ticket-skeptic) — real, well-scoped defect distinct from #613/#614/#617; presentable() silently truncates diffs and behavior-verifier verdict:error has no repair route, so a green implementation was killed and had to be hand-finished; root is DESIGN in brief_facts.rb/verdict routing, echoes unticketed #528 note.

Parent: #613

Context

This surfaced in implementation 5 of #613 (target os-sdlc, 7 production files, 62 insertions). The programmer dispatch 44 went green. The behavior-verifier dispatch 45 could not judge the diff and returned an error verdict, which the Runner treated as terminal.

Observed

The behavior-verifier handoff said: "code_changes is truncated mid-diff; three of six ticket tasks cannot be judged". It listed "MISSING: complete code_changes -- the diff ends mid-hunk in the header line for plugins/os-sdlc/lib/os_sdlc/pipeline/open_implementation.rb". It returned verdict: error.

verdict: error

implementation_failed, reason "verdict:error" (terminal, no resume)

The main loop verified the diff by hand, ran the suite green (1354 runs), and hand-finished the ticket. A green implementation failed because of a brief-side defect, not a code defect.

Two defects combine to cause this:

BriefFacts#presentable truncates a diff without a visible marker or a fact stating the cap and the omitted files. The verifier has no way to ask for the rest.
A verdict: error from behavior-verifier routes straight to implementation_failed, with no repair or re-brief route. A brief-side defect kills a green implementation outright.

This echoes an earlier note from #528: verifier error pre-empted the suite-check repair route (unticketed at the time).

Reproduce

Run the ticket map on a ticket whose production diff exceeds the presentable cap, then observe the dispatch 45-style handoff.

Expected

presentable appends an explicit line, for example "TRUNCATED: N of M diff lines shown; files omitted: ...". The verifier charter instructs it to verify omitted files with Bash (git diff on the named paths) instead of returning error.
A verdict: error from behavior-verifier routes to one bounded re-dispatch of behavior-verifier with the full diff, or escalates. It never routes straight to implementation_failed.

Origin

Trigger: /os-sdlc:implement #613, implementation 5, behavior-verifier dispatch 45.
Improvised this session: hand-verified the diff and suite, then hand-finished the ticket.
Chain: implementation_failed on a green diff ← verdict:error with no repair route ← presentable truncates the diff with no visible marker. Stop: DESIGN, plugins/os-sdlc/lib/os_sdlc/pipeline/brief_facts.rb (presentable) and the behavior-verifier verdict-routing map.
Root candidate: none; this is the root ticket.
Where: plugins/os-sdlc/lib/os_sdlc/pipeline/brief_facts.rb (presentable); behavior-verifier verdict handling in the map / routing.
Session: 4974deff-0140-4813-8cd1-e96c6356a214
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/4974deff-0140-4813-8cd1-e96c6356a214.jsonl

Decisions 2026-09-19

Remove truncation from the two diff facts, code_changes and test_changes. A diff is bounded by the increment and is the verifier's only evidence. presentable goes away.
Keep the cap on the two transcript facts, gate results and lint_offense_history, and append a visible marker line when the cap cuts.
A verdict: error from behavior-verifier routes to one bounded re-dispatch of behavior-verifier, then escalates. It never routes straight to implementation_failed.
This amends ADR-0169 and ADR-0173, which state that truncation stays in the brief layer for code_changes. Record the amending ADR with /os-adr:create before the code change.

Slice

The verifier always sees the whole diff. Transcript facts that are cut say so. An error verdict from the verifier gets one retry before it can end the implementation.

Acceptance criteria

BriefFacts#code_changes and BriefFacts#test_changes return the full diff, or nil when empty. BriefFacts#presentable is removed.
BriefFacts#truncate (used by gate results and lint_offense_history) appends a final line TRUNCATED: showing 4000 of N chars when it cuts, and returns the input unchanged otherwise.
In plugins/os-sdlc/maps/poodr-implementation.yaml, the behavior-verifier error edge becomes { to: behavior-verifier, bound: 1, note: "retry once with the same facts" }. When the bound is exhausted, the existing bound-exhausted escalation applies. The map validator accepts this shape.
A new ADR amends ADR-0169 and ADR-0173: diff facts are never truncated; transcript facts are truncated with a marker.

Non-goals

No change to the verifier charter beyond what the marker requires. No change to MAX_GATE_OUTPUT_CHARS.

Owner sketch

class Pipeline::BriefFacts
  def code_changes
    diff = changes.code_diff.to_s
    diff.empty? ? nil : diff
  end

  def truncate(output)
    text = output.to_s
    return text if text.size <= MAX_GATE_OUTPUT_CHARS
    "#{text[0, MAX_GATE_OUTPUT_CHARS]}\nTRUNCATED: showing #{MAX_GATE_OUTPUT_CHARS} of #{text.size} chars"
  end
end

Cases

Given a production diff longer than 4000 characters, when BriefFacts resolves code_changes, then the whole diff is returned.
Given a test diff longer than 4000 characters, when BriefFacts resolves test_changes, then the whole diff is returned.
Given gate output longer than 4000 characters, when BriefFacts resolves gate results, then the fact ends with a line TRUNCATED: showing 4000 of N chars.
Given gate output shorter than the cap, when BriefFacts resolves gate results, then the fact has no TRUNCATED line.
Given a behavior-verifier dispatch that returns verdict error, when the Runner routes it, then the next step is behavior-verifier again, once; a second error escalates instead of terminating as implementation_failed.

VERDICT: CREATE (ticket-skeptic) — real, well-scoped defect distinct from #613/#614/#617; presentable() silently truncates diffs and behavior-verifier verdict:error has no repair route, so a green implementation was killed and had to be hand-finished; root is DESIGN in brief_facts.rb/verdict routing, echoes unticketed #528 note. Parent: #613 ## Context This surfaced in implementation 5 of #613 (target os-sdlc, 7 production files, 62 insertions). The programmer dispatch 44 went green. The behavior-verifier dispatch 45 could not judge the diff and returned an error verdict, which the Runner treated as terminal. ## Observed The behavior-verifier handoff said: "code_changes is truncated mid-diff; three of six ticket tasks cannot be judged". It listed "MISSING: complete code_changes -- the diff ends mid-hunk in the header line for plugins/os-sdlc/lib/os_sdlc/pipeline/open_implementation.rb". It returned verdict: error. ``` verdict: error ``` ``` implementation_failed, reason "verdict:error" (terminal, no resume) ``` The main loop verified the diff by hand, ran the suite green (1354 runs), and hand-finished the ticket. A green implementation failed because of a brief-side defect, not a code defect. Two defects combine to cause this: 1. `BriefFacts#presentable` truncates a diff without a visible marker or a fact stating the cap and the omitted files. The verifier has no way to ask for the rest. 2. A `verdict: error` from behavior-verifier routes straight to `implementation_failed`, with no repair or re-brief route. A brief-side defect kills a green implementation outright. This echoes an earlier note from #528: verifier error pre-empted the suite-check repair route (unticketed at the time). ## Reproduce ``` Run the ticket map on a ticket whose production diff exceeds the presentable cap, then observe the dispatch 45-style handoff. ``` ## Expected 1. `presentable` appends an explicit line, for example "TRUNCATED: N of M diff lines shown; files omitted: ...". The verifier charter instructs it to verify omitted files with Bash (`git diff` on the named paths) instead of returning error. 2. A `verdict: error` from behavior-verifier routes to one bounded re-dispatch of behavior-verifier with the full diff, or escalates. It never routes straight to `implementation_failed`. ## Origin - Trigger: /os-sdlc:implement #613, implementation 5, behavior-verifier dispatch 45. - Improvised this session: hand-verified the diff and suite, then hand-finished the ticket. - Chain: implementation_failed on a green diff ← verdict:error with no repair route ← presentable truncates the diff with no visible marker. Stop: DESIGN, plugins/os-sdlc/lib/os_sdlc/pipeline/brief_facts.rb (`presentable`) and the behavior-verifier verdict-routing map. - Root candidate: none; this is the root ticket. - Where: plugins/os-sdlc/lib/os_sdlc/pipeline/brief_facts.rb (`presentable`); behavior-verifier verdict handling in the map / routing. - Session: 4974deff-0140-4813-8cd1-e96c6356a214 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/4974deff-0140-4813-8cd1-e96c6356a214.jsonl ## Decisions 2026-09-19 1. Remove truncation from the two diff facts, `code_changes` and `test_changes`. A diff is bounded by the increment and is the verifier's only evidence. `presentable` goes away. 2. Keep the cap on the two transcript facts, `gate results` and `lint_offense_history`, and append a visible marker line when the cap cuts. 3. A `verdict: error` from behavior-verifier routes to one bounded re-dispatch of behavior-verifier, then escalates. It never routes straight to `implementation_failed`. 4. This amends ADR-0169 and ADR-0173, which state that truncation stays in the brief layer for code_changes. Record the amending ADR with /os-adr:create before the code change. ## Slice The verifier always sees the whole diff. Transcript facts that are cut say so. An error verdict from the verifier gets one retry before it can end the implementation. ## Acceptance criteria * `BriefFacts#code_changes` and `BriefFacts#test_changes` return the full diff, or nil when empty. `BriefFacts#presentable` is removed. * `BriefFacts#truncate` (used by gate results and lint_offense_history) appends a final line `TRUNCATED: showing 4000 of N chars` when it cuts, and returns the input unchanged otherwise. * In `plugins/os-sdlc/maps/poodr-implementation.yaml`, the behavior-verifier `error` edge becomes `{ to: behavior-verifier, bound: 1, note: "retry once with the same facts" }`. When the bound is exhausted, the existing bound-exhausted escalation applies. The map validator accepts this shape. * A new ADR amends ADR-0169 and ADR-0173: diff facts are never truncated; transcript facts are truncated with a marker. ## Non-goals No change to the verifier charter beyond what the marker requires. No change to MAX_GATE_OUTPUT_CHARS. ## Owner sketch class Pipeline::BriefFacts def code_changes diff = changes.code_diff.to_s diff.empty? ? nil : diff end def truncate(output) text = output.to_s return text if text.size <= MAX_GATE_OUTPUT_CHARS "#{text[0, MAX_GATE_OUTPUT_CHARS]}\nTRUNCATED: showing #{MAX_GATE_OUTPUT_CHARS} of #{text.size} chars" end end ## Cases 1. Given a production diff longer than 4000 characters, when BriefFacts resolves code_changes, then the whole diff is returned. 2. Given a test diff longer than 4000 characters, when BriefFacts resolves test_changes, then the whole diff is returned. 3. Given gate output longer than 4000 characters, when BriefFacts resolves gate results, then the fact ends with a line `TRUNCATED: showing 4000 of N chars`. 4. Given gate output shorter than the cap, when BriefFacts resolves gate results, then the fact has no TRUNCATED line. 5. Given a behavior-verifier dispatch that returns verdict error, when the Runner routes it, then the next step is behavior-verifier again, once; a second error escalates instead of terminating as implementation_failed.

jared added the

P2

plugin/os-sdlc

filed-by/agent

labels

2026-09-18 15:29:36 +00:00

jared referenced this issue

2026-09-18 15:29:50 +00:00

test-writer rewrites Retired tests into absence tests instead of deleting them; behavior-verifier passes anyway #613

jared commented

2026-09-19 17:38:43 +00:00

Author

Owner

Body updated 2026-09-19 with the design decisions (see ## Decisions 2026-09-19). Implementation via /os-sdlc:implement starts now.

jared commented

2026-09-19 21:04:58 +00:00

Author

Owner

Work started on branch ticket-618 via /os-sdlc:implement (session 33dcab60). ADR-0204 committed on the branch first.

jared commented

2026-09-19 21:31:24 +00:00

Author

Owner

Resolution

Done: BriefFacts#code_changes and #test_changes return the full diff or nil; presentable removed. truncate appends 'TRUNCATED: showing 4000 of N chars' for gate results and lint_offense_history. poodr-implementation.yaml behavior-verifier error edge is a bounded self-retry (bound 1) then bound-exhausted escalation. ADR-0204 amends 0137/0169/0173.

Evidence: Branch ticket-618 commits 757a8770 (ADR) and HEAD; pipeline implementation 8, 17 dispatches; behavior-verifier pass x3, suite-check pass (dispatch 101), contract-auditor pass (102). Hand-finish: two map invariant tests in poodr_implementation_map_test.rb amended to allow the bounded error self-edge.

Follow-ups: 1) poodr-ticket-implementation.yaml still routes behavior-verifier error to the terminal error node; the ticket named only poodr-implementation.yaml. Left as is; apply the same edge when that map is next edited. 2) LSP daemon tests lsp_integration_test and cli_resolves_root_from_subdirectory_test fail under full-suite order twice this session and pass in isolation; pre-existing, not captured here.

## Resolution **Done:** BriefFacts#code_changes and #test_changes return the full diff or nil; presentable removed. truncate appends 'TRUNCATED: showing 4000 of N chars' for gate results and lint_offense_history. poodr-implementation.yaml behavior-verifier error edge is a bounded self-retry (bound 1) then bound-exhausted escalation. ADR-0204 amends 0137/0169/0173. **Evidence:** Branch ticket-618 commits 757a8770 (ADR) and HEAD; pipeline implementation 8, 17 dispatches; behavior-verifier pass x3, suite-check pass (dispatch 101), contract-auditor pass (102). Hand-finish: two map invariant tests in poodr_implementation_map_test.rb amended to allow the bounded error self-edge. **Follow-ups:** 1) poodr-ticket-implementation.yaml still routes behavior-verifier error to the terminal error node; the ticket named only poodr-implementation.yaml. Left as is; apply the same edge when that map is next edited. 2) LSP daemon tests lsp_integration_test and cli_resolves_root_from_subdirectory_test fail under full-suite order twice this session and pass in isolation; pre-existing, not captured here.

jared closed this issue

2026-09-19 21:31:25 +00:00

jared commented

2026-09-22 17:37:16 +00:00

Author

Owner

Flaky-test note: full-suite run of client_sends_absolute_root_test.rb failed once under a random seed while verifying this merge; passed in isolation and on 3 fixed-seed reruns (1363/1371/1376 executions, 0 failures). No defect in this ticket's work. Skeptic gate verdict: DROP — unconfirmed intermittent-test suspicion with no captured seed or assertion; provenance is a guess (IMPROVISATION-strength), not an observed, reproducible failure.

jared referenced this issue from a commit

2026-09-22 18:00:00 +00:00

docs(adr): ADR-0204 diff brief facts are never truncated; transcript facts keep the cap with a visible marker (amends 0137/0169/0173) (#618)

jared referenced this issue from a commit

2026-09-22 18:00:00 +00:00

feat(os-sdlc): code_changes and test_changes are never truncated; transcript facts append a TRUNCATED marker; behavior-verifier error edge is a bounded self-retry (#618, ADR-0204)

jared referenced this issue from a commit

2026-09-22 18:00:00 +00:00

Merge branch 'ticket-618' (#618, ADR-0204)

jared referenced this issue from a commit

2026-09-22 18:00:00 +00:00

docs(status): os-sdlc leaf History rows for #616 and #618