Stop the 60s suite budget from failing bin/fast-gate on a loaded host

jared commented

2026-10-22 20:32:53 +00:00

Owner

Problem

The os-sdlc suite has a 60s wall-time budget.
On a loaded host, bin/fast-gate fails on this budget with 0 failures and 0 errors.
Each time-only failure cost the builder a wasted repair attempt, or left the change uncommitted for the orchestrator to re-gate.

Observed

Evidence from 2026-10-22, fast-batch-12.
Finding B's builder saw 60.4s, 66.7s, and 64.3s at load average about 4.5 on 12 cores, with other agents active.
The orchestrator saw 68.1s on the final worktree run.
Idle runs the same day took 44.5s to 54s.

Reproduce

n/a: the failure depends on host load, not on a fixed command.

Approach

Decided 2026-10-23: option (a). The budget stays a signal and stops being a blocker. The budget value stays 60s. No load-aware logic.
The budget is in plugins/os-sdlc/tests/all.rb:21 (BUDGET_SECONDS = 60). Its Minitest.after_run hook (lines 23-30) runs exit 1 on an overrun with 0 failures.
That exit code feeds both FastGate::SuiteCheck#outcome (fast_gate/suite_check.rb:19) and the implement pipeline suite step (StepOutcome#classify_green, step_outcome.rb:34-36, which escalates :green_report_nonzero_exit). Both stop failing on a time-only overrun. This is intended: the overrun is the same false failure in both places.
tests/all.rb prints WARN Suite wall time X (budget 60s) on an overrun and exits 0 when the run is green.
A new SuiteOutput::WallTime parses the wall-time line. It returns nil when the line is absent.
FastGate::SuiteCheck#outcome adds the wall time to its summary, with an over budget marker on an overrun, so the PASS line shows it.

Required work:

Print a WARN line on a budget overrun in the suite runner — check: present WARN Suite wall time in plugins/os-sdlc/tests/all.rb
Parse the wall-time line — check: present WallTime in plugins/os-sdlc/lib/os_sdlc/suite_output/*.rb
Show the wall time in the fast-gate suite summary — check: present WallTime in plugins/os-sdlc/lib/os_sdlc/fast_gate/suite_check.rb
Update the green_report_nonzero_exit known cause — check: absent exceeded because the machine was overloaded in plugins/os-sdlc/references/escalation-procedure.md
Update the suite wall-time budget entry — check: absent BUDGET_SECONDS = 47 in docs/implementation-status/os-sdlc.md

Rejected: (b) raise the budget — the next load spike fails again. (c) load-aware budget — more code for the same signal.

Acceptance criteria

Given a suite run with 0 failures and 0 errors whose wall time exceeds the budget, when bin/fast-gate runs, then the gate passes.
Given that same run, when it finishes, then the PASS line contains the wall time and an over budget marker.
Given suite output with no wall-time line, when SuiteOutput::WallTime parses it, then it returns nil and the gate summary has no wall-time text.

Unchanged, not an acceptance criterion: a suite run with a failing test still fails the gate (the existing nonzero-exit path; no new test).

Owner sketch

class OsSdlc::SuiteOutput::WallTime
  def self.parse(text)
  def seconds
  def budget
  def over?
  def to_s
end

class OsSdlc::FastGate::SuiteCheck
  def outcome
end

Origin

Trigger: bin/fast-gate runs during fast-batch-12 on 2026-10-22
Improvised this session: builders re-ran the gate; the orchestrator re-gated
Chain: gate fail with 0 failures <- 60s wall-time budget <- DESIGN (budget in plugins/os-sdlc/tests/all.rb:21)
Root candidate: none found; searched budget, fast-gate, wall time, suite, 60s
Where: plugins/os-sdlc/tests/all.rb, plugins/os-sdlc/lib/os_sdlc/fast_gate/suite_check.rb
Session: 3ff277d2-0942-49d9-8ac0-4e778200b1e2
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/3ff277d2-0942-49d9-8ac0-4e778200b1e2.jsonl

VERDICT: CREATE

## Problem The os-sdlc suite has a 60s wall-time budget. On a loaded host, `bin/fast-gate` fails on this budget with 0 failures and 0 errors. Each time-only failure cost the builder a wasted repair attempt, or left the change uncommitted for the orchestrator to re-gate. ## Observed Evidence from 2026-10-22, fast-batch-12. Finding B's builder saw 60.4s, 66.7s, and 64.3s at load average about 4.5 on 12 cores, with other agents active. The orchestrator saw 68.1s on the final worktree run. Idle runs the same day took 44.5s to 54s. ## Reproduce n/a: the failure depends on host load, not on a fixed command. ## Approach Decided 2026-10-23: option (a). The budget stays a signal and stops being a blocker. The budget value stays 60s. No load-aware logic. The budget is in `plugins/os-sdlc/tests/all.rb:21` (`BUDGET_SECONDS = 60`). Its `Minitest.after_run` hook (lines 23-30) runs `exit 1` on an overrun with 0 failures. That exit code feeds both `FastGate::SuiteCheck#outcome` (`fast_gate/suite_check.rb:19`) and the implement pipeline suite step (`StepOutcome#classify_green`, `step_outcome.rb:34-36`, which escalates `:green_report_nonzero_exit`). Both stop failing on a time-only overrun. This is intended: the overrun is the same false failure in both places. `tests/all.rb` prints `WARN Suite wall time X (budget 60s)` on an overrun and exits 0 when the run is green. A new `SuiteOutput::WallTime` parses the wall-time line. It returns nil when the line is absent. `FastGate::SuiteCheck#outcome` adds the wall time to its summary, with an `over budget` marker on an overrun, so the PASS line shows it. Required work: 1. Print a WARN line on a budget overrun in the suite runner — check: present `WARN Suite wall time` in `plugins/os-sdlc/tests/all.rb` 2. Parse the wall-time line — check: present `WallTime` in `plugins/os-sdlc/lib/os_sdlc/suite_output/*.rb` 3. Show the wall time in the fast-gate suite summary — check: present `WallTime` in `plugins/os-sdlc/lib/os_sdlc/fast_gate/suite_check.rb` 4. Update the green_report_nonzero_exit known cause — check: absent `exceeded because the machine was overloaded` in `plugins/os-sdlc/references/escalation-procedure.md` 5. Update the suite wall-time budget entry — check: absent `BUDGET_SECONDS = 47` in `docs/implementation-status/os-sdlc.md` Rejected: (b) raise the budget — the next load spike fails again. (c) load-aware budget — more code for the same signal. ## Acceptance criteria 1. Given a suite run with 0 failures and 0 errors whose wall time exceeds the budget, when `bin/fast-gate` runs, then the gate passes. 2. Given that same run, when it finishes, then the PASS line contains the wall time and an `over budget` marker. 3. Given suite output with no wall-time line, when `SuiteOutput::WallTime` parses it, then it returns nil and the gate summary has no wall-time text. Unchanged, not an acceptance criterion: a suite run with a failing test still fails the gate (the existing nonzero-exit path; no new test). ## Owner sketch ```ruby class OsSdlc::SuiteOutput::WallTime def self.parse(text) def seconds def budget def over? def to_s end class OsSdlc::FastGate::SuiteCheck def outcome end ``` ## Origin - Trigger: `bin/fast-gate` runs during fast-batch-12 on 2026-10-22 - Improvised this session: builders re-ran the gate; the orchestrator re-gated - Chain: gate fail with 0 failures <- 60s wall-time budget <- DESIGN (budget in `plugins/os-sdlc/tests/all.rb:21`) - Root candidate: none found; searched budget, fast-gate, wall time, suite, 60s - Where: plugins/os-sdlc/tests/all.rb, plugins/os-sdlc/lib/os_sdlc/fast_gate/suite_check.rb - Session: 3ff277d2-0942-49d9-8ac0-4e778200b1e2 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/3ff277d2-0942-49d9-8ac0-4e778200b1e2.jsonl VERDICT: CREATE

jared added the

labels

2026-10-22 20:32:53 +00:00

jared referenced this issue

2026-10-23 15:22:59 +00:00

Gate each fast-path finding on its cop and touched tests; run the full suite once per batch #1254

jared added the

ready-for-agent

label

2026-10-23 15:23:10 +00:00

jared commented

2026-10-23 15:23:32 +00:00

Author

Owner

Work started on branch ticket-1239 (os-sdlc implement pipeline).

jared commented

2026-10-23 15:37:19 +00:00

Author

Owner

Implementation 115 failed at test-reviewer dispatch 962 (verdict:error, MISSING: red_tests). Cause: after orchestrator reroute (ticket AC 3 removed — it was a no-behavior criterion that looped test-writer/test-reviewer 4x to bound_exhausted), Dispatch::Brief::Facts#red_tests read the map-router dispatch 960 step summary ({reroute: []}) instead of the last test gate. Workflow defect captured separately. Restarting with a fresh implementation-open: 3 red tests remain in plugins/os-sdlc/tests/fast_gate/suite_check_wall_time_test.rb.

jared referenced this issue

2026-10-23 15:38:23 +00:00

Supply real gate facts to test-reviewer after a reroute #1255

jared referenced this issue

2026-10-23 15:38:31 +00:00

implement skill names no map for a behavior-preserving refactor; ticket-implementation map escalates on unexpected_pass #663

jared referenced this issue

2026-10-23 16:01:41 +00:00

behavior-verifier brief omits test_changes, producing a wrong fail and an unexpected_pass escalation #1111

jared commented

2026-10-23 16:01:42 +00:00

Author

Owner

Resolution

Done: A green suite run over the 60s budget prints 'WARN Suite wall time X (budget 60s)' and exits 0; SuiteOutput::WallTime parses it; FastGate::SuiteCheck shows 'wall time Xs over budget' in its summary. The implement pipeline suite step no longer escalates a time-only overrun.

Evidence: main 606016fc (commit 0c171c40, merged). Suite 2452 runs, 0 failures, wall 50.5s; rubocop clean on 4 files; one design audit (WallTime is Data.define; AC1 test sets START_TIME back). Implementation 115 failed (ticket AC defect, then reroute defect); implementation 118 hand-finished after escalation 64 and abandoned.

Follow-ups: #1255 (reroute gives test-reviewer empty red_tests), occurrence on #663 (no-behavior AC loops the red round), occurrence on #1111 (verifier brief omits changed files). Dropped: SuiteOutput::Summary Struct to Data.define tidy (no defect observed).

## Resolution **Done:** A green suite run over the 60s budget prints 'WARN Suite wall time X (budget 60s)' and exits 0; SuiteOutput::WallTime parses it; FastGate::SuiteCheck shows 'wall time Xs over budget' in its summary. The implement pipeline suite step no longer escalates a time-only overrun. **Evidence:** main 606016fc (commit 0c171c40, merged). Suite 2452 runs, 0 failures, wall 50.5s; rubocop clean on 4 files; one design audit (WallTime is Data.define; AC1 test sets START_TIME back). Implementation 115 failed (ticket AC defect, then reroute defect); implementation 118 hand-finished after escalation 64 and abandoned. **Follow-ups:** #1255 (reroute gives test-reviewer empty red_tests), occurrence on #663 (no-behavior AC loops the red round), occurrence on #1111 (verifier brief omits changed files). Dropped: SuiteOutput::Summary Struct to Data.define tidy (no defect observed).

jared closed this issue

2026-10-23 16:01:42 +00:00

jared referenced this issue from a commit

2026-10-23 16:02:23 +00:00

feat(os-sdlc): suite budget overrun warns instead of failing the gate (#1239)

Rows
Columns

Stop the 60s suite budget from failing bin/fast-gate on a loaded host #1239

Problem

Observed

Reproduce

Approach

Acceptance criteria

Owner sketch

Origin

Resolution