jared/cc-os

Watch

Fork 0

Suite gate fails green suites when the machine is under load #813

New issue

Closed

opened 2026-09-28 20:30:36 +00:00 by jared · 2 comments

jared commented

2026-09-28 20:30:36 +00:00

Owner

Problem

The os-sdlc suite gate fails a green suite when the machine is under load. plugins/os-sdlc/tests/all.rb:21 exits 1 when the wall time exceeds 47 s (OS_SDLC_SUITE_BUDGET_S). Wall time depends on the other sessions that run on the same machine. The programmer-repair node then has no failure to repair, the suite-check/fail edge bound runs out, and the implementation escalates. This blocks pipeline completion whenever sessions run in parallel.

Observed

Implementation 65 (#803) on 2026-09-28. Load average was 10.4. Each suite-check recorded 1334 green runs and failed on time only:

suite-check 684: suite 52269 ms exit 1
suite-check 690: suite 68445 ms exit 1
suite-check 696: suite 59541 ms exit 1
escalation reason: bound_exhausted:suite-check/fail

1334 runs, 2900 assertions, 0 failures, 0 errors, 10 skips
Suite wall time: 49.7s (budget 47s)

#808 escalated on the same budget in another session on the same day.

Reproduce

ruby plugins/os-sdlc/tests/all.rb   # while other sessions run suites; exit 1 with 0 failures

Origin

Trigger: /os-sdlc:implement 803, implementation 65, escalation after three suite-check fails
Improvised this session: #803 hand-finished; tests green by hand
Chain: escalation ← suite-check exit 1 with 0 failures ← wall-clock budget in tests/all.rb:21 ← DESIGN #624 suite speed budget (ADR-0205)
Root candidate: this ticket
Where: plugins/os-sdlc/tests/all.rb
Session: 2ffc9dea-8187-4719-a9da-644174a8d7b8
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/2ffc9dea-8187-4719-a9da-644174a8d7b8.jsonl

## Problem The os-sdlc suite gate fails a green suite when the machine is under load. `plugins/os-sdlc/tests/all.rb:21` exits 1 when the wall time exceeds 47 s (`OS_SDLC_SUITE_BUDGET_S`). Wall time depends on the other sessions that run on the same machine. The `programmer-repair` node then has no failure to repair, the `suite-check/fail` edge bound runs out, and the implementation escalates. This blocks pipeline completion whenever sessions run in parallel. ## Observed Implementation 65 (#803) on 2026-09-28. Load average was 10.4. Each suite-check recorded 1334 green runs and failed on time only: ``` suite-check 684: suite 52269 ms exit 1 suite-check 690: suite 68445 ms exit 1 suite-check 696: suite 59541 ms exit 1 escalation reason: bound_exhausted:suite-check/fail ``` ``` 1334 runs, 2900 assertions, 0 failures, 0 errors, 10 skips Suite wall time: 49.7s (budget 47s) ``` #808 escalated on the same budget in another session on the same day. ## Reproduce ``` ruby plugins/os-sdlc/tests/all.rb # while other sessions run suites; exit 1 with 0 failures ``` ## Origin - Trigger: /os-sdlc:implement 803, implementation 65, escalation after three suite-check fails - Improvised this session: #803 hand-finished; tests green by hand - Chain: escalation ← suite-check exit 1 with 0 failures ← wall-clock budget in tests/all.rb:21 ← DESIGN #624 suite speed budget (ADR-0205) - Root candidate: this ticket - Where: plugins/os-sdlc/tests/all.rb - Session: 2ffc9dea-8187-4719-a9da-644174a8d7b8 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/2ffc9dea-8187-4719-a9da-644174a8d7b8.jsonl

jared added the

plugin/os-sdlc

filed-by/agent

labels

2026-09-28 20:30:36 +00:00

jared referenced this issue

2026-09-28 20:30:51 +00:00

Callers stop asking Project for a default target; Facts#bound_target crashed every enriching hook on shaped projects (#795 regression) #803

jared referenced this issue

2026-09-28 20:45:23 +00:00

Extract TestReport and LintReport presenters from the brief and gate parsers #808

jared referenced this issue

2026-09-28 20:50:47 +00:00

Run each Required work check: command as a deterministic required-work gate #814

jared commented

2026-09-28 21:46:02 +00:00

Author

Owner

OS_SDLC_SUITE_BUDGET_S is removed (main e122af18). plugins/os-sdlc/tests/all.rb now has BUDGET_SECONDS = 47 as a constant and reads no ENV. A fix for this ticket must not bring back an environment override.

OS_SDLC_SUITE_BUDGET_S is removed (main e122af18). plugins/os-sdlc/tests/all.rb now has BUDGET_SECONDS = 47 as a constant and reads no ENV. A fix for this ticket must not bring back an environment override.

jared referenced this issue

2026-09-29 15:55:04 +00:00

Select the presenter by outcome; escalate replies must not ask for repair #826

jared referenced this issue

2026-09-29 15:55:04 +00:00

Store the gate step classification in step_results #827

jared referenced this issue

2026-09-29 15:55:05 +00:00

Rename UnexpectedPassReason to EscalationReason #828

jared commented

2026-09-29 15:55:17 +00:00

Author

Owner

Resolution

Done: A green minitest report with a nonzero exit (wall-time overrun) now classifies as green_report_nonzero_exit and escalates at once, with no programmer-repair. The reason text is green_report_nonzero_exit:/. references/escalation-procedure.md lists the known causes: check uptime, rerun when load drops, else investigate a real regression. No ENV override.

Evidence: main f20cc962 + b4a19bfa (merged ed3b8295). Red first: step_outcome_test, gate_test (fail->escalate), cli_next_instruction_test (dispatch->escalation, 0 edge_spends). Full suite 1328 runs, 0 failures, 0 errors after merge with #802.

Follow-ups: #826 presenter selection by outcome (escalate reply still asks for repair); #827 store classification in step_results; #828 rename UnexpectedPassReason

## Resolution **Done:** A green minitest report with a nonzero exit (wall-time overrun) now classifies as green_report_nonzero_exit and escalates at once, with no programmer-repair. The reason text is green_report_nonzero_exit:<position>/<gate>. references/escalation-procedure.md lists the known causes: check uptime, rerun when load drops, else investigate a real regression. No ENV override. **Evidence:** main f20cc962 + b4a19bfa (merged ed3b8295). Red first: step_outcome_test, gate_test (fail->escalate), cli_next_instruction_test (dispatch->escalation, 0 edge_spends). Full suite 1328 runs, 0 failures, 0 errors after merge with #802. **Follow-ups:** #826 presenter selection by outcome (escalate reply still asks for repair); #827 store classification in step_results; #828 rename UnexpectedPassReason

jared closed this issue

2026-09-29 15:55:17 +00:00

jared referenced this issue from a commit

2026-09-29 16:46:31 +00:00

fix(os-sdlc): #813 green report with nonzero exit escalates at once

jared referenced this issue from a commit

2026-09-29 16:46:31 +00:00

docs: history row for #813

jared referenced this issue from a commit

2026-10-01 18:24:32 +00:00

refactor(os-sdlc): #828 rename UnexpectedPassReason to EscalationReason