Sort the 13 mutation timeouts that remain at 8 workers into hang or real-clock wait #1714

New issue

Closed

opened 2026-11-12 17:56:19 +00:00 by jared · 1 comment

jared commented

2026-11-12 17:56:19 +00:00

Owner

Problem

Whole mutation pass 3 (2026-11-11) reported 37 timeouts. A check at 8 workers on 2026-11-12 sorted 24 of them, and 13 still time out. A timeout can be a real hang, which is a kill in effect. It can also be a test that waits on the real clock, which makes the mutation run slow and can hide a survivor. Nobody has sorted these 13. Ruling 138 forbids an exclusion rule, so each one needs a reason.

Observed

docs/research/mutation-testing/2026-11-11-third-pass/timeout_check.tsv lists each mutant with the result timeout.

lib/os_sdlc/daemon/client.rb:43: 6571817b6a7f
lib/os_sdlc/daemon/client.rb:58: c515ad5878f4
lib/os_sdlc/daemon/client.rb:69: 0cf71e82bc26, 99bab1be453c, a2a492345a91
lib/os_sdlc/daemon/lsp_session/poll.rb:11: 39d00e1aaa2d, c307fedc4936
lib/os_sdlc/daemon/lsp_session/symbols_request.rb:11: f8ad81143532
lib/os_sdlc/daemon/lsp_session/transport.rb:21: 01d0404efc12, beac02c8fd0d
lib/os_sdlc/daemon/termination.rb:12: 9fc3e4b6ddac
lib/os_sdlc/cli/invocation.rb:49: c3d42a93c1cc
lib/os_sdlc/yaml_file.rb:9: c9b56d42c797

Example of a real hang: daemon/termination.rb:45 runs sleep(@poll_interval) while alive?(pid) && Time.now < deadline. With @poll_interval = nil, sleep(nil) never returns.

Reproduce

From plugins/os-sdlc, run Mutineer on the 7 files above with all test files, the pass 3 operators, --strategy redefine, --jobs 8, and --no-matrix. The 13 IDs above are timeouts.

Approach

For each mutant, apply it by hand and run its covering tests with a stack dump at the hang. Sort it:

A real hang (an infinite loop or sleep(nil) that any caller would hit): no change. Record the reason in the pass 3 record.
A test that waits on the real clock (a deadline, a retry, a poll interval, IO.select with a time limit): change the test to inject the clock or the interval, so the mutant fails fast. Then the mutant is killed or survives, and a survivor goes through references/survivor-loop.md.
Neither: record the cause found.

No ignore list, no baseline, and no per-test time limit (rulings 92, 106, 138).

Required work: none beyond the acceptance criteria.

Acceptance criteria

Given the 13 mutants above, when the work is done, then the pass 3 record gives each one a sort result and its reason.
Given each mutant sorted as a real-clock wait, when the orchestrator runs its packet at --jobs 8, then the mutant is killed or ruled, not a timeout.
Given the full suite, when it runs after the change, then it has 0 failures and no test sleeps on the real clock for a mutant sorted as a real-clock wait.

Origin

Trigger: targeted timeout check, docs/research/mutation-testing/2026-11-11-third-pass/README.md section "Timeout check at 8 workers".
Improvised this session: none
Chain: 13 unsorted timeouts ← Mutineer stops each at 10 s and leaves it out of the score ← no step of the survivor loop sorts a timeout (DESIGN: plugins/os-sdlc/references/survivor-loop.md).
Root candidate: none found; searched timeout, daemon, client, lsp_session, termination, yaml_file, mutation
Where: lib/os_sdlc/daemon/, lib/os_sdlc/cli/invocation.rb, lib/os_sdlc/yaml_file.rb
Session: 01fe836c-833b-4fd9-a7f2-6982cee53d73
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/01fe836c-833b-4fd9-a7f2-6982cee53d73.jsonl

VERDICT: CREATE (ticket-skeptic, 2026-11-12; approved by ruling 140)

## Problem Whole mutation pass 3 (2026-11-11) reported 37 timeouts. A check at 8 workers on 2026-11-12 sorted 24 of them, and 13 still time out. A timeout can be a real hang, which is a kill in effect. It can also be a test that waits on the real clock, which makes the mutation run slow and can hide a survivor. Nobody has sorted these 13. Ruling 138 forbids an exclusion rule, so each one needs a reason. ## Observed `docs/research/mutation-testing/2026-11-11-third-pass/timeout_check.tsv` lists each mutant with the result `timeout`. - `lib/os_sdlc/daemon/client.rb:43`: 6571817b6a7f - `lib/os_sdlc/daemon/client.rb:58`: c515ad5878f4 - `lib/os_sdlc/daemon/client.rb:69`: 0cf71e82bc26, 99bab1be453c, a2a492345a91 - `lib/os_sdlc/daemon/lsp_session/poll.rb:11`: 39d00e1aaa2d, c307fedc4936 - `lib/os_sdlc/daemon/lsp_session/symbols_request.rb:11`: f8ad81143532 - `lib/os_sdlc/daemon/lsp_session/transport.rb:21`: 01d0404efc12, beac02c8fd0d - `lib/os_sdlc/daemon/termination.rb:12`: 9fc3e4b6ddac - `lib/os_sdlc/cli/invocation.rb:49`: c3d42a93c1cc - `lib/os_sdlc/yaml_file.rb:9`: c9b56d42c797 Example of a real hang: `daemon/termination.rb:45` runs `sleep(@poll_interval) while alive?(pid) && Time.now < deadline`. With `@poll_interval = nil`, `sleep(nil)` never returns. ## Reproduce From `plugins/os-sdlc`, run Mutineer on the 7 files above with all test files, the pass 3 operators, `--strategy redefine`, `--jobs 8`, and `--no-matrix`. The 13 IDs above are timeouts. ## Approach For each mutant, apply it by hand and run its covering tests with a stack dump at the hang. Sort it: - A real hang (an infinite loop or `sleep(nil)` that any caller would hit): no change. Record the reason in the pass 3 record. - A test that waits on the real clock (a deadline, a retry, a poll interval, `IO.select` with a time limit): change the test to inject the clock or the interval, so the mutant fails fast. Then the mutant is killed or survives, and a survivor goes through `references/survivor-loop.md`. - Neither: record the cause found. No ignore list, no baseline, and no per-test time limit (rulings 92, 106, 138). Required work: none beyond the acceptance criteria. ## Acceptance criteria 1. Given the 13 mutants above, when the work is done, then the pass 3 record gives each one a sort result and its reason. 2. Given each mutant sorted as a real-clock wait, when the orchestrator runs its packet at `--jobs 8`, then the mutant is killed or ruled, not a timeout. 3. Given the full suite, when it runs after the change, then it has 0 failures and no test sleeps on the real clock for a mutant sorted as a real-clock wait. ## Origin - Trigger: targeted timeout check, `docs/research/mutation-testing/2026-11-11-third-pass/README.md` section "Timeout check at 8 workers". - Improvised this session: none - Chain: 13 unsorted timeouts ← Mutineer stops each at 10 s and leaves it out of the score ← no step of the survivor loop sorts a timeout (DESIGN: `plugins/os-sdlc/references/survivor-loop.md`). - Root candidate: none found; searched timeout, daemon, client, lsp_session, termination, yaml_file, mutation - Where: lib/os_sdlc/daemon/, lib/os_sdlc/cli/invocation.rb, lib/os_sdlc/yaml_file.rb - Session: 01fe836c-833b-4fd9-a7f2-6982cee53d73 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/01fe836c-833b-4fd9-a7f2-6982cee53d73.jsonl VERDICT: CREATE (ticket-skeptic, 2026-11-12; approved by ruling 140)

jared added the

plugin/os-sdlc

filed-by/agent

category/investigation

labels

2026-11-12 17:56:19 +00:00

jared referenced this issue

2026-11-12 18:31:46 +00:00

Kill the 10 pass 3 survivors hidden as load timeouts #1713

jared commented

2026-11-12 18:52:17 +00:00

Author

Owner

Resolution

Done: Sorted the 13 pass 3 timeouts that remain at 8 workers: 9 hangs (no action), 2 real-clock mutants on poll.rb:11 (tests no longer sleep on the real interval; killed in about 1 s), 2 hidden survivors (invocation.rb print_result guard deleted, ruling 107; yaml_file.rb tests added).

Evidence: Commit 63b1b30ef merged to main. Table in docs/research/mutation-testing/2026-11-11-third-pass/README.md. Packets: poll.rb 0 of 15, invocation.rb + yaml_file.rb 0 of 31, no timeouts. Suite 5,369 runs, 0 failures, 7.5 s; rubocop clean.

Follow-ups: none

## Resolution **Done:** Sorted the 13 pass 3 timeouts that remain at 8 workers: 9 hangs (no action), 2 real-clock mutants on poll.rb:11 (tests no longer sleep on the real interval; killed in about 1 s), 2 hidden survivors (invocation.rb print_result guard deleted, ruling 107; yaml_file.rb tests added). **Evidence:** Commit 63b1b30ef merged to main. Table in docs/research/mutation-testing/2026-11-11-third-pass/README.md. Packets: poll.rb 0 of 15, invocation.rb + yaml_file.rb 0 of 31, no timeouts. Suite 5,369 runs, 0 failures, 7.5 s; rubocop clean. **Follow-ups:** none

jared closed this issue

2026-11-12 18:52:17 +00:00

jared referenced this issue from a commit

2026-11-12 18:52:20 +00:00

fix(os-sdlc): sort the pass 3 timeouts at 8 workers (#1714)