Kill the 10 pass 3 survivors hidden as load timeouts #1713

New issue

Closed

opened 2026-11-12 17:56:18 +00:00 by jared · 1 comment

jared commented

2026-11-12 17:56:18 +00:00

Owner

Problem

Whole mutation pass 3 (2026-11-11, main 9a1895a1f) reported 10 mutants as timeouts. A check at 8 workers on 2026-11-12 shows that they are survivors. With 12 workers on 12 processors, each survivor ran all of its covering tests for more than 10 s, so the pass reported a timeout. No pass 3 ticket saw them. None has a ledger row in .sdlc/mutation-rulings.tsv, and none is a known visible mutant (ruling 137).

Observed

docs/research/mutation-testing/2026-11-11-third-pass/timeout_check.tsv lists each mutant with the result survived.

lib/os_sdlc/cops/structural/def_key.rb:12: 64d19d451d51 (statement_removal), abe00e636740 (condition_false)
lib/os_sdlc/cops/structural/same_file_defs/call_keys.rb:43: 0000bee9aa33 (condition_true)
lib/os_sdlc/detector/cop_sites.rb:19: 64a4ae8ecc38 (boolean_literal)
lib/os_sdlc/detector/cop_sites.rb:28: da1b7c21104e (statement_removal)
lib/os_sdlc/system_model/extractor/class_visitor.rb:24: 8808b3d94bb7 (arithmetic)
lib/os_sdlc/system_model/extractor/entry_scripts.rb:30: a4ef25fdfe8e (arithmetic)
lib/os_sdlc/system_model/extractor/forwarding.rb:18: 789a59f41a74 (statement_removal)
lib/os_sdlc/system_model/extractor/visibility.rb:13: 39ba99805709 (boolean_connector)
lib/os_sdlc/system_model/extractor/walk.rb:10: 84278eb2ccf2 (statement_removal)

Reproduce

From plugins/os-sdlc, run Mutineer on the 9 files above with all test files, the pass 3 operators (tier 1 plus condition_true, condition_false, negation_removal), --strategy redefine, --jobs 8, and --no-matrix. The 10 IDs above are survivors.

Approach

Run each file through plugins/os-sdlc/references/survivor-loop.md, as with #1702-#1708: a paired test kills the mutant, a dead-code deletion removes it (ruling 107), or a ledger row rules it with a reason that holds for every input. The packet --test list holds every covering test plus each paired test/os_sdlc/<lib path>_test.rb. The orchestrator re-runs every packet at --jobs 8 before the close.

Required work: none beyond the acceptance criteria.

Acceptance criteria

Given the 9 source files above, when the orchestrator runs their packets at --jobs 8 with the pass 3 operators, then the run shows 0 unruled survivors and none of the 10 IDs is a timeout.
Given the full suite, when it runs after the change, then it has 0 failures.

Origin

Trigger: targeted timeout check, docs/research/mutation-testing/2026-11-11-third-pass/README.md section "Timeout check at 8 workers".
Improvised this session: none
Chain: 10 survivors not ticketed ← pass 3 reported them as timeouts ← 12 workers on 12 processors made each full covering-test run exceed 10 s (ruling 112) ← IMPROVISATION: pass 3 ran at the default worker count because --jobs did not exist yet (#1701).
Root candidate: none found; searched timeout, survivor, mutation, pass 3, hidden, def_key, cop_sites, extractor
Where: lib/os_sdlc/cops/structural/def_key.rb, lib/os_sdlc/detector/cop_sites.rb, lib/os_sdlc/system_model/extractor/
Session: 01fe836c-833b-4fd9-a7f2-6982cee53d73
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/01fe836c-833b-4fd9-a7f2-6982cee53d73.jsonl

VERDICT: CREATE (ticket-skeptic, 2026-11-12; approved by ruling 140)

## Problem Whole mutation pass 3 (2026-11-11, main 9a1895a1f) reported 10 mutants as timeouts. A check at 8 workers on 2026-11-12 shows that they are survivors. With 12 workers on 12 processors, each survivor ran all of its covering tests for more than 10 s, so the pass reported a timeout. No pass 3 ticket saw them. None has a ledger row in `.sdlc/mutation-rulings.tsv`, and none is a known visible mutant (ruling 137). ## Observed `docs/research/mutation-testing/2026-11-11-third-pass/timeout_check.tsv` lists each mutant with the result `survived`. - `lib/os_sdlc/cops/structural/def_key.rb:12`: 64d19d451d51 (statement_removal), abe00e636740 (condition_false) - `lib/os_sdlc/cops/structural/same_file_defs/call_keys.rb:43`: 0000bee9aa33 (condition_true) - `lib/os_sdlc/detector/cop_sites.rb:19`: 64a4ae8ecc38 (boolean_literal) - `lib/os_sdlc/detector/cop_sites.rb:28`: da1b7c21104e (statement_removal) - `lib/os_sdlc/system_model/extractor/class_visitor.rb:24`: 8808b3d94bb7 (arithmetic) - `lib/os_sdlc/system_model/extractor/entry_scripts.rb:30`: a4ef25fdfe8e (arithmetic) - `lib/os_sdlc/system_model/extractor/forwarding.rb:18`: 789a59f41a74 (statement_removal) - `lib/os_sdlc/system_model/extractor/visibility.rb:13`: 39ba99805709 (boolean_connector) - `lib/os_sdlc/system_model/extractor/walk.rb:10`: 84278eb2ccf2 (statement_removal) ## Reproduce From `plugins/os-sdlc`, run Mutineer on the 9 files above with all test files, the pass 3 operators (tier 1 plus `condition_true`, `condition_false`, `negation_removal`), `--strategy redefine`, `--jobs 8`, and `--no-matrix`. The 10 IDs above are survivors. ## Approach Run each file through `plugins/os-sdlc/references/survivor-loop.md`, as with #1702-#1708: a paired test kills the mutant, a dead-code deletion removes it (ruling 107), or a ledger row rules it with a reason that holds for every input. The packet `--test` list holds every covering test plus each paired `test/os_sdlc/<lib path>_test.rb`. The orchestrator re-runs every packet at `--jobs 8` before the close. Required work: none beyond the acceptance criteria. ## Acceptance criteria 1. Given the 9 source files above, when the orchestrator runs their packets at `--jobs 8` with the pass 3 operators, then the run shows 0 unruled survivors and none of the 10 IDs is a timeout. 2. Given the full suite, when it runs after the change, then it has 0 failures. ## Origin - Trigger: targeted timeout check, `docs/research/mutation-testing/2026-11-11-third-pass/README.md` section "Timeout check at 8 workers". - Improvised this session: none - Chain: 10 survivors not ticketed ← pass 3 reported them as timeouts ← 12 workers on 12 processors made each full covering-test run exceed 10 s (ruling 112) ← IMPROVISATION: pass 3 ran at the default worker count because `--jobs` did not exist yet (#1701). - Root candidate: none found; searched timeout, survivor, mutation, pass 3, hidden, def_key, cop_sites, extractor - Where: lib/os_sdlc/cops/structural/def_key.rb, lib/os_sdlc/detector/cop_sites.rb, lib/os_sdlc/system_model/extractor/ - Session: 01fe836c-833b-4fd9-a7f2-6982cee53d73 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/01fe836c-833b-4fd9-a7f2-6982cee53d73.jsonl VERDICT: CREATE (ticket-skeptic, 2026-11-12; approved by ruling 140)

jared added the

plugin/os-sdlc

filed-by/agent

category/code-change

labels

2026-11-12 17:56:18 +00:00

jared commented

2026-11-12 18:31:46 +00:00

Author

Owner

Resolution

Done: 8 unruled survivors that pass 3 reported as load timeouts, plus entry_scripts.rb:35 (90a4fe6e2789), went to 0 through the survivor loop: 38 tests in the QueryMethodReachesWrite cop test, test/detector/cop_sites_test.rb, and new paired files for ClassVisitor, EntryScripts, and Visibility. No code change. The ticket said 10; #1704 had already ruled 789a59f41a74 and 84278eb2ccf2 equivalent.

Evidence: Commit b7f28465f, merged to main after main merge f0d94e9c2. Orchestrator packet re-runs on the merged branch: 0 survivors of 35 (def_key, call_keys), 0 of 10 (cop_sites), 0 unruled of 74 (extractor; 2 ledger rows). Full suite 5364 runs, 0 failures; rubocop 5 files, no offenses; sonnet test review: no blocking findings, 3 minor applied.

Follow-ups: #1714 (sort the 13 remaining timeouts) is already open. Ledger row 83 (key 9836f490e1b0) needs a new key after the git-off-process-io merge; recorded in the handoff, no new ticket.

## Resolution **Done:** 8 unruled survivors that pass 3 reported as load timeouts, plus entry_scripts.rb:35 (90a4fe6e2789), went to 0 through the survivor loop: 38 tests in the QueryMethodReachesWrite cop test, test/detector/cop_sites_test.rb, and new paired files for ClassVisitor, EntryScripts, and Visibility. No code change. The ticket said 10; #1704 had already ruled 789a59f41a74 and 84278eb2ccf2 equivalent. **Evidence:** Commit b7f28465f, merged to main after main merge f0d94e9c2. Orchestrator packet re-runs on the merged branch: 0 survivors of 35 (def_key, call_keys), 0 of 10 (cop_sites), 0 unruled of 74 (extractor; 2 ledger rows). Full suite 5364 runs, 0 failures; rubocop 5 files, no offenses; sonnet test review: no blocking findings, 3 minor applied. **Follow-ups:** #1714 (sort the 13 remaining timeouts) is already open. Ledger row 83 (key 9836f490e1b0) needs a new key after the git-off-process-io merge; recorded in the handoff, no new ticket.

jared closed this issue

2026-11-12 18:31:46 +00:00

jared referenced this issue from a commit

2026-11-12 18:31:48 +00:00

test(os-sdlc): kill the survivors that pass 3 timeouts hid (#1713)