Cover the 388 mutants that no test runs #1683

Closed
opened 2026-11-10 21:58:49 +00:00 by jared · 7 comments
Owner

Problem

The second whole-codebase mutation pass found 388 mutants that no test covers. The survivor tickets #1656-#1664 cover only survived mutants, so no ticket covers these. An uncovered mutant is code that the suite never runs.

Observed

The report plugins/os-sdlc/.mutineer/pass.json (2026-11-09) has this summary: total 15813, killed 13439, survived 1619, no_coverage 388, timeout 235, no_verdict 247, score 89.2. Uncovered mutants by top directory under lib/os_sdlc/: daemon 181, system_model 43, cops 33, lint 30, implementation 18, crc 15, daemon.rb 10, gate 9, evidence 6, boot 5, detector 5, project_shell.rb 5, others fewer.

On 2026-11-10, the #1658 targeted run over the daemon files still had 45 uncovered mutants: daemon.rb 10, daemon/lsp_session.rb 17, daemon/navigation.rb 13, daemon/lsp_session/handshake.rb 4, daemon/client.rb 1. Most of this code talks to the ruby-lsp process. Ruling 88 says such a test needs an adapter and a recorded fixture, not a real process.

Reproduce

cd plugins/os-sdlc && ruby -rjson -e 'p JSON.parse(File.read(".mutineer/pass.json"))["no_coverage"].size'

Approach

Split the 388 uncovered mutants into tickets by concept, as #1656-#1664 split the survivors. For each concept, write the paired unit tests that run the uncovered code (ruling 100). Where the code calls a process, add or reuse the adapter and its recorded fixture (ruling 88). Delete code that no caller reaches (ruling 107).

Required work: none beyond the acceptance criteria.

Acceptance criteria

  1. Given a targeted --no-matrix Mutineer run on the files of each child ticket, when the run ends, then it reports no uncovered mutant, except code that the ticket deletes or rules unreachable with a reason.

Origin

  • Trigger: #1658 survivor loop verify run, 2026-11-10
  • Improvised this session: none
  • Chain: 45 uncovered daemon mutants after #1658 ← survivor tickets #1656-#1664 take only survived mutants ← the second pass split only survivors into tickets ← DESIGN (docs/research/mutation-testing/2026-11-09-second-pass/)
  • Root candidate: none found; searched uncovered, no_coverage, no test runs, mutant, mutation, survivor
  • Where: plugins/os-sdlc/lib/os_sdlc/daemon/
  • Session: e04dcfb2-d544-497a-9020-55066ed00123
  • Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/e04dcfb2-d544-497a-9020-55066ed00123.jsonl

VERDICT: CREATE (ticket-skeptic, 2026-11-10): no open ticket covers the 388 uncovered mutants.

## Problem The second whole-codebase mutation pass found 388 mutants that no test covers. The survivor tickets #1656-#1664 cover only survived mutants, so no ticket covers these. An uncovered mutant is code that the suite never runs. ## Observed The report `plugins/os-sdlc/.mutineer/pass.json` (2026-11-09) has this summary: `total 15813, killed 13439, survived 1619, no_coverage 388, timeout 235, no_verdict 247, score 89.2`. Uncovered mutants by top directory under `lib/os_sdlc/`: `daemon` 181, `system_model` 43, `cops` 33, `lint` 30, `implementation` 18, `crc` 15, `daemon.rb` 10, `gate` 9, `evidence` 6, `boot` 5, `detector` 5, `project_shell.rb` 5, others fewer. On 2026-11-10, the #1658 targeted run over the daemon files still had 45 uncovered mutants: `daemon.rb` 10, `daemon/lsp_session.rb` 17, `daemon/navigation.rb` 13, `daemon/lsp_session/handshake.rb` 4, `daemon/client.rb` 1. Most of this code talks to the ruby-lsp process. Ruling 88 says such a test needs an adapter and a recorded fixture, not a real process. ## Reproduce ``` cd plugins/os-sdlc && ruby -rjson -e 'p JSON.parse(File.read(".mutineer/pass.json"))["no_coverage"].size' ``` ## Approach Split the 388 uncovered mutants into tickets by concept, as #1656-#1664 split the survivors. For each concept, write the paired unit tests that run the uncovered code (ruling 100). Where the code calls a process, add or reuse the adapter and its recorded fixture (ruling 88). Delete code that no caller reaches (ruling 107). Required work: none beyond the acceptance criteria. ## Acceptance criteria 1. Given a targeted `--no-matrix` Mutineer run on the files of each child ticket, when the run ends, then it reports no uncovered mutant, except code that the ticket deletes or rules unreachable with a reason. ## Origin - Trigger: #1658 survivor loop verify run, 2026-11-10 - Improvised this session: none - Chain: 45 uncovered daemon mutants after #1658 ← survivor tickets #1656-#1664 take only survived mutants ← the second pass split only `survivors` into tickets ← DESIGN (`docs/research/mutation-testing/2026-11-09-second-pass/`) - Root candidate: none found; searched uncovered, no_coverage, no test runs, mutant, mutation, survivor - Where: `plugins/os-sdlc/lib/os_sdlc/daemon/` - Session: e04dcfb2-d544-497a-9020-55066ed00123 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/e04dcfb2-d544-497a-9020-55066ed00123.jsonl VERDICT: CREATE (ticket-skeptic, 2026-11-10): no open ticket covers the 388 uncovered mutants.
Author
Owner

Line item from #1691: the Db.bind! freeze mutant at lib/os_sdlc/db.rb:25 (statement_removal survived; condition mutants ran_at_load). Cause: test/mutation_prelude.rb:9 binds the database once with freeze: false before any mutant, and no test observes freeze_descendants. Kill it with a test against a separate model root, or rule it equivalent with that reason.

Line item from #1691: the Db.bind! freeze mutant at lib/os_sdlc/db.rb:25 (statement_removal survived; condition mutants ran_at_load). Cause: test/mutation_prelude.rb:9 binds the database once with freeze: false before any mutant, and no test observes freeze_descendants. Kill it with a test against a separate model root, or rule it equivalent with that reason.
Author
Owner

The split into child tickets is done. The 7 children cover all 385 uncovered mutants (388 in the pass, minus 3 in lib/os_sdlc/boot/inputs/detector_argv.rb, which #1646 deleted).

  • #1692 Cover the uncovered daemon server mutants: 94
  • #1693 Cover the uncovered daemon LSP session mutants with a recorded ruby-lsp fixture: 97
  • #1694 Cover the uncovered system_model mutants: 43
  • #1695 Cover the uncovered cops mutants: 33
  • #1696 Cover the uncovered lint and shell mutants: 35
  • #1697 Cover the uncovered implementation and crc mutants: 33
  • #1698 Cover the uncovered mutants in the small directories: 50 (also takes the Db.bind! freeze survivor at lib/os_sdlc/db.rb:25)
The split into child tickets is done. The 7 children cover all 385 uncovered mutants (388 in the pass, minus 3 in `lib/os_sdlc/boot/inputs/detector_argv.rb`, which #1646 deleted). - #1692 Cover the uncovered daemon server mutants: 94 - #1693 Cover the uncovered daemon LSP session mutants with a recorded ruby-lsp fixture: 97 - #1694 Cover the uncovered system_model mutants: 43 - #1695 Cover the uncovered cops mutants: 33 - #1696 Cover the uncovered lint and shell mutants: 35 - #1697 Cover the uncovered implementation and crc mutants: 33 - #1698 Cover the uncovered mutants in the small directories: 50 (also takes the `Db.bind!` freeze survivor at `lib/os_sdlc/db.rb:25`)
Author
Owner

Known uncovered mutants found by #1694 (ruling 137): (1) ten Recorder::Trace mutants (trace.rb lines 21-32) run inside a TracePoint callback; Ruby records no line coverage there, so Mutineer runs no test, but applied by hand each one fails test/os_sdlc/system_model/recorder/trace_test.rb. (2) ChildProcess#call (child_process.rb:8) and PatternSite::ExampleRun#printed (example_run.rb:10) are adapter process calls that ruling 88 forbids a test to run. They are not equivalent and stay visible as known (ruling 106 shape). #1693 will meet case (2) for the ruby-lsp adapter; apply the same treatment. A separate ticket for a ledger verdict was dropped by ticket-skeptic (depth-1).

Known uncovered mutants found by #1694 (ruling 137): (1) ten Recorder::Trace mutants (trace.rb lines 21-32) run inside a TracePoint callback; Ruby records no line coverage there, so Mutineer runs no test, but applied by hand each one fails test/os_sdlc/system_model/recorder/trace_test.rb. (2) ChildProcess#call (child_process.rb:8) and PatternSite::ExampleRun#printed (example_run.rb:10) are adapter process calls that ruling 88 forbids a test to run. They are not equivalent and stay visible as known (ruling 106 shape). #1693 will meet case (2) for the ruby-lsp adapter; apply the same treatment. A separate ticket for a ledger verdict was dropped by ticket-skeptic (depth-1).
Author
Owner

#1698 adds 5 known adapter process-call mutants (ruling 137): e1453baf79c2 GitRepositories#temporary_checkout; b298a7822ebd, b929cba6a296 Evidence::CodeSearch#capture; e7d42941b0b6, 6d638c75dd48 Tracker::IssueSource::Shell#capture.

#1698 adds 5 known adapter process-call mutants (ruling 137): e1453baf79c2 GitRepositories#temporary_checkout; b298a7822ebd, b929cba6a296 Evidence::CodeSearch#capture; e7d42941b0b6, 6d638c75dd48 Tracker::IssueSource::Shell#capture.
Author
Owner

#1696 adds 6 known adapter Open3-call mutants (ruling 137): ff621c465379, 28d2ed668ddc Lint::RubocopProcess#capture; b9dcc73d08e9, ef9aa24cd1b3 ProjectShell#capture3; d95a763df127, d21a9fb9b25f ProjectShell#capture2e.

#1696 adds 6 known adapter Open3-call mutants (ruling 137): ff621c465379, 28d2ed668ddc Lint::RubocopProcess#capture; b9dcc73d08e9, ef9aa24cd1b3 ProjectShell#capture3; d95a763df127, d21a9fb9b25f ProjectShell#capture2e.
Author
Owner

#1693 adds 3 known adapter Open3-call mutants (ruling 137): 6814b5f00d46, cfc15075e137, 79fe1bc39584 in Daemon::LspSession::ServerProcess#start. All seven children (#1692-#1698) are now closed.

#1693 adds 3 known adapter Open3-call mutants (ruling 137): 6814b5f00d46, cfc15075e137, 79fe1bc39584 in Daemon::LspSession::ServerProcess#start. All seven children (#1692-#1698) are now closed.
Author
Owner

Resolution

Done: Split the 388 uncovered mutants into 7 concept tickets (#1692 daemon server 94, #1693 daemon LSP 97, #1694 system_model 43, #1695 cops 33, #1696 lint and shell 35, #1697 implementation and crc 33, #1698 small directories 50) = 385; the other 3 were in detector_argv.rb, deleted in #1646. All 7 are closed: each ran the survivor loop with paired tests (ruling 100), deleted dead code (ruling 107), and recorded equivalent rulings with reasons. #1693 added the recorded ruby-lsp fixture and the LspSession::ServerProcess adapter (ruling 88).

Evidence: Each child's close comment holds the orchestrator's re-run packet headings. main 4b94bd14b, pushed. Final suite 5046 runs, 0 failures (#1693). Ruling 137 (2026-11-11) came from this work: a mutant Mutineer cannot run is never ruled equivalent.

Follow-ups: none. Known visible mutants on real process calls (ruling 137) are listed in the comments above: 5 from #1698, 6 from #1696, 3 from #1693, plus the #1694 TracePoint and continuation-line mutants. The next whole pass, which needs Jared, will show the true uncovered count.

## Resolution **Done:** Split the 388 uncovered mutants into 7 concept tickets (#1692 daemon server 94, #1693 daemon LSP 97, #1694 system_model 43, #1695 cops 33, #1696 lint and shell 35, #1697 implementation and crc 33, #1698 small directories 50) = 385; the other 3 were in detector_argv.rb, deleted in #1646. All 7 are closed: each ran the survivor loop with paired tests (ruling 100), deleted dead code (ruling 107), and recorded equivalent rulings with reasons. #1693 added the recorded ruby-lsp fixture and the LspSession::ServerProcess adapter (ruling 88). **Evidence:** Each child's close comment holds the orchestrator's re-run packet headings. main 4b94bd14b, pushed. Final suite 5046 runs, 0 failures (#1693). Ruling 137 (2026-11-11) came from this work: a mutant Mutineer cannot run is never ruled equivalent. **Follow-ups:** none. Known visible mutants on real process calls (ruling 137) are listed in the comments above: 5 from #1698, 6 from #1696, 3 from #1693, plus the #1694 TracePoint and continuation-line mutants. The next whole pass, which needs Jared, will show the true uncovered count. <!-- os-backlog kind: resolution -->
jared closed this issue 2026-11-11 18:03:19 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
jared/cc-os#1683
No description provided.