os-sdlc: MutationTest objects and os-sdlc-survivors command (#1638 part B)

jared commented

2026-11-08 15:58:27 +00:00

Owner

Problem

os-sdlc cannot tell which mutants on a ticket's changed lines survive the ticket's tests. A surviving mutant shows a missing test or dead code.

Approach

Design: docs/system-design-format/proof/2026-11-08-os-sdlc-mutation-survivors.md, sections 2 to 6, build slices 3 to 5. Parent: #1638. The design's "Rules to keep" table is binding.

MutationTest depends on two roles: a runner (survivors(scope) returns MutationTest::Survivors or raises MutationTest::RunError) and a switch (enabled?). It never names Mutineer, a shell, or a file. MutationTest::MutineerRunner is the only object that knows the Mutineer argv, JSON schema, and operator names. Tests of MutationTest use a fake runner.

Required work:

Add MutationTest with outcome — check: present def outcome in plugins/os-sdlc/lib/os_sdlc/mutation_test.rb
Add MutationTest::Scope, MutationTest::Outcome (to_json and self.parse own the step contract), MutationTest::Survivors, MutationTest::Mutant, MutationTest::RunError (a Loud::Error), and MutationTest::Config — check: present def self.parse in plugins/os-sdlc/lib/os_sdlc/mutation_test/outcome.rb
Add MutationTest::MutineerRunner. It runs bundle exec mutineer run <sources> --test <t>... --since <ref> --no-matrix --format json --output .mutineer/gate-<id>.json from the target root, reads the JSON file, and folds each boolean_literal row into a statement_removal row with the same file, subject, and line whose token contains it — check: present --no-matrix in plugins/os-sdlc/lib/os_sdlc/mutation_test/**/*.rb
Add Implementation#mutation_test(root). It asks Project::BaseBranch#merge_base and Project::Changes#since, and injects the Mutineer runner and the config switch — check: present def mutation_test in plugins/os-sdlc/lib/os_sdlc/**/*.rb
Add the controller MutationTest::Check, the Boot table entry survivors (parse the id with Inputs.implementation_check("survivors"), as the mutation-check entry does), and bin/os-sdlc-survivors (the one-line entry script shape from CLAUDE.md "Hook contract"). Check returns exit 1 itself for survivors; it never raises to signal survivors. Only a raised error reaches ProcessIO#guarded and exit 3 — check: present "survivors" in plugins/os-sdlc/lib/os_sdlc/boot/commands.rb

Acceptance criteria

Given a target with no .mutineer.yml, when the outcome is asked, then it is skipped with reason no_config, the runner is not asked, and os-sdlc-survivors exits 0.
Given no changed Ruby code in the target, when the outcome is asked, then it is skipped with reason no_changed_code and the runner is not asked.
Given changed code with no mapped test file, when the outcome is asked, then it is skipped with reason no_tests, it lists the untested files, and the runner is not asked.
Given a runner that returns no survivors, when os-sdlc-survivors runs, then it prints the outcome JSON and exits 0.
Given a runner that returns survivors, when os-sdlc-survivors runs, then it prints one row per survivor with id, file, line, kind, subject, and diff, and exits 1.
Given a runner that raises MutationTest::RunError, when os-sdlc-survivors runs, then it writes one os-sdlc- stderr line that carries the runner's error text and exits 3.
Given a Mutineer report with a boolean_literal and a statement_removal survivor on the same file, subject, and line where the statement token contains the literal token, when the Mutineer runner reads it, then only the statement_removal row is returned.
Given a Mutineer process that exits 1 or 2, when the Mutineer runner runs it, then it raises MutationTest::RunError carrying Mutineer's stderr.
Given Mutineer exits 0 with survivors in the JSON report, when the Mutineer runner runs it, then it returns those survivors (exit 0 does not mean none).
Given a target with .mutineer.yml whose bundle has no mutineer gem, when the Mutineer runner runs, then it raises MutationTest::RunError; it does not skip.
Given an outcome written with to_json, when MutationTest::Outcome.parse reads it, then it returns an equal outcome.

Owner sketch

class MutationTest
  def initialize(switch:, runner:, scope:)
  def outcome
end

class MutationTest::MutineerRunner
  def initialize(root)
  def survivors(scope)
end

class MutationTest::Outcome
  def self.skipped(reason, untested: [])
  def self.parse(json)
  def failed?
  def survivors
  def to_json(*)
end

class Implementation
  def mutation_test(root)
end

Notes from blast radius

ProjectShell runs with the unbundled environment, so bundle exec mutineer uses the target's own Gemfile. Only a target whose Gemfile has mutineer can run the gate.
Implementation#mutation_test and Implementation::MutationCheck are different checks. Keep the names apart in tests and messages.

Blocking edges

Blocked by #1639.

Origin

Trigger: domain-object design docs/system-design-format/proof/2026-11-08-os-sdlc-mutation-survivors.md (commit f0e172b7) for parent #1638
Improvised this session: none
Chain: survivor gate needs one owner per fact ← design section 3 ← DESIGN (docs/system-design-format/proof/2026-11-08-os-sdlc-mutation-survivors.md)
Root candidate: none found; searched mutation, mutineer, mutant, survivor, merge_base, changes since
Where: lib/os_sdlc/mutation_test.rb (new)
Session: ff703e34-8046-4b24-af68-6ef76d8aa338
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/ff703e34-8046-4b24-af68-6ef76d8aa338.jsonl

VERDICT: CREATE

## Problem os-sdlc cannot tell which mutants on a ticket's changed lines survive the ticket's tests. A surviving mutant shows a missing test or dead code. ## Approach Design: `docs/system-design-format/proof/2026-11-08-os-sdlc-mutation-survivors.md`, sections 2 to 6, build slices 3 to 5. Parent: #1638. The design's "Rules to keep" table is binding. `MutationTest` depends on two roles: a runner (`survivors(scope)` returns `MutationTest::Survivors` or raises `MutationTest::RunError`) and a switch (`enabled?`). It never names Mutineer, a shell, or a file. `MutationTest::MutineerRunner` is the only object that knows the Mutineer argv, JSON schema, and operator names. Tests of `MutationTest` use a fake runner. Required work: 1. Add `MutationTest` with `outcome` — check: present `def outcome` in `plugins/os-sdlc/lib/os_sdlc/mutation_test.rb` 2. Add `MutationTest::Scope`, `MutationTest::Outcome` (`to_json` and `self.parse` own the step contract), `MutationTest::Survivors`, `MutationTest::Mutant`, `MutationTest::RunError` (a `Loud::Error`), and `MutationTest::Config` — check: present `def self.parse` in `plugins/os-sdlc/lib/os_sdlc/mutation_test/outcome.rb` 3. Add `MutationTest::MutineerRunner`. It runs `bundle exec mutineer run <sources> --test <t>... --since <ref> --no-matrix --format json --output .mutineer/gate-<id>.json` from the target root, reads the JSON file, and folds each `boolean_literal` row into a `statement_removal` row with the same file, subject, and line whose token contains it — check: present `--no-matrix` in `plugins/os-sdlc/lib/os_sdlc/mutation_test/**/*.rb` 4. Add `Implementation#mutation_test(root)`. It asks `Project::BaseBranch#merge_base` and `Project::Changes#since`, and injects the Mutineer runner and the config switch — check: present `def mutation_test` in `plugins/os-sdlc/lib/os_sdlc/**/*.rb` 5. Add the controller `MutationTest::Check`, the Boot table entry `survivors` (parse the id with `Inputs.implementation_check("survivors")`, as the `mutation-check` entry does), and `bin/os-sdlc-survivors` (the one-line entry script shape from CLAUDE.md "Hook contract"). `Check` returns exit 1 itself for survivors; it never raises to signal survivors. Only a raised error reaches `ProcessIO#guarded` and exit 3 — check: present `"survivors"` in `plugins/os-sdlc/lib/os_sdlc/boot/commands.rb` ## Acceptance criteria 1. Given a target with no `.mutineer.yml`, when the outcome is asked, then it is skipped with reason `no_config`, the runner is not asked, and `os-sdlc-survivors` exits 0. 2. Given no changed Ruby code in the target, when the outcome is asked, then it is skipped with reason `no_changed_code` and the runner is not asked. 3. Given changed code with no mapped test file, when the outcome is asked, then it is skipped with reason `no_tests`, it lists the untested files, and the runner is not asked. 4. Given a runner that returns no survivors, when `os-sdlc-survivors` runs, then it prints the outcome JSON and exits 0. 5. Given a runner that returns survivors, when `os-sdlc-survivors` runs, then it prints one row per survivor with id, file, line, kind, subject, and diff, and exits 1. 6. Given a runner that raises `MutationTest::RunError`, when `os-sdlc-survivors` runs, then it writes one `os-sdlc-` stderr line that carries the runner's error text and exits 3. 7. Given a Mutineer report with a `boolean_literal` and a `statement_removal` survivor on the same file, subject, and line where the statement token contains the literal token, when the Mutineer runner reads it, then only the `statement_removal` row is returned. 8. Given a Mutineer process that exits 1 or 2, when the Mutineer runner runs it, then it raises `MutationTest::RunError` carrying Mutineer's stderr. 9. Given Mutineer exits 0 with survivors in the JSON report, when the Mutineer runner runs it, then it returns those survivors (exit 0 does not mean none). 10. Given a target with `.mutineer.yml` whose bundle has no `mutineer` gem, when the Mutineer runner runs, then it raises `MutationTest::RunError`; it does not skip. 11. Given an outcome written with `to_json`, when `MutationTest::Outcome.parse` reads it, then it returns an equal outcome. ## Owner sketch ```ruby class MutationTest def initialize(switch:, runner:, scope:) def outcome end class MutationTest::MutineerRunner def initialize(root) def survivors(scope) end class MutationTest::Outcome def self.skipped(reason, untested: []) def self.parse(json) def failed? def survivors def to_json(*) end class Implementation def mutation_test(root) end ``` ## Notes from blast radius - `ProjectShell` runs with the unbundled environment, so `bundle exec mutineer` uses the target's own Gemfile. Only a target whose Gemfile has `mutineer` can run the gate. - `Implementation#mutation_test` and `Implementation::MutationCheck` are different checks. Keep the names apart in tests and messages. ## Blocking edges Blocked by #1639. ## Origin - Trigger: domain-object design `docs/system-design-format/proof/2026-11-08-os-sdlc-mutation-survivors.md` (commit f0e172b7) for parent #1638 - Improvised this session: none - Chain: survivor gate needs one owner per fact ← design section 3 ← DESIGN (`docs/system-design-format/proof/2026-11-08-os-sdlc-mutation-survivors.md`) - Root candidate: none found; searched mutation, mutineer, mutant, survivor, merge_base, changes since - Where: lib/os_sdlc/mutation_test.rb (new) - Session: ff703e34-8046-4b24-af68-6ef76d8aa338 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/ff703e34-8046-4b24-af68-6ef76d8aa338.jsonl VERDICT: CREATE

jared added the

labels

2026-11-08 15:58:27 +00:00

jared commented

2026-11-08 15:58:38 +00:00

Author

Owner

Blocked on #1639 (Project::BaseBranch#merge_base and Project::Changes#since).

jared referenced this issue

2026-11-08 15:58:39 +00:00

os-sdlc: per-ticket mutation survivor gate (mutation testing increment 1) #1638

jared referenced this issue

2026-11-08 16:01:00 +00:00

os-sdlc: survivors gate step and slow-step warning (#1638 part C) #1641

jared referenced this issue

2026-11-08 16:01:03 +00:00

os-sdlc: survivors gate step and slow-step warning (#1638 part C) #1641

jared referenced this issue

2026-11-08 16:09:20 +00:00

os-sdlc: one owner for the base commit and changes since it (#1638 part A) #1639

jared commented

2026-11-08 16:10:21 +00:00

Author

Owner

Unblocked: #1639 merged (Project::BaseBranch#merge_base, Project::Changes#since).

jared commented

2026-11-08 16:10:34 +00:00

Author

Owner

Work started on branch ticket-1640 (worktree .claude/worktrees/ticket-1640) through /os-sdlc:implement.

jared commented

2026-11-08 16:45:17 +00:00

Author

Owner

Design-audit note (skeptic DROP, need-to-exist): MutationTest::Scope.from makes test paths relative to the target, and Target#relative passes an outside path through unchanged. A target with a sibling test_path (path: lib, test_path: tests) would give Mutineer wrong test paths. No project uses that configuration, so no ticket. File it when such a target is onboarded.

jared commented

2026-11-08 16:45:18 +00:00

Author

Owner

Resolution

Done: MutationTest (switch, runner, scope roles), MutineerRunner (argv, JSON report, boolean_literal fold), Implementation#mutation_test wired to Project::BaseBranch and Project::Changes#since, os-sdlc-survivors CLI. Post-run fixes: Mutineer runs in the target directory with target-relative paths (1596f6ba); Project::Target#directory owns the target's absolute directory (bd83e03e).

Evidence: Implementation 264 implementation_complete (21 dispatches, 25 min 20 s); commits 09ad8f39, 1596f6ba, bd83e03e on ticket-1640. Full suite 3528 runs, 0 failures (61.9 s). Rubocop clean. Design audit of 1596f6ba: 4 findings, 1 fixed, 1 removed by that fix, 2 answered. Real run on plugins/os-sdlc: survivors abfd2fb825c1 (return_nil) and 559f08c32fd1 (statement_removal) at lib/os_sdlc/offense_directive.rb:28.

Follow-ups: #1641 (survivors gate step) is unblocked. Sibling test_path note: dropped by ticket-skeptic, recorded as a comment on this ticket.

## Resolution **Done:** MutationTest (switch, runner, scope roles), MutineerRunner (argv, JSON report, boolean_literal fold), Implementation#mutation_test wired to Project::BaseBranch and Project::Changes#since, os-sdlc-survivors CLI. Post-run fixes: Mutineer runs in the target directory with target-relative paths (1596f6ba); Project::Target#directory owns the target's absolute directory (bd83e03e). **Evidence:** Implementation 264 implementation_complete (21 dispatches, 25 min 20 s); commits 09ad8f39, 1596f6ba, bd83e03e on ticket-1640. Full suite 3528 runs, 0 failures (61.9 s). Rubocop clean. Design audit of 1596f6ba: 4 findings, 1 fixed, 1 removed by that fix, 2 answered. Real run on plugins/os-sdlc: survivors abfd2fb825c1 (return_nil) and 559f08c32fd1 (statement_removal) at lib/os_sdlc/offense_directive.rb:28. **Follow-ups:** #1641 (survivors gate step) is unblocked. Sibling test_path note: dropped by ticket-skeptic, recorded as a comment on this ticket.

jared closed this issue

2026-11-08 16:45:18 +00:00

jared referenced this issue

2026-11-08 16:45:19 +00:00

os-sdlc: survivors gate step and slow-step warning (#1638 part C) #1641

jared referenced this issue

2026-11-08 17:06:20 +00:00

os-sdlc: per-ticket mutation survivor gate (mutation testing increment 1) #1638

jared referenced this issue from a commit

2026-11-08 18:29:55 +00:00

feat(os-sdlc): POODR audit passed: MutationTest depends only on the runner and switch roles; MutineerRunner owns argv, JSON, and fold; Implementation#mutation_test wires BaseBranch and Changes; R1-R5 present. (#1640)

jared referenced this issue from a commit