jared/cc-os

Watch

Fork 0

os-sdlc: per-ticket mutation survivor gate (mutation testing increment 1) #1638

New issue

Closed

opened 2026-11-08 15:24:52 +00:00 by jared · 4 comments

jared commented

2026-11-08 15:24:52 +00:00

Owner

Problem

The os-sdlc gate does not show which changed lines no test protects. A surviving mutant on a changed line shows a missing test or dead code. The Mutineer pilot showed that Mutineer measures this suite at changed-line scope.

Reproduce

n/a: feature ticket.

Approach

Source: docs/research/mutation-testing/plan-v1-2026-11-08.md, table "Increments, in order", row 1. The section "Rulings" is binding; do not re-decide it.

Add mutineer to the Gemfile test group.
Add tests/mutation_helper.rb. It loads the normal helper without minitest/parallel_fork and adds the Process._fork hook from docs/research/mutation-testing/2026-11-07-pilot-evidence/test_helper-fork-hook.rb. Do not edit tests/test_helper.rb.
Add .mutineer.yml as in the plan (capture_timeout: 1800).
Add a Ruby adapter under lib/os_sdlc/gate/ (or where the architecture lens places it). It runs Mutineer with --since <base> --coverage-from <capture> --format json. It folds boolean_literal into the sibling statement_removal, drops ignore: ids, and returns survivor rows with id, file:line, and diff. No agent reads raw Mutineer JSON.
The adapter measures Mutineer wall time. Above 500 s, it adds a warning to the gate reply. The warning never changes the gate result.

Rejected: file scope — the plan rules line scope for the gate. Mutant — license. Heckle — does not run on Ruby 4.0.2.

Origin

Trigger: plan docs/research/mutation-testing/plan-v1-2026-11-08.md (commit b90d8253), ruled by Jared 2026-11-07 and 2026-11-08
Improvised this session: none
Chain: suite doubled in six weeks (757 test files, 3,572 test methods) ← suspected duplicated behavior and tests with no demonstrated value ← no mutation evidence in the gate. End: DESIGN (docs/research/mutation-testing/plan-v1-2026-11-08.md)
Root candidate: none found; searched mutation, mutineer, mutant, survivor, taxonomy, redundant test, blind test, duplicate, overlap, ledger
Where: plugins/os-sdlc/lib/os_sdlc/gate/
Session: ff703e34-8046-4b24-af68-6ef76d8aa338
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/ff703e34-8046-4b24-af68-6ef76d8aa338.jsonl

VERDICT: CREATE

## Problem The os-sdlc gate does not show which changed lines no test protects. A surviving mutant on a changed line shows a missing test or dead code. The Mutineer pilot showed that Mutineer measures this suite at changed-line scope. ## Reproduce n/a: feature ticket. ## Approach Source: `docs/research/mutation-testing/plan-v1-2026-11-08.md`, table "Increments, in order", row 1. The section "Rulings" is binding; do not re-decide it. 1. Add `mutineer` to the `Gemfile` test group. 2. Add `tests/mutation_helper.rb`. It loads the normal helper without `minitest/parallel_fork` and adds the `Process._fork` hook from `docs/research/mutation-testing/2026-11-07-pilot-evidence/test_helper-fork-hook.rb`. Do not edit `tests/test_helper.rb`. 3. Add `.mutineer.yml` as in the plan (`capture_timeout: 1800`). 4. Add a Ruby adapter under `lib/os_sdlc/gate/` (or where the architecture lens places it). It runs Mutineer with `--since <base> --coverage-from <capture> --format json`. It folds `boolean_literal` into the sibling `statement_removal`, drops `ignore:` ids, and returns survivor rows with id, file:line, and diff. No agent reads raw Mutineer JSON. 5. The adapter measures Mutineer wall time. Above 500 s, it adds a warning to the gate reply. The warning never changes the gate result. Rejected: file scope — the plan rules line scope for the gate. Mutant — license. Heckle — does not run on Ruby 4.0.2. ## Origin - Trigger: plan `docs/research/mutation-testing/plan-v1-2026-11-08.md` (commit b90d8253), ruled by Jared 2026-11-07 and 2026-11-08 - Improvised this session: none - Chain: suite doubled in six weeks (757 test files, 3,572 test methods) ← suspected duplicated behavior and tests with no demonstrated value ← no mutation evidence in the gate. End: DESIGN (`docs/research/mutation-testing/plan-v1-2026-11-08.md`) - Root candidate: none found; searched mutation, mutineer, mutant, survivor, taxonomy, redundant test, blind test, duplicate, overlap, ledger - Where: plugins/os-sdlc/lib/os_sdlc/gate/ - Session: ff703e34-8046-4b24-af68-6ef76d8aa338 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/ff703e34-8046-4b24-af68-6ef76d8aa338.jsonl VERDICT: CREATE

jared added the

plugin/os-sdlc

filed-by/user

category/code-change

labels

2026-11-08 15:24:52 +00:00

jared referenced this issue

2026-11-08 15:25:02 +00:00

os-sdlc: whole-codebase mutation pass (mutation testing increment 4) #1636

jared commented

2026-11-08 15:26:27 +00:00

Author

Owner

Decompose routed to /os-sdlc:ticket-takedown (new object, more than one subsystem).

a. One change: a new gate step runs Mutineer at changed-line scope and adds survivor rows to the gate reply. Gate::Reply#to_h (lib/os_sdlc/gate/reply.rb:6) has no survivor field and no non-blocking warning field today.

b. Owner: none exists. New object: a Gate adapter whose secret is Mutineer's JSON shape, the boolean_literal fold, the ignore: filter, and the 500 s slowdown warning.

Facts for takedown:

Base for --since: Gate::UnchangedCheck::BaseCheckout#merge_base (lib/os_sdlc/gate/unchanged_check/base_checkout.rb:22) with Project::BaseBranch#name.
Implementation::MutationCheck (lib/os_sdlc/implementation/mutation_check.rb) runs one test-writer-supplied seeded mutation. Decide: keep beside the new step, or replace.
Open question: does a survivor make the gate red, or is the survivor list informational for the reviewer? The plan says 'survivor rows as a gate reply'; gate pass/fail must live in tested Ruby.
ADR check: no blocking ADR. ADR-0166 (commit Gemfile.lock), ADR-0090/0302 (Zeitwerk paths) apply. The ADR judge ruled the gem and the reply fields a habit; no ADR.

Decompose routed to /os-sdlc:ticket-takedown (new object, more than one subsystem). a. One change: a new gate step runs Mutineer at changed-line scope and adds survivor rows to the gate reply. Gate::Reply#to_h (lib/os_sdlc/gate/reply.rb:6) has no survivor field and no non-blocking warning field today. b. Owner: none exists. New object: a Gate adapter whose secret is Mutineer's JSON shape, the boolean_literal fold, the ignore: filter, and the 500 s slowdown warning. Facts for takedown: - Base for --since: Gate::UnchangedCheck::BaseCheckout#merge_base (lib/os_sdlc/gate/unchanged_check/base_checkout.rb:22) with Project::BaseBranch#name. - Implementation::MutationCheck (lib/os_sdlc/implementation/mutation_check.rb) runs one test-writer-supplied seeded mutation. Decide: keep beside the new step, or replace. - Open question: does a survivor make the gate red, or is the survivor list informational for the reviewer? The plan says 'survivor rows as a gate reply'; gate pass/fail must live in tested Ruby. - ADR check: no blocking ADR. ADR-0166 (commit Gemfile.lock), ADR-0090/0302 (Zeitwerk paths) apply. The ADR judge ruled the gem and the reply fields a habit; no ADR.

jared referenced this issue

2026-11-08 15:58:26 +00:00

os-sdlc: one owner for the base commit and changes since it (#1638 part A) #1639

jared referenced this issue

2026-11-08 15:58:27 +00:00

os-sdlc: MutationTest objects and os-sdlc-survivors command (#1638 part B) #1640

jared commented

2026-11-08 15:58:39 +00:00

Author

Owner

Design: docs/system-design-format/proof/2026-11-08-os-sdlc-mutation-survivors.md (f0e172b7, f9d79c14). Slice 1 (gem, tests/mutation_helper.rb, .mutineer.yml) done in 8e28c94e.
Children, in order:

#1639 part A: one owner for the base commit and changes since it (slice 2)
#1640 part B: MutationTest objects and os-sdlc-survivors (slices 3-5), blocked on #1639
part C: survivors gate step, 500 s warning, survivor-check map node (slices 6-7), filed after its blast radius, blocked on #1640
Close-out: close #1638 when A, B, and C are closed.

Design: docs/system-design-format/proof/2026-11-08-os-sdlc-mutation-survivors.md (f0e172b7, f9d79c14). Slice 1 (gem, tests/mutation_helper.rb, .mutineer.yml) done in 8e28c94e. Children, in order: - #1639 part A: one owner for the base commit and changes since it (slice 2) - #1640 part B: MutationTest objects and os-sdlc-survivors (slices 3-5), blocked on #1639 - part C: survivors gate step, 500 s warning, survivor-check map node (slices 6-7), filed after its blast radius, blocked on #1640 - [ ] Close-out: close #1638 when A, B, and C are closed.

jared referenced this issue

2026-11-08 16:01:00 +00:00

os-sdlc: survivors gate step and slow-step warning (#1638 part C) #1641

jared commented

2026-11-08 16:01:03 +00:00

Author

Owner

Part C filed as #1641 (survivors gate step, slow-step warning). The map edit (survivor-check and survivor-tests nodes) is orchestrator work after #1641 merges; see the design doc, slice 7.

jared referenced this issue

2026-11-08 17:01:09 +00:00

os-sdlc: survivors gate step and slow-step warning (#1638 part C) #1641

jared commented

2026-11-08 17:06:20 +00:00

Author

Owner

Resolution

Done: Mutation testing increment 1 built: Project::BaseBranch#merge_base and Project::Changes#since (#1639); MutationTest with switch, runner and scope roles, MutineerRunner, os-sdlc-survivors CLI (#1640); gate survivors of type survivor with the non-blocking 500 s slow-step warning (#1641); poodr-ticket-implementation map routes suite-check pass through new nodes survivor-check and survivor-tests (d10d4c2e).

Evidence: Children #1639, #1640, #1641 closed and merged to main. Map wiring commit d10d4c2e: Map::Document.load passes (16 nodes), tests/map/document_test.rb 22 runs 0 failures, full suite 3535 runs 0 failures 61.8 s. First real survivor list: lib/os_sdlc/offense_directive.rb:28 abfd2fb825c1 (return_nil), 559f08c32fd1 (statement_removal). Design: docs/system-design-format/proof/2026-11-08-os-sdlc-mutation-survivors.md (slice 7 done).

Follow-ups: Increments 2-5 are open as #1634-#1637. Design change at wiring: Map::Check required a bound on the new loop, so survivor-check fail has bound 2 (doc updated). The first ticket run on the map after this merge is the first live run of the survivor nodes.

## Resolution **Done:** Mutation testing increment 1 built: Project::BaseBranch#merge_base and Project::Changes#since (#1639); MutationTest with switch, runner and scope roles, MutineerRunner, os-sdlc-survivors CLI (#1640); gate survivors of type survivor with the non-blocking 500 s slow-step warning (#1641); poodr-ticket-implementation map routes suite-check pass through new nodes survivor-check and survivor-tests (d10d4c2e). **Evidence:** Children #1639, #1640, #1641 closed and merged to main. Map wiring commit d10d4c2e: Map::Document.load passes (16 nodes), tests/map/document_test.rb 22 runs 0 failures, full suite 3535 runs 0 failures 61.8 s. First real survivor list: lib/os_sdlc/offense_directive.rb:28 abfd2fb825c1 (return_nil), 559f08c32fd1 (statement_removal). Design: docs/system-design-format/proof/2026-11-08-os-sdlc-mutation-survivors.md (slice 7 done). **Follow-ups:** Increments 2-5 are open as #1634-#1637. Design change at wiring: Map::Check required a bound on the new loop, so survivor-check fail has bound 2 (doc updated). The first ticket run on the map after this merge is the first live run of the survivor nodes.

jared closed this issue

2026-11-08 17:06:20 +00:00

jared referenced this issue

2026-11-08 18:07:27 +00:00

os-sdlc: test taxonomy classifier (mutation testing increment 2) #1634

jared referenced this issue from a commit

2026-11-08 18:29:55 +00:00

docs(os-sdlc): domain-object design for the mutation survivor gate (#1638)

jared referenced this issue from a commit

2026-11-08 18:29:55 +00:00

build(os-sdlc): add mutineer test-group gem, mutation helper, and .mutineer.yml (#1638)

jared referenced this issue from a commit