lint rule (consensus): seed-sweep-assertion couples tests to RNG internals #203

New issue

Closed

opened 2026-08-01 14:16:52 +00:00 by jared · 1 comment

jared commented

2026-08-01 14:16:52 +00:00

Owner

Migrated from jared/os-sdlc#28 (repo retired).

Problem

Looping an assertion over many RNG seeds (e.g. 200.times { |seed| ...; refute_equal(...) }) couples a test to the RNG's internal behavior rather than asserting the domain rule directly.

Detection

Inputs: *_test.rb files.
Algorithm: flag any assert/refute/assert_equal/etc. call nested inside an Integer#times/#each block body in a test file.
Failure message: "asserts inside a N.times loop -- this couples the test to RNG/iteration internals. Assert the underlying rule directly, or stub randomness to a deterministic value."

Correction

Assert the rule directly (e.g. an excluded case never appears in the candidate set), or stub the source of randomness instead of sweeping seeds.

Pass/fail examples

Must fail: 200.times { |seed| ...; refute_equal(...) } -- an assertion nested in a seed-sweep loop.
Must pass: a single deterministic assertion against a stubbed RNG or against the underlying rule/logic.

Provenance

Run 18 (ticket #18, decision-dice sandbox, now retired), Fable + Codex dual review, 2026-07-21.

Implementation plan

Candidate custom cop for plugins/os-sdlc/lib/os_sdlc/cops/ (Sdlc/Minitest namespace), same shape as the existing TestMethodAfterPrivate cop.

Migrated from jared/os-sdlc#28 (repo retired). ## Problem Looping an assertion over many RNG seeds (e.g. `200.times { |seed| ...; refute_equal(...) }`) couples a test to the RNG's internal behavior rather than asserting the domain rule directly. ## Detection - **Inputs:** *_test.rb files. - **Algorithm:** flag any assert/refute/assert_equal/etc. call nested inside an Integer#times/#each block body in a test file. - **Failure message:** "asserts inside a N.times loop -- this couples the test to RNG/iteration internals. Assert the underlying rule directly, or stub randomness to a deterministic value." ## Correction Assert the rule directly (e.g. an excluded case never appears in the candidate set), or stub the source of randomness instead of sweeping seeds. ## Pass/fail examples - **Must fail:** `200.times { |seed| ...; refute_equal(...) }` -- an assertion nested in a seed-sweep loop. - **Must pass:** a single deterministic assertion against a stubbed RNG or against the underlying rule/logic. ## Provenance Run 18 (ticket #18, decision-dice sandbox, now retired), Fable + Codex dual review, 2026-07-21. ## Implementation plan Candidate custom cop for plugins/os-sdlc/lib/os_sdlc/cops/ (Sdlc/Minitest namespace), same shape as the existing TestMethodAfterPrivate cop.

jared added the

ready-for-agent

enhancement

labels

2026-08-01 14:16:52 +00:00

jared commented

2026-08-01 14:33:16 +00:00

Author

Owner

Implemented as Sdlc/Minitest/SeedSweepAssertion (commit e49252f, branch worktree-sdlc-lint-cops). 5 tests green, rubocop clean.

Implemented as Sdlc/Minitest/SeedSweepAssertion (commit e49252f, branch worktree-sdlc-lint-cops). 5 tests green, rubocop clean.

jared closed this issue