jared/cc-os

Watch

Fork 0

Run os-sdlc detector bin tests in-process, not as subprocesses #1330

New issue

Closed

opened 2026-10-25 19:48:30 +00:00 by jared · 2 comments

jared commented

2026-10-25 19:48:30 +00:00

Owner

Problem

Five detector test files start our own bin scripts with Open3. detector_bin_test alone spawns in 7 places. The os-sdlc suite runs 65.8 s against a 60 s budget.

Approach

Replace each spawn with OsSdlc::Boot.run("<name>", OsSdlc::ProcessIO.new(cwd:, stdin: StringIO.new, stdout: StringIO.new, stderr: StringIO.new, env: {}), argv). The files are cycle_edges_bin_test.rb (6 tests), detector_bin_test.rb (8), error_lineage_bin_test.rb (8), holders_bin_test.rb (12), and reach_through_bin_test.rb (3). Expected saving: about 3.5 s.

Behavior is unchanged. The existing suite is the proof. Each bin keeps exactly one smoke test, in a file named <bin>_bin_smoke_test.rb. The smoke test runs the bin as a subprocess to prove the plain-ruby boot (ADR-0166) and the shebang. Calls to external tools (git, rubocop, likec4) stay as subprocess calls. The ADR-0232 os-sdlc-<command> contract (stdin, argv, stdout byte-identical; harness errors on stderr with exit 3) must stay asserted by the in-process tests or the smoke tests.

Required work:

Replace each Open3 spawn in the five detector bin test files with OsSdlc::Boot.run and OsSdlc::ProcessIO.
Add one <bin>_bin_smoke_test.rb file per detector bin that runs the bin as a subprocess with --help and expects exit 0.
Remove Open3 from the converted test files — check: absent Open3 in plugins/os-sdlc/tests/detector/*_bin_test.rb

Acceptance criteria

Given the converted tests, when the suite runs, then no test outside a *_bin_smoke_test.rb file starts a plugin bin/ script as a subprocess.
Given each detector bin, when its smoke test runs, then it boots with plain ruby and exits 0 on --help.
Given the converted detector tests, when they run, then they pass with the same assertions as before.

Owner sketch

OsSdlc::Boot.run(name, io, argv) returns the exit code. The test reads stdout and stderr from the StringIO objects in OsSdlc::ProcessIO.

Non-goals

The 11.8 s DetectorEntryPointsTest case. Another ticket covers it.

Origin

Trigger: ruby plugins/os-sdlc/tests/all.rb took 65.8 s against a 60 s budget. About 70% of summed test time is tests that start our own bin scripts with Open3.
Improvised this session: none
Chain: suite over budget ← tests start our own Ruby bin, hook, or CLI as a subprocess (Ruby boot cost per call) ← no rule says to call the in-process entry (Jared ruling, 2026-10-25; habit, no ADR)
Root candidate: none found; searched Open3, subprocess, in-process, bin_test, run_bin, Boot.run, ProcessIO
Where: plugins/os-sdlc/tests/detector/*_bin_test.rb
Session: 01c29715-c945-43b4-86fc-6c5fc6c4dd3f
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/01c29715-c945-43b4-86fc-6c5fc6c4dd3f.jsonl

## Problem Five detector test files start our own bin scripts with Open3. `detector_bin_test` alone spawns in 7 places. The os-sdlc suite runs 65.8 s against a 60 s budget. ## Approach Replace each spawn with `OsSdlc::Boot.run("<name>", OsSdlc::ProcessIO.new(cwd:, stdin: StringIO.new, stdout: StringIO.new, stderr: StringIO.new, env: {}), argv)`. The files are `cycle_edges_bin_test.rb` (6 tests), `detector_bin_test.rb` (8), `error_lineage_bin_test.rb` (8), `holders_bin_test.rb` (12), and `reach_through_bin_test.rb` (3). Expected saving: about 3.5 s. Behavior is unchanged. The existing suite is the proof. Each bin keeps exactly one smoke test, in a file named `<bin>_bin_smoke_test.rb`. The smoke test runs the bin as a subprocess to prove the plain-ruby boot (ADR-0166) and the shebang. Calls to external tools (git, rubocop, likec4) stay as subprocess calls. The ADR-0232 `os-sdlc-<command>` contract (stdin, argv, stdout byte-identical; harness errors on stderr with exit 3) must stay asserted by the in-process tests or the smoke tests. Required work: 1. Replace each Open3 spawn in the five detector bin test files with `OsSdlc::Boot.run` and `OsSdlc::ProcessIO`. 2. Add one `<bin>_bin_smoke_test.rb` file per detector bin that runs the bin as a subprocess with `--help` and expects exit 0. 3. Remove `Open3` from the converted test files — check: absent `Open3` in `plugins/os-sdlc/tests/detector/*_bin_test.rb` ## Acceptance criteria 1. Given the converted tests, when the suite runs, then no test outside a `*_bin_smoke_test.rb` file starts a plugin `bin/` script as a subprocess. 2. Given each detector bin, when its smoke test runs, then it boots with plain ruby and exits 0 on `--help`. 3. Given the converted detector tests, when they run, then they pass with the same assertions as before. ## Owner sketch `OsSdlc::Boot.run(name, io, argv)` returns the exit code. The test reads `stdout` and `stderr` from the `StringIO` objects in `OsSdlc::ProcessIO`. ## Non-goals The 11.8 s `DetectorEntryPointsTest` case. Another ticket covers it. ## Origin - Trigger: `ruby plugins/os-sdlc/tests/all.rb` took 65.8 s against a 60 s budget. About 70% of summed test time is tests that start our own bin scripts with Open3. - Improvised this session: none - Chain: suite over budget ← tests start our own Ruby bin, hook, or CLI as a subprocess (Ruby boot cost per call) ← no rule says to call the in-process entry (Jared ruling, 2026-10-25; habit, no ADR) - Root candidate: none found; searched Open3, subprocess, in-process, bin_test, run_bin, Boot.run, ProcessIO - Where: plugins/os-sdlc/tests/detector/*_bin_test.rb - Session: 01c29715-c945-43b4-86fc-6c5fc6c4dd3f - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/01c29715-c945-43b4-86fc-6c5fc6c4dd3f.jsonl

jared added the

plugin/os-sdlc

filed-by/user

category/structure

labels

2026-10-25 19:48:30 +00:00

~~jared referenced this issue 2026-10-25 19:49:02 +00:00~~

Add a cop that flags tests starting our own plugin bin or hook as a subprocess #1337

jared referenced this issue

2026-10-25 19:49:03 +00:00

Find why DetectorEntryPointsTest#test_bin_script_maps_to_its_first_constant takes 11.8 s #1338

jared commented

2026-10-25 19:49:48 +00:00

Author

Owner

Work started in /os-sdlc:fast-batch fast-batch-15, branch fast-batch-15.

jared referenced this issue

2026-10-25 20:13:35 +00:00

CopSource target Ruby version depends on the current directory #1226

jared referenced this issue from a commit

2026-10-26 21:19:26 +00:00

test(os-sdlc): detector bin tests and crc ledger and shares scenarios call the Boot entry in a forked worker (#1330)

jared commented

2026-10-26 21:21:24 +00:00

Author

Owner

Resolution

Done: detector bin tests and the crc ledger/shares scenarios use InProcessBin; one smoke file each.

Evidence: Finding B 2fea2e8a. Suites at c9f5db8f: os-sdlc 2701 runs 0 failures (53.1 s, budget 60 s), os 47, os-backlog 327, os-aidd-lint 491, os-adr 111, os-me 399, cc-architect plugin_config 106, all green. Merged to main c9f5db8f, pushed be5d2c5d. Batch state and rulings: docs/crc/os-sdlc/fast-batch-15/.

Follow-ups: #1226 (R2: CopSites target Ruby version; occurrence posted; DetectorHoldersBinTest keeps one Open3 run until it is fixed). #1337 (cop) runs next.

## Resolution **Done:** detector bin tests and the crc ledger/shares scenarios use InProcessBin; one smoke file each. **Evidence:** Finding B 2fea2e8a. Suites at c9f5db8f: os-sdlc 2701 runs 0 failures (53.1 s, budget 60 s), os 47, os-backlog 327, os-aidd-lint 491, os-adr 111, os-me 399, cc-architect plugin_config 106, all green. Merged to main c9f5db8f, pushed be5d2c5d. Batch state and rulings: docs/crc/os-sdlc/fast-batch-15/. **Follow-ups:** #1226 (R2: CopSites target Ruby version; occurrence posted; DetectorHoldersBinTest keeps one Open3 run until it is fixed). #1337 (cop) runs next.

jared closed this issue

2026-10-26 21:21:24 +00:00