Run os-sdlc crc bin tests in-process, not as subprocesses #1329

New issue

Closed

opened 2026-10-25 19:48:29 +00:00 by jared · 2 comments

jared commented

2026-10-25 19:48:29 +00:00

Owner

Problem

Ten crc test files start our own crc-* bin scripts with Open3. They make 83 subprocess calls. These calls use 26.5 s of the 29.3 s that these files take. The os-sdlc suite runs 65.8 s against a 60 s budget.

Approach

Each file has a private run_bin helper that calls Open3.capture2 or Open3.capture3. Replace run_bin with OsSdlc::Boot.run("<name>", OsSdlc::ProcessIO.new(cwd:, stdin: StringIO.new, stdout: StringIO.new, stderr: StringIO.new, env: {}), argv). See tests/crc_boot_entry_test.rb:209 and tests/system_model_cli_test.rb:129 for the pattern. The files are census (37 tests), census_since (13), filing (13), inward_delta (10), jobs (13), placement (7), records (39), score (6), sweep (39), and tickets (47). Also convert tests/support/crc_census_scenario.rb. Expected saving: about 24 s of wall time.

Behavior is unchanged. The existing suite is the proof. Each bin keeps exactly one smoke test, in a file named <bin>_bin_smoke_test.rb. The smoke test runs the bin as a subprocess to prove the plain-ruby boot (ADR-0166) and the shebang. Calls to external tools (git, rubocop, likec4) stay as subprocess calls. The ADR-0232 os-sdlc-<command> contract (stdin, argv, stdout byte-identical; harness errors on stderr with exit 3) must stay asserted by the in-process tests or the smoke tests.

Required work:

Replace the run_bin helper in each tests/crc/*_bin_test.rb file with a call to OsSdlc::Boot.run and OsSdlc::ProcessIO.
Convert tests/support/crc_census_scenario.rb to the in-process call.
Add one <bin>_bin_smoke_test.rb file per crc bin that runs the bin as a subprocess with --help and expects exit 0.
Remove Open3 from the converted test files — check: absent Open3 in plugins/os-sdlc/tests/crc/*_bin_test.rb

Acceptance criteria

Given the converted tests, when the suite runs, then no test outside a *_bin_smoke_test.rb file starts a plugin bin/ script as a subprocess.
Given each crc bin, when its smoke test runs, then it boots with plain ruby and exits 0 on --help.
Given the converted crc tests, when they run, then they pass with the same assertions as before.
Given the os-sdlc suite, when it runs, then the crc bin test files take less than 5 s together.

Owner sketch

OsSdlc::Boot.run(name, io, argv) returns the exit code. The test passes an OsSdlc::ProcessIO built from StringIO objects and reads stdout and stderr from them.

Non-goals

Other os-sdlc spawning tests. Another ticket covers them.

Origin

Trigger: ruby plugins/os-sdlc/tests/all.rb took 65.8 s against a 60 s budget. About 70% of summed test time is tests that start our own bin scripts with Open3.
Improvised this session: none
Chain: suite over budget ← tests start our own Ruby bin, hook, or CLI as a subprocess (Ruby boot cost per call) ← no rule says to call the in-process entry (Jared ruling, 2026-10-25; habit, no ADR)
Root candidate: none found; searched Open3, subprocess, in-process, bin_test, run_bin, Boot.run, ProcessIO
Where: plugins/os-sdlc/tests/crc/*_bin_test.rb, plugins/os-sdlc/tests/support/crc_census_scenario.rb
Session: 01c29715-c945-43b4-86fc-6c5fc6c4dd3f
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/01c29715-c945-43b4-86fc-6c5fc6c4dd3f.jsonl

## Problem Ten crc test files start our own `crc-*` bin scripts with Open3. They make 83 subprocess calls. These calls use 26.5 s of the 29.3 s that these files take. The os-sdlc suite runs 65.8 s against a 60 s budget. ## Approach Each file has a private `run_bin` helper that calls `Open3.capture2` or `Open3.capture3`. Replace `run_bin` with `OsSdlc::Boot.run("<name>", OsSdlc::ProcessIO.new(cwd:, stdin: StringIO.new, stdout: StringIO.new, stderr: StringIO.new, env: {}), argv)`. See `tests/crc_boot_entry_test.rb:209` and `tests/system_model_cli_test.rb:129` for the pattern. The files are `census` (37 tests), `census_since` (13), `filing` (13), `inward_delta` (10), `jobs` (13), `placement` (7), `records` (39), `score` (6), `sweep` (39), and `tickets` (47). Also convert `tests/support/crc_census_scenario.rb`. Expected saving: about 24 s of wall time. Behavior is unchanged. The existing suite is the proof. Each bin keeps exactly one smoke test, in a file named `<bin>_bin_smoke_test.rb`. The smoke test runs the bin as a subprocess to prove the plain-ruby boot (ADR-0166) and the shebang. Calls to external tools (git, rubocop, likec4) stay as subprocess calls. The ADR-0232 `os-sdlc-<command>` contract (stdin, argv, stdout byte-identical; harness errors on stderr with exit 3) must stay asserted by the in-process tests or the smoke tests. Required work: 1. Replace the `run_bin` helper in each `tests/crc/*_bin_test.rb` file with a call to `OsSdlc::Boot.run` and `OsSdlc::ProcessIO`. 2. Convert `tests/support/crc_census_scenario.rb` to the in-process call. 3. Add one `<bin>_bin_smoke_test.rb` file per crc bin that runs the bin as a subprocess with `--help` and expects exit 0. 4. Remove `Open3` from the converted test files — check: absent `Open3` in `plugins/os-sdlc/tests/crc/*_bin_test.rb` ## Acceptance criteria 1. Given the converted tests, when the suite runs, then no test outside a `*_bin_smoke_test.rb` file starts a plugin `bin/` script as a subprocess. 2. Given each crc bin, when its smoke test runs, then it boots with plain ruby and exits 0 on `--help`. 3. Given the converted crc tests, when they run, then they pass with the same assertions as before. 4. Given the os-sdlc suite, when it runs, then the crc bin test files take less than 5 s together. ## Owner sketch `OsSdlc::Boot.run(name, io, argv)` returns the exit code. The test passes an `OsSdlc::ProcessIO` built from `StringIO` objects and reads `stdout` and `stderr` from them. ## Non-goals Other os-sdlc spawning tests. Another ticket covers them. ## Origin - Trigger: `ruby plugins/os-sdlc/tests/all.rb` took 65.8 s against a 60 s budget. About 70% of summed test time is tests that start our own bin scripts with Open3. - Improvised this session: none - Chain: suite over budget ← tests start our own Ruby bin, hook, or CLI as a subprocess (Ruby boot cost per call) ← no rule says to call the in-process entry (Jared ruling, 2026-10-25; habit, no ADR) - Root candidate: none found; searched Open3, subprocess, in-process, bin_test, run_bin, Boot.run, ProcessIO - Where: plugins/os-sdlc/tests/crc/*_bin_test.rb, plugins/os-sdlc/tests/support/crc_census_scenario.rb - Session: 01c29715-c945-43b4-86fc-6c5fc6c4dd3f - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/01c29715-c945-43b4-86fc-6c5fc6c4dd3f.jsonl

jared added the

plugin/os-sdlc

filed-by/user

category/structure

labels

2026-10-25 19:48:29 +00:00

~~jared referenced this issue 2026-10-25 19:49:02 +00:00~~

Add a cop that flags tests starting our own plugin bin or hook as a subprocess #1337

jared commented

2026-10-25 19:49:48 +00:00

Author

Owner

Work started in /os-sdlc:fast-batch fast-batch-15, branch fast-batch-15.

jared referenced this issue from a commit

2026-10-26 21:19:26 +00:00

test(os-sdlc): crc bin tests call the Boot entry in a forked worker; one smoke test per bin (#1329)

jared commented

2026-10-26 21:21:23 +00:00

Author

Owner

Resolution

Done: crc bin tests call Boot.run through InProcessBin (forked); one *_bin_smoke_test.rb per crc bin.

Evidence: Finding A 5813d542; fix R18 aa1b19e2 (InProcessBin error reporting). Suites at c9f5db8f: os-sdlc 2701 runs 0 failures (53.1 s, budget 60 s), os 47, os-backlog 327, os-aidd-lint 491, os-adr 111, os-me 399, cc-architect plugin_config 106, all green. Merged to main c9f5db8f, pushed be5d2c5d. Batch state and rulings: docs/crc/os-sdlc/fast-batch-15/.

Follow-ups: #1390 (R1: census/score load the target loader, so InProcessBin forks; occurrence posted). #1337 (cop) runs next.

## Resolution **Done:** crc bin tests call Boot.run through InProcessBin (forked); one *_bin_smoke_test.rb per crc bin. **Evidence:** Finding A 5813d542; fix R18 aa1b19e2 (InProcessBin error reporting). Suites at c9f5db8f: os-sdlc 2701 runs 0 failures (53.1 s, budget 60 s), os 47, os-backlog 327, os-aidd-lint 491, os-adr 111, os-me 399, cc-architect plugin_config 106, all green. Merged to main c9f5db8f, pushed be5d2c5d. Batch state and rulings: docs/crc/os-sdlc/fast-batch-15/. **Follow-ups:** #1390 (R1: census/score load the target loader, so InProcessBin forks; occurrence posted). #1337 (cop) runs next.

jared closed this issue

2026-10-26 21:21:23 +00:00