Challenge the subprocess test in project_test_path_default_test.rb #923

New issue

Open

opened 2026-10-03 13:20:04 +00:00 by jared · 0 comments

jared commented

2026-10-03 13:20:04 +00:00

Owner

Problem

Jared 2026-10-03: "Now I think we should be challenging the slow test."
The test test_project_config_writes_spec_test_path_when_only_spec_dir_exists in plugins/os-sdlc/tests/model/project_test_path_default_test.rb:43 starts a child process: Open3.capture3(File.expand_path("../../bin/os-sdlc", __dir__), "project-config", "widgets", chdir: dir).
Jared treats the slow test as a symptom that something is wrong with the test.
This ticket needs a design ruling from Jared: delete the test, or reshape it.

Observed

Commit 5d0a1cd6 (#860) changed the test from one subprocess to another subprocess.
Before 5d0a1cd6 it ran ruby -I lib -e with a script that built OsSdlc::Project::ConfigCommand (child process, not in-process).
Jared's premise was that the test built ConfigCommand in-process.
That premise is false: both versions spawn a child process.
The only change is that the child now boots bin/os-sdlc and goes through Onboarding.start.
The test discards the exit status and stdout, and asserts only written["test_path"] == "spec" (line 52).
The whole file runs in 0.24 s (4 runs); ruby -Itests -Ilib tests/model/project_test_path_default_test.rb takes 0.92 s wall time. The slowness is therefore a subprocess boot cost, not a long test.

Reproduce

n/a: test-design challenge, nothing to reproduce.

Approach

Challenge the test with Jared's questions, then rule: delete or reshape.

What is the point of this test? It pins that project-config writes test_path: spec when only spec/ exists.
Does it need to exist at all? plugins/os-sdlc/tests/controller/project/cli_project_config_test.rb:40 (test_project_config_detects_spec_directory_and_writes_test_path) creates spec/ and asserts test_path: "spec" through Project::Config.load. It looks like a duplicate in-process test of the same behavior.
Does it test more than it should? It runs the bin script, Thor dispatch, and ConfigCommand to reach one detection rule (ConfigCommand#with_detected_test_path, plugins/os-sdlc/lib/os_sdlc/command/project/config_command.rb:28). The same file also holds three Project::Config.load default tests (lines 8-39) that need no subprocess.
How is it set up compared with similar tests? plugins/os-sdlc/tests/controller/project/cli_project_config_help_test.rb:66 calls OsSdlc::Project::Onboarding.start(argv, invocation: io) in-process with a ProcessIO built over StringIO. This test uses a real subprocess instead, and the file is also in tests/model/ while its subject is a controller/command.
Result to rule on: delete the test (if cli_project_config_test.rb:40 covers it), or reshape it to the in-process Onboarding.start pattern, or move it beside the other project-config controller tests. State the reason with the ruling.

Required work: none beyond the acceptance criteria.

Acceptance criteria

Given the ruling, when the ticket closes, then the ticket states whether the test was deleted or reshaped and why.
Given the test is kept, when plugins/os-sdlc/tests/model/project_test_path_default_test.rb runs, then no test in it starts a child process.

Origin

Trigger: Jared asked to challenge the slow test after #860 shipped (2026-10-03).
Improvised this session: none
Chain: slow subprocess test ← #860 (5d0a1cd6) kept a subprocess and moved it to bin/os-sdlc ← the original test (before #860) already used a subprocess (DESIGN: no ADR; file plugins/os-sdlc/tests/model/project_test_path_default_test.rb)
Root candidate: none
Where: plugins/os-sdlc/tests/model/project_test_path_default_test.rb
Session: a0dbd5c4-ce77-4eaa-bb8a-9517044ae3ad
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/a0dbd5c4-ce77-4eaa-bb8a-9517044ae3ad.jsonl

VERDICT: CREATE

## Problem Jared 2026-10-03: "Now I think we should be challenging the slow test." The test `test_project_config_writes_spec_test_path_when_only_spec_dir_exists` in `plugins/os-sdlc/tests/model/project_test_path_default_test.rb:43` starts a child process: `Open3.capture3(File.expand_path("../../bin/os-sdlc", __dir__), "project-config", "widgets", chdir: dir)`. Jared treats the slow test as a symptom that something is wrong with the test. This ticket needs a design ruling from Jared: delete the test, or reshape it. ## Observed Commit 5d0a1cd6 (#860) changed the test from one subprocess to another subprocess. Before 5d0a1cd6 it ran `ruby -I lib -e` with a script that built `OsSdlc::Project::ConfigCommand` (child process, not in-process). Jared's premise was that the test built `ConfigCommand` in-process. That premise is false: both versions spawn a child process. The only change is that the child now boots `bin/os-sdlc` and goes through `Onboarding.start`. The test discards the exit status and stdout, and asserts only `written["test_path"] == "spec"` (line 52). The whole file runs in 0.24 s (4 runs); `ruby -Itests -Ilib tests/model/project_test_path_default_test.rb` takes 0.92 s wall time. The slowness is therefore a subprocess boot cost, not a long test. ## Reproduce n/a: test-design challenge, nothing to reproduce. ## Approach Challenge the test with Jared's questions, then rule: delete or reshape. 1. What is the point of this test? It pins that `project-config` writes `test_path: spec` when only `spec/` exists. 2. Does it need to exist at all? `plugins/os-sdlc/tests/controller/project/cli_project_config_test.rb:40` (`test_project_config_detects_spec_directory_and_writes_test_path`) creates `spec/` and asserts `test_path: "spec"` through `Project::Config.load`. It looks like a duplicate in-process test of the same behavior. 3. Does it test more than it should? It runs the bin script, Thor dispatch, and `ConfigCommand` to reach one detection rule (`ConfigCommand#with_detected_test_path`, `plugins/os-sdlc/lib/os_sdlc/command/project/config_command.rb:28`). The same file also holds three `Project::Config.load` default tests (lines 8-39) that need no subprocess. 4. How is it set up compared with similar tests? `plugins/os-sdlc/tests/controller/project/cli_project_config_help_test.rb:66` calls `OsSdlc::Project::Onboarding.start(argv, invocation: io)` in-process with a `ProcessIO` built over `StringIO`. This test uses a real subprocess instead, and the file is also in `tests/model/` while its subject is a controller/command. 5. Result to rule on: delete the test (if cli_project_config_test.rb:40 covers it), or reshape it to the in-process `Onboarding.start` pattern, or move it beside the other project-config controller tests. State the reason with the ruling. Required work: none beyond the acceptance criteria. ## Acceptance criteria 1. Given the ruling, when the ticket closes, then the ticket states whether the test was deleted or reshaped and why. 2. Given the test is kept, when `plugins/os-sdlc/tests/model/project_test_path_default_test.rb` runs, then no test in it starts a child process. ## Origin - Trigger: Jared asked to challenge the slow test after #860 shipped (2026-10-03). - Improvised this session: none - Chain: slow subprocess test ← #860 (5d0a1cd6) kept a subprocess and moved it to `bin/os-sdlc` ← the original test (before #860) already used a subprocess (DESIGN: no ADR; file `plugins/os-sdlc/tests/model/project_test_path_default_test.rb`) - Root candidate: none - Where: plugins/os-sdlc/tests/model/project_test_path_default_test.rb - Session: a0dbd5c4-ce77-4eaa-bb8a-9517044ae3ad - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/a0dbd5c4-ce77-4eaa-bb8a-9517044ae3ad.jsonl VERDICT: CREATE

jared added the

plugin/os-sdlc

filed-by/agent

labels

2026-10-03 13:20:04 +00:00

jared added the

category/owner-ruling

label

2026-10-10 13:31:49 +00:00