Repo-wide guard against forks of this repo's own bin/ executables in tests; hook-test harness runner calls in-process; hang-guard 30 s -> 5 s #642

Closed
opened 2026-09-23 13:05:40 +00:00 by jared · 4 comments
Owner

Map: #631
Blocked by #638, #639, #640, #641

Problem

Child of #631, split from the user's 631d so that each child routes to one pipeline. It mirrors plugins/os-sdlc/tests/lsp_probe_real_daemon_guard_test.rb (ADR-0205 item 5; commit a295f828).

Observed

Commit 8615acc9 (#624, 2026-09-22) raised Timeout.timeout(5) to 30 s in plugins/os-sdlc/tests/pipeline/post_tool_use_async_dispatch_test.rb:10 and post_tool_use_non_pipeline_async_dispatch_test.rb:9, 23, 37. Its message: "5 s Timeout around two ruby process boots in PostToolUse tests is a hang guard that load alone exceeded; raised to 30 s". Inside the guard, tests/support/post_tool_use_hook_test_case.rb forks bin/os-sdlc-runner through run_cli (line 37) and run_subagent_start (line 42), and forks hooks/post_tool_use.rb (line 53). tests/pipeline/subagent_start_hook_test.rb:93 and subagent_stop_hook_test.rb:90,95 also fork the runner.

Reproduce

cd plugins/os-sdlc && ruby -Itests tests/pipeline/post_tool_use_non_pipeline_async_dispatch_test.rb -v

Expected

One prose sentence: a guard test fails when any file under a plugin test tree spawns an executable under a plugins/**/bin/ directory and is not on its allowlist; the PostToolUse hang guard is 5 s around a single hook boot.

Origin

  • Trigger: decomposition of #631, 2026-09-23
  • Improvised this session: none
  • Chain: 30 s hang guard ← runner forks inside the guarded block (DESIGN, post_tool_use_hook_test_case.rb:37,42) ← 8615acc9 masked the boot cost
  • Root candidate: this ticket; hook-script forks are #632
  • Where: plugins/os-sdlc/tests/support/post_tool_use_hook_test_case.rb; plugins/os-sdlc/tests/lsp_probe_real_daemon_guard_test.rb
  • Session: 5bb052ff-ddf2-4d47-a283-d00d071e9b79
  • Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/5bb052ff-ddf2-4d47-a283-d00d071e9b79.jsonl

Owner sketch

class OsSdlcOwnBinaryForkGuardTest < Minitest::Test   # plugins/os-sdlc/tests/own_binary_fork_guard_test.rb
  ALLOWED_FILES = [].freeze
  def test_only_allowlisted_files_fork_a_repo_binary
end

class PostToolUseHookTestCase < Minitest::Test
  def run_cli(*args)
  def run_subagent_start(rest)
end

Acceptance criteria

  1. Given a test-tree file that references a repo bin/ executable path and calls Open3, Process.spawn, system, or backticks, and that is not in ALLOWED_FILES, when the guard runs, then it fails and names the file.
  2. Given that only the allowlisted files spawn repo binaries, when the guard runs, then it passes.
  3. Given PostToolUseHookTestCase, when a test opens an implementation and a dispatch, then both runner calls run in-process through InProcessRunnerCli and only the hook script is a subprocess.
  4. Given the four PostToolUse async-dispatch tests, when read, then each Timeout.timeout argument is 5.

Approach

  • Scan plugins//tests/**/.rb and plugins/cc-architect/scripts/plugin_config/test/**/*.rb. Detect at file level, not line level: the BIN constant and the capture3 call sit on different lines. Build the binary names from Dir.glob("plugins/**/bin/*"). A reference is a literal bin/<name> or a "bin", "<name>" join.
  • Expected allowlist after #638-#641, one reason per entry: each smoke test file; tests/support/evidence_bin_runner.rb, tests/support/cli_shared_helpers.rb, tests/lint_worklist_residual_cli_test.rb (unconverted binaries, follow-up); tests/support/lsp_probe_bin_runner.rb (real daemon, ADR-0205); false positives tests/lint/rubocop_report_fetcher_test.rb and tests/cops/bin_script_load_in_test_test.rb (bin path as data). TODO: unverified. Run the guard and confirm the list.
  • Timeout revert condition: one hooks/post_tool_use.rb boot must finish under 5 s under the full ruby tests/all.rb load. #624 measured 5-6 s per runner boot under 12-way load. Measure the hook boot before the revert. If it is over 5 s, stop and report; do not raise the value again.
  • ADRs to respect: ADR-0205 (guard pattern; no process-global writes), ADR-0195 (one Minitest class per file), ADR-0172/0128 (hooks stay thin couriers until #632).
    Required work:
  1. Add a repo-wide guard test scanning plugin test trees (and the plugin_config test dir) for non-allowlisted files that fork a plugins/**/bin/* executable. (proven by AC 1, AC 2)
  2. Convert PostToolUseHookTestCase's runner calls to run in-process via InProcessRunnerCli, leaving only the hook script as a subprocess. (proven by AC 3)
  3. Revert the four PostToolUse async-dispatch Timeout.timeout guards from 30 to 5, only after measuring one hook boot under full-suite load at under 5 s. (proven by AC 4; check: grep -c 'Timeout.timeout(5)' plugins/os-sdlc/tests/pipeline/post_tool_use_async_dispatch_test.rb plugins/os-sdlc/tests/pipeline/post_tool_use_non_pipeline_async_dispatch_test.rb)

Structural follow-through

  • One History row in docs/implementation-status/os-sdlc.md.
Map: #631 Blocked by #638, #639, #640, #641 ## Problem Child of #631, split from the user's 631d so that each child routes to one pipeline. It mirrors plugins/os-sdlc/tests/lsp_probe_real_daemon_guard_test.rb (ADR-0205 item 5; commit a295f828). ## Observed Commit 8615acc9 (#624, 2026-09-22) raised `Timeout.timeout(5)` to 30 s in plugins/os-sdlc/tests/pipeline/post_tool_use_async_dispatch_test.rb:10 and post_tool_use_non_pipeline_async_dispatch_test.rb:9, 23, 37. Its message: "5 s Timeout around two ruby process boots in PostToolUse tests is a hang guard that load alone exceeded; raised to 30 s". Inside the guard, tests/support/post_tool_use_hook_test_case.rb forks bin/os-sdlc-runner through `run_cli` (line 37) and `run_subagent_start` (line 42), and forks hooks/post_tool_use.rb (line 53). tests/pipeline/subagent_start_hook_test.rb:93 and subagent_stop_hook_test.rb:90,95 also fork the runner. ## Reproduce ``` cd plugins/os-sdlc && ruby -Itests tests/pipeline/post_tool_use_non_pipeline_async_dispatch_test.rb -v ``` ## Expected One prose sentence: a guard test fails when any file under a plugin test tree spawns an executable under a plugins/**/bin/ directory and is not on its allowlist; the PostToolUse hang guard is 5 s around a single hook boot. ## Origin - Trigger: decomposition of #631, 2026-09-23 - Improvised this session: none - Chain: 30 s hang guard ← runner forks inside the guarded block (DESIGN, post_tool_use_hook_test_case.rb:37,42) ← 8615acc9 masked the boot cost - Root candidate: this ticket; hook-script forks are #632 - Where: plugins/os-sdlc/tests/support/post_tool_use_hook_test_case.rb; plugins/os-sdlc/tests/lsp_probe_real_daemon_guard_test.rb - Session: 5bb052ff-ddf2-4d47-a283-d00d071e9b79 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/5bb052ff-ddf2-4d47-a283-d00d071e9b79.jsonl ## Owner sketch ```ruby class OsSdlcOwnBinaryForkGuardTest < Minitest::Test # plugins/os-sdlc/tests/own_binary_fork_guard_test.rb ALLOWED_FILES = [].freeze def test_only_allowlisted_files_fork_a_repo_binary end class PostToolUseHookTestCase < Minitest::Test def run_cli(*args) def run_subagent_start(rest) end ``` ## Acceptance criteria 1. Given a test-tree file that references a repo bin/ executable path and calls Open3, Process.spawn, system, or backticks, and that is not in ALLOWED_FILES, when the guard runs, then it fails and names the file. 2. Given that only the allowlisted files spawn repo binaries, when the guard runs, then it passes. 3. Given PostToolUseHookTestCase, when a test opens an implementation and a dispatch, then both runner calls run in-process through InProcessRunnerCli and only the hook script is a subprocess. 4. Given the four PostToolUse async-dispatch tests, when read, then each `Timeout.timeout` argument is 5. ## Approach - Scan plugins/*/tests/**/*.rb and plugins/cc-architect/scripts/plugin_config/test/**/*.rb. Detect at file level, not line level: the `BIN` constant and the `capture3` call sit on different lines. Build the binary names from `Dir.glob("plugins/**/bin/*")`. A reference is a literal `bin/<name>` or a `"bin", "<name>"` join. - Expected allowlist after #638-#641, one reason per entry: each smoke test file; tests/support/evidence_bin_runner.rb, tests/support/cli_shared_helpers.rb, tests/lint_worklist_residual_cli_test.rb (unconverted binaries, follow-up); tests/support/lsp_probe_bin_runner.rb (real daemon, ADR-0205); false positives tests/lint/rubocop_report_fetcher_test.rb and tests/cops/bin_script_load_in_test_test.rb (bin path as data). TODO: unverified. Run the guard and confirm the list. - Timeout revert condition: one hooks/post_tool_use.rb boot must finish under 5 s under the full `ruby tests/all.rb` load. #624 measured 5-6 s per runner boot under 12-way load. Measure the hook boot before the revert. If it is over 5 s, stop and report; do not raise the value again. - ADRs to respect: ADR-0205 (guard pattern; no process-global writes), ADR-0195 (one Minitest class per file), ADR-0172/0128 (hooks stay thin couriers until #632). Required work: 1. Add a repo-wide guard test scanning plugin test trees (and the plugin_config test dir) for non-allowlisted files that fork a `plugins/**/bin/*` executable. (proven by AC 1, AC 2) 2. Convert `PostToolUseHookTestCase`'s runner calls to run in-process via `InProcessRunnerCli`, leaving only the hook script as a subprocess. (proven by AC 3) 3. Revert the four PostToolUse async-dispatch `Timeout.timeout` guards from 30 to 5, only after measuring one hook boot under full-suite load at under 5 s. (proven by AC 4; check: `grep -c 'Timeout.timeout(5)' plugins/os-sdlc/tests/pipeline/post_tool_use_async_dispatch_test.rb plugins/os-sdlc/tests/pipeline/post_tool_use_non_pipeline_async_dispatch_test.rb`) ## Structural follow-through - One History row in docs/implementation-status/os-sdlc.md.
Author
Owner

Body before Required-work migration, 2026-09-23

Map: #631
Blocked by #638, #639, #640, #641

Problem

Child of #631, split from the user's 631d so that each child routes to one pipeline. It mirrors plugins/os-sdlc/tests/lsp_probe_real_daemon_guard_test.rb (ADR-0205 item 5; commit a295f828).

Observed

Commit 8615acc9 (#624, 2026-09-22) raised Timeout.timeout(5) to 30 s in plugins/os-sdlc/tests/pipeline/post_tool_use_async_dispatch_test.rb:10 and post_tool_use_non_pipeline_async_dispatch_test.rb:9, 23, 37. Its message: "5 s Timeout around two ruby process boots in PostToolUse tests is a hang guard that load alone exceeded; raised to 30 s". Inside the guard, tests/support/post_tool_use_hook_test_case.rb forks bin/os-sdlc-runner through run_cli (line 37) and run_subagent_start (line 42), and forks hooks/post_tool_use.rb (line 53). tests/pipeline/subagent_start_hook_test.rb:93 and subagent_stop_hook_test.rb:90,95 also fork the runner.

Reproduce

cd plugins/os-sdlc && ruby -Itests tests/pipeline/post_tool_use_non_pipeline_async_dispatch_test.rb -v

Expected

One prose sentence: a guard test fails when any file under a plugin test tree spawns an executable under a plugins/**/bin/ directory and is not on its allowlist; the PostToolUse hang guard is 5 s around a single hook boot.

Origin

  • Trigger: decomposition of #631, 2026-09-23
  • Improvised this session: none
  • Chain: 30 s hang guard ← runner forks inside the guarded block (DESIGN, post_tool_use_hook_test_case.rb:37,42) ← 8615acc9 masked the boot cost
  • Root candidate: this ticket; hook-script forks are #632
  • Where: plugins/os-sdlc/tests/support/post_tool_use_hook_test_case.rb; plugins/os-sdlc/tests/lsp_probe_real_daemon_guard_test.rb
  • Session: 5bb052ff-ddf2-4d47-a283-d00d071e9b79
  • Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/5bb052ff-ddf2-4d47-a283-d00d071e9b79.jsonl

Owner sketch

class OsSdlcOwnBinaryForkGuardTest < Minitest::Test   # plugins/os-sdlc/tests/own_binary_fork_guard_test.rb
  ALLOWED_FILES = [].freeze
  def test_only_allowlisted_files_fork_a_repo_binary
end

class PostToolUseHookTestCase < Minitest::Test
  def run_cli(*args)
  def run_subagent_start(rest)
end

Acceptance criteria

  1. Given a test-tree file that references a repo bin/ executable path and calls Open3, Process.spawn, system, or backticks, and that is not in ALLOWED_FILES, when the guard runs, then it fails and names the file.
  2. Given that only the allowlisted files spawn repo binaries, when the guard runs, then it passes.
  3. Given PostToolUseHookTestCase, when a test opens an implementation and a dispatch, then both runner calls run in-process through InProcessRunnerCli and only the hook script is a subprocess.
  4. Given the four PostToolUse async-dispatch tests, when read, then each Timeout.timeout argument is 5.

Approach

  • Scan plugins//tests/**/.rb and plugins/cc-architect/scripts/plugin_config/test/**/*.rb. Detect at file level, not line level: the BIN constant and the capture3 call sit on different lines. Build the binary names from Dir.glob("plugins/**/bin/*"). A reference is a literal bin/<name> or a "bin", "<name>" join.
  • Expected allowlist after #638-#641, one reason per entry: each smoke test file; tests/support/evidence_bin_runner.rb, tests/support/cli_shared_helpers.rb, tests/lint_worklist_residual_cli_test.rb (unconverted binaries, follow-up); tests/support/lsp_probe_bin_runner.rb (real daemon, ADR-0205); false positives tests/lint/rubocop_report_fetcher_test.rb and tests/cops/bin_script_load_in_test_test.rb (bin path as data). TODO: unverified. Run the guard and confirm the list.
  • Timeout revert condition: one hooks/post_tool_use.rb boot must finish under 5 s under the full ruby tests/all.rb load. #624 measured 5-6 s per runner boot under 12-way load. Measure the hook boot before the revert. If it is over 5 s, stop and report; do not raise the value again.
  • ADRs to respect: ADR-0205 (guard pattern; no process-global writes), ADR-0195 (one Minitest class per file), ADR-0172/0128 (hooks stay thin couriers until #632).

Structural follow-through

  • One History row in docs/implementation-status/os-sdlc.md.
Body before Required-work migration, 2026-09-23 Map: #631 Blocked by #638, #639, #640, #641 ## Problem Child of #631, split from the user's 631d so that each child routes to one pipeline. It mirrors plugins/os-sdlc/tests/lsp_probe_real_daemon_guard_test.rb (ADR-0205 item 5; commit a295f828). ## Observed Commit 8615acc9 (#624, 2026-09-22) raised `Timeout.timeout(5)` to 30 s in plugins/os-sdlc/tests/pipeline/post_tool_use_async_dispatch_test.rb:10 and post_tool_use_non_pipeline_async_dispatch_test.rb:9, 23, 37. Its message: "5 s Timeout around two ruby process boots in PostToolUse tests is a hang guard that load alone exceeded; raised to 30 s". Inside the guard, tests/support/post_tool_use_hook_test_case.rb forks bin/os-sdlc-runner through `run_cli` (line 37) and `run_subagent_start` (line 42), and forks hooks/post_tool_use.rb (line 53). tests/pipeline/subagent_start_hook_test.rb:93 and subagent_stop_hook_test.rb:90,95 also fork the runner. ## Reproduce ``` cd plugins/os-sdlc && ruby -Itests tests/pipeline/post_tool_use_non_pipeline_async_dispatch_test.rb -v ``` ## Expected One prose sentence: a guard test fails when any file under a plugin test tree spawns an executable under a plugins/**/bin/ directory and is not on its allowlist; the PostToolUse hang guard is 5 s around a single hook boot. ## Origin - Trigger: decomposition of #631, 2026-09-23 - Improvised this session: none - Chain: 30 s hang guard ← runner forks inside the guarded block (DESIGN, post_tool_use_hook_test_case.rb:37,42) ← 8615acc9 masked the boot cost - Root candidate: this ticket; hook-script forks are #632 - Where: plugins/os-sdlc/tests/support/post_tool_use_hook_test_case.rb; plugins/os-sdlc/tests/lsp_probe_real_daemon_guard_test.rb - Session: 5bb052ff-ddf2-4d47-a283-d00d071e9b79 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/5bb052ff-ddf2-4d47-a283-d00d071e9b79.jsonl ## Owner sketch ```ruby class OsSdlcOwnBinaryForkGuardTest < Minitest::Test # plugins/os-sdlc/tests/own_binary_fork_guard_test.rb ALLOWED_FILES = [].freeze def test_only_allowlisted_files_fork_a_repo_binary end class PostToolUseHookTestCase < Minitest::Test def run_cli(*args) def run_subagent_start(rest) end ``` ## Acceptance criteria 1. Given a test-tree file that references a repo bin/ executable path and calls Open3, Process.spawn, system, or backticks, and that is not in ALLOWED_FILES, when the guard runs, then it fails and names the file. 2. Given that only the allowlisted files spawn repo binaries, when the guard runs, then it passes. 3. Given PostToolUseHookTestCase, when a test opens an implementation and a dispatch, then both runner calls run in-process through InProcessRunnerCli and only the hook script is a subprocess. 4. Given the four PostToolUse async-dispatch tests, when read, then each `Timeout.timeout` argument is 5. ## Approach - Scan plugins/*/tests/**/*.rb and plugins/cc-architect/scripts/plugin_config/test/**/*.rb. Detect at file level, not line level: the `BIN` constant and the `capture3` call sit on different lines. Build the binary names from `Dir.glob("plugins/**/bin/*")`. A reference is a literal `bin/<name>` or a `"bin", "<name>"` join. - Expected allowlist after #638-#641, one reason per entry: each smoke test file; tests/support/evidence_bin_runner.rb, tests/support/cli_shared_helpers.rb, tests/lint_worklist_residual_cli_test.rb (unconverted binaries, follow-up); tests/support/lsp_probe_bin_runner.rb (real daemon, ADR-0205); false positives tests/lint/rubocop_report_fetcher_test.rb and tests/cops/bin_script_load_in_test_test.rb (bin path as data). TODO: unverified. Run the guard and confirm the list. - Timeout revert condition: one hooks/post_tool_use.rb boot must finish under 5 s under the full `ruby tests/all.rb` load. #624 measured 5-6 s per runner boot under 12-way load. Measure the hook boot before the revert. If it is over 5 s, stop and report; do not raise the value again. - ADRs to respect: ADR-0205 (guard pattern; no process-global writes), ADR-0195 (one Minitest class per file), ADR-0172/0128 (hooks stay thin couriers until #632). ## Structural follow-through - One History row in docs/implementation-status/os-sdlc.md.
Author
Owner

Work started on branch ticket-642 (session 557d47dd). Ticket is test-only, so it is hand-finished in the worktree rather than pipeline-routed (#665: the map has no route for test-file edits).

Work started on branch ticket-642 (session 557d47dd). Ticket is test-only, so it is hand-finished in the worktree rather than pipeline-routed (#665: the map has no route for test-file edits).
Author
Owner

Deviation: hand-finished outside the pipeline (#665)

Problem. Every change in this ticket is a test-file edit: a new guard test, three hook tests converted from Open3 forks of bin/os-sdlc-runner to InProcessRunnerCli, and four Timeout.timeout arguments. The map poodr-ticket-implementation.yaml routes red diff-tests only to programmer, who may not edit tests, and the ticket body has no ## Approach section, so implementation-open would refuse it. #665 tracks the missing route.

Solution. A Sonnet agent implemented the ticket in worktree ticket-642 red-first: the guard failed and named each offender, then the three os-sdlc hook tests were converted in-process until only allowlisted files remained. Result on main: adfb114c.

Check Result
os-sdlc suite 1431 runs, 3262 assertions, 0 failures, 0 errors, 40.1 s
rubocop on changed files 0 offenses
Guard allowlist 17 files

Allowlist judgment. Seven os-backlog CLI contract tests are allowlisted because os-backlog has no in-process CLI seam (os-sdlc, os-aidd-lint, cc-architect got theirs from #638, #639, #641). Building one is outside this ticket. Two entries (cops/bin_script_load_in_test_test.rb, lint/rubocop_report_fetcher_test.rb) are false positives: the bin path is fixture text, not a spawn.

## Deviation: hand-finished outside the pipeline (#665) **Problem.** Every change in this ticket is a test-file edit: a new guard test, three hook tests converted from `Open3` forks of `bin/os-sdlc-runner` to `InProcessRunnerCli`, and four `Timeout.timeout` arguments. The map `poodr-ticket-implementation.yaml` routes red diff-tests only to `programmer`, who may not edit tests, and the ticket body has no `## Approach` section, so `implementation-open` would refuse it. #665 tracks the missing route. **Solution.** A Sonnet agent implemented the ticket in worktree `ticket-642` red-first: the guard failed and named each offender, then the three os-sdlc hook tests were converted in-process until only allowlisted files remained. Result on main: adfb114c. | Check | Result | |---|---| | os-sdlc suite | 1431 runs, 3262 assertions, 0 failures, 0 errors, 40.1 s | | rubocop on changed files | 0 offenses | | Guard allowlist | 17 files | **Allowlist judgment.** Seven os-backlog CLI contract tests are allowlisted because os-backlog has no in-process CLI seam (os-sdlc, os-aidd-lint, cc-architect got theirs from #638, #639, #641). Building one is outside this ticket. Two entries (`cops/bin_script_load_in_test_test.rb`, `lint/rubocop_report_fetcher_test.rb`) are false positives: the bin path is fixture text, not a spawn.
Author
Owner

Resolution

Done: Guard test tests/own_binary_fork_guard_test.rb fails when a plugin test file spawns a plugins/**/bin/ executable outside its allowlist; PostToolUseHookTestCase and the subagent_start/subagent_stop hook tests call the runner in-process through InProcessRunnerCli, only hooks/post_tool_use.rb forks; all four PostToolUse hang guards are Timeout.timeout(5).

Evidence: main adfb114c (branch ticket-642, fast-forward). os-sdlc suite 1431 runs, 3262 assertions, 0 failures, 0 errors, 40.1 s wall. rubocop 0 offenses on changed files. Hand-finished outside the pipeline because the change is test-only and the map has no test-file repair route (#665); write-up in the comment above.

Follow-ups: os-backlog in-process CLI seam so its seven allowlisted contract tests leave the guard allowlist: dropped, no ticket-skeptic verdict yet and the allowlist already enforces the boundary; revisit when os-backlog tests are worked. Pipeline defect: #665 (already filed).

## Resolution **Done:** Guard test tests/own_binary_fork_guard_test.rb fails when a plugin test file spawns a plugins/**/bin/ executable outside its allowlist; PostToolUseHookTestCase and the subagent_start/subagent_stop hook tests call the runner in-process through InProcessRunnerCli, only hooks/post_tool_use.rb forks; all four PostToolUse hang guards are Timeout.timeout(5). **Evidence:** main adfb114c (branch ticket-642, fast-forward). os-sdlc suite 1431 runs, 3262 assertions, 0 failures, 0 errors, 40.1 s wall. rubocop 0 offenses on changed files. Hand-finished outside the pipeline because the change is test-only and the map has no test-file repair route (#665); write-up in the comment above. **Follow-ups:** os-backlog in-process CLI seam so its seven allowlisted contract tests leave the guard allowlist: dropped, no ticket-skeptic verdict yet and the allowlist already enforces the boundary; revisit when os-backlog tests are worked. Pipeline defect: #665 (already filed).
jared closed this issue 2026-09-24 13:14:31 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
jared/cc-os#642
No description provided.