jared/cc-os
Watch
1
0
Fork
You've already forked cc-os
 0
Code Issues 378 Pull requests Projects Releases Packages Wiki Activity

CLI invocation seam + in-process CLI tests #631

New issue
Closed
opened 2026-09-23 12:03:43 +00:00 by jared · 2 comments
jared commented 2026-09-23 12:03:43 +00:00
Owner
Copy link

Context

Follow-up to #624 (merged main cf017e0c, ADR-0205), which cut suite wall time from 112-129 s to 45-53 s. The user asked for this ticket directly after reviewing the remaining profile.

Observed

About 35 test files fork ruby bin/<own-cli> per test. Each fork boots Ruby, Bundler, Zeitwerk, and runs a SQLite schema migration, costing 5-6 s under 12-way parallel load. Session c15fe063 measured: CliNextInstructionTest 43 s, CliImplementationOpenTest 42 s, CliSubagentStopTest 22 s, OsSdlcEvidenceBinTest 17 s. os-aidd-lint's tests/aidd_lint_bin_test.rb runs 13 forks of bin/aidd-lint at 2.7 s solo. A hang-guard timeout around two Ruby boots was raised from 5 s to 30 s in c15fe063 to mask boot cost instead of removing it.

plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb reads Dir.pwd or $stdin directly in ten Thor command methods instead of receiving them as parameters:

plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:31  emit(DispatchCommands.implementation_open(Dir.pwd, args, target: options[:target]))
plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:39  emit(DispatchCommands.next_instruction(Dir.pwd, args))
plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:51  def subagent_stop = emit(DispatchCommands.subagent_stop_from_stdin(Dir.pwd, $stdin))
plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:54  def gate(*args) = emit(DispatchCommands.gate(Dir.pwd, args))
plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:59  emit(DispatchCommands.brief(Dir.pwd, options[:dispatch_id]))
plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:64  result = DispatchCommands.retired_tests_check(Dir.pwd, args)
plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:71  def implementation_close(*args) = emit(DispatchCommands.implementation_close(Dir.pwd, args))
plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:76  puts DispatchCommands.violations(Dir.pwd, args, format: options[:format])
plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:87  DispatchCommands.sweep_missing_usage(Dir.pwd, args)
plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:91  emit(DispatchCommands.implementation_report(Dir.pwd, args))

Because these methods read process globals, the only way a test can vary cwd or stdin is to fork a real subprocess. That fork is the 5-6 s cost above.

Related: #628 (open) already covers the os-sdlc-specific slice of this same problem (per-test binary fork plus a missing template-DB copy). This ticket supplies the general invocation seam #628's in-process tests need as their foundation, then extends the same fix to os-aidd-lint, os-adr, and cc-architect. Do not duplicate #628's os-sdlc task list; this ticket's os-sdlc scope is "apply the seam," #628 owns the template-DB and per-binary smoke-test details.

Reproduce

cd plugins/os-sdlc && ruby tests/all.rb -v 2>&1 | sort -t= -k2 -rn | head -20

Expected

def test_implementation_open_runs_in_process_without_forking
  invocation = OsSdlc::CLI::Invocation.new(cwd: tmp_project, stdin: StringIO.new, stdout: out, stderr: err, env: {})
  OsSdlc::Pipeline::Cli.new(invocation).implementation_open(*args)
  assert_no_subprocess_forked
end

Illustration (optional)

# illustration, not spec
class OsSdlc::CLI::Invocation
  attr_reader :cwd, :stdin, :stdout, :stderr, :env
  def self.from_process = new(cwd: Dir.pwd, stdin: $stdin, stdout: $stdout, stderr: $stderr, env: ENV.to_h)
end
# bin/os-sdlc-runner: Pipeline::Cli.start(ARGV, invocation: OsSdlc::CLI::Invocation.from_process)

Solution

Give each CLI an invocation object (cwd, stdin, stdout, stderr, env); only bin/* entry points read real process globals and construct that object. Tests call the CLI command class in-process. Keep exactly one real-binary smoke test per binary. Revert the c15fe063 hang-guard timeout from 30 s back to 5 s. Add a guard test that fails when a test forks one of this repo's own binaries outside a named allowlist, mirroring the daemon-spawn guard added in #624/ADR-0205.

Scope order: os-sdlc first (lib/os_sdlc/pipeline/cli.rb and its Cli*Test files, coordinating with #628's template-DB work), then os-aidd-lint (tests/aidd_lint_bin_test.rb), then os-adr (os-adr/tests/cli_test.rb), then cc-architect plugin_config tests.

Acceptance criteria

  • os-sdlc CLI Thor methods take an injected invocation object; Dir.pwd/$stdin reads move to bin/os-sdlc-runner only.
  • Cli*Test files call the command class in-process, not via subprocess fork.
  • One real-binary smoke test remains per os-sdlc binary.
  • Same seam applied to os-aidd-lint, os-adr, cc-architect plugin_config tests.
  • Hang-guard timeout reverted 30 s -> 5 s.
  • A guard test fails on any un-allowlisted fork of this repo's own binaries.

Origin

  • Trigger: profile run after #624, session c15fe063 (2026-09-22); this ticket requested directly by the user in session 16bf37b3-8cff-4121-896c-4a5fc61f2e9e.
  • Improvised this session: none
  • Chain: slow suite <- per-test process boot in Cli*Test <- DESIGN (tests exercise the binary instead of the command object, ADR-0205 removed serialization but not the fork).
  • Root candidate: this ticket
  • Where: plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb; plugins/os-sdlc/tests/pipeline/cli_*_test.rb
  • Session: c15fe063-c64c-417b-b4e1-53a52ffca319 (suite-speed work); 16bf37b3-8cff-4121-896c-4a5fc61f2e9e (shell-call/ENV audit that produced this ticket)
  • Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/c15fe063-c64c-417b-b4e1-53a52ffca319.jsonl ; /home/jared/.claude/projects/-home-jared-dev-cc-os/16bf37b3-8cff-4121-896c-4a5fc61f2e9e.jsonl

Relationship to #628

#628 is open and covers the os-sdlc-specific in-process test conversion plus copying a pre-migrated template SQLite DB per test. This ticket supplies the general invocation-seam mechanism #628's in-process tests depend on, and extends the same fix to three other plugins. Reference #628; do not duplicate its task list.

## Context Follow-up to #624 (merged main cf017e0c, ADR-0205), which cut suite wall time from 112-129 s to 45-53 s. The user asked for this ticket directly after reviewing the remaining profile. ## Observed About 35 test files fork `ruby bin/<own-cli>` per test. Each fork boots Ruby, Bundler, Zeitwerk, and runs a SQLite schema migration, costing 5-6 s under 12-way parallel load. Session c15fe063 measured: CliNextInstructionTest 43 s, CliImplementationOpenTest 42 s, CliSubagentStopTest 22 s, OsSdlcEvidenceBinTest 17 s. os-aidd-lint's tests/aidd_lint_bin_test.rb runs 13 forks of bin/aidd-lint at 2.7 s solo. A hang-guard timeout around two Ruby boots was raised from 5 s to 30 s in c15fe063 to mask boot cost instead of removing it. plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb reads Dir.pwd or $stdin directly in ten Thor command methods instead of receiving them as parameters: ``` plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:31 emit(DispatchCommands.implementation_open(Dir.pwd, args, target: options[:target])) plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:39 emit(DispatchCommands.next_instruction(Dir.pwd, args)) plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:51 def subagent_stop = emit(DispatchCommands.subagent_stop_from_stdin(Dir.pwd, $stdin)) plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:54 def gate(*args) = emit(DispatchCommands.gate(Dir.pwd, args)) plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:59 emit(DispatchCommands.brief(Dir.pwd, options[:dispatch_id])) plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:64 result = DispatchCommands.retired_tests_check(Dir.pwd, args) plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:71 def implementation_close(*args) = emit(DispatchCommands.implementation_close(Dir.pwd, args)) plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:76 puts DispatchCommands.violations(Dir.pwd, args, format: options[:format]) plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:87 DispatchCommands.sweep_missing_usage(Dir.pwd, args) plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb:91 emit(DispatchCommands.implementation_report(Dir.pwd, args)) ``` Because these methods read process globals, the only way a test can vary cwd or stdin is to fork a real subprocess. That fork is the 5-6 s cost above. Related: #628 (open) already covers the os-sdlc-specific slice of this same problem (per-test binary fork plus a missing template-DB copy). This ticket supplies the general invocation seam #628's in-process tests need as their foundation, then extends the same fix to os-aidd-lint, os-adr, and cc-architect. Do not duplicate #628's os-sdlc task list; this ticket's os-sdlc scope is "apply the seam," #628 owns the template-DB and per-binary smoke-test details. ## Reproduce ``` cd plugins/os-sdlc && ruby tests/all.rb -v 2>&1 | sort -t= -k2 -rn | head -20 ``` ## Expected ```ruby def test_implementation_open_runs_in_process_without_forking invocation = OsSdlc::CLI::Invocation.new(cwd: tmp_project, stdin: StringIO.new, stdout: out, stderr: err, env: {}) OsSdlc::Pipeline::Cli.new(invocation).implementation_open(*args) assert_no_subprocess_forked end ``` ## Illustration (optional) ```ruby # illustration, not spec class OsSdlc::CLI::Invocation attr_reader :cwd, :stdin, :stdout, :stderr, :env def self.from_process = new(cwd: Dir.pwd, stdin: $stdin, stdout: $stdout, stderr: $stderr, env: ENV.to_h) end # bin/os-sdlc-runner: Pipeline::Cli.start(ARGV, invocation: OsSdlc::CLI::Invocation.from_process) ``` ## Solution Give each CLI an invocation object (cwd, stdin, stdout, stderr, env); only `bin/*` entry points read real process globals and construct that object. Tests call the CLI command class in-process. Keep exactly one real-binary smoke test per binary. Revert the c15fe063 hang-guard timeout from 30 s back to 5 s. Add a guard test that fails when a test forks one of this repo's own binaries outside a named allowlist, mirroring the daemon-spawn guard added in #624/ADR-0205. Scope order: os-sdlc first (lib/os_sdlc/pipeline/cli.rb and its Cli*Test files, coordinating with #628's template-DB work), then os-aidd-lint (tests/aidd_lint_bin_test.rb), then os-adr (os-adr/tests/cli_test.rb), then cc-architect plugin_config tests. ## Acceptance criteria - [ ] os-sdlc CLI Thor methods take an injected invocation object; Dir.pwd/$stdin reads move to bin/os-sdlc-runner only. - [ ] Cli*Test files call the command class in-process, not via subprocess fork. - [ ] One real-binary smoke test remains per os-sdlc binary. - [ ] Same seam applied to os-aidd-lint, os-adr, cc-architect plugin_config tests. - [ ] Hang-guard timeout reverted 30 s -> 5 s. - [ ] A guard test fails on any un-allowlisted fork of this repo's own binaries. ## Origin - Trigger: profile run after #624, session c15fe063 (2026-09-22); this ticket requested directly by the user in session 16bf37b3-8cff-4121-896c-4a5fc61f2e9e. - Improvised this session: none - Chain: slow suite <- per-test process boot in Cli*Test <- DESIGN (tests exercise the binary instead of the command object, ADR-0205 removed serialization but not the fork). - Root candidate: this ticket - Where: plugins/os-sdlc/lib/os_sdlc/pipeline/cli.rb; plugins/os-sdlc/tests/pipeline/cli_*_test.rb - Session: c15fe063-c64c-417b-b4e1-53a52ffca319 (suite-speed work); 16bf37b3-8cff-4121-896c-4a5fc61f2e9e (shell-call/ENV audit that produced this ticket) - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/c15fe063-c64c-417b-b4e1-53a52ffca319.jsonl ; /home/jared/.claude/projects/-home-jared-dev-cc-os/16bf37b3-8cff-4121-896c-4a5fc61f2e9e.jsonl ## Relationship to #628 #628 is open and covers the os-sdlc-specific in-process test conversion plus copying a pre-migrated template SQLite DB per test. This ticket supplies the general invocation-seam mechanism #628's in-process tests depend on, and extends the same fix to three other plugins. Reference #628; do not duplicate its task list.
jared commented 2026-09-23 13:06:24 +00:00
Author
Owner
Copy link

Decomposed 2026-09-23 into children (each body has Map: #631):

  • #638 os-sdlc: Pipeline::CLI receives OsSdlc::CLI::Invocation; bin/os-sdlc-runner is the only reader of process globals; Cli*Test forks go in-process; one runner smoke test. Absorbs #628's in-process scope. First.
  • #639 os-aidd-lint: bin/aidd-lint logic moves to AiddLint::CLI; 20 bin tests go in-process; one smoke test. Blocked by #638 (ADR only).
  • #640 os-adr: adr-new/adr-init move to Adr::CLI::New/Init; cli_test goes in-process; one smoke test per binary. Blocked by #638; needs an os-adr target in .sdlc/project.yaml.
  • #641 cc-architect: PluginConfig::Cli moves out of bin/plugin-config with cwd/HOME injected; 19 CLI tests go in-process. Blocked by #638.
  • #642 repo-wide own-binary fork guard, hook-harness runner calls in-process, hang-guard 30 s -> 5 s. Test-only (ready-for-human). Blocked by #638-#641.

b, c, and d can run in parallel after a. #628 is narrowed to the template-DB copy.
Not yet ticketed: in-process tests for the other os-sdlc binaries (evidence, os-sdlc project-config, lint-worklist); the cc-architect plugin_config.rb and validate_marketplaces.rb entry points.

Decomposed 2026-09-23 into children (each body has `Map: #631`): - #638 os-sdlc: Pipeline::CLI receives OsSdlc::CLI::Invocation; bin/os-sdlc-runner is the only reader of process globals; Cli*Test forks go in-process; one runner smoke test. Absorbs #628's in-process scope. First. - #639 os-aidd-lint: bin/aidd-lint logic moves to AiddLint::CLI; 20 bin tests go in-process; one smoke test. Blocked by #638 (ADR only). - #640 os-adr: adr-new/adr-init move to Adr::CLI::New/Init; cli_test goes in-process; one smoke test per binary. Blocked by #638; needs an os-adr target in .sdlc/project.yaml. - #641 cc-architect: PluginConfig::Cli moves out of bin/plugin-config with cwd/HOME injected; 19 CLI tests go in-process. Blocked by #638. - #642 repo-wide own-binary fork guard, hook-harness runner calls in-process, hang-guard 30 s -> 5 s. Test-only (ready-for-human). Blocked by #638-#641. b, c, and d can run in parallel after a. #628 is narrowed to the template-DB copy. Not yet ticketed: in-process tests for the other os-sdlc binaries (evidence, os-sdlc project-config, lint-worklist); the cc-architect plugin_config.rb and validate_marketplaces.rb entry points.
jared commented 2026-09-24 15:44:24 +00:00
Author
Owner
Copy link

Resolution

Done: Superseded by its children, all closed and merged: #638 Invocation seam in os-sdlc (main 4b0ef96c, ADR-0207), #639 os-aidd-lint in-process CLI (main 7ff78767), #640 os-adr (closed, no os-adr target), #641 cc-architect PluginConfig::Cli + Invocation (main 7414b978), #642 repo-wide fork guard, in-process hook tests, 5 s hang guard (main adfb114c). No work remains on this parent.

Evidence: Each child ticket carries its own os-backlog close comment with commits and suite counts. os-sdlc suite on main 0fe1ee9a: 1383 runs, 0 failures; own_binary_fork_guard_test.rb enforces the no-fork boundary this ticket asked for. Session 557d47dd, 2026-09-24.

Follow-ups: none: the children's follow-ups (#665 test-file repair route, #670 gate output cap, #662 charter brief-fetch carve-out, #628 template-DB copy) are filed on their own tickets.

## Resolution **Done:** Superseded by its children, all closed and merged: #638 Invocation seam in os-sdlc (main 4b0ef96c, ADR-0207), #639 os-aidd-lint in-process CLI (main 7ff78767), #640 os-adr (closed, no os-adr target), #641 cc-architect PluginConfig::Cli + Invocation (main 7414b978), #642 repo-wide fork guard, in-process hook tests, 5 s hang guard (main adfb114c). No work remains on this parent. **Evidence:** Each child ticket carries its own os-backlog close comment with commits and suite counts. os-sdlc suite on main 0fe1ee9a: 1383 runs, 0 failures; own_binary_fork_guard_test.rb enforces the no-fork boundary this ticket asked for. Session 557d47dd, 2026-09-24. **Follow-ups:** none: the children's follow-ups (#665 test-file repair route, #670 gate output cap, #662 charter brief-fetch carve-out, #628 template-DB copy) are filed on their own tickets.
jared closed this issue 2026-09-24 15:44:25 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
ticket-1337
No results found.
Labels
Clear labels   
P0
Priority 0 — drop everything

  
P1
Priority 1 — next up

  
P2
Priority 2 — normal

  
P3
Priority 3 — someday

  
bug
Something is broken

  
category/code-change
Ticket category

  
category/cross-repo
Ticket category

  
category/eval-run
Ticket category

  
category/investigation
Ticket category

  
category/no-test-target
Ticket category

  
category/owner-ruling
Ticket category

  
category/prose-artifact
Ticket category

  
category/recurring-chore
Ticket category

  
category/structure

   
category/umbrella
Ticket category

  
crc-refactor
CRC refactor run work record; exempt from the ticket-skeptic

  
create
Ticket creates something new

  
delete
Ticket removes something

  
enhancement
New feature or improvement

  
filed-by/agent

   
filed-by/user

   
frozen
Pre-reset backlog freeze 2026-08-16; do not work unless rediscovered by a live run

  
lint-rule
Ticket proposes/modifies a lint rule or cop

  
needs-info
Waiting on reporter for more information

  
needs-triage
Maintainer needs to evaluate this issue

  
next
Human-curated next up — AI never applies or removes

  
plugin/cc-architect
Scope: the cc-architect plugin

  
plugin/os
Scope: the os plugin

  
plugin/os-adr
Scope: the os-adr plugin

  
plugin/os-aidd-lint
Scope: the os-aidd-lint plugin

  
plugin/os-backlog
Scope: the os-backlog plugin

  
plugin/os-context
Scope: the os-context plugin

  
plugin/os-doc-hygiene
Scope: the os-doc-hygiene plugin

  
plugin/os-sdlc
Scope: the os-sdlc plugin

  
plugin/os-vault
Scope: the os-vault plugin

  
project/cc-os
Parent-scope: cc-os repo-wide, not specific to one plugin

  
ready-for-agent
Triage complete; an agent can pick this up

  
ready-for-human
Human-owned; never picked up autonomously

  
recurring
Recurring flavor: occurrence outcomes commented, never closed by AI

  
review
Shipped semi work awaiting human sign-off

  
rule-proposal
A rule change that survived the gate; Jared reads these when he chooses

  
update
Ticket updates something existing

  
waiting
Blocked; blocker named in a comment

  
wayfinder:grilling

   
wayfinder:map

   
wayfinder:map
wayfinder map ticket

  
wayfinder:research

   
wayfinder:task

   
wayfinder:task
wayfinder map child: task

  
wontfix
Will not be actioned

  
worklist/deviations
Deviation incidents from skill/pipeline runs — evidence pile for correcting the owning artifact

  
worklist/lint-rule
Proposed os-sdlc or os-aidd-lint rule

  
worklist/new-implement-build
Updated implementation harness (ast-grep + stepper)

No labels
P0
P1
P2
P3
bug
category/code-change
category/cross-repo
category/eval-run
category/investigation
category/no-test-target
category/owner-ruling
category/prose-artifact
category/recurring-chore
category/structure
category/umbrella
crc-refactor
create
delete
enhancement
filed-by/agent
filed-by/user
frozen
lint-rule
needs-info
needs-triage
next
plugin/cc-architect
plugin/os
plugin/os-adr
plugin/os-aidd-lint
plugin/os-backlog
plugin/os-context
plugin/os-doc-hygiene
plugin/os-sdlc
plugin/os-vault
project/cc-os
ready-for-agent
ready-for-human
recurring
review
rule-proposal
update
waiting
wayfinder:grilling
wayfinder:map
wayfinder:map
wayfinder:research
wayfinder:task
wayfinder:task
wontfix
worklist/deviations
worklist/lint-rule
worklist/new-implement-build
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
jared/cc-os#631
No description provided.