Add a cop that flags tests starting our own plugin bin or hook as a subprocess #1337
Labels
No labels
P0
P1
P2
P3
bug
category/code-change
category/cross-repo
category/eval-run
category/investigation
category/no-test-target
category/owner-ruling
category/prose-artifact
category/recurring-chore
category/structure
category/umbrella
crc-refactor
create
delete
enhancement
filed-by/agent
filed-by/user
frozen
lint-rule
needs-info
needs-triage
next
plugin/cc-architect
plugin/os
plugin/os-adr
plugin/os-aidd-lint
plugin/os-backlog
plugin/os-context
plugin/os-doc-hygiene
plugin/os-sdlc
plugin/os-vault
project/cc-os
ready-for-agent
ready-for-human
recurring
review
rule-proposal
update
waiting
wayfinder:grilling
wayfinder:map
wayfinder:map
wayfinder:research
wayfinder:task
wayfinder:task
wontfix
worklist/deviations
worklist/lint-rule
worklist/new-implement-build
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
jared/cc-os#1337
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem
A test that starts this repository's own script as a process pays one Ruby boot per test, and two failures go silent: data the test seeds in its own process does not reach the script, and a crash exits 1 like an expected failure. Jared ruled on 2026-10-25 that a test calls the entry in-process and that one smoke test per script starts it as a process. On 2026-10-26 he ruled that the cop must be general (smell analysis section 8, option B): it must not name an entry class for each plugin, and it must not hard-code folder or file-name lists. Nothing stops a new test from starting its own script today. The
Sdlc/Structural/ExecutableLoadedInTestmessage still says scripts are tested "only as subprocesses", which contradicts the ruling and that cop's own lesson.One test outside a smoke file starts its script today:
plugins/os-sdlc/tests/detector/holders_bin_test.rb:56. It stays a process becauseDetector::Holders::SiteOffensesbuildsDetector::CopSiteswith noruby_version:, soCopSource.target_versionasks RuboCop, which finds.ruby-versionthrough the process working folder and memoizes it once per process.Approach
Add the cop
Sdlc/Structural/OwnEntryPointSpawnedInTest. It reports a call in a test file when the call starts a process (Open3.*,system,spawn,IO.popen, backticks) and one argument resolves to an entry point of this repository. An entry point is a file whose first line starts with#!or a file in a folder namedbin: the same definition asSdlc/Structural/ExitOutsideEntryPoint. For a spawned path, the shebang check reads the first line of the resolved file on disk. A test file is a file whose name ends in_test.rbor starts withtest_. Extract that definition into one object that both cops use. The argument resolves when it is aFile.expand_path(..., __dir__)call or a constant assigned one in the same file; extendExecutableLoadedInTest::ResolvedPathor move it so both cops share it. The cop config keySmokeTestPatterns(default["*_bin_smoke_test.rb", "test_*_bin_smoke.rb"]) names the smoke files, where the cop reports nothing. No Exclude list.The message prescribes the shape, not a class: "Call the script's entry in the test process: arguments and streams in, status out. Only the smoke test starts the script."
Make
holders_bin_test.rbcall the entry in-process:SiteOffensespasses the Ruby version read from the target root's.ruby-versiontoCopSites. Delete the two-line comment above the spawn with the spawn.Reword the
ExecutableLoadedInTestmessage to match its lesson's correction: a test calls the library entry; one smoke test runs the script as a process.Add the lint lesson and its smell analysis. The smell analysis is the 2026-10-26 one (scratch copy in the session; write it to
docs/lint-lessons/analyses/sdlc-structural-own-entry-point-spawned-in-test.md).Required work:
plugins/os-sdlc/.rubocop.ymlwith no Exclude list. check: presentOwnEntryPointSpawnedInTestinplugins/os-sdlc/.rubocop.ymlSmokeTestPatternsto the cop config. check: presentSmokeTestPatternsinplugins/os-sdlc/.rubocop.ymlExecutableLoadedInTestmessage. check: absentonly as subprocessesinplugins/os-sdlc/lib/os_sdlc/cops/structural/executable_loaded_in_test.rbOpen3.capture2(SCRIPTinplugins/os-sdlc/tests/detector/holders_bin_test.rbSdlc/Structural/OwnEntryPointSpawnedInTestinplugins/os-sdlc/lint-lessons/sdlc-structural-own-entry-point-spawned-in-test.ymlOwnEntryPointSpawnedInTestindocs/lint-lessons/analyses/sdlc-structural-own-entry-point-spawned-in-test.mdAcceptance criteria
SmokeTestPatternsand callsOpen3.capture2e(BIN)whereBIN = File.expand_path("../bin/tool", __dir__)in the same file, when rubocop runs, then the cop reports one offense with the message above.#!/usr/bin/env ruby, outside abinfolder, withOpen3.capture3("ruby", HOOK), when rubocop runs, then the cop reports one offense.SmokeTestPatternsand starts its script, when rubocop runs, then the cop reports no offense.git,RbConfig.ruby, "-e", code, or a path held in another file's constant, when rubocop runs, then the cop reports no offense..ruby-versiondiffers from the process working folder's, when the holders detector runs in-process for that root, then it parses with the target's Ruby version.plugins/, then the cop reports zero offenses.Owner sketch
Non-goals
Record and replay of external tools (
rubocop, the suite command) in tests. That is a separate time saving and is not measured.Origin
Unblocked: #1329-#1336 are merged (main
c9f5db8f) and closed. Known spawn sites the cop's zero-offense criterion (AC5) must meet:Source: docs/crc/os-sdlc/fast-batch-15/batch-state.md, Open threads.
Work started on branch ticket-1337 (os-sdlc implement pipeline). The last known spawn site (os_sdlc_lint_bin_test.rb) is converted on main
f4cfb8fb.Blocked by #1226. Implementation 177 escalated (83, behavior-verifier bound). Hand-finished on branch ticket-1337 (
714fe599, pushed): AC1-AC4 hold, os-sdlc suite 2750 runs green. AC5 fails on one offense: plugins/os-sdlc/tests/detector/holders_bin_test.rb:56, the Open3 run kept on purpose until #1226 (CopSites reads the target Ruby version from the process cwd; RuboCop memoizes it in-process). No Exclude or inline disable, per house rule. Resume: after #1226, convert holders_bin_test to InProcessBin, run the cop over the repo (0 offenses), one design audit of the branch, merge. The branch also replaces the Minitest/NoTestCases Exclude with Include: [/test//, /tests//] (cop files in lib/ named *_test.rb were false positives); confirm at merge. Body paths corrected to cops/structural/, plugins/os-sdlc/.rubocop.yml, and the sdlc-structural- lesson name.Removed the stale
waitinglabel and the "Blocked by: #1226" line. The holders change in this ticket (SiteOffensespasses the Ruby version from the target root's.ruby-versiontoCopSites) removes the dependency on #1226. Implementation starts now with option B (general copSdlc/Structural/OwnEntryPointSpawnedInTest).Work started on branch
ticket-1337(fresh from maine12f9297). The old narrow-design branch is kept asticket-1337-narrowfor reference.Follow-up dropped by ticket-skeptic: SiteOffenses reads .ruby-version, while RuboCop gives a target .rubocop.yml TargetRubyVersion priority. No target has both with different values, so the need is speculative (design-audit minor item, A12).
Resolution
Done: Added cop Sdlc/Structural/OwnEntryPointSpawnedInTest (option B): it reports a test file (name test.rb or test.rb) that starts a process (Open3.*, system, spawn, IO.popen, backticks) on an entry point of this repository (first line #!, or a file in a bin folder), resolved from File.expand_path(..., dir) or a same-file constant. Config key SmokeTestPatterns (default bin_smoke_test.rb, test_bin_smoke.rb); no Exclude list. EntryPoint shared with ExitOutsideEntryPoint; ResolvedPath moved to Structural. ExecutableLoadedInTest message reworded. Holders::SiteOffenses passes the target root's .ruby-version to CopSites; holders_bin_test calls the entry in-process. Two more true hits made in-process: os-backlog checkout_test (env: passed to the CLI) and os-me reply_shape (run.rb is one line calling ReplyShape::EvalHarness; one smoke test). Lesson and smell analysis added. Also Minitest/NoTestCases Include for #1445.
Evidence: main
23b064dc(pushed). os-sdlc suite 2836 runs green in 35 s; os-me 443 and os-backlog 344 runs green. rubocop --only Sdlc/Structural/OwnEntryPointSpawnedInTest plugins/: 2750 files, no offenses. Lint worklist counts for os-sdlc, os-me, os-backlog are at or below main's. Hand-finished after the session-ID collision (#1447); implementation 201 abandoned.Follow-ups: none (SiteOffenses .ruby-version precedence follow-up was dropped by ticket-skeptic)