Revisit test fixture and factory strategy across plugin test suites #1019
Labels
No labels
P0
P1
P2
P3
bug
category/code-change
category/cross-repo
category/eval-run
category/investigation
category/no-test-target
category/owner-ruling
category/prose-artifact
category/recurring-chore
category/structure
category/umbrella
crc-refactor
create
delete
enhancement
filed-by/agent
filed-by/user
frozen
lint-rule
needs-info
needs-triage
next
plugin/cc-architect
plugin/os
plugin/os-adr
plugin/os-aidd-lint
plugin/os-backlog
plugin/os-context
plugin/os-doc-hygiene
plugin/os-sdlc
plugin/os-vault
project/cc-os
ready-for-agent
ready-for-human
recurring
review
rule-proposal
update
waiting
wayfinder:grilling
wayfinder:map
wayfinder:map
wayfinder:research
wayfinder:task
wayfinder:task
wontfix
worklist/deviations
worklist/lint-rule
worklist/new-implement-build
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
jared/cc-os#1019
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem
Test fixture practice is scattered across the plugin test suites.
Suites share fixtures per concern, not per domain object, and sharing stops at each plugin boundary.
No lens or convention document prescribes fixture shape.
The only test-lens text on fixtures warns about bare count pins.
Evidence of duplication, from a survey on 2026-10-16:
.sdlc/project.yamlwith targets is hand-built at 15 sites: 11 os-sdlc tests, 4 os-sdlctests/supportfiles, and all 4 takedown-eval tests that need it. Meanwhile 48 os-sdlc tests useProjectScenario.TicketFixture::COMPLETE_BODYexists in 4 copies:tests/support/ticket_fixture.rb:7,command/implementation/open_ticket_shape_test.rb:10,command/implementation/open_map_name_test.rb:8,parser/ticket_shape_parity_test.rb:7.FakeRunneris defined 4 times with different signatures: os-sdlctests/test_helper.rb:114, takedowntest/run_support.rb:12, os-metest/test_helper.rb:72,tests/adapter/probe/lsp_test.rb:12.Target.new(name:is hand-built in 3 files with no helper.test/command/final_suite_test.rbholds the same two-target Hash in twosetupmethods (lines 38 and 206).The one place that matches the per-domain-object model is os-adr:
AdrFixtures#with_projectandsample_adr(id:, title:, status:, ...).The takedown eval cannot reach
plugins/os-sdlc/tests/support, because ADR-0297 keeps the eval to its own difference from the os-sdlc library. Shared test support has no home that both suites can load.Reproduce
n/a: design decision ticket; the evidence above comes from a survey, not a failing command.
Approach
This is a decision ticket, not a build ticket. Expected output: a decision memo to Jared, then ADR(s) through
/os-adr:createif a one-way door is chosen, then build tickets.Open questions:
Required work: none beyond the acceptance criteria.
Acceptance criteria
/os-adr:create.Non-goals
Origin
Decision memo and implementation plan (2026-10-28)
Rulings by Jared (prediction accepted):
OsSdlc::Project::Target), no fixture.build(**overrides).tests/support, modules named<Record>Fixture, one constant per file, Zeitwerk-loaded (ADR-0241). Delivery: a ~30-line reference under lens-testing, one link sentence in lens-testing SKILL.md, one index line in the test-writer agent. No cop up front; re-measure after the first pipeline batch that writes tests. Reference stays general so other plugins adopt it unchanged. No ADR: none of these is a one-way door. The ADR-0297 premise in this ticket is stale: 0297 is retired.Rules (go in the reference):
build(**overrides)passes overrides to the real constructor and never re-states a constructor default..subon a returned value.Re-verified evidence 2026-10-28 (os-sdlc): 16 test files hand-build a complete ticket body; nothing references
TicketFixture::COMPLETE_BODY;TicketFixture.seeddoes disk I/O (20 callers); 10 hand-built project.yaml-with-targets sites (6 os-sdlc, 4 takedown eval) whileProjectScenario#write_multi_target_project_yamlexists; three unrelated fakes namedFakeRunner.Measurement (run from plugins/os-sdlc; baseline at
d5cf9f96yaml=10 body=16):Cop decision rule: after the first pipeline batch that writes tests following ticket 5, re-run. If
yamlorbodyrises above the post-ticket-5 value, file a cop for that record via mine-lints. If flat, no cop; close #1019.Tickets (filed via capture; numbers in the follow-up comment):
TicketFixtureinto value-only fixture (complete_body,without(section),with_required_work(item)) andTicketScenarioowning the disk writes (fast-batch; depends on 1).HomeAndStartout ofproject_scenario.rb(fast-batch).require_relativeshome_and_start.rb+project_scenario.rband replaces its 4 hand-built Hashes (fast-batch; depends on 3).Out of scope follow-ups: rename the three
FakeRunnerfakes; audit other*_fixture.rbfor I/O (git_repo_fixture,changed_project_fixture,gate_commands_project_fixture,open_implementation_fixture); convert os-adr, os-me, os-backlog; finding-builder index line if builders keep hand-building records.Filed from the 1019 memo: #1458 Add record-fixture reference, lens link, and test-writer index line; #1459 Split TicketFixture into a value-only fixture and TicketScenario; #1460 Move HomeAndStart out of project_scenario.rb into its own file; #1461 Replace hand-built ticket bodies and project.yaml Hashes in os-sdlc tests with fixtures. The fifth ticket (Use ProjectScenario in the takedown eval tests) is held: the skeptic found open #1011 with the same mechanism.
fast-batch-21 shipped on main
c74e1bde: #1460 #1459 #1461 closed. Measurement after the batch (from plugins/os-sdlc): yaml=4 (all in eval/ticket-takedown, #1011 occurrence) body=3 (kept parser-input files only). Baseline was yaml=10 body=16. Next: after the first pipeline batch that writes tests, re-run the measurement; if either count rises, file a cop for that record via mine-lints; if flat, close this ticket.Conversion tickets filed: #1463 Rename the nine I/O-doing *_fixture.rb support files to scenarios; #1464 WorkOrderFixture.build for the 73 tests that hand-build WorkOrder; #1465 Record fixtures for ProcessIO, Finding, Signal, StepOutcome, SourceTree, Location; #1466 Consolidate git identity, git init, and rubocop AllCops literals in tests; #1467 Split AdrFixtures into AdrFixture (value) and AdrScenario (disk); #1468 Rename the two os-sdlc FakeRunner fakes after their collaborator. Order: #1463 → #1466; #1464 → #1465; #1467, #1468 independent. Held: the os-me Zeitwerk and FakeRunner ticket (skeptic verdict OCCURRENCE 1019).
os-me conversion filed as #1469 (user-approved; skeptic had marked it an occurrence of this ruling ticket, which holds no build work). Full conversion set: #1463 → #1466, #1464 → #1465, #1467 (os-adr, by hand), #1468, #1469 (os-me, by hand). Re-measure step still pending the first test-writing pipeline batch.
Resolution
Done: Re-measure done; no cop (ruling M1, predicted 72%). The convention shipped (#1458) and every conversion ticket is closed (#1463 #1464 #1465 #1466 #1467 #1468 #1469 #1269, #1011 occurrence).
Evidence: Main
2fe1b7f0, measurement from plugins/os-sdlc: yaml=0 body=4. The post-fast-batch-21 baseline on main was yaml=4 body=4: fast-batch-21 measured body=3 on its branch, before #1450 (5ccc305f, pre-convention) added tests/implementation/acceptance_test.rb BODY. #1453 (7246f609), the first test-writing batch after the convention, left the counts flat. The 4 body files are kept literals that are input under test; acceptance_test.rb was checked in fast-batch-22 fix 9 (no TicketFixture call yields typed required work plus a Retired tests section; ruling L1b). Note: M1 assumed body=3 after L1; the count is still flat. Rulings: docs/crc/os-sdlc/fast-batch-22/rulings.tsv, memo rulings/M1.md.Follow-ups: none (ruling M1: no re-run habit)
Skeptic DROP for follow-up 'os-sdlc test-support leftovers after fast-batch-22': nothing fails today, no observed failure (speculative-hardening test). Leftovers, fix on next touch: redundant require_relative "cop_helpers" in plugins/os-sdlc/tests/support/rescue_source.rb; SeededGitRepoHelpers#run_git ignores exit status (GitRepoScenario#run_git! raises); CrcLedgerScenario has class and instance methods both named write.