Takedown design must list every existing test that pins changed behavior #1004

New issue

Closed

opened 2026-10-14 15:18:06 +00:00 by jared · 5 comments

jared commented 2026-10-14 15:18:06 +00:00

Owner

Copy link

Problem

The takedown design phase proves new tests red at base (final-red, #995). It does not run the whole target suite against the complete design. An existing test that pins the old behavior therefore fails only at implement time.

Design

/home/jared/dev/cc-os/.claude/worktrees/prism-architect/docs/takedown/1004-20261014-124517/final-design.json

Approach

Add a final-suite gate (FinalSuiteStep over SuiteCheck) that applies the complete design with FilePatch and runs every target's test_command from .sdlc/project.yaml, the same source as the implement-side suite gate. Takedown::Flow writes final-design.json only after final-red, final-check and final-suite pass.

Required work:

1. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/file_patch.rb
2. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/project_suites.rb
3. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/suite_runner.rb
4. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/suite_run.rb
5. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/suite_check.rb
6. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/final_suite_step.rb
7. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/red_check.rb
8. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/step_list.rb
9. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/flow.rb
10. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/run_plan.rb
11. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/run_state.rb
12. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/collaborators.rb
13. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/run_builder.rb
14. Write plugins/os-sdlc/eval/ticket-takedown/lib/takedown/controller/run_cli.rb
15. Write plugins/os-sdlc/eval/ticket-takedown/test/controller/run_cli_test.rb
16. Delete plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/test_patch.rb
17. Write test plugins/os-sdlc/eval/ticket-takedown/test/command/final_suite_test.rb

Acceptance criteria

• Live design breaks an existing test it does not list: final-suite fails, names the test path; no final-design.json; no grade
• A different target's suite breaks: Same: all configured targets run
• Design lists a changed test in code_files: The changed file runs
• Design adds failing_tests: They are applied and run with the suite, so a collision with an existing test fails here
• Design deletes a test file: The file is absent while the suite runs and is restored afterwards
• All targets green: final-design.json has suite_check with green, and each target's command, pass flag and failing paths
• Suite fails with no parseable test path (load error, timeout, exit 127): Not green; the note names the target
• .sdlc/project.yaml, its targets, or a test_command is missing: Failure, loudly
• final-red or final-check fails: Stops before final-suite; no final-design.json
• Design path outside the repo: FilePatch raises Failure
• Eval-case run (suite false): No final-suite step; unchanged; final_record has no suite_check key
• Debate ends with no acceptable design: Unchanged; no final design

Owner sketch

Responsibility	Owner
Order of the final gates; final-design.json written only after they pass; the gate runs only when the plan asks for it	Takedown::Flow
Whether a plan runs the suite (live plans yes, eval-case plans no)	Takedown::RunPlan#suite?
Apply code_files and failing_tests, remove deleted_files, restore afterwards	Takedown::FilePatch (replaces TestPatch; one patcher)
Read target names and test commands from .sdlc/project.yaml; fail loudly when missing	Takedown::ProjectSuites (new, adapter)
Run one target command in the repo root	Takedown::SuiteRunner (new, adapter)
One target result and its failing test paths parsed from Minitest output	Takedown::SuiteRun (new, model)
Suite result: apply the design, run all targets, green or not, note naming the failing paths	Takedown::SuiteCheck (new, model)
Step wiring: outcome, slot record, run.suite_checks	Takedown::FinalSuiteStep (new, command)
Keep the suite record per design step; put it in the final record	Takedown::RunState
Carry the suite runner to the step list	Takedown::Collaborators, RunBuilder, RunCLI.collaborators

plugins/os-sdlc/eval/ticket-takedown/test/command/final_suite_test.rb

# frozen_string_literal: true

require_relative "../run_support"

class ScriptedSuites
  attr_reader :calls, :seen

  def initialize(failing: {}, watch: nil)
    @failing = failing
    @watch = watch
    @calls = []
    @seen = []
  end

  def call(repo, target)
    @calls << target.name
    @seen << @watch.to_h { |path| [path, File.exist?(File.join(repo, path)) ? File.read(File.join(repo, path)) : nil] } if @watch
    output = @failing[target.name]
    Takedown::SuiteRun.new(target: target.name, command: target.command, passed: output.nil?, output: output || "ok")
  end
end

class FinalSuiteTest < TakedownTest
  include RunSupport

  PIN = "test/pin_test.rb"

  def setup
    super
    @repo = File.join(@root, "repo")
    write(File.join(@repo, ".sdlc", "project.yaml"),
          { "targets" => { "alpha" => { "test_command" => "run-a" }, "beta" => { "test_command" => "run-b" } } }.to_yaml)
    write(File.join(@repo, "lib", "y.rb"), "y")
    write(File.join(@repo, PIN), "ORIGINAL")
  end

  def failure_block(path) = "  1) Failure:\nPinTest#test_x [#{path}:3]:\nboom\n\n1 runs, 1 assertions, 1 failures\n"

  def design(code: [], deleted: [], failing: [{ "path" => "test/new_test.rb", "content" => "NEW" }])
    { "code_files" => code, "deleted_files" => deleted, "failing_tests" => failing }
  end

  def test_project_suites_lists_every_target_with_its_command
    targets = Takedown::ProjectSuites.new(@repo).targets
    assert_equal [%w[alpha run-a], %w[beta run-b]], targets.map { [_1.name, _1.command] }
  end

  def test_project_suites_without_a_project_file_fails
    FileUtils.rm_f(File.join(@repo, ".sdlc", "project.yaml"))
    assert_raises(Takedown::Failure) { Takedown::ProjectSuites.new(@repo).targets }
  end

  def test_project_suites_with_a_target_without_a_command_fails
    write(File.join(@repo, ".sdlc", "project.yaml"), { "targets" => { "alpha" => { "path" => "x" } } }.to_yaml)
    assert_raises(Takedown::Failure) { Takedown::ProjectSuites.new(@repo).targets }
  end

  def test_project_suites_with_no_targets_fails
    write(File.join(@repo, ".sdlc", "project.yaml"), { "name" => "x" }.to_yaml)
    assert_raises(Takedown::Failure) { Takedown::ProjectSuites.new(@repo).targets }
  end

  def test_a_suite_run_names_failure_and_error_paths_relative_to_the_repo
    output = "  1) Failure:\nATest#test_a [#{@repo}/test/a_test.rb:3]:\nx\n\n  2) Error:\nBTest#test_b:\nRuntimeError: y\n    test/b_test.rb:9:in 'm'\n"
    run = Takedown::SuiteRun.new(target: "alpha", command: "run-a", passed: false, output: output)
    assert_equal %w[test/a_test.rb test/b_test.rb], run.failing_paths(@repo)
  end

  def test_a_passing_suite_run_names_no_paths
    run = Takedown::SuiteRun.new(target: "alpha", command: "run-a", passed: true, output: failure_block(PIN))
    assert_empty run.failing_paths(@repo)
  end

  def test_the_runner_runs_the_command_in_the_repo_root_and_maps_the_exit_code
    runner = FakeRunner.new(process(stdout: "so", stderr: "se", exit_code: 1))
    target = Takedown::ProjectSuites::Target.new(name: "alpha", command: "run-a")
    run = Takedown::SuiteRunner.new(runner: runner).call(@repo, target)
    call = runner.calls.first
    assert_equal [["sh", "-c", "run-a"], @repo, false], [call[:argv], call[:cwd], run.passed]
    assert_includes run.output, "so"
    assert_includes run.output, "se"
  end

  def patch(**options) = Takedown::FilePatch.new(@repo, [], **options)

  def test_the_patch_removes_deleted_files_while_applied_and_restores_them
    path = File.join(@repo, "lib", "y.rb")
    seen = patch(deleted: ["lib/y.rb"]).applied { File.exist?(path) }
    assert_equal [false, "y"], [seen, File.read(path)]
  end

  def test_the_patch_restores_deleted_files_after_an_error
    path = File.join(@repo, "lib", "y.rb")
    assert_raises(RuntimeError) { patch(deleted: ["lib/y.rb"]).applied { raise "boom" } }
    assert_equal "y", File.read(path)
  end

  def test_a_deleted_path_outside_the_repository_fails
    assert_raises(Takedown::Failure) { patch(deleted: ["../escape.rb"]) }
  end

  def test_a_green_check_runs_every_target_and_lists_its_command
    suites = ScriptedSuites.new
    check = Takedown::SuiteCheck.run(suites, @repo, design).to_h
    assert_equal [true, %w[alpha beta], %w[run-a run-b]],
                 [check["green"], suites.calls, check["targets"].map { _1["command"] }]
  end

  def test_a_failing_target_makes_the_check_not_green_and_the_note_names_the_test_path
    suites = ScriptedSuites.new(failing: { "beta" => failure_block(PIN) })
    check = Takedown::SuiteCheck.run(suites, @repo, design).to_h
    assert_equal [false, "breaks existing tests: #{PIN}"], check.values_at("green", "note")
    assert_equal [[], [PIN]], check["targets"].map { _1["failing_paths"] }
  end

  def test_a_failing_target_with_no_test_path_is_named_by_target
    suites = ScriptedSuites.new(failing: { "alpha" => "LoadError" })
    check = Takedown::SuiteCheck.run(suites, @repo, design).to_h
    assert_includes check["note"], "target alpha failed with no test path"
  end

  def test_the_suite_sees_the_complete_design_applied_and_the_repo_restored_afterwards
    suites = ScriptedSuites.new(watch: ["test/new_test.rb", "lib/y.rb", PIN])
    code = [{ "path" => PIN, "content" => "CHANGED" }]
    Takedown::SuiteCheck.run(suites, @repo, design(code: code, deleted: ["lib/y.rb"]))
    assert_equal({ "test/new_test.rb" => "NEW", "lib/y.rb" => nil, PIN => "CHANGED" }, suites.seen.first)
    assert_equal [false, "y", "ORIGINAL"],
                 [File.exist?(File.join(@repo, "test/new_test.rb")), File.read(File.join(@repo, "lib", "y.rb")),
                  File.read(File.join(@repo, PIN))]
  end

  def test_the_targets_are_read_before_the_design_is_applied
    suites = ScriptedSuites.new
    Takedown::SuiteCheck.run(suites, @repo, design(deleted: [".sdlc/project.yaml"]))
    assert_equal %w[alpha beta], suites.calls
  end
end

class FinalSuitePipelineTest < TakedownTest
  include RunSupport

  PIN = "test/pin_test.rb"
  Source = Data.define(:markdown_text) do
    def markdown(_ticket) = markdown_text
    def comments(_ticket) = []
  end

  def setup
    super
    @repo = File.join(@root, "scratch", "repo")
    FileUtils.mkdir_p(@repo)
    write(File.join(@repo, ".sdlc", "project.yaml"),
          { "targets" => { "alpha" => { "test_command" => "run-a" }, "beta" => { "test_command" => "run-b" } } }.to_yaml)
    write(File.join(@repo, PIN), "ORIGINAL")
    @run_dir = File.join(@root, "runs", "907-20261004-101010")
    @problems = File.join(@root, "problems")
  end

  def run_state(suite)
    ticket = "# REAL TITLE\n\n## Problem\n\nPROBLEM TEXT\n\n## Approach\n\nAPPROACH TEXT\n"
    source = Source.new(ticket)
    full = Takedown::TicketBody.fetch("907", source: source).to_s
    blind = Takedown::TicketBody.fetch("907", exclude: Takedown::TicketBody::BLIND, source: source).to_s
    views = Takedown::TicketViews.new(full: full, blind: blind, redactions: 0)
    Takedown::RunState.new(ticket: "907", dir: Takedown::RunDir.new(@run_dir), views: views,
                           plan: Takedown::RunPlan.new(designers: 1, suite: suite))
  end

  def recorder
    metrics = Takedown::MetricsLog.new([File.join(@run_dir, "metrics.tsv"), File.join(@root, "metrics-all.tsv")])
    Takedown::StepRecorder.new(metrics: metrics, clock: -> { Time.at(0) })
  end

  def call_pipeline(suites, replies: {}, suite: true, final_check: FakeFinalCheck.new)
    models = Takedown::ModelSteps.new(executor: FakeExecutor.new(replies), prompts: Takedown::Prompts.new(FakeGradeInputs.new),
                                      fresh_followup: false)
    list = Takedown::StepList.new(models: models, tests: FakeTests.new(["t_test.rb"], flip: true), suites: suites,
                                  setup_step: FakeSetup.new(@repo), cache: Takedown::ProblemCache.new(@problems),
                                  final_check: final_check)
    Takedown::Pipeline.new(plan: list, run: run_state(suite), recorder: recorder,
                           scratch: Takedown::ScratchDir.new(path: File.join(@root, "released"))).call
  end

  def row_names = File.readlines(File.join(@run_dir, "metrics.tsv"), chomp: true).drop(1).map { _1.split("\t", -1)[2] }

  def final_path = File.join(@run_dir, "final-design.json")

  def final_json = JSON.parse(File.read(final_path))

  def breaking = "  1) Failure:\nPinTest#test_x [#{PIN}:3]:\nboom\n"

  def test_a_design_that_breaks_an_unlisted_existing_test_is_not_final_and_names_it
    report = call_pipeline(ScriptedSuites.new(failing: { "beta" => breaking }))
    assert_equal ["final-suite", false, false],
                 [report.failed_step, File.exist?(final_path), row_names.include?("grade")]
    assert_includes report.failure_text, PIN
  end

  def test_a_green_suite_run_is_recorded_with_its_commands_in_the_final_design
    report = call_pipeline(ScriptedSuites.new)
    check = final_json["suite_check"]
    assert_equal [true, true, %w[run-a run-b], %w[final-red final-check final-suite grade]],
                 [report.ok?, check["green"], check["targets"].map { _1["command"] }, row_names.last(4)]
  end

  def test_the_suite_runs_a_test_file_the_design_changes_in_its_changed_form_and_restores_the_original
    suites = ScriptedSuites.new(watch: [PIN])
    design = RunSupport.design("D").merge("code_files" => [{ "path" => PIN, "content" => "CHANGED" }])
    call_pipeline(suites, replies: { "design-1" => design })
    assert_equal [{ PIN => "CHANGED" }, "ORIGINAL"], [suites.seen.first, File.read(File.join(@repo, PIN))]
  end

  def test_the_suite_does_not_see_a_test_file_the_design_deletes_and_restores_it
    suites = ScriptedSuites.new(watch: [PIN])
    design = RunSupport.design("D").merge("deleted_files" => [PIN])
    report = call_pipeline(suites, replies: { "design-1" => design })
    assert_equal [true, { PIN => nil }, "ORIGINAL"], [report.ok?, suites.seen.first, File.read(File.join(@repo, PIN))]
  end

  def test_a_plan_without_the_suite_runs_no_final_suite_and_records_no_suite_check
    suites = ScriptedSuites.new
    call_pipeline(suites, suite: false)
    assert_equal [[], false, false], [suites.calls, row_names.include?("final-suite"), final_json.key?("suite_check")]
  end

  def test_a_failed_final_check_writes_no_final_design
    report = call_pipeline(ScriptedSuites.new, final_check: FakeFinalCheck.new("failed"))
    assert_equal ["final-check", false], [report.failed_step, File.exist?(final_path)]
  end
end

## Problem The takedown design phase proves new tests red at base (final-red, #995). It does not run the whole target suite against the complete design. An existing test that pins the old behavior therefore fails only at implement time. ## Design /home/jared/dev/cc-os/.claude/worktrees/prism-architect/docs/takedown/1004-20261014-124517/final-design.json ## Approach Add a final-suite gate (FinalSuiteStep over SuiteCheck) that applies the complete design with FilePatch and runs every target's test_command from .sdlc/project.yaml, the same source as the implement-side suite gate. Takedown::Flow writes final-design.json only after final-red, final-check and final-suite pass. Required work: 1. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/file_patch.rb` 2. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/project_suites.rb` 3. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/suite_runner.rb` 4. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/suite_run.rb` 5. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/suite_check.rb` 6. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/final_suite_step.rb` 7. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/red_check.rb` 8. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/step_list.rb` 9. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/flow.rb` 10. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/run_plan.rb` 11. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/run_state.rb` 12. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/model/collaborators.rb` 13. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/run_builder.rb` 14. Write `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/controller/run_cli.rb` 15. Write `plugins/os-sdlc/eval/ticket-takedown/test/controller/run_cli_test.rb` 16. Delete `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/test_patch.rb` 17. Write test `plugins/os-sdlc/eval/ticket-takedown/test/command/final_suite_test.rb` ## Acceptance criteria - Live design breaks an existing test it does not list: `final-suite` fails, names the test path; no final-design.json; no grade - A different target's suite breaks: Same: all configured targets run - Design lists a changed test in code_files: The changed file runs - Design adds failing_tests: They are applied and run with the suite, so a collision with an existing test fails here - Design deletes a test file: The file is absent while the suite runs and is restored afterwards - All targets green: `final-design.json` has `suite_check` with green, and each target's command, pass flag and failing paths - Suite fails with no parseable test path (load error, timeout, exit 127): Not green; the note names the target - `.sdlc/project.yaml`, its targets, or a `test_command` is missing: Failure, loudly - `final-red` or `final-check` fails: Stops before `final-suite`; no final-design.json - Design path outside the repo: FilePatch raises Failure - Eval-case run (`suite` false): No final-suite step; unchanged; `final_record` has no `suite_check` key - Debate ends with no acceptable design: Unchanged; no final design ## Owner sketch | Responsibility | Owner | |---|---| | Order of the final gates; `final-design.json` written only after they pass; the gate runs only when the plan asks for it | `Takedown::Flow` | | Whether a plan runs the suite (live plans yes, eval-case plans no) | `Takedown::RunPlan#suite?` | | Apply code_files and failing_tests, remove deleted_files, restore afterwards | `Takedown::FilePatch` (replaces `TestPatch`; one patcher) | | Read target names and test commands from `.sdlc/project.yaml`; fail loudly when missing | `Takedown::ProjectSuites` (new, adapter) | | Run one target command in the repo root | `Takedown::SuiteRunner` (new, adapter) | | One target result and its failing test paths parsed from Minitest output | `Takedown::SuiteRun` (new, model) | | Suite result: apply the design, run all targets, green or not, note naming the failing paths | `Takedown::SuiteCheck` (new, model) | | Step wiring: outcome, slot record, `run.suite_checks` | `Takedown::FinalSuiteStep` (new, command) | | Keep the suite record per design step; put it in the final record | `Takedown::RunState` | | Carry the suite runner to the step list | `Takedown::Collaborators`, `RunBuilder`, `RunCLI.collaborators` | `plugins/os-sdlc/eval/ticket-takedown/test/command/final_suite_test.rb` ```ruby # frozen_string_literal: true require_relative "../run_support" class ScriptedSuites attr_reader :calls, :seen def initialize(failing: {}, watch: nil) @failing = failing @watch = watch @calls = [] @seen = [] end def call(repo, target) @calls << target.name @seen << @watch.to_h { |path| [path, File.exist?(File.join(repo, path)) ? File.read(File.join(repo, path)) : nil] } if @watch output = @failing[target.name] Takedown::SuiteRun.new(target: target.name, command: target.command, passed: output.nil?, output: output || "ok") end end class FinalSuiteTest < TakedownTest include RunSupport PIN = "test/pin_test.rb" def setup super @repo = File.join(@root, "repo") write(File.join(@repo, ".sdlc", "project.yaml"), { "targets" => { "alpha" => { "test_command" => "run-a" }, "beta" => { "test_command" => "run-b" } } }.to_yaml) write(File.join(@repo, "lib", "y.rb"), "y") write(File.join(@repo, PIN), "ORIGINAL") end def failure_block(path) = " 1) Failure:\nPinTest#test_x [#{path}:3]:\nboom\n\n1 runs, 1 assertions, 1 failures\n" def design(code: [], deleted: [], failing: [{ "path" => "test/new_test.rb", "content" => "NEW" }]) { "code_files" => code, "deleted_files" => deleted, "failing_tests" => failing } end def test_project_suites_lists_every_target_with_its_command targets = Takedown::ProjectSuites.new(@repo).targets assert_equal [%w[alpha run-a], %w[beta run-b]], targets.map { [_1.name, _1.command] } end def test_project_suites_without_a_project_file_fails FileUtils.rm_f(File.join(@repo, ".sdlc", "project.yaml")) assert_raises(Takedown::Failure) { Takedown::ProjectSuites.new(@repo).targets } end def test_project_suites_with_a_target_without_a_command_fails write(File.join(@repo, ".sdlc", "project.yaml"), { "targets" => { "alpha" => { "path" => "x" } } }.to_yaml) assert_raises(Takedown::Failure) { Takedown::ProjectSuites.new(@repo).targets } end def test_project_suites_with_no_targets_fails write(File.join(@repo, ".sdlc", "project.yaml"), { "name" => "x" }.to_yaml) assert_raises(Takedown::Failure) { Takedown::ProjectSuites.new(@repo).targets } end def test_a_suite_run_names_failure_and_error_paths_relative_to_the_repo output = " 1) Failure:\nATest#test_a [#{@repo}/test/a_test.rb:3]:\nx\n\n 2) Error:\nBTest#test_b:\nRuntimeError: y\n test/b_test.rb:9:in 'm'\n" run = Takedown::SuiteRun.new(target: "alpha", command: "run-a", passed: false, output: output) assert_equal %w[test/a_test.rb test/b_test.rb], run.failing_paths(@repo) end def test_a_passing_suite_run_names_no_paths run = Takedown::SuiteRun.new(target: "alpha", command: "run-a", passed: true, output: failure_block(PIN)) assert_empty run.failing_paths(@repo) end def test_the_runner_runs_the_command_in_the_repo_root_and_maps_the_exit_code runner = FakeRunner.new(process(stdout: "so", stderr: "se", exit_code: 1)) target = Takedown::ProjectSuites::Target.new(name: "alpha", command: "run-a") run = Takedown::SuiteRunner.new(runner: runner).call(@repo, target) call = runner.calls.first assert_equal [["sh", "-c", "run-a"], @repo, false], [call[:argv], call[:cwd], run.passed] assert_includes run.output, "so" assert_includes run.output, "se" end def patch(**options) = Takedown::FilePatch.new(@repo, [], **options) def test_the_patch_removes_deleted_files_while_applied_and_restores_them path = File.join(@repo, "lib", "y.rb") seen = patch(deleted: ["lib/y.rb"]).applied { File.exist?(path) } assert_equal [false, "y"], [seen, File.read(path)] end def test_the_patch_restores_deleted_files_after_an_error path = File.join(@repo, "lib", "y.rb") assert_raises(RuntimeError) { patch(deleted: ["lib/y.rb"]).applied { raise "boom" } } assert_equal "y", File.read(path) end def test_a_deleted_path_outside_the_repository_fails assert_raises(Takedown::Failure) { patch(deleted: ["../escape.rb"]) } end def test_a_green_check_runs_every_target_and_lists_its_command suites = ScriptedSuites.new check = Takedown::SuiteCheck.run(suites, @repo, design).to_h assert_equal [true, %w[alpha beta], %w[run-a run-b]], [check["green"], suites.calls, check["targets"].map { _1["command"] }] end def test_a_failing_target_makes_the_check_not_green_and_the_note_names_the_test_path suites = ScriptedSuites.new(failing: { "beta" => failure_block(PIN) }) check = Takedown::SuiteCheck.run(suites, @repo, design).to_h assert_equal [false, "breaks existing tests: #{PIN}"], check.values_at("green", "note") assert_equal [[], [PIN]], check["targets"].map { _1["failing_paths"] } end def test_a_failing_target_with_no_test_path_is_named_by_target suites = ScriptedSuites.new(failing: { "alpha" => "LoadError" }) check = Takedown::SuiteCheck.run(suites, @repo, design).to_h assert_includes check["note"], "target alpha failed with no test path" end def test_the_suite_sees_the_complete_design_applied_and_the_repo_restored_afterwards suites = ScriptedSuites.new(watch: ["test/new_test.rb", "lib/y.rb", PIN]) code = [{ "path" => PIN, "content" => "CHANGED" }] Takedown::SuiteCheck.run(suites, @repo, design(code: code, deleted: ["lib/y.rb"])) assert_equal({ "test/new_test.rb" => "NEW", "lib/y.rb" => nil, PIN => "CHANGED" }, suites.seen.first) assert_equal [false, "y", "ORIGINAL"], [File.exist?(File.join(@repo, "test/new_test.rb")), File.read(File.join(@repo, "lib", "y.rb")), File.read(File.join(@repo, PIN))] end def test_the_targets_are_read_before_the_design_is_applied suites = ScriptedSuites.new Takedown::SuiteCheck.run(suites, @repo, design(deleted: [".sdlc/project.yaml"])) assert_equal %w[alpha beta], suites.calls end end class FinalSuitePipelineTest < TakedownTest include RunSupport PIN = "test/pin_test.rb" Source = Data.define(:markdown_text) do def markdown(_ticket) = markdown_text def comments(_ticket) = [] end def setup super @repo = File.join(@root, "scratch", "repo") FileUtils.mkdir_p(@repo) write(File.join(@repo, ".sdlc", "project.yaml"), { "targets" => { "alpha" => { "test_command" => "run-a" }, "beta" => { "test_command" => "run-b" } } }.to_yaml) write(File.join(@repo, PIN), "ORIGINAL") @run_dir = File.join(@root, "runs", "907-20261004-101010") @problems = File.join(@root, "problems") end def run_state(suite) ticket = "# REAL TITLE\n\n## Problem\n\nPROBLEM TEXT\n\n## Approach\n\nAPPROACH TEXT\n" source = Source.new(ticket) full = Takedown::TicketBody.fetch("907", source: source).to_s blind = Takedown::TicketBody.fetch("907", exclude: Takedown::TicketBody::BLIND, source: source).to_s views = Takedown::TicketViews.new(full: full, blind: blind, redactions: 0) Takedown::RunState.new(ticket: "907", dir: Takedown::RunDir.new(@run_dir), views: views, plan: Takedown::RunPlan.new(designers: 1, suite: suite)) end def recorder metrics = Takedown::MetricsLog.new([File.join(@run_dir, "metrics.tsv"), File.join(@root, "metrics-all.tsv")]) Takedown::StepRecorder.new(metrics: metrics, clock: -> { Time.at(0) }) end def call_pipeline(suites, replies: {}, suite: true, final_check: FakeFinalCheck.new) models = Takedown::ModelSteps.new(executor: FakeExecutor.new(replies), prompts: Takedown::Prompts.new(FakeGradeInputs.new), fresh_followup: false) list = Takedown::StepList.new(models: models, tests: FakeTests.new(["t_test.rb"], flip: true), suites: suites, setup_step: FakeSetup.new(@repo), cache: Takedown::ProblemCache.new(@problems), final_check: final_check) Takedown::Pipeline.new(plan: list, run: run_state(suite), recorder: recorder, scratch: Takedown::ScratchDir.new(path: File.join(@root, "released"))).call end def row_names = File.readlines(File.join(@run_dir, "metrics.tsv"), chomp: true).drop(1).map { _1.split("\t", -1)[2] } def final_path = File.join(@run_dir, "final-design.json") def final_json = JSON.parse(File.read(final_path)) def breaking = " 1) Failure:\nPinTest#test_x [#{PIN}:3]:\nboom\n" def test_a_design_that_breaks_an_unlisted_existing_test_is_not_final_and_names_it report = call_pipeline(ScriptedSuites.new(failing: { "beta" => breaking })) assert_equal ["final-suite", false, false], [report.failed_step, File.exist?(final_path), row_names.include?("grade")] assert_includes report.failure_text, PIN end def test_a_green_suite_run_is_recorded_with_its_commands_in_the_final_design report = call_pipeline(ScriptedSuites.new) check = final_json["suite_check"] assert_equal [true, true, %w[run-a run-b], %w[final-red final-check final-suite grade]], [report.ok?, check["green"], check["targets"].map { _1["command"] }, row_names.last(4)] end def test_the_suite_runs_a_test_file_the_design_changes_in_its_changed_form_and_restores_the_original suites = ScriptedSuites.new(watch: [PIN]) design = RunSupport.design("D").merge("code_files" => [{ "path" => PIN, "content" => "CHANGED" }]) call_pipeline(suites, replies: { "design-1" => design }) assert_equal [{ PIN => "CHANGED" }, "ORIGINAL"], [suites.seen.first, File.read(File.join(@repo, PIN))] end def test_the_suite_does_not_see_a_test_file_the_design_deletes_and_restores_it suites = ScriptedSuites.new(watch: [PIN]) design = RunSupport.design("D").merge("deleted_files" => [PIN]) report = call_pipeline(suites, replies: { "design-1" => design }) assert_equal [true, { PIN => nil }, "ORIGINAL"], [report.ok?, suites.seen.first, File.read(File.join(@repo, PIN))] end def test_a_plan_without_the_suite_runs_no_final_suite_and_records_no_suite_check suites = ScriptedSuites.new call_pipeline(suites, suite: false) assert_equal [[], false, false], [suites.calls, row_names.include?("final-suite"), final_json.key?("suite_check")] end def test_a_failed_final_check_writes_no_final_design report = call_pipeline(ScriptedSuites.new, final_check: FakeFinalCheck.new("failed")) assert_equal ["final-check", false], [report.failed_step, File.exist?(final_path)] end end ```

jared added the

plugin/os-sdlc

filed-by/agent

category/code-change

labels 2026-10-14 15:18:06 +00:00

jared referenced this issue 2026-10-14 16:01:52 +00:00

Route changed existing tests away from the programmer at place-code #1003

jared commented 2026-10-14 17:11:09 +00:00

Author

Owner

Copy link

Work started: /os-sdlc:implement 1004 takedown-implement on branch prism-architect. Design: docs/takedown/1004-20261014-124517/final-design.json (db9dea98). ADR-0291 (Proposed, predicted ruling) amends ADR-0283.

Work started: /os-sdlc:implement 1004 takedown-implement on branch prism-architect. Design: docs/takedown/1004-20261014-124517/final-design.json (db9dea98). ADR-0291 (Proposed, predicted ruling) amends ADR-0283.

jared commented 2026-10-14 17:14:04 +00:00

Author

Owner

Copy link

Implementation 43 (takedown-implement) abandoned: escalation 22 at place-tests; test-lint found 10+ offenses in the design test final_suite_test.rb; the takedown design phase never lints. Per the integration brief, #1004 now runs on poodr-ticket-implementation, branch prism-architect. Lint defect captured separately.

Implementation 43 (takedown-implement) abandoned: escalation 22 at place-tests; test-lint found 10+ offenses in the design test final_suite_test.rb; the takedown design phase never lints. Per the integration brief, #1004 now runs on poodr-ticket-implementation, branch prism-architect. Lint defect captured separately.

jared referenced this issue 2026-10-14 17:14:24 +00:00

Takedown design phase must lint the design's tests and code before the design is final #1005

jared commented 2026-10-14 17:38:43 +00:00

Author

Owner

Copy link

Lens follow-up dropped (skeptic DROP, depth-1, speculative hardening): move apply-and-run out of model SuiteCheck.run and RedCheck.run into the command layer (A11). Effects in a model, no observed failure.

Lens follow-up dropped (skeptic DROP, depth-1, speculative hardening): move apply-and-run out of model SuiteCheck.run and RedCheck.run into the command layer (A11). Effects in a model, no observed failure.

jared commented 2026-10-14 17:38:43 +00:00

Author

Owner

Copy link

Lens follow-up dropped (skeptic DROP, depth-1, speculative hardening): nest single-caller top-level objects RunOptions, SuiteRunner, ProjectSuites per A2/ADR-0237. Same root as the A11 note; one takedown conformance sweep if ever wanted.

Lens follow-up dropped (skeptic DROP, depth-1, speculative hardening): nest single-caller top-level objects RunOptions, SuiteRunner, ProjectSuites per A2/ADR-0237. Same root as the A11 note; one takedown conformance sweep if ever wanted.

jared commented 2026-10-14 17:39:10 +00:00

Author

Owner

Copy link

Resolution

Done: Takedown design phase now runs a final-suite gate: after final-red and final-check it applies the complete design (code_files, failing_tests, deleted_files) to the scratch copy and runs every target's test_command from .sdlc/project.yaml; final-design.json is written only after all three pass and records suite_check; a failing existing test is named and the design is not final. Live plans only; eval-case plans skip it via RunPlan suite field. ADR-0291 (Proposed, predicted ruling at 65%, awaiting Jared) amends ADR-0283.

Evidence: Commit d3cfd3e6 (implementation 44, poodr-ticket-implementation, 13 dispatches, 21 min 28 s; suite-check green twice, behavior-verifier pass on all 12 AC, contract-auditor pass). Records e2b6d485 (README step table, os-sdlc leaf row), a059ba70 (lens signals), ADR 643b5b97, takedown run db9dea98. Implementation 43 (takedown-implement) escalated at place-tests on test-lint (escalation 22) and was abandoned; implementation 44 needed one orchestrator test hand-fix in run_cli_test.rb (escalation 23) and one reroute to programmer.

Follow-ups: #1005 captured (VERDICT: CREATE): takedown design phase must lint design tests and code. A11 and A2 lens follow-ups dropped by skeptic (DROP, depth-1), reasons posted as comments on this ticket. ADR-0291 confirmation is Jared's ruling, recorded in docs/takedown/integration-plan.md Rulings table.

## Resolution **Done:** Takedown design phase now runs a final-suite gate: after final-red and final-check it applies the complete design (code_files, failing_tests, deleted_files) to the scratch copy and runs every target's test_command from .sdlc/project.yaml; final-design.json is written only after all three pass and records suite_check; a failing existing test is named and the design is not final. Live plans only; eval-case plans skip it via RunPlan suite field. ADR-0291 (Proposed, predicted ruling at 65%, awaiting Jared) amends ADR-0283. **Evidence:** Commit d3cfd3e6 (implementation 44, poodr-ticket-implementation, 13 dispatches, 21 min 28 s; suite-check green twice, behavior-verifier pass on all 12 AC, contract-auditor pass). Records e2b6d485 (README step table, os-sdlc leaf row), a059ba70 (lens signals), ADR 643b5b97, takedown run db9dea98. Implementation 43 (takedown-implement) escalated at place-tests on test-lint (escalation 22) and was abandoned; implementation 44 needed one orchestrator test hand-fix in run_cli_test.rb (escalation 23) and one reroute to programmer. **Follow-ups:** #1005 captured (VERDICT: CREATE): takedown design phase must lint design tests and code. A11 and A2 lens follow-ups dropped by skeptic (DROP, depth-1), reasons posted as comments on this ticket. ADR-0291 confirmation is Jared's ruling, recorded in docs/takedown/integration-plan.md Rulings table.

jared closed this issue 2026-10-14 17:39:10 +00:00

jared referenced this issue 2026-10-14 17:40:33 +00:00

ticket-takedown: start and finish its worktree via /os:worktree #957

jared referenced this issue 2026-10-15 15:38:00 +00:00

Redesign RawFileContentAssertion into OwnedFileReadInTest (owner-index cop) #1008

jared referenced this issue 2026-10-15 17:55:21 +00:00

Redesign RepeatedLiteralArgument into RepeatedPathLiteral (whole-path cop) #1014

jared referenced this issue 2026-10-15 17:55:27 +00:00

Add cop for Metrics/PerceivedComplexity number gaming #1015

jared referenced this issue from a commit 2026-10-16 16:51:23 +00:00

docs(takedown): integration brief, phased plan, session-1 handoff (#1003 ruled A, #1004 filed)

jared referenced this issue from a commit 2026-10-16 16:51:23 +00:00

docs(adr): ADR-0291 takedown design is final only after full target suites pass (amends ADR-0283, #1004); Proposed, predicted ruling

jared referenced this issue from a commit 2026-10-16 16:51:23 +00:00

docs(takedown): #1004 takedown run record (final design, 3 rounds, $3.62)

jared referenced this issue from a commit 2026-10-16 16:51:23 +00:00

feat(takedown): POODR audit of the final-suite gate: placement follows the owner sketch, dependencies point inward, contracts are protected; pass. (#1004)

jared referenced this issue from a commit 2026-10-16 16:51:23 +00:00

docs(takedown): final-suite step in README and os-sdlc leaf History row (#1004)

jared referenced this issue from a commit 2026-10-16 16:51:23 +00:00

chore(os-sdlc): lens signals from #1004 lens challenge (A11 x2, P6, A2)

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

ticket-1337

No results found.

Labels

Clear labels   

P0

Priority 0 — drop everything

  

P1

Priority 1 — next up

  

P2

Priority 2 — normal

  

P3

Priority 3 — someday

  

bug

Something is broken

  

category/code-change

Ticket category

  

category/cross-repo

Ticket category

  

category/eval-run

Ticket category

  

category/investigation

Ticket category

  

category/no-test-target

Ticket category

  

category/owner-ruling

Ticket category

  

category/prose-artifact

Ticket category

  

category/recurring-chore

Ticket category

  

category/structure

  

category/umbrella

Ticket category

  

crc-refactor

CRC refactor run work record; exempt from the ticket-skeptic

  

create

Ticket creates something new

  

delete

Ticket removes something

  

enhancement

New feature or improvement

  

filed-by/agent

  

filed-by/user

  

frozen

Pre-reset backlog freeze 2026-08-16; do not work unless rediscovered by a live run

  

lint-rule

Ticket proposes/modifies a lint rule or cop

  

needs-info

Waiting on reporter for more information

  

needs-triage

Maintainer needs to evaluate this issue

  

next

Human-curated next up — AI never applies or removes

  

plugin/cc-architect

Scope: the cc-architect plugin

  

plugin/os

Scope: the os plugin

  

plugin/os-adr

Scope: the os-adr plugin

  

plugin/os-aidd-lint

Scope: the os-aidd-lint plugin

  

plugin/os-backlog

Scope: the os-backlog plugin

  

plugin/os-context

Scope: the os-context plugin

  

plugin/os-doc-hygiene

Scope: the os-doc-hygiene plugin

  

plugin/os-sdlc

Scope: the os-sdlc plugin

  

plugin/os-vault

Scope: the os-vault plugin

  

project/cc-os

Parent-scope: cc-os repo-wide, not specific to one plugin

  

ready-for-agent

Triage complete; an agent can pick this up

  

ready-for-human

Human-owned; never picked up autonomously

  

recurring

Recurring flavor: occurrence outcomes commented, never closed by AI

  

review

Shipped semi work awaiting human sign-off

  

rule-proposal

A rule change that survived the gate; Jared reads these when he chooses

  

update

Ticket updates something existing

  

waiting

Blocked; blocker named in a comment

  

wayfinder:grilling

  

wayfinder:map

  

wayfinder:map

wayfinder map ticket

  

wayfinder:research

  

wayfinder:task

  

wayfinder:task

wayfinder map child: task

  

wontfix

Will not be actioned

  

worklist/deviations

Deviation incidents from skill/pipeline runs — evidence pile for correcting the owning artifact

  

worklist/lint-rule

Proposed os-sdlc or os-aidd-lint rule

  

worklist/new-implement-build

Updated implementation harness (ast-grep + stepper)

No labels

P0

P1

P2

P3

bug

category/code-change

category/cross-repo

category/eval-run

category/investigation

category/no-test-target

category/owner-ruling

category/prose-artifact

category/recurring-chore

category/structure

category/umbrella

crc-refactor

create

delete

enhancement

filed-by/agent

filed-by/user

frozen

lint-rule

needs-info

needs-triage

next

plugin/cc-architect

plugin/os

plugin/os-adr

plugin/os-aidd-lint

plugin/os-backlog

plugin/os-context

plugin/os-doc-hygiene

plugin/os-sdlc

plugin/os-vault

project/cc-os

ready-for-agent

ready-for-human

recurring

review

rule-proposal

update

waiting

wayfinder:grilling

wayfinder:map

wayfinder:map

wayfinder:research

wayfinder:task

wayfinder:task

wontfix

worklist/deviations

worklist/lint-rule

worklist/new-implement-build

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

jared/cc-os#1004

No description provided.