jared/cc-os

Watch

Fork 0

project.yaml test_path default is test/ while every cc-os plugin uses tests/; setup-project and implementation-open do not detect the mismatch #616

New issue

Closed

opened 2026-09-18 14:19:11 +00:00 by jared · 3 comments

jared commented

2026-09-18 14:19:11 +00:00

Owner

project.yaml test_path default is test/ while every cc-os plugin uses tests/ [capture]

Context

This came up during the #612 pipeline run, on the new plugins/os-backlog/.sdlc/project.yaml.
It caused a gate to silently auto-pass with nothing checked.

Observed

Project::Test#initialize defaults path: to "test".
Every plugin in this repo keeps its tests under tests/, including plugins/os-sdlc/.sdlc/project.yaml, which has no test_path line at all.
In #612 implementation 1, the new plugins/os-backlog/.sdlc/project.yaml also had no test_path line.

GateEvaluator#run_gate for diff-test, target plugins/os-backlog

exit 0, empty output, 0 ms

Changes#test_paths was empty because the default test/ directory does not exist in this repo.
GateCommands#diff_test_command returned nil, so GateEvaluator#run_gate short-circuited to a pass with nothing run.
The Runner recorded unexpected_pass:test-writer/diff-test.
The tests were red when run directly outside the gate.

Reproduce

cd plugins/os-backlog && ls test 2>&1; ls tests

test does not exist; tests holds the suite the gate should have selected from.

Expected

def test_implementation_open_refuses_a_test_path_that_does_not_exist
  # arrange: a project.yaml whose test.path names a directory absent from the repo root
  # act: run implementation-open against that project
  # assert: it refuses, naming the missing path in its message
end

Origin

Trigger: diff-test gate for the #612 run recording unexpected_pass with 0 ms and no output
Improvised this session: none; the gap was not caught until the tests were run directly
Chain: gate auto-passed with nothing selected ← test_paths empty ← test_path default "test" does not match this repo's "tests" ← DESIGN (plugins/os-sdlc/lib/os_sdlc/model/project/test.rb, no existence check)
Root candidate: this ticket
Where: plugins/os-sdlc/lib/os_sdlc/model/project/test.rb; cli implementation-open
Session: edb23374-6b44-4410-b666-a485a1923501
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/edb23374-6b44-4410-b666-a485a1923501.jsonl

Decisions 2026-09-19

Refuse at implementation-open with an instruction the orchestrator can act on, not a bare abort. Keep the non-zero exit.
Change the Project::Test default path from test to tests, and keep the refusal. Both fixes ship in this ticket.
Centralize the check as Project::Validation, called from OpenImplementation#validate_target!, which runs before retired tests are deleted and before the implementations row is inserted. No orphan implementation rows.
Both plugins/os-sdlc/.sdlc/project.yaml and plugins/os-backlog/.sdlc/project.yaml already carry test_path: tests. That criterion is done.

Slice

A project.yaml whose paths do not exist on disk is refused at implementation-open with a message that names the path, the target, the candidate directories, and the fix. The default test path becomes tests.

Acceptance criteria

Model::Project::Validation checks, relative to the project root: code_path is a directory, test_path is a directory, and lint_config (when set) is a file. It returns the list of problems.
OpenImplementation#validate_target! raises when the list is not empty, before load_ticket_body! and before the implementations row is opened.
The CLI rescues that error in DispatchCommands.implementation_open and exits loudly with a message shaped like:

implementation-open project_validation: test_path "test" does not exist for target os-backlog.
Candidates under plugins/os-backlog: tests/
Update test_path in plugins/os-backlog/.sdlc/project.yaml and rerun implementation-open.

Candidates are the root-level directories among test, tests, spec that exist. For code_path, candidates are lib, app, src.
Project::Test#initialize defaults path: to "tests". The shared fixture default in tests/support/project_attr_assertions.rb and the tests that assert the default move to tests. Tests that write test_path: "test" explicitly into a fixture yaml keep working unchanged.
/os-sdlc:setup-project SKILL.md tells the generator to write test_path from the directory it finds among tests, test, spec, in that order, rather than omitting the key.

Non-goals

This ticket does not touch the path-base mismatch in Adapter::Git::Repo#changed_paths. It does not remove the "nil command means pass" branch in GateEvaluator#run_gate.

Owner sketch

class Model::Project::Validation
  def initialize(root, project)
  def problems  # => [] or ["test_path \"test\" does not exist ...", ...]
end

class Pipeline::OpenImplementation
  def validate_target!
    project = Model::Project.load(@root, target: @work_order.target)
    problems = Model::Project::Validation.new(@root, project).problems
    raise ProjectValidationError, problems.join("\n") if problems.any?
    project
  end
end

Cases

Given a project.yaml whose test_path names a directory that does not exist under root, when implementation-open runs, then it exits non-zero with a message naming the path, the target, the existing candidate directories, and the instruction to update project.yaml and rerun; and no implementations row is created.
Given a project.yaml whose code_path names a directory that does not exist under root, when implementation-open runs, then it refuses the same way.
Given a project.yaml whose lint_config names a file that does not exist, when implementation-open runs, then it refuses the same way.
Given a project.yaml with no test_path line and a tests/ directory, when the project loads, then project.test.path is tests.
Given a project with all paths present, when implementation-open runs, then it opens as before.

Skeptic verdict

VERDICT: CREATE
REASON: Diff-test gate silently auto-passed with zero tests run because test_path default ("test") doesn't match this repo's convention ("tests"); Origin chain traces to a design gap (no existence check on test_path), not an agent improvisation, and is validation-class (silent false-pass), so it clears the gauntlet.

# project.yaml test_path default is test/ while every cc-os plugin uses tests/ [capture] ## Context This came up during the #612 pipeline run, on the new `plugins/os-backlog/.sdlc/project.yaml`. It caused a gate to silently auto-pass with nothing checked. ## Observed `Project::Test#initialize` defaults `path:` to `"test"`. Every plugin in this repo keeps its tests under `tests/`, including `plugins/os-sdlc/.sdlc/project.yaml`, which has no `test_path` line at all. In #612 implementation 1, the new `plugins/os-backlog/.sdlc/project.yaml` also had no `test_path` line. ``` GateEvaluator#run_gate for diff-test, target plugins/os-backlog ``` ``` exit 0, empty output, 0 ms ``` `Changes#test_paths` was empty because the default `test/` directory does not exist in this repo. `GateCommands#diff_test_command` returned nil, so `GateEvaluator#run_gate` short-circuited to a pass with nothing run. The Runner recorded `unexpected_pass:test-writer/diff-test`. The tests were red when run directly outside the gate. ## Reproduce ``` cd plugins/os-backlog && ls test 2>&1; ls tests ``` `test` does not exist; `tests` holds the suite the gate should have selected from. ## Expected ```ruby def test_implementation_open_refuses_a_test_path_that_does_not_exist # arrange: a project.yaml whose test.path names a directory absent from the repo root # act: run implementation-open against that project # assert: it refuses, naming the missing path in its message end ``` ## Origin - Trigger: diff-test gate for the #612 run recording unexpected_pass with 0 ms and no output - Improvised this session: none; the gap was not caught until the tests were run directly - Chain: gate auto-passed with nothing selected ← test_paths empty ← test_path default "test" does not match this repo's "tests" ← DESIGN (plugins/os-sdlc/lib/os_sdlc/model/project/test.rb, no existence check) - Root candidate: this ticket - Where: plugins/os-sdlc/lib/os_sdlc/model/project/test.rb; cli implementation-open - Session: edb23374-6b44-4410-b666-a485a1923501 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/edb23374-6b44-4410-b666-a485a1923501.jsonl ## Decisions 2026-09-19 1. Refuse at implementation-open with an instruction the orchestrator can act on, not a bare abort. Keep the non-zero exit. 2. Change the `Project::Test` default path from `test` to `tests`, and keep the refusal. Both fixes ship in this ticket. 3. Centralize the check as `Project::Validation`, called from `OpenImplementation#validate_target!`, which runs before retired tests are deleted and before the implementations row is inserted. No orphan implementation rows. 4. Both `plugins/os-sdlc/.sdlc/project.yaml` and `plugins/os-backlog/.sdlc/project.yaml` already carry `test_path: tests`. That criterion is done. ## Slice A project.yaml whose paths do not exist on disk is refused at implementation-open with a message that names the path, the target, the candidate directories, and the fix. The default test path becomes `tests`. ## Acceptance criteria * `Model::Project::Validation` checks, relative to the project root: `code_path` is a directory, `test_path` is a directory, and `lint_config` (when set) is a file. It returns the list of problems. * `OpenImplementation#validate_target!` raises when the list is not empty, before `load_ticket_body!` and before the implementations row is opened. * The CLI rescues that error in `DispatchCommands.implementation_open` and exits loudly with a message shaped like: implementation-open project_validation: test_path "test" does not exist for target os-backlog. Candidates under plugins/os-backlog: tests/ Update test_path in plugins/os-backlog/.sdlc/project.yaml and rerun implementation-open. Candidates are the root-level directories among `test`, `tests`, `spec` that exist. For code_path, candidates are `lib`, `app`, `src`. * `Project::Test#initialize` defaults `path:` to `"tests"`. The shared fixture default in `tests/support/project_attr_assertions.rb` and the tests that assert the default move to `tests`. Tests that write `test_path: "test"` explicitly into a fixture yaml keep working unchanged. * `/os-sdlc:setup-project` SKILL.md tells the generator to write `test_path` from the directory it finds among `tests`, `test`, `spec`, in that order, rather than omitting the key. ## Non-goals This ticket does not touch the path-base mismatch in `Adapter::Git::Repo#changed_paths`. It does not remove the "nil command means pass" branch in `GateEvaluator#run_gate`. ## Owner sketch class Model::Project::Validation def initialize(root, project) def problems # => [] or ["test_path \"test\" does not exist ...", ...] end class Pipeline::OpenImplementation def validate_target! project = Model::Project.load(@root, target: @work_order.target) problems = Model::Project::Validation.new(@root, project).problems raise ProjectValidationError, problems.join("\n") if problems.any? project end end ## Cases 1. Given a project.yaml whose test_path names a directory that does not exist under root, when implementation-open runs, then it exits non-zero with a message naming the path, the target, the existing candidate directories, and the instruction to update project.yaml and rerun; and no implementations row is created. 2. Given a project.yaml whose code_path names a directory that does not exist under root, when implementation-open runs, then it refuses the same way. 3. Given a project.yaml whose lint_config names a file that does not exist, when implementation-open runs, then it refuses the same way. 4. Given a project.yaml with no test_path line and a `tests/` directory, when the project loads, then `project.test.path` is `tests`. 5. Given a project with all paths present, when implementation-open runs, then it opens as before. ## Skeptic verdict VERDICT: CREATE REASON: Diff-test gate silently auto-passed with zero tests run because test_path default ("test") doesn't match this repo's convention ("tests"); Origin chain traces to a design gap (no existence check on test_path), not an agent improvisation, and is validation-class (silent false-pass), so it clears the gauntlet.

jared added the

plugin/os-sdlc

filed-by/agent

labels

2026-09-18 14:19:11 +00:00

jared referenced this issue

2026-09-18 14:19:36 +00:00

os-backlog validator: optional sections list for ## Retired tests and ## Approach #612

jared referenced this issue

2026-09-18 14:36:28 +00:00

lsp-probe daemon: concurrent first connects leave an orphan daemon until idle timeout #611

jared referenced this issue

2026-09-18 17:25:43 +00:00

lsp-probe daemon: concurrent first connects leave an orphan daemon until idle timeout #611

jared referenced this issue from a commit

2026-09-18 18:49:54 +00:00

feat(os-sdlc): lsp-probe Daemon#connect holds a per-root spawn lock and waits for the socket, so concurrent cold connects spawn one daemon (#611, amends ADR-0201)

jared commented

2026-09-19 17:38:41 +00:00

Author

Owner

Body updated 2026-09-19 with the design decisions (see ## Decisions 2026-09-19). Implementation via /os-sdlc:implement starts now.

jared commented

2026-09-19 17:39:06 +00:00

Author

Owner

Work started on branch ticket-616 via /os-sdlc:implement (session 33dcab60).

jared commented

2026-09-19 21:01:58 +00:00

Author

Owner

Resolution

Done: Model::Project::Validation refuses implementation-open when code_path, test_path, or lint_config does not exist under the root, before retired-test deletion and before the implementations row; CLI label project_validation with a three-line message naming target, existing candidate dirs, and the project.yaml to edit. Project::Test default path is now tests; FlatAttrs picks tests/ or test/ when no test_path line is set; project-config detects tests/test/spec at setup.

Evidence: Branch ticket-616 commit e011d051; pipeline implementation 7, 26 dispatches, behavior-verifier pass (dispatch 80), suite-check pass twice (81, 84; 1368 runs 0 failures), contract-auditor pass (85). Hand-finish: validation initially checked only explicitly configured keys; corrected to check effective paths and 10 fixtures now create lib/ and tests/.

Follow-ups: none captured. Dropped as minor: the candidate directory lists (tests/test/spec, lib/app/src) are repeated in Validation, FlatAttrs, and ProjectConfigCommand. The nil-command-means-pass branch in GateEvaluator stays per the ticket non-goals.

## Resolution **Done:** Model::Project::Validation refuses implementation-open when code_path, test_path, or lint_config does not exist under the root, before retired-test deletion and before the implementations row; CLI label project_validation with a three-line message naming target, existing candidate dirs, and the project.yaml to edit. Project::Test default path is now tests; FlatAttrs picks tests/ or test/ when no test_path line is set; project-config detects tests/test/spec at setup. **Evidence:** Branch ticket-616 commit e011d051; pipeline implementation 7, 26 dispatches, behavior-verifier pass (dispatch 80), suite-check pass twice (81, 84; 1368 runs 0 failures), contract-auditor pass (85). Hand-finish: validation initially checked only explicitly configured keys; corrected to check effective paths and 10 fixtures now create lib/ and tests/. **Follow-ups:** none captured. Dropped as minor: the candidate directory lists (tests/test/spec, lib/app/src) are repeated in Validation, FlatAttrs, and ProjectConfigCommand. The nil-command-means-pass branch in GateEvaluator stays per the ticket non-goals.

jared closed this issue

2026-09-19 21:01:58 +00:00

jared referenced this issue from a commit

2026-09-22 18:00:00 +00:00

feat(os-sdlc): implementation-open refuses a project whose code_path, test_path, or lint_config does not exist, naming candidates and the fix; Project::Test default path is tests (#616)

jared referenced this issue from a commit

2026-09-22 18:00:00 +00:00

Merge branch 'ticket-616' (#616)

jared referenced this issue from a commit

2026-09-22 18:00:00 +00:00

docs(status): os-sdlc leaf History rows for #616 and #618