bin/evidence: a new file resolves to an unrelated file with the same basename #1760

New issue

Closed

opened 2026-11-13 20:39:47 +00:00 by jared · 2 comments

jared commented

2026-11-13 20:39:47 +00:00

Owner

Problem

bin/evidence collect resolves a path that does not exist to an unrelated file with the same basename.
TokenResolution#file_path does this in its fallback Dir.glob(root/**/basename).first.
The fallback runs before --expect-new is applied, so a new file in a design gets the lookups of an unrelated file.
The takedown final check then runs past its timeout and the takedown writes no design.

Observed

The takedown final check (Takedown::EvidenceCheck, plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/evidence_check.rb, TIMEOUT = 600) runs bin/evidence collect --root <scratch repo> <paths> --expect-new <paths> on the files of the final design.
The scratch repo is an export of the base commit. The same result occurs from the main checkout.
For a path that does not exist, TokenResolution#file_path falls back to Dir.glob(root/**/basename).first.
A new file therefore resolves to an unrelated file with the same basename, and the tool traces each method of that file through the language server.
The run on 2026-11-13 was the takedown of #1748. Its run directory is docs/takedown/1748-20261113-150953/, in the worktree .claude/worktrees/ticket-1742.
The last line of the run was FAILED final-check: evidence timed out after 600s.
The design had 16 paths. 14 of them were new.
Six new files resolved to unrelated files:

concept_cycle_check/kinds.rb resolved to docs/research/lint-lessons/2026-11-13-concept-cycle-kinds/simulation/kinds.rb (574 lookups).
graph.rb resolved to plugins/os-adr/lib/adr/record/matches/graph.rb (30 lookups).
kinds/record.rb resolved to plugins/os-adr/lib/adr/record.rb (22 lookups).
kinds/report.rb, kinds/ledger.rb, and kinds/report_test.rb resolved to files under docs/system-design-format/proof/ (17, 8, and 3 lookups).
The total was 721 lookups. 654 of them were about unrelated files.
One existing file, references.rb, took 78 s for 39 lookups. This is about 2 s for each lookup. 721 lookups need about 24 minutes.
The run wrote no final-design.json and an empty final-evidence.txt. The implementation phase cannot start from the takedown.
A larger timeout would give evidence about unrelated files.
A related defect is on the same code path. units_in_file makes one unit for each class in a file, and each unit traces all methods of the file.
The lookups of a file therefore multiply by its class count. The two existing files had 28 and 39 lookups in place of 14 and 13.

Reproduce

Export a commit that holds docs/research/lint-lessons/2026-11-13-concept-cycle-kinds/simulation/kinds.rb with git archive into an empty directory <export>.
Run this command:

plugins/os-sdlc/bin/evidence collect --root <export> plugins/os-sdlc/lib/os_sdlc/lint/concept_cycle_check/kinds.rb --expect-new plugins/os-sdlc/lib/os_sdlc/lint/concept_cycle_check/kinds.rb

The command traces the simulation file kinds.rb. It does not return in 100 s.

Origin

Trigger: the takedown of #1748 (docs/takedown/1748-20261113-150953/ in the worktree .claude/worktrees/ticket-1742), which ended with FAILED final-check: evidence timed out after 600s
Improvised this session: none
Chain: the takedown writes no design ← the final check bin/evidence collect times out ← a new file resolves to an unrelated file with the same basename ← DESIGN: the fallback in plugins/os-sdlc/lib/os_sdlc/evidence/resolution/token_resolution.rb (TokenResolution#file_path) runs before --expect-new
Root candidate: none found; searched evidence, TokenResolution, token_resolution, expect-new, basename, units_in_file
Where: plugins/os-sdlc/lib/os_sdlc/evidence/resolution/token_resolution.rb (TokenResolution#file_path)
Session: fdf2ac7d-9590-47d4-97e8-ebff9711ce46
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/fdf2ac7d-9590-47d4-97e8-ebff9711ce46.jsonl
VERDICT: CREATE - measured tool defect that blocked a pipeline run (hard floor); open #1678 covers a different fault.

## Problem `bin/evidence collect` resolves a path that does not exist to an unrelated file with the same basename. `TokenResolution#file_path` does this in its fallback `Dir.glob(root/**/basename).first`. The fallback runs before `--expect-new` is applied, so a new file in a design gets the lookups of an unrelated file. The takedown final check then runs past its timeout and the takedown writes no design. ## Observed The takedown final check (`Takedown::EvidenceCheck`, `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/command/evidence_check.rb`, `TIMEOUT = 600`) runs `bin/evidence collect --root <scratch repo> <paths> --expect-new <paths>` on the files of the final design. The scratch repo is an export of the base commit. The same result occurs from the main checkout. For a path that does not exist, `TokenResolution#file_path` falls back to `Dir.glob(root/**/basename).first`. A new file therefore resolves to an unrelated file with the same basename, and the tool traces each method of that file through the language server. The run on 2026-11-13 was the takedown of #1748. Its run directory is `docs/takedown/1748-20261113-150953/`, in the worktree `.claude/worktrees/ticket-1742`. The last line of the run was `FAILED final-check: evidence timed out after 600s`. The design had 16 paths. 14 of them were new. Six new files resolved to unrelated files: - `concept_cycle_check/kinds.rb` resolved to `docs/research/lint-lessons/2026-11-13-concept-cycle-kinds/simulation/kinds.rb` (574 lookups). - `graph.rb` resolved to `plugins/os-adr/lib/adr/record/matches/graph.rb` (30 lookups). - `kinds/record.rb` resolved to `plugins/os-adr/lib/adr/record.rb` (22 lookups). - `kinds/report.rb`, `kinds/ledger.rb`, and `kinds/report_test.rb` resolved to files under `docs/system-design-format/proof/` (17, 8, and 3 lookups). The total was 721 lookups. 654 of them were about unrelated files. One existing file, `references.rb`, took 78 s for 39 lookups. This is about 2 s for each lookup. 721 lookups need about 24 minutes. The run wrote no `final-design.json` and an empty `final-evidence.txt`. The implementation phase cannot start from the takedown. A larger timeout would give evidence about unrelated files. A related defect is on the same code path. `units_in_file` makes one unit for each class in a file, and each unit traces all methods of the file. The lookups of a file therefore multiply by its class count. The two existing files had 28 and 39 lookups in place of 14 and 13. ## Reproduce 1. Export a commit that holds `docs/research/lint-lessons/2026-11-13-concept-cycle-kinds/simulation/kinds.rb` with `git archive` into an empty directory `<export>`. 2. Run this command: ``` plugins/os-sdlc/bin/evidence collect --root <export> plugins/os-sdlc/lib/os_sdlc/lint/concept_cycle_check/kinds.rb --expect-new plugins/os-sdlc/lib/os_sdlc/lint/concept_cycle_check/kinds.rb ``` 3. The command traces the simulation file `kinds.rb`. It does not return in 100 s. ## Origin - Trigger: the takedown of #1748 (`docs/takedown/1748-20261113-150953/` in the worktree `.claude/worktrees/ticket-1742`), which ended with `FAILED final-check: evidence timed out after 600s` - Improvised this session: none - Chain: the takedown writes no design ← the final check `bin/evidence collect` times out ← a new file resolves to an unrelated file with the same basename ← DESIGN: the fallback in `plugins/os-sdlc/lib/os_sdlc/evidence/resolution/token_resolution.rb` (`TokenResolution#file_path`) runs before `--expect-new` - Root candidate: none found; searched evidence, TokenResolution, token_resolution, expect-new, basename, units_in_file - Where: plugins/os-sdlc/lib/os_sdlc/evidence/resolution/token_resolution.rb (`TokenResolution#file_path`) - Session: fdf2ac7d-9590-47d4-97e8-ebff9711ce46 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os-plugins-os-sdlc/fdf2ac7d-9590-47d4-97e8-ebff9711ce46.jsonl VERDICT: CREATE - measured tool defect that blocked a pipeline run (hard floor); open #1678 covers a different fault.

jared added the

plugin/os-sdlc

filed-by/agent

category/code-change

labels

2026-11-13 20:39:47 +00:00

jared referenced this issue

2026-11-13 20:39:52 +00:00

ConceptCycleCheck: classify each loop reference by kind #1748

jared referenced this issue

2026-11-13 20:54:40 +00:00

ConceptCycleCheck: classify each loop reference by kind #1748

jared referenced this issue from a commit

2026-11-14 15:24:11 +00:00

fix(os-sdlc): evidence does not resolve an expected-new path to a same-basename file (#1760)

jared commented

2026-11-14 15:24:29 +00:00

Author

Owner

Resolution

Done: A path named in --expect-new no longer resolves to an unrelated file with the same basename; the basename fallback stays for other paths.

Evidence: fdc97824a merged to main 51a5613db; test_expected_new_path_does_not_resolve_to_a_same_basename_file; suite 5566 runs 0 failures

Follow-ups: units_in_file multiplies lookups by class count: pending capture with ticket-skeptic verdict

## Resolution **Done:** A path named in --expect-new no longer resolves to an unrelated file with the same basename; the basename fallback stays for other paths. **Evidence:** fdc97824a merged to main 51a5613db; test_expected_new_path_does_not_resolve_to_a_same_basename_file; suite 5566 runs 0 failures **Follow-ups:** units_in_file multiplies lookups by class count: pending capture with ticket-skeptic verdict

jared closed this issue

2026-11-14 15:24:30 +00:00

jared commented

2026-11-14 15:25:41 +00:00

Author

Owner

Follow-up resolved: the units_in_file per-class lookup multiplication is DROPPED by ticket-skeptic (2026-11-14): depth-2 follow-up, cost not failure now that the basename fallback is fixed; re-file only if a real run times out on per-class fan-out.