Make evidence collect skip git-ignored files #967

New issue

Open

opened 2026-10-07 18:49:57 +00:00 by jared · 2 comments

jared commented

2026-10-07 18:49:57 +00:00

Owner

Problem

bin/evidence collect reads files that git ignores. The ruby-lsp probe indexes the whole --root, including the ignored takedown copies under docs/takedown/*/scratch-*/. The collect run becomes 13 times slower, and the report gives evidence from copies that are not the project code.

Observed

On 2026-10-07 the worktree held 47,695 .rb files under docs/takedown, against about 1,540 tracked files. The same five tokens (IssueBody, IssueBody::Lines, FenceStack, TicketShape::Section, section_body) were collected twice on an idle machine:

Root	Time	Output lines
worktree, scratch copies present	208.8 s	1,490, of which 896 point into `scratch-*`
clean copy, no `docs/takedown`	16.1 s	none from scratch

The collect run takes longer than the 120 s Bash timeout, so it moves to the background. In takedown runs #963 and #965, blast-radius agents then waited 11 to 29 minutes for the background notification.

Reproduce

bin/evidence collect --root . IssueBody IssueBody::Lines FenceStack TicketShape::Section section_body

Run it in a worktree that contains docs/takedown/963/scratch-*.

Approach

evidence collect and its LSP workspace use only files that git tracks or that git does not ignore (git ls-files --cached --others --exclude-standard). This fixes the whole class: any ignored copy, cache, or build output is out of scope, not only the takedown scratch copies.

Rejected: exclude docs/takedown/*/scratch-* by a fixed path — another ignored copy elsewhere causes the same defect.
Rejected: move the scratch copies outside the repo — the #966 pipeline removes scratch builds, but the evidence CLI still reads other ignored files.

Origin

Trigger: #966 diagnosis of blast-radius idle time in takedown runs #963 and #965 (2026-10-07)
Improvised this session: none
Chain: blast-radius agent idles 11-29 min ← evidence collect exceeds the 120 s Bash timeout and runs in the background ← the ruby-lsp probe indexes ignored scratch copies (timed 208.8 s vs 16.1 s) ← evidence collect takes the whole --root with no git filter. End: DESIGN (bin/evidence collect, os-sdlc).
Root candidate: none; this ticket.
Where: os-sdlc bin/evidence collect and its LSP workspace setup
Session: 87a712ed-cbfb-44d0-a970-d1bbff2fd148
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-prism-architect/87a712ed-cbfb-44d0-a970-d1bbff2fd148.jsonl

VERDICT: CREATE — measured 208.8 s vs 16.1 s, chain ends in DESIGN, causes 11-29 min idle in #963/#965.

## Problem `bin/evidence collect` reads files that git ignores. The ruby-lsp probe indexes the whole `--root`, including the ignored takedown copies under `docs/takedown/*/scratch-*/`. The collect run becomes 13 times slower, and the report gives evidence from copies that are not the project code. ## Observed On 2026-10-07 the worktree held 47,695 `.rb` files under `docs/takedown`, against about 1,540 tracked files. The same five tokens (IssueBody, IssueBody::Lines, FenceStack, TicketShape::Section, section_body) were collected twice on an idle machine: | Root | Time | Output lines | | --- | --- | --- | | worktree, scratch copies present | 208.8 s | 1,490, of which 896 point into `scratch-*` | | clean copy, no `docs/takedown` | 16.1 s | none from scratch | The collect run takes longer than the 120 s Bash timeout, so it moves to the background. In takedown runs #963 and #965, blast-radius agents then waited 11 to 29 minutes for the background notification. ## Reproduce ``` bin/evidence collect --root . IssueBody IssueBody::Lines FenceStack TicketShape::Section section_body ``` Run it in a worktree that contains `docs/takedown/963/scratch-*`. ## Approach `evidence collect` and its LSP workspace use only files that git tracks or that git does not ignore (`git ls-files --cached --others --exclude-standard`). This fixes the whole class: any ignored copy, cache, or build output is out of scope, not only the takedown scratch copies. Rejected: exclude `docs/takedown/*/scratch-*` by a fixed path — another ignored copy elsewhere causes the same defect. Rejected: move the scratch copies outside the repo — the #966 pipeline removes scratch builds, but the evidence CLI still reads other ignored files. ## Origin - Trigger: #966 diagnosis of blast-radius idle time in takedown runs #963 and #965 (2026-10-07) - Improvised this session: none - Chain: blast-radius agent idles 11-29 min ← `evidence collect` exceeds the 120 s Bash timeout and runs in the background ← the ruby-lsp probe indexes ignored scratch copies (timed 208.8 s vs 16.1 s) ← `evidence collect` takes the whole `--root` with no git filter. End: DESIGN (`bin/evidence collect`, os-sdlc). - Root candidate: none; this ticket. - Where: os-sdlc `bin/evidence collect` and its LSP workspace setup - Session: 87a712ed-cbfb-44d0-a970-d1bbff2fd148 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-prism-architect/87a712ed-cbfb-44d0-a970-d1bbff2fd148.jsonl VERDICT: CREATE — measured 208.8 s vs 16.1 s, chain ends in DESIGN, causes 11-29 min idle in #963/#965.

jared added the

plugin/os-sdlc

filed-by/agent

labels

2026-10-07 18:49:57 +00:00

jared commented

2026-10-07 18:59:59 +00:00

Author

Owner

Design note (session 87a712ed): os-sdlc already has one hardcoded exclusion list, Codebase::EXCLUDED_SEGMENTS = %w[.sdlc vendor node_modules tmp] (plugins/os-sdlc/lib/os_sdlc/model/project/codebase.rb:6). bin/evidence does not use it; it queries the ruby-lsp workspace through LspRunner. Fix the class with one owner: Codebase decides which files belong to the project (git-ignored files excluded), and both the LSP workspace and evidence collect ask it, so a second exclusion list does not appear. Note: the eval harness scratch repos live outside the repo at ~/.cache/cc-os/takedown-runs (locations.rb:5), so they do not trigger this defect; the in-repo copies came from the production skill's designers writing docs/takedown//scratch-.

Design note (session 87a712ed): os-sdlc already has one hardcoded exclusion list, `Codebase::EXCLUDED_SEGMENTS = %w[.sdlc vendor node_modules tmp]` (plugins/os-sdlc/lib/os_sdlc/model/project/codebase.rb:6). `bin/evidence` does not use it; it queries the ruby-lsp workspace through `LspRunner`. Fix the class with one owner: `Codebase` decides which files belong to the project (git-ignored files excluded), and both the LSP workspace and evidence collect ask it, so a second exclusion list does not appear. Note: the eval harness scratch repos live outside the repo at ~/.cache/cc-os/takedown-runs (locations.rb:5), so they do not trigger this defect; the in-repo copies came from the production skill's designers writing docs/takedown/*/scratch-*.

jared referenced this issue

2026-10-07 19:00:42 +00:00

Delete takedown scratch repos after each run #968

jared commented

2026-10-09 16:58:53 +00:00

Author

Owner

Close ruling still open (2026-10-09). Remaining cost: ruby-lsp textDocument/references globs the whole workspace regardless of excludedPatterns (64 s to 52 s at 38.6k ignored files). Options: symlinked workspace from Codebase, ast-grep for references, or upstream. Owner has not ruled. See docs/handoffs/2026-10-08-takedown-autoresearch.md.

jared referenced this issue from a commit

2026-10-09 18:43:29 +00:00

docs(handoff): blast-radius diagnosis done, #967 filed

jared referenced this issue from a commit

2026-10-09 18:43:29 +00:00

docs(handoff): grader calibration committed; #967 #968 filed

jared referenced this issue from a commit

2026-10-09 18:43:29 +00:00

feat(os-sdlc): Codebase asks git which files are project files (#967)

jared referenced this issue from a commit

2026-10-09 18:43:29 +00:00

Merge #967: Codebase owns project files; evidence and LSP ask it