Exclude test/ and .pbm/ from the cc-os graph #1565

New issue

Closed

opened 2026-11-05 14:19:06 +00:00 by jared · 2 comments

jared commented

2026-11-05 14:19:06 +00:00

Owner

Problem

The cc-os .graphifyignore excludes tests/ and **/tests/, but the repo uses test/ directories. It also omits .pbm/, which .gitignore line 28 already ignores.
So 2,543 test nodes and 997 .pbm nodes are in the cc-os graph (32,134 nodes).
Noise takes seed slots. In question 5 the top seed is glob from .pbm/pb_data/types.d.ts. In question 7 a .pbm create node takes a seed slot.
Evidence: /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/graphify-diagnosis-evidence.md (copied below; do not depend on the path).

Observed

Whole graph (32,134 nodes): generated .pbm style files 997 (3.1%); docs, openspec and no-path nodes 7,222 (22.5%); tests (test/, tests/, eval/) 5,576 (17.3%); src 18,339 (57.1%).
Nodes under test/ directories: 2,543.
The 10-question set scores 3 of 10 on this graph. The limit is 7 of 10.
Test nodes reach 49% of the nodes in the output for question 10.

Reproduce

n/a: a configuration gap. Count the nodes whose source_file starts with test/ or .pbm/ in graphify-out/graph.json.

Approach

Add test/ and .pbm/ to .graphifyignore. Run the 10-question set (testset.tsv from the #1540 audit) before and after the rebuild. Record both scores and the node counts.

Required work:

Add test/ and .pbm/ lines to .graphifyignore.

Orchestrator work: rebuild the graph with graphify update. Measure the 10-question set before the change and after the rebuild. Record both results on this ticket.

Acceptance criteria

Given the updated .graphifyignore, when the graph is rebuilt, then no node has a source_file under test/ or .pbm/.
Given the same 10 questions, when they run before and after the rebuild, then both scores and the node counts are recorded on this ticket.

Non-goals

Raising the 10-question score to 7 is not the pass rule here. Other causes (label-only matching, seed de-duplication) have their own tickets.

Origin

Trigger: graphify 3/10 diagnosis on the cc-os graph, 2026-11-05.
Improvised this session: none (read-only; no rebuild).
Chain: question 5 and 7 lose a seed slot ← .pbm nodes are indexed ← .graphifyignore lacks .pbm/ and test/ ← DESIGN (.graphifyignore; .gitignore line 28 lists .pbm/).
Root candidate: none found; searched backlog terms for this ticket (VERDICT: CREATE)
Where: .graphifyignore
Session: b10276a0-3582-48b5-b0fe-f13795bdcc75
Transcript: /home/jared/.claude/projects/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75.jsonl
Parent: #1540.

## Problem The cc-os `.graphifyignore` excludes `tests/` and `**/tests/`, but the repo uses `test/` directories. It also omits `.pbm/`, which `.gitignore` line 28 already ignores. So 2,543 test nodes and 997 `.pbm` nodes are in the cc-os graph (32,134 nodes). Noise takes seed slots. In question 5 the top seed is `glob` from `.pbm/pb_data/types.d.ts`. In question 7 a `.pbm` `create` node takes a seed slot. Evidence: /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/graphify-diagnosis-evidence.md (copied below; do not depend on the path). ## Observed - Whole graph (32,134 nodes): generated `.pbm` style files 997 (3.1%); docs, openspec and no-path nodes 7,222 (22.5%); tests (`test/`, `tests/`, `eval/`) 5,576 (17.3%); src 18,339 (57.1%). - Nodes under `test/` directories: 2,543. - The 10-question set scores 3 of 10 on this graph. The limit is 7 of 10. - Test nodes reach 49% of the nodes in the output for question 10. ## Reproduce n/a: a configuration gap. Count the nodes whose `source_file` starts with `test/` or `.pbm/` in `graphify-out/graph.json`. ## Approach Add `test/` and `.pbm/` to `.graphifyignore`. Run the 10-question set (testset.tsv from the #1540 audit) before and after the rebuild. Record both scores and the node counts. Required work: 1. Add `test/` and `.pbm/` lines to `.graphifyignore`. Orchestrator work: rebuild the graph with `graphify update`. Measure the 10-question set before the change and after the rebuild. Record both results on this ticket. ## Acceptance criteria 1. Given the updated `.graphifyignore`, when the graph is rebuilt, then no node has a `source_file` under `test/` or `.pbm/`. 2. Given the same 10 questions, when they run before and after the rebuild, then both scores and the node counts are recorded on this ticket. ## Non-goals Raising the 10-question score to 7 is not the pass rule here. Other causes (label-only matching, seed de-duplication) have their own tickets. ## Origin - Trigger: graphify 3/10 diagnosis on the cc-os graph, 2026-11-05. - Improvised this session: none (read-only; no rebuild). - Chain: question 5 and 7 lose a seed slot ← `.pbm` nodes are indexed ← `.graphifyignore` lacks `.pbm/` and `test/` ← DESIGN (.graphifyignore; .gitignore line 28 lists `.pbm/`). - Root candidate: none found; searched backlog terms for this ticket (VERDICT: CREATE) - Where: .graphifyignore - Session: b10276a0-3582-48b5-b0fe-f13795bdcc75 - Transcript: /home/jared/.claude/projects/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75.jsonl - Parent: #1540.

jared added the

project/cc-os

filed-by/user

category/code-change

labels

2026-11-05 14:19:06 +00:00

jared commented

2026-11-05 14:34:34 +00:00

Author

Owner

Start: checking graph consumers, then adding test/ and .pbm/ to .graphifyignore and measuring the 10-question set before and after.

jared referenced this issue

2026-11-05 14:37:28 +00:00

Find why extensionless scripts and one vault note are missing from the graphs #1566

jared commented

2026-11-05 14:43:24 +00:00

Author

Owner

Resolution

Done: Added test/, **/test/ and .pbm/ to .graphifyignore. Rebuilt the cc-os graph with graphify update . --force (no LLM): nodes under test/ 2543 to 0, under .pbm/ 997 to 0, non-test nodes kept (docs 8000, lib 133).

Evidence: Commit 57d4beff, merged to main. 10-question set: baseline 3/10; forced rebuild without the ignore change 5/10; with the ignore change 5/10 (the ignore change itself added 0; the 3 to 5 gain comes from the full rebuild). Nodes: baseline 32134, after 33572 (control 36233). Q9 rank 1 to 3, still pass. Q2, Q3, Q6, Q7, Q10 still fail, limit of 7 not met. Detail: /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/harness/diff-1565.md and results-after-1565.json. Confound: the worktree lacks the gitignored .pbm/, so the .pbm effect is shown by a detect test, not by the control.

Follow-ups: none

## Resolution **Done:** Added test/, **/test/ and .pbm/ to .graphifyignore. Rebuilt the cc-os graph with graphify update . --force (no LLM): nodes under test/ 2543 to 0, under .pbm/ 997 to 0, non-test nodes kept (docs 8000, lib 133). **Evidence:** Commit 57d4beff, merged to main. 10-question set: baseline 3/10; forced rebuild without the ignore change 5/10; with the ignore change 5/10 (the ignore change itself added 0; the 3 to 5 gain comes from the full rebuild). Nodes: baseline 32134, after 33572 (control 36233). Q9 rank 1 to 3, still pass. Q2, Q3, Q6, Q7, Q10 still fail, limit of 7 not met. Detail: /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/harness/diff-1565.md and results-after-1565.json. Confound: the worktree lacks the gitignored .pbm/, so the .pbm effect is shown by a detect test, not by the control. **Follow-ups:** none

jared closed this issue

2026-11-05 14:43:25 +00:00

jared referenced this issue from a commit

2026-11-05 16:11:53 +00:00

fix(graphify): ignore test/ and .pbm/ so they stay out of the cc-os graph (#1565)