jared/cc-os

Watch

Fork 0

Find why extensionless scripts and one vault note are missing from the graphs #1566

New issue

Closed

opened 2026-11-05 14:19:07 +00:00 by jared · 4 comments

jared commented

2026-11-05 14:19:07 +00:00

Owner

Problem

Two inputs are missing from graphs that should hold them. The cause is unknown for both.

In the cc-os graph, 101 of 102 tracked extensionless scripts under bin/ have no node. Only plugins/os/bin/handoff has one. Three of them are in the root bin/: arch-metrics, refresh-plugins, bin/test. All start with #!/usr/bin/env ruby.
In the vault graph, reference/gitignore-trailing-slash-does-not-match-symlinks.md has no node. It is not in manifest.json or the log, and it is not git-ignored.
The missing bin/refresh-plugins node makes question 10 of the 10-question set unanswerable.
Evidence: /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/graphify-diagnosis-evidence.md and /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/vault-graph-weak-retrieval-evidence.md (copied below; do not depend on the paths).

Observed

detect.py:530-553 classifies extensionless files by shebang, yet 1 of 102 is in the graph (graphify 0.9.102).
GRAPH_REPORT: 605 files unclassified (.tsv 307, .c4 251, no-extension 23).
Other file types are covered: 1,192 of 1,210 non-ignored .md files, 1,796 of 1,796 .rb, 6 of 6 .js, 25 of 25 .py.
Vault graph: 329 of 330 non-journal notes are present. The absent note is the one above. A dangling stub, "Git ignore trailing slash does not match symlinks", sits on two citing notes.
Not determined: why 101 of 102 are skipped (no rebuild was allowed); why the vault note was never scanned.

Reproduce

git ls-files 'bin/*' | grep -v '\.'

Then search graphify-out/graph.json for source_file equal to bin/refresh-plugins. It has no match.

Approach

Investigate first. Find where each file drops out: detect, the ignore rules, the manifest, or extraction. Then correct the cause, and rebuild both graphs. Do not add a workaround for one file.

Required work:

Name the cause for the extensionless scripts and record it on this ticket.
Name the cause for the missing vault note and record it on this ticket.
Correct each cause in the code or configuration that owns it.

Orchestrator work: rebuild each graph and count the nodes.

Acceptance criteria

Given the corrected cc-os graph, when it is rebuilt, then bin/refresh-plugins has a node.
Given the corrected cc-os graph, when it is rebuilt, then at least 101 of the 102 tracked extensionless scripts have a node, or this ticket records the reason for each exclusion.
Given the corrected vault graph, when it is rebuilt, then reference/gitignore-trailing-slash-does-not-match-symlinks.md has a node.

Non-goals

Upstream reports go on the graphify defects ticket. Open ticket #1557 asks whether the vault note was skipped as hollow or under --allow-partial. It may resolve the vault half; check it first.

Origin

Trigger: graphify 3/10 diagnosis (question 10) and vault graph test (question 6), 2026-11-05.
Improvised this session: none.
Chain: question 10 fails ← bin/refresh-plugins has no node ← cause unknown, stop. The cause is not determined, so the chain ends at IMPROVISATION (none applied; the cause is open).
Root candidate: none found; searched backlog terms for this ticket (VERDICT: CREATE)
Where: graphify detect step (detect.py:530-553, graphify 0.9.102) and the vault scan manifest
Session: b10276a0-3582-48b5-b0fe-f13795bdcc75
Transcript: /home/jared/.claude/projects/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75.jsonl
Parent: #1540.

## Problem Two inputs are missing from graphs that should hold them. The cause is unknown for both. 1. In the cc-os graph, 101 of 102 tracked extensionless scripts under `bin/` have no node. Only `plugins/os/bin/handoff` has one. Three of them are in the root `bin/`: `arch-metrics`, `refresh-plugins`, `bin/test`. All start with `#!/usr/bin/env ruby`. 2. In the vault graph, `reference/gitignore-trailing-slash-does-not-match-symlinks.md` has no node. It is not in `manifest.json` or the log, and it is not git-ignored. The missing `bin/refresh-plugins` node makes question 10 of the 10-question set unanswerable. Evidence: /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/graphify-diagnosis-evidence.md and /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/vault-graph-weak-retrieval-evidence.md (copied below; do not depend on the paths). ## Observed - `detect.py:530-553` classifies extensionless files by shebang, yet 1 of 102 is in the graph (graphify 0.9.102). - GRAPH_REPORT: 605 files unclassified (.tsv 307, .c4 251, no-extension 23). - Other file types are covered: 1,192 of 1,210 non-ignored `.md` files, 1,796 of 1,796 `.rb`, 6 of 6 `.js`, 25 of 25 `.py`. - Vault graph: 329 of 330 non-journal notes are present. The absent note is the one above. A dangling stub, "Git ignore trailing slash does not match symlinks", sits on two citing notes. - Not determined: why 101 of 102 are skipped (no rebuild was allowed); why the vault note was never scanned. ## Reproduce ``` git ls-files 'bin/*' | grep -v '\.' ``` Then search `graphify-out/graph.json` for `source_file` equal to `bin/refresh-plugins`. It has no match. ## Approach Investigate first. Find where each file drops out: detect, the ignore rules, the manifest, or extraction. Then correct the cause, and rebuild both graphs. Do not add a workaround for one file. Required work: 1. Name the cause for the extensionless scripts and record it on this ticket. 2. Name the cause for the missing vault note and record it on this ticket. 3. Correct each cause in the code or configuration that owns it. Orchestrator work: rebuild each graph and count the nodes. ## Acceptance criteria 1. Given the corrected cc-os graph, when it is rebuilt, then `bin/refresh-plugins` has a node. 2. Given the corrected cc-os graph, when it is rebuilt, then at least 101 of the 102 tracked extensionless scripts have a node, or this ticket records the reason for each exclusion. 3. Given the corrected vault graph, when it is rebuilt, then `reference/gitignore-trailing-slash-does-not-match-symlinks.md` has a node. ## Non-goals Upstream reports go on the graphify defects ticket. Open ticket #1557 asks whether the vault note was skipped as hollow or under --allow-partial. It may resolve the vault half; check it first. ## Origin - Trigger: graphify 3/10 diagnosis (question 10) and vault graph test (question 6), 2026-11-05. - Improvised this session: none. - Chain: question 10 fails ← `bin/refresh-plugins` has no node ← cause unknown, stop. The cause is not determined, so the chain ends at IMPROVISATION (none applied; the cause is open). - Root candidate: none found; searched backlog terms for this ticket (VERDICT: CREATE) - Where: graphify detect step (detect.py:530-553, graphify 0.9.102) and the vault scan manifest - Session: b10276a0-3582-48b5-b0fe-f13795bdcc75 - Transcript: /home/jared/.claude/projects/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75.jsonl - Parent: #1540.

jared added the

project/cc-os

filed-by/user

category/investigation

labels

2026-11-05 14:19:07 +00:00

jared commented

2026-11-05 14:34:33 +00:00

Author

Owner

Start (code half): tracing why 101 of 102 extensionless scripts have no node. Vault-note half is left open for another agent.

jared commented

2026-11-05 14:37:28 +00:00

Author

Owner

Code half result (vault note half still open, not checked here).

Cause: not our config and not detect. Graphify state (graphify 0.9.102).

detect classifies all 101 scripts as code. A clean checkout scan gives 101 extensionless in files.code; classify_file(bin/refresh-plugins) is CODE via shebang ruby (detect.py:530-553 works).
Extraction works. extract() on the same 101 files gives 562 nodes across 102 source files (bin/refresh-plugins: 11 nodes). _get_extractor maps ruby shebangs to extract_ruby (extract.py:9598, 9733-9741).
The drop is in the manifest. graphify-out/manifest.json has a row with an ast_hash for all 101 scripts (all seen 2026-11-04 10:52:51) but no node in graph.json. Only plugins/os/bin/handoff was edited later (mtime 11:55), so only it was re-extracted. graphify update (cli.py:3089 -> watch.py _rebuild_code, incremental, watch.py:2419-2442) takes the change set from manifest ast_hash, so it skips the 101 files for good.
Proof, on a copy of the live graph.json and manifest.json in a worktree: graphify update . re-extracted 1 of 3838 files, 0 of the scripts gained a node. graphify update . --force gave 101 of 101 scripts with nodes, bin/refresh-plugins included (36,233 nodes, up from 29,922 in that same worktree copy).
What I could not prove: which earlier run stamped the rows without extracting them. Likely a graphify build before the shebang dispatch in extract.py (comment at 9590-9597 says files were "classified as code and then silently dropped"); the stamp survived the upgrade. Evidence for #1567: the incremental gate trusts ast_hash rows even when the graph has no node for that source.

Config change: none. .graphifyignore does not exclude these files. The repair is a one-time graphify update . --force on the live graph (graphify-out/ is gitignored, so no commit). I have not run it on the live graph. #1565 will rebuild it with --force anyway.
Acceptance 1 and 2 hold on the forced rebuild (101 of 101). Not closing: the vault half (criterion 3) is open.

Code half result (vault note half still open, not checked here). Cause: not our config and not detect. Graphify state (graphify 0.9.102). - detect classifies all 101 scripts as code. A clean checkout scan gives 101 extensionless in files.code; `classify_file(bin/refresh-plugins)` is CODE via shebang `ruby` (detect.py:530-553 works). - Extraction works. `extract()` on the same 101 files gives 562 nodes across 102 source files (bin/refresh-plugins: 11 nodes). `_get_extractor` maps ruby shebangs to extract_ruby (extract.py:9598, 9733-9741). - The drop is in the manifest. `graphify-out/manifest.json` has a row with an `ast_hash` for all 101 scripts (all `seen` 2026-11-04 10:52:51) but no node in graph.json. Only `plugins/os/bin/handoff` was edited later (mtime 11:55), so only it was re-extracted. `graphify update` (cli.py:3089 -> watch.py `_rebuild_code`, incremental, watch.py:2419-2442) takes the change set from manifest `ast_hash`, so it skips the 101 files for good. - Proof, on a copy of the live graph.json and manifest.json in a worktree: `graphify update .` re-extracted 1 of 3838 files, 0 of the scripts gained a node. `graphify update . --force` gave 101 of 101 scripts with nodes, `bin/refresh-plugins` included (36,233 nodes, up from 29,922 in that same worktree copy). - What I could not prove: which earlier run stamped the rows without extracting them. Likely a graphify build before the shebang dispatch in extract.py (comment at 9590-9597 says files were "classified as code and then silently dropped"); the stamp survived the upgrade. Evidence for #1567: the incremental gate trusts `ast_hash` rows even when the graph has no node for that source. Config change: none. `.graphifyignore` does not exclude these files. The repair is a one-time `graphify update . --force` on the live graph (graphify-out/ is gitignored, so no commit). I have not run it on the live graph. #1565 will rebuild it with --force anyway. Acceptance 1 and 2 hold on the forced rebuild (101 of 101). Not closing: the vault half (criterion 3) is open.

jared commented

2026-11-05 14:39:03 +00:00

Author

Owner

Vault note result, session b10276a0. After graphify update --force on the live vault, reference/gitignore-trailing-slash-does-not-match-symlinks.md has 5 nodes, and test question Q6 now ranks it first. The original skip cause is still not known. The same pass will not run on a scheduled rebuild until the #1545 step merges.

Vault note result, session b10276a0. After `graphify update --force` on the live vault, reference/gitignore-trailing-slash-does-not-match-symlinks.md has 5 nodes, and test question Q6 now ranks it first. The original skip cause is still not known. The same pass will not run on a scheduled rebuild until the #1545 step merges.

jared referenced this issue

2026-11-05 14:48:58 +00:00

Report graphify 0.9.102 query defects upstream #1567

jared commented

2026-11-05 14:48:59 +00:00

Author

Owner

Resolution

Done: Code half: 101 scripts lacked nodes although manifest.json held ast_hash rows, so incremental update skipped them. update --force repaired all 101 on the live cc-os graph. Vault half: the gitignore note had a manifest row and no node, and update --force gave it 5 nodes.

Evidence: classify_file returns CODE for bin/refresh-plugins. Live graph: bin/refresh-plugins has 11 nodes. Vault Q6 ranks the note first. Copy test: plain update added 0 script nodes, update --force added 101. How the rows became stale is not proved, and the issue did not reproduce through a normal update.

Follow-ups: none: a forced update is the repair, and #1545 carries the rebuild step

## Resolution **Done:** Code half: 101 scripts lacked nodes although manifest.json held ast_hash rows, so incremental update skipped them. update --force repaired all 101 on the live cc-os graph. Vault half: the gitignore note had a manifest row and no node, and update --force gave it 5 nodes. **Evidence:** classify_file returns CODE for bin/refresh-plugins. Live graph: bin/refresh-plugins has 11 nodes. Vault Q6 ranks the note first. Copy test: plain update added 0 script nodes, update --force added 101. How the rows became stale is not proved, and the issue did not reproduce through a normal update. **Follow-ups:** none: a forced update is the repair, and #1545 carries the rebuild step

jared closed this issue

2026-11-05 14:48:59 +00:00