Report graphify 0.9.102 query defects upstream #1567

New issue

Closed

opened 2026-11-05 14:19:07 +00:00 by jared · 1 comment

jared commented

2026-11-05 14:19:07 +00:00

Owner

Problem

Graphify 0.9.102 query code makes the cc-os 10-question set score 3 of 10. The diagnosis found four behaviors in serve.py that cost seed slots or matches. One is a certain defect. The others may be design choices. The owner of graphify must decide, so this ticket sorts them and reports the defects upstream.
Evidence: /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/graphify-diagnosis-evidence.md and /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/vault-graph-weak-retrieval-evidence.md (copied below; do not depend on the paths).

Observed

Installed path: ~/.local/share/uv/tools/graphifyy/lib/python3.12/site-packages/graphify/. Query is token match on label, source_file path, and rationale. It is not embedding-based.

Same-label seed de-duplication (certain defect). serve.py:835-935 _pick_seeds de-duplicates seeds by label (_seed_label_key, #1766). Two different nodes with one label count as one. In question 4 the os-adr PreToolUse class ranked 2nd (501.18 against 501.46) and was dropped, because the os hook class has the same label "PreToolUse". Result: wrong plugin returned.
No stemming (serve.py:317-336, 339-342). "ensures" does not match "ensure"; "labels" does match. Query ensure_labels returns 0 nodes, because _ splits the query into tokens while the label is ensure_labels.rb.
Seed cap with score gap (serve.py:835-935). max_k=3, and selection stops below 20% of the top score. A per-term guaranteed seed is then added with no cap (3 to 7 seeds seen). In questions 2, 3, 5, 6 and 7 the expected node is in the graph and scored (rank 2, 70, 4, 44 and 6) but is never a seed. For example question 3 ranks 70 with score 10.0 against a top of 161.2.
Markdown demotion (serve.py:564-587 _query_path_weight). Weight 0.3 for tests, nodes with no source_file, document nodes, and files ending .md .rst .txt .yml .yaml .toml .cfg .ini .json or under doc/docs/example/examples, unless the question has test, doc, docs, documentation or readme. Every SKILL.md node and heading is scored x0.3. Question 7 still fails with "docs" (expected rank 6, score 130 against 305).
Rewrites with bare class or label names pass 5 of 7 failed questions (Q2 to Q6): "EnsureLabels", "GlobFiles", "CopCheck", "adr PreToolUse". Q7 and Q10 do not pass. Measurement only; not a fix.

Reproduce

cd /home/jared/dev/cc-os && graphify query "what does the os-adr PreToolUse hook class do" --budget 2000

The output holds plugins/os/lib/os_hooks/pre_tool_use.rb and not plugins/os-adr/lib/adr/pre_tool_use.rb.

Approach

Rule on each behavior: defect or design choice. Item 1 is a defect. Report each defect upstream with its evidence and the reproduction above. Record each design choice and the reason on this ticket, and do not report it as a bug. Use a workaround in cc-os only when the ruling says so.

Required work: none beyond the acceptance criteria.

Orchestrator work: file the upstream reports on the graphify tracker. Record the report links and the rulings on this ticket.

Acceptance criteria

Given the four behaviors, when the ticket closes, then each one carries a recorded ruling: defect or design choice, with the reason.
Given the same-label de-duplication, when the ticket closes, then an upstream report exists that cites the question 4 evidence.
Given each behavior ruled a defect, when the ticket closes, then an upstream report exists for it.

Non-goals

Patching graphify locally. Changing cc-os configuration (see the .graphifyignore ticket).

Origin

Trigger: graphify 3/10 diagnosis on the cc-os graph, 2026-11-05.
Improvised this session: none; reworded queries were measured only.
Chain: expected file not in the output ← node never chosen as a seed ← seed selection de-duplicates by label, caps seeds, and scores without stemming ← UPSTREAM-TOOL (graphify 0.9.102, serve.py).
Root candidate: none found; searched backlog terms for this ticket (VERDICT: CREATE)
Where: graphify 0.9.102 serve.py (_pick_seeds, _query_terms, _query_path_weight)
Session: b10276a0-3582-48b5-b0fe-f13795bdcc75
Transcript: /home/jared/.claude/projects/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75.jsonl
Parent: #1540.

## Problem Graphify 0.9.102 query code makes the cc-os 10-question set score 3 of 10. The diagnosis found four behaviors in `serve.py` that cost seed slots or matches. One is a certain defect. The others may be design choices. The owner of graphify must decide, so this ticket sorts them and reports the defects upstream. Evidence: /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/graphify-diagnosis-evidence.md and /tmp/claude-1000/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75/scratchpad/vault-graph-weak-retrieval-evidence.md (copied below; do not depend on the paths). ## Observed Installed path: `~/.local/share/uv/tools/graphifyy/lib/python3.12/site-packages/graphify/`. Query is token match on label, source_file path, and rationale. It is not embedding-based. 1. Same-label seed de-duplication (certain defect). `serve.py:835-935` `_pick_seeds` de-duplicates seeds by label (`_seed_label_key`, #1766). Two different nodes with one label count as one. In question 4 the os-adr `PreToolUse` class ranked 2nd (501.18 against 501.46) and was dropped, because the os hook class has the same label "PreToolUse". Result: wrong plugin returned. 2. No stemming (`serve.py:317-336`, `339-342`). "ensures" does not match "ensure"; "labels" does match. Query `ensure_labels` returns 0 nodes, because `_` splits the query into tokens while the label is `ensure_labels.rb`. 3. Seed cap with score gap (`serve.py:835-935`). `max_k=3`, and selection stops below 20% of the top score. A per-term guaranteed seed is then added with no cap (3 to 7 seeds seen). In questions 2, 3, 5, 6 and 7 the expected node is in the graph and scored (rank 2, 70, 4, 44 and 6) but is never a seed. For example question 3 ranks 70 with score 10.0 against a top of 161.2. 4. Markdown demotion (`serve.py:564-587` `_query_path_weight`). Weight 0.3 for tests, nodes with no source_file, document nodes, and files ending `.md .rst .txt .yml .yaml .toml .cfg .ini .json` or under doc/docs/example/examples, unless the question has test, doc, docs, documentation or readme. Every SKILL.md node and heading is scored x0.3. Question 7 still fails with "docs" (expected rank 6, score 130 against 305). Rewrites with bare class or label names pass 5 of 7 failed questions (Q2 to Q6): "EnsureLabels", "GlobFiles", "CopCheck", "adr PreToolUse". Q7 and Q10 do not pass. Measurement only; not a fix. ## Reproduce ``` cd /home/jared/dev/cc-os && graphify query "what does the os-adr PreToolUse hook class do" --budget 2000 ``` The output holds `plugins/os/lib/os_hooks/pre_tool_use.rb` and not `plugins/os-adr/lib/adr/pre_tool_use.rb`. ## Approach Rule on each behavior: defect or design choice. Item 1 is a defect. Report each defect upstream with its evidence and the reproduction above. Record each design choice and the reason on this ticket, and do not report it as a bug. Use a workaround in cc-os only when the ruling says so. Required work: none beyond the acceptance criteria. Orchestrator work: file the upstream reports on the graphify tracker. Record the report links and the rulings on this ticket. ## Acceptance criteria 1. Given the four behaviors, when the ticket closes, then each one carries a recorded ruling: defect or design choice, with the reason. 2. Given the same-label de-duplication, when the ticket closes, then an upstream report exists that cites the question 4 evidence. 3. Given each behavior ruled a defect, when the ticket closes, then an upstream report exists for it. ## Non-goals Patching graphify locally. Changing cc-os configuration (see the .graphifyignore ticket). ## Origin - Trigger: graphify 3/10 diagnosis on the cc-os graph, 2026-11-05. - Improvised this session: none; reworded queries were measured only. - Chain: expected file not in the output ← node never chosen as a seed ← seed selection de-duplicates by label, caps seeds, and scores without stemming ← UPSTREAM-TOOL (graphify 0.9.102, serve.py). - Root candidate: none found; searched backlog terms for this ticket (VERDICT: CREATE) - Where: graphify 0.9.102 serve.py (_pick_seeds, _query_terms, _query_path_weight) - Session: b10276a0-3582-48b5-b0fe-f13795bdcc75 - Transcript: /home/jared/.claude/projects/-home-jared-Documents-SecondBrain/b10276a0-3582-48b5-b0fe-f13795bdcc75.jsonl - Parent: #1540.

jared added the

plugin/os-vault

filed-by/user

category/cross-repo

labels

2026-11-05 14:19:07 +00:00

jared referenced this issue

2026-11-05 14:37:28 +00:00

Find why extensionless scripts and one vault note are missing from the graphs #1566

jared commented

2026-11-05 14:48:58 +00:00

Author

Owner

Resolution

Done: Tested six graphify behaviors with minimal repros on 0.9.102, the latest release. Ruled one a defect and reported it upstream. Ruled four design choices. Held one as not reportable.

Evidence: Upstream issue: https://github.com/Graphify-Labs/graphify/issues/5027. Rulings: A same-label seed drop is a defect. B no stemming, C seed cap, D Markdown weight, F directed path default are design choices. E stale ast_hash rows reproduce only with a hand-edited graph.json, so the cause is unproved. Repro output confirmed: grep count 0.

Follow-ups: none: E stays recorded on #1566

## Resolution **Done:** Tested six graphify behaviors with minimal repros on 0.9.102, the latest release. Ruled one a defect and reported it upstream. Ruled four design choices. Held one as not reportable. **Evidence:** Upstream issue: https://github.com/Graphify-Labs/graphify/issues/5027. Rulings: A same-label seed drop is a defect. B no stemming, C seed cap, D Markdown weight, F directed path default are design choices. E stale ast_hash rows reproduce only with a hand-edited graph.json, so the cause is unproved. Repro output confirmed: grep count 0. **Follow-ups:** none: E stays recorded on #1566

jared closed this issue

2026-11-05 14:48:58 +00:00