cc-os/docs/graphify/06-querying-and-god-nodes.md

232 lines
14 KiB
Markdown
Raw Permalink Normal View History

Add Graphify setup & best-practices guide (11 docs + research) Research: dispatched 11 subagents to synthesize a creator interview into 11 standalone documents (overview, install, code/doc ingestion, backends, querying, token economics, workflows, best practices, and community tips), then reconciled cross-document contradictions against the GitHub repo (v0.8.30 release). Verified all substantive claims against primary sources (GitHub, interview, community, PyPI). Corrected interview errors: package is graphifyy (double-y), shell scripts now AST-supported, Slack/meetings/OneNote connectors are roadmap not shipped. Settled flag version disputes (--token-budget vs. --budget) by grepping raw README bytes. Applied honest token-savings framing (1–49x measured, not 70–90x marketing). Every claim tagged inline by source confidence ([github] / [interview] / [community] / [unverified]). Zero broken links, zero residual contradictions. Deliverable: docs/graphify/ - 00-README.md: index, reading order, provenance - 01-overview-concepts.md: god nodes, neuro-symbolic, vs. Obsidian - 02-installation-setup.md: install, register, first run - 03-ingesting-code-ast.md: tree-sitter, 33 languages, multi-repo - 04-ingesting-docs-knowledge.md: PDF/media/YouTube/Google Workspace - 05-local-models-and-backends.md: Ollama vs. cloud, privacy - 06-querying-and-god-nodes.md: god-nodes-first discipline - 07-token-economics-and-updates.md: savings honestly, --update, SHA hashing - 08-workflows-and-use-cases.md: onboarding, bug-trace, audits, second brain - 09-best-practices-checklist.md: do/don't reference + quick commands - external-tips.md: community tips, gotchas, savings debate Also included: graphify-interview (raw creator interview, source material), memory-systems-compared060326 (research reference).
2026-06-03 20:45:07 +00:00
# Querying the Graph & God Nodes
Querying is the highest-leverage skill in Graphify: a good query makes the graph *do the reading for you* so your assistant spends tokens on an answer, not on re-grepping the corpus. This page is the retrieval playbook — start at god nodes, scalpel down, and trust the graph as your memory.
Prerequisite: you've built a graph (`graphify-out/graph.json`, `GRAPH_REPORT.md`, `graph.html`). See [03-ingesting-code-ast.md](03-ingesting-code-ast.md) and [04-ingesting-docs-knowledge.md](04-ingesting-docs-knowledge.md). The cost story behind "ask the graph, not the corpus" lives in [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
---
## The core principle: prompt the graph, don't make the LLM read the corpus
The single most important habit, in the creator's words:
> "When you are prompting the graph, don't ask the LLM to go through the graph — rather ask the LLM to extract from the graph. So the graph is the memory, the graph is the context." `[interview]`
The graph **is** the context/memory. You want the assistant to *extract from* it (pull the specific nodes/edges that answer the question), not *traverse* the whole thing (which just re-spends the tokens you built the graph to save) `[interview]`.
This is backed by how the tool installs itself. The persistent install config tells your assistant to:
> "consult the knowledge graph for codebase questions — preferring scoped queries like `graphify query "<question>"` over reading the full report or grepping raw files." `[github]`
On Claude Code and Gemini CLI a hook even fires *before* search-style tool calls (and before reading source files one-by-one) to nudge the assistant onto the graph path `[github]`. So the discipline is partly automated — but you still drive it with good queries.
---
## Step 1 — Always ask for the god nodes first
**God nodes** are the most-connected concepts — the load-bearing spine of the project.
> "God nodes — the most-connected concepts in your project. Everything flows through these." `[github]`
The retrieval playbook starts here every time:
> "First of all you prompt to give you the god nodes from the whole corpus first of all ... from god nodes you can easily see where you have to prompt and write to minimize as much tokens as possible." `[interview]`
Get the high-level architecture in one shot, *then* "scalpel" down into the specific nodes you actually need `[interview]`. That ordering — map first, drill second — is where the token savings come from.
### How to actually surface god nodes (important)
There is **no** dedicated `graphify god-nodes` command or `--god-nodes` flag. God nodes are surfaced in two honest ways:
1. **Read the top of `GRAPH_REPORT.md`.** God nodes are the first thing in the report `[github]`:
> "What's in the report: **God nodes** — the most-connected concepts ... Everything flows through these." `[github]`
2. **Ask your assistant** for the high-level architecture; with the graph installed it reads the graph rather than the files `[github]`.
Do *not* expect `graphify query "what are the god nodes?"` to return the structural ranking — `query` is *semantic search* over content, while the god-node ranking is a structural (degree-based) result that lives in the report `[github]`. You can ask a semantic question to get oriented, but the authoritative hub list is the report.
To tune that ranking, see `--exclude-hubs` under [clusters](#step-4--community--cluster-detection) below.
---
## Step 2 — Scalpel down: query / path / explain
Three verified commands cover the drill-down. (Inside an IDE skill they take a `/` prefix, e.g. `/graphify query`; from a plain terminal drop the slash, e.g. `graphify query`. Same command, two entry points `[github]`. On Windows PowerShell, always use the no-slash form `[github]`.)
| Command | What it does | Use it to |
|---|---|---|
| `graphify query "<question>"` | Semantic search over the graph `[github]` | Find the nodes relevant to a question |
| `graphify path "Node1" "Node2"` | Find connections between two nodes `[github]` | Trace how two things relate |
| `graphify explain "Node"` | Single-node deep-dive `[github]` | Scalpel into one specific node |
```bash
# semantic search — "extract from the graph"
graphify query "what connects auth to the database?" # [github]
graphify query "show the auth flow" # [github]
# point at a specific graph file if not in the default location
graphify query "what connects DigestAuth to Response?" --graph graphify-out/graph.json # [github]
# bound the search to keep the answer (and tokens) tight
graphify query "..." --dfs --budget 1500 # [github]
# connection between two named nodes
graphify path "UserService" "DatabasePool" # [github]
graphify path "DigestAuth" "Response" # [github]
# deep-dive a single node — the literal "scalpel"
graphify explain "RateLimiter" # [github]
```
The `--dfs --budget` flags are the direct, runnable expression of "minimize tokens": you cap the traversal/budget so the assistant extracts a bounded slice instead of wandering the whole graph `[github]`.
### Architecture / call-flow diagram
For a visual map of how the system fits together:
```bash
graphify export callflow-html # Mermaid architecture/call-flow HTML [github]
graphify export callflow-html --max-sections 8 # cap the number of generated sections [github]
graphify export callflow-html --output docs/arch.html
graphify export callflow-html ./some-repo/graphify-out
```
If you installed the git hook, this auto-regenerates on every commit `[github]`. This pairs well with the god-node read: the report tells you *which* nodes matter, the call-flow HTML shows you *how* they connect.
---
## Step 3 — Read the confidence tags
Every **inferred** relationship is labeled so you know how much to trust it:
> "Confidence tags — every inferred relationship is marked `EXTRACTED`, `INFERRED`, or `AMBIGUOUS`. You always know what was found vs guessed." `[github]`
| Tag | Meaning `[github]` (overview doc) | How much to trust |
|---|---|---|
| `EXTRACTED` | "directly found in source" | High — deterministic; for code this is the AST `[github]` |
| `INFERRED` | "logically deduced" | Medium — a model reasoned it out |
| `AMBIGUOUS` | "uncertain connections" | Low — verify before acting |
Rule of thumb: code edges are `EXTRACTED` from the tree-sitter AST with no model involved `[github]`, so they're solid; `INFERRED`/`AMBIGUOUS` edges came from a language model reading prose `[github]` — weight them down and confirm before you act. (Confidence-tag concept also covered in [01-overview-concepts.md](01-overview-concepts.md).)
---
## Step 4 — Community / cluster detection
Graphify groups related entities into **communities**, and you can rerun clustering at adjustable granularity *without* re-extracting (so it's free — no API calls):
```bash
graphify cluster-only ./my-project # rerun clustering on existing graph [github]
graphify cluster-only ./my-project --resolution 1.5 # more, smaller communities [github]
graphify cluster-only ./my-project --exclude-hubs 99 # exclude p99-degree nodes from partitioning [github]
```
The same controls exist as flags on a build run:
```bash
/graphify . --cluster-only # rerun clustering, no re-extract [github]
/graphify . --cluster-only --resolution 1.5 # more granular communities [github]
/graphify . --cluster-only --exclude-hubs 99 # suppress utility super-hubs from god-node rankings [github]
```
- **`--resolution`** raises granularity: a higher value (e.g. `1.5`) yields *more, smaller* communities; lower values yield fewer, larger ones `[github]`. Sweep it to find the topic-cluster scale that matches how you think about the project.
- **`--exclude-hubs 99`** drops the very highest-degree utility nodes (e.g. a logger imported everywhere) from the partitioning / god-node ranking so they don't swamp the real structure `[github]`.
**Cross-community links** — relationships that bridge otherwise-separate clusters — are the thing the creator says plain visualizers (e.g. Obsidian) can't surface `[interview]`. There is **no** dedicated flag for them; they appear in the report as **"surprising connections"**:
> "Surprising connections — links between things that live in different files or modules. Ranked by how unexpected they are." `[github]`
So: clusters come from `cluster-only`/`--resolution`; the bridges between them are the ranked "surprising connections" in `GRAPH_REPORT.md`.
---
## Step 5 — The god-node diagnostic heuristic
A rough health signal from the creator:
> "If you have large amount of god nodes that means that something has gone wrong with the cohesion or maybe with your codebase ... if you have a decent number of god nodes ... things have worked fine." `[interview]`
Treat this as an `[interview]` heuristic, not a measured threshold — there's no defined "too many." `[unverified claim]` But it's testable: run `--exclude-hubs 99` to strip out the utility super-hubs and see whether a sane architecture remains underneath `[github]`. If a huge, sprawling god-node set survives even after excluding the obvious utility hubs, that's a soft signal of poor cohesion worth a closer look. (`--exclude-hubs` adjusts the *ranking*; it does not *fix* cohesion.)
---
## Putting it together: inspecting a fresh graph
A concrete sequence for a graph you've never seen (e.g. a new hire on an unfamiliar codebase — the interview's headline scenario `[interview]`):
```bash
# 1. Build the graph (AST for code = free, no API calls) [github][interview]
/graphify .
# 2. Read the high-level map FIRST: god nodes + surprising
# connections are at the top of the report [github][interview]
# -> open graphify-out/GRAPH_REPORT.md
# 3. See how the spine connects, visually
graphify export callflow-html # [github]
# 4. Sanity-check cohesion: strip utility super-hubs, re-look
graphify cluster-only . --exclude-hubs 99 # [github]
# too many god nodes left? possible cohesion problem [interview / unverified]
# 5. Scalpel into specifics — extract from the graph, bounded
graphify query "what connects auth to the database?" --dfs --budget 1500 # [github]
graphify path "UserService" "DatabasePool" # [github]
graphify explain "RateLimiter" # [github]
# 6. Tune cluster granularity if topics feel too coarse/fine
graphify cluster-only . --resolution 1.5 # [github]
```
Throughout: read the **confidence tag** on any edge before you act on it — `EXTRACTED` is trustworthy, `INFERRED`/`AMBIGUOUS` needs a second look `[github]`.
### Example queries to copy
```bash
graphify query "what connects attention to the optimizer?" # [github]
graphify query "show the auth flow" # [github]
graphify path "DigestAuth" "Response" # [github]
graphify explain "SwinTransformer" # [github]
```
---
## Repeated / programmatic access (MCP)
For an assistant that hits the graph many times in a session, expose it as an MCP server instead of one-off CLI calls:
```bash
python -m graphify.serve graphify-out/graph.json # [github]
```
This gives structured tools: `query_graph`, `get_node`, `get_neighbors`, `shortest_path`, `list_prs`, `get_pr_impact`, `triage_prs` `[github]`. Note `get_neighbors` / `shortest_path` are the programmatic equivalents of `explain` / `path`. (PR-impact tooling — `graphify prs` — is covered in [08-workflows-and-use-cases.md](08-workflows-and-use-cases.md).)
---
## Quick reference
| Goal | Do this |
|---|---|
| See the architecture spine | Read god nodes at top of `GRAPH_REPORT.md` `[github]` |
| Visual call-flow | `graphify export callflow-html` `[github]` |
| Find relevant nodes (semantic) | `graphify query "..."` (add `--dfs --budget N`) `[github]` |
| Connect two named nodes | `graphify path "A" "B"` `[github]` |
| Deep-dive one node | `graphify explain "Node"` `[github]` |
| Re-cluster (free) | `graphify cluster-only . --resolution 1.5` `[github]` |
| De-noise god-node ranking | `--exclude-hubs 99` `[github]` |
| Bridges between clusters | "surprising connections" in the report `[github]` |
| Judge trust of an edge | Read its `EXTRACTED` / `INFERRED` / `AMBIGUOUS` tag `[github]` |
Index: [00-README.md](00-README.md) · Next: [07-token-economics-and-updates.md](07-token-economics-and-updates.md) · Workflows: [08-workflows-and-use-cases.md](08-workflows-and-use-cases.md)
---
## Open questions / unverified
- **No `god-nodes` command exists.** God nodes are surfaced via `GRAPH_REPORT.md` and the assistant, not a dedicated CLI verb. `graphify query` is *semantic search*, not the structural hub ranking — don't conflate them. `[github]`
- **"Too many god nodes = poor cohesion"** is an `[interview]` heuristic with no defined threshold. `[unverified claim]` `--exclude-hubs` lets you *test* it but does not fix cohesion.
- **Cross-community links have no flag.** They map to the report's ranked "surprising connections" `[github]`; the interview's framing that Obsidian "can't do cross-community interaction" `[interview]` is the creator's claim, not independently verified.
- **`--resolution` direction** (higher = more, smaller communities) is from the README's inline comments `[github]`; exact algorithm/parameter semantics aren't documented in the README.
- **`--dfs --budget`** appears in the README command reference `[github]` but the exact budget units (tokens? nodes? hops?) and the `--budget` default are not spelled out there. `[unverified claim]`
- **graphifylabs.ai** returns HTTP 403 to plain fetch, so any query syntax documented only on the official site is not directly verified here.
- **Token-savings figures** ("70x"/"90x"/"20x") are sales claims handled in [07-token-economics-and-updates.md](07-token-economics-and-updates.md), not retrieval facts. `[unverified claim]`