--- summary: Evaluation of Graphify as the knowledge-layer engine for a personal AI memory system — fit by layer, rationale for adoption, empirical findings, and limitations discovered. tags: - type/reference - domain/knowledge-graphs - domain/llm - tool/graphify - scope/global source: cc-os date: 2026-06-09 --- # Graphify Evaluation: Fit as a Knowledge-Layer Engine > Evaluated: 2026-06-03. Source docs: Graphify GitHub + handbook. --- ## Summary Verdict **Graphify is a strong fit for a knowledge layer over a markdown vault and a natural fit for project-file code analysis. It is NOT the right tool for episodic/session logs, and it does not "connect" multiple sources on its own — that bridge still needs custom glue.** --- ## What Graphify Actually Is Graphify builds a knowledge graph from code and documents: - **Nodes**: entities (functions, classes, concepts, topics) - **Edges**: typed relationships with confidence tags (`EXTRACTED` = deterministic AST, `INFERRED` = LLM, `AMBIGUOUS` = flagged) - **Storage**: `graph.json` + optional exports (Obsidian sidecar, GraphML, Cypher) - **Query**: semantic graph traversal (`query`, `path`, `explain`) — not vector/embedding search - **MCP server**: `graphify.serve` exposes `query_graph`, `get_node`, `shortest_path` as Claude tools The core cost split: - **Code** → tree-sitter AST, 33 languages, deterministic, **zero LLM cost** - **Docs** → LLM entity extraction, can use local SLMs via Ollama (`--backend ollama`) --- ## Fit by Layer ### Layer 1: Episodic ("what happened") — NOT a fit for Graphify Episodic queries are time-anchored semantic lookups: "what was I working on last Tuesday?", "what client did we discuss the Stripe integration for?" Vector similarity over session logs is well-matched to this. A knowledge graph of session entities would add build overhead without improving recall for timeline queries. **Keep memsearch (Milvus Lite).** ### Layer 2: Knowledge ("how do we…") — strong fit **What Graphify replaces**: A hand-built tag-index CLI and a deferred vector semantic layer. Instead of manually adding tags to every note, a local SLM (Ollama + Qwen2.5 7B or Phi-4 14B) extracts entities automatically. You get entity nodes and relationship edges from the vault without requiring exhaustive frontmatter authoring. **What it adds that tags cannot**: Inferred cross-note relationships. Tags only filter ("give me notes tagged tool/semrush"). A graph can say "tool/semrush is connected to client/X via 3 project notes, and both reference convention/seo-workflow." That's a graph query, not a tag query. **What it does NOT replace**: The `summary` field in the frontmatter schema. Graphify extracts entities, not prose summaries. If summaries are the primary token-efficiency mechanism (the AI picks a file by reading its summary without opening it), they must remain author-controlled. **Known limitation — stale node drift**: Graphify's `--update` does not prune deleted symbols/notes — you must `--force` rebuild to clear ghost nodes. The graph is a snapshot with known drift. Mitigate by scheduling a periodic `--force` rebuild (e.g., triggered by a session-start staleness check when the rebuild stamp is older than N days). ### Project Files (code) — no-brainer add Project-file code analysis via AST is free (no LLM, no API key needed at runtime), deterministic, and produces exactly the call-graph / symbol map that "what does this codebase do?" queries need. Every project gets a `graph.json` at zero ongoing cost. Query via the MCP server. **Limitation** [github]: `graphify extract` may demand a backend credential even for pure-code extraction. Workaround: set a dummy key or use `--backend ollama` with a running (or not) Ollama instance — confirm at install time. --- ## The "Multiple Sources Connected" Question Graphify does **not** natively connect three separate graphs. It builds one graph per ingestion run. To connect project files, vault, and session logs in one queryable surface, you'd need: 1. **A combined ingestion run** that points at all three source trees, OR 2. **Three separate graphs** queried via three MCP tool instances and synthesized at the Claude layer Option 1 risks cross-contamination of unrelated client projects. Option 2 is the safer model: each source has its own graph, and Claude stitches results from multiple MCP calls. The more honest framing: Graphify replaces tags as the **vault entity index**, adds a **free code graph per project**, and leaves episodic (memsearch) alone. That's a meaningful simplification — you no longer need a bespoke tag index CLI — but it's a layer replacement, not a unified connector. --- ## What Changes vs. a Tag-Index Design | Prior design | With Graphify | |---|---| | Bespoke tag index CLI (e.g., Ruby/SQLite) | Replaced by `graphify query` + MCP server | | Manual tag-discipline on every note | Auto-extracted by local SLM on vault ingestion | | `summary` frontmatter (author-written, router hint) | Unchanged — Graphify doesn't generate these | | Deferred vector/semantic layer (e.g., QMD) | Graphify fills this role without vectors | | memsearch (episodic) | Unchanged | | Project files (unindexed) | Free code graph via AST | **What's saved**: The entire bespoke index build (a significant chunk of critical-path work) can be skipped. Significant scope reduction. **What's added**: A local Ollama SLM running on this machine for vault doc extraction. This is a new infrastructure dependency. On low-RAM machines, a 7B model at 4-bit takes ~5 GB VRAM/RAM. --- ## Empirical Findings (Graphify v0.8.31 + qwen2.5-coder:7b) [primary/measured — 2026-06-05] A discriminating test compared a cached-replay graph against a clean single-pass deep extraction (`--mode deep --token-budget 8000`) against a real vault. **Key finding — Graphify is a structure extractor, not a topic clusterer.** Even at `--mode deep`, no emergent shared-topic hub nodes appeared (no auto-synthesized "Pest Control" node, no "Niche Prospecting" node). All cross-note edges observed came from explicit references, wikilinks, or document-level semantic similarity — not from shared thematic identity. **Practical impact**: A test query ("how do we do niche prospecting outreach for pest control?") returned 3 starting notes, and traversal could not reach thematically related notes in separate communities because there were no connecting edges. This confirms that useful retrieval is gated on **migration scaffolding** (hub notes + wikilinks), not on Graphify's extraction power. **Build artifact caveat**: The cached graph was partially a build artifact — cross-note edges rose from 41% to 78% in a single-pass run. The structural finding (no emergent hub nodes) held in both runs. **Open empirical question**: Whether shared frontmatter facet tags (`client/`, `tool/`, `domain/`, etc.) cause Graphify to create edges between notes was NOT tested. Do not assume facet tags contribute to graph traversal retrieval until this is verified. **`reasoning_effort:"none"` issue**: The clean run required a local patch to `graphify/llm.py`. Treat the installed version as pinned until an official fix appears upstream. --- ## Design Conclusions from Evaluation 1. **Hub notes and wikilinks are mandatory, not optional**: The connective spine of the knowledge graph must be author-provided as part of any vault migration. Deferring this to "someday" means graph traversal retrieval will fail for cross-topic queries. 2. **Migration means more than frontmatter**: Adding `summary:` and six-facet tags to existing notes is necessary but not sufficient. Hub notes and wikilinks for key concepts must be authored alongside the frontmatter migration. 3. **Summary frontmatter survives the pivot**: Even though Graphify auto-extracts entities, the human-written `summary` field remains the token-efficient router hint that lets the AI pick a file without opening it. 4. **Stale-node drift is real and must be scheduled**: Rely on the lazy freshness model (write-hook `--update` + periodic session-start `--force` rebuild), not continuous re-extraction. --- ## Open Questions 1. **Rebuild cadence**: How often do you rebuild the vault graph? On every session-start (expensive), on vault git commit (right cadence), or manually? The `--update` flag helps but stale-node drift means periodic `--force` rebuilds are needed. 2. **Model choice for vault extraction**: Test Qwen2.5 7B and Phi-4 14B against a sample of vault notes and review god-node quality before committing. [unverified claim: no official recommendation exists] 3. **Facet-tag-to-graph-edge question**: Do shared frontmatter facet tags cause Graphify to create edges between notes, or does graph connectivity come only from explicit wikilinks/references and semantic similarity? Resolve before designing graph-traversal retrieval skills. 4. **Larger extraction model**: Whether a substantially larger SLM (14B, 30B) would synthesize emergent topic-hub nodes is untested. Secondary — the design does not depend on it. 5. **Cross-client isolation**: Project code graphs should be per-client-project, not merged. Separate `graphify-out/` per project is the safe default. 6. **MCP server management**: Running `graphify.serve` for vault + N project graphs means N+1 MCP server instances. Manageable for small N; a single meta-server may be needed at scale.