SecondBrain/2026-06-09-graphify-evaluation-as-knowledge-layer.md

---
summary: Evaluation of Graphify as the knowledge-layer engine for a personal AI memory system — fit by layer, rationale for adoption, empirical findings, and limitations discovered.
tags:
  - type/reference
  - domain/knowledge-graphs
  - domain/llm
  - tool/graphify
  - scope/global
source: cc-os
date: 2026-06-09
---

# Graphify Evaluation: Fit as a Knowledge-Layer Engine

> Evaluated: 2026-06-03. Source docs: Graphify GitHub + handbook.

---

## Summary Verdict

**Graphify is a strong fit for a knowledge layer over a markdown vault and a natural fit for project-file code analysis. It is NOT the right tool for episodic/session logs, and it does not "connect" multiple sources on its own — that bridge still needs custom glue.**

---

## What Graphify Actually Is

Graphify builds a knowledge graph from code and documents:

- **Nodes**: entities (functions, classes, concepts, topics)
- **Edges**: typed relationships with confidence tags (`EXTRACTED` = deterministic AST, `INFERRED` = LLM, `AMBIGUOUS` = flagged)
- **Storage**: `graph.json` + optional exports (Obsidian sidecar, GraphML, Cypher)
- **Query**: semantic graph traversal (`query`, `path`, `explain`) — not vector/embedding search
- **MCP server**: `graphify.serve` exposes `query_graph`, `get_node`, `shortest_path` as Claude tools

The core cost split:
- **Code** → tree-sitter AST, 33 languages, deterministic, **zero LLM cost**
- **Docs** → LLM entity extraction, can use local SLMs via Ollama (`--backend ollama`)

---

## Fit by Layer

### Layer 1: Episodic ("what happened") — NOT a fit for Graphify

Episodic queries are time-anchored semantic lookups: "what was I working on last Tuesday?", "what client did we discuss the Stripe integration for?" Vector similarity over session logs is well-matched to this. A knowledge graph of session entities would add build overhead without improving recall for timeline queries. **Keep memsearch (Milvus Lite).**

### Layer 2: Knowledge ("how do we…") — strong fit

**What Graphify replaces**: A hand-built tag-index CLI and a deferred vector semantic layer. Instead of manually adding tags to every note, a local SLM (Ollama + Qwen2.5 7B or Phi-4 14B) extracts entities automatically. You get entity nodes and relationship edges from the vault without requiring exhaustive frontmatter authoring.

**What it adds that tags cannot**: Inferred cross-note relationships. Tags only filter ("give me notes tagged tool/semrush"). A graph can say "tool/semrush is connected to client/X via 3 project notes, and both reference convention/seo-workflow." That's a graph query, not a tag query.

**What it does NOT replace**: The `summary` field in the frontmatter schema. Graphify extracts entities, not prose summaries. If summaries are the primary token-efficiency mechanism (the AI picks a file by reading its summary without opening it), they must remain author-controlled.

**Known limitation — stale node drift**: Graphify's `--update` does not prune deleted symbols/notes — you must `--force` rebuild to clear ghost nodes. The graph is a snapshot with known drift. Mitigate by scheduling a periodic `--force` rebuild (e.g., triggered by a session-start staleness check when the rebuild stamp is older than N days).

### Project Files (code) — no-brainer add

Project-file code analysis via AST is free (no LLM, no API key needed at runtime), deterministic, and produces exactly the call-graph / symbol map that "what does this codebase do?" queries need. Every project gets a `graph.json` at zero ongoing cost. Query via the MCP server.

**Limitation** [github]: `graphify extract` may demand a backend credential even for pure-code extraction. Workaround: set a dummy key or use `--backend ollama` with a running (or not) Ollama instance — confirm at install time.

---

## The "Multiple Sources Connected" Question

Graphify does **not** natively connect three separate graphs. It builds one graph per ingestion run. To connect project files, vault, and session logs in one queryable surface, you'd need:

1. **A combined ingestion run** that points at all three source trees, OR
2. **Three separate graphs** queried via three MCP tool instances and synthesized at the Claude layer

Option 1 risks cross-contamination of unrelated client projects. Option 2 is the safer model: each source has its own graph, and Claude stitches results from multiple MCP calls.

The more honest framing: Graphify replaces tags as the **vault entity index**, adds a **free code graph per project**, and leaves episodic (memsearch) alone. That's a meaningful simplification — you no longer need a bespoke tag index CLI — but it's a layer replacement, not a unified connector.

---

## What Changes vs. a Tag-Index Design

| Prior design | With Graphify |
|---|---|
| Bespoke tag index CLI (e.g., Ruby/SQLite) | Replaced by `graphify query` + MCP server |
| Manual tag-discipline on every note | Auto-extracted by local SLM on vault ingestion |
| `summary` frontmatter (author-written, router hint) | Unchanged — Graphify doesn't generate these |
| Deferred vector/semantic layer (e.g., QMD) | Graphify fills this role without vectors |
| memsearch (episodic) | Unchanged |
| Project files (unindexed) | Free code graph via AST |

**What's saved**: The entire bespoke index build (a significant chunk of critical-path work) can be skipped. Significant scope reduction.

**What's added**: A local Ollama SLM running on this machine for vault doc extraction. This is a new infrastructure dependency. On low-RAM machines, a 7B model at 4-bit takes ~5 GB VRAM/RAM.

---

## Empirical Findings (Graphify v0.8.31 + qwen2.5-coder:7b) [primary/measured — 2026-06-05]

A discriminating test compared a cached-replay graph against a clean single-pass deep extraction (`--mode deep --token-budget 8000`) against a real vault.

**Key finding — Graphify is a structure extractor, not a topic clusterer.** Even at `--mode deep`, no emergent shared-topic hub nodes appeared (no auto-synthesized "Pest Control" node, no "Niche Prospecting" node). All cross-note edges observed came from explicit references, wikilinks, or document-level semantic similarity — not from shared thematic identity.

**Practical impact**: A test query ("how do we do niche prospecting outreach for pest control?") returned 3 starting notes, and traversal could not reach thematically related notes in separate communities because there were no connecting edges. This confirms that useful retrieval is gated on **migration scaffolding** (hub notes + wikilinks), not on Graphify's extraction power.

**Build artifact caveat**: The cached graph was partially a build artifact — cross-note edges rose from 41% to 78% in a single-pass run. The structural finding (no emergent hub nodes) held in both runs.

**Open empirical question**: Whether shared frontmatter facet tags (`client/`, `tool/`, `domain/`, etc.) cause Graphify to create edges between notes was NOT tested. Do not assume facet tags contribute to graph traversal retrieval until this is verified.

**`reasoning_effort:"none"` issue**: The clean run required a local patch to `graphify/llm.py`. Treat the installed version as pinned until an official fix appears upstream.

---

## Design Conclusions from Evaluation

1. **Hub notes and wikilinks are mandatory, not optional**: The connective spine of the knowledge graph must be author-provided as part of any vault migration. Deferring this to "someday" means graph traversal retrieval will fail for cross-topic queries.

2. **Migration means more than frontmatter**: Adding `summary:` and six-facet tags to existing notes is necessary but not sufficient. Hub notes and wikilinks for key concepts must be authored alongside the frontmatter migration.

3. **Summary frontmatter survives the pivot**: Even though Graphify auto-extracts entities, the human-written `summary` field remains the token-efficient router hint that lets the AI pick a file without opening it.

4. **Stale-node drift is real and must be scheduled**: Rely on the lazy freshness model (write-hook `--update` + periodic session-start `--force` rebuild), not continuous re-extraction.

---

## Open Questions

1. **Rebuild cadence**: How often do you rebuild the vault graph? On every session-start (expensive), on vault git commit (right cadence), or manually? The `--update` flag helps but stale-node drift means periodic `--force` rebuilds are needed.

2. **Model choice for vault extraction**: Test Qwen2.5 7B and Phi-4 14B against a sample of vault notes and review god-node quality before committing. [unverified claim: no official recommendation exists]

3. **Facet-tag-to-graph-edge question**: Do shared frontmatter facet tags cause Graphify to create edges between notes, or does graph connectivity come only from explicit wikilinks/references and semantic similarity? Resolve before designing graph-traversal retrieval skills.

4. **Larger extraction model**: Whether a substantially larger SLM (14B, 30B) would synthesize emergent topic-hub nodes is untested. Secondary — the design does not depend on it.

5. **Cross-client isolation**: Project code graphs should be per-client-project, not merged. Separate `graphify-out/` per project is the safe default.

6. **MCP server management**: Running `graphify.serve` for vault + N project graphs means N+1 MCP server instances. Manageable for small N; a single meta-server may be needed at scale.