cc-os/openspec/changes/graphify-ollama-setup/design.md

Context

This change implements Steps 2a–2d of docs/memory-system/05-implementation-process.md — the critical path of the memory system. Step 2c (the Claude reference-set gate) is already executed: 18 gold-standard fragments (6 fixtures × 3 tiers) exist in docs/memory-system/benchmark/reference-outputs/, with the claude-opus-4-8 outputs as the scoring rubric (specced under reference-extraction-benchmark). What remains is to stand up the local extraction toolchain, score local Ollama models against that rubric, pick the model, and build the first vault graph.

Constraints:

• The vault is the existing ~/Documents/SecondBrain (ADR-012); the build runs against it as-is, no bulk migration (ADR-013, incremental-migration spec).
• Markdown is the single source of truth; graph artifacts are disposable (ADR-008).
• A feasibility gut-check found gemma4:e4b runs at ~74 tok/s on the local GPU, but quality against the references has not been measured. The implementation-process doc is internally inconsistent on the model (e4b vs e2b) — a tell that nothing is locked.

Goals / Non-Goals

Goals:

• Install and verify Graphify; configure Ollama for extraction.
• Score candidate Ollama models against the existing Opus rubric on the 6 fixtures (quality + speed) and select the extraction model by evidence.
• Build the initial vault graph with the selected model and sanity-check god-nodes.

Non-Goals:

• Regenerating or modifying the Step 2c reference set (consumed, not produced here).
• Per-project code graphs / Step 2e (separate, free tree-sitter path; out of range).
• Hooks, memsearch, sync, plugin packaging (Steps 3–6).
• Bulk vault migration (deferred to last per ADR-013).
• Choosing the sync mechanism or stale-rebuild threshold (Open questions §2–3).

Decisions

Selection by scoring, not by gut-check. The model is chosen by comparing each candidate's Graphify-shaped output to the Opus reference per fixture (entity correctness, relationship typing, confidence-tag accuracy) plus measured wall-clock speed. Alternative considered: adopt gemma4:e4b directly since feasibility passed — rejected because the gut-check validated speed, not extraction quality, and Open-question §6 explicitly says do not hardcode. gemma4:e4b enters as the front-runner candidate, nothing more.

Score against the Opus tier as the rubric. Haiku/Sonnet references exist but Opus is the gold standard (per reference-extraction-benchmark). Candidates are scored primarily against Opus; the other tiers provide a quality gradient for context.

Ollama config travels with this step. OLLAMA_FLASH_ATTENTION=1 (KV-cache VRAM savings) and GRAPHIFY_OLLAMA_NUM_CTX=8192 (sufficient for 200–2000-word notes with prompt headroom) are set in the shell profile now and re-baked into the plugin env block at Step 6. GRAPHIFY_OLLAMA_KEEP_ALIVE is deferred to packaging. Verify with ollama ps after the first call.

Build the full vault, then review god-nodes. Rather than a synthetic subset, build over the real ~/Documents/SecondBrain and use GRAPH_REPORT.md's most-connected nodes as the sanity signal — the highest-traffic tools/clients/domains should surface as god-nodes. Cheap, and it exercises the real extraction path.

Risks / Trade-offs

• No candidate scores acceptably against the rubric → If even the best candidate is well below the Opus reference, surface it as a finding rather than forcing a selection; the front-runner's speed does not rescue poor extraction quality. Selection may need a larger candidate or a revisit of token budget / context.
• Scoring is partly qualitative → Entity/relationship/confidence-tag comparison against references is judgement-based, not a single numeric pass/fail. Mitigation: record the per-fixture comparison and rationale in the result artifact so the choice is auditable, not asserted.
• Vault content drift during build → Building over the live vault means notes could change mid-run. Mitigation: the graph is disposable and rebuildable (--force); a one-shot initial build is acceptable.
• GPU/VRAM regressions → The ~74 tok/s figure was post-reboot; throughput can vary. Mitigation: --max-concurrency 2 and a bounded --token-budget keep memory pressure predictable; record actual speed per candidate.

Migration Plan

Sequential, low blast radius (local-only, no production system touched):

1. pip install graphifyy; verify graphify --version.
2. Export Ollama env vars; pull candidate model(s); verify context via ollama ps.
3. Run each candidate over the 6 fixtures; score vs. the Opus references; record results.
4. Select the model; build the initial vault graph; review GRAPH_REPORT.md.

Rollback: artifacts are disposable — delete graphify-out/ and re-run; uninstall graphifyy if needed. No data is mutated in the vault (extraction is read-only over notes).

Open Questions

• Exact candidate set beyond gemma4:e4b (e.g. whether to also score a smaller/larger sibling) — decided at run time based on what is pulled and how the front-runner scores.
• Where the scoring-result artifact lives (under docs/memory-system/benchmark/ alongside the references is the natural home) — settle when writing it.
• --token-budget / --max-concurrency tuning — start from the doc's 512 / 2 and adjust if quality or VRAM demands it.