# Implementation Process _Last updated: 2026-06-05_ | _Status: Steps 2a/2b/2d executed — toolchain installed, context fix applied, vault graph built; model selected (qwen2.5-coder:7b); Step 2c (reference set) previously complete. Step 2 fully executed._ This document distills `04-build-plan.md` into a concrete, staged build process and folds in two locked decisions: **ADR-011** (faceted six-namespace taxonomy) and **ADR-012** (reuse the existing `~/Documents/SecondBrain` vault rather than creating a new one). Read `04-build-plan.md` for the underlying rationale, query/CRUD conventions, and plugin internals; read `03-architecture-decisions.md` for the decision log. This document is the runbook. > This is still a build process outline, not a detailed implementation plan. **Step 2 is fully > executed as of 2026-06-05**: toolchain installed (2a), context-baking approach confirmed (2b), > reference set generated (2c), vault graph built (2d). Selected model: **qwen2.5-coder:7b** > (Modelfile variant `qwen25-coder-7b-16k`, num_ctx 16384). The next phase is Step 3 (Hooks). > Most open questions in Steps 3–6 can be defaulted at build time. --- ## Build order Bottom-up: fixture selection first (feeds the Step 2c gate), then Graphify setup, then hooks, then plugin. Bulk vault migration is deferred to after the system is validated end-to-end. --- ## Step 1 — Select benchmark fixtures (and defer bulk migration) **Per ADR-012:** `~/Documents/SecondBrain` is the vault. The vault is already flat, already scoped to durable knowledge, and already articulates the correct governance philosophy. **Bulk vault migration is DEFERRED.** Do not migrate all notes now. The only pre-build activity in this step is selecting a small fixture set that feeds the Step 2c benchmark gate. Everything else — tagging ~20 existing notes, updating governance docs, initializing git, fixing broken references — happens after the system is validated end-to-end. > **Four sequencing concepts to keep distinct — they are not interchangeable:** > > 1. **Benchmark fixtures** — 5–10 notes selected NOW, in this step. They feed the Step 2c > extraction benchmark gate. They are living fixtures, kept and reused throughout development. > 2. **Bulk vault migration** — converting all ~20 existing notes to the six-facet tag schema, > updating governance docs, initializing git. DEFERRED to last, after the system is fully > validated. > 3. **Initial validation project** — the first real end-to-end test after the system is built: > one small project that contains BOTH code AND documents. POST-BUILD, before any rollout. > 4. **Project-by-project rollout** — onboarding remaining projects one at a time, observing > and adjusting between each. Follows the initial validation project. ### 1a — Select fixture notes **Status (2026-06-04): DONE.** 6 cross-domain fixtures selected and listed in `benchmark/dispatch-prompt.md`. Run as-is (no vault frontmatter modification per the methodology decision — fixtures used exactly as they exist in the vault). Choose 5–10 notes from `~/Documents/SecondBrain` with deliberate variety. The selection must include at minimum: - A **tool note** (e.g., a note about a CLI tool or SaaS product) - A **client/project note** (scoped, not global) - A **convention note** (a `type/convention` or workflow note) - A **domain note** (a topic or subject-area note) - At least one **relationship-dense note** — a note where several concepts interrelate and Graphify should emit multiple typed edges and confidence tags These notes must be usable as-is by a subagent that receives only the note text plus the shared extraction spec (see Step 2c). Apply `summary:` frontmatter and the six-facet tag schema to each fixture note now; do not wait for bulk migration. **Frontmatter contract** (required for fixture notes immediately; apply to all notes at bulk migration time): ```yaml --- summary: "One-line, human-written router hint (required; Graphify does not generate this)" tags: - type/ # listed first by convention - client/ # as applicable - project/ # as applicable — first-class for a freelancer - domain/ # as applicable - tool/ # as applicable - convention/ # as applicable - scope/global # or scope/project --- ``` `type/` listed first preserves the vault's existing type-first ordering habit and makes note kind immediately visible. ### 1b — Deferred bulk migration work (do not start until post-validation) The following items are deferred to after the system is validated end-to-end on fixtures and on the initial validation project (see §Post-build sequence below): - **Initialize git:** `git init ~/Documents/SecondBrain` (enables ADR-008 sync) — before the first commit, add `graphify-out/` to the vault's `.gitignore`: `~/Documents/SecondBrain/graphify-out/cache/` (AST/tree-sitter parse cache) currently sits inside the vault and must stay untracked (ADR-008 — indexes are disposable/excluded from version control) - **Update vault governance notes:** `CLAUDE.md` (vault path refs, Graphify commands, ADR-011 namespace list), `vault-conventions.md` (six-facet tag list; preserve the "act without being asked" proactive-retrieval section), and project-config hub note(s) (update tag-inference table to namespaced form, e.g. `semrush-work → tool/semrush`) - **Migrate remaining notes (~20):** add `summary:` frontmatter and convert unnamespaced tags to six-facet form (e.g. `research → type/research`, `pest-control → domain/pest-control`, `semrush → tool/semrush`). Add `scope/global` or `scope/project` to each note. - **Create `_templates` for core note types:** `research`, `howto`, `adr`, `hub`. The long tail stays freeform until a pattern earns a template (per ADR-011). - **Fix the broken vault search reference:** `vault-conventions.md` (or `CLAUDE.md`) currently references `~/.claude/scripts/vault_search.rb`, which does not exist. Replace every reference with `graphify query` (per ADR-010). ### Post-build sequence (after Steps 2–6 are complete) After the system is built and validated on fixtures: **First:** Validate end-to-end on **one small project that contains BOTH code AND documents.** This project is the initial validation target — not a production migration. It exercises both extraction paths simultaneously: vault notes use the local-SLM extraction path; project code uses the tree-sitter AST path. These are different extraction paths, which is precisely why the first validation project must contain both. Observe how the two paths compose before proceeding. **Then:** Onboard remaining projects **one at a time.** Observe extraction quality, god-node shape, and hook behavior between each project before onboarding the next. Adjust configuration as needed. **Finally:** Execute bulk vault migration (items in 1b above) and run full vault graph build. **Migration-unit granularity open question:** the right unit — a whole project repo, a vault-note cluster, a client boundary — is not yet established. The first validation project ("one small project with both code and documents") is the anchor; refine the granularity definition after observing that first real migration. --- ## Step 2 — Graphify + Ollama setup (CRITICAL PATH) This step is the genuine gate. All hook behavior, model selection, and extraction tuning depend on the benchmark in 2c. Do not skip or defer 2c. ### 2a — Install and verify Graphify **Status (2026-06-05): EXECUTED.** graphify 0.8.31 installed at `~/.local/bin/graphify`; `graphify --version` → `graphify 0.8.31`. ollama 0.30.3, systemd service on 127.0.0.1:11434; GPU RTX 3060 12GB VRAM confirmed; all extraction ran 100% on GPU (no CPU spill per `ollama ps`). [verified: local shell, 2026-06-05] The PyPI package name has a double-y — this is the correct install command: ```bash pip install graphifyy ``` Verify: `graphify --version` ### 2b — Configure Ollama **Status (2026-06-05): EXECUTED — but the env-var approach was superseded.** `GRAPHIFY_OLLAMA_NUM_CTX` does NOT propagate through graphify's ollama `/v1/chat/completions` endpoint: ollama's OpenAI-compatible `/v1` endpoint silently ignores per-request `options.num_ctx`, pinning context at the model's baked default (4096 by default). At 4096, graphify's extraction output is truncated mid-response and the chunk is discarded — producing an empty graph. [verified: local A/B test, 2026-06-05] **SUPERSEDED APPROACH — do not use `GRAPHIFY_OLLAMA_NUM_CTX`:** ```bash # This env var has NO EFFECT through graphify's /v1 path — do not set # GRAPHIFY_OLLAMA_NUM_CTX=8192 ``` **CORRECT APPROACH — bake context into a Modelfile variant:** ```bash # Create a Modelfile variant with the desired context baked in: # FROM # PARAMETER num_ctx 16384 # ollama create -16k -f Modelfile ``` The `/v1` endpoint DOES honor the model's baked default. Verified `ollama ps` CONTEXT=16384, 100% GPU. This is the same lever the vault build (Step 2d) and plugin (Step 6) must use. Non-invasive: no sudo, no systemd restart, no shell-profile edit; reversible via `ollama rm`. Still set in shell profile (these remain valid): ```bash OLLAMA_FLASH_ATTENTION=1 # 30–50% VRAM savings on KV cache — always set GRAPHIFY_OLLAMA_KEEP_ALIVE=5 # set when packaging in Step 6 ``` Two additional operational requirements confirmed: `OLLAMA_BASE_URL` must end in `/v1` (e.g., `http://127.0.0.1:11434/v1`) or every graphify call 404s; `OLLAMA_API_KEY` must be set to any non-empty value unless the host is loopback. Verify context allocation after the first extraction call: `ollama ps` shows allocated context. ### 2c — Claude reference-set benchmark (THE GATE) **Status (2026-06-04): EXECUTED (prior round).** 6 cross-domain fixtures × 3 Claude tiers = 18 reference fragments generated in `benchmark/reference-outputs/`. Run as-is (no vault frontmatter modification); verified clean. Gate is passed — Ollama model scoring is unblocked. Full local-model scoring (task 3.4) and model selection (task 3.6) are also complete — see `benchmark/scoring-results-2026-06-04.md`. Produce a reference set of Graphify-shaped extraction outputs before committing to any local Ollama model. Use the 5–10 fixture notes selected in Step 1a as the input set. **What this step produces — and what it does NOT decide:** This step produces the **gold-standard reference set**: one structured extraction output per fixture note per Claude tier. The reference set is the scoring rubric against which local Ollama models are scored in a later step. This step does NOT choose the final extraction model — local-model selection happens when Ollama models are timed and scored against these references. **Why Claude-only here:** local Ollama models cannot be run in this benchmarking environment. Only Claude tiers are reachable via dispatched subagents. The Claude outputs serve as quality anchors; Ollama speed and quality are measured separately, against these anchors. **Dispatch one Claude Code subagent per tier:** | Tier | Model ID | Role | |---|---|---| | Haiku | `claude-haiku-4-5` | Lightweight reference | | Sonnet | `claude-sonnet-4-6` | Mid-tier reference | | Opus | `claude-opus-4-8` | **Gold standard** | Each subagent receives only the fixture note text plus the shared extraction spec at `docs/memory-system/benchmark/extraction-spec.md`. It must not read `CLAUDE.md`, design docs, or pull any project context — each subagent operates with the note and spec alone (fairness contract). Each subagent emits a Graphify-shaped structured fragment containing: - Extracted entities (named concepts, tools, people, projects) - Typed relationships between entities - Confidence tags: `INFERRED` (inferred but not stated) and `AMBIGUOUS` (could be interpreted multiple ways) applied where appropriate **Metrics — quality only (wall-clock speed is explicitly out of scope here):** Wall-clock timing is untrackable across dispatched subagents and is not measured in this step. Speed re-enters the picture in the later Ollama-scoring step, where local models are timed against these reference outputs. For this step, measure quality only: 1. **Entity correctness** — right concepts extracted, no hallucinated entities 2. **Relationship plausibility and typing** — edges plausible, correctly typed, no missing key edges 3. **Confidence-tag accuracy** — `INFERRED` vs `AMBIGUOUS` applied appropriately **Deliverable files:** - Dispatch prompt (copy/paste-able): `docs/memory-system/benchmark/dispatch-prompt.md` - Shared extraction schema: `docs/memory-system/benchmark/extraction-spec.md` - Per-model outputs: `docs/memory-system/benchmark/reference-outputs/..md` where `tier` is one of `haiku`, `sonnet`, or `opus` `claude-opus-4-8` output is the gold standard. When Ollama models are benchmarked later, their outputs are scored by how closely they match the Opus reference for each fixture note. ### 2d — Build the initial vault graph **Status (2026-06-05): EXECUTED.** Full vault graph built with qwen2.5-coder:7b (Modelfile variant `qwen25-coder-7b-16k`, num_ctx 16384). Result: **57 nodes / 43 edges / 15 communities**. God-nodes are plausible: Speed-to-Lead is the dominant hub, consistent with the vault's current content emphasis. [verified: local shell, 2026-06-05] Command used (deviates from the template below — actual values recorded for reference): ```bash graphify extract ~/Documents/SecondBrain \ --backend ollama --model qwen25-coder-7b-16k \ --max-concurrency 1 --token-budget 4000 \ --exclude .obsidian --out /tmp/graphify-bench/vault-graph ``` Note: `--max-concurrency 1` (not 2 — untested at 2 on 12GB GPU); `--token-budget 4000` (not 512 — fits the 16k context comfortably). Template for future runs: ```bash graphify extract --path ~/Documents/SecondBrain \ --backend ollama --model qwen25-coder-7b-16k \ --token-budget 4000 --max-concurrency 1 --exclude .obsidian ``` Review `GRAPH_REPORT.md`. God nodes should be the most-connected tools, clients, and domain concepts — confirm they make sense given the vault's content. ### 2e — Per-project code graphs (free; no model) For each active client project: ```bash graphify extract --path ~/projects// --no-docs ``` `--no-docs` runs only tree-sitter AST — zero token cost. Use `--update` on subsequent runs; use `--force` when files have been deleted (to clear stale nodes). Keep each project's `graphify-out/` alongside the project; do not merge client projects into one graph. --- ## Step 3 — Hooks Thin shell wrappers around Graphify. The logic lives in Graphify; the hooks only invoke it. | Hook | Trigger | Action | |---|---|---| | **PostToolUse** | AI writes/edits a vault `.md` | `graphify update --file ` | | **SessionStart** | Session opens | Stale check → `--force` rebuild if needed → inject context | | **SessionEnd** | Session closes | Append dated journal note to vault | **Stale-check mechanism (SessionStart):** read the mtime of `~/.cache/graphify/vault-rebuild.stamp`. If older than N days (7 is the starting guess — tune after observing drift), run `graphify --force` on the vault and write a new stamp. Then inject: god-node summary from `GRAPH_REPORT.md` + `convention/*` note summaries + journal pointer. **Note on `--update` vs `--force`:** `graphify update --file` does not prune deleted nodes. Stale/ghost nodes accumulate from manual deletes and renames. The periodic `--force` rebuild triggered by the stale check is the mitigation — it rebuilds the graph clean. --- ## Step 4 — Episodic layer (memsearch) Install memsearch for time-anchored "what happened / what was I working on" queries. Graphify does **not** replace this — they serve different query patterns (knowledge graph vs timeline recall). ``` /plugin marketplace add zilliztech/memsearch plugin install memsearch ``` Verify daily memory files appear after a few conversations. The open question of whether memsearch indexes SessionEnd journal notes or only its own auto-capture is deferred to build time (see Open questions §7). --- ## Step 5 — Vault sync Sync the markdown vault to the VPS. Pick one: - **git** — versioned, hourly push/pull via cron; explicit audit trail. - **Syncthing** — continuous, bidirectional, zero-thought after setup. **What to sync:** the vault only (`~/Documents/SecondBrain`). **What NOT to sync:** `graphify-out/` directories, Milvus Lite caches, Ollama models. These are disposable — rebuilt per machine from the vault (markdown is the single source of truth, per ADR-008). --- ## Step 6 — Package as a global Claude Code plugin One global install so every project and machine shares the same vault conventions, hooks, and Graphify config. **Plugin contents:** - **Hooks** registered in settings: SessionStart, PostToolUse, SessionEnd — shell wrappers from Step 3, parameterized by vault path and Graphify output dir. - **Graphify CLI on PATH** — the AI calls `graphify query`, `graphify path`, `graphify explain` via the Bash tool. No server process per graph; project-specific graphs are queried with `--graph /graphify-out/graph.json`. - **Skills** (carry the know-how to the model): - `memory-write` — when to record evergreen knowledge, frontmatter contract, scope rule, vault not repo. - `memory-query` — `graphify query` vs memsearch; god-node discipline; `--budget`/`--dfs`; cross-client lookups; progressive disclosure via `summary`. - `memory-reorganize` — plan-mode consolidation/promotion procedure; when to trigger `--force` rebuild; human-approval guardrails. - **Env vars** baked in: `OLLAMA_FLASH_ATTENTION=1`, `GRAPHIFY_OLLAMA_KEEP_ALIVE=5`, `OLLAMA_BASE_URL=http://127.0.0.1:11434/v1`, `OLLAMA_API_KEY=`. Note: `GRAPHIFY_OLLAMA_NUM_CTX` is NOT effective through graphify's `/v1` path — context must be baked into the Modelfile variant instead (see Step 2b). - **Config** (set once at user level): vault path, Graphify output dir, Ollama model name, stale-rebuild threshold in days. --- ## Open questions / decisions still to settle These are deferred to build time. Most can be defaulted without blocking. **§6 (model selection) is RESOLVED** — qwen2.5-coder:7b selected, vault graph built. Remaining questions (§1–5, §7–8) do not block Step 3 or later. 1. **Vault symlink** — `~/Documents/SecondBrain` is confirmed as the vault (RESOLVED per ADR-012). The open sub-question: symlink it into `~/.claude/memory` or not? Only needed if a tool requires that path. 2. **Sync mechanism** — git (versioned, hourly) vs Syncthing (continuous). Both are valid; choose at build time based on preference. 3. **Stale rebuild threshold** — 7 days is the starting guess. Tune after observing how quickly ghost-node drift becomes noticeable in practice. 4. **Per-project graph path convention** — how does SessionStart know which `graph.json` to inject? Proposed convention: `/graphify-out/graph.json`, injected only if the file exists at that path. Not yet made explicit in the plugin spec. 5. **`summary` field discipline** — Graphify extracts entities and edges but does not write summaries. The human (or AI at note-creation time) must write `summary:` frontmatter. Confirm this holds in practice and add a lint/reminder to the memory-write skill if it drifts. 6. **Step 2c reference set + model selection** — **RESOLVED (2026-06-04/05):** The reference set was generated: 18 fragments (6 fixtures × 3 tiers) in `benchmark/reference-outputs/`. Local-model scoring complete (see `benchmark/scoring-results-2026-06-04.md`). **Selected model: qwen2.5-coder:7b** (run as Modelfile variant `qwen25-coder-7b-16k`, num_ctx 16384) — graphify's shipped default and the only tested candidate that cleared the relationship/edge gate. All smaller candidates tested (gemma4:e4b 8.0B, gemma4:e2b 5.1B, qwen3.5:2b 2.3B) failed to produce relationship-bearing graphs through graphify's extraction prompt on this hardware. Model selection is complete. 7. **memsearch + journal integration** — does memsearch index SessionEnd journal notes or only its own auto-capture? How does the journal pointer injected at SessionStart reference the vault? Nail down at memsearch install time (Step 4). 8. **MCP server management** — vault graph + N project graphs = N+1 potential instances. The build plan leans toward a single Graphify CLI on PATH with `--graph ` rather than running multiple `graphify serve` instances, but this is not yet made explicit in the plugin spec. Confirm the CLI-only approach or define the server lifecycle. **Resolved:** The type-tag taxonomy is settled via ADR-011 (faceted, six independent namespaces; `type/` listed first). No further debate needed on taxonomy shape. --- ## Recommended next move **Step 2 is complete.** The next phase is **Step 3 (Hooks)** — thin shell wrappers around Graphify for PostToolUse, SessionStart, and SessionEnd — followed by Step 4 (memsearch episodic layer) and Step 6 (package as global plugin). Open questions §1–5 and §7–8 can be defaulted or deferred at build time. §6 (model selection) is resolved.