cc-os/docs/memory-system/05-implementation-process.md

# Implementation Process

_Last updated: 2026-06-04_ | _Status: Ready to plan — awaiting Step 2c reference set_

This document distills `04-build-plan.md` into a concrete, staged build process and folds in
two locked decisions: **ADR-011** (faceted six-namespace taxonomy) and **ADR-012** (reuse the
existing `~/Documents/SecondBrain` vault rather than creating a new one). Read `04-build-plan.md`
for the underlying rationale, query/CRUD conventions, and plugin internals; read
`03-architecture-decisions.md` for the decision log. This document is the runbook.

> This is still a build process outline, not a detailed implementation plan. The recommended
> next move is to feed Step 2 (the critical path) into the **writing-plans skill** to produce an
> executable, task-level plan. Most open questions in Steps 3–6 can be defaulted; Step 2c's
> Claude reference-set run is the genuine gate — it produces the gold-standard rubric against
> which local Ollama models are then scored. Do not hardcode a model before that scoring runs.

---

## Build order

Bottom-up: fixture selection first (feeds the Step 2c gate), then Graphify setup, then hooks, then plugin. Bulk vault migration is deferred to after the system is validated end-to-end.

---

## Step 1 — Select benchmark fixtures (and defer bulk migration)

**Per ADR-012:** `~/Documents/SecondBrain` is the vault. The vault is already flat, already
scoped to durable knowledge, and already articulates the correct governance philosophy.

**Bulk vault migration is DEFERRED.** Do not migrate all notes now. The only pre-build activity
in this step is selecting a small fixture set that feeds the Step 2c benchmark gate. Everything
else — tagging ~20 existing notes, updating governance docs, initializing git, fixing broken
references — happens after the system is validated end-to-end.

> **Four sequencing concepts to keep distinct — they are not interchangeable:**
>
> 1. **Benchmark fixtures** — 5–10 notes selected NOW, in this step. They feed the Step 2c
>    extraction benchmark gate. They are living fixtures, kept and reused throughout development.
> 2. **Bulk vault migration** — converting all ~20 existing notes to the six-facet tag schema,
>    updating governance docs, initializing git. DEFERRED to last, after the system is fully
>    validated.
> 3. **Initial validation project** — the first real end-to-end test after the system is built:
>    one small project that contains BOTH code AND documents. POST-BUILD, before any rollout.
> 4. **Project-by-project rollout** — onboarding remaining projects one at a time, observing
>    and adjusting between each. Follows the initial validation project.

### 1a — Select fixture notes

Choose 5–10 notes from `~/Documents/SecondBrain` with deliberate variety. The selection must
include at minimum:

- A **tool note** (e.g., a note about a CLI tool or SaaS product)
- A **client/project note** (scoped, not global)
- A **convention note** (a `type/convention` or workflow note)
- A **domain note** (a topic or subject-area note)
- At least one **relationship-dense note** — a note where several concepts interrelate and
  Graphify should emit multiple typed edges and confidence tags

These notes must be usable as-is by a subagent that receives only the note text plus the shared
extraction spec (see Step 2c). Apply `summary:` frontmatter and the six-facet tag schema to
each fixture note now; do not wait for bulk migration.

**Frontmatter contract** (required for fixture notes immediately; apply to all notes at bulk
migration time):

```yaml
---
summary: "One-line, human-written router hint (required; Graphify does not generate this)"
tags:
  - type/<kind>        # listed first by convention
  - client/<name>      # as applicable
  - project/<name>     # as applicable — first-class for a freelancer
  - domain/<topic>     # as applicable
  - tool/<name>        # as applicable
  - convention/<name>  # as applicable
  - scope/global       # or scope/project
---
```

`type/` listed first preserves the vault's existing type-first ordering habit and makes note
kind immediately visible.

### 1b — Deferred bulk migration work (do not start until post-validation)

The following items are deferred to after the system is validated end-to-end on fixtures and
on the initial validation project (see §Post-build sequence below):

- **Initialize git:** `git init ~/Documents/SecondBrain` (enables ADR-008 sync)
- **Update vault governance notes:** `CLAUDE.md` (vault path refs, Graphify commands,
  ADR-011 namespace list), `vault-conventions.md` (six-facet tag list; preserve the
  "act without being asked" proactive-retrieval section), and project-config hub note(s)
  (update tag-inference table to namespaced form, e.g. `semrush-work → tool/semrush`)
- **Migrate remaining notes (~20):** add `summary:` frontmatter and convert unnamespaced tags
  to six-facet form (e.g. `research → type/research`, `pest-control → domain/pest-control`,
  `semrush → tool/semrush`). Add `scope/global` or `scope/project` to each note.
- **Create `_templates` for core note types:** `research`, `howto`, `adr`, `hub`. The long
  tail stays freeform until a pattern earns a template (per ADR-011).
- **Fix the broken vault search reference:** `vault-conventions.md` (or `CLAUDE.md`) currently
  references `~/.claude/scripts/vault_search.rb`, which does not exist. Replace every reference
  with `graphify query` (per ADR-010).

### Post-build sequence (after Steps 2–6 are complete)

After the system is built and validated on fixtures:

**First:** Validate end-to-end on **one small project that contains BOTH code AND documents.**
This project is the initial validation target — not a production migration. It exercises both
extraction paths simultaneously: vault notes use the local-SLM extraction path; project code
uses the tree-sitter AST path. These are different extraction paths, which is precisely why the
first validation project must contain both. Observe how the two paths compose before proceeding.

**Then:** Onboard remaining projects **one at a time.** Observe extraction quality, god-node
shape, and hook behavior between each project before onboarding the next. Adjust configuration
as needed.

**Finally:** Execute bulk vault migration (items in 1b above) and run full vault graph build.

**Migration-unit granularity open question:** the right unit — a whole project repo, a
vault-note cluster, a client boundary — is not yet established. The first validation project
("one small project with both code and documents") is the anchor; refine the granularity
definition after observing that first real migration.

---

## Step 2 — Graphify + Ollama setup (CRITICAL PATH)

This step is the genuine gate. All hook behavior, model selection, and extraction tuning depend
on the benchmark in 2c. Do not skip or defer 2c.

### 2a — Install and verify Graphify

The PyPI package name has a double-y — this is the correct install command:

```bash
pip install graphifyy
```

Verify: `graphify --version`

### 2b — Configure Ollama

Set these in your shell profile (or the plugin env block in Step 6):

```bash
OLLAMA_FLASH_ATTENTION=1        # 30–50% VRAM savings on KV cache — always set
GRAPHIFY_OLLAMA_NUM_CTX=8192    # 8K is sufficient for vault notes (200–2000 words)
                                # leaves comfortable headroom for prompt overhead
GRAPHIFY_OLLAMA_KEEP_ALIVE=5    # set when packaging in Step 6
```

Verify context allocation after the first extraction call: `ollama ps` shows allocated context.

### 2c — Claude reference-set benchmark (THE GATE)

Produce a reference set of Graphify-shaped extraction outputs before committing to any local
Ollama model. Use the 5–10 fixture notes selected in Step 1a as the input set.

**What this step produces — and what it does NOT decide:**

This step produces the **gold-standard reference set**: one structured extraction output per
fixture note per Claude tier. The reference set is the scoring rubric against which local Ollama
models are scored in a later step. This step does NOT choose the final extraction model —
local-model selection happens when Ollama models are timed and scored against these references.

**Why Claude-only here:** local Ollama models cannot be run in this benchmarking environment.
Only Claude tiers are reachable via dispatched subagents. The Claude outputs serve as quality
anchors; Ollama speed and quality are measured separately, against these anchors.

**Dispatch one Claude Code subagent per tier:**

| Tier | Model ID | Role |
|---|---|---|
| Haiku | `claude-haiku-4-5` | Lightweight reference |
| Sonnet | `claude-sonnet-4-6` | Mid-tier reference |
| Opus | `claude-opus-4-8` | **Gold standard** |

Each subagent receives only the fixture note text plus the shared extraction spec at
`docs/memory-system/benchmark/extraction-spec.md`. It must not read `CLAUDE.md`, design docs,
or pull any project context — each subagent operates with the note and spec alone (fairness
contract).

Each subagent emits a Graphify-shaped structured fragment containing:
- Extracted entities (named concepts, tools, people, projects)
- Typed relationships between entities
- Confidence tags: `INFERRED` (inferred but not stated) and `AMBIGUOUS` (could be interpreted
  multiple ways) applied where appropriate

**Metrics — quality only (wall-clock speed is explicitly out of scope here):**

Wall-clock timing is untrackable across dispatched subagents and is not measured in this step.
Speed re-enters the picture in the later Ollama-scoring step, where local models are timed
against these reference outputs. For this step, measure quality only:

1. **Entity correctness** — right concepts extracted, no hallucinated entities
2. **Relationship plausibility and typing** — edges plausible, correctly typed, no missing key edges
3. **Confidence-tag accuracy** — `INFERRED` vs `AMBIGUOUS` applied appropriately

**Deliverable files:**

- Dispatch prompt (copy/paste-able): `docs/memory-system/benchmark/dispatch-prompt.md`
- Shared extraction schema: `docs/memory-system/benchmark/extraction-spec.md`
- Per-model outputs: `docs/memory-system/benchmark/reference-outputs/<note-slug>.<tier>.md`
  where `tier` is one of `haiku`, `sonnet`, or `opus`

`claude-opus-4-8` output is the gold standard. When Ollama models are benchmarked later, their
outputs are scored by how closely they match the Opus reference for each fixture note.

### 2d — Build the initial vault graph

After a local Ollama model has been chosen (scored against the Step 2c references):

```bash
graphify extract --path ~/Documents/SecondBrain \
  --backend ollama --model <chosen-model> \
  --token-budget 512 --max-concurrency 2
```

Review `GRAPH_REPORT.md`. God nodes should be the most-connected tools, clients, and domain
concepts — confirm they make sense given the vault's content.

### 2e — Per-project code graphs (free; no model)

For each active client project:

```bash
graphify extract --path ~/projects/<client>/<project> --no-docs
```

`--no-docs` runs only tree-sitter AST — zero token cost. Use `--update` on subsequent runs;
use `--force` when files have been deleted (to clear stale nodes). Keep each project's
`graphify-out/` alongside the project; do not merge client projects into one graph.

---

## Step 3 — Hooks

Thin shell wrappers around Graphify. The logic lives in Graphify; the hooks only invoke it.

| Hook | Trigger | Action |
|---|---|---|
| **PostToolUse** | AI writes/edits a vault `.md` | `graphify update --file <path>` |
| **SessionStart** | Session opens | Stale check → `--force` rebuild if needed → inject context |
| **SessionEnd** | Session closes | Append dated journal note to vault |

**Stale-check mechanism (SessionStart):** read the mtime of
`~/.cache/graphify/vault-rebuild.stamp`. If older than N days (7 is the starting guess — tune
after observing drift), run `graphify --force` on the vault and write a new stamp. Then inject:
god-node summary from `GRAPH_REPORT.md` + `convention/*` note summaries + journal pointer.

**Note on `--update` vs `--force`:** `graphify update --file` does not prune deleted nodes.
Stale/ghost nodes accumulate from manual deletes and renames. The periodic `--force` rebuild
triggered by the stale check is the mitigation — it rebuilds the graph clean.

---

## Step 4 — Episodic layer (memsearch)

Install memsearch for time-anchored "what happened / what was I working on" queries. Graphify
does **not** replace this — they serve different query patterns (knowledge graph vs timeline
recall).

```
/plugin marketplace add zilliztech/memsearch
plugin install memsearch
```

Verify daily memory files appear after a few conversations. The open question of whether
memsearch indexes SessionEnd journal notes or only its own auto-capture is deferred to build
time (see Open questions §7).

---

## Step 5 — Vault sync

Sync the markdown vault to the VPS. Pick one:

- **git** — versioned, hourly push/pull via cron; explicit audit trail.
- **Syncthing** — continuous, bidirectional, zero-thought after setup.

**What to sync:** the vault only (`~/Documents/SecondBrain`).

**What NOT to sync:** `graphify-out/` directories, Milvus Lite caches, Ollama models. These are
disposable — rebuilt per machine from the vault (markdown is the single source of truth, per
ADR-008).

---

## Step 6 — Package as a global Claude Code plugin

One global install so every project and machine shares the same vault conventions, hooks, and
Graphify config.

**Plugin contents:**

- **Hooks** registered in settings: SessionStart, PostToolUse, SessionEnd — shell wrappers from
  Step 3, parameterized by vault path and Graphify output dir.
- **Graphify CLI on PATH** — the AI calls `graphify query`, `graphify path`, `graphify explain`
  via the Bash tool. No server process per graph; project-specific graphs are queried with
  `--graph <project-root>/graphify-out/graph.json`.
- **Skills** (carry the know-how to the model):
  - `memory-write` — when to record evergreen knowledge, frontmatter contract, scope rule, vault not repo.
  - `memory-query` — `graphify query` vs memsearch; god-node discipline; `--budget`/`--dfs`; cross-client lookups; progressive disclosure via `summary`.
  - `memory-reorganize` — plan-mode consolidation/promotion procedure; when to trigger `--force` rebuild; human-approval guardrails.
- **Env vars** baked in: `OLLAMA_FLASH_ATTENTION=1`, `GRAPHIFY_OLLAMA_NUM_CTX=8192`,
  `GRAPHIFY_OLLAMA_KEEP_ALIVE=5`.
- **Config** (set once at user level): vault path, Graphify output dir, Ollama model name,
  stale-rebuild threshold in days.

---

## Open questions / decisions still to settle

These are deferred to build time. Most can be defaulted without blocking; the only genuine gate
is **§6 (Step 2c benchmark)** — local-model selection is blocked until the reference set exists
and Ollama models can be scored against it.

1. **Vault symlink** — `~/Documents/SecondBrain` is confirmed as the vault (RESOLVED per
   ADR-012). The open sub-question: symlink it into `~/.claude/memory` or not? Only needed if
   a tool requires that path.

2. **Sync mechanism** — git (versioned, hourly) vs Syncthing (continuous). Both are valid;
   choose at build time based on preference.

3. **Stale rebuild threshold** — 7 days is the starting guess. Tune after observing how quickly
   ghost-node drift becomes noticeable in practice.

4. **Per-project graph path convention** — how does SessionStart know which `graph.json` to
   inject? Proposed convention: `<project-root>/graphify-out/graph.json`, injected only if the
   file exists at that path. Not yet made explicit in the plugin spec.

5. **`summary` field discipline** — Graphify extracts entities and edges but does not write
   summaries. The human (or AI at note-creation time) must write `summary:` frontmatter. Confirm
   this holds in practice and add a lint/reminder to the memory-write skill if it drifts.

6. **Step 2c reference set (Claude gold-standard outputs)** — Step 2c produces the scoring
   rubric (Opus output as gold standard, Haiku/Sonnet as lighter-tier references), not the final
   model choice. Local Ollama model selection is deferred until those references exist and Ollama
   models can be timed and quality-scored against them. Do not hardcode `gemma4:e2b` — the Ollama
   scoring step decides. The Step 2c outcome is the reference set, not the chosen model.

7. **memsearch + journal integration** — does memsearch index SessionEnd journal notes or only
   its own auto-capture? How does the journal pointer injected at SessionStart reference the
   vault? Nail down at memsearch install time (Step 4).

8. **MCP server management** — vault graph + N project graphs = N+1 potential instances. The
   build plan leans toward a single Graphify CLI on PATH with `--graph <path>` rather than
   running multiple `graphify serve` instances, but this is not yet made explicit in the plugin
   spec. Confirm the CLI-only approach or define the server lifecycle.

**Resolved:** The type-tag taxonomy is settled via ADR-011 (faceted, six independent namespaces;
`type/` listed first). No further debate needed on taxonomy shape.

---

## Recommended next move

Turn **Step 2** into a detailed implementation plan via the **writing-plans skill**. It is the
critical path: Graphify install, Ollama configuration, the Step 2c reference-set run, and the
initial graph build are the smallest set of tasks that unblock everything else. Open questions
§1–5 and §7–8 can be defaulted or deferred; only §6 (producing the reference set and then
scoring Ollama models against it) genuinely blocks model-dependent decisions.