cc-os/docs/memory-system/04-build-plan.md

# Build Plan

How a human builds this system, step by step, and answers to the operational questions:
which scripts and hooks, how the AI knows when to write and what conventions to follow, how and
when it queries, the CRUD hooks, and how it's packaged as a global Claude Code plugin with
skills.

> This is a build outline, not the implementation plan. The next session should turn this into
> a proper implementation plan (writing-plans skill) and execute it.

**Architecture decision (2026-06-03):** Graphify replaces the Ruby/SQLite tag index.
The Ruby CLI (old Step 2) is skipped. Graphify handles both knowledge-graph queries over the
vault and free AST-based code graphs per project. The deferred QMD semantic layer (old Step 7)
is also skipped — Graphify covers it without vectors. See `06-graphify-evaluation.md` for
the full rationale.

---

## Part A — Build order (human builder's path)

Build bottom-up: the vault and Graphify first (usable standalone), then the hooks, then the
plugin that packages it.

### Step 1 — Vault skeleton & conventions
- Decide the vault location (default: a synced home dir, e.g. `~/brain`; symlink into
  `~/.claude/memory` only if a tool insists). **Vault is the single source of truth.**
- Write `CONVENTIONS.md` in the vault: the frontmatter contract. The required fields are
  `summary` (one-line, author-written — this is the human-readable router hint Graphify
  does not generate) and `scope` (`scope/global` or `scope/project`). Tags are now
  supplementary, not the primary query mechanism.
- Seed a few real notes (e.g. `tool/semrush`, `client/<x>`) to use as extraction test cases
  in Step 2.

### Step 2 — Graphify + Ollama setup (the knowledge graph layer)

This replaces the Ruby tag-index CLI. Graphify builds a knowledge graph over vault docs via a
local SLM, and a free code graph over project files via AST.

#### 2a — Install Graphify
```
pip install graphify   # or follow current install docs
```
Verify: `graphify --version`

#### 2b — Configure Ollama for extraction
The 2024 approach (Modelfile with `PARAMETER num_ctx`) still works but the preferred
programmatic method is now per-request via the API (Graphify supports this via
`GRAPHIFY_OLLAMA_NUM_CTX`):

```bash
# Set in your shell profile or service env:
OLLAMA_FLASH_ATTENTION=1          # 30-50% VRAM savings on KV cache — always set this
GRAPHIFY_OLLAMA_NUM_CTX=8192      # 8K is sufficient for vault notes (200-2000 words)
                                  # and leaves comfortable headroom for prompt overhead
```

To verify context is being used: `ollama ps` shows allocated context after first call.

Available models (as of 2026-06-03, in order of interest):
- `gemma4:e2b` — 7.2 GB, ~2B effective params, 128K native window, fast; **start here**
- `qwen3.5:2b` — 2.7 GB, smallest, good fallback if VRAM is constrained
- `gemma4:e4b` — 9.6 GB, more capable, slower

#### 2c — Model benchmarking (before committing to a model)

Run a small extraction test across all local models plus the three Claude API models. The goal
is to find the best speed/accuracy tradeoff for entity+relationship extraction from vault notes.

**Test set:** 5–10 representative vault notes spanning different note types (tool note, client
note, convention note, domain note). Include one note that is dense with relationships.

**What to measure per model:**
1. Extraction speed (tokens/sec or wall-clock time per note)
2. Entity quality: are the right concepts extracted? Any hallucinated entities?
3. Relationship quality: are edges plausible and correctly typed? Missing relationships?
4. Confidence tag accuracy: are `INFERRED` vs `AMBIGUOUS` edges appropriately flagged?

**Models to test:**
| Model | Backend | Notes |
|---|---|---|
| `gemma4:e2b` | Ollama local | Primary candidate — fast, 7.2 GB |
| `qwen3.5:2b` | Ollama local | Smallest, fastest |
| `gemma4:e4b` | Ollama local | Highest local quality |
| `claude-haiku-4-5` | Claude API | Baseline API option |
| `claude-sonnet-4-6` | Claude API | Mid-tier reference |
| `claude-opus-4-8` | Claude API | **Gold standard** — judge quality against this |

Run using subagents in Claude Code: dispatch one agent per model, each extracting the same
test set, return structured JSON of entities+relationships. Review god-node quality in
`GRAPH_REPORT.md` after each run. Use Opus output as the scoring rubric for the local models.

**Decision rule:** Choose the fastest local model whose entity/relationship quality is
"close enough" to Opus (subjective; likely Gemma4:e2b or Gemma4:e4b based on the description).
API models are a fallback option for high-stakes notes, not the default.

#### 2d — Initial vault graph build
```bash
graphify extract --path ~/brain --backend ollama --model gemma4:e2b \
  --token-budget 512 --max-concurrency 2
```
Tune `--token-budget` (semantic chunk size) and `--max-concurrency` based on VRAM headroom.
Review `GRAPH_REPORT.md` — check god nodes make sense (they should be your most-connected
tools, clients, and domain concepts).

#### 2e — Per-project code graphs (free, no model needed)
For each client project:
```bash
graphify extract --path ~/projects/<client>/<project> --no-docs
```
`--no-docs` skips LLM extraction; only AST is run. Zero token cost. Add `--update` on
subsequent runs; use `--force` when you've deleted files (to clear stale nodes).

Store each project's `graphify-out/` alongside the project, or in a configurable cache dir.
Keep per-project graphs separate — do not merge client projects into one graph.

### Step 3 — Hooks (maintenance + retrieval)

Hooks now call Graphify instead of the Ruby CLI.

- **PostToolUse** (on Write/Edit of vault `.md`): `graphify update --file <path>` to merge
  the changed note into the vault graph. Note: Graphify's `--update` does not prune deleted
  nodes — a periodic `--force` rebuild (e.g. weekly, or triggered by SessionStart stale
  check) is needed to clear ghost nodes.
- **SessionStart**: reconcile (check vault graph mtime vs last rebuild; trigger `--force` if
  stale threshold exceeded) + inject context (query MCP server for god nodes + inject
  `convention/*` note summaries + journal pointer).
- **SessionEnd**: append daily journal note with pointers to vault notes touched.

**Stale-node mitigation strategy:** track last `--force` rebuild time in a stamp file
(e.g. `~/.cache/graphify/vault-rebuild.stamp`). SessionStart hook triggers `--force` if
older than N days (7 is a reasonable starting point).

### Step 4 — Episodic layer (memsearch)
- Install memsearch (`/plugin marketplace add zilliztech/memsearch`, then `plugin install
  memsearch`), local to start. Verify daily memory files appear after a few conversations.
- Decide whether memsearch indexes our session-end journal notes or its own capture (likely its
  own; our journal can point into the knowledge vault).
- Graphify does **not** replace memsearch here. Time-anchored semantic queries ("what was I
  working on last Tuesday?") are better served by vector similarity over session logs.

### Step 5 — Sync
- Pick **git** (versioned, hourly push/pull) or **Syncthing** (continuous, zero-thought) for
  the vault → VPS. Configure on each machine.
- **Do not sync** the Graphify `graphify-out/` directories, Milvus caches, or the Ollama
  models. Graphs are rebuilt per machine from the vault (the single source of truth).

### Step 6 — Package as a global plugin (Part D)
- Wrap Steps 2–3 into a Claude Code plugin with skills; install at user level.

### Step 7 — (SKIPPED) QMD semantic layer
Covered by Graphify. The knowledge-graph approach provides structured semantic retrieval
without vectors. Only revisit if Graphify's graph queries prove insufficient for a use case
where embedding similarity would outperform graph traversal.

---

## Part B — The AI's write & query conventions (skills teach these)

### When the AI WRITES to the vault
Trigger writes when the AI learns something **evergreen and reusable across projects** — not
project-ephemeral state (that's the episodic layer / project files). Concretely:
- It worked out how a tool/API behaves (e.g. SEMrush auth, rate limits, an endpoint quirk).
- It established a convention, decision, or preference that should apply beyond this task.
- It discovered a client-specific fact worth reusing (how *this* client uses a tool).

**What conventions to follow when writing** (enforced by the frontmatter contract):
- One concept per note; keep notes small (the L1 "under ~200 lines / one topic" discipline).
- **Required frontmatter**: a one-line `summary` (written now, not deferred — this is the
  human-authored router hint; Graphify extracts entities but does not write summaries),
  and `scope/global` or `scope/project`.
- **Scope rule**: default new tool/domain knowledge to `scope/global`; mark `scope/project`
  when it's specific to how a client uses something.
- **Write only to the vault**, never silently into a project repo.
- **Promotion to global** and **consolidation** happen in the reorganize step (Part C), in
  plan mode, for human review.

### When & how the AI QUERIES

**Knowledge queries (vault graph)** via Graphify CLI (Bash tool):
- `graphify query "how do we authenticate with SEMrush?"` — semantic graph search
- `graphify path "client/sesame3g" "tool/semrush"` — how these two concepts connect
- `graphify explain "convention/seo-workflow"` — node deep-dive with neighbors
- Add `--budget N` to cap answer size (default ~2000 tokens); `--dfs --budget N` for bounded traversal
- Add `--graph <path>` to query a specific project's `graph.json` instead of the vault graph

**At session start**: god-node summary (injected by hook from `GRAPH_REPORT.md`) gives the
architecture spine — what are the highest-connectivity concepts in the vault. The AI then
queries specific nodes on demand rather than reading everything.

**On demand, during a task**: run `graphify query` when the task touches a tool/client/domain.
The graph returns relevant node clusters; the AI opens only the vault notes whose `summary`
matches the task. The `summary` frontmatter field remains the human-written progressive-disclosure hint.

**Cross-client lookups**: `graphify query "tool semrush"` returns all nodes connected to the
SEMrush concept, regardless of which client project they came from. Graph edges show provenance.

**"What happened" questions** go to **memsearch** in natural language — not the graph.

---

## Part C — Hooks & CRUD mapping

| Operation | Trigger | Mechanism |
|-----------|---------|-----------|
| **Create** | AI writes a new vault note | AI writes `.md` (Write tool) → **PostToolUse** hook → `graphify update --file` |
| **Read / query** | AI needs knowledge | AI runs `graphify query` / `graphify explain` via Bash (no hook; on-demand) |
| **Update** | AI/user edits a note | AI edit → **PostToolUse** → `graphify update --file`; user edit → **SessionStart** reconcile |
| **Delete / rename** | note removed/renamed | `graphify update --file` prunes AI-deleted paths; **SessionStart** triggers `--force` rebuild if stale (catches manual deletes) |
| **Inject** | new session starts | **SessionStart** hook: god-node summary + `convention/*` summaries + journal pointer |
| **Journal** | session ends | **SessionEnd** hook appends a dated journal note with pointers |
| **Reorganize** | periodic, user-invoked | `reorganize memory` in **plan mode**: dedupe, merge, split, re-scope, then `graphify --force` to rebuild clean graph — human approves |
| **Stale graph rebuild** | SessionStart stale check | If rebuild stamp > N days old: `graphify --force` on vault; resets stamp |

Hooks are thin shell wrappers; the logic lives in Graphify.

**Hooks summary:**
- **SessionStart** — stale check + `--force` if needed, then inject (god nodes + conventions + journal pointer).
- **PostToolUse** (on `Write`/`Edit` of vault `.md`) — `graphify update --file`.
- **SessionEnd** — append daily journal note.
- (memsearch brings its own hooks for the episodic layer.)

---

## Part D — Claude Code plugin with skills (global install)

Goal: one global install so every project/machine knows the vault, conventions, hooks, and
Graphify config.

**Plugin contents:**
- **Hooks** registered in settings: SessionStart, PostToolUse, SessionEnd (shell wrappers
  from Part C), pointed at a configurable vault path + Graphify output dir.
- **Graphify CLI** on `PATH` — the AI uses the Bash tool to run `graphify query`,
  `graphify path`, `graphify explain` directly. No server process. Project-specific graphs
  are queried with `--graph <path>` pointing at the project's `graphify-out/graph.json`.
- **Skills** (these carry the *know-how* to the model):
  - `memory-write` — when to record evergreen knowledge, the frontmatter contract (summary
    required, scope rule), "vault not repo."
  - `memory-query` — how/when to use `graphify query` vs memsearch; god-node discipline;
    `--budget`/`--dfs` usage; cross-client lookups via `--graph`; progressive-disclosure
    via summary field.
  - `memory-reorganize` — the plan-mode consolidation/promotion procedure + guardrails;
    when to trigger `graphify --force` rebuild.
- **Config**: vault path, Graphify output dir, Ollama model name, `num_ctx`, rebuild-stale
  threshold in days — set once at user level.
- **Env vars** set by plugin: `OLLAMA_FLASH_ATTENTION=1`, `GRAPHIFY_OLLAMA_NUM_CTX=8192`,
  `GRAPHIFY_OLLAMA_KEEP_ALIVE=5`.

**Why a plugin + skills (not just CLAUDE.md):** hooks must be registered by the harness;
conventions taught as skills load on demand without bloating every project's context. A single
global install keeps conventions a single source of truth.

**Convention notes placement:** coding `convention/*` notes live as **data in the vault**
(edited freely, resolved by SessionStart hook); memory-system **skills live in the plugin**
(behavior, versioned).

---

## Open questions

1. **Vault location** — `~/brain` (synced home dir)? Symlink into `~/.claude/memory`?
2. **Sync mechanism** — git (versioned, hourly) vs Syncthing (continuous)?
3. **Stale rebuild threshold** — how many days before SessionStart triggers `--force`?
   7 days is the starting guess; tune after observing drift in practice.
4. **Per-project graph path** — how does the AI know which `graph.json` to pass to
   `--graph` for the current project? Convention: `<project-root>/graphify-out/graph.json`,
   injected by SessionStart hook if a project graph exists at that path.
5. **Summary field ownership** — Graphify extracts entities but does NOT write summaries.
   The human (or AI, on vault note creation) must write the `summary` frontmatter. Confirm
   this discipline holds in practice.
6. **Model benchmark results** — defer all model decisions until Step 2c benchmark is done.
   Don't hardcode `gemma4:e2b` until testing confirms it's the right pick.
7. **memsearch + journal** — does memsearch index our SessionEnd journal notes, or only
   its own auto-capture, and how does the journal point into the knowledge vault?