cc-os/docs/memory-system/04-build-plan.md

# Build Plan

How a human builds this system, step by step, and answers to the operational questions:
which scripts and hooks, how the AI knows when to write and what conventions to follow, how and
when it queries, the CRUD hooks, and how it's packaged as a global Claude Code plugin with
skills.

> This is a build outline, not the implementation plan. The next session should turn this into
> a proper implementation plan (writing-plans skill) and execute it.

---

## Part A — Build order (human builder's path)

Build bottom-up: the vault and CLI first (usable standalone), then the hooks, then the plugin
that packages it.

### Step 1 — Vault skeleton & conventions
- Decide the vault location (default: a synced home dir, e.g. `~/brain`; symlink into
  `~/.claude/memory` only if a tool insists). **Vault is the single source of truth.**
- Write `CONVENTIONS.md` in the vault: the frontmatter contract and the tag namespaces
  (`tool/`, `client/`, `domain/`, `convention/`, `scope/`). This doubles as the spec the
  skills teach the AI.
- Seed a few real notes (e.g. `tool/semrush` global + a `client/<x>` project note) to test
  against.

### Step 2 — The Ruby tag-index CLI (the core build)
- Ruby project, `sequel` + `sqlite3` gems. SQLite file in a cache dir (e.g.
  `~/.cache/memory-index/index.sqlite`) — disposable, never synced.
- **Schema / migrations**: `files(id, path, mtime, summary, scope)`, `tags(id, name)`,
  `files_tags(file_id, tag_id)`; `many_to_many` between files and tags.
- **Frontmatter parser**: read YAML frontmatter from a `.md` file → `{summary, scope, tags[]}`.
  Fail loudly (or quarantine) notes missing the required `summary`/tags so the contract is
  enforced.
- **Commands** (a thin CLI dispatch — `thor`/`optparse` or plain):
  - `index update --since <iso8601|auto>` — find `.md` with `mtime >` last-cache-time, upsert
    them; **reconcile**: delete rows for paths no longer on disk.
  - `index update --file <path>` — upsert a single file (used by the write hook); prune if the
    path is gone.
  - `index update --rebuild` (default off) — drop & rebuild from a full vault scan.
  - `index query --client X --tool Y [--domain Z] [--scope global]` — AND the filters; output
    JSON lines of `{path, summary, tags[]}` (option C).
  - `index tags [--namespace tool/]` — enumerate the vocabulary (virtual index).
- Store `last_cache_time` (a tiny meta table or a stamp file) so `--since auto` works.
- **Tests**: create/edit/delete/rename a note, assert the index reflects it; rebuild equals
  incremental.

### Step 3 — Hooks (maintenance + retrieval)
See Part C for exact mapping. Implement as small shell wrappers that call the Ruby CLI:
- `pre-tool-memory` style **PostToolUse** updater (write path → `index update --file`).
- **SessionStart** reconcile + inject (index overview + resolved `convention/*` files + journal
  pointer).
- **SessionEnd** journal appender (daily note with pointers).

### Step 4 — Episodic layer (memsearch)
- Install memsearch (`/plugin marketplace add zilliztech/memsearch`, then `plugin install
  memsearch`), local to start. Verify daily memory files appear after a few conversations.
- Decide whether memsearch indexes our session-end journal notes or its own capture (likely its
  own; our journal can point into the knowledge vault).

### Step 5 — Sync
- Pick **git** (versioned, hourly push/pull) or **Syncthing** (continuous, zero-thought) for the
  vault → VPS. Configure on each machine. **Do not sync the SQLite/Milvus caches.**

### Step 6 — Package as a global plugin (Part D)
- Wrap Steps 2–3 into a Claude Code plugin with skills; install at user level.

### Step 7 (deferred) — QMD semantic layer
- Only when structured-only retrieval misses notes you know exist: `qmd` over the vault,
  optionally as an MCP server the AI can query.

---

## Part B — The AI's write & query conventions (skills teach these)

### When the AI WRITES to the vault
Trigger writes when the AI learns something **evergreen and reusable across projects** — not
project-ephemeral state (that's the episodic layer / project files). Concretely:
- It worked out how a tool/API behaves (e.g. SEMrush auth, rate limits, an endpoint quirk).
- It established a convention, decision, or preference that should apply beyond this task.
- It discovered a client-specific fact worth reusing (how *this* client uses a tool).

**What conventions to follow when writing** (enforced by the frontmatter contract):
- One concept per note; keep notes small (the L1 "under ~200 lines / one topic" discipline).
- **Required frontmatter**: a one-line `summary` (written now, not deferred), `scope/global`
  or `scope/project`, and namespaced tags (`tool/…`, `client/…`, `domain/…`, optionally
  `convention/…`).
- **Scope rule**: default new tool/domain knowledge to `scope/global`; mark `scope/project` +
  a `client/<name>` tag when it's specific to how a client uses something.
- **Write only to the vault**, never silently into a project repo.
- **Promotion to global** (project → global) and **consolidation** are *not* done inline —
  they happen in the reorganize step (Part C), in plan mode, for human review.

### When & how the AI QUERIES
- **At session start**: the injected index overview + resolved `convention/*` files tell it
  what's available without reading everything.
- **On demand, during a task**: when the task touches a tool/client/domain, run
  `index query --tool semrush --client sesame3g` → get `{path, summary, tags}` → open only the
  files whose summary matches. This is the progressive-disclosure loop that keeps tokens low.
- **Cross-client lookups**: query by `--tool` alone (omit client) to surface what was learned
  with *any* client; tags in the output show provenance.
- **"What happened" questions** go to the **episodic layer (memsearch)** in natural language,
  not the tag index.

---

## Part C — Hooks & CRUD mapping

CRUD over the knowledge vault, and which hook/command services each operation:

| Operation | Trigger | Mechanism |
|-----------|---------|-----------|
| **Create** | AI writes a new note | AI writes `.md` (Write tool) → **PostToolUse** hook → `index update --file` |
| **Read / query** | AI needs knowledge | AI calls `index query …` (no hook; on-demand CLI) |
| **Update** | AI/user edits a note | AI edit → **PostToolUse** hook → `index update --file`; user edit → **SessionStart** reconcile |
| **Delete / rename** | note removed/renamed | covered by `--file` (prune) on AI ops; by **SessionStart** reconcile (prune vanished paths) on manual ops |
| **Inject** | new session/sub-agent starts | **SessionStart** hook injects index overview + resolved `convention/*` + journal pointer |
| **Journal** | session ends | **SessionEnd** hook appends a dated journal note with pointers |
| **Reorganize** | periodic, user-invoked | `reorganize memory` run in **plan mode**: dedupe, merge, split, re-tag, promote `scope/global`, rebuild index — human approves |

Hooks are thin shell wrappers over the Ruby CLI so the logic lives in one place.

**Hooks summary:**
- **SessionStart** — reconcile (`index update --since auto`) + inject context.
- **PostToolUse** (on `Write`/`Edit` of vault `.md`) — `index update --file`.
- **SessionEnd** — append daily journal note.
- (memsearch brings its own `UserPromptSubmit` + capture hooks for the episodic layer.)

---

## Part D — Claude Code plugin with skills (global install)

Goal: one global install so every project/machine knows the vault, conventions, hooks, and CLI.

**Plugin contents:**
- **Hooks** registered in settings: SessionStart, PostToolUse, SessionEnd (the shell wrappers
  from Part C), pointed at a configurable vault path + cache dir.
- **The Ruby CLI** (the tag index) bundled or installed as a dependency, on `PATH`.
- **Skills** (these carry the *know-how* to the model):
  - `memory-write` — when to record evergreen knowledge, the frontmatter contract, scope rules,
    "vault not repo." (Part B write section.)
  - `memory-query` — how/when to query the tag index vs the episodic layer; the progressive-
    disclosure loop; cross-client lookups. (Part B query section.)
  - `memory-reorganize` — the plan-mode consolidation/promotion procedure + guardrails.
- **Config**: vault path, cache path, sync method — set once at user level.

**Why a plugin + skills (not just CLAUDE.md):** hooks must be registered by the harness (the
plugin does that), and the conventions are taught as skills so they load on demand without
bloating every project's context. A single global install keeps the conventions themselves a
single source of truth — edit the skill once, every project follows.

**Open question for build time**: do the coding-`convention/*` notes live *in the vault* (data,
edited freely, resolved by the SessionStart hook) while the *memory-system skills* live *in the
plugin* (behavior, versioned)? Recommended: yes — conventions are data in the vault; the skills
are the mechanism.