cc-os/docs/memory-system/04-build-plan.md

Build Plan

How a human builds this system, step by step, and answers to the operational questions: which scripts and hooks, how the AI knows when to write and what conventions to follow, how and when it queries, the CRUD hooks, and how it's packaged as a global Claude Code plugin with skills.

This is a build outline, not the implementation plan. The next session should turn this into a proper implementation plan (writing-plans skill) and execute it.

---

Part A — Build order (human builder's path)

Build bottom-up: the vault and CLI first (usable standalone), then the hooks, then the plugin that packages it.

Step 1 — Vault skeleton & conventions

• Decide the vault location (default: a synced home dir, e.g. ~/brain; symlink into ~/.claude/memory only if a tool insists). Vault is the single source of truth.
• Write CONVENTIONS.md in the vault: the frontmatter contract and the tag namespaces (tool/, client/, domain/, convention/, scope/). This doubles as the spec the skills teach the AI.
• Seed a few real notes (e.g. tool/semrush global + a client/<x> project note) to test against.

Step 2 — The Ruby tag-index CLI (the core build)

• Ruby project, sequel + sqlite3 gems. SQLite file in a cache dir (e.g. ~/.cache/memory-index/index.sqlite) — disposable, never synced.
• Schema / migrations: files(id, path, mtime, summary, scope), tags(id, name), files_tags(file_id, tag_id); many_to_many between files and tags.
• Frontmatter parser: read YAML frontmatter from a .md file → {summary, scope, tags[]}. Fail loudly (or quarantine) notes missing the required summary/tags so the contract is enforced.
• Commands (a thin CLI dispatch — thor/optparse or plain):
  • index update --since <iso8601|auto> — find .md with mtime > last-cache-time, upsert them; reconcile: delete rows for paths no longer on disk.
  • index update --file <path> — upsert a single file (used by the write hook); prune if the path is gone.
  • index update --rebuild (default off) — drop & rebuild from a full vault scan.
  • index query --client X --tool Y [--domain Z] [--scope global] — AND the filters; output JSON lines of {path, summary, tags[]} (option C).
  • index tags [--namespace tool/] — enumerate the vocabulary (virtual index).
• Store last_cache_time (a tiny meta table or a stamp file) so --since auto works.
• Tests: create/edit/delete/rename a note, assert the index reflects it; rebuild equals incremental.

Step 3 — Hooks (maintenance + retrieval)

See Part C for exact mapping. Implement as small shell wrappers that call the Ruby CLI:

• pre-tool-memory style PostToolUse updater (write path → index update --file).
• SessionStart reconcile + inject (index overview + resolved convention/* files + journal pointer).
• SessionEnd journal appender (daily note with pointers).

Step 4 — Episodic layer (memsearch)

• Install memsearch (/plugin marketplace add zilliztech/memsearch, then plugin install memsearch), local to start. Verify daily memory files appear after a few conversations.
• Decide whether memsearch indexes our session-end journal notes or its own capture (likely its own; our journal can point into the knowledge vault).

Step 5 — Sync

• Pick git (versioned, hourly push/pull) or Syncthing (continuous, zero-thought) for the vault → VPS. Configure on each machine. Do not sync the SQLite/Milvus caches.

Step 6 — Package as a global plugin (Part D)

• Wrap Steps 2–3 into a Claude Code plugin with skills; install at user level.

Step 7 (deferred) — QMD semantic layer

• Only when structured-only retrieval misses notes you know exist: qmd over the vault, optionally as an MCP server the AI can query.

---

Part B — The AI's write & query conventions (skills teach these)

When the AI WRITES to the vault

Trigger writes when the AI learns something evergreen and reusable across projects — not project-ephemeral state (that's the episodic layer / project files). Concretely:

• It worked out how a tool/API behaves (e.g. SEMrush auth, rate limits, an endpoint quirk).
• It established a convention, decision, or preference that should apply beyond this task.
• It discovered a client-specific fact worth reusing (how this client uses a tool).

What conventions to follow when writing (enforced by the frontmatter contract):

• One concept per note; keep notes small (the L1 "under ~200 lines / one topic" discipline).
• Required frontmatter: a one-line summary (written now, not deferred), scope/global or scope/project, and namespaced tags (tool/…, client/…, domain/…, optionally convention/…).
• Scope rule: default new tool/domain knowledge to scope/global; mark scope/project + a client/<name> tag when it's specific to how a client uses something.
• Write only to the vault, never silently into a project repo.
• Promotion to global (project → global) and consolidation are not done inline — they happen in the reorganize step (Part C), in plan mode, for human review.

When & how the AI QUERIES

• At session start: the injected index overview + resolved convention/* files tell it what's available without reading everything.
• On demand, during a task: when the task touches a tool/client/domain, run index query --tool semrush --client sesame3g → get {path, summary, tags} → open only the files whose summary matches. This is the progressive-disclosure loop that keeps tokens low.
• Cross-client lookups: query by --tool alone (omit client) to surface what was learned with any client; tags in the output show provenance.
• "What happened" questions go to the episodic layer (memsearch) in natural language, not the tag index.

---

Part C — Hooks & CRUD mapping

CRUD over the knowledge vault, and which hook/command services each operation:

Operation	Trigger	Mechanism
Create	AI writes a new note	AI writes .md (Write tool) → PostToolUse hook → index update --file
Read / query	AI needs knowledge	AI calls index query … (no hook; on-demand CLI)
Update	AI/user edits a note	AI edit → PostToolUse hook → index update --file; user edit → SessionStart reconcile
Delete / rename	note removed/renamed	covered by --file (prune) on AI ops; by SessionStart reconcile (prune vanished paths) on manual ops
Inject	new session/sub-agent starts	SessionStart hook injects index overview + resolved convention/* + journal pointer
Journal	session ends	SessionEnd hook appends a dated journal note with pointers
Reorganize	periodic, user-invoked	reorganize memory run in plan mode: dedupe, merge, split, re-tag, promote scope/global, rebuild index — human approves

Hooks are thin shell wrappers over the Ruby CLI so the logic lives in one place.

Hooks summary:

• SessionStart — reconcile (index update --since auto) + inject context.
• PostToolUse (on Write/Edit of vault .md) — index update --file.
• SessionEnd — append daily journal note.
• (memsearch brings its own UserPromptSubmit + capture hooks for the episodic layer.)

---

Part D — Claude Code plugin with skills (global install)

Goal: one global install so every project/machine knows the vault, conventions, hooks, and CLI.

Plugin contents:

• Hooks registered in settings: SessionStart, PostToolUse, SessionEnd (the shell wrappers from Part C), pointed at a configurable vault path + cache dir.
• The Ruby CLI (the tag index) bundled or installed as a dependency, on PATH.
• Skills (these carry the know-how to the model):
  • memory-write — when to record evergreen knowledge, the frontmatter contract, scope rules, "vault not repo." (Part B write section.)
  • memory-query — how/when to query the tag index vs the episodic layer; the progressive- disclosure loop; cross-client lookups. (Part B query section.)
  • memory-reorganize — the plan-mode consolidation/promotion procedure + guardrails.
• Config: vault path, cache path, sync method — set once at user level.

Why a plugin + skills (not just CLAUDE.md): hooks must be registered by the harness (the plugin does that), and the conventions are taught as skills so they load on demand without bloating every project's context. A single global install keeps the conventions themselves a single source of truth — edit the skill once, every project follows.

Open question for build time: do the coding-convention/* notes live in the vault (data, edited freely, resolved by the SessionStart hook) while the memory-system skills live in the plugin (behavior, versioned)? Recommended: yes — conventions are data in the vault; the skills are the mechanism.