156 lines
8.8 KiB
Markdown
156 lines
8.8 KiB
Markdown
# Build Plan
|
||
|
||
How a human builds this system, step by step, and answers to the operational questions:
|
||
which scripts and hooks, how the AI knows when to write and what conventions to follow, how and
|
||
when it queries, the CRUD hooks, and how it's packaged as a global Claude Code plugin with
|
||
skills.
|
||
|
||
> This is a build outline, not the implementation plan. The next session should turn this into
|
||
> a proper implementation plan (writing-plans skill) and execute it.
|
||
|
||
---
|
||
|
||
## Part A — Build order (human builder's path)
|
||
|
||
Build bottom-up: the vault and CLI first (usable standalone), then the hooks, then the plugin
|
||
that packages it.
|
||
|
||
### Step 1 — Vault skeleton & conventions
|
||
- Decide the vault location (default: a synced home dir, e.g. `~/brain`; symlink into
|
||
`~/.claude/memory` only if a tool insists). **Vault is the single source of truth.**
|
||
- Write `CONVENTIONS.md` in the vault: the frontmatter contract and the tag namespaces
|
||
(`tool/`, `client/`, `domain/`, `convention/`, `scope/`). This doubles as the spec the
|
||
skills teach the AI.
|
||
- Seed a few real notes (e.g. `tool/semrush` global + a `client/<x>` project note) to test
|
||
against.
|
||
|
||
### Step 2 — The Ruby tag-index CLI (the core build)
|
||
- Ruby project, `sequel` + `sqlite3` gems. SQLite file in a cache dir (e.g.
|
||
`~/.cache/memory-index/index.sqlite`) — disposable, never synced.
|
||
- **Schema / migrations**: `files(id, path, mtime, summary, scope)`, `tags(id, name)`,
|
||
`files_tags(file_id, tag_id)`; `many_to_many` between files and tags.
|
||
- **Frontmatter parser**: read YAML frontmatter from a `.md` file → `{summary, scope, tags[]}`.
|
||
Fail loudly (or quarantine) notes missing the required `summary`/tags so the contract is
|
||
enforced.
|
||
- **Commands** (a thin CLI dispatch — `thor`/`optparse` or plain):
|
||
- `index update --since <iso8601|auto>` — find `.md` with `mtime >` last-cache-time, upsert
|
||
them; **reconcile**: delete rows for paths no longer on disk.
|
||
- `index update --file <path>` — upsert a single file (used by the write hook); prune if the
|
||
path is gone.
|
||
- `index update --rebuild` (default off) — drop & rebuild from a full vault scan.
|
||
- `index query --client X --tool Y [--domain Z] [--scope global]` — AND the filters; output
|
||
JSON lines of `{path, summary, tags[]}` (option C).
|
||
- `index tags [--namespace tool/]` — enumerate the vocabulary (virtual index).
|
||
- Store `last_cache_time` (a tiny meta table or a stamp file) so `--since auto` works.
|
||
- **Tests**: create/edit/delete/rename a note, assert the index reflects it; rebuild equals
|
||
incremental.
|
||
|
||
### Step 3 — Hooks (maintenance + retrieval)
|
||
See Part C for exact mapping. Implement as small shell wrappers that call the Ruby CLI:
|
||
- `pre-tool-memory` style **PostToolUse** updater (write path → `index update --file`).
|
||
- **SessionStart** reconcile + inject (index overview + resolved `convention/*` files + journal
|
||
pointer).
|
||
- **SessionEnd** journal appender (daily note with pointers).
|
||
|
||
### Step 4 — Episodic layer (memsearch)
|
||
- Install memsearch (`/plugin marketplace add zilliztech/memsearch`, then `plugin install
|
||
memsearch`), local to start. Verify daily memory files appear after a few conversations.
|
||
- Decide whether memsearch indexes our session-end journal notes or its own capture (likely its
|
||
own; our journal can point into the knowledge vault).
|
||
|
||
### Step 5 — Sync
|
||
- Pick **git** (versioned, hourly push/pull) or **Syncthing** (continuous, zero-thought) for the
|
||
vault → VPS. Configure on each machine. **Do not sync the SQLite/Milvus caches.**
|
||
|
||
### Step 6 — Package as a global plugin (Part D)
|
||
- Wrap Steps 2–3 into a Claude Code plugin with skills; install at user level.
|
||
|
||
### Step 7 (deferred) — QMD semantic layer
|
||
- Only when structured-only retrieval misses notes you know exist: `qmd` over the vault,
|
||
optionally as an MCP server the AI can query.
|
||
|
||
---
|
||
|
||
## Part B — The AI's write & query conventions (skills teach these)
|
||
|
||
### When the AI WRITES to the vault
|
||
Trigger writes when the AI learns something **evergreen and reusable across projects** — not
|
||
project-ephemeral state (that's the episodic layer / project files). Concretely:
|
||
- It worked out how a tool/API behaves (e.g. SEMrush auth, rate limits, an endpoint quirk).
|
||
- It established a convention, decision, or preference that should apply beyond this task.
|
||
- It discovered a client-specific fact worth reusing (how *this* client uses a tool).
|
||
|
||
**What conventions to follow when writing** (enforced by the frontmatter contract):
|
||
- One concept per note; keep notes small (the L1 "under ~200 lines / one topic" discipline).
|
||
- **Required frontmatter**: a one-line `summary` (written now, not deferred), `scope/global`
|
||
or `scope/project`, and namespaced tags (`tool/…`, `client/…`, `domain/…`, optionally
|
||
`convention/…`).
|
||
- **Scope rule**: default new tool/domain knowledge to `scope/global`; mark `scope/project` +
|
||
a `client/<name>` tag when it's specific to how a client uses something.
|
||
- **Write only to the vault**, never silently into a project repo.
|
||
- **Promotion to global** (project → global) and **consolidation** are *not* done inline —
|
||
they happen in the reorganize step (Part C), in plan mode, for human review.
|
||
|
||
### When & how the AI QUERIES
|
||
- **At session start**: the injected index overview + resolved `convention/*` files tell it
|
||
what's available without reading everything.
|
||
- **On demand, during a task**: when the task touches a tool/client/domain, run
|
||
`index query --tool semrush --client sesame3g` → get `{path, summary, tags}` → open only the
|
||
files whose summary matches. This is the progressive-disclosure loop that keeps tokens low.
|
||
- **Cross-client lookups**: query by `--tool` alone (omit client) to surface what was learned
|
||
with *any* client; tags in the output show provenance.
|
||
- **"What happened" questions** go to the **episodic layer (memsearch)** in natural language,
|
||
not the tag index.
|
||
|
||
---
|
||
|
||
## Part C — Hooks & CRUD mapping
|
||
|
||
CRUD over the knowledge vault, and which hook/command services each operation:
|
||
|
||
| Operation | Trigger | Mechanism |
|
||
|-----------|---------|-----------|
|
||
| **Create** | AI writes a new note | AI writes `.md` (Write tool) → **PostToolUse** hook → `index update --file` |
|
||
| **Read / query** | AI needs knowledge | AI calls `index query …` (no hook; on-demand CLI) |
|
||
| **Update** | AI/user edits a note | AI edit → **PostToolUse** hook → `index update --file`; user edit → **SessionStart** reconcile |
|
||
| **Delete / rename** | note removed/renamed | covered by `--file` (prune) on AI ops; by **SessionStart** reconcile (prune vanished paths) on manual ops |
|
||
| **Inject** | new session/sub-agent starts | **SessionStart** hook injects index overview + resolved `convention/*` + journal pointer |
|
||
| **Journal** | session ends | **SessionEnd** hook appends a dated journal note with pointers |
|
||
| **Reorganize** | periodic, user-invoked | `reorganize memory` run in **plan mode**: dedupe, merge, split, re-tag, promote `scope/global`, rebuild index — human approves |
|
||
|
||
Hooks are thin shell wrappers over the Ruby CLI so the logic lives in one place.
|
||
|
||
**Hooks summary:**
|
||
- **SessionStart** — reconcile (`index update --since auto`) + inject context.
|
||
- **PostToolUse** (on `Write`/`Edit` of vault `.md`) — `index update --file`.
|
||
- **SessionEnd** — append daily journal note.
|
||
- (memsearch brings its own `UserPromptSubmit` + capture hooks for the episodic layer.)
|
||
|
||
---
|
||
|
||
## Part D — Claude Code plugin with skills (global install)
|
||
|
||
Goal: one global install so every project/machine knows the vault, conventions, hooks, and CLI.
|
||
|
||
**Plugin contents:**
|
||
- **Hooks** registered in settings: SessionStart, PostToolUse, SessionEnd (the shell wrappers
|
||
from Part C), pointed at a configurable vault path + cache dir.
|
||
- **The Ruby CLI** (the tag index) bundled or installed as a dependency, on `PATH`.
|
||
- **Skills** (these carry the *know-how* to the model):
|
||
- `memory-write` — when to record evergreen knowledge, the frontmatter contract, scope rules,
|
||
"vault not repo." (Part B write section.)
|
||
- `memory-query` — how/when to query the tag index vs the episodic layer; the progressive-
|
||
disclosure loop; cross-client lookups. (Part B query section.)
|
||
- `memory-reorganize` — the plan-mode consolidation/promotion procedure + guardrails.
|
||
- **Config**: vault path, cache path, sync method — set once at user level.
|
||
|
||
**Why a plugin + skills (not just CLAUDE.md):** hooks must be registered by the harness (the
|
||
plugin does that), and the conventions are taught as skills so they load on demand without
|
||
bloating every project's context. A single global install keeps the conventions themselves a
|
||
single source of truth — edit the skill once, every project follows.
|
||
|
||
**Open question for build time**: do the coding-`convention/*` notes live *in the vault* (data,
|
||
edited freely, resolved by the SessionStart hook) while the *memory-system skills* live *in the
|
||
plugin* (behavior, versioned)? Recommended: yes — conventions are data in the vault; the skills
|
||
are the mechanism.
|