cc-os/docs/memory-system/02-system-design.md

# System Design

_Status: approved 2026-06-03. Implementation not yet started._

## Goals (what this system must do)

1. **Thin projects** — keep as little AI context inside each project repo as possible. Projects
   focus on project files; knowledge is pulled in on demand or injected by hooks.
2. **Cross-project / cross-client knowledge** — the AI learns something once (e.g. the SEMrush
   API) and references it from anywhere. Two scopes: **global** (broadly useful) and
   **project/client-specific** (how a given client uses a tool) — both globally reachable.
   Ask anything client- or project-related from any project.
3. **Timeline awareness** — from any project, lightweight awareness of recent activity ("what
   was I doing an hour ago / yesterday"), with the ability to drill deeper.
4. **Remote, local-fast** — accessible anywhere (VPS / personal OS) but runs local-fast; lazy
   sync (minutes/hourly) is fine; real-time is overkill.

Desired properties: **lightweight** (low tokens), **fast** (out of the way), **flexible**
(cross project/client), **self-evolving** (AI maintains it under clear rules), **easy to
manage** (AI-managed), **semi-structured** (organization that can evolve).

## Core principle: two memory types, kept separate

| Type | Question | Lifecycle | Write path | Our tool |
|------|----------|-----------|-----------|----------|
| **Episodic** | "What happened, when?" | accretes & decays | auto-captured | **memsearch** |
| **Semantic / knowledge** | "How do we…?" | deliberately maintained | curated by you/AI | **Obsidian vault + tag index** |

This is the classic **episodic vs. semantic** memory split. Keeping them separate is the key
architectural decision — they have different lifecycles, write paths, and query patterns, and
forcing one tool to do both is what made every earlier design feel forced.

## The three layers

```
┌─────────────────────────────────────────────────────────────────┐
│ EPISODIC  ── memsearch (Milvus Lite, embedded, no Docker)         │
│   auto-captured session/journal notes · NL semantic recall        │
│   answers "when did we…", "what was I doing yesterday"            │
├─────────────────────────────────────────────────────────────────┤
│ KNOWLEDGE ── flat Obsidian vault (single source of truth)         │
│   + Ruby/Sequel/SQLite tag index (CLI)                            │
│   structured tag filtering · answers "how do we…", "what do we    │
│   know about X for client Y"                                       │
├─────────────────────────────────────────────────────────────────┤
│ SEMANTIC-OVER-KNOWLEDGE ── QMD (SQLite+vector, MCP)   [DEFERRED]   │
│   meaning-based recall over the vault when tags miss              │
│   add ONLY when structured tagging proves insufficient            │
└─────────────────────────────────────────────────────────────────┘
```

All three are **local-first, markdown-as-truth, no Docker, no server, no API keys.**

## Layer 1 — Episodic (memsearch)

- **What it is**: a Claude Code plugin (by Zilliz) that auto-captures session notes as daily
  markdown, chunks them, and stores a **shadow index** in **Milvus Lite** (a single embedded
  file — no server, no Docker). Hybrid search = BM25 + dense vectors + RRF, local ONNX
  embeddings (`bge-m3`, no API key/cost). A FileWatcher (1500ms debounce) handles updates and
  deletions; markdown stays the source of truth.
- **Why off-the-shelf**: it already implements the OpenClaw daily-notes + "dreaming" pattern
  and the markdown-as-truth / disposable-shadow-index philosophy we'd otherwise hand-build.
- **Role in our system**: satisfies Goal 3 (timeline). The AI queries it in natural language
  ("what was decided about X last week"). We do **not** make it filter by our tags — it owns
  the episodic corpus only.

## Layer 2 — Knowledge (vault + tag index)

The heart of the system, and the part we build.

### Vault
- **Flat markdown directory**, single source of truth, **configurable location** (NOT forced
  into `~/.claude/`; symlink if a tool insists). Browsable in Obsidian as a viewer.
- **Replaces project-local documentation**: instead of docs scattered per repo, knowledge
  lives once in the vault and is pulled into any project on demand.

### Frontmatter contract (every note)
```yaml
---
summary: One line, written at creation. The router shows this so the AI can pick a
  file without opening it.
tags:
  - tool/semrush          # namespaced, nested (slash = Obsidian nested tag)
  - client/sesame3g
  - domain/seo
  - scope/project         # or scope/global
---
```
- **Namespaces** are the "virtual indexes": `tool/`, `client/`, `domain/`, `convention/`,
  `scope/`. `#tool` matches all children — native prefix filtering, no folders needed.
- **Two knowledge scopes** via `scope/global` vs `scope/project` (+ a `client/` tag): global =
  broadly useful tool/domain knowledge; project = how a specific client uses it. Both are
  globally queryable; the scope tag is the shortcut that avoids scanning every client's usage.

### Index (the tag cache)
A small **Ruby program, Sequel ORM, SQLite** — the disposable structured cache over the vault.

- **Schema**:
  - `files(id, path, mtime, summary, scope)` — one row per note.
  - `tags(id, name)` — one row per distinct tag (enables enumerating the vocabulary:
    "what clients/tools do I have notes on?").
  - `files_tags(file_id, tag_id)` — `many_to_many` join.
- **CLI** (the only interface; the AI never touches SQLite directly):
  - `index update --since <mtime>` — incremental: pulls `.md` files with `mtime >` last cache
    time, re-reads their frontmatter, upserts; **also reconciles** (prunes rows whose path no
    longer exists).
  - `index update --rebuild` — full rebuild from scratch (default `false`).
  - `index query --client X --tool Y [--scope global] [--domain Z]` — returns **path +
    summary + matched tags** for each hit (decided: option C). Tags shown so the AI sees *why*
    a file matched.
  - `index tags --namespace tool/` — enumerate a virtual index.
- **Source of truth rule**: markdown is authoritative; the SQLite file is a rebuildable cache
  that is **never synced** and can be deleted/rebuilt anytime.

### Freshness (lazy — chosen Option A)
- **AI writes** → a `PostToolUse` hook on `Write`/`Edit` targeting vault `.md` files calls
  `index update --file <path>` (updates exactly that file, prunes if deleted). Event-driven,
  no polling, no staleness for AI edits.
- **Manual edits** (rare) → caught by a **session-start reconcile** (`index update --since` +
  prune). **No daemon, no cron.**

### Retrieval (hook-injected + on-demand)
- **Session-start hook** injects: (a) a compact index/overview, (b) the current project's
  declared `convention/*` tags **resolved to files** (so coding conventions auto-pull and a
  convention edit propagates to every project using that tag), (c) a pointer to recent
  episodic journal.
- **On demand**: the AI runs `index query` to pull specific knowledge into context only when
  the task needs it. Projects stay thin — their CLAUDE.md holds **tags/pointers**, not content.

## Layer 3 — Semantic over knowledge (QMD) — DEFERRED

- **What**: QMD (github.com/tobi/qmd) — local markdown search, **SQLite + FTS5/BM25 + local
  vector embeddings (EmbeddingGemma-300M GGUF) + LLM rerank**. CLI + optional **MCP server**.
  No Docker, no API keys. Proves SQLite + a local vector model is enough — no Milvus/Postgres
  for knowledge.
- **Why deferred**: start structured-only. Add QMD as a **set-and-forget** semantic layer over
  the vault **only when** we catch ourselves failing to retrieve notes we know exist (the
  video's "only level up when it bites"). It complements, not replaces, the tag index (QMD
  filters by path/collection context, not first-class frontmatter tags).

## Timeline (Goal 3) details
- A **session-end hook** appends a daily journal note (one file per date) with pointers to the
  project/knowledge files touched. memsearch indexes these; today+yesterday are cheap to load,
  older entries are reachable by query for drill-down.

## Self-evolution guardrails
- The AI **writes only to the vault**, never silently into project repos.
- **Required frontmatter schema** (summary + namespaced tags) is enforced so the index stays
  queryable.
- **Daily notes are append-only**; consolidation/reorg is a **separate, reviewable step run in
  plan mode** (Connelly's reorganize + Huryn's propose-and-approve loop).
- **Promotion to `scope/global`** requires a rule (e.g. a fact recurring N times) — not every
  stray note gets promoted.

## Sync (Goal 4)
- The **vault** syncs to the VPS via **git** (versioned history, hourly) or **Syncthing**
  (continuous, zero-thought). Decision deferred to build time.
- **Indexes are never synced** — Milvus Lite and (later) QMD shadow indexes are rebuilt per
  machine. Sync only the markdown.

## Packaging
- The whole thing ships as a **global Claude Code plugin with skills** (hooks + scripts +
  CRUD know-how) so every project, on every machine, knows how to use the vault effectively.
  See [04-build-plan.md](04-build-plan.md).

## How each goal is met

| Goal | Met by |
|------|--------|
| 1. Thin projects | Knowledge in the vault, not repos; CLAUDE.md holds tags/pointers; on-demand `index query` |
| 2. Cross-project/client knowledge, global vs project scopes | Flat vault + namespaced tags + `scope/` + `client/`; enumerable virtual indexes |
| 3. Timeline | memsearch episodic layer + session-end journal hook |
| 4. Remote, local-fast | Markdown vault synced via git/Syncthing; disposable per-machine indexes |