SecondBrain/2026-06-09-claude-code-memory-system-design.md

---
summary: Architecture for a two-layer personal memory system for Claude Code — episodic (memsearch) and semantic/knowledge (Obsidian vault + Graphify), hook-injected and on-demand.
tags:
  - type/reference
  - domain/knowledge-graphs
  - domain/llm
  - tool/graphify
  - scope/global
source: cc-os
date: 2026-06-09
---

# Claude Code Memory System — Architecture Design

## Core principle: two memory types, kept separate

| Type | Question | Lifecycle | Write path | Tool |
|------|----------|-----------|-----------|------|
| **Episodic** | "What happened, when?" | accretes & decays | auto-captured | **memsearch** |
| **Semantic / knowledge** | "How do we…?" | deliberately maintained | curated by AI/author | **Obsidian vault + Graphify knowledge graph** |

The episodic vs. semantic split is the key architectural decision. They have different lifecycles, write paths, and query patterns. Forcing one tool to do both is what makes every unified-memory design feel forced.

## Goals the architecture must satisfy

1. **Thin projects** — keep as little AI context inside each project repo as possible. Knowledge is pulled in on demand or injected by hooks.
2. **Cross-project / cross-client knowledge** — learn something once (e.g. a tool's API) and reference it from anywhere. Two scopes: **global** (broadly useful) and **project/client-specific** — both globally reachable.
3. **Timeline awareness** — lightweight awareness of recent activity ("what was I doing yesterday"), with the ability to drill deeper.
4. **Remote, local-fast** — accessible anywhere but runs local-fast; lazy sync (minutes/hourly) is fine; real-time is overkill.

Desired properties: **lightweight** (low tokens), **fast** (out of the way), **flexible** (cross project/client), **self-evolving** (AI maintains it under clear rules), **semi-structured** (organization that can evolve).

Both layers are **local-first, markdown-as-truth, no Docker, no server, no API keys** (Graphify extraction runs against a local Ollama model).

## Layer 1 — Episodic (memsearch)

- **What it is**: A Claude Code plugin (by Zilliz) that auto-captures session notes as daily markdown, chunks them, and stores a **shadow index** in **Milvus Lite** (a single embedded file — no server, no Docker). Hybrid search = BM25 + dense vectors + RRF, local ONNX embeddings (`bge-m3`, no API key/cost). A FileWatcher (1500ms debounce) handles updates and deletions.
- **Why off-the-shelf**: It already implements the daily-notes + "dreaming" pattern and the markdown-as-truth / disposable-shadow-index philosophy.
- **Role**: Satisfies timeline goal. The AI queries it in natural language ("what was decided about X last week"). Does **not** filter by knowledge tags — it owns the episodic corpus only.

## Layer 2 — Knowledge (vault + Graphify knowledge graph)

### Vault structure

- **Flat markdown directory**, single source of truth — reuses an existing Obsidian vault (`~/Documents/SecondBrain`). Browsable in Obsidian as a viewer.
- Replaces project-local documentation: instead of docs scattered per repo, knowledge lives once in the vault and is pulled into any project on demand.

### Frontmatter contract (every note)

```yaml
---
summary: One line, written at creation. The router shows this so the AI can pick a file without opening it.
tags:
  - type/reference        # listed first by convention; e.g. type/hub, type/how-to
  - client/clientname
  - project/project-name
  - domain/seo
  - tool/semrush
  - convention/api-style
  - scope/project         # or scope/global
---
```

- **Six flat facets**: `type/`, `client/`, `project/`, `domain/`, `tool/`, `convention/` — plus `scope/`. Each facet is independent and parallel (never nested into each other). `#tool` matches all `tool/*` values — native Obsidian prefix filtering, no folders needed.
- **Hierarchy and relationships** are expressed via **hub notes** (`type/hub`), **wikilinks**, and **Graphify graph edges** — NOT via nested tag paths.
- **Two knowledge scopes** via `scope/global` vs `scope/project` (+ a `client/` tag): global = broadly useful tool/domain knowledge; project = how a specific client uses it. Both are globally queryable.

### Knowledge graph (Graphify)

Graphify turns the vault into a queryable **knowledge graph** — a disposable, rebuildable structure over the markdown. It replaces a bespoke tag index CLI *and* a deferred vector layer: one graph gives both structured and semantic retrieval, without vectors.

**Extraction:**
- **Vault docs** → a **local Ollama SLM** extracts entities + typed relationships from each note (confidence-tagged `EXTRACTED` / `INFERRED` / `AMBIGUOUS`). Local model = no API cost, no data leaving the machine.
- **Project code** → free **tree-sitter AST** (`--no-docs`), no model, no token cost. Kept as separate per-project graphs, not merged with the vault graph.

**What it produces**: `graphify-out/` with `graph.json`, an interactive `graph.html`, and a `GRAPH_REPORT.md` whose top lists the **god nodes** (the most-connected concepts — highest-value entry points).

**Query** (via CLI and MCP server exposing `query_graph` / `get_node` / `shortest_path`): ask for **god nodes first**, then scalpel down with `graphify query` / `path` / `explain`. Prompt the graph; don't dump the corpus into context.

**Metadata still matters**: the `summary` + six facet tags remain first-class note attributes — `summary` is the human-written router hint Graphify does **not** generate. Facet namespaces stay useful for Obsidian filtering and as node attributes.

**Source of truth rule**: markdown is authoritative; `graphify-out/` is a rebuildable artifact that is **never synced** and can be deleted/rebuilt anytime (`graphify ... --force`).

### Freshness (lazy)

- **AI writes** → a `PostToolUse` hook on `Write`/`Edit` targeting vault `.md` files runs `graphify ... --update` to merge the changed note into the vault graph. Event-driven, no polling.
- **Stale-node caveat**: Graphify's `--update` merges (SHA-256 + dedup) but does **not** prune deleted notes — ghost nodes accumulate. A periodic `--force` rebuild clears them, triggered by the **session-start reconcile** when a rebuild stamp is older than N days. **No daemon, no cron.**

### Retrieval (hook-injected + on-demand)

- **Session-start hook** injects: (a) a compact overview — the vault graph's **god nodes** as the map of what's known, (b) the current project's declared `convention/*` notes resolved to their summaries (coding conventions auto-pull; a convention edit propagates to every project using that tag), (c) a pointer to recent episodic journal.
- **On demand**: the AI runs `graphify query` / `path` / `explain` (or the MCP tools) to pull specific knowledge into context only when the task needs it. Projects stay thin — their CLAUDE.md holds **tags/pointers**, not content.

## Semantic recall — covered by Graphify

An earlier design earmarked a separate vector layer for "when structured tag filtering misses a note whose wording doesn't match the query." Graphify's knowledge graph covers that need without a second system or vectors: relationship traversal and `explain` surface notes by *connection*, not just exact tag match. Revisit a vector layer only if graph traversal demonstrably misses cases where embedding similarity would clearly win.

## Timeline details

A **session-end hook** appends a daily journal note (one file per date) with pointers to the project/knowledge files touched. memsearch indexes these; today+yesterday are cheap to load, older entries are reachable by query for drill-down.

## Self-evolution guardrails

- The AI **writes only to the vault**, never silently into project repos.
- **Required frontmatter schema** (summary + namespaced tags) is enforced so the index stays queryable.
- **Daily notes are append-only**; consolidation/reorg is a **separate, reviewable step** run in plan mode.
- **Promotion to `scope/global`** requires a rule (e.g. a fact recurring N times) — not every stray note gets promoted.

## Sync

- The **vault** syncs to a remote machine via **git** (versioned history) or **Syncthing** (continuous, zero-thought).
- **Graphs/indexes are never synced** — the Milvus Lite episodic index and Graphify `graphify-out/` graphs are rebuilt per machine. Sync only the markdown.

## Packaging

Ships as a **global Claude Code plugin with skills** (hooks + scripts + CRUD know-how) so every project, on every machine, knows how to use the vault effectively.

## How each goal is met

| Goal | Met by |
|------|--------|
| 1. Thin projects | Knowledge in the vault, not repos; CLAUDE.md holds tags/pointers; on-demand graph query |
| 2. Cross-project/client knowledge | Vault + six-facet tags + Graphify knowledge graph (god nodes + traversal) |
| 3. Timeline | memsearch episodic layer + session-end journal hook |
| 4. Remote, local-fast | Markdown vault synced via git/Syncthing; disposable per-machine graphs/indexes |