jared
/
cc-os
1
0
Fork 0
Code Issues 18 Pull Requests Packages Projects Releases Wiki Activity
cc-os/docs/memory-system/02-system-design.md

193 lines
12 KiB
Markdown
Raw Normal View History

Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
# System Design
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
_Status: approved 2026-06-03; knowledge layer revised 2026-06-04 (Graphify replaces the Ruby
Update design docs and add implementation runbook per ADR-011/012 Update CLAUDE.md: clarify the knowledge layer now specifies the ~/Documents/SecondBrain vault, six-facet taxonomy, and reference the locked decisions (ADR-011, ADR-012). Update 02-system-design.md: align with faceted taxonomy (type/ client/project/domain/tool/convention/ plus scope/), express hierarchy via hubs + wikilinks + Graphify edges rather than nested paths, name the vault explicitly, update build-plan timestamp. Add 05-implementation-process.md: concrete build runbook integrating the locked decisions — seven steps bottom-up from vault migration through plugin packaging, with the Graphify + Ollama model benchmark (Step 2c) marked as the critical gate. Open questions deferred or defaulted; most are non-blocking. Design milestone: tag taxonomy and vault location locked.
2026-06-04 13:35:43 +00:00
tag-index CLI and the deferred QMD layer — see ADR-010); tag taxonomy and vault location locked
2026-06-04 (six-facet taxonomy — see ADR-011; reuse ~/Documents/SecondBrain — see ADR-012).
Implementation not yet started._
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
## Goals (what this system must do)
1. **Thin projects** — keep as little AI context inside each project repo as possible. Projects
focus on project files; knowledge is pulled in on demand or injected by hooks.
2. **Cross-project / cross-client knowledge** — the AI learns something once (e.g. the SEMrush
API) and references it from anywhere. Two scopes: **global** (broadly useful) and
**project/client-specific** (how a given client uses a tool) — both globally reachable.
Ask anything client- or project-related from any project.
3. **Timeline awareness** — from any project, lightweight awareness of recent activity ("what
was I doing an hour ago / yesterday"), with the ability to drill deeper.
4. **Remote, local-fast** — accessible anywhere (VPS / personal OS) but runs local-fast; lazy
sync (minutes/hourly) is fine; real-time is overkill.
Desired properties: **lightweight** (low tokens), **fast** (out of the way), **flexible**
(cross project/client), **self-evolving** (AI maintains it under clear rules), **easy to
manage** (AI-managed), **semi-structured** (organization that can evolve).
## Core principle: two memory types, kept separate
| Type | Question | Lifecycle | Write path | Our tool |
|------|----------|-----------|-----------|----------|
| **Episodic** | "What happened, when?" | accretes & decays | auto-captured | **memsearch** |
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
| **Semantic / knowledge** | "How do we…?" | deliberately maintained | curated by you/AI | **Obsidian vault + Graphify knowledge graph** |
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
This is the classic **episodic vs. semantic** memory split. Keeping them separate is the key
architectural decision — they have different lifecycles, write paths, and query patterns, and
forcing one tool to do both is what made every earlier design feel forced.
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
## The two layers
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
```
┌─────────────────────────────────────────────────────────────────┐
│ EPISODIC ── memsearch (Milvus Lite, embedded, no Docker) │
│ auto-captured session/journal notes · NL semantic recall │
│ answers "when did we…", "what was I doing yesterday" │
├─────────────────────────────────────────────────────────────────┤
Update design docs and add implementation runbook per ADR-011/012 Update CLAUDE.md: clarify the knowledge layer now specifies the ~/Documents/SecondBrain vault, six-facet taxonomy, and reference the locked decisions (ADR-011, ADR-012). Update 02-system-design.md: align with faceted taxonomy (type/ client/project/domain/tool/convention/ plus scope/), express hierarchy via hubs + wikilinks + Graphify edges rather than nested paths, name the vault explicitly, update build-plan timestamp. Add 05-implementation-process.md: concrete build runbook integrating the locked decisions — seven steps bottom-up from vault migration through plugin packaging, with the Graphify + Ollama model benchmark (Step 2c) marked as the critical gate. Open questions deferred or defaulted; most are non-blocking. Design milestone: tag taxonomy and vault location locked.
2026-06-04 13:35:43 +00:00
│ KNOWLEDGE ── ~/Documents/SecondBrain Obsidian vault (single SOT) │
│ notes carry summary + six facet tags + scope/ (metadata) │
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
│ + Graphify knowledge graph (local SLM over docs; AST over code) │
│ graph queries (god nodes / query / path) · answers "how do we…",│
│ "what do we know about X for client Y", "what relates to Y" │
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
└─────────────────────────────────────────────────────────────────┘
```
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
Both are **local-first, markdown-as-truth, no Docker, no server, no API keys** (Graphify
extraction runs against a **local Ollama** model). An earlier design split the knowledge layer
into a Ruby/SQLite tag index plus a deferred QMD vector layer; **Graphify replaces both** — it
provides structured *and* semantic retrieval over the vault as a single graph (ADR-010). The
`summary` + tag frontmatter is **retained** as note metadata (router hint + cross-cutting
filters), it is just no longer backed by a bespoke index.
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
## Layer 1 — Episodic (memsearch)
- **What it is**: a Claude Code plugin (by Zilliz) that auto-captures session notes as daily
markdown, chunks them, and stores a **shadow index** in **Milvus Lite** (a single embedded
file — no server, no Docker). Hybrid search = BM25 + dense vectors + RRF, local ONNX
embeddings (`bge-m3`, no API key/cost). A FileWatcher (1500ms debounce) handles updates and
deletions; markdown stays the source of truth.
- **Why off-the-shelf**: it already implements the OpenClaw daily-notes + "dreaming" pattern
and the markdown-as-truth / disposable-shadow-index philosophy we'd otherwise hand-build.
- **Role in our system**: satisfies Goal 3 (timeline). The AI queries it in natural language
("what was decided about X last week"). We do **not** make it filter by our tags — it owns
the episodic corpus only.
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
## Layer 2 — Knowledge (vault + Graphify knowledge graph)
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
The heart of the system, and the part we build.
### Vault
Update design docs and add implementation runbook per ADR-011/012 Update CLAUDE.md: clarify the knowledge layer now specifies the ~/Documents/SecondBrain vault, six-facet taxonomy, and reference the locked decisions (ADR-011, ADR-012). Update 02-system-design.md: align with faceted taxonomy (type/ client/project/domain/tool/convention/ plus scope/), express hierarchy via hubs + wikilinks + Graphify edges rather than nested paths, name the vault explicitly, update build-plan timestamp. Add 05-implementation-process.md: concrete build runbook integrating the locked decisions — seven steps bottom-up from vault migration through plugin packaging, with the Graphify + Ollama model benchmark (Step 2c) marked as the critical gate. Open questions deferred or defaulted; most are non-blocking. Design milestone: tag taxonomy and vault location locked.
2026-06-04 13:35:43 +00:00
- **Flat markdown directory**, single source of truth — reuses the existing **`~/Documents/SecondBrain`** Obsidian vault (ADR-012); not forced into `~/.claude/`; symlink if a tool insists. Browsable in Obsidian as a viewer.
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
- **Replaces project-local documentation**: instead of docs scattered per repo, knowledge
lives once in the vault and is pulled into any project on demand.
### Frontmatter contract (every note)
```yaml
---
summary: One line, written at creation. The router shows this so the AI can pick a
file without opening it.
tags:
Update design docs and add implementation runbook per ADR-011/012 Update CLAUDE.md: clarify the knowledge layer now specifies the ~/Documents/SecondBrain vault, six-facet taxonomy, and reference the locked decisions (ADR-011, ADR-012). Update 02-system-design.md: align with faceted taxonomy (type/ client/project/domain/tool/convention/ plus scope/), express hierarchy via hubs + wikilinks + Graphify edges rather than nested paths, name the vault explicitly, update build-plan timestamp. Add 05-implementation-process.md: concrete build runbook integrating the locked decisions — seven steps bottom-up from vault migration through plugin packaging, with the Graphify + Ollama model benchmark (Step 2c) marked as the critical gate. Open questions deferred or defaulted; most are non-blocking. Design milestone: tag taxonomy and vault location locked.
2026-06-04 13:35:43 +00:00
- type/reference # listed first by convention; e.g. type/hub, type/how-to
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
- client/sesame3g
Update design docs and add implementation runbook per ADR-011/012 Update CLAUDE.md: clarify the knowledge layer now specifies the ~/Documents/SecondBrain vault, six-facet taxonomy, and reference the locked decisions (ADR-011, ADR-012). Update 02-system-design.md: align with faceted taxonomy (type/ client/project/domain/tool/convention/ plus scope/), express hierarchy via hubs + wikilinks + Graphify edges rather than nested paths, name the vault explicitly, update build-plan timestamp. Add 05-implementation-process.md: concrete build runbook integrating the locked decisions — seven steps bottom-up from vault migration through plugin packaging, with the Graphify + Ollama model benchmark (Step 2c) marked as the critical gate. Open questions deferred or defaulted; most are non-blocking. Design milestone: tag taxonomy and vault location locked.
2026-06-04 13:35:43 +00:00
- project/website-redesign
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
- domain/seo
Update design docs and add implementation runbook per ADR-011/012 Update CLAUDE.md: clarify the knowledge layer now specifies the ~/Documents/SecondBrain vault, six-facet taxonomy, and reference the locked decisions (ADR-011, ADR-012). Update 02-system-design.md: align with faceted taxonomy (type/ client/project/domain/tool/convention/ plus scope/), express hierarchy via hubs + wikilinks + Graphify edges rather than nested paths, name the vault explicitly, update build-plan timestamp. Add 05-implementation-process.md: concrete build runbook integrating the locked decisions — seven steps bottom-up from vault migration through plugin packaging, with the Graphify + Ollama model benchmark (Step 2c) marked as the critical gate. Open questions deferred or defaulted; most are non-blocking. Design milestone: tag taxonomy and vault location locked.
2026-06-04 13:35:43 +00:00
- tool/semrush
- convention/api-style
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
- scope/project # or scope/global
---
```
Update design docs and add implementation runbook per ADR-011/012 Update CLAUDE.md: clarify the knowledge layer now specifies the ~/Documents/SecondBrain vault, six-facet taxonomy, and reference the locked decisions (ADR-011, ADR-012). Update 02-system-design.md: align with faceted taxonomy (type/ client/project/domain/tool/convention/ plus scope/), express hierarchy via hubs + wikilinks + Graphify edges rather than nested paths, name the vault explicitly, update build-plan timestamp. Add 05-implementation-process.md: concrete build runbook integrating the locked decisions — seven steps bottom-up from vault migration through plugin packaging, with the Graphify + Ollama model benchmark (Step 2c) marked as the critical gate. Open questions deferred or defaulted; most are non-blocking. Design milestone: tag taxonomy and vault location locked.
2026-06-04 13:35:43 +00:00
- **Six flat facets** (ADR-011): `type/`, `client/`, `project/`, `domain/`, `tool/`,
`convention/` — plus `scope/`. Each facet is independent and parallel (never nested into each
other). `#tool` matches all `tool/*` values — native Obsidian prefix filtering, no folders
needed. Hierarchy and relationships are expressed via **hub notes** (`type/hub`),
**wikilinks**, and **Graphify graph edges**, not nested tag paths.
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
- **Two knowledge scopes** via `scope/global` vs `scope/project` (+ a `client/` tag): global =
broadly useful tool/domain knowledge; project = how a specific client uses it. Both are
globally queryable; the scope tag is the shortcut that avoids scanning every client's usage.
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
### Knowledge graph (Graphify)
**Graphify** ([safishamsi/graphify](https://github.com/safishamsi/graphify), command `graphify`)
turns the vault into a queryable **knowledge graph** — the disposable, rebuildable structure
over the markdown. It replaces the earlier Ruby/SQLite tag index *and* the deferred QMD vector
layer (ADR-010): one graph gives both structured and semantic retrieval, without vectors.
- **Extraction**:
- **Vault docs** → a **local Ollama SLM** extracts entities + typed relationships from each
note (confidence-tagged `EXTRACTED` / `INFERRED` / `AMBIGUOUS`). Local model = no API cost,
no data leaving the machine.
- **Project code** → free **tree-sitter AST** (`--no-docs`), no model, no token cost. Kept as
separate per-project graphs, not merged with the vault graph.
- **What it produces**: `graphify-out/` with `graph.json`, an interactive `graph.html`, and a
`GRAPH_REPORT.md` whose top lists the **god nodes** (the most-connected concepts — your
highest-value entry points).
- **Query** (via CLI and an MCP server exposing `query_graph` / `get_node` / `shortest_path`):
ask for **god nodes first**, then scalpel down with `graphify query` / `path` / `explain`.
Prompt the graph; don't dump the corpus into context.
Update design docs and add implementation runbook per ADR-011/012 Update CLAUDE.md: clarify the knowledge layer now specifies the ~/Documents/SecondBrain vault, six-facet taxonomy, and reference the locked decisions (ADR-011, ADR-012). Update 02-system-design.md: align with faceted taxonomy (type/ client/project/domain/tool/convention/ plus scope/), express hierarchy via hubs + wikilinks + Graphify edges rather than nested paths, name the vault explicitly, update build-plan timestamp. Add 05-implementation-process.md: concrete build runbook integrating the locked decisions — seven steps bottom-up from vault migration through plugin packaging, with the Graphify + Ollama model benchmark (Step 2c) marked as the critical gate. Open questions deferred or defaulted; most are non-blocking. Design milestone: tag taxonomy and vault location locked.
2026-06-04 13:35:43 +00:00
- **Metadata still matters**: the `summary` + six facet tags remain first-class note
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
attributes — `summary` is the human-written router hint Graphify does **not** generate, and
Update design docs and add implementation runbook per ADR-011/012 Update CLAUDE.md: clarify the knowledge layer now specifies the ~/Documents/SecondBrain vault, six-facet taxonomy, and reference the locked decisions (ADR-011, ADR-012). Update 02-system-design.md: align with faceted taxonomy (type/ client/project/domain/tool/convention/ plus scope/), express hierarchy via hubs + wikilinks + Graphify edges rather than nested paths, name the vault explicitly, update build-plan timestamp. Add 05-implementation-process.md: concrete build runbook integrating the locked decisions — seven steps bottom-up from vault migration through plugin packaging, with the Graphify + Ollama model benchmark (Step 2c) marked as the critical gate. Open questions deferred or defaulted; most are non-blocking. Design milestone: tag taxonomy and vault location locked.
2026-06-04 13:35:43 +00:00
the facet namespaces (`type/`, `client/`, `project/`, `domain/`, `tool/`, `convention/`,
`scope/`) stay useful for Obsidian filtering and as node attributes. They are retained even
though they no longer back a bespoke index. *(How tightly metadata feeds graph queries is a
refinement for build time.)*
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
- **Source of truth rule**: markdown is authoritative; the graph (`graphify-out/`) is a
rebuildable artifact that is **never synced** and can be deleted/rebuilt anytime
(`graphify ... --force`).
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
### Freshness (lazy — chosen Option A)
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
- **AI writes** → a `PostToolUse` hook on `Write`/`Edit` targeting vault `.md` files runs
`graphify ... --update` to merge the changed note into the vault graph. Event-driven, no
polling.
- **Stale-node caveat**: Graphify's `--update` merges (SHA-256 + dedup) but does **not** prune
deleted notes/symbols — ghost nodes accumulate. A periodic `--force` rebuild clears them,
triggered by the **session-start reconcile** when a rebuild stamp is older than N days
(7 to start). **No daemon, no cron.**
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
### Retrieval (hook-injected + on-demand)
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
- **Session-start hook** injects: (a) a compact overview — the vault graph's **god nodes** as
the map of what's known, (b) the current project's declared `convention/*` notes resolved to
their summaries (so coding conventions auto-pull and a convention edit propagates to every
project using that tag), (c) a pointer to recent episodic journal.
- **On demand**: the AI runs `graphify query` / `path` / `explain` (or the MCP tools) to pull
specific knowledge into context only when the task needs it. Projects stay thin — their
CLAUDE.md holds **tags/pointers**, not content.
## Semantic recall over the vault — covered by Graphify
The earlier design earmarked a separate vector layer (**QMD**) for "when structured tag
filtering misses a note whose wording doesn't match the query." Graphify's knowledge graph
covers that need without a second system or vectors: relationship traversal and `explain`
surface notes by *connection*, not just exact tag match. So there is no separate deferred
semantic layer — if graph traversal ever proves insufficient for a case where embedding
similarity would clearly win, revisit then (the video's "only level up when it bites").
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
## Timeline (Goal 3) details
- A **session-end hook** appends a daily journal note (one file per date) with pointers to the
project/knowledge files touched. memsearch indexes these; today+yesterday are cheap to load,
older entries are reachable by query for drill-down.
## Self-evolution guardrails
- The AI **writes only to the vault**, never silently into project repos.
- **Required frontmatter schema** (summary + namespaced tags) is enforced so the index stays
queryable.
- **Daily notes are append-only**; consolidation/reorg is a **separate, reviewable step run in
plan mode** (Connelly's reorganize + Huryn's propose-and-approve loop).
- **Promotion to `scope/global`** requires a rule (e.g. a fact recurring N times) — not every
stray note gets promoted.
## Sync (Goal 4)
- The **vault** syncs to the VPS via **git** (versioned history, hourly) or **Syncthing**
(continuous, zero-thought). Decision deferred to build time.
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
- **Graphs/indexes are never synced** — the Milvus Lite episodic index and the Graphify
`graphify-out/` graphs are rebuilt per machine. Sync only the markdown.
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
## Packaging
- The whole thing ships as a **global Claude Code plugin with skills** (hooks + scripts +
CRUD know-how) so every project, on every machine, knows how to use the vault effectively.
See [04-build-plan.md](04-build-plan.md).
## How each goal is met
| Goal | Met by |
|------|--------|
| 1. Thin projects | Knowledge in the vault, not repos; CLAUDE.md holds tags/pointers; on-demand `index query` |
Update design docs and add implementation runbook per ADR-011/012 Update CLAUDE.md: clarify the knowledge layer now specifies the ~/Documents/SecondBrain vault, six-facet taxonomy, and reference the locked decisions (ADR-011, ADR-012). Update 02-system-design.md: align with faceted taxonomy (type/ client/project/domain/tool/convention/ plus scope/), express hierarchy via hubs + wikilinks + Graphify edges rather than nested paths, name the vault explicitly, update build-plan timestamp. Add 05-implementation-process.md: concrete build runbook integrating the locked decisions — seven steps bottom-up from vault migration through plugin packaging, with the Graphify + Ollama model benchmark (Step 2c) marked as the critical gate. Open questions deferred or defaulted; most are non-blocking. Design milestone: tag taxonomy and vault location locked.
2026-06-04 13:35:43 +00:00
| 2. Cross-project/client knowledge, global vs project scopes | `~/Documents/SecondBrain` vault + six-facet tags (`type/`/`client/`/`project/`/`domain/`/`tool/`/`convention/`) + `scope/`; Graphify knowledge graph (god nodes + traversal) over it |
Add 3-layer memory system design documentation This documents the complete architecture for a personal, cross-project memory system for Claude Code: three-layer (episodic/knowledge/deferred- semantic) design using Milvus Lite for fast local storage with lazy remote sync. Includes video synthesis, architecture decisions with rejected alternatives (MemPalace, Mem0, Postgres, LightRAG), concrete build plan with scripts and hooks, and initial handoff for implementation.
2026-06-03 20:47:50 +00:00
| 3. Timeline | memsearch episodic layer + session-end journal hook |
Remove handoff doc and pivot to Graphify knowledge layer The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
2026-06-04 11:57:39 +00:00
| 4. Remote, local-fast | Markdown vault synced via git/Syncthing; disposable per-machine graphs/indexes |