SecondBrain/2026-06-09-claude-code-memo...

21 KiB
Raw Permalink Blame History

summary tags source date
Architecture Decision Records for the Claude Code personal memory system — 14 decisions covering memory-type split, tooling choices, vault structure, freshness model, and graph connectivity.
type/adr
domain/knowledge-graphs
domain/llm
tool/graphify
scope/global
cc-os 2026-06-09

Claude Code Memory System — Architecture Decision Records

A running log of decisions and why. Format per entry: Context · Decision · Rationale · Alternatives rejected · Status. Newest decisions extend the log; supersede rather than delete.


ADR-001 — Two memory types, kept as separate systems

  • Context: Earlier attempts to make one tool serve both "what happened" and "how do we do X" felt forced (e.g. trying to make memsearch filter knowledge by tags).
  • Decision: Model episodic memory and semantic/knowledge memory as two separate systems with different tools.
  • Rationale: They have different lifecycles (episodic accretes and decays; knowledge is deliberately maintained), different write paths (auto-captured vs curated with guardrails), and different query patterns ("when did we…" vs "how do we…"). Separation dissolves the earlier integration tension entirely.
  • Alternatives rejected: One unified store (memsearch-for-everything, or a single thoughts table) — conflates the two and forces awkward filtering.
  • Status: Accepted.

ADR-002 — memsearch for the episodic layer

  • Context: Need timeline/"what happened" memory that's NL-queryable and lazy.
  • Decision: Adopt memsearch (Zilliz) off-the-shelf for episodic memory.
  • Rationale: It already implements the daily-notes + "dreaming" pattern and the markdown-as-truth / disposable-shadow-index philosophy. Embedded Milvus Lite (single file), hybrid BM25+vector+RRF search, local ONNX embeddings (no API key/cost), a FileWatcher that handles deletions — no Docker, no server. Two-line install.
  • Alternatives rejected: claude-mem (MCP-based — Claude must actively call search; opaque blobs vs readable markdown; overkill features). Hand-building daily notes + dreaming ourselves (reinventing a solved tool).
  • Status: Accepted.

ADR-003 — Flat vault with namespaced tags, not folders

  • Context: Want a flat Obsidian vault with tags as virtual indexes, and cross-cutting filters (client × tool × convention).
  • Decision: One flat markdown vault; organize via namespaced, nested tags (tool/, client/, domain/, convention/, scope/). Slashes are valid Obsidian nested tags, so #tool matches all children.
  • Rationale: A note can carry several namespaces at once (tool/semrush + client/clientx + convention/react-ts) — folders can't express that. Enables "filter by client+tool to narrow the index." Enumerable virtual indexes ("what clients/tools exist").
  • Alternatives rejected: Folder hierarchy (single-axis; can't do cross-cutting filters). Pure-prefix path filtering via memsearch source_prefix (would force directories back in).
  • Trade-off accepted: Tags give the human/Obsidian free filtering, but the AI gets nothing for free from tags — they must be materialized into a queryable index.
  • Status: Refined by ADR-011 (type/ and project/ namespaces added; hierarchy-vs-facets clarified). Core decision — flat vault, namespaced tags — stands.

ADR-004 — SQLite + Sequel (Ruby) tag index as the knowledge-layer cache

  • Context: The AI can't use Obsidian tags directly; tag filtering needs a machine-queryable index.
  • Decision: A small Ruby program using the Sequel ORM over SQLite, exposed as a CLI. Schema: files(path, mtime, summary, scope), tags(name), files_tags join (many_to_many). The summary is a column on files (an attribute), not a join.
  • Rationale: Normalized tags table makes enumerating the vocabulary a first-class cheap query. The summary column is what turns the index from a finder into a router — the AI sees enough to pick a file without opening it (progressive disclosure, low tokens). Failure-mode guard: markdown is always authoritative; the SQLite file is a disposable cache that is never synced and can be rebuilt from frontmatter anytime.
  • Alternatives rejected: Plain-markdown generated INDEX.md (must regenerate; grep-at-scale is token-heavy). Frontmatter grep on demand (scales badly). Milvus/Postgres for knowledge (overkill).
  • Query output: returns path + summary + matched tags — tags are cheap and show why a result matched.
  • Status: Superseded by ADR-010 (Graphify replaces the Ruby/SQLite tag index). The summary + namespaced-tag frontmatter is retained as note metadata; only the bespoke Ruby/SQLite index and CLI are dropped.

ADR-005 — Structured-first; semantic search over the vault deferred

  • Context: Tag filtering may miss notes whose wording doesn't match the query.
  • Decision: Ship the knowledge layer structured-only (tags + summaries). Defer meaning-based search until it demonstrably bites.
  • Rationale: Structured tagging is the lightweight/fast thing; follow the "only level up when it bites" principle.
  • Status: Superseded by ADR-010. Graphify makes the knowledge layer a graph from day one, giving structured and connection-based recall together. The "only level up when it bites" instinct carries forward to whether a vector layer is ever needed on top of the graph.

ADR-006 — QMD as the (deferred) semantic-over-knowledge layer

  • Context: When ADR-005's structured-only proves insufficient, want a set-and-forget semantic layer over the vault, local and Docker-free.
  • Decision: Earmark QMD (github.com/tobi/qmd) for that role; do not install yet.
  • Rationale: Local markdown search using SQLite + FTS5/BM25 + local vector embeddings (EmbeddingGemma-300M GGUF) + LLM rerank; CLI + optional MCP server; no Docker, no API keys. Complements the tag index (QMD filters by path/collection context, not first-class frontmatter tags).
  • Alternatives rejected: Pointing memsearch at the vault (mixes episodic and knowledge corpora; its filtering is path-prefix not tags). A bespoke embedding index (reinvents QMD).
  • Status: Superseded by ADR-010. Graphify's knowledge graph fills the semantic-recall role (traversal/explain over connections) without a separate vector system, so QMD is no longer earmarked. Revisit a vector layer only if graph traversal demonstrably misses cases where embedding similarity would win.

ADR-007 — Lazy freshness: write-hook + session-start reconcile, no daemon/cron

  • Context: The cache must reflect new/edited/deleted/renamed notes without becoming a resource hog or going stale.
  • Decision: Option A (lazy). A PostToolUse hook updates the index on AI writes (single-file, prunes on delete). Manual edits are caught by a session-start reconcile (index update --since + prune of vanished paths). No daemon, no cron.
  • Rationale: The AI is the primary writer, so write-time hooks give event-driven freshness with no polling. The user rarely edits the vault by hand, so a session-start reconcile is enough; a continuous inotify daemon would add an always-on process to manage for negligible benefit.
  • Alternatives rejected: inotify/listen daemon (live freshness, but always-on process to manage — unnecessary). Cron reconcile (session-start covers it).
  • Status: Accepted.

ADR-008 — Markdown-as-truth; sync the vault, not the indexes

  • Context: Must be accessible on a VPS / multiple machines but run local-fast.
  • Decision: Sync the markdown vault to a remote machine via git or Syncthing (choice deferred to build time). Graphs/indexes (Milvus Lite, Graphify graphify-out/) are rebuilt per machine and never synced.
  • Rationale: Markdown is plain text — git/Syncthing sync it trivially; lazy (hourly or continuous-async) is enough. Indexes are disposable caches; syncing binary DBs invites conflicts for no gain. Local reads stay fast; ownership and portability stay with the user.
  • Alternatives rejected: Hosted DBs (always-remote, adds per-query latency and monthly cost, conflicts with local-fast; ownership weaker). Only worth it for real-time cross-tool memory, which was deemed overkill.
  • Status: Accepted.

ADR-009 — Package as a global Claude Code plugin with skills

  • Context: Every project, on every machine, should know how to use the vault — write conventions, query patterns, the hooks, and the CLI — without per-project setup.
  • Decision: Ship hooks + scripts + CRUD know-how as a global Claude Code plugin with skills, installed at the user level.
  • Rationale: Skills carry the "when to write / what conventions / how & when to query" guidance to the model; the plugin registers the session-start / session-end / PostToolUse hooks and wires up Graphify. Global install = consistent behavior everywhere; single source of truth for the conventions themselves.
  • Status: Accepted.

ADR-010 — Graphify knowledge graph as the knowledge layer (supersedes ADR-004/005/006)

  • Context: ADR-004 specced a hand-built Ruby/Sequel/SQLite tag index (+ CLI) as the machine-queryable layer over the vault, with ADR-005/006 deferring meaning-based recall to a future QMD vector layer. Before building any of it, Graphify (graphify, PyPI graphifyy) was evaluated — a tool that turns a folder into a queryable knowledge graph (local tree-sitter AST for code, local-SLM entity/relationship extraction for docs).
  • Decision: Use Graphify as the knowledge-layer engine over the vault, with a local Ollama backend for doc extraction and free AST for per-project code graphs. Drop the Ruby/SQLite tag-index CLI (ADR-004) and the earmarked QMD layer (ADR-006); retain the summary + namespaced-tag frontmatter from ADR-003/004 as note metadata.
  • Rationale: One off-the-shelf tool delivers both what the tag index was for (structured retrieval) and what QMD was deferred for (connection/meaning-based recall via graph traversal + explain) — without writing or maintaining a bespoke index, and without a vector store. Code graphs come free. Keeps the markdown-as-truth, no-Docker, no-API-key, local-first properties.
  • What's retained / changed: summary stays the human-written router hint Graphify does not generate; namespaced tags stay useful for Obsidian filtering and as node attributes. How tightly metadata feeds graph queries is a build-time refinement, not settled here.
  • Trade-off accepted: Graphify's --update doesn't prune deleted nodes (stale-node drift) — mitigated by a periodic --force rebuild on the session-start staleness check (ADR-007's lazy model still applies). Graphify also moves fast (flags are version-dependent; anchor to the installed version) and its headline token-savings numbers are corpus-dependent — benchmark your own.
  • Alternatives rejected: Building the Ruby/SQLite index as originally planned (more code to own; no semantic recall); adding QMD as a second system on top (two stores where one graph suffices).
  • Status: Accepted.

ADR-011 — Faceted tag taxonomy: six independent namespaces (refines ADR-003)

Date: 2026-06-04

  • Context: ADR-003 introduced five namespaces (tool/, client/, domain/, convention/, scope/). During vault-reuse assessment it became clear that (1) the existing vault uses a de-facto first-tag convention for note kind that should be made explicit and machine-queryable, and (2) for a freelancer working many projects per client, project identity deserves a first-class namespace.

  • Decision: Knowledge-vault notes are classified by six independent, flat tag facets that sit side-by-side, never nested into one another:

    • type/ — note kind: research, howto, adr, hub, plan, log, clip, etc.
    • client/ — which client
    • project/ — which project (first-class; a freelancer's projects are the primary unit of work)
    • domain/ — knowledge domain / topic area
    • tool/ — tool-specific knowledge
    • convention/ — conventions
    • …plus scope/global or scope/project

    Hierarchy and relationships are expressed via hub notes (type/hub), wikilinks, and Graphify knowledge-graph edges — NOT via nested tag paths.

    By convention type/ is listed first in frontmatter.

  • Rationale: The vault is flat — hierarchy is not expressed through folder paths or tag nesting. A many-to-many reality (many projects per client, knowledge domains spanning clients) maps badly onto a single-parent tree. A project hub note links out to both its client/ and relevant domain/ tags rather than being buried under either. Per-type _templates for core types only (research, howto, adr, hub); the long tail stays freeform. Consistent per-type structure also improves Graphify's local-SLM extraction reliability.

  • Alternatives rejected: Hierarchical nesting in the style of domain/{product}/{project}.md folder structure. Rejected because: (1) the vault is flat; (2) the many-to-many reality maps badly onto a single-parent folder tree; (3) nesting one facet through another creates traversal coupling. Faceted parallel tags are the flat-vault analogue of what the Graphify graph already does with edges, so they compose naturally with the knowledge layer.

  • Status: Accepted (supersedes the namespace list in ADR-003; core flat-vault + namespaced-tags decision stands).

ADR-012 — Reuse an existing vault rather than creating a new one

Date: 2026-06-04

  • Context: The design called for a flat markdown vault as the semantic knowledge layer. The question was whether to stand up a new vault from scratch or adopt an existing one.
  • Decision: Adopt an existing vault rather than creating a new vault.
  • Rationale: An existing vault that is already flat (all notes at root, only a _templates/ exception), already articulates the correct "durable knowledge, not working memory" role, and contains real notes is worth preserving. Two patterns the existing vault may already include that improve on a bare-spec design:
    1. An "act without being asked" section specifying when the AI should proactively query the vault — a behavioral spec.
    2. Project-config hub notes with a tag-inference table (auto-tag by path pattern) that operationalizes how to tag a note from a given project.
  • Adaptations typically required (migration cost):
    • Add summary: frontmatter to existing notes.
    • Migrate flat unnamespaced tags to the six-facet namespaced form (per ADR-011).
    • Add scope/global or scope/project to each note.
    • Initialize git in the vault (required by ADR-008's sync strategy).
    • Replace any legacy vault-search script references with graphify query (ADR-010). These are mechanical schema migrations, not structural rework.
  • Alternatives rejected: Starting fresh. Rejected because the hardest design decision — flat structure, durable-knowledge-only role, governance philosophy — is already made and practiced in an existing vault. The improved behavioral patterns and the existing notes are worth preserving.
  • Status: Accepted.

ADR-013 — Build-first / migrate-incrementally (build-order inversion)

Date: 2026-06-04

  • Context: The original build order front-loaded bulk vault migration as Step 1 — migrating all existing notes and all projects to the ADR-011 six-facet taxonomy before the system existed to validate them. This committed to a schema and workflow before any end-to-end path had been exercised. The risk: locking in an approach that fails at scale, with no feedback loop until the entire vault has been touched.
  • Decision: Invert the build order. The full system is built and validated against a small 510 note fixture set first. Bulk vault migration is deferred to the final stage. The first real-data validation uses one small project that contains both code AND documents, exercising both the local-SLM doc-extraction path and the tree-sitter code path in the same run. After that single project validates end-to-end, remaining projects are onboarded one at a time with an observe-and-adjust step between each.
  • Rationale: Validates the ADR-011 taxonomy and vault conventions against the real Graphify extraction pipeline before the entire vault is committed. The first mixed code+docs project surfaces both extraction paths (SLM for docs, tree-sitter for code) early, when corrections are cheap. Per-project rollout keeps the blast radius of any schema or workflow correction small.
  • Alternatives rejected:
    • Keep migration-first: Front-loads all notes and all projects before any end-to-end validation exists.
    • Big-bang migrate everything after build: Build against fixtures, then migrate all notes and all projects in one batch at the end. Per-project rollout with intermediate checkpoints is strictly safer.
  • Status: Accepted.

ADR-014 — Graph connectivity comes from authored structure; migration scaffolding is a first-class prerequisite

Date: 2026-06-05

  • Context: ADR-011 specified hub notes + wikilinks + Graphify graph edges as the mechanism for expressing hierarchy and cross-note relationships, with ADR-013 deferring bulk vault migration. Before build began, an empirical test compared a cached-replay graph (per-fixture isolated extractions) against a clean single-pass deep extraction against a real vault under Graphify 0.8.31 + qwen2.5-coder:7b. [primary/measured — 2026-06-05]
  • Decision: The connective spine of the knowledge graph must be author-provided. Hub notes and wikilinks are not optional scaffolding to add "someday" — they are the mechanism by which Graphify connects thematically related notes, and they must be authored as part of the migration step, not deferred to bulk import. Migration scaffolding (hub notes + wikilinks for key concepts) is a first-class build deliverable.
  • Rationale: The empirical test found that Graphify is a structure extractor, not a topic clusterer. Even at --mode deep --token-budget 8000, no emergent shared-topic hub nodes appeared. All cross-note edges observed came from explicit references, wikilinks, or document-level semantic similarity — not from shared thematic identity. A practical test query ("how do we do niche prospecting outreach for pest control?") returned 3 starting notes and traversal could not reach related notes in separate communities (no connecting edges). This confirms that useful retrieval is gated on migration scaffolding, not on Graphify's extraction power.
  • Relationship to ADR-011: Validates the hub-notes + wikilinks half of ADR-011 empirically. The facet-tag half is not yet validated: no edge was observed to arise from shared frontmatter facet tags alone. Whether client/X or tool/Y tags create graph connectivity is an open question — do not assume facet tags contribute to graph traversal retrieval until tested.
  • Relationship to ADR-013: Refines the migrate-incrementally stage. "Migration" must include hub note authoring and wikilink addition for key concepts, not just frontmatter schema migration.
  • Alternatives rejected: Relying on the SLM to auto-cluster topics and synthesize hub entities — empirically does not happen at 7B model size with --mode deep. The design already intended human-authored hub notes; the test confirms that intent was correct and the fallback assumption ("maybe the LLM will do it") is false.
  • Deferred:
    1. Facet-tag-to-graph-edge question: Do shared frontmatter facet tags (client/, tool/, domain/, etc.) cause Graphify to create edges between notes, or does graph connectivity come only from explicit wikilinks/references and semantic similarity? This was NOT tested. Resolve before designing graph-traversal retrieval skills.
    2. Larger extraction model: Whether a substantially larger SLM (14B, 30B) would synthesize emergent topic-hub nodes is untested.
    3. reasoning_effort:"none" patch: The clean run required a local patch to graphify/llm.py. Track the upstream Graphify issue tracker for an official fix; treat the installed version as pinned until resolved.
  • Status: Accepted. Refines ADR-013 (migrate-incrementally phase scope) and empirically validates the hub-notes/wikilinks mechanism of ADR-011 while flagging its facet-tag half as an open question.

Rejected tools (summary)

Tool Why rejected
MemPalace Storage not readable markdown; isolated drawers (knowledge not interconnected); fights self-managing + cross-linking goals
Recall / LightRAG Content knowledge bases / deep research, not operational memory; Recall = hosted, you don't own data; LightRAG = enterprise overkill
OpenBrain / Mem0 Always-remote DB → latency + cost; conflicts with local-fast lazy-sync; only pays off for real-time cross-tool memory (overkill)
Postgres / Milvus server Unnecessary — Graphify's local graph (knowledge) + Milvus Lite (memsearch episodic) cover everything locally with no Docker
claude-mem MCP-based (Claude must call search); opaque blobs vs readable markdown; feature overkill
Ruby/SQLite tag index CLI; QMD vector layer Superseded by Graphify before build — one knowledge graph replaces both the structured index and the deferred semantic layer (ADR-010)