cc-os/docs/memory-system/03-architecture-decisions.md

Architecture Decision Records

Last updated: 2026-06-04

A running log of decisions and why. Format per entry: Context · Decision · Rationale · Alternatives rejected · Status. Newest decisions extend the log; supersede rather than delete.

---

ADR-001 — Two memory types, kept as separate systems

• Context: Earlier attempts to make one tool serve both "what happened" and "how do we do X" felt forced (e.g. trying to make memsearch filter knowledge by tags).
• Decision: Model episodic memory and semantic/knowledge memory as two separate systems with different tools.
• Rationale: They have different lifecycles (episodic accretes and decays; knowledge is deliberately maintained), different write paths (auto-captured vs curated with guardrails), and different query patterns ("when did we…" vs "how do we…"). Separation dissolves the earlier integration tension entirely.
• Alternatives rejected: One unified store (memsearch-for-everything, or OpenBrain's single thoughts table) — conflates the two and forces awkward filtering.
• Status: Accepted.

ADR-002 — memsearch for the episodic layer

• Context: Need timeline/"what happened" memory (Goal 3) that's NL-queryable and lazy.
• Decision: Adopt memsearch (Zilliz) off-the-shelf for episodic memory.
• Rationale: It already implements the OpenClaw daily-notes + "dreaming" pattern and the markdown-as-truth / disposable-shadow-index philosophy we'd otherwise build. Embedded Milvus Lite (single file), hybrid BM25+vector+RRF search, local ONNX embeddings (no API key/cost), a FileWatcher that handles deletions — no Docker, no server. Two-line install.
• Alternatives rejected: claude-mem (MCP-based — Claude must actively call search; opaque blobs vs readable markdown; overkill features). Hand-building daily notes + dreaming ourselves (reinventing a solved tool).
• Status: Accepted.

ADR-003 — Flat vault with namespaced tags, not folders

• Context: Connelly/Huryn organize by folders (tools/, domain/). User wants a flat Obsidian vault with tags as virtual indexes, and cross-cutting filters (client × tool × convention).
• Decision: One flat markdown vault; organize via namespaced, nested tags (tool/, client/, domain/, convention/, scope/). Slashes are valid Obsidian nested tags, so #tool matches all children.
• Rationale: A note can carry several namespaces at once (tool/semrush + client/sesame3g + convention/react-ts) — folders can't express that. Enables "filter by client+tool to narrow the index." Enumerable virtual indexes ("what clients/tools exist").
• Alternatives rejected: Folder hierarchy (single-axis; can't do cross-cutting filters). Pure-prefix path filtering via memsearch source_prefix (would force directories back in).
• Trade-off accepted: Tags give the human/Obsidian free filtering, but the AI gets nothing for free from tags — we must materialize them into a queryable index (see ADR-004).
• Status: Refined by ADR-011 (type/ and project/ namespaces added; hierarchy-vs-facets clarified). Core decision — flat vault, namespaced tags — stands.

ADR-004 — SQLite + Sequel (Ruby) tag index as the knowledge-layer cache

• Context: The AI can't use Obsidian tags directly; tag filtering needs a machine-queryable index. A previous ~/Documents/SecondBrain/ tag database was lost track of.
• Decision: A small Ruby program using the Sequel ORM over SQLite, exposed as a CLI. Schema: files(path, mtime, summary, scope), tags(name), files_tags join (many_to_many). The summary is a column on files (an attribute), not a join.
• Rationale: Normalized tags table makes enumerating the vocabulary a first-class cheap query (the "virtual index" goal). The summary column is what turns the index from a finder into a router — the AI sees enough to pick a file without opening it (progressive disclosure, low tokens). Ruby + Sequel + CLI keeps the contract clean and the DB swappable; the AI never touches SQLite directly.
• Failure-mode guard (the lost-SecondBrain lesson): markdown is always authoritative; the SQLite file is a disposable cache that is never synced and can be rebuilt from frontmatter anytime (index update --rebuild).
• Alternatives rejected: Plain-markdown generated INDEX.md (must regenerate; grep-at-scale is token-heavy). Frontmatter grep on demand (scales badly). Milvus/Postgres for knowledge (overkill; QMD/memsearch prove SQLite is enough — see ADR-006/008).
• Query output: returns path + summary + matched tags (option C) — tags are cheap and show why a result matched, useful for cross-client queries.
• Status: Superseded by ADR-010 (Graphify replaces the Ruby/SQLite tag index). The summary + namespaced-tag frontmatter this ADR introduced is retained as note metadata; only the bespoke Ruby/SQLite index and its CLI are dropped.

ADR-005 — Structured-first; semantic search over the vault deferred

• Context: Tag filtering ("client/sesame3g + tool/semrush") may miss notes whose wording doesn't match the query ("how do we use semrush" vs a note titled "search analytics integration").
• Decision: Ship the knowledge layer structured-only (tags + summaries). Defer meaning-based search over the vault until it demonstrably bites.
• Rationale: Structured tagging is the lightweight/fast thing the user wants, and the summary+tag design is built to make it work. Follow the video's "only level up when it bites."
• Status: Superseded by ADR-010. The premise (ship structured-only, bolt on semantics later) no longer holds: Graphify makes the knowledge layer a graph from day one, giving structured and connection-based recall together. The "only level up when it bites" instinct carries forward to whether a vector layer is ever needed on top of the graph.

ADR-006 — QMD as the (deferred) semantic-over-knowledge layer

• Context: When ADR-005's structured-only proves insufficient, we want a set-and-forget semantic layer over the vault, local and Docker-free.
• Decision: Earmark QMD (github.com/tobi/qmd) for that role; do not install yet.
• Rationale: Local markdown search using SQLite + FTS5/BM25 + local vector embeddings (EmbeddingGemma-300M GGUF) + LLM rerank; CLI + optional MCP server; no Docker, no API keys. Validates that SQLite + a local vector model suffices (no Milvus/Postgres for knowledge). Complements the tag index (QMD filters by path/collection context, not first-class frontmatter tags), so it adds semantic recall without replacing structured filtering.
• Alternatives rejected: Pointing memsearch at the vault (mixes episodic and knowledge corpora; its filtering is path-prefix not tags). A bespoke embedding index (reinvents QMD).
• Status: Superseded by ADR-010. Graphify's knowledge graph fills the semantic-recall role (traversal/explain over connections) without a separate vector system, so QMD is no longer earmarked. Revisit a vector layer only if graph traversal demonstrably misses cases where embedding similarity would win.

ADR-007 — Lazy freshness: write-hook + session-start reconcile, no daemon/cron

• Context: The cache must reflect new/edited/deleted/renamed notes without becoming a resource hog or going stale on renames.
• Decision: Option A (lazy). A PostToolUse hook updates the index on AI writes (single-file, prunes on delete). Manual edits are caught by a session-start reconcile (index update --since + prune of vanished paths). No daemon, no cron.
• Rationale: The AI is the primary writer, so write-time hooks give event-driven freshness with no polling. The user rarely edits the vault by hand, so a session-start reconcile is enough; a continuous inotify daemon (the listen gem) would add an always-on process to manage/sync for negligible benefit. Matches the user's "lazy sync is fine" stance.
• Alternatives rejected: inotify/listen daemon (live freshness, but always-on process to manage — unnecessary). Cron reconcile ("seems silly" per user; session-start covers it).
• Status: Accepted.

ADR-008 — Markdown-as-truth; sync the vault, not the indexes

• Context: Must be accessible on a VPS / multiple machines but run local-fast (Goal 4).
• Decision: Sync the markdown vault to the VPS via git or Syncthing (choice deferred to build time). Graphs/indexes (Milvus Lite, Graphify graphify-out/) are rebuilt per machine and never synced.
• Rationale: Markdown is plain text — git/Syncthing sync it trivially; lazy (hourly or continuous-async) is enough. Indexes are disposable caches; syncing binary DBs invites conflicts for no gain. Local reads stay fast; ownership and portability stay with the user.
• Alternatives rejected: OpenBrain / Mem0 hosted DBs — always-remote, adds per-query latency and monthly cost, conflicts with local-fast; ownership weaker (Mem0 especially). Only worth it for real-time cross-tool memory, which the user called overkill.
• Status: Accepted.

ADR-009 — Package as a global Claude Code plugin with skills

• Context: Every project, on every machine, should know how to use the vault — write conventions, query patterns, the hooks, and the CLI — without per-project setup.
• Decision: Ship hooks + scripts + CRUD know-how as a global Claude Code plugin with skills, installed at the user level.
• Rationale: Skills carry the "when to write / what conventions / how & when to query" guidance to the model; the plugin registers the session-start / session-end / PostToolUse hooks and wires up Graphify (extraction/update/query + MCP server). Global install = consistent behavior everywhere; single source of truth for the conventions themselves.
• Status: Accepted (to be built — see 04-build-plan.md).

ADR-010 — Graphify knowledge graph as the knowledge layer (supersedes ADR-004/005/006)

• Context: ADR-004 specced a hand-built Ruby/Sequel/SQLite tag index (+ CLI) as the machine-queryable layer over the vault, with ADR-005/006 deferring meaning-based recall to a future QMD vector layer. Before building any of it, we evaluated Graphify (graphify, PyPI graphifyy) — a tool that turns a folder into a queryable knowledge graph (local tree-sitter AST for code, local-SLM entity/relationship extraction for docs). See 06-graphify-evaluation.md.
• Decision: Use Graphify as the knowledge-layer engine over the vault, with a local Ollama backend for doc extraction and free AST for per-project code graphs. Drop the Ruby/SQLite tag-index CLI (ADR-004) and the earmarked QMD layer (ADR-006); retain the summary + namespaced-tag frontmatter from ADR-003/004 as note metadata.
• Rationale: One off-the-shelf tool delivers both what the tag index was for (structured retrieval) and what QMD was deferred for (connection/meaning-based recall via graph traversal + explain) — without writing or maintaining a bespoke index, and without a vector store. Code graphs come free. Keeps the markdown-as-truth, no-Docker, no-API-key, local-first properties (extraction runs against local Ollama). Net scope reduction: the entire Ruby build (old critical-path Step 2) and the QMD layer are removed.
• What's retained / changed: summary stays the human-written router hint Graphify does not generate; namespaced tags stay useful for Obsidian filtering and as node attributes. How tightly metadata should feed graph queries is a build-time refinement, not settled here.
• Trade-off accepted: Graphify's --update doesn't prune deleted nodes (stale-node drift) — mitigated by a periodic --force rebuild on the session-start staleness check (ADR-007's lazy model still applies). Graphify also moves fast (flags are version-dependent; anchored to v0.8.30) and its headline token-savings numbers are corpus-dependent — benchmark our own.
• Alternatives rejected: Building the Ruby/SQLite index as originally planned (more code to own; no semantic recall); adding QMD as a second system on top (two stores where one graph suffices).
• Status: Accepted (to be built — see 04-build-plan.md and 06-graphify-evaluation.md).

ADR-011 — Faceted tag taxonomy: six independent namespaces (refines ADR-003)

Date: 2026-06-04

• Context: ADR-003 introduced five namespaces (tool/, client/, domain/, convention/, scope/). During vault-reuse assessment (ADR-012) it became clear that (1) the existing SecondBrain vault uses a de-facto first-tag convention for note kind (research/plan/log/adr/howto) that should be made explicit and machine-queryable, and (2) for a freelancer working many projects per client, project identity deserves a first-class namespace rather than being implied by client/ or domain/.

• Decision: Knowledge-vault notes are classified by six independent, flat tag facets that sit side-by-side, never nested into one another:

  • type/ — note kind: research, howto, adr, hub, plan, log, clip, etc.
  • client/ — which client
  • project/ — which project (first-class; a freelancer's projects are the primary unit of work)
  • domain/ — knowledge domain / topic area
  • tool/ — tool-specific knowledge
  • convention/ — conventions
  • …plus scope/global or scope/project (retained from ADR-003)

  Hierarchy and relationships are expressed via hub notes (type/hub), wikilinks, and Graphify knowledge-graph edges — NOT via nested tag paths.

  By convention type/ is listed first in frontmatter, preserving the SecondBrain vault's existing type-first ordering habit and making the note kind immediately visible.

• Rationale: The vault is flat — hierarchy is not expressed through folder paths or tag nesting. The user's reality is many-to-many (many projects per client, knowledge domains spanning clients), which a single-parent tree models badly and forces false hierarchy. A project hub note links out to both its client/ and relevant domain/ tags rather than being buried under either. Per-type _templates will be provided for core types only (research, howto, adr, hub); the long tail stays freeform until a pattern earns a template. Consistent per-type structure also improves Graphify's local-SLM extraction reliability.

• Alternatives rejected: Hierarchical nesting in the style of John Conneely's domain/{product}/{project}.md folder structure (from the youngleaders.tech article "How I finally sorted my Claude Code memory" — secondary/interview-grade source, not verified against primary implementation). Rejected because: (1) the vault is flat — hierarchy is not expressed through folder paths; (2) the user's many-to-many reality maps badly onto a single-parent folder tree and forces false hierarchy; (3) nesting one facet through another (e.g. domain/client/project) creates Law-of-Demeter-style traversal coupling. Conneely's structure was the inspiration but diverges here on hierarchy-vs-facets. Faceted parallel tags are the flat-vault analogue of what the Graphify graph already does with edges, so they compose naturally with the chosen knowledge layer.

• Status: Accepted (supersedes the namespace list in ADR-003; core flat-vault + namespaced-tags decision stands).

ADR-012 — Reuse the existing SecondBrain vault as the knowledge vault

Date: 2026-06-04

• Context: The design called for a flat markdown vault as the semantic knowledge layer (ADR-003/008/010). The question was whether to stand up a new ~/brain vault from scratch or adopt the existing ~/Documents/SecondBrain vault.
• Decision: Adopt ~/Documents/SecondBrain as the knowledge vault rather than creating a new vault.
• Rationale: Assessment found the SecondBrain vault is already flat (all notes at root, only a _templates/ exception — exactly what the design permits), already articulates the correct "durable knowledge, not working memory" role in its CLAUDE.md and vault-conventions.md, and contains ~20 real notes. It also includes two patterns that improve on the current design and should be adopted:
  1. vault-conventions.md's "act without being asked" section specifying when the AI should proactively query the vault — a behavioral spec the cc-os docs lacked.
  2. Project-config hub notes with a tag-inference table (auto-tag by path pattern) that operationalizes how to tag a note from a given project.
• Adaptations required (migration cost):
  • Add summary: frontmatter to existing notes.
  • Migrate flat unnamespaced tags to the six-facet namespaced form (per ADR-011).
  • Add scope/global or scope/project to each note.
  • Initialize git in the vault (no .git exists yet — required by ADR-008's sync strategy).
  • Replace the vault's ~/.claude/scripts/vault_search.rb reference (script does not exist) with graphify query (ADR-010). These are mechanical schema migrations, not structural rework.
• Alternatives rejected: Starting fresh with a new ~/brain vault. Rejected because the hardest design decision — flat structure, durable-knowledge-only role, governance philosophy — is already made and practiced in SecondBrain. The improved behavioral patterns (proactive-query spec, tag-inference table) and the existing notes are worth preserving; the remaining work is mechanical migration.
• Status: Accepted.

ADR-013 — Build-first / migrate-incrementally (build-order inversion)

Date: 2026-06-04

• Context: The build runbook (05-implementation-process.md) originally front-loaded bulk vault migration as Step 1 — migrating all ~20 existing SecondBrain notes and all projects to the ADR-011 six-facet taxonomy before the system existed to validate them. This committed to a schema and workflow (the tag taxonomy from ADR-011, the vault-reuse choice from ADR-012, and Graphify extraction behavior) before any end-to-end path had been exercised. The risk: locking in an approach that fails at scale, with no feedback loop until the entire vault has been touched.
• Decision: Invert the build order. The full system is built and validated against a small 5–10 note fixture set first. Bulk vault migration is deferred to the final stage. The first real-data validation uses one small project that contains both code AND documents, exercising both the local-SLM doc-extraction path and the tree-sitter code path in the same run. After that single project validates end-to-end, remaining projects are onboarded one at a time with an observe-and-adjust step between each.
• Rationale: Validates the ADR-011 taxonomy and ADR-012 vault conventions against the real Graphify extraction pipeline before the entire vault is committed. The first mixed code+docs project surfaces both extraction paths (SLM for docs, tree-sitter for code) early, when corrections are cheap. Per-project rollout keeps the blast radius of any schema or workflow correction small; each project is an opportunity to observe and adjust rather than discover problems across 20 notes at once. This is consistent with the "markdown-as-truth, indexes are disposable" principle (ADR-008): the vault notes are durable, but the extraction schema should be validated before it shapes all of them.
• Alternatives rejected:
  • Keep migration-first (status quo): Front-loads all ~20 notes and all projects before any end-to-end validation exists. Commits to ADR-011's taxonomy and ADR-012's migration steps against the full vault without a feedback loop — exactly the gap this decision closes.
  • Big-bang migrate everything after build: Build against fixtures, then migrate all notes and all projects in one batch at the end. Avoids the pre-build commitment problem but still risks a single large irreversible migration with no observe-and-adjust loop between units. Per-project rollout with intermediate checkpoints is strictly safer.
• Cross-references: ADR-011 (six-facet tag taxonomy — the schema being validated); ADR-012 (SecondBrain vault reuse — the migration steps this order defers).
• Status: Accepted (updates 05-implementation-process.md build order).

Rejected tools (summary)

Tool	Why rejected for our use
MemPalace (L4)	Storage not readable markdown; isolated drawers (knowledge not interconnected); fights self-managing + cross-linking goals
Recall / LightRAG (L5)	Content knowledge bases / deep research, not operational memory; Recall = hosted, you don't own data; LightRAG = enterprise overkill
OpenBrain / Mem0 (L6)	Always-remote DB → latency + cost; conflicts with local-fast lazy-sync; only pays off for real-time cross-tool memory (user: overkill)
Postgres / Milvus server	Unnecessary — Graphify's local graph (knowledge) + Milvus Lite (memsearch episodic) cover everything locally with no Docker
claude-mem	MCP-based (Claude must call search); opaque blobs vs readable markdown; feature overkill
Ruby/SQLite tag index CLI; QMD vector layer	Superseded by Graphify before build — one knowledge graph replaces both the structured index and the deferred semantic layer (ADR-010)