cc-os/docs/memory-system/03-architecture-decisions.md

Architecture Decision Records

Last updated: 2026-06-05

A running log of decisions and why. Format per entry: Context · Decision · Rationale · Alternatives rejected · Status. Newest decisions extend the log; supersede rather than delete.

---

ADR-001 — Two memory types, kept as separate systems

• Context: Earlier attempts to make one tool serve both "what happened" and "how do we do X" felt forced (e.g. trying to make memsearch filter knowledge by tags).
• Decision: Model episodic memory and semantic/knowledge memory as two separate systems with different tools.
• Rationale: They have different lifecycles (episodic accretes and decays; knowledge is deliberately maintained), different write paths (auto-captured vs curated with guardrails), and different query patterns ("when did we…" vs "how do we…"). Separation dissolves the earlier integration tension entirely.
• Alternatives rejected: One unified store (memsearch-for-everything, or OpenBrain's single thoughts table) — conflates the two and forces awkward filtering.
• Status: Accepted.

ADR-002 — memsearch for the episodic layer

• Context: Need timeline/"what happened" memory (Goal 3) that's NL-queryable and lazy.
• Decision: Adopt memsearch (Zilliz) off-the-shelf for episodic memory.
• Rationale: It already implements the OpenClaw daily-notes + "dreaming" pattern and the markdown-as-truth / disposable-shadow-index philosophy we'd otherwise build. Embedded Milvus Lite (single file), hybrid BM25+vector+RRF search, local ONNX embeddings (no API key/cost), a FileWatcher that handles deletions — no Docker, no server. Two-line install.
• Alternatives rejected: claude-mem (MCP-based — Claude must actively call search; opaque blobs vs readable markdown; overkill features). Hand-building daily notes + dreaming ourselves (reinventing a solved tool).
• Status: Accepted.

ADR-003 — Flat vault with namespaced tags, not folders

• Context: Connelly/Huryn organize by folders (tools/, domain/). User wants a flat Obsidian vault with tags as virtual indexes, and cross-cutting filters (client × tool × convention).
• Decision: One flat markdown vault; organize via namespaced, nested tags (tool/, client/, domain/, convention/, scope/). Slashes are valid Obsidian nested tags, so #tool matches all children.
• Rationale: A note can carry several namespaces at once (tool/semrush + client/sesame3g + convention/react-ts) — folders can't express that. Enables "filter by client+tool to narrow the index." Enumerable virtual indexes ("what clients/tools exist").
• Alternatives rejected: Folder hierarchy (single-axis; can't do cross-cutting filters). Pure-prefix path filtering via memsearch source_prefix (would force directories back in).
• Trade-off accepted: Tags give the human/Obsidian free filtering, but the AI gets nothing for free from tags — we must materialize them into a queryable index (see ADR-004).
• Status: Refined by ADR-011 (type/ and project/ namespaces added; hierarchy-vs-facets clarified). Core decision — flat vault, namespaced tags — stands.

ADR-004 — SQLite + Sequel (Ruby) tag index as the knowledge-layer cache

• Context: The AI can't use Obsidian tags directly; tag filtering needs a machine-queryable index. A previous ~/Documents/SecondBrain/ tag database was lost track of.
• Decision: A small Ruby program using the Sequel ORM over SQLite, exposed as a CLI. Schema: files(path, mtime, summary, scope), tags(name), files_tags join (many_to_many). The summary is a column on files (an attribute), not a join.
• Rationale: Normalized tags table makes enumerating the vocabulary a first-class cheap query (the "virtual index" goal). The summary column is what turns the index from a finder into a router — the AI sees enough to pick a file without opening it (progressive disclosure, low tokens). Ruby + Sequel + CLI keeps the contract clean and the DB swappable; the AI never touches SQLite directly.
• Failure-mode guard (the lost-SecondBrain lesson): markdown is always authoritative; the SQLite file is a disposable cache that is never synced and can be rebuilt from frontmatter anytime (index update --rebuild).
• Alternatives rejected: Plain-markdown generated INDEX.md (must regenerate; grep-at-scale is token-heavy). Frontmatter grep on demand (scales badly). Milvus/Postgres for knowledge (overkill; QMD/memsearch prove SQLite is enough — see ADR-006/008).
• Query output: returns path + summary + matched tags (option C) — tags are cheap and show why a result matched, useful for cross-client queries.
• Status: Superseded by ADR-010 (Graphify replaces the Ruby/SQLite tag index). The summary + namespaced-tag frontmatter this ADR introduced is retained as note metadata; only the bespoke Ruby/SQLite index and its CLI are dropped.

ADR-005 — Structured-first; semantic search over the vault deferred

• Context: Tag filtering ("client/sesame3g + tool/semrush") may miss notes whose wording doesn't match the query ("how do we use semrush" vs a note titled "search analytics integration").
• Decision: Ship the knowledge layer structured-only (tags + summaries). Defer meaning-based search over the vault until it demonstrably bites.
• Rationale: Structured tagging is the lightweight/fast thing the user wants, and the summary+tag design is built to make it work. Follow the video's "only level up when it bites."
• Status: Superseded by ADR-010. The premise (ship structured-only, bolt on semantics later) no longer holds: Graphify makes the knowledge layer a graph from day one, giving structured and connection-based recall together. The "only level up when it bites" instinct carries forward to whether a vector layer is ever needed on top of the graph.

ADR-006 — QMD as the (deferred) semantic-over-knowledge layer

• Context: When ADR-005's structured-only proves insufficient, we want a set-and-forget semantic layer over the vault, local and Docker-free.
• Decision: Earmark QMD (github.com/tobi/qmd) for that role; do not install yet.
• Rationale: Local markdown search using SQLite + FTS5/BM25 + local vector embeddings (EmbeddingGemma-300M GGUF) + LLM rerank; CLI + optional MCP server; no Docker, no API keys. Validates that SQLite + a local vector model suffices (no Milvus/Postgres for knowledge). Complements the tag index (QMD filters by path/collection context, not first-class frontmatter tags), so it adds semantic recall without replacing structured filtering.
• Alternatives rejected: Pointing memsearch at the vault (mixes episodic and knowledge corpora; its filtering is path-prefix not tags). A bespoke embedding index (reinvents QMD).
• Status: Superseded by ADR-010. Graphify's knowledge graph fills the semantic-recall role (traversal/explain over connections) without a separate vector system, so QMD is no longer earmarked. Revisit a vector layer only if graph traversal demonstrably misses cases where embedding similarity would win.

ADR-007 — Lazy freshness: write-hook + session-start reconcile, no daemon/cron

• Context: The cache must reflect new/edited/deleted/renamed notes without becoming a resource hog or going stale on renames.
• Decision: Option A (lazy). A PostToolUse hook updates the index on AI writes (single-file, prunes on delete). Manual edits are caught by a session-start reconcile (index update --since + prune of vanished paths). No daemon, no cron.
• Rationale: The AI is the primary writer, so write-time hooks give event-driven freshness with no polling. The user rarely edits the vault by hand, so a session-start reconcile is enough; a continuous inotify daemon (the listen gem) would add an always-on process to manage/sync for negligible benefit. Matches the user's "lazy sync is fine" stance.
• Alternatives rejected: inotify/listen daemon (live freshness, but always-on process to manage — unnecessary). Cron reconcile ("seems silly" per user; session-start covers it).
• Status: Accepted.

ADR-008 — Markdown-as-truth; sync the vault, not the indexes

• Context: Must be accessible on a VPS / multiple machines but run local-fast (Goal 4).
• Decision: Sync the markdown vault to the VPS via git or Syncthing (choice deferred to build time). Graphs/indexes (Milvus Lite, Graphify graphify-out/) are rebuilt per machine and never synced.
• Rationale: Markdown is plain text — git/Syncthing sync it trivially; lazy (hourly or continuous-async) is enough. Indexes are disposable caches; syncing binary DBs invites conflicts for no gain. Local reads stay fast; ownership and portability stay with the user.
• Alternatives rejected: OpenBrain / Mem0 hosted DBs — always-remote, adds per-query latency and monthly cost, conflicts with local-fast; ownership weaker (Mem0 especially). Only worth it for real-time cross-tool memory, which the user called overkill.
• Status: Accepted.

ADR-009 — Package as a global Claude Code plugin with skills

• Context: Every project, on every machine, should know how to use the vault — write conventions, query patterns, the hooks, and the CLI — without per-project setup.
• Decision: Ship hooks + scripts + CRUD know-how as a global Claude Code plugin with skills, installed at the user level.
• Rationale: Skills carry the "when to write / what conventions / how & when to query" guidance to the model; the plugin registers the session-start / session-end / PostToolUse hooks and wires up Graphify (extraction/update/query + MCP server). Global install = consistent behavior everywhere; single source of truth for the conventions themselves.
• Status: Accepted (to be built — see 04-build-plan.md).

ADR-010 — Graphify knowledge graph as the knowledge layer (supersedes ADR-004/005/006)

• Context: ADR-004 specced a hand-built Ruby/Sequel/SQLite tag index (+ CLI) as the machine-queryable layer over the vault, with ADR-005/006 deferring meaning-based recall to a future QMD vector layer. Before building any of it, we evaluated Graphify (graphify, PyPI graphifyy) — a tool that turns a folder into a queryable knowledge graph (local tree-sitter AST for code, local-SLM entity/relationship extraction for docs). See 06-graphify-evaluation.md.
• Decision: Use Graphify as the knowledge-layer engine over the vault, with a local Ollama backend for doc extraction and free AST for per-project code graphs. Drop the Ruby/SQLite tag-index CLI (ADR-004) and the earmarked QMD layer (ADR-006); retain the summary + namespaced-tag frontmatter from ADR-003/004 as note metadata.
• Rationale: One off-the-shelf tool delivers both what the tag index was for (structured retrieval) and what QMD was deferred for (connection/meaning-based recall via graph traversal + explain) — without writing or maintaining a bespoke index, and without a vector store. Code graphs come free. Keeps the markdown-as-truth, no-Docker, no-API-key, local-first properties (extraction runs against local Ollama). Net scope reduction: the entire Ruby build (old critical-path Step 2) and the QMD layer are removed.
• What's retained / changed: summary stays the human-written router hint Graphify does not generate; namespaced tags stay useful for Obsidian filtering and as node attributes. How tightly metadata should feed graph queries is a build-time refinement, not settled here.
• Trade-off accepted: Graphify's --update doesn't prune deleted nodes (stale-node drift) — mitigated by a periodic --force rebuild on the session-start staleness check (ADR-007's lazy model still applies). Graphify also moves fast (flags are version-dependent; anchored to v0.8.30) and its headline token-savings numbers are corpus-dependent — benchmark our own.
• Alternatives rejected: Building the Ruby/SQLite index as originally planned (more code to own; no semantic recall); adding QMD as a second system on top (two stores where one graph suffices).
• Status: Accepted (to be built — see 04-build-plan.md and 06-graphify-evaluation.md).

ADR-011 — Faceted tag taxonomy: six independent namespaces (refines ADR-003)

Date: 2026-06-04

• Context: ADR-003 introduced five namespaces (tool/, client/, domain/, convention/, scope/). During vault-reuse assessment (ADR-012) it became clear that (1) the existing SecondBrain vault uses a de-facto first-tag convention for note kind (research/plan/log/adr/howto) that should be made explicit and machine-queryable, and (2) for a freelancer working many projects per client, project identity deserves a first-class namespace rather than being implied by client/ or domain/.

• Decision: Knowledge-vault notes are classified by six independent, flat tag facets that sit side-by-side, never nested into one another:

  • type/ — note kind: research, howto, adr, hub, plan, log, clip, etc.
  • client/ — which client
  • project/ — which project (first-class; a freelancer's projects are the primary unit of work)
  • domain/ — knowledge domain / topic area
  • tool/ — tool-specific knowledge
  • convention/ — conventions
  • …plus scope/global or scope/project (retained from ADR-003)

  Hierarchy and relationships are expressed via hub notes (type/hub), wikilinks, and Graphify knowledge-graph edges — NOT via nested tag paths.

  By convention type/ is listed first in frontmatter, preserving the SecondBrain vault's existing type-first ordering habit and making the note kind immediately visible.

• Rationale: The vault is flat — hierarchy is not expressed through folder paths or tag nesting. The user's reality is many-to-many (many projects per client, knowledge domains spanning clients), which a single-parent tree models badly and forces false hierarchy. A project hub note links out to both its client/ and relevant domain/ tags rather than being buried under either. Per-type _templates will be provided for core types only (research, howto, adr, hub); the long tail stays freeform until a pattern earns a template. Consistent per-type structure also improves Graphify's local-SLM extraction reliability.

• Alternatives rejected: Hierarchical nesting in the style of John Conneely's domain/{product}/{project}.md folder structure (from the youngleaders.tech article "How I finally sorted my Claude Code memory" — secondary/interview-grade source, not verified against primary implementation). Rejected because: (1) the vault is flat — hierarchy is not expressed through folder paths; (2) the user's many-to-many reality maps badly onto a single-parent folder tree and forces false hierarchy; (3) nesting one facet through another (e.g. domain/client/project) creates Law-of-Demeter-style traversal coupling. Conneely's structure was the inspiration but diverges here on hierarchy-vs-facets. Faceted parallel tags are the flat-vault analogue of what the Graphify graph already does with edges, so they compose naturally with the chosen knowledge layer.

• Status: Accepted (supersedes the namespace list in ADR-003; core flat-vault + namespaced-tags decision stands).

ADR-012 — Reuse the existing SecondBrain vault as the knowledge vault

Date: 2026-06-04

• Context: The design called for a flat markdown vault as the semantic knowledge layer (ADR-003/008/010). The question was whether to stand up a new ~/brain vault from scratch or adopt the existing ~/Documents/SecondBrain vault.
• Decision: Adopt ~/Documents/SecondBrain as the knowledge vault rather than creating a new vault.
• Rationale: Assessment found the SecondBrain vault is already flat (all notes at root, only a _templates/ exception — exactly what the design permits), already articulates the correct "durable knowledge, not working memory" role in its CLAUDE.md and vault-conventions.md, and contains ~20 real notes. It also includes two patterns that improve on the current design and should be adopted:
  1. vault-conventions.md's "act without being asked" section specifying when the AI should proactively query the vault — a behavioral spec the cc-os docs lacked.
  2. Project-config hub notes with a tag-inference table (auto-tag by path pattern) that operationalizes how to tag a note from a given project.
• Adaptations required (migration cost):
  • Add summary: frontmatter to existing notes.
  • Migrate flat unnamespaced tags to the six-facet namespaced form (per ADR-011).
  • Add scope/global or scope/project to each note.
  • Initialize git in the vault (no .git exists yet — required by ADR-008's sync strategy).
  • Replace the vault's ~/.claude/scripts/vault_search.rb reference (script does not exist) with graphify query (ADR-010). These are mechanical schema migrations, not structural rework.
• Alternatives rejected: Starting fresh with a new ~/brain vault. Rejected because the hardest design decision — flat structure, durable-knowledge-only role, governance philosophy — is already made and practiced in SecondBrain. The improved behavioral patterns (proactive-query spec, tag-inference table) and the existing notes are worth preserving; the remaining work is mechanical migration.
• Status: Accepted.

ADR-013 — Build-first / migrate-incrementally (build-order inversion)

Date: 2026-06-04

• Context: The build runbook (05-implementation-process.md) originally front-loaded bulk vault migration as Step 1 — migrating all ~20 existing SecondBrain notes and all projects to the ADR-011 six-facet taxonomy before the system existed to validate them. This committed to a schema and workflow (the tag taxonomy from ADR-011, the vault-reuse choice from ADR-012, and Graphify extraction behavior) before any end-to-end path had been exercised. The risk: locking in an approach that fails at scale, with no feedback loop until the entire vault has been touched.
• Decision: Invert the build order. The full system is built and validated against a small 5–10 note fixture set first. Bulk vault migration is deferred to the final stage. The first real-data validation uses one small project that contains both code AND documents, exercising both the local-SLM doc-extraction path and the tree-sitter code path in the same run. After that single project validates end-to-end, remaining projects are onboarded one at a time with an observe-and-adjust step between each.
• Rationale: Validates the ADR-011 taxonomy and ADR-012 vault conventions against the real Graphify extraction pipeline before the entire vault is committed. The first mixed code+docs project surfaces both extraction paths (SLM for docs, tree-sitter for code) early, when corrections are cheap. Per-project rollout keeps the blast radius of any schema or workflow correction small; each project is an opportunity to observe and adjust rather than discover problems across 20 notes at once. This is consistent with the "markdown-as-truth, indexes are disposable" principle (ADR-008): the vault notes are durable, but the extraction schema should be validated before it shapes all of them.
• Alternatives rejected:
  • Keep migration-first (status quo): Front-loads all ~20 notes and all projects before any end-to-end validation exists. Commits to ADR-011's taxonomy and ADR-012's migration steps against the full vault without a feedback loop — exactly the gap this decision closes.
  • Big-bang migrate everything after build: Build against fixtures, then migrate all notes and all projects in one batch at the end. Avoids the pre-build commitment problem but still risks a single large irreversible migration with no observe-and-adjust loop between units. Per-project rollout with intermediate checkpoints is strictly safer.
• Cross-references: ADR-011 (six-facet tag taxonomy — the schema being validated); ADR-012 (SecondBrain vault reuse — the migration steps this order defers).
• Status: Accepted (updates 05-implementation-process.md build order).

ADR-014 — Graph connectivity comes from authored structure; migration scaffolding is a first-class prerequisite

Date: 2026-06-05

• Context: ADR-011 specified hub notes + wikilinks + Graphify graph edges as the mechanism for expressing hierarchy and cross-note relationships, with ADR-013 deferring bulk vault migration to the final stage. Before build began, a discriminating empirical test compared a cached-replay graph (per-fixture isolated extractions) against a clean single-pass deep extraction (graphify extract ~/Documents/SecondBrain --backend ollama --model qwen25-coder-7b-16k --max-concurrency 1 --token-budget 8000 --mode deep --exclude .obsidian) against the real ~/Documents/SecondBrain vault under Graphify 0.8.31 + qwen2.5-coder:7b. See 07-graph-connectivity-findings.md for the full data and methodology. [primary/measured — 2026-06-05 session]
• Decision: The connective spine of the knowledge graph must be author-provided. Hub notes and wikilinks are not optional scaffolding to add "someday" — they are the mechanism by which Graphify connects thematically related notes, and they must be authored as part of the migration step, not deferred to bulk import. Migration scaffolding (hub notes + wikilinks for key concepts) is treated as a first-class build deliverable in the migrate-incrementally phase of ADR-013.
• Rationale: The empirical test found that Graphify is a structure extractor, not a topic clusterer. Even at --mode deep --token-budget 8000, no emergent shared-topic hub nodes appeared (no "Pest Control" node, no "Niche Prospecting" node). All cross-note edges observed came from explicit references, wikilinks, or document-level semantic similarity — not from shared thematic identity. A practical test query ("how do we do niche prospecting outreach for pest control?") returned 3 starting notes and traversal could not reach the email templates / ACV data / business-model notes (separate communities, no connecting edges). This confirms that useful retrieval is gated on migration scaffolding, not on Graphify's extraction power. The clean single-pass run also showed the cached graph was partially a build artifact (cross- note edges rose from 41% to 78% in a single-pass run), but the structural finding — no emergent hub nodes — held in both runs.
• Relationship to ADR-011: Validates the hub-notes + wikilinks half of ADR-011 empirically. The facet-tag half is not yet validated: no edge was observed to arise from shared frontmatter facet tags alone. Whether client/X or tool/Y tags create graph connectivity is an open question — see "Deferred" below. Do not assume facet tags contribute to graph traversal retrieval until tested.
• Relationship to ADR-013: Refines the migrate-incrementally stage. "Migration" must be defined to include hub note authoring and wikilink addition for key concepts, not just frontmatter schema migration (adding summary: and namespaced facet tags). The build plan (04-build-plan.md) should be updated to name this deliverable explicitly.
• Alternatives rejected: Relying on the SLM to auto-cluster topics and synthesize hub entities — empirically does not happen at 7B model size with --mode deep. The design already intended human-authored hub notes for this; the test confirms that intent was correct and the fallback assumption ("maybe the LLM will do it") is false.
• Deferred:
  1. Facet-tag-to-graph-edge question: Do shared frontmatter facet tags (client/, tool/, domain/, etc.) cause Graphify to create edges between notes, or does graph connectivity come only from explicit wikilinks/references and semantic similarity? This was NOT tested. Resolve before designing graph-traversal retrieval skills.
  2. Larger extraction model: Whether a substantially larger SLM (14B, 30B) would synthesize emergent topic-hub nodes is untested. Secondary — the design does not depend on it — but worth one test run before the build ships.
  3. reasoning_effort:"none" patch: The clean run required a local patch to graphify/ llm.py. Track the upstream Graphify issue tracker for an official fix; treat the installed version as pinned until resolved.
• Status: Accepted. Refines ADR-013 (migrate-incrementally phase scope) and empirically validates the hub-notes/wikilinks mechanism of ADR-011 while flagging its facet-tag half as an open question.

Rejected tools (summary)

Tool	Why rejected for our use
MemPalace (L4)	Storage not readable markdown; isolated drawers (knowledge not interconnected); fights self-managing + cross-linking goals
Recall / LightRAG (L5)	Content knowledge bases / deep research, not operational memory; Recall = hosted, you don't own data; LightRAG = enterprise overkill
OpenBrain / Mem0 (L6)	Always-remote DB → latency + cost; conflicts with local-fast lazy-sync; only pays off for real-time cross-tool memory (user: overkill)
Postgres / Milvus server	Unnecessary — Graphify's local graph (knowledge) + Milvus Lite (memsearch episodic) cover everything locally with no Docker
claude-mem	MCP-based (Claude must call search); opaque blobs vs readable markdown; feature overkill
Ruby/SQLite tag index CLI; QMD vector layer	Superseded by Graphify before build — one knowledge graph replaces both the structured index and the deferred semantic layer (ADR-010)