cc-os/docs/memory-system/03-architecture-decisions.md

# Architecture Decision Records

_Last updated: 2026-06-04_

A running log of decisions and *why*. Format per entry: Context · Decision · Rationale ·
Alternatives rejected · Status. Newest decisions extend the log; supersede rather than delete.

---

## ADR-001 — Two memory types, kept as separate systems

- **Context**: Earlier attempts to make one tool serve both "what happened" and "how do we do
  X" felt forced (e.g. trying to make memsearch filter knowledge by tags).
- **Decision**: Model **episodic** memory and **semantic/knowledge** memory as two separate
  systems with different tools.
- **Rationale**: They have different lifecycles (episodic accretes and decays; knowledge is
  deliberately maintained), different write paths (auto-captured vs curated with guardrails),
  and different query patterns ("when did we…" vs "how do we…"). Separation dissolves the
  earlier integration tension entirely.
- **Alternatives rejected**: One unified store (memsearch-for-everything, or OpenBrain's single
  `thoughts` table) — conflates the two and forces awkward filtering.
- **Status**: Accepted.

## ADR-002 — memsearch for the episodic layer

- **Context**: Need timeline/"what happened" memory (Goal 3) that's NL-queryable and lazy.
- **Decision**: Adopt **memsearch** (Zilliz) off-the-shelf for episodic memory.
- **Rationale**: It already implements the OpenClaw daily-notes + "dreaming" pattern and the
  markdown-as-truth / disposable-shadow-index philosophy we'd otherwise build. Embedded
  **Milvus Lite** (single file), hybrid BM25+vector+RRF search, local ONNX embeddings (no API
  key/cost), a FileWatcher that handles deletions — **no Docker, no server**. Two-line install.
- **Alternatives rejected**: claude-mem (MCP-based — Claude must actively call search; opaque
  blobs vs readable markdown; overkill features). Hand-building daily notes + dreaming
  ourselves (reinventing a solved tool).
- **Status**: Accepted.

## ADR-003 — Flat vault with namespaced tags, not folders

- **Context**: Connelly/Huryn organize by folders (`tools/`, `domain/`). User wants a flat
  Obsidian vault with tags as virtual indexes, and cross-cutting filters (client × tool ×
  convention).
- **Decision**: One **flat markdown vault**; organize via **namespaced, nested tags**
  (`tool/`, `client/`, `domain/`, `convention/`, `scope/`). Slashes are valid Obsidian nested
  tags, so `#tool` matches all children.
- **Rationale**: A note can carry several namespaces at once (`tool/semrush` +
  `client/sesame3g` + `convention/react-ts`) — folders can't express that. Enables "filter by
  client+tool to narrow the index." Enumerable virtual indexes ("what clients/tools exist").
- **Alternatives rejected**: Folder hierarchy (single-axis; can't do cross-cutting filters).
  Pure-prefix path filtering via memsearch `source_prefix` (would force directories back in).
- **Trade-off accepted**: Tags give the *human/Obsidian* free filtering, but the *AI* gets
  nothing for free from tags — we must materialize them into a queryable index (see ADR-004).
- **Status**: **Refined by ADR-011** (type/ and project/ namespaces added; hierarchy-vs-facets
  clarified). Core decision — flat vault, namespaced tags — stands.

## ADR-004 — SQLite + Sequel (Ruby) tag index as the knowledge-layer cache

- **Context**: The AI can't use Obsidian tags directly; tag filtering needs a machine-queryable
  index. A previous `~/Documents/SecondBrain/` tag database was lost track of.
- **Decision**: A small **Ruby program using the Sequel ORM over SQLite**, exposed as a **CLI**.
  Schema: `files(path, mtime, summary, scope)`, `tags(name)`, `files_tags` join
  (`many_to_many`). The summary is a **column on `files`** (an attribute), not a join.
- **Rationale**: Normalized `tags` table makes enumerating the vocabulary a first-class cheap
  query (the "virtual index" goal). The `summary` column is what turns the index from a
  *finder* into a *router* — the AI sees enough to pick a file without opening it (progressive
  disclosure, low tokens). Ruby + Sequel + CLI keeps the contract clean and the DB swappable;
  the AI never touches SQLite directly.
- **Failure-mode guard (the lost-SecondBrain lesson)**: **markdown is always authoritative; the
  SQLite file is a disposable cache** that is never synced and can be rebuilt from frontmatter
  anytime (`index update --rebuild`).
- **Alternatives rejected**: Plain-markdown generated `INDEX.md` (must regenerate; grep-at-scale
  is token-heavy). Frontmatter grep on demand (scales badly). Milvus/Postgres for knowledge
  (overkill; QMD/memsearch prove SQLite is enough — see ADR-006/008).
- **Query output**: returns **path + summary + matched tags** (option C) — tags are cheap and
  show *why* a result matched, useful for cross-client queries.
- **Status**: **Superseded by ADR-010** (Graphify replaces the Ruby/SQLite tag index). The
  `summary` + namespaced-tag frontmatter this ADR introduced is **retained as note metadata**;
  only the bespoke Ruby/SQLite index and its CLI are dropped.

## ADR-005 — Structured-first; semantic search over the vault deferred

- **Context**: Tag filtering ("client/sesame3g + tool/semrush") may miss notes whose wording
  doesn't match the query ("how do we use semrush" vs a note titled "search analytics
  integration").
- **Decision**: Ship the knowledge layer **structured-only** (tags + summaries). **Defer**
  meaning-based search over the vault until it demonstrably bites.
- **Rationale**: Structured tagging is the lightweight/fast thing the user wants, and the
  summary+tag design is built to make it work. Follow the video's "only level up when it bites."
- **Status**: **Superseded by ADR-010.** The premise (ship structured-only, bolt on semantics
  later) no longer holds: Graphify makes the knowledge layer a graph from day one, giving
  structured *and* connection-based recall together. The "only level up when it bites" instinct
  carries forward to whether a *vector* layer is ever needed on top of the graph.

## ADR-006 — QMD as the (deferred) semantic-over-knowledge layer

- **Context**: When ADR-005's structured-only proves insufficient, we want a set-and-forget
  semantic layer over the vault, local and Docker-free.
- **Decision**: Earmark **QMD** (github.com/tobi/qmd) for that role; do **not** install yet.
- **Rationale**: Local markdown search using **SQLite + FTS5/BM25 + local vector embeddings
  (EmbeddingGemma-300M GGUF) + LLM rerank**; CLI + optional **MCP server**; no Docker, no API
  keys. Validates that SQLite + a local vector model suffices (no Milvus/Postgres for
  knowledge). Complements the tag index (QMD filters by path/collection context, not
  first-class frontmatter tags), so it adds semantic recall without replacing structured
  filtering.
- **Alternatives rejected**: Pointing memsearch at the vault (mixes episodic and knowledge
  corpora; its filtering is path-prefix not tags). A bespoke embedding index (reinvents QMD).
- **Status**: **Superseded by ADR-010.** Graphify's knowledge graph fills the
  semantic-recall role (traversal/`explain` over connections) without a separate vector system,
  so QMD is no longer earmarked. Revisit a vector layer only if graph traversal demonstrably
  misses cases where embedding similarity would win.

## ADR-007 — Lazy freshness: write-hook + session-start reconcile, no daemon/cron

- **Context**: The cache must reflect new/edited/deleted/renamed notes without becoming a
  resource hog or going stale on renames.
- **Decision**: **Option A (lazy).** A `PostToolUse` hook updates the index on **AI** writes
  (single-file, prunes on delete). **Manual** edits are caught by a **session-start reconcile**
  (`index update --since` + prune of vanished paths). **No daemon, no cron.**
- **Rationale**: The AI is the primary writer, so write-time hooks give event-driven freshness
  with no polling. The user rarely edits the vault by hand, so a session-start reconcile is
  enough; a continuous `inotify` daemon (the `listen` gem) would add an always-on process to
  manage/sync for negligible benefit. Matches the user's "lazy sync is fine" stance.
- **Alternatives rejected**: `inotify`/`listen` daemon (live freshness, but always-on process
  to manage — unnecessary). Cron reconcile ("seems silly" per user; session-start covers it).
- **Status**: Accepted.

## ADR-008 — Markdown-as-truth; sync the vault, not the indexes

- **Context**: Must be accessible on a VPS / multiple machines but run local-fast (Goal 4).
- **Decision**: Sync the **markdown vault** to the VPS via **git or Syncthing** (choice deferred
  to build time). **Graphs/indexes (Milvus Lite, Graphify `graphify-out/`) are rebuilt per
  machine and never synced.**
- **Rationale**: Markdown is plain text — git/Syncthing sync it trivially; lazy (hourly or
  continuous-async) is enough. Indexes are disposable caches; syncing binary DBs invites
  conflicts for no gain. Local reads stay fast; ownership and portability stay with the user.
- **Alternatives rejected**: **OpenBrain / Mem0** hosted DBs — always-remote, adds per-query
  latency and monthly cost, conflicts with local-fast; ownership weaker (Mem0 especially).
  Only worth it for real-time cross-tool memory, which the user called overkill.
- **Status**: Accepted.

## ADR-009 — Package as a global Claude Code plugin with skills

- **Context**: Every project, on every machine, should know how to use the vault — write
  conventions, query patterns, the hooks, and the CLI — without per-project setup.
- **Decision**: Ship hooks + scripts + CRUD know-how as a **global Claude Code plugin with
  skills**, installed at the user level.
- **Rationale**: Skills carry the "when to write / what conventions / how & when to query"
  guidance to the model; the plugin registers the session-start / session-end / PostToolUse
  hooks and wires up Graphify (extraction/update/query + MCP server). Global install =
  consistent behavior everywhere; single source of truth for the conventions themselves.
- **Status**: Accepted (to be built — see 04-build-plan.md).

## ADR-010 — Graphify knowledge graph as the knowledge layer (supersedes ADR-004/005/006)

- **Context**: ADR-004 specced a hand-built Ruby/Sequel/SQLite tag index (+ CLI) as the
  machine-queryable layer over the vault, with ADR-005/006 deferring meaning-based recall to a
  future QMD vector layer. Before building any of it, we evaluated **Graphify** (`graphify`,
  PyPI `graphifyy`) — a tool that turns a folder into a queryable knowledge graph (local
  tree-sitter AST for code, local-SLM entity/relationship extraction for docs). See
  `06-graphify-evaluation.md`.
- **Decision**: Use **Graphify as the knowledge-layer engine** over the vault, with a **local
  Ollama** backend for doc extraction and free AST for per-project code graphs. **Drop** the
  Ruby/SQLite tag-index CLI (ADR-004) and the earmarked QMD layer (ADR-006); **retain** the
  `summary` + namespaced-tag frontmatter from ADR-003/004 as note metadata.
- **Rationale**: One off-the-shelf tool delivers both what the tag index was for (structured
  retrieval) and what QMD was deferred for (connection/meaning-based recall via graph
  traversal + `explain`) — without writing or maintaining a bespoke index, and without a vector
  store. Code graphs come free. Keeps the markdown-as-truth, no-Docker, no-API-key, local-first
  properties (extraction runs against local Ollama). Net scope reduction: the entire Ruby build
  (old critical-path Step 2) and the QMD layer are removed.
- **What's retained / changed**: `summary` stays the human-written router hint Graphify does not
  generate; namespaced tags stay useful for Obsidian filtering and as node attributes. How
  tightly metadata should feed graph queries is a **build-time refinement**, not settled here.
- **Trade-off accepted**: Graphify's `--update` doesn't prune deleted nodes (stale-node drift) —
  mitigated by a periodic `--force` rebuild on the session-start staleness check (ADR-007's lazy
  model still applies). Graphify also moves fast (flags are version-dependent; anchored to
  v0.8.30) and its headline token-savings numbers are corpus-dependent — benchmark our own.
- **Alternatives rejected**: Building the Ruby/SQLite index as originally planned (more code to
  own; no semantic recall); adding QMD as a second system on top (two stores where one graph
  suffices).
- **Status**: Accepted (to be built — see 04-build-plan.md and 06-graphify-evaluation.md).

## ADR-011 — Faceted tag taxonomy: six independent namespaces (refines ADR-003)

_Date: 2026-06-04_

- **Context**: ADR-003 introduced five namespaces (`tool/`, `client/`, `domain/`,
  `convention/`, `scope/`). During vault-reuse assessment (ADR-012) it became clear that (1)
  the existing SecondBrain vault uses a de-facto first-tag convention for note kind
  (research/plan/log/adr/howto) that should be made explicit and machine-queryable, and (2)
  for a freelancer working many projects per client, project identity deserves a first-class
  namespace rather than being implied by `client/` or `domain/`.
- **Decision**: Knowledge-vault notes are classified by **six independent, flat tag facets**
  that sit side-by-side, never nested into one another:
  - `type/` — note kind: `research`, `howto`, `adr`, `hub`, `plan`, `log`, `clip`, etc.
  - `client/` — which client
  - `project/` — which project (first-class; a freelancer's projects are the primary unit of
    work)
  - `domain/` — knowledge domain / topic area
  - `tool/` — tool-specific knowledge
  - `convention/` — conventions
  - …plus `scope/global` or `scope/project` (retained from ADR-003)

  Hierarchy and relationships are expressed via **hub notes** (`type/hub`), **wikilinks**, and
  **Graphify knowledge-graph edges** — NOT via nested tag paths.

  By convention `type/` is listed **first** in frontmatter, preserving the SecondBrain vault's
  existing type-first ordering habit and making the note kind immediately visible.
- **Rationale**: The vault is flat — hierarchy is not expressed through folder paths or tag
  nesting. The user's reality is many-to-many (many projects per client, knowledge domains
  spanning clients), which a single-parent tree models badly and forces false hierarchy. A
  project hub note links out to both its `client/` and relevant `domain/` tags rather than
  being buried under either. Per-type `_templates` will be provided for **core types only**
  (research, howto, adr, hub); the long tail stays freeform until a pattern earns a template.
  Consistent per-type structure also improves Graphify's local-SLM extraction reliability.
- **Alternatives rejected**: Hierarchical nesting in the style of John Conneely's
  `domain/{product}/{project}.md` folder structure (from the youngleaders.tech article "How I
  finally sorted my Claude Code memory" — **secondary/interview-grade source, not verified
  against primary implementation**). Rejected because: (1) the vault is flat — hierarchy is
  not expressed through folder paths; (2) the user's many-to-many reality maps badly onto a
  single-parent folder tree and forces false hierarchy; (3) nesting one facet through another
  (e.g. `domain/client/project`) creates Law-of-Demeter-style traversal coupling. Conneely's
  structure was the inspiration but diverges here on hierarchy-vs-facets. Faceted parallel tags
  are the flat-vault analogue of what the Graphify graph already does with edges, so they
  compose naturally with the chosen knowledge layer.
- **Status**: Accepted (supersedes the namespace list in ADR-003; core flat-vault +
  namespaced-tags decision stands).

## ADR-012 — Reuse the existing SecondBrain vault as the knowledge vault

_Date: 2026-06-04_

- **Context**: The design called for a flat markdown vault as the semantic knowledge layer
  (ADR-003/008/010). The question was whether to stand up a new `~/brain` vault from scratch
  or adopt the existing `~/Documents/SecondBrain` vault.
- **Decision**: **Adopt `~/Documents/SecondBrain`** as the knowledge vault rather than
  creating a new vault.
- **Rationale**: Assessment found the SecondBrain vault is already flat (all notes at root,
  only a `_templates/` exception — exactly what the design permits), already articulates the
  correct "durable knowledge, not working memory" role in its `CLAUDE.md` and
  `vault-conventions.md`, and contains ~20 real notes. It also includes two patterns that
  **improve on the current design** and should be adopted:
  1. `vault-conventions.md`'s "act without being asked" section specifying *when* the AI
     should proactively query the vault — a behavioral spec the cc-os docs lacked.
  2. Project-config hub notes with a tag-inference table (auto-tag by path pattern) that
     operationalizes *how* to tag a note from a given project.
- **Adaptations required (migration cost)**:
  - Add `summary:` frontmatter to existing notes.
  - Migrate flat unnamespaced tags to the six-facet namespaced form (per ADR-011).
  - Add `scope/global` or `scope/project` to each note.
  - Initialize git in the vault (no `.git` exists yet — required by ADR-008's sync strategy).
  - Replace the vault's `~/.claude/scripts/vault_search.rb` reference (script does not exist)
    with `graphify query` (ADR-010).
  These are mechanical schema migrations, not structural rework.
- **Alternatives rejected**: Starting fresh with a new `~/brain` vault. Rejected because the
  hardest design decision — flat structure, durable-knowledge-only role, governance philosophy
  — is already made and practiced in SecondBrain. The improved behavioral patterns
  (proactive-query spec, tag-inference table) and the existing notes are worth preserving; the
  remaining work is mechanical migration.
- **Status**: Accepted.

## ADR-013 — Build-first / migrate-incrementally (build-order inversion)

_Date: 2026-06-04_

- **Context**: The build runbook (`05-implementation-process.md`) originally front-loaded bulk
  vault migration as Step 1 — migrating all ~20 existing SecondBrain notes and all projects to
  the ADR-011 six-facet taxonomy before the system existed to validate them. This committed to
  a schema and workflow (the tag taxonomy from ADR-011, the vault-reuse choice from ADR-012,
  and Graphify extraction behavior) before any end-to-end path had been exercised. The risk:
  locking in an approach that fails at scale, with no feedback loop until the entire vault has
  been touched.
- **Decision**: **Invert the build order.** The full system is built and validated against a
  small **5–10 note fixture set** first. Bulk vault migration is deferred to the final stage.
  The first real-data validation uses **one small project that contains both code AND
  documents**, exercising both the local-SLM doc-extraction path and the tree-sitter code path
  in the same run. After that single project validates end-to-end, remaining projects are
  onboarded **one at a time** with an observe-and-adjust step between each.
- **Rationale**: Validates the ADR-011 taxonomy and ADR-012 vault conventions against the real
  Graphify extraction pipeline before the entire vault is committed. The first mixed code+docs
  project surfaces both extraction paths (SLM for docs, tree-sitter for code) early, when
  corrections are cheap. Per-project rollout keeps the blast radius of any schema or workflow
  correction small; each project is an opportunity to observe and adjust rather than discover
  problems across 20 notes at once. This is consistent with the "markdown-as-truth, indexes are
  disposable" principle (ADR-008): the vault notes are durable, but the extraction schema should
  be validated before it shapes all of them.
- **Alternatives rejected**:
  - **Keep migration-first (status quo)**: Front-loads all ~20 notes and all projects before
    any end-to-end validation exists. Commits to ADR-011's taxonomy and ADR-012's migration
    steps against the full vault without a feedback loop — exactly the gap this decision closes.
  - **Big-bang migrate everything after build**: Build against fixtures, then migrate all notes
    and all projects in one batch at the end. Avoids the pre-build commitment problem but still
    risks a single large irreversible migration with no observe-and-adjust loop between units.
    Per-project rollout with intermediate checkpoints is strictly safer.
- **Cross-references**: ADR-011 (six-facet tag taxonomy — the schema being validated);
  ADR-012 (SecondBrain vault reuse — the migration steps this order defers).
- **Status**: Accepted (updates `05-implementation-process.md` build order).

## Rejected tools (summary)

| Tool | Why rejected for our use |
|------|--------------------------|
| MemPalace (L4) | Storage not readable markdown; isolated drawers (knowledge not interconnected); fights self-managing + cross-linking goals |
| Recall / LightRAG (L5) | Content knowledge bases / deep research, not operational memory; Recall = hosted, you don't own data; LightRAG = enterprise overkill |
| OpenBrain / Mem0 (L6) | Always-remote DB → latency + cost; conflicts with local-fast lazy-sync; only pays off for real-time cross-tool memory (user: overkill) |
| Postgres / Milvus server | Unnecessary — Graphify's local graph (knowledge) + Milvus Lite (memsearch episodic) cover everything locally with no Docker |
| claude-mem | MCP-based (Claude must call search); opaque blobs vs readable markdown; feature overkill |
| Ruby/SQLite tag index CLI; QMD vector layer | Superseded by Graphify before build — one knowledge graph replaces both the structured index and the deferred semantic layer (ADR-010) |