150 lines
9.7 KiB
Markdown
150 lines
9.7 KiB
Markdown
# Architecture Decision Records
|
||
|
||
A running log of decisions and *why*. Format per entry: Context · Decision · Rationale ·
|
||
Alternatives rejected · Status. Newest decisions extend the log; supersede rather than delete.
|
||
|
||
---
|
||
|
||
## ADR-001 — Two memory types, kept as separate systems
|
||
|
||
- **Context**: Earlier attempts to make one tool serve both "what happened" and "how do we do
|
||
X" felt forced (e.g. trying to make memsearch filter knowledge by tags).
|
||
- **Decision**: Model **episodic** memory and **semantic/knowledge** memory as two separate
|
||
systems with different tools.
|
||
- **Rationale**: They have different lifecycles (episodic accretes and decays; knowledge is
|
||
deliberately maintained), different write paths (auto-captured vs curated with guardrails),
|
||
and different query patterns ("when did we…" vs "how do we…"). Separation dissolves the
|
||
earlier integration tension entirely.
|
||
- **Alternatives rejected**: One unified store (memsearch-for-everything, or OpenBrain's single
|
||
`thoughts` table) — conflates the two and forces awkward filtering.
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-002 — memsearch for the episodic layer
|
||
|
||
- **Context**: Need timeline/"what happened" memory (Goal 3) that's NL-queryable and lazy.
|
||
- **Decision**: Adopt **memsearch** (Zilliz) off-the-shelf for episodic memory.
|
||
- **Rationale**: It already implements the OpenClaw daily-notes + "dreaming" pattern and the
|
||
markdown-as-truth / disposable-shadow-index philosophy we'd otherwise build. Embedded
|
||
**Milvus Lite** (single file), hybrid BM25+vector+RRF search, local ONNX embeddings (no API
|
||
key/cost), a FileWatcher that handles deletions — **no Docker, no server**. Two-line install.
|
||
- **Alternatives rejected**: claude-mem (MCP-based — Claude must actively call search; opaque
|
||
blobs vs readable markdown; overkill features). Hand-building daily notes + dreaming
|
||
ourselves (reinventing a solved tool).
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-003 — Flat vault with namespaced tags, not folders
|
||
|
||
- **Context**: Connelly/Huryn organize by folders (`tools/`, `domain/`). User wants a flat
|
||
Obsidian vault with tags as virtual indexes, and cross-cutting filters (client × tool ×
|
||
convention).
|
||
- **Decision**: One **flat markdown vault**; organize via **namespaced, nested tags**
|
||
(`tool/`, `client/`, `domain/`, `convention/`, `scope/`). Slashes are valid Obsidian nested
|
||
tags, so `#tool` matches all children.
|
||
- **Rationale**: A note can carry several namespaces at once (`tool/semrush` +
|
||
`client/sesame3g` + `convention/react-ts`) — folders can't express that. Enables "filter by
|
||
client+tool to narrow the index." Enumerable virtual indexes ("what clients/tools exist").
|
||
- **Alternatives rejected**: Folder hierarchy (single-axis; can't do cross-cutting filters).
|
||
Pure-prefix path filtering via memsearch `source_prefix` (would force directories back in).
|
||
- **Trade-off accepted**: Tags give the *human/Obsidian* free filtering, but the *AI* gets
|
||
nothing for free from tags — we must materialize them into a queryable index (see ADR-004).
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-004 — SQLite + Sequel (Ruby) tag index as the knowledge-layer cache
|
||
|
||
- **Context**: The AI can't use Obsidian tags directly; tag filtering needs a machine-queryable
|
||
index. A previous `~/Documents/SecondBrain/` tag database was lost track of.
|
||
- **Decision**: A small **Ruby program using the Sequel ORM over SQLite**, exposed as a **CLI**.
|
||
Schema: `files(path, mtime, summary, scope)`, `tags(name)`, `files_tags` join
|
||
(`many_to_many`). The summary is a **column on `files`** (an attribute), not a join.
|
||
- **Rationale**: Normalized `tags` table makes enumerating the vocabulary a first-class cheap
|
||
query (the "virtual index" goal). The `summary` column is what turns the index from a
|
||
*finder* into a *router* — the AI sees enough to pick a file without opening it (progressive
|
||
disclosure, low tokens). Ruby + Sequel + CLI keeps the contract clean and the DB swappable;
|
||
the AI never touches SQLite directly.
|
||
- **Failure-mode guard (the lost-SecondBrain lesson)**: **markdown is always authoritative; the
|
||
SQLite file is a disposable cache** that is never synced and can be rebuilt from frontmatter
|
||
anytime (`index update --rebuild`).
|
||
- **Alternatives rejected**: Plain-markdown generated `INDEX.md` (must regenerate; grep-at-scale
|
||
is token-heavy). Frontmatter grep on demand (scales badly). Milvus/Postgres for knowledge
|
||
(overkill; QMD/memsearch prove SQLite is enough — see ADR-006/008).
|
||
- **Query output**: returns **path + summary + matched tags** (option C) — tags are cheap and
|
||
show *why* a result matched, useful for cross-client queries.
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-005 — Structured-first; semantic search over the vault deferred
|
||
|
||
- **Context**: Tag filtering ("client/sesame3g + tool/semrush") may miss notes whose wording
|
||
doesn't match the query ("how do we use semrush" vs a note titled "search analytics
|
||
integration").
|
||
- **Decision**: Ship the knowledge layer **structured-only** (tags + summaries). **Defer**
|
||
meaning-based search over the vault until it demonstrably bites.
|
||
- **Rationale**: Structured tagging is the lightweight/fast thing the user wants, and the
|
||
summary+tag design is built to make it work. Follow the video's "only level up when it bites."
|
||
- **Status**: Accepted (semantic deferred).
|
||
|
||
## ADR-006 — QMD as the (deferred) semantic-over-knowledge layer
|
||
|
||
- **Context**: When ADR-005's structured-only proves insufficient, we want a set-and-forget
|
||
semantic layer over the vault, local and Docker-free.
|
||
- **Decision**: Earmark **QMD** (github.com/tobi/qmd) for that role; do **not** install yet.
|
||
- **Rationale**: Local markdown search using **SQLite + FTS5/BM25 + local vector embeddings
|
||
(EmbeddingGemma-300M GGUF) + LLM rerank**; CLI + optional **MCP server**; no Docker, no API
|
||
keys. Validates that SQLite + a local vector model suffices (no Milvus/Postgres for
|
||
knowledge). Complements the tag index (QMD filters by path/collection context, not
|
||
first-class frontmatter tags), so it adds semantic recall without replacing structured
|
||
filtering.
|
||
- **Alternatives rejected**: Pointing memsearch at the vault (mixes episodic and knowledge
|
||
corpora; its filtering is path-prefix not tags). A bespoke embedding index (reinvents QMD).
|
||
- **Status**: Deferred / earmarked.
|
||
|
||
## ADR-007 — Lazy freshness: write-hook + session-start reconcile, no daemon/cron
|
||
|
||
- **Context**: The cache must reflect new/edited/deleted/renamed notes without becoming a
|
||
resource hog or going stale on renames.
|
||
- **Decision**: **Option A (lazy).** A `PostToolUse` hook updates the index on **AI** writes
|
||
(single-file, prunes on delete). **Manual** edits are caught by a **session-start reconcile**
|
||
(`index update --since` + prune of vanished paths). **No daemon, no cron.**
|
||
- **Rationale**: The AI is the primary writer, so write-time hooks give event-driven freshness
|
||
with no polling. The user rarely edits the vault by hand, so a session-start reconcile is
|
||
enough; a continuous `inotify` daemon (the `listen` gem) would add an always-on process to
|
||
manage/sync for negligible benefit. Matches the user's "lazy sync is fine" stance.
|
||
- **Alternatives rejected**: `inotify`/`listen` daemon (live freshness, but always-on process
|
||
to manage — unnecessary). Cron reconcile ("seems silly" per user; session-start covers it).
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-008 — Markdown-as-truth; sync the vault, not the indexes
|
||
|
||
- **Context**: Must be accessible on a VPS / multiple machines but run local-fast (Goal 4).
|
||
- **Decision**: Sync the **markdown vault** to the VPS via **git or Syncthing** (choice deferred
|
||
to build time). **Indexes (Milvus Lite, future QMD) are rebuilt per machine and never
|
||
synced.**
|
||
- **Rationale**: Markdown is plain text — git/Syncthing sync it trivially; lazy (hourly or
|
||
continuous-async) is enough. Indexes are disposable caches; syncing binary DBs invites
|
||
conflicts for no gain. Local reads stay fast; ownership and portability stay with the user.
|
||
- **Alternatives rejected**: **OpenBrain / Mem0** hosted DBs — always-remote, adds per-query
|
||
latency and monthly cost, conflicts with local-fast; ownership weaker (Mem0 especially).
|
||
Only worth it for real-time cross-tool memory, which the user called overkill.
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-009 — Package as a global Claude Code plugin with skills
|
||
|
||
- **Context**: Every project, on every machine, should know how to use the vault — write
|
||
conventions, query patterns, the hooks, and the CLI — without per-project setup.
|
||
- **Decision**: Ship hooks + scripts + CRUD know-how as a **global Claude Code plugin with
|
||
skills**, installed at the user level.
|
||
- **Rationale**: Skills carry the "when to write / what conventions / how & when to query"
|
||
guidance to the model; the plugin registers the session-start / session-end / PostToolUse
|
||
hooks and bundles the Ruby CLI. Global install = consistent behavior everywhere; single
|
||
source of truth for the conventions themselves.
|
||
- **Status**: Accepted (to be built — see 04-build-plan.md).
|
||
|
||
## Rejected tools (summary)
|
||
|
||
| Tool | Why rejected for our use |
|
||
|------|--------------------------|
|
||
| MemPalace (L4) | Storage not readable markdown; isolated drawers (knowledge not interconnected); fights self-managing + cross-linking goals |
|
||
| Recall / LightRAG (L5) | Content knowledge bases / deep research, not operational memory; Recall = hosted, you don't own data; LightRAG = enterprise overkill |
|
||
| OpenBrain / Mem0 (L6) | Always-remote DB → latency + cost; conflicts with local-fast lazy-sync; only pays off for real-time cross-tool memory (user: overkill) |
|
||
| Postgres / Milvus server | Unnecessary — SQLite (tag index) + Milvus Lite (memsearch) + QMD's SQLite cover everything locally with no Docker |
|
||
| claude-mem | MCP-based (Claude must call search); opaque blobs vs readable markdown; feature overkill |
|