550 lines
42 KiB
Markdown
550 lines
42 KiB
Markdown
# Architecture Decision Records
|
||
|
||
_Last updated: 2026-06-17_
|
||
|
||
A running log of decisions and *why*. Format per entry: Context · Decision · Rationale ·
|
||
Alternatives rejected · Status. Newest decisions extend the log; supersede rather than delete.
|
||
|
||
---
|
||
|
||
## ADR-001 — Two memory types, kept as separate systems
|
||
|
||
- **Context**: Earlier attempts to make one tool serve both "what happened" and "how do we do
|
||
X" felt forced (e.g. trying to make memsearch filter knowledge by tags).
|
||
- **Decision**: Model **episodic** memory and **semantic/knowledge** memory as two separate
|
||
systems with different tools.
|
||
- **Rationale**: They have different lifecycles (episodic accretes and decays; knowledge is
|
||
deliberately maintained), different write paths (auto-captured vs curated with guardrails),
|
||
and different query patterns ("when did we…" vs "how do we…"). Separation dissolves the
|
||
earlier integration tension entirely.
|
||
- **Alternatives rejected**: One unified store (memsearch-for-everything, or OpenBrain's single
|
||
`thoughts` table) — conflates the two and forces awkward filtering.
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-002 — memsearch for the episodic layer
|
||
|
||
- **Context**: Need timeline/"what happened" memory (Goal 3) that's NL-queryable and lazy.
|
||
- **Decision**: Adopt **memsearch** (Zilliz) off-the-shelf for episodic memory.
|
||
- **Rationale**: It already implements the OpenClaw daily-notes + "dreaming" pattern and the
|
||
markdown-as-truth / disposable-shadow-index philosophy we'd otherwise build. Embedded
|
||
**Milvus Lite** (single file), hybrid BM25+vector+RRF search, local ONNX embeddings (no API
|
||
key/cost), a FileWatcher that handles deletions — **no Docker, no server**. Two-line install.
|
||
- **Alternatives rejected**: claude-mem (MCP-based — Claude must actively call search; opaque
|
||
blobs vs readable markdown; overkill features). Hand-building daily notes + dreaming
|
||
ourselves (reinventing a solved tool).
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-003 — Flat vault with namespaced tags, not folders
|
||
|
||
- **Context**: Connelly/Huryn organize by folders (`tools/`, `domain/`). User wants a flat
|
||
Obsidian vault with tags as virtual indexes, and cross-cutting filters (client × tool ×
|
||
convention).
|
||
- **Decision**: One **flat markdown vault**; organize via **namespaced, nested tags**
|
||
(`tool/`, `client/`, `domain/`, `convention/`, `scope/`). Slashes are valid Obsidian nested
|
||
tags, so `#tool` matches all children.
|
||
- **Rationale**: A note can carry several namespaces at once (`tool/semrush` +
|
||
`client/sesame3g` + `convention/react-ts`) — folders can't express that. Enables "filter by
|
||
client+tool to narrow the index." Enumerable virtual indexes ("what clients/tools exist").
|
||
- **Alternatives rejected**: Folder hierarchy (single-axis; can't do cross-cutting filters).
|
||
Pure-prefix path filtering via memsearch `source_prefix` (would force directories back in).
|
||
- **Trade-off accepted**: Tags give the *human/Obsidian* free filtering, but the *AI* gets
|
||
nothing for free from tags — we must materialize them into a queryable index (see ADR-004).
|
||
- **Status**: **Refined by ADR-011** (type/ and project/ namespaces added; hierarchy-vs-facets
|
||
clarified). Core decision — flat vault, namespaced tags — stands.
|
||
|
||
## ADR-004 — SQLite + Sequel (Ruby) tag index as the knowledge-layer cache
|
||
|
||
- **Context**: The AI can't use Obsidian tags directly; tag filtering needs a machine-queryable
|
||
index. A previous `~/Documents/SecondBrain/` tag database was lost track of.
|
||
- **Decision**: A small **Ruby program using the Sequel ORM over SQLite**, exposed as a **CLI**.
|
||
Schema: `files(path, mtime, summary, scope)`, `tags(name)`, `files_tags` join
|
||
(`many_to_many`). The summary is a **column on `files`** (an attribute), not a join.
|
||
- **Rationale**: Normalized `tags` table makes enumerating the vocabulary a first-class cheap
|
||
query (the "virtual index" goal). The `summary` column is what turns the index from a
|
||
*finder* into a *router* — the AI sees enough to pick a file without opening it (progressive
|
||
disclosure, low tokens). Ruby + Sequel + CLI keeps the contract clean and the DB swappable;
|
||
the AI never touches SQLite directly.
|
||
- **Failure-mode guard (the lost-SecondBrain lesson)**: **markdown is always authoritative; the
|
||
SQLite file is a disposable cache** that is never synced and can be rebuilt from frontmatter
|
||
anytime (`index update --rebuild`).
|
||
- **Alternatives rejected**: Plain-markdown generated `INDEX.md` (must regenerate; grep-at-scale
|
||
is token-heavy). Frontmatter grep on demand (scales badly). Milvus/Postgres for knowledge
|
||
(overkill; QMD/memsearch prove SQLite is enough — see ADR-006/008).
|
||
- **Query output**: returns **path + summary + matched tags** (option C) — tags are cheap and
|
||
show *why* a result matched, useful for cross-client queries.
|
||
- **Status**: **Superseded by ADR-010** (Graphify replaces the Ruby/SQLite tag index). The
|
||
`summary` + namespaced-tag frontmatter this ADR introduced is **retained as note metadata**;
|
||
only the bespoke Ruby/SQLite index and its CLI are dropped.
|
||
|
||
## ADR-005 — Structured-first; semantic search over the vault deferred
|
||
|
||
- **Context**: Tag filtering ("client/sesame3g + tool/semrush") may miss notes whose wording
|
||
doesn't match the query ("how do we use semrush" vs a note titled "search analytics
|
||
integration").
|
||
- **Decision**: Ship the knowledge layer **structured-only** (tags + summaries). **Defer**
|
||
meaning-based search over the vault until it demonstrably bites.
|
||
- **Rationale**: Structured tagging is the lightweight/fast thing the user wants, and the
|
||
summary+tag design is built to make it work. Follow the video's "only level up when it bites."
|
||
- **Status**: **Superseded by ADR-010.** The premise (ship structured-only, bolt on semantics
|
||
later) no longer holds: Graphify makes the knowledge layer a graph from day one, giving
|
||
structured *and* connection-based recall together. The "only level up when it bites" instinct
|
||
carries forward to whether a *vector* layer is ever needed on top of the graph.
|
||
|
||
## ADR-006 — QMD as the (deferred) semantic-over-knowledge layer
|
||
|
||
- **Context**: When ADR-005's structured-only proves insufficient, we want a set-and-forget
|
||
semantic layer over the vault, local and Docker-free.
|
||
- **Decision**: Earmark **QMD** (github.com/tobi/qmd) for that role; do **not** install yet.
|
||
- **Rationale**: Local markdown search using **SQLite + FTS5/BM25 + local vector embeddings
|
||
(EmbeddingGemma-300M GGUF) + LLM rerank**; CLI + optional **MCP server**; no Docker, no API
|
||
keys. Validates that SQLite + a local vector model suffices (no Milvus/Postgres for
|
||
knowledge). Complements the tag index (QMD filters by path/collection context, not
|
||
first-class frontmatter tags), so it adds semantic recall without replacing structured
|
||
filtering.
|
||
- **Alternatives rejected**: Pointing memsearch at the vault (mixes episodic and knowledge
|
||
corpora; its filtering is path-prefix not tags). A bespoke embedding index (reinvents QMD).
|
||
- **Status**: **Superseded by ADR-010.** Graphify's knowledge graph fills the
|
||
semantic-recall role (traversal/`explain` over connections) without a separate vector system,
|
||
so QMD is no longer earmarked. Revisit a vector layer only if graph traversal demonstrably
|
||
misses cases where embedding similarity would win.
|
||
|
||
## ADR-007 — Lazy freshness: write-hook + session-start reconcile, no daemon/cron
|
||
|
||
- **Context**: The cache must reflect new/edited/deleted/renamed notes without becoming a
|
||
resource hog or going stale on renames.
|
||
- **Decision**: **Option A (lazy).** A `PostToolUse` hook updates the index on **AI** writes
|
||
(single-file, prunes on delete). **Manual** edits are caught by a **session-start reconcile**
|
||
(`index update --since` + prune of vanished paths). **No daemon, no cron.**
|
||
- **Rationale**: The AI is the primary writer, so write-time hooks give event-driven freshness
|
||
with no polling. The user rarely edits the vault by hand, so a session-start reconcile is
|
||
enough; a continuous `inotify` daemon (the `listen` gem) would add an always-on process to
|
||
manage/sync for negligible benefit. Matches the user's "lazy sync is fine" stance.
|
||
- **Alternatives rejected**: `inotify`/`listen` daemon (live freshness, but always-on process
|
||
to manage — unnecessary). Cron reconcile ("seems silly" per user; session-start covers it).
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-008 — Markdown-as-truth; sync the vault, not the indexes
|
||
|
||
- **Context**: Must be accessible on a VPS / multiple machines but run local-fast (Goal 4).
|
||
- **Decision**: Sync the **markdown vault** to the VPS via **git or Syncthing** (choice deferred
|
||
to build time). **Graphs/indexes (Milvus Lite, Graphify `graphify-out/`) are rebuilt per
|
||
machine and never synced.**
|
||
- **Rationale**: Markdown is plain text — git/Syncthing sync it trivially; lazy (hourly or
|
||
continuous-async) is enough. Indexes are disposable caches; syncing binary DBs invites
|
||
conflicts for no gain. Local reads stay fast; ownership and portability stay with the user.
|
||
- **Alternatives rejected**: **OpenBrain / Mem0** hosted DBs — always-remote, adds per-query
|
||
latency and monthly cost, conflicts with local-fast; ownership weaker (Mem0 especially).
|
||
Only worth it for real-time cross-tool memory, which the user called overkill.
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-009 — Package as a global Claude Code plugin with skills
|
||
|
||
- **Context**: Every project, on every machine, should know how to use the vault — write
|
||
conventions, query patterns, the hooks, and the CLI — without per-project setup.
|
||
- **Decision**: Ship hooks + scripts + CRUD know-how as a **global Claude Code plugin with
|
||
skills**, installed at the user level.
|
||
- **Rationale**: Skills carry the "when to write / what conventions / how & when to query"
|
||
guidance to the model; the plugin registers the session-start / session-end / PostToolUse
|
||
hooks and wires up Graphify (extraction/update/query + MCP server). Global install =
|
||
consistent behavior everywhere; single source of truth for the conventions themselves.
|
||
- **Status**: Accepted (to be built — see 04-build-plan.md).
|
||
|
||
## ADR-010 — Graphify knowledge graph as the knowledge layer (supersedes ADR-004/005/006)
|
||
|
||
- **Context**: ADR-004 specced a hand-built Ruby/Sequel/SQLite tag index (+ CLI) as the
|
||
machine-queryable layer over the vault, with ADR-005/006 deferring meaning-based recall to a
|
||
future QMD vector layer. Before building any of it, we evaluated **Graphify** (`graphify`,
|
||
PyPI `graphifyy`) — a tool that turns a folder into a queryable knowledge graph (local
|
||
tree-sitter AST for code, local-SLM entity/relationship extraction for docs). See
|
||
`06-graphify-evaluation.md`.
|
||
- **Decision**: Use **Graphify as the knowledge-layer engine** over the vault, with a **local
|
||
Ollama** backend for doc extraction and free AST for per-project code graphs. **Drop** the
|
||
Ruby/SQLite tag-index CLI (ADR-004) and the earmarked QMD layer (ADR-006); **retain** the
|
||
`summary` + namespaced-tag frontmatter from ADR-003/004 as note metadata.
|
||
- **Rationale**: One off-the-shelf tool delivers both what the tag index was for (structured
|
||
retrieval) and what QMD was deferred for (connection/meaning-based recall via graph
|
||
traversal + `explain`) — without writing or maintaining a bespoke index, and without a vector
|
||
store. Code graphs come free. Keeps the markdown-as-truth, no-Docker, no-API-key, local-first
|
||
properties (extraction runs against local Ollama). Net scope reduction: the entire Ruby build
|
||
(old critical-path Step 2) and the QMD layer are removed.
|
||
- **What's retained / changed**: `summary` stays the human-written router hint Graphify does not
|
||
generate; namespaced tags stay useful for Obsidian filtering and as node attributes. How
|
||
tightly metadata should feed graph queries is a **build-time refinement**, not settled here.
|
||
- **Trade-off accepted**: Graphify's `--update` doesn't prune deleted nodes (stale-node drift) —
|
||
mitigated by a periodic `--force` rebuild on the session-start staleness check (ADR-007's lazy
|
||
model still applies). Graphify also moves fast (flags are version-dependent; anchored to
|
||
v0.8.30) and its headline token-savings numbers are corpus-dependent — benchmark our own.
|
||
- **Alternatives rejected**: Building the Ruby/SQLite index as originally planned (more code to
|
||
own; no semantic recall); adding QMD as a second system on top (two stores where one graph
|
||
suffices).
|
||
- **Status**: Accepted (to be built — see 04-build-plan.md and 06-graphify-evaluation.md).
|
||
|
||
## ADR-011 — Faceted tag taxonomy: six independent namespaces (refines ADR-003)
|
||
|
||
_Date: 2026-06-04_
|
||
|
||
- **Context**: ADR-003 introduced five namespaces (`tool/`, `client/`, `domain/`,
|
||
`convention/`, `scope/`). During vault-reuse assessment (ADR-012) it became clear that (1)
|
||
the existing SecondBrain vault uses a de-facto first-tag convention for note kind
|
||
(research/plan/log/adr/howto) that should be made explicit and machine-queryable, and (2)
|
||
for a freelancer working many projects per client, project identity deserves a first-class
|
||
namespace rather than being implied by `client/` or `domain/`.
|
||
- **Decision**: Knowledge-vault notes are classified by **six independent, flat tag facets**
|
||
that sit side-by-side, never nested into one another:
|
||
- `type/` — note kind: `research`, `howto`, `adr`, `hub`, `plan`, `log`, `clip`, etc.
|
||
- `client/` — which client
|
||
- `project/` — which project (first-class; a freelancer's projects are the primary unit of
|
||
work)
|
||
- `domain/` — knowledge domain / topic area
|
||
- `tool/` — tool-specific knowledge
|
||
- `convention/` — conventions
|
||
- …plus `scope/global` or `scope/project` (retained from ADR-003)
|
||
|
||
Hierarchy and relationships are expressed via **hub notes** (`type/hub`), **wikilinks**, and
|
||
**Graphify knowledge-graph edges** — NOT via nested tag paths.
|
||
|
||
By convention `type/` is listed **first** in frontmatter, preserving the SecondBrain vault's
|
||
existing type-first ordering habit and making the note kind immediately visible.
|
||
- **Rationale**: The vault is flat — hierarchy is not expressed through folder paths or tag
|
||
nesting. The user's reality is many-to-many (many projects per client, knowledge domains
|
||
spanning clients), which a single-parent tree models badly and forces false hierarchy. A
|
||
project hub note links out to both its `client/` and relevant `domain/` tags rather than
|
||
being buried under either. Per-type `_templates` will be provided for **core types only**
|
||
(research, howto, adr, hub); the long tail stays freeform until a pattern earns a template.
|
||
Consistent per-type structure also improves Graphify's local-SLM extraction reliability.
|
||
- **Alternatives rejected**: Hierarchical nesting in the style of John Conneely's
|
||
`domain/{product}/{project}.md` folder structure (from the youngleaders.tech article "How I
|
||
finally sorted my Claude Code memory" — **secondary/interview-grade source, not verified
|
||
against primary implementation**). Rejected because: (1) the vault is flat — hierarchy is
|
||
not expressed through folder paths; (2) the user's many-to-many reality maps badly onto a
|
||
single-parent folder tree and forces false hierarchy; (3) nesting one facet through another
|
||
(e.g. `domain/client/project`) creates Law-of-Demeter-style traversal coupling. Conneely's
|
||
structure was the inspiration but diverges here on hierarchy-vs-facets. Faceted parallel tags
|
||
are the flat-vault analogue of what the Graphify graph already does with edges, so they
|
||
compose naturally with the chosen knowledge layer.
|
||
- **Status**: Accepted (supersedes the namespace list in ADR-003; core flat-vault +
|
||
namespaced-tags decision stands).
|
||
|
||
## ADR-012 — Reuse the existing SecondBrain vault as the knowledge vault
|
||
|
||
_Date: 2026-06-04_
|
||
|
||
- **Context**: The design called for a flat markdown vault as the semantic knowledge layer
|
||
(ADR-003/008/010). The question was whether to stand up a new `~/brain` vault from scratch
|
||
or adopt the existing `~/Documents/SecondBrain` vault.
|
||
- **Decision**: **Adopt `~/Documents/SecondBrain`** as the knowledge vault rather than
|
||
creating a new vault.
|
||
- **Rationale**: Assessment found the SecondBrain vault is already flat (all notes at root,
|
||
only a `_templates/` exception — exactly what the design permits), already articulates the
|
||
correct "durable knowledge, not working memory" role in its `CLAUDE.md` and
|
||
`vault-conventions.md`, and contains ~20 real notes. It also includes two patterns that
|
||
**improve on the current design** and should be adopted:
|
||
1. `vault-conventions.md`'s "act without being asked" section specifying *when* the AI
|
||
should proactively query the vault — a behavioral spec the cc-os docs lacked.
|
||
2. Project-config hub notes with a tag-inference table (auto-tag by path pattern) that
|
||
operationalizes *how* to tag a note from a given project.
|
||
- **Adaptations required (migration cost)**:
|
||
- Add `summary:` frontmatter to existing notes.
|
||
- Migrate flat unnamespaced tags to the six-facet namespaced form (per ADR-011).
|
||
- Add `scope/global` or `scope/project` to each note.
|
||
- Initialize git in the vault (no `.git` exists yet — required by ADR-008's sync strategy).
|
||
- Replace the vault's `~/.claude/scripts/vault_search.rb` reference (script does not exist)
|
||
with `graphify query` (ADR-010).
|
||
These are mechanical schema migrations, not structural rework.
|
||
- **Alternatives rejected**: Starting fresh with a new `~/brain` vault. Rejected because the
|
||
hardest design decision — flat structure, durable-knowledge-only role, governance philosophy
|
||
— is already made and practiced in SecondBrain. The improved behavioral patterns
|
||
(proactive-query spec, tag-inference table) and the existing notes are worth preserving; the
|
||
remaining work is mechanical migration.
|
||
- **Status**: Accepted.
|
||
|
||
## ADR-013 — Build-first / migrate-incrementally (build-order inversion)
|
||
|
||
_Date: 2026-06-04_
|
||
|
||
- **Context**: The build runbook (`05-implementation-process.md`) originally front-loaded bulk
|
||
vault migration as Step 1 — migrating all ~20 existing SecondBrain notes and all projects to
|
||
the ADR-011 six-facet taxonomy before the system existed to validate them. This committed to
|
||
a schema and workflow (the tag taxonomy from ADR-011, the vault-reuse choice from ADR-012,
|
||
and Graphify extraction behavior) before any end-to-end path had been exercised. The risk:
|
||
locking in an approach that fails at scale, with no feedback loop until the entire vault has
|
||
been touched.
|
||
- **Decision**: **Invert the build order.** The full system is built and validated against a
|
||
small **5–10 note fixture set** first. Bulk vault migration is deferred to the final stage.
|
||
The first real-data validation uses **one small project that contains both code AND
|
||
documents**, exercising both the local-SLM doc-extraction path and the tree-sitter code path
|
||
in the same run. After that single project validates end-to-end, remaining projects are
|
||
onboarded **one at a time** with an observe-and-adjust step between each.
|
||
- **Rationale**: Validates the ADR-011 taxonomy and ADR-012 vault conventions against the real
|
||
Graphify extraction pipeline before the entire vault is committed. The first mixed code+docs
|
||
project surfaces both extraction paths (SLM for docs, tree-sitter for code) early, when
|
||
corrections are cheap. Per-project rollout keeps the blast radius of any schema or workflow
|
||
correction small; each project is an opportunity to observe and adjust rather than discover
|
||
problems across 20 notes at once. This is consistent with the "markdown-as-truth, indexes are
|
||
disposable" principle (ADR-008): the vault notes are durable, but the extraction schema should
|
||
be validated before it shapes all of them.
|
||
- **Alternatives rejected**:
|
||
- **Keep migration-first (status quo)**: Front-loads all ~20 notes and all projects before
|
||
any end-to-end validation exists. Commits to ADR-011's taxonomy and ADR-012's migration
|
||
steps against the full vault without a feedback loop — exactly the gap this decision closes.
|
||
- **Big-bang migrate everything after build**: Build against fixtures, then migrate all notes
|
||
and all projects in one batch at the end. Avoids the pre-build commitment problem but still
|
||
risks a single large irreversible migration with no observe-and-adjust loop between units.
|
||
Per-project rollout with intermediate checkpoints is strictly safer.
|
||
- **Cross-references**: ADR-011 (six-facet tag taxonomy — the schema being validated);
|
||
ADR-012 (SecondBrain vault reuse — the migration steps this order defers).
|
||
- **Status**: Accepted (updates `05-implementation-process.md` build order).
|
||
|
||
## ADR-014 — Graph connectivity comes from authored structure; migration scaffolding is a first-class prerequisite
|
||
|
||
_Date: 2026-06-05_
|
||
|
||
- **Context**: ADR-011 specified hub notes + wikilinks + Graphify graph edges as the mechanism
|
||
for expressing hierarchy and cross-note relationships, with ADR-013 deferring bulk vault
|
||
migration to the final stage. Before build began, a discriminating empirical test compared a
|
||
cached-replay graph (per-fixture isolated extractions) against a clean single-pass deep
|
||
extraction (`graphify extract ~/Documents/SecondBrain --backend ollama --model
|
||
qwen25-coder-7b-16k --max-concurrency 1 --token-budget 8000 --mode deep --exclude
|
||
.obsidian`) against the real `~/Documents/SecondBrain` vault under Graphify 0.8.31 +
|
||
qwen2.5-coder:7b. See `07-graph-connectivity-findings.md` for the full data and methodology.
|
||
[primary/measured — 2026-06-05 session]
|
||
- **Decision**: **The connective spine of the knowledge graph must be author-provided.** Hub
|
||
notes and wikilinks are not optional scaffolding to add "someday" — they are the mechanism
|
||
by which Graphify connects thematically related notes, and they must be authored **as part of
|
||
the migration step**, not deferred to bulk import. Migration scaffolding (hub notes +
|
||
wikilinks for key concepts) is treated as a **first-class build deliverable** in the
|
||
migrate-incrementally phase of ADR-013.
|
||
- **Rationale**: The empirical test found that Graphify is a **structure extractor, not a topic
|
||
clusterer**. Even at `--mode deep --token-budget 8000`, no emergent shared-topic hub nodes
|
||
appeared (no "Pest Control" node, no "Niche Prospecting" node). All cross-note edges observed
|
||
came from explicit references, wikilinks, or document-level semantic similarity — not from
|
||
shared thematic identity. A practical test query ("how do we do niche prospecting outreach for
|
||
pest control?") returned 3 starting notes and traversal could not reach the email templates /
|
||
ACV data / business-model notes (separate communities, no connecting edges). This confirms
|
||
that useful retrieval is gated on migration scaffolding, not on Graphify's extraction power.
|
||
The clean single-pass run also showed the cached graph was partially a build artifact (cross-
|
||
note edges rose from 41% to 78% in a single-pass run), but the structural finding — no
|
||
emergent hub nodes — held in both runs.
|
||
- **Relationship to ADR-011**: Validates the hub-notes + wikilinks half of ADR-011 empirically.
|
||
The facet-tag half is **not yet validated**: no edge was observed to arise from shared
|
||
frontmatter facet tags alone. Whether `client/X` or `tool/Y` tags create graph connectivity
|
||
is an **open question** — see "Deferred" below. Do not assume facet tags contribute to graph
|
||
traversal retrieval until tested.
|
||
- **Relationship to ADR-013**: Refines the migrate-incrementally stage. "Migration" must be
|
||
defined to include hub note authoring and wikilink addition for key concepts, not just
|
||
frontmatter schema migration (adding `summary:` and namespaced facet tags). The build plan
|
||
(`04-build-plan.md`) should be updated to name this deliverable explicitly.
|
||
- **Alternatives rejected**: Relying on the SLM to auto-cluster topics and synthesize hub
|
||
entities — **empirically does not happen** at 7B model size with `--mode deep`. The design
|
||
already intended human-authored hub notes for this; the test confirms that intent was correct
|
||
and the fallback assumption ("maybe the LLM will do it") is false.
|
||
- **Deferred**:
|
||
1. **Facet-tag-to-graph-edge question**: Do shared frontmatter facet tags (`client/`,
|
||
`tool/`, `domain/`, etc.) cause Graphify to create edges between notes, or does graph
|
||
connectivity come only from explicit wikilinks/references and semantic similarity? This was
|
||
NOT tested. Resolve before designing graph-traversal retrieval skills.
|
||
2. **Larger extraction model**: Whether a substantially larger SLM (14B, 30B) would
|
||
synthesize emergent topic-hub nodes is untested. Secondary — the design does not depend on
|
||
it — but worth one test run before the build ships.
|
||
3. **`reasoning_effort:"none"` patch**: The clean run required a local patch to `graphify/
|
||
llm.py`. Track the upstream Graphify issue tracker for an official fix; treat the
|
||
installed version as pinned until resolved.
|
||
- **Status**: Accepted. Refines ADR-013 (migrate-incrementally phase scope) and empirically
|
||
validates the hub-notes/wikilinks mechanism of ADR-011 while flagging its facet-tag half as
|
||
an open question.
|
||
|
||
## ADR-015 — memsearch episodic memory version-controlled in a dedicated private repo, auto-synced via cc-os SessionEnd hook
|
||
|
||
_Date: 2026-06-09_
|
||
|
||
- **Context**: memsearch's memory store at `~/.memsearch` accumulates daily session-summary
|
||
files (`memory/YYYY-MM-DD.md`) that are the only irreplaceable data in the episodic layer —
|
||
the Milvus Lite index (`milvus.db`) and the bge-m3 ONNX embeddings model are derived/
|
||
disposable and can be rebuilt at any time via `memsearch index`. With Step 4 (episodic layer)
|
||
now live, preserving episodic memory across machines and protecting against local disk loss
|
||
required a sync strategy. The question was: what venue, what scope, and who owns the sync?
|
||
memsearch's own stated design philosophy is "markdown files are the canonical data store; the
|
||
vector database is a derived index" and notes that markdown is "git-friendly" and the index
|
||
rebuildable from markdown — making git a natural fit for the markdown layer.
|
||
- **Decision**: `~/.memsearch` is a **dedicated git repo** on branch `main` with remote `origin`
|
||
pointing to a **private self-hosted Forgejo repo**
|
||
(`ssh://git@forgejo.swansoncloud.com:2222/jared/memsearch.git`;
|
||
web: `https://forgejo.swansoncloud.com/jared/memsearch`). A whitelist `.gitignore` tracks
|
||
**only** `memory/*.md` (the daily session files) and `.gitignore` itself. Excluded as
|
||
derived/disposable: `milvus.db` (Milvus Lite index — rebuildable any time via
|
||
`memsearch index`), `config.toml`, and the bge-m3 ONNX embeddings model (lives in
|
||
`~/.cache/huggingface`, not in the store). Auto-commit and push are wired into the **cc-os
|
||
memory plugin's own `session-end.sh`** hook, not the marketplace plugin. The appended block
|
||
guards on `~/.memsearch/.git` existing, runs `git add -A` (whitelist makes it safe), commits
|
||
only when something is staged (message: `memsearch: session memory <date>`), and pushes with
|
||
`timeout 30 ... || true`. The entire block is wrapped in a subshell with a trailing `|| true`
|
||
so it can **never fail session shutdown**.
|
||
- **Rationale**:
|
||
- **Dedicated repo (not folded into the vault or a project repo):** ADR-001 established
|
||
episodic and semantic/knowledge as separate systems by design, and `~/.memsearch` is a
|
||
global, cross-project store with no natural home in any single project repo. Committing
|
||
episodic session logs into the Obsidian vault repo would conflate the two systems and
|
||
violate the separation of concerns that ADR-001 is built on.
|
||
- **Whitelist — commit markdown, exclude rebuildable index/model/config:** consistent with
|
||
memsearch's own design philosophy ("markdown is canonical; index is derived") and with the
|
||
cc-os principle from ADR-008 ("sync the vault, not the indexes"). The 544 MB bge-m3 ONNX
|
||
model is not even in the store; the Milvus Lite DB rebuilds in one command. Committing them
|
||
would bloat the repo for zero durability gain.
|
||
- **Sync lives in cc-os hook, not the marketplace plugin:** the marketplace plugin's hook
|
||
scripts (`stop.sh`, `session-start.sh`, `user-prompt-submit.sh`, `session-end.sh`) perform
|
||
no git operations — only a read-only `git rev-parse --show-toplevel` for scoping. Adding
|
||
sync to a marketplace plugin that can be clobbered by an upstream update is fragile; owning
|
||
it in the cc-os `session-end.sh` keeps the sync logic under our control and version-tracked
|
||
in this repo.
|
||
- **Fail-safe contract:** the `|| true` wrap and `timeout 30` on push ensure that a network
|
||
outage, an unreachable Forgejo instance, or any git error cannot prevent a session from
|
||
closing cleanly. The SessionEnd hook's harness timeout is ~10 s, so push is effectively
|
||
capped; commits land locally first, and any unpushed commit is carried forward by the next
|
||
session's push (daily files are append-only, so no conflict risk).
|
||
- **Commingling — resolved 2026-06-09**: `~/.memsearch` aggregates session summaries across
|
||
**all clients** into one global store. This is an **explicit design choice**, not a tolerated
|
||
risk: the user deliberately accepts a single commingled global store across all clients.
|
||
Private self-hosted Forgejo (user-owned infrastructure) is the chosen sync venue; client-
|
||
agreement compliance is the user's knowing responsibility. The concern was previously recorded
|
||
as user-owned and unresolved; that is now closed: single global store is the intended design.
|
||
This also aligns with a forward direction of minimizing dedicated per-client working
|
||
directories — since memsearch captures memory globally regardless of cwd, a single
|
||
`clients/` area for non-project client work becomes viable (direction stated; not yet designed).
|
||
- **Alternatives rejected**:
|
||
- **Fold into the Obsidian vault repo:** violates ADR-001's episodic/semantic separation.
|
||
Episodic logs have different lifecycle (accrete and decay), different write patterns
|
||
(auto-captured every session), and would clutter a vault intended for curated durable
|
||
knowledge.
|
||
- **Auto-commit in the marketplace plugin:** the marketplace plugin can be overwritten on
|
||
update, losing the sync logic silently. Out-of-band ownership in cc-os is safer.
|
||
- **Commit the Milvus Lite index or the embeddings model:** both are large binaries, derived
|
||
from the markdown source, and rebuildable. Committing them wastes space and provides no
|
||
additional durability. The markdown files are the canonical source; the index follows from
|
||
them.
|
||
- **Syncthing or rsync instead of git:** git provides both version history and conflict-free
|
||
daily-append semantics; Syncthing is continuous-async (suitable for the vault where changes
|
||
are sparse); git's push-on-session-end cadence matches how memsearch produces data (one
|
||
daily file per day, append-only). Git was already chosen for ADR-008's vault sync rationale;
|
||
applying the same mechanism to the episodic store is consistent.
|
||
- **Status**: Accepted (commingling resolved 2026-06-09). Initial commit `106cebc` in the
|
||
Forgejo repo; git repo initialized 2026-06-09 via the `git-context:repo-init` skill. The
|
||
memsearch auto-commit+push was relocated to a dedicated SessionEnd hook entry
|
||
(`memsearch_sync.py`) by ADR-016, 2026-06-12; behavior preserved.
|
||
|
||
## ADR-016 — Memory plugin sourced from cc-os git repo; bash→Python deep-module port; memsearch-sync split; symlink cutover
|
||
|
||
_Date: 2026-06-12_
|
||
|
||
- **Context**: The global Claude Code memory plugin was developed iteratively in-place at
|
||
`~/.claude/plugins/memory/` — never tracked in version control. Four bash hook scripts
|
||
(`session-start.sh`, `session-context.sh`, `post_tool_use_write.sh`, `session_end.sh`) each
|
||
independently parsed YAML config, decoded stdin JSON, and referenced shared filesystem
|
||
contracts (e.g. a temp file keyed by `$SESSION_ID`) via ad-hoc inline code with no
|
||
abstraction boundary. Additionally, the memsearch auto-commit+push block (ADR-015 behavior)
|
||
was embedded in `session_end.sh` alongside the vault journal logic. The plugin was untracked
|
||
and fragile; the bash scripts had no shared modules, no tests, and a jq-vs-python3
|
||
inconsistency across hooks. An OpenSpec change (`memory-plugin-source-and-port`) was
|
||
initiated to move the plugin into git and port the hooks to Python.
|
||
- **Decision (1) — Source location**: The canonical source for the global memory plugin is now
|
||
`cc-os/plugins/memory/` (this repo, git-tracked). `~/.claude/plugins/memory/` is a symlink
|
||
pointing to `cc-os/plugins/memory/`; it is no longer an independent directory.
|
||
- **Decision (2) — Module shape: deep-module Python architecture**: The four bash hooks were
|
||
ported to Python as a **deep-module** design — three shared modules providing clean, stable
|
||
abstractions, plus thin entry-point scripts:
|
||
- `config.py`: `load_config() -> Config` frozen dataclass — handles YAML parse, path
|
||
expansion, defaults, missing-file case. Replaces 3 separate inline YAML parsers.
|
||
- `hook_io.py`: `read_input() -> HookInput` dataclass (session_id, cwd, file_path, reason)
|
||
— replaces 4 separate inline stdin parsers and eliminates the jq-vs-python3 inconsistency.
|
||
- `session_state.py`: `record_touch(session_id, path)` / `read_touches(session_id) ->
|
||
list[str]` — encapsulates the `/tmp/claude-vault-touched-$SESSION_ID` temp-file contract.
|
||
The temp file still exists by necessity (separate process invocations); the win is an
|
||
explicit, named, testable contract.
|
||
- Thin entry-point scripts: `hooks/session_start.py`, `hooks/session_context.py`,
|
||
`hooks/post_tool_use_write.py`, `hooks/session_end.py`, `hooks/memsearch_sync.py`.
|
||
- Every `main()` is wrapped in `try/except`: uncaught exceptions log to stderr and `exit 0`
|
||
(fail-open invariant — a hook crash cannot block a session).
|
||
- **Decision (3) — memsearch-sync split**: The memsearch auto-commit+push block from ADR-015
|
||
was extracted out of `session_end.py` (vault journal) into its own dedicated SessionEnd hook
|
||
entry `memsearch_sync.py`, registered as a separate hook in `~/.claude/settings.json` with a
|
||
30-second timeout. This is a **relocation of ADR-015 behavior, not a reversal** — the
|
||
auto-commit+push logic, whitelist `.gitignore`, fail-safe `|| true` contract, and Forgejo
|
||
remote are byte-for-byte preserved. The benefit is separation of concerns: vault journal
|
||
failures and memsearch sync failures are now independently observable and independently
|
||
fail-open.
|
||
- **Decision (4) — Deployment-mechanism deviation (symlink)**: The OpenSpec plan described
|
||
repointing the `local-plugins` marketplace source path to `cc-os/plugins/memory/`. In
|
||
practice, there is no `marketplace.json` registry manifest to repoint — the marketplace's
|
||
`known_marketplaces.json` manages marketplace sources, not per-plugin paths, and the hook
|
||
scripts are entirely decoupled from the marketplace (they are registered in `settings.json`
|
||
by hardcoded absolute path). The functional cutover was therefore accomplished via two
|
||
actions: (1) `~/.claude/settings.json` hook entries were rewritten to invoke `/usr/bin/python3`
|
||
with absolute paths into `cc-os/plugins/memory/hooks/`, which is the real behavioral cutover;
|
||
(2) a symlink `~/.claude/plugins/memory → cc-os/plugins/memory/` was created so that skill
|
||
resolution (which follows the plugins directory) continues to resolve correctly. Pre-cutover
|
||
backups were taken: `~/.claude/settings.json.bak-precutover-2026-06-12`,
|
||
`~/.claude/known_marketplaces.json.bak-precutover-2026-06-12`, and
|
||
`~/.claude/plugins/memory.bak-bash-precutover/`.
|
||
- **Rationale**:
|
||
- **Source in git**: an untracked plugin is invisible to review, diff, and rollback; living in
|
||
cc-os gives change history, golden fixtures, and CI-comparable testing.
|
||
- **Deep-module over OO hook-class hierarchy**: four hooks that run as separate processes
|
||
with no shared runtime have nothing to gain from a class hierarchy — inheritance is just
|
||
complexity. Shared modules with named functions and typed dataclasses give all the benefits
|
||
(single parse path, named contracts) with no class machinery. The three modules provide a
|
||
deep public surface (one function call = full parse + validation) behind a thin API, which
|
||
is the Ousterhout deep-module criterion.
|
||
- **memsearch-sync split**: the original monolithic `session_end.sh` mixed two distinct
|
||
concerns (vault journal vs external git push). Split entry points mean each has its own
|
||
timeout budget, its own failure domain, and its own log line.
|
||
- **Symlink + settings.json rewrite**: the symlink is the minimum viable mechanism consistent
|
||
with how Claude Code actually resolves plugins. Attempting to retool `known_marketplaces.json`
|
||
for per-plugin path overrides would have required reverse-engineering an undocumented format;
|
||
direct hook path rewriting in settings.json is explicit, transparent, and reversible.
|
||
- **Alternatives rejected**:
|
||
- **OO hook-class hierarchy**: a base `Hook` class with subclasses per hook type. Rejected
|
||
because hooks run in separate processes — there is no shared runtime state to inherit.
|
||
Shared logic belongs in functions, not class trees.
|
||
- **Keep bash, just move into git**: bash lacks typed dataclasses, structured exceptions, and
|
||
unit-testable modules. The jq-vs-python3 inconsistency (pre-existing bug) would be carried
|
||
forward unchanged, and the temp-file contract would remain implicit.
|
||
- **Repoint `known_marketplaces.json`**: no per-plugin path override semantics confirmed in
|
||
the actual marketplace format; would require undocumented hacking with no rollback path.
|
||
- **Consequences / ongoing contracts**:
|
||
- `~/.claude/plugins/memory` is a symlink; do not replace it with a directory (e.g. on plugin
|
||
reinstall) without checking cc-os source first.
|
||
- `~/.claude/settings.json` hook entries now invoke Python 3 via absolute path into cc-os;
|
||
keep these entries current when hook filenames change.
|
||
- The fail-open invariant (`exit 0` on any unhandled exception) is a hard contract: never
|
||
wrap a hook's `main()` in code that propagates exceptions to the harness.
|
||
- A fresh-session cutover test was run 2026-06-12 and passed: all five hooks fired from the
|
||
cc-os Python sources.
|
||
- **Cross-references**: ADR-009 (global plugin design); ADR-015 (memsearch sync — relocated,
|
||
not reversed).
|
||
- **Status**: Accepted. Cutover verified 2026-06-12.
|
||
|
||
|
||
## ADR-017 — Project graph onboarding assesses the repo and writes `.graphifyignore` before extracting
|
||
|
||
_Date: 2026-06-17_
|
||
|
||
- **Context**: The `/memory:memory-project` skill's onboarding flow ran `graphify extract . --backend ollama --model qwen2.5-coder:7b` directly. graphify does NOT honor `.gitignore`; it uses a separate `.graphifyignore` file (same syntax — see `docs/graphify/02-installation-setup.md`). With no `.graphifyignore`, onboarding `viking-warrior-training-log` (135 tracked files) walked `node_modules/` (5,858 files / 161MB) and routed every non-code file there through the Ollama doc pass. The run was killed after ~1 hour with no `graph.json` produced. Cost driver: only non-code files hit the Ollama LLM; code uses the free tree-sitter AST pass.
|
||
- **Decision**: Onboarding is now assessment-first. Before running `graphify extract`, the skill surveys the repo, classifies directories and file types into include/exclude (weighing non-code file count since only non-code files cost LLM time), confirms the proposed ignore list with the user, writes `.graphifyignore` at the repo root, ensures `graphify-out/` is in the project's `.gitignore`, then runs extract. The ignore list is generated per-project — what counts as noise varies by stack — not from a static template.
|
||
- **Rationale**: graphify's `.gitignore` blindness is by design; the correct lever is `.graphifyignore`, not a workaround. Assessment before extraction surfaces borderline dirs (migrations, seeds, fixtures, sample data) that need a human call. Per-project generation avoids the false safety of a shared template that silently mismatches a new stack.
|
||
- **Alternatives rejected**: **Static `.graphifyignore` template** — rejected; dependency/build/cache dirs vary by project stack; a template would both over-exclude (suppressing wanted source) and under-exclude (missing stack-specific dirs) for any given project. **Auto-write without user confirmation** — rejected; borderline directories (e.g. migrations, seeds, fixtures) require human judgment; mechanical exclusion would silently drop legitimate content. **AST-only mode (skip doc pass entirely)** — deferred; the `--backend ollama` flag stays; the ignore list is the correct cost lever, not backend switching.
|
||
- **Status**: Accepted 2026-06-17.
|
||
|
||
**Addendum — 2026-06-17:**
|
||
- **Default-exclude taxonomy is type-based, not name-based.** The skill's assessment step organizes exclude candidates into 11 categories by KIND (fetched deps, build/generated output, caches, VCS internals, editor/AI-tooling dirs, lockfiles, coverage/logs, bulk data/databases, binaries, secrets, graphify-out/) with a meta-principle: index what a human authored; exclude anything fetched, generated, cached, tooling config, bulk data/binary, or secret. Example names within each category are illustrative, not exhaustive — the goal is to recognize the kind even in an unfamiliar stack. See SKILL.md for the full table.
|
||
- **Onboarding uses the 16k-context model.** The `graphify extract` command now passes `--model qwen25-coder-7b-16k` (the `ollama_model` from config.yaml), a 16k-context build of qwen2.5-coder:7b. Inference is GPU-bound (~65 tok/s); the main speed lever is chunk-count × generation — fewer chunks per doc via a larger context window. `GRAPHIFY_OLLAMA_NUM_CTX` does not propagate through graphify's OpenAI-compatible endpoint; the larger context must come from the model's Modelfile.
|
||
|
||
## Rejected tools (summary)
|
||
|
||
| Tool | Why rejected for our use |
|
||
|------|--------------------------|
|
||
| MemPalace (L4) | Storage not readable markdown; isolated drawers (knowledge not interconnected); fights self-managing + cross-linking goals |
|
||
| Recall / LightRAG (L5) | Content knowledge bases / deep research, not operational memory; Recall = hosted, you don't own data; LightRAG = enterprise overkill |
|
||
| OpenBrain / Mem0 (L6) | Always-remote DB → latency + cost; conflicts with local-fast lazy-sync; only pays off for real-time cross-tool memory (user: overkill) |
|
||
| Postgres / Milvus server | Unnecessary — Graphify's local graph (knowledge) + Milvus Lite (memsearch episodic) cover everything locally with no Docker |
|
||
| claude-mem | MCP-based (Claude must call search); opaque blobs vs readable markdown; feature overkill |
|
||
| Ruby/SQLite tag index CLI; QMD vector layer | Superseded by Graphify before build — one knowledge graph replaces both the structured index and the deferred semantic layer (ADR-010) |
|