Remove handoff doc and pivot to Graphify knowledge layer
The handoff document was a session artifact; it is no longer needed. Update all architecture docs to reflect the design pivot: Graphify knowledge graph supersedes the earlier Ruby/SQLite tag-index CLI and the deferred QMD semantic layer. The vault metadata (summary + tags) is retained; only the bespoke index is removed. Update ADRs 003, 005, and 006 to note they are superseded by ADR-010 (Graphify), and update all system-design doc references from the three-layer model to the final two-layer model (episodic + knowledge).
This commit is contained in:
jared
parent
2bc95ed30e
commit
334d5fc703
|
|
@ -18,22 +18,22 @@ wants the AI to learn things once and reuse them everywhere, without bloating pr
|
|||
4. **[04-build-plan.md](04-build-plan.md)** — How a human builds this, step by step. The
|
||||
scripts, the hooks, the CRUD lifecycle, the AI's write/query conventions, and the
|
||||
Claude Code plugin + skills that package it for global install.
|
||||
5. **[05-handoff.md](05-handoff.md)** — Where we are, what's decided, what's open, and the
|
||||
first concrete actions for the next session.
|
||||
|
||||
## One-paragraph summary
|
||||
|
||||
Two complementary memory types kept as **separate systems**: **episodic** ("what happened,
|
||||
when") handled by **memsearch** (Milvus Lite, embedded, auto-captured session/journal notes),
|
||||
and **semantic/knowledge** ("how do we…") handled by a **flat Obsidian markdown vault** that
|
||||
is the single source of truth, organized by **namespaced nested tags** (`tool/`, `client/`,
|
||||
`domain/`, `convention/`, `scope/`) rather than folders, and indexed by a small **Ruby +
|
||||
Sequel + SQLite** tag index exposed through a CLI. Retrieval is **hook-injected and
|
||||
on-demand** so project repos stay thin. Freshness is **lazy** (a write-time hook plus a
|
||||
session-start reconcile — no daemon, no cron). Everything is markdown-as-truth and syncs to a
|
||||
VPS via **git/Syncthing**; the indexes are disposable and rebuilt per machine. A meaning-based
|
||||
search layer over the vault (**QMD**) is designed-in but **deferred** until structured tagging
|
||||
proves insufficient. The whole thing ships as a **global Claude Code plugin with skills** so
|
||||
every project knows how to use it.
|
||||
is the single source of truth. Every note carries **`summary` + namespaced tags** (`tool/`,
|
||||
`client/`, `domain/`, `convention/`, `scope/`) as first-class metadata — the human-written
|
||||
router hint and the cross-cutting filters that organize the vault without folders. The vault is
|
||||
made queryable by a **Graphify knowledge graph** (local Ollama SLM extracts entities/relations
|
||||
from the notes; free tree-sitter AST builds a code graph per project). Retrieval is
|
||||
**hook-injected and on-demand** so project repos stay thin. Freshness is **lazy** (a write-time
|
||||
hook plus a session-start reconcile — no daemon, no cron). Everything is markdown-as-truth and
|
||||
syncs to a VPS via **git/Syncthing**; the graphs are disposable and rebuilt per machine. The
|
||||
whole thing ships as a **global Claude Code plugin with skills** so every project knows how to
|
||||
use it.
|
||||
|
||||
_Last updated: 2026-06-03_
|
||||
_Last updated: 2026-06-04 · knowledge layer = Graphify knowledge graph (supersedes the earlier
|
||||
Ruby tag-index CLI; metadata frontmatter retained)_
|
||||
|
|
|
|||
|
|
@ -119,4 +119,4 @@ detail only when needed.
|
|||
- MemPalace — github.com/MemPalace/mempalace
|
||||
- Karpathy LLM wiki — gist.github.com/karpathy
|
||||
- Recall — recall.it · Mem0 — mem0.ai · OpenBrain — github.com/NateBJones-Project
|
||||
- QMD (our deferred semantic layer) — github.com/tobi/qmd
|
||||
- QMD — github.com/tobi/qmd (evaluated as a semantic layer; superseded by Graphify — see ADR-010)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
# System Design
|
||||
|
||||
_Status: approved 2026-06-03. Implementation not yet started._
|
||||
_Status: approved 2026-06-03; knowledge layer revised 2026-06-04 (Graphify replaces the Ruby
|
||||
tag-index CLI and the deferred QMD layer — see ADR-010). Implementation not yet started._
|
||||
|
||||
## Goals (what this system must do)
|
||||
|
||||
|
|
@ -24,13 +25,13 @@ manage** (AI-managed), **semi-structured** (organization that can evolve).
|
|||
| Type | Question | Lifecycle | Write path | Our tool |
|
||||
|------|----------|-----------|-----------|----------|
|
||||
| **Episodic** | "What happened, when?" | accretes & decays | auto-captured | **memsearch** |
|
||||
| **Semantic / knowledge** | "How do we…?" | deliberately maintained | curated by you/AI | **Obsidian vault + tag index** |
|
||||
| **Semantic / knowledge** | "How do we…?" | deliberately maintained | curated by you/AI | **Obsidian vault + Graphify knowledge graph** |
|
||||
|
||||
This is the classic **episodic vs. semantic** memory split. Keeping them separate is the key
|
||||
architectural decision — they have different lifecycles, write paths, and query patterns, and
|
||||
forcing one tool to do both is what made every earlier design feel forced.
|
||||
|
||||
## The three layers
|
||||
## The two layers
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
|
|
@ -39,17 +40,19 @@ forcing one tool to do both is what made every earlier design feel forced.
|
|||
│ answers "when did we…", "what was I doing yesterday" │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ KNOWLEDGE ── flat Obsidian vault (single source of truth) │
|
||||
│ + Ruby/Sequel/SQLite tag index (CLI) │
|
||||
│ structured tag filtering · answers "how do we…", "what do we │
|
||||
│ know about X for client Y" │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ SEMANTIC-OVER-KNOWLEDGE ── QMD (SQLite+vector, MCP) [DEFERRED] │
|
||||
│ meaning-based recall over the vault when tags miss │
|
||||
│ add ONLY when structured tagging proves insufficient │
|
||||
│ notes carry summary + namespaced tags (metadata) │
|
||||
│ + Graphify knowledge graph (local SLM over docs; AST over code) │
|
||||
│ graph queries (god nodes / query / path) · answers "how do we…",│
|
||||
│ "what do we know about X for client Y", "what relates to Y" │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
All three are **local-first, markdown-as-truth, no Docker, no server, no API keys.**
|
||||
Both are **local-first, markdown-as-truth, no Docker, no server, no API keys** (Graphify
|
||||
extraction runs against a **local Ollama** model). An earlier design split the knowledge layer
|
||||
into a Ruby/SQLite tag index plus a deferred QMD vector layer; **Graphify replaces both** — it
|
||||
provides structured *and* semantic retrieval over the vault as a single graph (ADR-010). The
|
||||
`summary` + tag frontmatter is **retained** as note metadata (router hint + cross-cutting
|
||||
filters), it is just no longer backed by a bespoke index.
|
||||
|
||||
## Layer 1 — Episodic (memsearch)
|
||||
|
||||
|
|
@ -64,7 +67,7 @@ All three are **local-first, markdown-as-truth, no Docker, no server, no API key
|
|||
("what was decided about X last week"). We do **not** make it filter by our tags — it owns
|
||||
the episodic corpus only.
|
||||
|
||||
## Layer 2 — Knowledge (vault + tag index)
|
||||
## Layer 2 — Knowledge (vault + Graphify knowledge graph)
|
||||
|
||||
The heart of the system, and the part we build.
|
||||
|
||||
|
|
@ -92,51 +95,59 @@ tags:
|
|||
broadly useful tool/domain knowledge; project = how a specific client uses it. Both are
|
||||
globally queryable; the scope tag is the shortcut that avoids scanning every client's usage.
|
||||
|
||||
### Index (the tag cache)
|
||||
A small **Ruby program, Sequel ORM, SQLite** — the disposable structured cache over the vault.
|
||||
### Knowledge graph (Graphify)
|
||||
**Graphify** ([safishamsi/graphify](https://github.com/safishamsi/graphify), command `graphify`)
|
||||
turns the vault into a queryable **knowledge graph** — the disposable, rebuildable structure
|
||||
over the markdown. It replaces the earlier Ruby/SQLite tag index *and* the deferred QMD vector
|
||||
layer (ADR-010): one graph gives both structured and semantic retrieval, without vectors.
|
||||
|
||||
- **Schema**:
|
||||
- `files(id, path, mtime, summary, scope)` — one row per note.
|
||||
- `tags(id, name)` — one row per distinct tag (enables enumerating the vocabulary:
|
||||
"what clients/tools do I have notes on?").
|
||||
- `files_tags(file_id, tag_id)` — `many_to_many` join.
|
||||
- **CLI** (the only interface; the AI never touches SQLite directly):
|
||||
- `index update --since <mtime>` — incremental: pulls `.md` files with `mtime >` last cache
|
||||
time, re-reads their frontmatter, upserts; **also reconciles** (prunes rows whose path no
|
||||
longer exists).
|
||||
- `index update --rebuild` — full rebuild from scratch (default `false`).
|
||||
- `index query --client X --tool Y [--scope global] [--domain Z]` — returns **path +
|
||||
summary + matched tags** for each hit (decided: option C). Tags shown so the AI sees *why*
|
||||
a file matched.
|
||||
- `index tags --namespace tool/` — enumerate a virtual index.
|
||||
- **Source of truth rule**: markdown is authoritative; the SQLite file is a rebuildable cache
|
||||
that is **never synced** and can be deleted/rebuilt anytime.
|
||||
- **Extraction**:
|
||||
- **Vault docs** → a **local Ollama SLM** extracts entities + typed relationships from each
|
||||
note (confidence-tagged `EXTRACTED` / `INFERRED` / `AMBIGUOUS`). Local model = no API cost,
|
||||
no data leaving the machine.
|
||||
- **Project code** → free **tree-sitter AST** (`--no-docs`), no model, no token cost. Kept as
|
||||
separate per-project graphs, not merged with the vault graph.
|
||||
- **What it produces**: `graphify-out/` with `graph.json`, an interactive `graph.html`, and a
|
||||
`GRAPH_REPORT.md` whose top lists the **god nodes** (the most-connected concepts — your
|
||||
highest-value entry points).
|
||||
- **Query** (via CLI and an MCP server exposing `query_graph` / `get_node` / `shortest_path`):
|
||||
ask for **god nodes first**, then scalpel down with `graphify query` / `path` / `explain`.
|
||||
Prompt the graph; don't dump the corpus into context.
|
||||
- **Metadata still matters**: the `summary` + namespaced tags remain first-class note
|
||||
attributes — `summary` is the human-written router hint Graphify does **not** generate, and
|
||||
the `tag/` namespaces stay useful for Obsidian filtering and as node attributes. They are
|
||||
retained even though they no longer back a bespoke index. *(How tightly metadata feeds graph
|
||||
queries is a refinement for build time.)*
|
||||
- **Source of truth rule**: markdown is authoritative; the graph (`graphify-out/`) is a
|
||||
rebuildable artifact that is **never synced** and can be deleted/rebuilt anytime
|
||||
(`graphify ... --force`).
|
||||
|
||||
### Freshness (lazy — chosen Option A)
|
||||
- **AI writes** → a `PostToolUse` hook on `Write`/`Edit` targeting vault `.md` files calls
|
||||
`index update --file <path>` (updates exactly that file, prunes if deleted). Event-driven,
|
||||
no polling, no staleness for AI edits.
|
||||
- **Manual edits** (rare) → caught by a **session-start reconcile** (`index update --since` +
|
||||
prune). **No daemon, no cron.**
|
||||
- **AI writes** → a `PostToolUse` hook on `Write`/`Edit` targeting vault `.md` files runs
|
||||
`graphify ... --update` to merge the changed note into the vault graph. Event-driven, no
|
||||
polling.
|
||||
- **Stale-node caveat**: Graphify's `--update` merges (SHA-256 + dedup) but does **not** prune
|
||||
deleted notes/symbols — ghost nodes accumulate. A periodic `--force` rebuild clears them,
|
||||
triggered by the **session-start reconcile** when a rebuild stamp is older than N days
|
||||
(7 to start). **No daemon, no cron.**
|
||||
|
||||
### Retrieval (hook-injected + on-demand)
|
||||
- **Session-start hook** injects: (a) a compact index/overview, (b) the current project's
|
||||
declared `convention/*` tags **resolved to files** (so coding conventions auto-pull and a
|
||||
convention edit propagates to every project using that tag), (c) a pointer to recent
|
||||
episodic journal.
|
||||
- **On demand**: the AI runs `index query` to pull specific knowledge into context only when
|
||||
the task needs it. Projects stay thin — their CLAUDE.md holds **tags/pointers**, not content.
|
||||
- **Session-start hook** injects: (a) a compact overview — the vault graph's **god nodes** as
|
||||
the map of what's known, (b) the current project's declared `convention/*` notes resolved to
|
||||
their summaries (so coding conventions auto-pull and a convention edit propagates to every
|
||||
project using that tag), (c) a pointer to recent episodic journal.
|
||||
- **On demand**: the AI runs `graphify query` / `path` / `explain` (or the MCP tools) to pull
|
||||
specific knowledge into context only when the task needs it. Projects stay thin — their
|
||||
CLAUDE.md holds **tags/pointers**, not content.
|
||||
|
||||
## Layer 3 — Semantic over knowledge (QMD) — DEFERRED
|
||||
## Semantic recall over the vault — covered by Graphify
|
||||
|
||||
- **What**: QMD (github.com/tobi/qmd) — local markdown search, **SQLite + FTS5/BM25 + local
|
||||
vector embeddings (EmbeddingGemma-300M GGUF) + LLM rerank**. CLI + optional **MCP server**.
|
||||
No Docker, no API keys. Proves SQLite + a local vector model is enough — no Milvus/Postgres
|
||||
for knowledge.
|
||||
- **Why deferred**: start structured-only. Add QMD as a **set-and-forget** semantic layer over
|
||||
the vault **only when** we catch ourselves failing to retrieve notes we know exist (the
|
||||
video's "only level up when it bites"). It complements, not replaces, the tag index (QMD
|
||||
filters by path/collection context, not first-class frontmatter tags).
|
||||
The earlier design earmarked a separate vector layer (**QMD**) for "when structured tag
|
||||
filtering misses a note whose wording doesn't match the query." Graphify's knowledge graph
|
||||
covers that need without a second system or vectors: relationship traversal and `explain`
|
||||
surface notes by *connection*, not just exact tag match. So there is no separate deferred
|
||||
semantic layer — if graph traversal ever proves insufficient for a case where embedding
|
||||
similarity would clearly win, revisit then (the video's "only level up when it bites").
|
||||
|
||||
## Timeline (Goal 3) details
|
||||
- A **session-end hook** appends a daily journal note (one file per date) with pointers to the
|
||||
|
|
@ -155,8 +166,8 @@ A small **Ruby program, Sequel ORM, SQLite** — the disposable structured cache
|
|||
## Sync (Goal 4)
|
||||
- The **vault** syncs to the VPS via **git** (versioned history, hourly) or **Syncthing**
|
||||
(continuous, zero-thought). Decision deferred to build time.
|
||||
- **Indexes are never synced** — Milvus Lite and (later) QMD shadow indexes are rebuilt per
|
||||
machine. Sync only the markdown.
|
||||
- **Graphs/indexes are never synced** — the Milvus Lite episodic index and the Graphify
|
||||
`graphify-out/` graphs are rebuilt per machine. Sync only the markdown.
|
||||
|
||||
## Packaging
|
||||
- The whole thing ships as a **global Claude Code plugin with skills** (hooks + scripts +
|
||||
|
|
@ -168,6 +179,6 @@ A small **Ruby program, Sequel ORM, SQLite** — the disposable structured cache
|
|||
| Goal | Met by |
|
||||
|------|--------|
|
||||
| 1. Thin projects | Knowledge in the vault, not repos; CLAUDE.md holds tags/pointers; on-demand `index query` |
|
||||
| 2. Cross-project/client knowledge, global vs project scopes | Flat vault + namespaced tags + `scope/` + `client/`; enumerable virtual indexes |
|
||||
| 2. Cross-project/client knowledge, global vs project scopes | Flat vault + namespaced tags + `scope/` + `client/`; Graphify knowledge graph (god nodes + traversal) over it |
|
||||
| 3. Timeline | memsearch episodic layer + session-end journal hook |
|
||||
| 4. Remote, local-fast | Markdown vault synced via git/Syncthing; disposable per-machine indexes |
|
||||
| 4. Remote, local-fast | Markdown vault synced via git/Syncthing; disposable per-machine graphs/indexes |
|
||||
|
|
|
|||
|
|
@ -69,7 +69,9 @@ Alternatives rejected · Status. Newest decisions extend the log; supersede rath
|
|||
(overkill; QMD/memsearch prove SQLite is enough — see ADR-006/008).
|
||||
- **Query output**: returns **path + summary + matched tags** (option C) — tags are cheap and
|
||||
show *why* a result matched, useful for cross-client queries.
|
||||
- **Status**: Accepted.
|
||||
- **Status**: **Superseded by ADR-010** (Graphify replaces the Ruby/SQLite tag index). The
|
||||
`summary` + namespaced-tag frontmatter this ADR introduced is **retained as note metadata**;
|
||||
only the bespoke Ruby/SQLite index and its CLI are dropped.
|
||||
|
||||
## ADR-005 — Structured-first; semantic search over the vault deferred
|
||||
|
||||
|
|
@ -80,7 +82,10 @@ Alternatives rejected · Status. Newest decisions extend the log; supersede rath
|
|||
meaning-based search over the vault until it demonstrably bites.
|
||||
- **Rationale**: Structured tagging is the lightweight/fast thing the user wants, and the
|
||||
summary+tag design is built to make it work. Follow the video's "only level up when it bites."
|
||||
- **Status**: Accepted (semantic deferred).
|
||||
- **Status**: **Superseded by ADR-010.** The premise (ship structured-only, bolt on semantics
|
||||
later) no longer holds: Graphify makes the knowledge layer a graph from day one, giving
|
||||
structured *and* connection-based recall together. The "only level up when it bites" instinct
|
||||
carries forward to whether a *vector* layer is ever needed on top of the graph.
|
||||
|
||||
## ADR-006 — QMD as the (deferred) semantic-over-knowledge layer
|
||||
|
||||
|
|
@ -95,7 +100,10 @@ Alternatives rejected · Status. Newest decisions extend the log; supersede rath
|
|||
filtering.
|
||||
- **Alternatives rejected**: Pointing memsearch at the vault (mixes episodic and knowledge
|
||||
corpora; its filtering is path-prefix not tags). A bespoke embedding index (reinvents QMD).
|
||||
- **Status**: Deferred / earmarked.
|
||||
- **Status**: **Superseded by ADR-010.** Graphify's knowledge graph fills the
|
||||
semantic-recall role (traversal/`explain` over connections) without a separate vector system,
|
||||
so QMD is no longer earmarked. Revisit a vector layer only if graph traversal demonstrably
|
||||
misses cases where embedding similarity would win.
|
||||
|
||||
## ADR-007 — Lazy freshness: write-hook + session-start reconcile, no daemon/cron
|
||||
|
||||
|
|
@ -116,8 +124,8 @@ Alternatives rejected · Status. Newest decisions extend the log; supersede rath
|
|||
|
||||
- **Context**: Must be accessible on a VPS / multiple machines but run local-fast (Goal 4).
|
||||
- **Decision**: Sync the **markdown vault** to the VPS via **git or Syncthing** (choice deferred
|
||||
to build time). **Indexes (Milvus Lite, future QMD) are rebuilt per machine and never
|
||||
synced.**
|
||||
to build time). **Graphs/indexes (Milvus Lite, Graphify `graphify-out/`) are rebuilt per
|
||||
machine and never synced.**
|
||||
- **Rationale**: Markdown is plain text — git/Syncthing sync it trivially; lazy (hourly or
|
||||
continuous-async) is enough. Indexes are disposable caches; syncing binary DBs invites
|
||||
conflicts for no gain. Local reads stay fast; ownership and portability stay with the user.
|
||||
|
|
@ -134,10 +142,40 @@ Alternatives rejected · Status. Newest decisions extend the log; supersede rath
|
|||
skills**, installed at the user level.
|
||||
- **Rationale**: Skills carry the "when to write / what conventions / how & when to query"
|
||||
guidance to the model; the plugin registers the session-start / session-end / PostToolUse
|
||||
hooks and bundles the Ruby CLI. Global install = consistent behavior everywhere; single
|
||||
source of truth for the conventions themselves.
|
||||
hooks and wires up Graphify (extraction/update/query + MCP server). Global install =
|
||||
consistent behavior everywhere; single source of truth for the conventions themselves.
|
||||
- **Status**: Accepted (to be built — see 04-build-plan.md).
|
||||
|
||||
## ADR-010 — Graphify knowledge graph as the knowledge layer (supersedes ADR-004/005/006)
|
||||
|
||||
- **Context**: ADR-004 specced a hand-built Ruby/Sequel/SQLite tag index (+ CLI) as the
|
||||
machine-queryable layer over the vault, with ADR-005/006 deferring meaning-based recall to a
|
||||
future QMD vector layer. Before building any of it, we evaluated **Graphify** (`graphify`,
|
||||
PyPI `graphifyy`) — a tool that turns a folder into a queryable knowledge graph (local
|
||||
tree-sitter AST for code, local-SLM entity/relationship extraction for docs). See
|
||||
`06-graphify-evaluation.md`.
|
||||
- **Decision**: Use **Graphify as the knowledge-layer engine** over the vault, with a **local
|
||||
Ollama** backend for doc extraction and free AST for per-project code graphs. **Drop** the
|
||||
Ruby/SQLite tag-index CLI (ADR-004) and the earmarked QMD layer (ADR-006); **retain** the
|
||||
`summary` + namespaced-tag frontmatter from ADR-003/004 as note metadata.
|
||||
- **Rationale**: One off-the-shelf tool delivers both what the tag index was for (structured
|
||||
retrieval) and what QMD was deferred for (connection/meaning-based recall via graph
|
||||
traversal + `explain`) — without writing or maintaining a bespoke index, and without a vector
|
||||
store. Code graphs come free. Keeps the markdown-as-truth, no-Docker, no-API-key, local-first
|
||||
properties (extraction runs against local Ollama). Net scope reduction: the entire Ruby build
|
||||
(old critical-path Step 2) and the QMD layer are removed.
|
||||
- **What's retained / changed**: `summary` stays the human-written router hint Graphify does not
|
||||
generate; namespaced tags stay useful for Obsidian filtering and as node attributes. How
|
||||
tightly metadata should feed graph queries is a **build-time refinement**, not settled here.
|
||||
- **Trade-off accepted**: Graphify's `--update` doesn't prune deleted nodes (stale-node drift) —
|
||||
mitigated by a periodic `--force` rebuild on the session-start staleness check (ADR-007's lazy
|
||||
model still applies). Graphify also moves fast (flags are version-dependent; anchored to
|
||||
v0.8.30) and its headline token-savings numbers are corpus-dependent — benchmark our own.
|
||||
- **Alternatives rejected**: Building the Ruby/SQLite index as originally planned (more code to
|
||||
own; no semantic recall); adding QMD as a second system on top (two stores where one graph
|
||||
suffices).
|
||||
- **Status**: Accepted (to be built — see 04-build-plan.md and 06-graphify-evaluation.md).
|
||||
|
||||
## Rejected tools (summary)
|
||||
|
||||
| Tool | Why rejected for our use |
|
||||
|
|
@ -145,5 +183,6 @@ Alternatives rejected · Status. Newest decisions extend the log; supersede rath
|
|||
| MemPalace (L4) | Storage not readable markdown; isolated drawers (knowledge not interconnected); fights self-managing + cross-linking goals |
|
||||
| Recall / LightRAG (L5) | Content knowledge bases / deep research, not operational memory; Recall = hosted, you don't own data; LightRAG = enterprise overkill |
|
||||
| OpenBrain / Mem0 (L6) | Always-remote DB → latency + cost; conflicts with local-fast lazy-sync; only pays off for real-time cross-tool memory (user: overkill) |
|
||||
| Postgres / Milvus server | Unnecessary — SQLite (tag index) + Milvus Lite (memsearch) + QMD's SQLite cover everything locally with no Docker |
|
||||
| Postgres / Milvus server | Unnecessary — Graphify's local graph (knowledge) + Milvus Lite (memsearch episodic) cover everything locally with no Docker |
|
||||
| claude-mem | MCP-based (Claude must call search); opaque blobs vs readable markdown; feature overkill |
|
||||
| Ruby/SQLite tag index CLI; QMD vector layer | Superseded by Graphify before build — one knowledge graph replaces both the structured index and the deferred semantic layer (ADR-010) |
|
||||
|
|
|
|||
|
|
@ -255,7 +255,7 @@ global install keeps conventions a single source of truth.
|
|||
|
||||
---
|
||||
|
||||
## Open questions (carried from 05-handoff.md + updated)
|
||||
## Open questions
|
||||
|
||||
1. **Vault location** — `~/brain` (synced home dir)? Symlink into `~/.claude/memory`?
|
||||
2. **Sync mechanism** — git (versioned, hourly) vs Syncthing (continuous)?
|
||||
|
|
@ -270,4 +270,4 @@ global install keeps conventions a single source of truth.
|
|||
6. **Model benchmark results** — defer all model decisions until Step 2c benchmark is done.
|
||||
Don't hardcode `gemma4:e2b` until testing confirms it's the right pick.
|
||||
7. **memsearch + journal** — does memsearch index our SessionEnd journal notes, or only
|
||||
its own auto-capture? See `05-handoff.md:55-56`.
|
||||
its own auto-capture, and how does the journal point into the knowledge vault?
|
||||
|
|
|
|||
|
|
@ -1,82 +0,0 @@
|
|||
# Session Handoff
|
||||
|
||||
_Written 2026-06-03 at the end of the brainstorming session. For the next session to pick up._
|
||||
|
||||
## What this is
|
||||
|
||||
Designing a personal, cross-project **memory system for Claude Code** for a multi-client
|
||||
freelancer. We finished **brainstorming/architecture** and wrote the design + ADRs. **No
|
||||
implementation has started.** The source material is the transcript at
|
||||
`/home/jared/Documents/cc-os/memory-systems-compared060326` (the "6 Levels of Claude Code
|
||||
Memory" video).
|
||||
|
||||
## Where we are
|
||||
|
||||
- ✅ Synthesized the video (`01-video-synthesis.md`).
|
||||
- ✅ Settled the architecture and got user sign-off (`02-system-design.md`).
|
||||
- ✅ Recorded every decision + rejected/deferred options (`03-architecture-decisions.md`).
|
||||
- ✅ Wrote the build outline + operational answers (`04-build-plan.md`).
|
||||
- ⏳ **Next**: turn `04-build-plan.md` into a real implementation plan and start building.
|
||||
|
||||
## The design in 30 seconds
|
||||
|
||||
Three local, markdown-as-truth, no-Docker layers:
|
||||
1. **Episodic** ("what happened, when") = **memsearch** (Milvus Lite, off-the-shelf).
|
||||
2. **Knowledge** ("how do we…") = **flat Obsidian vault** (single source of truth) + a
|
||||
**Ruby/Sequel/SQLite tag index** with a CLI. Organized by **namespaced nested tags**
|
||||
(`tool/ client/ domain/ convention/ scope/`), not folders.
|
||||
3. **Semantic over the vault** = **QMD**, **deferred** until structured tagging proves
|
||||
insufficient.
|
||||
|
||||
Retrieval is hook-injected + on-demand (projects stay thin). Freshness is lazy (write-time
|
||||
hook + session-start reconcile; no daemon/cron). Vault syncs to a VPS via git/Syncthing;
|
||||
indexes are disposable and rebuilt per machine. Ships as a **global Claude Code plugin with
|
||||
skills**.
|
||||
|
||||
## Decisions locked (don't relitigate without reason)
|
||||
|
||||
- Two separate systems for episodic vs knowledge (ADR-001).
|
||||
- memsearch for episodic (ADR-002). Flat vault + tags, not folders (ADR-003).
|
||||
- SQLite + Sequel (Ruby) CLI tag index; markdown authoritative, cache disposable (ADR-004).
|
||||
- Structured-first; semantic (QMD) deferred (ADR-005/006).
|
||||
- Lazy freshness: PostToolUse write hook + SessionStart reconcile, no daemon/cron (ADR-007).
|
||||
- Sync the vault not the indexes; reject OpenBrain/Mem0/Postgres (ADR-008).
|
||||
- Package as a global plugin with skills (ADR-009).
|
||||
- Query output = path + summary + matched tags (option C).
|
||||
|
||||
## Open questions for build time
|
||||
|
||||
1. **Vault location** — default `~/brain` (or similar synced home dir)? Symlink into
|
||||
`~/.claude/memory` only if a tool requires it.
|
||||
2. **Sync mechanism** — git (versioned history, hourly) vs Syncthing (continuous, zero-thought).
|
||||
3. **Convention notes placement** — confirmed direction: coding `convention/*` live as **data
|
||||
in the vault** (resolved by the SessionStart hook); the memory-system **skills live in the
|
||||
plugin** (behavior, versioned). Validate when building.
|
||||
4. **memsearch journal** — does memsearch index our SessionEnd journal notes, or only its own
|
||||
auto-capture? Decide how the journal points into the knowledge vault.
|
||||
5. **Promotion rule** — the concrete threshold for project→`scope/global` promotion (e.g.
|
||||
recurrence count) used by the reorganize step.
|
||||
6. **CLI ergonomics** — command framework (thor vs optparse), output format details, where
|
||||
`last_cache_time` is stored.
|
||||
|
||||
## Recommended first actions next session
|
||||
|
||||
1. Re-read `02-system-design.md` and `04-build-plan.md`.
|
||||
2. Invoke the **writing-plans** skill to convert `04-build-plan.md` Part A into a staged
|
||||
implementation plan (Step 2, the Ruby tag-index CLI, is the critical path — build and test
|
||||
it standalone first).
|
||||
3. Resolve open questions 1–2 (vault location + sync) before writing code, since they affect
|
||||
paths in the hooks and CLI.
|
||||
4. Build Step 2 (CLI) with tests, then Step 3 (hooks), then validate end-to-end on the seeded
|
||||
notes before touching the plugin packaging (Step 6).
|
||||
|
||||
## Context notes
|
||||
|
||||
- Working dir `/home/jared/Documents/cc-os` is **not a git repo**; consider `git init` if we
|
||||
want history for these docs and the build.
|
||||
- User prefers **Docker when reasonable** — but this design deliberately needs **none** (all
|
||||
layers are embedded/local). Only full Milvus or QMD-server modes would involve Docker, and
|
||||
we're not using those.
|
||||
- User is Ruby-comfortable (chose Ruby + Sequel deliberately).
|
||||
- A past `~/Documents/SecondBrain/` attempt (tag DB) was lost track of — the "markdown
|
||||
authoritative, cache disposable & rebuildable" rule (ADR-004) is the explicit fix for that.
|
||||
|
|
@ -111,7 +111,6 @@ The more honest framing: Graphify replaces tags as the **vault entity index**, a
|
|||
| Query verbs + bounded traversal | `docs/graphify/06-querying-and-god-nodes.md:51–93` |
|
||||
| Stale node drift limitation | `docs/graphify/07-token-economics-and-updates.md:136–148` |
|
||||
| Token savings by repo size | `docs/graphify/external-tips.md:31–45` |
|
||||
| Tag index schema and role | `docs/memory-system/02-system-design.md:95–113` |
|
||||
| Knowledge layer + retained metadata (summary/tags) | `docs/memory-system/02-system-design.md` § Layer 2 |
|
||||
| Graph rejection decisions (LightRAG) | `docs/memory-system/03-architecture-decisions.md:83, 145` |
|
||||
| Critical path (Ruby CLI = Step 2) | `docs/memory-system/04-build-plan.md` |
|
||||
| Open handoff questions | `docs/memory-system/05-handoff.md:47–60` |
|
||||
| Build order + open questions | `docs/memory-system/04-build-plan.md` |
|
||||
|
|
|
|||
Loading…
Reference in New Issue