Archive memsearch-episodic-layer change
- Sync memsearch-episodic delta spec to openspec/specs/memsearch-episodic/spec.md - Update README.md: memsearch now active alongside Graphify - Move change to openspec/changes/archive/2026-06-09-memsearch-episodic-layer/ Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
jared
parent
e6d925df29
commit
accae31fba
11
CLAUDE.md
11
CLAUDE.md
|
|
@ -61,7 +61,7 @@ to those two and fix the stale doc.
|
|||
|
||||
**Empirical finding locked (2026-06-05):** Graphify is a structure extractor, not a topic clusterer — no emergent hub nodes appear even at `--mode deep`; hub notes + wikilinks must be author-provided during migration (not deferred). Migration scaffolding is now a first-class deliverable. Open question: do facet tags create graph edges? (ADR-014; findings: `docs/memory-system/07-graph-connectivity-findings.md`).
|
||||
|
||||
**Implementation status (2026-06-09):** The global Claude Code plugin is live (`~/.claude/plugins/memory/`). Steps 1, 2a, 2b, 3, and 6 of the build plan are complete (including the `memory-project` skill, previously TODO under Step 6/Part D). Step 1 completed 2026-06-09: vault-conventions.md exists, all 6 fixture notes seeded, and 14 Graphify handbook + memory-system design notes migrated to the vault as migration scaffolding. Steps 4 (memsearch) and 5 (sync) remain. See `docs/memory-system/04-build-plan.md` for full step status.
|
||||
**Implementation status (2026-06-09):** The global Claude Code plugin is live (`~/.claude/plugins/memory/`). Steps 1, 2a, 2b, 3, and 6 of the build plan are complete (including the `memory-project` skill, previously TODO under Step 6/Part D). Step 1 completed 2026-06-09: vault-conventions.md exists, all 6 fixture notes seeded, and 14 Graphify handbook + memory-system design notes migrated to the vault as migration scaffolding. Step 4 (memsearch) completed 2026-06-09: plugin installed via marketplace, MEMSEARCH_DIR set global (~/.memsearch), Stop hook verified producing daily memory files, search confirmed working. Step 5 (sync) remains. See `docs/memory-system/04-build-plan.md` for full step status.
|
||||
|
||||
## Implemented Components
|
||||
|
||||
|
|
@ -76,7 +76,14 @@ to those two and fix the stale doc.
|
|||
- Project graph: `<project-root>/graphify-out/` — same pattern; gitignore it in each project repo
|
||||
- Vault conventions: `~/Documents/SecondBrain/vault-conventions.md` — frontmatter contract + tag taxonomy (canonical name decided 2026-06-09; formerly referred to as CONVENTIONS.md)
|
||||
|
||||
**Not yet implemented:** memsearch episodic layer (Step 4), VPS sync (Step 5)
|
||||
**memsearch** — v0.4.6 via Claude Code plugin marketplace (`memsearch@memsearch-plugins`)
|
||||
- Hooks: Stop, SessionStart, UserPromptSubmit, SessionEnd (ship with plugin)
|
||||
- Memory store: `~/.memsearch/memory/YYYY-MM-DD.md` daily files (global, cross-project)
|
||||
- Index: `~/.memsearch/milvus.db` (Milvus Lite, local); embeddings via ONNX bge-m3
|
||||
- Config: `MEMSEARCH_DIR=~/.memsearch` in `~/.zshrc` for global scope
|
||||
- Skills: `/memory-recall`, `/memory-config` (ship with plugin)
|
||||
|
||||
**Not yet implemented:** VPS sync (Step 5)
|
||||
|
||||
## OpenSpec workflow
|
||||
|
||||
|
|
|
|||
|
|
@ -7,8 +7,8 @@ start of each Claude Code session.
|
|||
## Status
|
||||
|
||||
The memory plugin is **live and working.** Design documentation lives in `docs/`.
|
||||
Episodic memory (memsearch) is not yet built — only semantic/knowledge memory via
|
||||
Graphify is active.
|
||||
Both episodic memory (memsearch) and semantic/knowledge memory (Graphify + Obsidian vault)
|
||||
are active.
|
||||
|
||||
## What it does for you
|
||||
|
||||
|
|
|
|||
|
|
@ -187,7 +187,7 @@ Hooks now call Graphify instead of the Ruby CLI.
|
|||
(e.g. `~/.cache/graphify/vault-rebuild.stamp`). SessionStart hook triggers `--force` if
|
||||
older than N days (7 is a reasonable starting point).
|
||||
|
||||
### Step 4 — Episodic layer (memsearch) — NOT STARTED
|
||||
### Step 4 — Episodic layer (memsearch) — DONE
|
||||
- Install memsearch (`/plugin marketplace add zilliztech/memsearch`, then `plugin install
|
||||
memsearch`), local to start. Verify daily memory files appear after a few conversations.
|
||||
- Decide whether memsearch indexes our session-end journal notes or its own capture (likely its
|
||||
|
|
|
|||
|
|
@ -0,0 +1,2 @@
|
|||
schema: spec-driven
|
||||
created: 2026-06-09
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
## Context
|
||||
|
||||
The personal Claude Code memory system currently has two intended layers: episodic (per-turn Q&A history across all projects) and semantic/knowledge (the Obsidian vault + Graphify). The semantic layer is live. The episodic layer does not exist yet — there is no durable record of what was discussed across sessions.
|
||||
|
||||
memsearch is a purpose-built episodic memory plugin for Claude Code, developed by Zilliz (the company behind Milvus). It ships with its own Stop hook, Claude guidance, and tool definitions. The design principle for this step is: install first, audit defaults, deviate only if evidence demands it.
|
||||
|
||||
**Current state:** No episodic memory. No memsearch installed.
|
||||
|
||||
**Constraints:**
|
||||
- Must be a global install (all projects share one episodic store)
|
||||
- No external API keys — embeddings must run locally
|
||||
- No daemon or cron — freshness is lazy (write-time + incremental index)
|
||||
- No custom code unless audit reveals concrete gaps
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Install memsearch via the Claude Code plugin marketplace
|
||||
- Confirm `memory_dir` points to a global path (`~/.memsearch/memory/`)
|
||||
- Verify the memsearch Stop hook fires and produces `~/.memsearch/memory/YYYY-MM-DD.md` after a session
|
||||
- Audit the built-in plugin guidance to check for routing conflicts with `memory-vault`/`memory-project`
|
||||
|
||||
**Non-Goals:**
|
||||
- Does not index the Obsidian vault — Graphify owns semantic search
|
||||
- Does not add a SessionStart hook injection (evaluate after seeing what the default install provides)
|
||||
- Does not configure Milvus Server or Zilliz Cloud (Milvus Lite is correct for personal use)
|
||||
- Does not write a custom `memory-search` skill unless the audit reveals gaps
|
||||
- Does not modify `session-end.sh` or any existing hooks
|
||||
|
||||
## Decisions
|
||||
|
||||
**Use the plugin marketplace, not manual install**
|
||||
memsearch ships as a complete Claude Code plugin. Installing via `/plugin marketplace add` + `/plugin install` wires the Stop hook, tool definitions, and Claude guidance automatically. Manual installation would require replicating that work without benefit.
|
||||
|
||||
*Alternative considered: clone repo and wire manually* — rejected; more work with no gain given a working plugin exists.
|
||||
|
||||
**Milvus Lite over Milvus Server**
|
||||
Personal use case: one developer, one machine, low write volume. Milvus Lite runs in-process with no daemon. Milvus Server adds operational overhead that isn't warranted here.
|
||||
|
||||
*Alternative considered: Zilliz Cloud* — rejected; requires account, network dependency, and cost for a personal tool.
|
||||
|
||||
**ONNX bge-m3 embeddings over API-based embeddings**
|
||||
Ships with the plugin; runs on CPU; ~558MB one-time download; no API key required. Consistent with the system's principle of local-first tooling (same reason Ollama is used for Graphify extraction).
|
||||
|
||||
**Audit-first policy**
|
||||
memsearch's built-in guidance may already handle cross-project episodic recall correctly. Adding a custom skill before seeing what the default install provides would be premature. Evaluate after one real session.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **`memory_dir` defaults to project-local path** → Mitigation: run `memsearch config list` immediately after install; set global path in `~/.memsearch/config.toml` if needed. Spec requires this verification step.
|
||||
- **Plugin marketplace unavailability** → Mitigation: if `/plugin marketplace add` fails, check memsearch GitHub for alternative install path.
|
||||
- **558MB embedding model download** → Mitigation: one-time cost; happens on first use. No action needed, just awareness.
|
||||
- **Built-in guidance conflicts with `memory-vault`/`memory-project` routing** → Mitigation: audit after install; add a minimal `memory-search` skill only if routing confusion is observed in practice.
|
||||
- **Stop hook not auto-registering** → Mitigation: verification step in the checklist confirms hook is wired; manual registration in `~/.claude/settings.json` is fallback.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. Install plugin via marketplace commands
|
||||
2. Run `memsearch config list` — record actual defaults (do not assume)
|
||||
3. If `memory_dir` is project-local, set `memory_dir = "~/.memsearch/memory/"` in `~/.memsearch/config.toml`
|
||||
4. Verify Stop hook is registered (check `~/.claude/settings.json` or plugin config)
|
||||
5. Run one test session — confirm `~/.memsearch/memory/YYYY-MM-DD.md` is created
|
||||
6. Audit Claude plugin instructions — note what guidance memsearch adds to context
|
||||
7. If routing gaps exist, write a minimal `memory-search` skill
|
||||
|
||||
**Rollback:** Uninstall the plugin (`/plugin uninstall memsearch`). The Milvus database and daily memory files are inert without the plugin. No changes to existing hooks or skills, so nothing to revert there.
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
## Why
|
||||
|
||||
The memory system has two layers — episodic ("what happened, when") and semantic/knowledge ("how do we…"). The semantic layer is live (Obsidian vault + Graphify). The episodic layer is missing: there is no way to answer "what were we working on last Tuesday?" or "did we discuss X recently?" across projects. memsearch is the purpose-built tool for this and fills that gap with zero custom code.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Install memsearch as a global Claude Code plugin via the marketplace
|
||||
- Confirm `memory_dir` is set to a global path (`~/.memsearch/memory/`) so all projects share one episodic store
|
||||
- Verify the memsearch Stop hook is registering per-turn Q&A summaries correctly
|
||||
- Audit memsearch's built-in Claude guidance after install before adding anything custom
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `memsearch-episodic`: Episodic memory layer — install memsearch plugin, confirm global memory_dir, verify Stop hook produces daily memory files, and audit built-in guidance for routing gaps
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
_(none — existing hooks and skills are unchanged; memsearch ships with its own Stop hook)_
|
||||
|
||||
## Impact
|
||||
|
||||
- New tool available in all Claude Code sessions: `memsearch search "<query>"`
|
||||
- memsearch Stop hook fires after every response turn (ships with plugin; no custom code)
|
||||
- `~/.memsearch/memory/YYYY-MM-DD.md` daily files accumulate across all projects
|
||||
- `~/.memsearch/milvus.db` is the persistent vector store (Milvus Lite, local)
|
||||
- No changes to `session-end.sh`, `memory-vault`, `memory-project`, or Graphify
|
||||
- A `memory-search` skill is conditional: only warranted if built-in guidance proves insufficient after install
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
## ADDED Requirements
|
||||
|
||||
### Requirement: Plugin installed globally
|
||||
The system SHALL install memsearch as a Claude Code plugin via the marketplace so that episodic memory is available in all sessions.
|
||||
|
||||
#### Scenario: Successful marketplace install
|
||||
- **WHEN** the user runs `/plugin marketplace add zilliztech/memsearch` followed by `/plugin install memsearch`
|
||||
- **THEN** the memsearch plugin is active, its Stop hook is registered, and `memsearch` tool definitions are available in the session
|
||||
|
||||
### Requirement: Global memory directory
|
||||
The system SHALL store all episodic memory files at a global path (`~/.memsearch/memory/`) so that memories from all projects accumulate in one place and are queryable cross-project.
|
||||
|
||||
#### Scenario: memory_dir is already global after install
|
||||
- **WHEN** the user runs `memsearch config list` after install
|
||||
- **THEN** `memory_dir` resolves to `~/.memsearch/memory/` (or equivalent global home-relative path)
|
||||
|
||||
#### Scenario: memory_dir requires explicit configuration
|
||||
- **WHEN** `memsearch config list` shows a project-local `memory_dir`
|
||||
- **THEN** the user sets `memory_dir = "~/.memsearch/memory/"` in `~/.memsearch/config.toml` and re-verifies with `memsearch config list`
|
||||
|
||||
### Requirement: Stop hook produces daily memory files
|
||||
The system SHALL produce a daily memory file at `~/.memsearch/memory/YYYY-MM-DD.md` after each session turn so that episodic history accumulates automatically without manual action.
|
||||
|
||||
#### Scenario: Memory file created after first session turn
|
||||
- **WHEN** a session turn completes (user message + Claude response) with memsearch installed and configured
|
||||
- **THEN** a file exists at `~/.memsearch/memory/<today-date>.md` containing a summary of that turn
|
||||
|
||||
#### Scenario: Stop hook fires on every turn
|
||||
- **WHEN** multiple turns occur in a session
|
||||
- **THEN** the daily memory file grows with entries for each turn (SHA-256 dedup prevents duplicate entries)
|
||||
|
||||
### Requirement: Built-in guidance audited before customization
|
||||
The system SHALL audit memsearch's built-in Claude plugin instructions after install before adding any custom skill or configuration, so that custom code is only added when a concrete gap is identified.
|
||||
|
||||
#### Scenario: No routing conflict found
|
||||
- **WHEN** the user reviews memsearch's built-in guidance and runs a test session
|
||||
- **THEN** episodic recall queries are handled by memsearch's built-in tools without conflicting with `memory-vault` or `memory-project` routing
|
||||
- **THEN** no custom `memory-search` skill is created
|
||||
|
||||
#### Scenario: Routing conflict or gap identified
|
||||
- **WHEN** memsearch's built-in guidance fails to distinguish episodic from semantic queries, or does not handle cross-project recall correctly
|
||||
- **THEN** a minimal `memory-search` skill is authored that supplements (not replaces) the built-in guidance
|
||||
|
||||
### Requirement: Cross-project episodic recall available
|
||||
The system SHALL enable the user to query episodic memory across all projects via `memsearch search "<query>"` so that questions like "what were we working on last Tuesday?" are answerable.
|
||||
|
||||
#### Scenario: Successful cross-project query
|
||||
- **WHEN** the user runs `memsearch search "what was I working on last week?"` after at least one session has produced memory files
|
||||
- **THEN** memsearch returns relevant summaries from `~/.memsearch/memory/` ranked by semantic similarity
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
## 1. Install Plugin
|
||||
|
||||
- [x] 1.1 Run `/plugin marketplace add zilliztech/memsearch` in a Claude Code session
|
||||
- [x] 1.2 Run `/plugin install memsearch` to activate the plugin
|
||||
- [x] 1.3 Confirm the plugin is listed as active (no error messages)
|
||||
|
||||
## 2. Audit and Configure
|
||||
|
||||
- [x] 2.1 Run `memsearch config list` and record the actual defaults (especially `memory_dir` and `milvus_db` paths)
|
||||
- [x] 2.2 If `memory_dir` is project-local, set `memory_dir = "~/.memsearch/memory/"` in `~/.memsearch/config.toml`
|
||||
- [x] 2.3 Re-run `memsearch config list` to confirm `memory_dir` resolves to a global path
|
||||
|
||||
## 3. Verify Stop Hook
|
||||
|
||||
- [x] 3.1 Check `~/.claude/settings.json` (or plugin config) to confirm the memsearch Stop hook is registered
|
||||
- [x] 3.2 Run a short test session (a few turns) and exit
|
||||
- [x] 3.3 Confirm `~/.memsearch/memory/YYYY-MM-DD.md` exists with content from the test session
|
||||
|
||||
## 4. Audit Built-in Guidance
|
||||
|
||||
- [x] 4.1 Review what memsearch adds to Claude's context (system prompt / tool definitions)
|
||||
- [x] 4.2 Run a test query: `memsearch search "what were we working on in the test session?"`
|
||||
- [x] 4.3 Assess whether built-in guidance handles cross-project episodic recall without conflicting with `memory-vault`/`memory-project`
|
||||
- [x] 4.4 If routing gaps are found, author a minimal `memory-search` skill; otherwise skip
|
||||
|
||||
## 5. Mark Complete
|
||||
|
||||
- [x] 5.1 Update `docs/memory-system/04-build-plan.md` — mark Step 4 complete
|
||||
- [x] 5.2 Update `CLAUDE.md` — add memsearch to Implemented Components, update design paragraph
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
# Spec: memsearch-episodic
|
||||
|
||||
## Purpose
|
||||
|
||||
Defines the episodic memory layer backed by the memsearch Claude Code plugin. Episodic memory captures what happened and when — session summaries stored as daily markdown files, queryable by semantic similarity across all projects.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: Plugin installed globally
|
||||
The system SHALL install memsearch as a Claude Code plugin via the marketplace so that episodic memory is available in all sessions.
|
||||
|
||||
#### Scenario: Successful marketplace install
|
||||
- **WHEN** the user runs `/plugin marketplace add zilliztech/memsearch` followed by `/plugin install memsearch`
|
||||
- **THEN** the memsearch plugin is active, its Stop hook is registered, and `memsearch` tool definitions are available in the session
|
||||
|
||||
### Requirement: Global memory directory
|
||||
The system SHALL store all episodic memory files at a global path (`~/.memsearch/memory/`) so that memories from all projects accumulate in one place and are queryable cross-project.
|
||||
|
||||
#### Scenario: memory_dir is already global after install
|
||||
- **WHEN** the user runs `memsearch config list` after install
|
||||
- **THEN** `memory_dir` resolves to `~/.memsearch/memory/` (or equivalent global home-relative path)
|
||||
|
||||
#### Scenario: memory_dir requires explicit configuration
|
||||
- **WHEN** `memsearch config list` shows a project-local `memory_dir`
|
||||
- **THEN** the user sets `memory_dir = "~/.memsearch/memory/"` in `~/.memsearch/config.toml` and re-verifies with `memsearch config list`
|
||||
|
||||
### Requirement: Stop hook produces daily memory files
|
||||
The system SHALL produce a daily memory file at `~/.memsearch/memory/YYYY-MM-DD.md` after each session turn so that episodic history accumulates automatically without manual action.
|
||||
|
||||
#### Scenario: Memory file created after first session turn
|
||||
- **WHEN** a session turn completes (user message + Claude response) with memsearch installed and configured
|
||||
- **THEN** a file exists at `~/.memsearch/memory/<today-date>.md` containing a summary of that turn
|
||||
|
||||
#### Scenario: Stop hook fires on every turn
|
||||
- **WHEN** multiple turns occur in a session
|
||||
- **THEN** the daily memory file grows with entries for each turn (SHA-256 dedup prevents duplicate entries)
|
||||
|
||||
### Requirement: Built-in guidance audited before customization
|
||||
The system SHALL audit memsearch's built-in Claude plugin instructions after install before adding any custom skill or configuration, so that custom code is only added when a concrete gap is identified.
|
||||
|
||||
#### Scenario: No routing conflict found
|
||||
- **WHEN** the user reviews memsearch's built-in guidance and runs a test session
|
||||
- **THEN** episodic recall queries are handled by memsearch's built-in tools without conflicting with `memory-vault` or `memory-project` routing
|
||||
- **THEN** no custom `memory-search` skill is created
|
||||
|
||||
#### Scenario: Routing conflict or gap identified
|
||||
- **WHEN** memsearch's built-in guidance fails to distinguish episodic from semantic queries, or does not handle cross-project recall correctly
|
||||
- **THEN** a minimal `memory-search` skill is authored that supplements (not replaces) the built-in guidance
|
||||
|
||||
### Requirement: Cross-project episodic recall available
|
||||
The system SHALL enable the user to query episodic memory across all projects via `memsearch search "<query>"` so that questions like "what were we working on last Tuesday?" are answerable.
|
||||
|
||||
#### Scenario: Successful cross-project query
|
||||
- **WHEN** the user runs `memsearch search "what was I working on last week?"` after at least one session has produced memory files
|
||||
- **THEN** memsearch returns relevant summaries from `~/.memsearch/memory/` ranked by semantic similarity
|
||||
Loading…
Reference in New Issue