95 lines
6.7 KiB
Markdown
95 lines
6.7 KiB
Markdown
|
|
---
|
||
|
|
id: "0015"
|
||
|
|
date: 2026-06-09
|
||
|
|
status: Accepted
|
||
|
|
supersedes:
|
||
|
|
superseded-by:
|
||
|
|
affected-paths: []
|
||
|
|
affected-components: []
|
||
|
|
migration_confidence: medium
|
||
|
|
migration_source: "docs/memory-system/03-architecture-decisions.md### ADR-015 — memsearch episodic memory version-controlled in a dedicated private repo, auto-synced via cc-os SessionEnd hook"
|
||
|
|
---
|
||
|
|
|
||
|
|
# 0015 — memsearch episodic memory version-controlled in a dedicated private repo, auto-synced via cc-os SessionEnd hook
|
||
|
|
|
||
|
|
## Context
|
||
|
|
|
||
|
|
memsearch's memory store at `~/.memsearch` accumulates daily session-summary
|
||
|
|
files (`memory/YYYY-MM-DD.md`) that are the only irreplaceable data in the episodic layer —
|
||
|
|
the Milvus Lite index (`milvus.db`) and the bge-m3 ONNX embeddings model are derived/
|
||
|
|
disposable and can be rebuilt at any time via `memsearch index`. With Step 4 (episodic layer)
|
||
|
|
now live, preserving episodic memory across machines and protecting against local disk loss
|
||
|
|
required a sync strategy. The question was: what venue, what scope, and who owns the sync?
|
||
|
|
memsearch's own stated design philosophy is "markdown files are the canonical data store; the
|
||
|
|
vector database is a derived index" and notes that markdown is "git-friendly" and the index
|
||
|
|
rebuildable from markdown — making git a natural fit for the markdown layer.
|
||
|
|
|
||
|
|
## Decision
|
||
|
|
|
||
|
|
`~/.memsearch` is a **dedicated git repo** on branch `main` with remote `origin`
|
||
|
|
pointing to a **private self-hosted Forgejo repo**
|
||
|
|
(`ssh://git@forgejo.swansoncloud.com:2222/jared/memsearch.git`;
|
||
|
|
web: `https://forgejo.swansoncloud.com/jared/memsearch`). A whitelist `.gitignore` tracks
|
||
|
|
**only** `memory/*.md` (the daily session files) and `.gitignore` itself. Excluded as
|
||
|
|
derived/disposable: `milvus.db` (Milvus Lite index — rebuildable any time via
|
||
|
|
`memsearch index`), `config.toml`, and the bge-m3 ONNX embeddings model (lives in
|
||
|
|
`~/.cache/huggingface`, not in the store). Auto-commit and push are wired into the **cc-os
|
||
|
|
memory plugin's own `session-end.sh`** hook, not the marketplace plugin. The appended block
|
||
|
|
guards on `~/.memsearch/.git` existing, runs `git add -A` (whitelist makes it safe), commits
|
||
|
|
only when something is staged (message: `memsearch: session memory <date>`), and pushes with
|
||
|
|
`timeout 30 ... || true`. The entire block is wrapped in a subshell with a trailing `|| true`
|
||
|
|
so it can **never fail session shutdown**.
|
||
|
|
- **Rationale**:
|
||
|
|
- **Dedicated repo (not folded into the vault or a project repo):** ADR-001 established
|
||
|
|
episodic and semantic/knowledge as separate systems by design, and `~/.memsearch` is a
|
||
|
|
global, cross-project store with no natural home in any single project repo. Committing
|
||
|
|
episodic session logs into the Obsidian vault repo would conflate the two systems and
|
||
|
|
violate the separation of concerns that ADR-001 is built on.
|
||
|
|
- **Whitelist — commit markdown, exclude rebuildable index/model/config:** consistent with
|
||
|
|
memsearch's own design philosophy ("markdown is canonical; index is derived") and with the
|
||
|
|
cc-os principle from ADR-008 ("sync the vault, not the indexes"). The 544 MB bge-m3 ONNX
|
||
|
|
model is not even in the store; the Milvus Lite DB rebuilds in one command. Committing them
|
||
|
|
would bloat the repo for zero durability gain.
|
||
|
|
- **Sync lives in cc-os hook, not the marketplace plugin:** the marketplace plugin's hook
|
||
|
|
scripts (`stop.sh`, `session-start.sh`, `user-prompt-submit.sh`, `session-end.sh`) perform
|
||
|
|
no git operations — only a read-only `git rev-parse --show-toplevel` for scoping. Adding
|
||
|
|
sync to a marketplace plugin that can be clobbered by an upstream update is fragile; owning
|
||
|
|
it in the cc-os `session-end.sh` keeps the sync logic under our control and version-tracked
|
||
|
|
in this repo.
|
||
|
|
- **Fail-safe contract:** the `|| true` wrap and `timeout 30` on push ensure that a network
|
||
|
|
outage, an unreachable Forgejo instance, or any git error cannot prevent a session from
|
||
|
|
closing cleanly. The SessionEnd hook's harness timeout is ~10 s, so push is effectively
|
||
|
|
capped; commits land locally first, and any unpushed commit is carried forward by the next
|
||
|
|
session's push (daily files are append-only, so no conflict risk).
|
||
|
|
- **Commingling — resolved 2026-06-09**: `~/.memsearch` aggregates session summaries across
|
||
|
|
**all clients** into one global store. This is an **explicit design choice**, not a tolerated
|
||
|
|
risk: the user deliberately accepts a single commingled global store across all clients.
|
||
|
|
Private self-hosted Forgejo (user-owned infrastructure) is the chosen sync venue; client-
|
||
|
|
agreement compliance is the user's knowing responsibility. The concern was previously recorded
|
||
|
|
as user-owned and unresolved; that is now closed: single global store is the intended design.
|
||
|
|
This also aligns with a forward direction of minimizing dedicated per-client working
|
||
|
|
directories — since memsearch captures memory globally regardless of cwd, a single
|
||
|
|
`clients/` area for non-project client work becomes viable (direction stated; not yet designed).
|
||
|
|
|
||
|
|
## Consequences
|
||
|
|
|
||
|
|
~/.memsearch becomes its own git repo synced to a private self-hosted Forgejo remote, tracking only the daily markdown session files (via a whitelist .gitignore) while excluding the derived/disposable Milvus index, config, and embeddings model; sync is wired into cc-os's own SessionEnd hook rather than the marketplace plugin, wrapped in a fail-safe subshell so it can never block session shutdown. The design explicitly accepts a single global store commingling all clients' episodic memory as an intentional choice, with client-agreement compliance left as the user's responsibility.
|
||
|
|
|
||
|
|
## Alternatives rejected
|
||
|
|
|
||
|
|
- **Fold into the Obsidian vault repo:** violates ADR-001's episodic/semantic separation.
|
||
|
|
Episodic logs have different lifecycle (accrete and decay), different write patterns
|
||
|
|
(auto-captured every session), and would clutter a vault intended for curated durable
|
||
|
|
knowledge.
|
||
|
|
- **Auto-commit in the marketplace plugin:** the marketplace plugin can be overwritten on
|
||
|
|
update, losing the sync logic silently. Out-of-band ownership in cc-os is safer.
|
||
|
|
- **Commit the Milvus Lite index or the embeddings model:** both are large binaries, derived
|
||
|
|
from the markdown source, and rebuildable. Committing them wastes space and provides no
|
||
|
|
additional durability. The markdown files are the canonical source; the index follows from
|
||
|
|
them.
|
||
|
|
- **Syncthing or rsync instead of git:** git provides both version history and conflict-free
|
||
|
|
daily-append semantics; Syncthing is continuous-async (suitable for the vault where changes
|
||
|
|
are sparse); git's push-on-session-end cadence matches how memsearch produces data (one
|
||
|
|
daily file per day, append-only). Git was already chosen for ADR-008's vault sync rationale;
|
||
|
|
applying the same mechanism to the episodic store is consistent.
|