cc-os/docs/memory-system/08-sb-active-use-plan.md

127 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

> **Superseded.** This plan was refactored into two focused plans following a design session on 2026-06-27:
> - [Plan A — Content](09-sb-content-plan.md): note types, templates, and vault population
> - [Plan B — Findability](10-sb-findability-plan.md): vault-index.json, /memory-find, SessionStart injection, SessionEnd agent
>
> Retain this file for historical reference only.
# SecondBrain Active Use Plan
_Last updated: 2026-07-02_
_Status: Planning_
## Problem
The Obsidian vault at `~/Documents/SecondBrain` has been dormant for 18 days, with no new notes added despite being positioned as the system's semantic memory layer. Projects and sessions lack clear lightweight triggers for when to write to the vault or query it, causing the vault to feel like a maintenance burden rather than an active tool. Without deliberate, low-friction surfaces that make the vault immediately useful, adoption stalls.
## Design Principles
- **Vault as library, not navigation:** Do not pre-compute "when to read X." Make querying cheap and let the AI decide at the point of need. The vault is a query-driven resource, not a prescribed workflow.
- **Write-once, reuse-many:** Notes are authored and improved in a single canonical location, shared across projects via tag queries. When `convention/ruby-coding-standards.md` improves, every Ruby project benefits automatically without duplication or sync overhead.
- **Progressive disclosure:** Start with lightweight tag-based search (one layer). Full notes are opened on-demand, not pre-loaded. Query results surface paths + one-line summaries; the AI reads summaries and decides which notes to fetch.
- **Minimal injection overhead:** SessionStart hook injection targets ~200 tokens total: a project note (if it exists) plus a 2-3 line instruction on how to use the vault. No embedded links, no pre-computed indexes in the project repo.
## Scope
This plan covers seven implementation pieces:
1. Note type templates and vault-conventions.md updates
2. High-priority knowledge migration (45 notes)
3. vault-index.json and rebuild hook
4. `/memory-find` query skill
5. Project note format and initial project notes
6. SessionStart hook extension for project note + vault instruction injection
7. `/memory-audit` skill for ongoing project-to-vault migration discovery
All pieces are self-contained; they build on each other but do not introduce external dependencies (no SQLite, no FTS5 at this stage, no PocketBase).
## Out of Scope
- **Bulk vault migration:** Migrate notes incrementally by type and project (ADR-013 principle). This plan seeds 45 high-priority notes; remaining notes migrate project-by-project as `/memory-audit` identifies them.
- **SQLite or full-text search upgrade:** vault-index.json (JSON mapping tag → metadata) is sufficient for current scale (targeting 200+ notes). Upgrade when measured query latency becomes a bottleneck.
- **Multi-machine vault pull:** Push-only design is locked (2026-06-15). SessionStart pull for multi-machine sync is explicitly deferred.
- **memsearch ↔ vault integration:** memsearch (episodic memory) and SecondBrain (semantic memory) remain separate by design (ADR-006, ADR-010). No cross-system queries in this phase.
## Surfaces Touched
- **Vault content:** `~/Documents/SecondBrain/` (notes, templates, vault-conventions.md updates)
- **Plugin code:** `cc-os/plugins/os-vault/` (hooks: `session_start.py` extended; new build step for index)
- **Plugin config:** `config.yaml` — no changes required (log index build status if helpful)
- **Settings:** `~/.claude/settings.json` — only if new hook is needed; likely already covered by SessionStart
- **Project repos:** No changes to project CLAUDE.md files. Project notes live in SecondBrain, not in repos.
## Risks
| Risk | Mitigation |
|------|-----------|
| **Index staleness** | Rebuild hook triggers on vault note write (SessionEnd), not on every query. Stale index is acceptable; index is regenerated within one session boundary. |
| **Tag explosion** | Enforce the six-facet taxonomy strictly during migration. Audit skill flags non-conforming tags and suggests corrections before vault-index.json rebuild. |
| **Over-querying vault** | `/memory-find` returns unfiltered tag matches. If a tag is too broad (e.g., `tool/ruby` in a large vault), result set is large. AI reads summaries and selects; no hard limit needed yet. Revisit if query result counts consistently exceed 10 matches. |
| **Project notes feel boilerplate** | Keep project notes minimal: name, 1-sentence description, tags only. No embedded reference links or pre-computed categories. Brevity prevents "maintenance tax" feeling. |
| **Hook performance** | vault-index.json rebuild is O(n) file scans. For 200 notes, ~500ms. Acceptable at SessionEnd; revisit if rebuild takes >2s. |
| **Audit skill churn** | `/memory-audit` discovers migration candidates but doesn't auto-migrate. Manual decision gate prevents vault sprawl. Migration is deliberate, not automatic. |
## Build Order
### 1. Templates + vault-conventions updates (Size: S)
Add note type templates and taxonomy guidance to `vault-conventions.md`. Define what each note type contains: howto (steps, prerequisites, gotchas), reference (lookup data, static facts), adr (decision, alternatives, rationale), convention (rule, rationale, exceptions), tool (commands, config, when to use). Update the vault-conventions.md to emphasize the DRY principle and tag-query-first workflow. Templates live in `~/Documents/SecondBrain/templates/` for Obsidian template picker.
### 2. Migrate 45 high-priority notes (Size: M)
Using templates and taxonomy guidance, author 45 high-value notes that already exist as scattered docs or tribal knowledge. Candidates identified in "High-Priority Migration Candidates" section below. Write in vault using correct tags; commit to SecondBrain git repo.
### 3. vault-index.json + rebuild hook (Size: M)
Create a JSON file at `~/Documents/SecondBrain/vault-index.json` with structure `{tag: [{path, title, summary}]}`. Extend `session_end.py` (or add a dedicated hook if `session_end.py` is already full) to rebuild the index by scanning vault notes (frontmatter tags + first 12 sentences as summary). Rebuild only when SessionEnd fires (vault is pushed); no real-time index updates. Save index to vault root so it's tracked in git and pushed with every vault sync.
### 4. `/memory-find` skill (Size: S)
Create a skill in `cc-os/plugins/os-vault/skills/memory-find.md` that accepts a tag argument, loads vault-index.json, filters for matching tags, and returns results as a bulleted list: path, title, summary. One line per match. No embedded note fetching; AI reads summaries and decides which notes to open with `/memory-recall` or manual Read tool calls.
### 5. Project note format + initial project notes (Size: S)
Define project note format: `project/[name].md` containing project name, one-sentence description, and a bulleted tag list (e.g., `tool/ruby`, `client/acme`, `project/inventory-service`). Create 23 pilot project notes for projects already in scope (e.g., cc-os itself, a client project, a tool project). No template links, no category pre-computation. Tags are the interface.
### 6. SessionStart injection (Size: S)
Extend `session_start.py` (or the equivalent hook) to check if a project note exists for the current project (by name). If it exists, inject the project note name + tags into the session context, plus 23 lines telling the AI: "SecondBrain is available. Use `/memory-find tag-name` to query notes by tag, then read full notes with `/memory-recall path`. DRY principle: notes are shared across projects." Total injection: ~200 tokens, at the top of the session context so it's visible but not verbose.
### 7. `/memory-audit` skill (Size: M)
Create a skill that scans the current project for knowledge artifacts (CLAUDE.md, README, docs/, ADRs, how-tos, etc.), extracts a brief summary of each, checks vault-index.json to see if equivalent coverage already exists, and reports: "Found 3 docs; 1 is already in vault under `howto/deployment`, suggest migrating 2 others to vault." Returns candidates for manual migration. Skill runs at user request, not automatically. Audit report feeds into step 2 loop: candidate → note template → authored → indexed.
## High-Priority Migration Candidates
These are real, existing documents identified in an audit of active project repos. Ordered by cross-project value.
| Source Path | Proposed Vault Path | Note Type | Rationale |
|-------------|---------------------|-----------|-----------|
| `~/dev/hyperthrive_dev/docs/agent-orchestration-patterns.md` | `type/reference/agent-orchestration-patterns.md` | reference | Framework for minimizing context accumulation across multiple agents. Needed in delta-refinery too — already being duplicated. |
| `~/dev/hyperthrive_dev/docs/agent-orchestration-cookbook.md` | `type/reference/agent-orchestration-cookbook.md` | reference | Companion recipes to patterns doc. Pair as hub note + detail note. |
| `~/dev/delta-refinery/docs/agent-pre-post-handoff.md` | `type/howto/pre-post-handoff-pattern.md` | howto | Pre/Post/Handoff pattern for multi-level agent coordination. Cross-project pattern currently locked in one repo. |
| `~/dev/hyperthrive_dev/PROGRESSIVE_DISCLOSURE.md` | `convention/progressive-disclosure.md` | convention | Context management pattern for large codebases. Directly relevant to how this vault itself is designed. |
| `~/dev/thinkfast/docs/conventions.md` (+ duplicate in `~/dev/verona-vocab/CLAUDE.md`) | `convention/sandi-metz-code-philosophy.md` | convention | Code philosophy derived from POODR/99 Bottles. Already duplicated across two repos — canonical vault note eliminates the copies. |
| `~/dev/hyperthrive_dev/AGENTS.md` | `type/reference/ai-team-role-hierarchy.md` | reference | CEO→PM→Architect→Dev→QA→Ops role framework for multi-agent projects. Medium priority; useful when spinning up new agent-heavy projects. |
**Not included (kept in-repo by design):** Per-project ADRs stay with their repos — they document project-specific decisions, not generalizable patterns. The memory system ADRs in cc-os similarly stay in `docs/memory-system/03-architecture-decisions.md`. If a decision *generalizes* (e.g., "always use progressive disclosure for large codebases"), it gets distilled into a convention or reference note rather than copying the ADR itself.
## Note Types and Templates Needed
- **howto:** Step-by-step instructions with prerequisites and gotchas. Example: "Running Graphify on a new project," "Debugging a memsearch query," "Deploying llf-schema to staging." Include terminal output / expected outcomes where applicable.
- **reference:** Lookup data, architecture diagrams (as text or links), static facts that rarely change. Example: "Memory system architecture," "Ruby stdlib conventions," "Client billing codes."
- **adr:** Architecture Decision Record. Problem statement, alternatives considered, decision, rationale, and deferred items. Links to related ADRs. Replaces inline ADRs in project CLAUDE.md.
- **convention:** Rules, patterns, and exceptions. Example: "File naming," "Commit message style," "When to use Graphify vs. full-text search." Rarely longer than one page.
- **tool:** Commands, common config patterns, when to use. Example: "Graphify cheat sheet," "memsearch CLI reference," "Obsidian query syntax for vault."
## Open Questions
1. **Project note auto-generation vs. manual:** Should `/memory-audit` suggest and auto-create project notes, or should they be authored by hand? Current design assumes manual (audit discovers, user confirms and authors). Revisit if manual authoring becomes a bottleneck.
2. **Index rebuild frequency vs. freshness:** Rebuild at SessionEnd only (batched), or on every vault note write? Current design: SessionEnd only (batches multiple edits, reduces hook latency). If vault is edited multiple times in one session and a later hook needs current index, we miss edits within the session. Acceptable risk for now; flag if `/memory-find` starts returning stale results.
3. **Result set size limits:** `/memory-find` returns all matches for a tag. If `tool/ruby` has 20 matches in a 200-note vault, should the skill limit to top 10 by recency/relevance, or return all with summaries and let the AI filter? Current design: return all, let AI filter by reading summaries. Revisit if query result sets consistently exceed 15 items and AI spends significant time skimming.
---
**Next step:** Proceed with step 1 (templates) or sync this plan with the project via ADR if design refinement is needed before build starts.