cc-os/docs/memory-system/01-video-synthesis.md

Video Synthesis — The 6 Levels of Claude Code Memory

Synthesis of the YouTube video "The 6 Levels of Claude Code Memory" plus the referenced articles (John Connelly / Pawel Huryn). This is background: the source of several ideas in our design. Our system borrows from Levels 1–3 and deliberately ignores 4–6.

The framing

Every memory system answers the same question: when you give Claude Code a task, how does it pull the right context at the right time? Each level differs on just two axes:

• Where memory lives — the storage mechanism and file structure (markdown vs vectors, local vs cloud).
• How Claude gets it — the retrieval stage (auto-injected into context, searched on demand in a DB, etc.).

The recurring enemy is context rot: as you load more context, the model recalls less of it reliably. The cure throughout is progressive disclosure — load a small index, pull the detail only when needed.

The levels

Level 1 — What ships natively (CLAUDE.md + memory.md / automemory)

• CLAUDE.md: plain markdown in the project, always loaded every session like a system prompt. Hierarchical (project-folder → root → individual project); lower levels inherit the parent but local rules win on conflict.
• Common mistake: stuffing it full (brand guide, tone doc, client list) → burns context → context rot. Rule of thumb: keep CLAUDE.md under 200 lines; push larger context into separate files referenced from CLAUDE.md so they load only when needed.
• memory.md / automemory: /memory shows imported/project/user memory. Auto-memory keeps a per-project memory.md that acts as an index of pointers to many small memory files — Claude quietly takes notes in the background and builds the index.
• Anthropic is clearly working on this natively (leaked references to an unreleased always-on consolidation daemon, "Chyros"). Native memory will only get better.

Level 2 — Forcing reliable recall (Connelly hook / Huryn structure)

• Paste a memory-management prompt + rules into CLAUDE.md. Structured memory rooted at ~/.claude/memory/: memory.md (index), general.md (cross-project facts/prefs/env), domain/<topic>.md (one file per topic), tools/<tool>.md (one file per tool, e.g. slack.md with config/workarounds/edge cases).
• A session-start hook auto-injects the index (not full content) into every session and sub-agent — not relying on Claude choosing to read it.
• A "reorganize memory" command periodically reads all memory files, removes duplicates/outdated entries, merges related ones, splits overloaded files, sorts by date, and rebuilds the index.
• Huryn's updated post (productcompass.pm) adds active hypothesis tracking, a catalog of "false beliefs," and an AI-proposes-reorganization / human-keeps-editorial-control loop. The transferable mechanic (not the content-writing domain) is the hypothesis/ false-belief log + propose-and-approve loop — useful for per-client "what we tried / decided / what didn't work."
• Author's take: most people should stop here.

Level 3 — Search by meaning, not keywords (memsearch; OpenClaw template)

• Add only if you've used Claude Code > 1 month, have many memory files, and have asked something you know is in your notes that Claude couldn't find.
• OpenClaw memory design (3 layers): (1) memory.md = long-term durable facts, loaded at session start; (2) daily note files = one per date, a running log, today+yesterday auto-loaded, older left on disk; (3) optional "dreaming" = background process that scores daily notes and promotes recurring ones into long-term memory, forgetting stale stuff.
• memsearch (by Zilliz): ports this into Claude Code as a two-line plugin. Markdown-first, same chunking/structure. Chunks into semantic vectors; a user-prompt-submit hook auto-injects the top-3 semantic matches into every prompt. Plain-readable markdown.
• Alternative claude-mem: MCP-based, 3-tier storage (summaries/timeline/observations), dashboard/team/cost features. Author's view: overkill, and MCP-based means Claude must actively call the search tool; stores opaque blobs vs memsearch's readable markdown.

Level 4 — Verbatim conversation recall (MemPalace)

• Local RAG, free, claims highest published benchmark. Stores words verbatim (nothing summarized) indexed in a dense symbolic dialect ("memory palace": wings → rooms → closets → drawers). Two DBs: SQL (entities/relationships) + Chroma (vector chunks). Background hooks on session-end/pre-compaction. ~42ms retrieval.
• Downsides: storage is not readable markdown; drawers are isolated (knowledge not interconnected); local-only.

Level 5 — Self-organizing knowledge base (Karpathy LLM wiki; Recall; LightRAG)

• Karpathy's LLM wiki: raw/ (you drop sources, Claude reads, never writes) + wiki/ (Claude owns entirely). Plain markdown, Obsidian graph. Recall is a hosted done-for-you version (you don't own the data). LightRAG is enterprise knowledge-graph overkill.
• Author's take: these are for content knowledge bases / deep research, NOT operational memory ("what did we decide about client X's landing page in March"). Skip for our use case.

Level 6 — One brain for all AI tools (OpenBrain; Mem0)

• OpenBrain: memory in a Postgres DB you own (Supabase), one thoughts table (text + embedding + tags + timestamp), MCP server fronting it so any AI tool shares the same memory. Most portable/future-proof. Downsides: longer/harder setup; always remote → latency on every query; small monthly cost. Mem0: cross-tool layer, fast setup, but data lives on their servers permanently.
• Author's take: only if you live across many AI tools and want real-time shared memory.

Author's recommendation

• Just starting → Level 1 done right.
• A bit in → Level 2 (Connelly hook). Most should stop here.
• Lots of context, losing old decisions → Level 3 (memsearch) or Level 4 (MemPalace).
• Levels 5–6 are a different realm for specific use cases.
• Levels 1+2+3 stack (similar folder structures). The author personally runs up to Level 3: OpenClaw conventions + semantic search + injection hooks.

What we took / left (see ADRs for why)

• Took: progressive-disclosure index (L1/L2), per-tool/per-domain granularity + reorganize command + session hooks (L2), Huryn's propose-and-approve reorg loop, OpenClaw daily-notes + dreaming and memsearch (L3).
• Replaced: L2's folders with namespaced tags in a flat vault.
• Left: MemPalace (opaque, isolated), Recall/LightRAG (content KB, not operational), OpenBrain/Mem0 (always-remote, fights local-fast).

Reference links

• John & Pawel's system — youngleaders.tech/p/how-i-finally-sorted-my-claude-code-memory
• Pawel Huryn (updated) — productcompass.pm/p/self-improving-claude-system
• memsearch — github.com/zilliztech/memsearch
• MemPalace — github.com/MemPalace/mempalace
• Karpathy LLM wiki — gist.github.com/karpathy
• Recall — recall.it · Mem0 — mem0.ai · OpenBrain — github.com/NateBJones-Project
• QMD — github.com/tobi/qmd (evaluated as a semantic layer; superseded by Graphify — see ADR-010)