From 72397a00b8e2f7f84d1baec34c325fc9836fceee Mon Sep 17 00:00:00 2001 From: jared Date: Wed, 17 Jun 2026 09:17:33 -0400 Subject: [PATCH] Add assessment-first onboarding to memory-project skill (ADR-017) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Onboarding ran a bare `graphify extract`, but graphify does not honor .gitignore — it needs its own .graphifyignore. Without one, onboarding viking-warrior-training-log walked node_modules/ (5,858 files) and routed its non-code files through the Ollama doc pass; the run was killed after ~1 hour with no graph.json. Onboarding is now assessment-first: survey the repo, generate a per-project .graphifyignore (the ignore set varies by stack), confirm with the user, then extract. Records the gotcha and decision in ADR-017. Co-Authored-By: Claude Opus 4.8 --- CLAUDE.md | 4 +++- .../03-architecture-decisions.md | 11 ++++++++++ docs/memory-system/04-build-plan.md | 1 + plugins/memory/skills/memory-project/SKILL.md | 21 +++++++++++++++---- 4 files changed, 32 insertions(+), 5 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 10bb310..036cd4a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -69,12 +69,14 @@ to those two and fix the stale doc. **Implementation status (2026-06-15):** Step 5b done and automated — vault (`~/Documents/SecondBrain`) initialized as git repo, pushed to private Forgejo (`ssh://git@forgejo.swansoncloud.com:2222/jared/SecondBrain.git`); 52 files, git chosen over Syncthing. Auto-commit+push wired via `vault_sync.py` SessionEnd hook (third SessionEnd hook, runs after `session_end.py` so the daily journal note is included; push-only — SessionStart pull is an optional future item for multi-machine). Step 2e pilot done — llf-schema (`/home/jared/dev/llf-schema`, PHP 8.2 WordPress plugin) onboarded via `memory-project` skill: 605 nodes / 930 edges / 52 communities; Ollama doc pass lossy on WordPress docs but AST pass solid. Step 2d closed — live vault `graphify-out/` is the baseline; fixture-only build superseded. +**Implementation status (2026-06-17):** `memory-project` skill now uses assessment-first onboarding: surveys the repo structure, generates a per-project `.graphifyignore` file to exclude non-code directories (node_modules/, etc.), confirms the ignore rules with the user, then extracts the graph. Fixes issue where repos with large dependency trees routed non-code files through the Ollama doc pass (see ADR-017). + ## Implemented Components **Global memory plugin** — `cc-os/plugins/memory/` (git-tracked, 2026-06-12); symlinked into `~/.claude/plugins/memory` - Hooks: `hooks/` — `session_start.py`, `session_context.py` (project graph path only), `post_tool_use_write.py`, `session_end.py` (vault journal), `memsearch_sync.py` (second SessionEnd hook; memsearch auto-commit+push, 30s timeout), `vault_sync.py` (third SessionEnd hook; vault auto-commit+push to forgejo.swansoncloud.com/jared/SecondBrain, 30s timeout; mirrors memsearch_sync.py) - Shared modules: `config.py` (load_config → frozen Config dataclass), `hook_io.py` (read_input → HookInput dataclass), `session_state.py` (record_touch/read_touches; encapsulates `/tmp/claude-vault-touched-$SESSION_ID` contract) -- Skills: `skills/` — memory-vault, memory-write, memory-reorganize, memory-project +- Skills: `skills/` — memory-vault, memory-write, memory-reorganize, memory-project (assessment-first onboarding: surveys repo, writes per-project `.graphifyignore`, confirms with user, then extracts — per ADR-017) - Config: `config.yaml` — vault path, Ollama model (qwen25-coder-7b-16k), env vars - Hook wiring: `~/.claude/settings.json` (hook entries invoke `/usr/bin/python3` with absolute paths into cc-os) diff --git a/docs/memory-system/03-architecture-decisions.md b/docs/memory-system/03-architecture-decisions.md index 2c637d2..c5fb028 100644 --- a/docs/memory-system/03-architecture-decisions.md +++ b/docs/memory-system/03-architecture-decisions.md @@ -522,6 +522,17 @@ _Date: 2026-06-12_ not reversed). - **Status**: Accepted. Cutover verified 2026-06-12. + +## ADR-017 — Project graph onboarding assesses the repo and writes `.graphifyignore` before extracting + +_Date: 2026-06-17_ + +- **Context**: The `/memory:memory-project` skill's onboarding flow ran `graphify extract . --backend ollama --model qwen2.5-coder:7b` directly. graphify does NOT honor `.gitignore`; it uses a separate `.graphifyignore` file (same syntax — see `docs/graphify/02-installation-setup.md`). With no `.graphifyignore`, onboarding `viking-warrior-training-log` (135 tracked files) walked `node_modules/` (5,858 files / 161MB) and routed every non-code file there through the Ollama doc pass. The run was killed after ~1 hour with no `graph.json` produced. Cost driver: only non-code files hit the Ollama LLM; code uses the free tree-sitter AST pass. +- **Decision**: Onboarding is now assessment-first. Before running `graphify extract`, the skill surveys the repo, classifies directories and file types into include/exclude (weighing non-code file count since only non-code files cost LLM time), confirms the proposed ignore list with the user, writes `.graphifyignore` at the repo root, ensures `graphify-out/` is in the project's `.gitignore`, then runs extract. The ignore list is generated per-project — what counts as noise varies by stack — not from a static template. +- **Rationale**: graphify's `.gitignore` blindness is by design; the correct lever is `.graphifyignore`, not a workaround. Assessment before extraction surfaces borderline dirs (migrations, seeds, fixtures, sample data) that need a human call. Per-project generation avoids the false safety of a shared template that silently mismatches a new stack. +- **Alternatives rejected**: **Static `.graphifyignore` template** — rejected; dependency/build/cache dirs vary by project stack; a template would both over-exclude (suppressing wanted source) and under-exclude (missing stack-specific dirs) for any given project. **Auto-write without user confirmation** — rejected; borderline directories (e.g. migrations, seeds, fixtures) require human judgment; mechanical exclusion would silently drop legitimate content. **AST-only mode (skip doc pass entirely)** — deferred; the `--backend ollama` flag stays; the ignore list is the correct cost lever, not backend switching. +- **Status**: Accepted 2026-06-17. + ## Rejected tools (summary) | Tool | Why rejected for our use | diff --git a/docs/memory-system/04-build-plan.md b/docs/memory-system/04-build-plan.md index f101204..794202f 100644 --- a/docs/memory-system/04-build-plan.md +++ b/docs/memory-system/04-build-plan.md @@ -357,6 +357,7 @@ Graphify config. when to trigger `graphify --force` rebuild. - `memory-project` **DONE (2026-06-08)** — full lifecycle skill for per-project Graphify graphs: - **Onboard**: `graphify extract . --backend ollama --model qwen2.5-coder:7b`, then `graphify cluster-only .`, add `graphify-out/` to `.gitignore` + - **Onboard flow (updated 2026-06-17, ADR-017)**: Assessment-first — the skill surveys the repo and generates a per-project `.graphifyignore` for user confirmation before running extract, because Graphify does not honor `.gitignore`. - **Query**: `graphify query ... --graph ` for project-context queries (wraps `memory-vault`) - **Update**: `graphify update .` for incremental updates (no API cost) or `--force` rebuild after structural changes - **Remove**: delete `graphify-out/` when no longer needed diff --git a/plugins/memory/skills/memory-project/SKILL.md b/plugins/memory/skills/memory-project/SKILL.md index 9a2ec46..e91264e 100644 --- a/plugins/memory/skills/memory-project/SKILL.md +++ b/plugins/memory/skills/memory-project/SKILL.md @@ -6,16 +6,29 @@ description: Manage per-project Graphify knowledge graphs — onboard, update, r ## Onboard -Run from the project root: +Assessment-first onboarding per ADR-017. Do not run `graphify extract` bare — graphify does not honor `.gitignore`; it uses a separate `.graphifyignore` (same syntax). Without one, repos with `node_modules/` or other dependency/build/cache dirs will walk every file and route all non-code content through the slow Ollama doc pass. Only non-code files hit the LLM; code uses the free tree-sitter AST pass. + +**Step 1 — Assess the repo.** Survey before touching graphify. List top-level dirs and get a file-type/size profile. Identify: + +- **Exclude candidates:** dependency dirs (`node_modules/`, `vendor/`, `venv/`, `.venv/`); build/generated output (`dist/`, `build/`, `.next/`, `target/`, `pb_public/`); caches (`.vite/`, `coverage/`, `__pycache__/`); VCS internals (`.git/`); lockfiles; databases and logs (`*.db`, `*.log`); binary/media blobs; and `graphify-out/` itself. +- **Include candidates:** source code (free AST pass) and genuine docs (README, design docs, ADRs). + +Weigh cost by non-code file count, not directory size. Surface borderline cases (migrations, seeds, fixtures, sample data) for a human call — do not make that judgment unilaterally. + +**Step 2 — Draft `.graphifyignore`.** Write `.gitignore`-syntax rules from the assessment. Add a one-line rationale comment above each rule so the intent is clear later. + +**Step 3 — Confirm with the user.** Present the proposed ignore list and rationale; adjust per feedback before writing anything. This is a judgment step, not a mechanical one — do not skip it. + +**Step 4 — Write the files.** Write `.graphifyignore` at the repo root. Ensure `graphify-out/` is in the project's `.gitignore`; add it if missing. + +**Step 5 — Build the graph.** ```bash graphify extract . --backend ollama --model qwen2.5-coder:7b graphify cluster-only . ``` -Then ensure `graphify-out/` is in `.gitignore`. Confirm `graphify-out/graph.json` exists before reporting done. - -Code-only repos use the free tree-sitter AST pass — no LLM cost. The `--backend ollama` flags are always specified so the single command handles mixed (code + docs) repos without requiring repo-type detection. +Confirm `./graphify-out/graph.json` exists before reporting done. ## Incremental update