Add assessment-first onboarding to memory-project skill (ADR-017)

Onboarding ran a bare `graphify extract`, but graphify does not honor
.gitignore — it needs its own .graphifyignore. Without one, onboarding
viking-warrior-training-log walked node_modules/ (5,858 files) and
routed its non-code files through the Ollama doc pass; the run was
killed after ~1 hour with no graph.json.

Onboarding is now assessment-first: survey the repo, generate a
per-project .graphifyignore (the ignore set varies by stack), confirm
with the user, then extract. Records the gotcha and decision in ADR-017.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
jared 2026-06-17 09:17:33 -04:00
parent 7ede69ec85
commit 72397a00b8
4 changed files with 32 additions and 5 deletions

View File

@ -69,12 +69,14 @@ to those two and fix the stale doc.
**Implementation status (2026-06-15):** Step 5b done and automated — vault (`~/Documents/SecondBrain`) initialized as git repo, pushed to private Forgejo (`ssh://git@forgejo.swansoncloud.com:2222/jared/SecondBrain.git`); 52 files, git chosen over Syncthing. Auto-commit+push wired via `vault_sync.py` SessionEnd hook (third SessionEnd hook, runs after `session_end.py` so the daily journal note is included; push-only — SessionStart pull is an optional future item for multi-machine). Step 2e pilot done — llf-schema (`/home/jared/dev/llf-schema`, PHP 8.2 WordPress plugin) onboarded via `memory-project` skill: 605 nodes / 930 edges / 52 communities; Ollama doc pass lossy on WordPress docs but AST pass solid. Step 2d closed — live vault `graphify-out/` is the baseline; fixture-only build superseded.
**Implementation status (2026-06-17):** `memory-project` skill now uses assessment-first onboarding: surveys the repo structure, generates a per-project `.graphifyignore` file to exclude non-code directories (node_modules/, etc.), confirms the ignore rules with the user, then extracts the graph. Fixes issue where repos with large dependency trees routed non-code files through the Ollama doc pass (see ADR-017).
## Implemented Components
**Global memory plugin** — `cc-os/plugins/memory/` (git-tracked, 2026-06-12); symlinked into `~/.claude/plugins/memory`
- Hooks: `hooks/``session_start.py`, `session_context.py` (project graph path only), `post_tool_use_write.py`, `session_end.py` (vault journal), `memsearch_sync.py` (second SessionEnd hook; memsearch auto-commit+push, 30s timeout), `vault_sync.py` (third SessionEnd hook; vault auto-commit+push to forgejo.swansoncloud.com/jared/SecondBrain, 30s timeout; mirrors memsearch_sync.py)
- Shared modules: `config.py` (load_config → frozen Config dataclass), `hook_io.py` (read_input → HookInput dataclass), `session_state.py` (record_touch/read_touches; encapsulates `/tmp/claude-vault-touched-$SESSION_ID` contract)
- Skills: `skills/` — memory-vault, memory-write, memory-reorganize, memory-project
- Skills: `skills/` — memory-vault, memory-write, memory-reorganize, memory-project (assessment-first onboarding: surveys repo, writes per-project `.graphifyignore`, confirms with user, then extracts — per ADR-017)
- Config: `config.yaml` — vault path, Ollama model (qwen25-coder-7b-16k), env vars
- Hook wiring: `~/.claude/settings.json` (hook entries invoke `/usr/bin/python3` with absolute paths into cc-os)

View File

@ -522,6 +522,17 @@ _Date: 2026-06-12_
not reversed).
- **Status**: Accepted. Cutover verified 2026-06-12.
## ADR-017 — Project graph onboarding assesses the repo and writes `.graphifyignore` before extracting
_Date: 2026-06-17_
- **Context**: The `/memory:memory-project` skill's onboarding flow ran `graphify extract . --backend ollama --model qwen2.5-coder:7b` directly. graphify does NOT honor `.gitignore`; it uses a separate `.graphifyignore` file (same syntax — see `docs/graphify/02-installation-setup.md`). With no `.graphifyignore`, onboarding `viking-warrior-training-log` (135 tracked files) walked `node_modules/` (5,858 files / 161MB) and routed every non-code file there through the Ollama doc pass. The run was killed after ~1 hour with no `graph.json` produced. Cost driver: only non-code files hit the Ollama LLM; code uses the free tree-sitter AST pass.
- **Decision**: Onboarding is now assessment-first. Before running `graphify extract`, the skill surveys the repo, classifies directories and file types into include/exclude (weighing non-code file count since only non-code files cost LLM time), confirms the proposed ignore list with the user, writes `.graphifyignore` at the repo root, ensures `graphify-out/` is in the project's `.gitignore`, then runs extract. The ignore list is generated per-project — what counts as noise varies by stack — not from a static template.
- **Rationale**: graphify's `.gitignore` blindness is by design; the correct lever is `.graphifyignore`, not a workaround. Assessment before extraction surfaces borderline dirs (migrations, seeds, fixtures, sample data) that need a human call. Per-project generation avoids the false safety of a shared template that silently mismatches a new stack.
- **Alternatives rejected**: **Static `.graphifyignore` template** — rejected; dependency/build/cache dirs vary by project stack; a template would both over-exclude (suppressing wanted source) and under-exclude (missing stack-specific dirs) for any given project. **Auto-write without user confirmation** — rejected; borderline directories (e.g. migrations, seeds, fixtures) require human judgment; mechanical exclusion would silently drop legitimate content. **AST-only mode (skip doc pass entirely)** — deferred; the `--backend ollama` flag stays; the ignore list is the correct cost lever, not backend switching.
- **Status**: Accepted 2026-06-17.
## Rejected tools (summary)
| Tool | Why rejected for our use |

View File

@ -357,6 +357,7 @@ Graphify config.
when to trigger `graphify --force` rebuild.
- `memory-project` **DONE (2026-06-08)** — full lifecycle skill for per-project Graphify graphs:
- **Onboard**: `graphify extract . --backend ollama --model qwen2.5-coder:7b`, then `graphify cluster-only .`, add `graphify-out/` to `.gitignore`
- **Onboard flow (updated 2026-06-17, ADR-017)**: Assessment-first — the skill surveys the repo and generates a per-project `.graphifyignore` for user confirmation before running extract, because Graphify does not honor `.gitignore`.
- **Query**: `graphify query ... --graph <path>` for project-context queries (wraps `memory-vault`)
- **Update**: `graphify update .` for incremental updates (no API cost) or `--force` rebuild after structural changes
- **Remove**: delete `graphify-out/` when no longer needed

View File

@ -6,16 +6,29 @@ description: Manage per-project Graphify knowledge graphs — onboard, update, r
## Onboard
Run from the project root:
Assessment-first onboarding per ADR-017. Do not run `graphify extract` bare — graphify does not honor `.gitignore`; it uses a separate `.graphifyignore` (same syntax). Without one, repos with `node_modules/` or other dependency/build/cache dirs will walk every file and route all non-code content through the slow Ollama doc pass. Only non-code files hit the LLM; code uses the free tree-sitter AST pass.
**Step 1 — Assess the repo.** Survey before touching graphify. List top-level dirs and get a file-type/size profile. Identify:
- **Exclude candidates:** dependency dirs (`node_modules/`, `vendor/`, `venv/`, `.venv/`); build/generated output (`dist/`, `build/`, `.next/`, `target/`, `pb_public/`); caches (`.vite/`, `coverage/`, `__pycache__/`); VCS internals (`.git/`); lockfiles; databases and logs (`*.db`, `*.log`); binary/media blobs; and `graphify-out/` itself.
- **Include candidates:** source code (free AST pass) and genuine docs (README, design docs, ADRs).
Weigh cost by non-code file count, not directory size. Surface borderline cases (migrations, seeds, fixtures, sample data) for a human call — do not make that judgment unilaterally.
**Step 2 — Draft `.graphifyignore`.** Write `.gitignore`-syntax rules from the assessment. Add a one-line rationale comment above each rule so the intent is clear later.
**Step 3 — Confirm with the user.** Present the proposed ignore list and rationale; adjust per feedback before writing anything. This is a judgment step, not a mechanical one — do not skip it.
**Step 4 — Write the files.** Write `.graphifyignore` at the repo root. Ensure `graphify-out/` is in the project's `.gitignore`; add it if missing.
**Step 5 — Build the graph.**
```bash
graphify extract . --backend ollama --model qwen2.5-coder:7b
graphify cluster-only .
```
Then ensure `graphify-out/` is in `.gitignore`. Confirm `graphify-out/graph.json` exists before reporting done.
Code-only repos use the free tree-sitter AST pass — no LLM cost. The `--backend ollama` flags are always specified so the single command handles mixed (code + docs) repos without requiring repo-type detection.
Confirm `./graphify-out/graph.json` exists before reporting done.
## Incremental update