Refine memory-project onboarding: type-based exclude taxonomy + 16k-context model

Assessment now reasons about ignorable content by category (fetched
deps, build output, caches, tooling/AI-assistant dirs, lockfiles, bulk
data, binaries) so it generalizes across stacks. Onboarding switches to
the config.yaml model qwen25-coder-7b-16k: inference is GPU-bound at
~65 tok/s, so the speed lever is fewer chunks per doc via a larger
context window (GRAPHIFY_OLLAMA_NUM_CTX does not propagate through the
OpenAI-compatible path; context must come from the modelfile). Extends
ADR-017.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

CLAUDE.md

@@ -69,12 +69,14 @@ to those two and fix the stale doc.
 
 **Implementation status (2026-06-15):** Step 5b done and automated — vault (`~/Documents/SecondBrain`) initialized as git repo, pushed to private Forgejo (`ssh://git@forgejo.swansoncloud.com:2222/jared/SecondBrain.git`); 52 files, git chosen over Syncthing. Auto-commit+push wired via `vault_sync.py` SessionEnd hook (third SessionEnd hook, runs after `session_end.py` so the daily journal note is included; push-only — SessionStart pull is an optional future item for multi-machine). Step 2e pilot done — llf-schema (`/home/jared/dev/llf-schema`, PHP 8.2 WordPress plugin) onboarded via `memory-project` skill: 605 nodes / 930 edges / 52 communities; Ollama doc pass lossy on WordPress docs but AST pass solid. Step 2d closed — live vault `graphify-out/` is the baseline; fixture-only build superseded.
 
+**Implementation status (2026-06-17):** `memory-project` skill now uses assessment-first onboarding: surveys the repo structure, classifies exclude candidates by TYPE (11 categories: fetched deps, build output, caches, VCS internals, editor/AI-tooling dirs, lockfiles, coverage/logs, bulk data, binaries, secrets, graphify-out/ — illustrative names, not fixed templates), generates a per-project `.graphifyignore`, confirms with the user, then extracts the graph using `qwen25-coder-7b-16k` (config.yaml `ollama_model`; 16k-context build of qwen2.5-coder:7b — larger context window cuts chunk count per doc, the main speed lever). Fixes issue where repos with large dependency trees routed non-code files through the Ollama doc pass (see ADR-017).
+
 ## Implemented Components
 
 **Global memory plugin** — `cc-os/plugins/memory/` (git-tracked, 2026-06-12); symlinked into `~/.claude/plugins/memory`
 - Hooks: `hooks/` — `session_start.py`, `session_context.py` (project graph path only), `post_tool_use_write.py`, `session_end.py` (vault journal), `memsearch_sync.py` (second SessionEnd hook; memsearch auto-commit+push, 30s timeout), `vault_sync.py` (third SessionEnd hook; vault auto-commit+push to forgejo.swansoncloud.com/jared/SecondBrain, 30s timeout; mirrors memsearch_sync.py)
 - Shared modules: `config.py` (load_config → frozen Config dataclass), `hook_io.py` (read_input → HookInput dataclass), `session_state.py` (record_touch/read_touches; encapsulates `/tmp/claude-vault-touched-$SESSION_ID` contract)
-- Skills: `skills/` — memory-vault, memory-write, memory-reorganize, memory-project
+- Skills: `skills/` — memory-vault, memory-write, memory-reorganize, memory-project (assessment-first onboarding: surveys repo, classifies excludes by type (11 categories), writes per-project `.graphifyignore`, confirms with user, then extracts using `qwen25-coder-7b-16k` — per ADR-017)
 - Config: `config.yaml` — vault path, Ollama model (qwen25-coder-7b-16k), env vars
 - Hook wiring: `~/.claude/settings.json` (hook entries invoke `/usr/bin/python3` with absolute paths into cc-os)
 

docs/memory-system/03-architecture-decisions.md

@@ -1,6 +1,6 @@
 # Architecture Decision Records
 
-_Last updated: 2026-06-12_
+_Last updated: 2026-06-17_
 
 A running log of decisions and *why*. Format per entry: Context · Decision · Rationale ·
 Alternatives rejected · Status. Newest decisions extend the log; supersede rather than delete.
@@ -522,6 +522,21 @@ _Date: 2026-06-12_
   not reversed).
 - **Status**: Accepted. Cutover verified 2026-06-12.
 
+
+## ADR-017 — Project graph onboarding assesses the repo and writes `.graphifyignore` before extracting
+
+_Date: 2026-06-17_
+
+- **Context**: The `/memory:memory-project` skill's onboarding flow ran `graphify extract . --backend ollama --model qwen2.5-coder:7b` directly. graphify does NOT honor `.gitignore`; it uses a separate `.graphifyignore` file (same syntax — see `docs/graphify/02-installation-setup.md`). With no `.graphifyignore`, onboarding `viking-warrior-training-log` (135 tracked files) walked `node_modules/` (5,858 files / 161MB) and routed every non-code file there through the Ollama doc pass. The run was killed after ~1 hour with no `graph.json` produced. Cost driver: only non-code files hit the Ollama LLM; code uses the free tree-sitter AST pass.
+- **Decision**: Onboarding is now assessment-first. Before running `graphify extract`, the skill surveys the repo, classifies directories and file types into include/exclude (weighing non-code file count since only non-code files cost LLM time), confirms the proposed ignore list with the user, writes `.graphifyignore` at the repo root, ensures `graphify-out/` is in the project's `.gitignore`, then runs extract. The ignore list is generated per-project — what counts as noise varies by stack — not from a static template.
+- **Rationale**: graphify's `.gitignore` blindness is by design; the correct lever is `.graphifyignore`, not a workaround. Assessment before extraction surfaces borderline dirs (migrations, seeds, fixtures, sample data) that need a human call. Per-project generation avoids the false safety of a shared template that silently mismatches a new stack.
+- **Alternatives rejected**: **Static `.graphifyignore` template** — rejected; dependency/build/cache dirs vary by project stack; a template would both over-exclude (suppressing wanted source) and under-exclude (missing stack-specific dirs) for any given project. **Auto-write without user confirmation** — rejected; borderline directories (e.g. migrations, seeds, fixtures) require human judgment; mechanical exclusion would silently drop legitimate content. **AST-only mode (skip doc pass entirely)** — deferred; the `--backend ollama` flag stays; the ignore list is the correct cost lever, not backend switching.
+- **Status**: Accepted 2026-06-17.
+
+**Addendum — 2026-06-17:**
+- **Default-exclude taxonomy is type-based, not name-based.** The skill's assessment step organizes exclude candidates into 11 categories by KIND (fetched deps, build/generated output, caches, VCS internals, editor/AI-tooling dirs, lockfiles, coverage/logs, bulk data/databases, binaries, secrets, graphify-out/) with a meta-principle: index what a human authored; exclude anything fetched, generated, cached, tooling config, bulk data/binary, or secret. Example names within each category are illustrative, not exhaustive — the goal is to recognize the kind even in an unfamiliar stack. See SKILL.md for the full table.
+- **Onboarding uses the 16k-context model.** The `graphify extract` command now passes `--model qwen25-coder-7b-16k` (the `ollama_model` from config.yaml), a 16k-context build of qwen2.5-coder:7b. Inference is GPU-bound (~65 tok/s); the main speed lever is chunk-count × generation — fewer chunks per doc via a larger context window. `GRAPHIFY_OLLAMA_NUM_CTX` does not propagate through graphify's OpenAI-compatible endpoint; the larger context must come from the model's Modelfile.
+
 ## Rejected tools (summary)
 
 | Tool | Why rejected for our use |

docs/memory-system/04-build-plan.md

@@ -357,6 +357,7 @@ Graphify config.
     when to trigger `graphify --force` rebuild.
   - `memory-project` **DONE (2026-06-08)** — full lifecycle skill for per-project Graphify graphs:
     - **Onboard**: `graphify extract . --backend ollama --model qwen2.5-coder:7b`, then `graphify cluster-only .`, add `graphify-out/` to `.gitignore`
+    - **Onboard flow (updated 2026-06-17, ADR-017)**: Assessment-first — the skill surveys the repo and generates a per-project `.graphifyignore` for user confirmation before running extract, because Graphify does not honor `.gitignore`.
     - **Query**: `graphify query ... --graph <path>` for project-context queries (wraps `memory-vault`)
     - **Update**: `graphify update .` for incremental updates (no API cost) or `--force` rebuild after structural changes
     - **Remove**: delete `graphify-out/` when no longer needed

plugins/memory/skills/memory-project/SKILL.md

@@ -6,16 +6,48 @@ description: Manage per-project Graphify knowledge graphs — onboard, update, r
 
 ## Onboard
 
-Run from the project root:
+Assessment-first onboarding per ADR-017. Do not run `graphify extract` bare — graphify does not honor `.gitignore`; it uses a separate `.graphifyignore` (same syntax). Without one, repos with `node_modules/` or other dependency/build/cache dirs will walk every file and route all non-code content through the slow Ollama doc pass. Only non-code files hit the LLM; code uses the free tree-sitter AST pass.
+
+**Step 1 — Assess the repo.** Survey before touching graphify. List top-level dirs and get a file-type/size profile. Identify what to exclude using the taxonomy below, then surface borderline cases (migrations, seeds, fixtures, sample data) for a human call — do not make that judgment unilaterally.
+
+**Meta-principle:** index what a human authored as this project's source and knowledge; exclude anything that is (a) fetched, (b) generated/derived/compiled, (c) cached, (d) tooling/environment config, (e) bulk data or binary, or (f) secret. Recognize the KIND even in an unfamiliar stack — the example names are illustrative, not exhaustive. Weight remaining cost by **non-code file count**: only non-code files go through the Ollama doc pass (~15–25s each), so a stray docs-heavy tooling dir or a large lockfile is what blows up runtime, not source-file count.
+
+**Default-exclude categories:**
+
+| # | Category | Illustrative names |
+|---|----------|--------------------|
+| 1 | Fetched dependencies | `node_modules/`, `vendor/`, `.venv/`/`venv/`, `target/`, `Pods/`, `site-packages/` |
+| 2 | Build / compiled / generated output | `dist/`, `build/`, `out/`, `.next/`, `.svelte-kit/`, `bin/`, `obj/`, `*.min.js`, source maps, generated dirs |
+| 3 | Caches | `.cache/`, `.vite/`, `.turbo/`, `__pycache__/`, `.pytest_cache/`, `.gradle/`, `.terraform/` |
+| 4 | VCS internals | `.git/`, `.hg/`, `.svn/` |
+| 5 | Editor / IDE & AI-assistant tooling dirs | `.vscode/`, `.idea/`, `.claude/`, `.codex/`, `.cursor/`, `.pi/`, `.impeccable/`, `.husky/` — these are full of markdown that floods the doc pass |
+| 6 | Lockfiles (machine-generated, hit the LLM) | `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `poetry.lock`, `Cargo.lock`, `composer.lock`, `Gemfile.lock` |
+| 7 | Coverage / reports / logs | `coverage/`, `htmlcov/`, `test-results/`, `playwright-report/`, `*.log`, `logs/` |
+| 8 | Data, databases & bulk dumps | `*.db`, `*.sqlite`, `pb_data/`, large `*.csv`/`*.json`/`*.parquet`, snapshots — keep small schema/seed files that define structure; drop bulk data |
+| 9 | Binary & media | images, audio, video, fonts, archives, PDFs/Office docs unless they ARE the knowledge |
+| 10 | Secrets / env | `.env`, `*.pem`, credentials |
+| 11 | graphify-out/ | graphify's own output — avoid self-ingestion |
+
+For non-coding/writing/research projects the same principle holds but offenders shift: exported duplicates of source notes (PDF/HTML/DOCX exports of the same markdown — index the source, not the export), attachment/media folders, app config (`.obsidian/`), and archived old-version folders.
+
+- **Include candidates:** source code (free AST pass) and genuine docs (README, design docs, ADRs).
+
+**Step 2 — Draft `.graphifyignore`.** Write `.gitignore`-syntax rules from the assessment. Add a one-line rationale comment above each rule so the intent is clear later.
+
+**Step 3 — Confirm with the user.** Present the proposed ignore list and rationale; adjust per feedback before writing anything. This is a judgment step, not a mechanical one — do not skip it.
+
+**Step 4 — Write the files.** Write `.graphifyignore` at the repo root. Ensure `graphify-out/` is in the project's `.gitignore`; add it if missing.
+
+**Step 5 — Build the graph.**
 
 ```bash
-graphify extract . --backend ollama --model qwen2.5-coder:7b
+graphify extract . --backend ollama --model qwen25-coder-7b-16k
 graphify cluster-only .
 ```
 
-Then ensure `graphify-out/` is in `.gitignore`. Confirm `graphify-out/graph.json` exists before reporting done.
+`qwen25-coder-7b-16k` is the `ollama_model` in `config.yaml` — a 16k-context build of qwen2.5-coder:7b. Its larger context window reduces the number of chunks per document, which is the main onboarding-speed lever: inference is GPU-bound (~65 tok/s), so the cost is chunk-count × generation, not hardware. Note: `GRAPHIFY_OLLAMA_NUM_CTX` does not propagate through graphify's OpenAI-compatible path; the larger context comes from the model's own Modelfile.
 
-Code-only repos use the free tree-sitter AST pass — no LLM cost. The `--backend ollama` flags are always specified so the single command handles mixed (code + docs) repos without requiring repo-type detection.
+Confirm `./graphify-out/graph.json` exists before reporting done.
 
 ## Incremental update
 
@@ -48,5 +80,5 @@ For traversal mechanics (`graphify path`, `graphify explain`, `--budget`, `--dfs
 ## Constraints
 
 - **Never commit `graphify-out/`** — it is local, disposable, and rebuildable.
-- Model is `qwen2.5-coder:7b` — do not substitute.
+- Model is `qwen25-coder-7b-16k` (config.yaml `ollama_model`) — do not substitute. This is a 16k-context build of qwen2.5-coder:7b; using the smaller base model silently reduces context and increases chunk count.
 - The `.gitignore` entry for `graphify-out/` must survive even if the directory is deleted.