cc-os/docs/graphify/02-installation-setup.md

225 lines
15 KiB
Markdown
Raw Permalink Normal View History

Add Graphify setup & best-practices guide (11 docs + research) Research: dispatched 11 subagents to synthesize a creator interview into 11 standalone documents (overview, install, code/doc ingestion, backends, querying, token economics, workflows, best practices, and community tips), then reconciled cross-document contradictions against the GitHub repo (v0.8.30 release). Verified all substantive claims against primary sources (GitHub, interview, community, PyPI). Corrected interview errors: package is graphifyy (double-y), shell scripts now AST-supported, Slack/meetings/OneNote connectors are roadmap not shipped. Settled flag version disputes (--token-budget vs. --budget) by grepping raw README bytes. Applied honest token-savings framing (1–49x measured, not 70–90x marketing). Every claim tagged inline by source confidence ([github] / [interview] / [community] / [unverified]). Zero broken links, zero residual contradictions. Deliverable: docs/graphify/ - 00-README.md: index, reading order, provenance - 01-overview-concepts.md: god nodes, neuro-symbolic, vs. Obsidian - 02-installation-setup.md: install, register, first run - 03-ingesting-code-ast.md: tree-sitter, 33 languages, multi-repo - 04-ingesting-docs-knowledge.md: PDF/media/YouTube/Google Workspace - 05-local-models-and-backends.md: Ollama vs. cloud, privacy - 06-querying-and-god-nodes.md: god-nodes-first discipline - 07-token-economics-and-updates.md: savings honestly, --update, SHA hashing - 08-workflows-and-use-cases.md: onboarding, bug-trace, audits, second brain - 09-best-practices-checklist.md: do/don't reference + quick commands - external-tips.md: community tips, gotchas, savings debate Also included: graphify-interview (raw creator interview, source material), memory-systems-compared060326 (research reference).
2026-06-03 20:45:07 +00:00
# Installation & First Run
How to install Graphify, register it with Claude Code (and other assistants), run your first graph, and confirm it worked — without accidentally burning through your token/usage limit on the first try.
> Provenance tags used throughout: `[github]` = official GitHub README (anchored to the **v0.8.30** release README, the comprehensive current version — fetched 2026-06-03; the shorter `main`-branch README omits some flags), `[pypi]` = PyPI page, `[site]` = graphifylabs.ai, `[interview]` = creator interview (sales/marketing context, treat as claims), `[community]` = third-party source, `[unverified claim]` = stated somewhere but not confirmed against a primary source.
---
## 1. Prerequisites
| Requirement | Notes |
|---|---|
| **Python 3.10+** | Required. Check with `python --version`. `[github][pypi]` |
| **An AI coding assistant** | Claude Code is the primary target; Codex, OpenCode, Cursor, and Gemini CLI are also supported. `[github]` |
| **`uv` or `pipx`** (recommended) | Either gives you isolated tool installs and puts `graphify` on your PATH automatically. `[github]` |
| **Ollama** (optional) | Only needed if you want a local LLM backend for *documents*. See [05-local-models-and-backends.md](05-local-models-and-backends.md). `[github]` (verified facts) |
| **`[google]` extra + Google auth** (optional) | Only for the Google Workspace connector. `[github]` (verified facts) |
| **AWS Bedrock access** (optional) | Only if you want a cloud LLM backend via `--backend bedrock`. `[github]` (verified facts) |
> **Code ingestion is done locally and needs no LLM.** Code files are parsed with a tree-sitter AST (abstract syntax tree) pass — no file contents leave your machine, no API calls. `[github]` Semantic extraction of *docs, papers, and images* is what calls a model (your assistant's provider, or a local backend). `[github]` Video/audio is transcribed locally with Whisper. `[github]` This split is the single most important thing to understand before your first run (see §6).
---
## 2. Install
The PyPI package is named **`graphifyy`** (double `y`) — the plain `graphify` name on PyPI is an unrelated package. The CLI command you type is still **`graphify`**. `[github]`
Install and register in one line. The README leads with `uv`:
```bash
# Recommended — works on Mac and Linux with no PATH setup needed
uv tool install graphifyy && graphify install
# or with pipx
pipx install graphifyy && graphify install
# or plain pip
pip install graphifyy && graphify install
```
`[github]`
**Why `uv` (or `pipx`) is recommended:** both put the CLI in a managed location that's automatically on your PATH, so you avoid the `graphify: command not found` problem. With plain `pip` you may have to add `~/.local/bin` (Linux) or `~/Library/Python/3.x/bin` (Mac) to your PATH yourself, or run `python -m graphify` instead. `[github]`
### Optional extras
Install feature groups with bracket syntax. Confirmed in the README's file-type table:
```bash
pip install "graphifyy[office]" # .docx / .xlsx ingestion
pip install "graphifyy[video]" # video/audio transcription (faster-whisper + yt-dlp)
pip install "graphifyy[mcp]" # MCP stdio server (graphify.serve)
```
`[github]` PDFs and images are supported out of the box (no extra needed). `[github]`
The Google Workspace connector uses a `[google]` extra (`uv tool install "graphifyy[google]"`, then `gws auth login`, run with `--google-workspace`), verified in the v0.8.30 README and detailed in [04-ingesting-docs-knowledge.md](04-ingesting-docs-knowledge.md). `[github]` Other extra names seen on the PyPI listing (`pdf`, `neo4j`, `anthropic`, `all`, …) should be confirmed against the PyPI page or `graphify --help` before you rely on them. `[pypi]`/`[unverified claim]`
---
## 3. Register with your assistant
`graphify install` (already chained into the install command above) drops the platform-specific skill manifest into your assistant's config directory so you can invoke `/graphify` from inside the assistant. For Claude Code on Mac/Linux this is all you need. `[github]`
Target a specific assistant with `--platform` (or a per-platform subcommand). From the README's platform table `[github]`:
| Platform | Install command |
|---|---|
| Claude Code (Linux/Mac) | `graphify install` |
| Claude Code (Windows) | `graphify install` (auto-detected) or `graphify install --platform windows` |
| Codex | `graphify install --platform codex` |
| OpenCode | `graphify install --platform opencode` |
| Gemini CLI | `graphify install --platform gemini` |
| GitHub Copilot CLI | `graphify install --platform copilot` |
| Cursor | `graphify cursor install` |
| Aider | `graphify install --platform aider` |
(Full list in the README also covers VS Code Copilot Chat, Trae, Kiro, Hermes, Factory Droid, OpenClaw, and Google Antigravity.) `[github]`
> Codex calls skills with `$` instead of `/`, so type `$graphify .` there. Codex also needs `multi_agent = true` under `[features]` in `~/.codex/config.toml` for parallel extraction. `[github]`
### Make the assistant *always* use the graph (recommended)
Registering the skill lets you call `/graphify` on demand. To make your assistant consult the graph automatically before grepping files, run the always-on installer **after** you've built a graph in a project:
```bash
graphify claude install # Claude Code: writes a CLAUDE.md section + a PreToolUse hook
```
For Claude Code this adds a `CLAUDE.md` note telling Claude to read `graphify-out/GRAPH_REPORT.md` before architecture questions, plus a PreToolUse hook (in `settings.json`) that fires before Glob/Grep and reminds Claude to navigate via the graph. Equivalent per-platform commands exist (`graphify codex install`, `graphify gemini install`, `graphify cursor install`, etc.); uninstall with the matching `... uninstall`. `[github]`
### Manual install (no package manager)
The README documents a curl-based fallback that copies the skill into `~/.claude/skills/graphify/SKILL.md` and adds a trigger line to `~/.claude/CLAUDE.md`. Use this only if you can't use `uv`/`pipx`. `[github]`
---
## 4. First run
From inside your assistant (Claude Code, etc.):
```
/graphify .
```
Or from a plain shell (on **Windows/PowerShell**, drop the slash — `/` is read as a path separator):
```bash
graphify .
```
This reads the files in the target folder, builds the knowledge graph, and writes everything into a `graphify-out/` directory. `[github]` You can point it at any folder — a codebase, a notes directory, a folder of papers. `[github]`
> **Three invocation contexts — know which one you're in** `[github]`:
> - **Inside your AI assistant:** `/graphify .` — semantic extraction uses your IDE session's model (no API key needed).
> - **Windows / plain shell:** `graphify .` (no leading slash) — same skill.
> - **Headless / CI:** `graphify extract ./path` (build) and `graphify update ./path` (incremental) — these run standalone and need a backend: an API key (`ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, …) **or** a local `--backend ollama`. Code (AST) is always free; only documents need the backend. See [05-local-models-and-backends.md](05-local-models-and-backends.md).
Useful first-run variants `[github]`:
```bash
graphify ./docs # run on a specific folder
graphify . --update # re-extract only changed files, merge into existing graph
graphify . --no-viz # skip the HTML, just produce the report + JSON
graphify . --mode deep # more aggressive INFERRED edge extraction
```
> A `--force` flag (or `GRAPHIFY_FORCE=1`) forces a full rebuild and overwrites the graph even when it ends up with fewer nodes — use it after a refactor that deleted files, to clear lingering "ghost" nodes. Verified in the v0.8.30 README. `[github]` See [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
> Add a `.graphifyignore` file (same syntax as `.gitignore`) at your repo root to exclude folders like `node_modules/`, `dist/`, or generated files from the graph. `[github]`
---
## 5. What you get and where it lands
Everything is written under **`graphify-out/`** in the directory you ran against. From the README's literal output tree `[github]`:
```
graphify-out/
├── graph.html interactive graph — open in any browser, click nodes, search, filter by community
├── GRAPH_REPORT.md god nodes, surprising connections, suggested questions
├── graph.json persistent graph — query weeks later without re-reading files
└── cache/ SHA256 cache — re-runs only process changed files
```
The three headline outputs map to the verified "interactive HTML graph, markdown report, JSON graph file": **`graph.html`**, **`GRAPH_REPORT.md`**, **`graph.json`**. `[github]`
Optional extra outputs are opt-in via flags, not produced by default: `--obsidian` writes an Obsidian vault, `--wiki` writes `index.md` + per-community articles, `--svg` writes `graph.svg`, `--graphml` writes `graph.graphml`, `--neo4j` writes `cypher.txt`. Video/audio runs also create `graphify-out/transcripts/`. `[github]`
### Confirming it worked
1. **Check the directory:** `graphify-out/` exists and contains `graph.html`, `GRAPH_REPORT.md`, and `graph.json`. `[github]`
2. **Open the report:** `GRAPH_REPORT.md` lists the **god nodes** (highest-degree concepts), surprising connections, and 45 suggested questions. If you see god nodes, extraction succeeded. `[github]` See [06-querying-and-god-nodes.md](06-querying-and-god-nodes.md).
3. **Watch the token benchmark:** Graphify prints a token benchmark automatically after every run, showing tokens-per-query vs. reading the raw files. `[github]`
4. **Open the graph:** open `graph.html` in a browser to click through nodes and communities. `[github]`
5. **Sanity-check the CLI:** `graphify --help` confirms the binary is on your PATH and lists available commands/flags. `[unverified claim]`
> On a tiny corpus (a handful of files) the token reduction may be ~1x — that's expected. At that size the value is structural clarity, not compression; reductions scale with corpus size. `[github]` More on this in [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
---
## 6. Avoid blowing your token limit on the first run
This is the #1 new-user mistake. In the interview, a user described running Graphify cold and burning through their entire daily limit plus ~$2530 extra on top of a $100/mo Claude plan. `[interview]`
The cause: pointing Graphify at a big pile of *documents* and letting it call a **cloud model** (your assistant's provider) to extract every chunk. The fix is to understand what actually costs tokens:
- **Code → local AST, free.** Code parsing uses tree-sitter AST locally with **no model calls** and no file contents leaving your machine. Run it on codebases freely. `[github][interview]`
- **Docs / papers / images → model, costs tokens.** Only these go to a model for semantic extraction. `[github]` To make that free too, point the document backend at **Ollama** (a local small language model). `[interview]` (verified facts: `--backend ollama`) See [05-local-models-and-backends.md](05-local-models-and-backends.md).
- **The first build costs tokens; queries are where you save.** The initial extraction pays the token cost; every later query reads the compact graph instead of raw files, and the SHA256 cache means re-runs only re-process changed files. `[github]`
- **Use `--update`, not full rebuilds.** When adding files later, re-run with `--update` so the cache reuses prior work instead of starting fresh. `[interview][github]` See [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
- **Split large doc sets** and re-ingest in pieces rather than dumping one giant chunk in a single run. `[interview]`
**Practical first run:** start on a code folder (free), confirm the graph looks right, then add documents with a local Ollama backend before scaling up.
---
## 7. Platform notes
**macOS / Linux:** `uv tool install graphifyy` or `pipx install graphifyy` both put `graphify` on your PATH with no setup. If you used plain `pip` and get `graphify: command not found`, add `~/.local/bin` (Linux) or `~/Library/Python/3.x/bin` (Mac) to your PATH, or run `python -m graphify`. `[github]`
**Windows:** Plain-`pip` scripts land in `%APPDATA%\Python\PythonXY\Scripts` — add that to your PATH, or just use `uv`/`pipx`. For Claude Code on Windows, `graphify install` auto-detects the platform (or use `--platform windows`). `[github]`
**WSL / Ubuntu:** Ubuntu ships `python3`, not `python`. For the MCP server path, install into a project venv to avoid PEP 668 conflicts: `python3 -m venv .venv && .venv/bin/pip install "graphifyy[mcp]"`. `[github]`
---
## 8. Copy-pasteable quickstart
```bash
# 1. Prereqs: Python 3.10+ and uv (or pipx). Check Python:
python --version
# 2. Install Graphify + register the skill (package is "graphifyy" with double y;
# command stays "graphify"). Recommended — no PATH setup on Mac/Linux:
uv tool install graphifyy && graphify install
# or: pipx install graphifyy && graphify install
# other assistants: graphify install --platform codex (Cursor: graphify cursor install)
# 3. (Optional) make the assistant always consult the graph before grepping:
graphify claude install
# 4. First run — start with a CODE folder (AST parsing is free, no LLM):
graphify .
# 5. Confirm it worked:
ls graphify-out/ # expect graph.html, GRAPH_REPORT.md, graph.json
# open graphify-out/GRAPH_REPORT.md -> check the god nodes
# open graphify-out/graph.html -> explore the graph
# 6. For DOCUMENTS, use a local model so you don't burn tokens — see doc 05.
# Then add incrementally:
graphify ./docs --update
```
Next: [05-local-models-and-backends.md](05-local-models-and-backends.md) (keep document ingestion free with Ollama) and [07-token-economics-and-updates.md](07-token-economics-and-updates.md) (incremental updates and where the savings come from). Concept refresher: [01-overview-concepts.md](01-overview-concepts.md).
---
## Open questions / unverified
- **README version skew (resolved).** The repo ships two READMEs: a short `main`-branch one (leads with `pip install`, omits several flags) and the comprehensive **v0.8.30 release** README (leads with `uv tool install graphifyy && graphify install`, documents the full flag set). This doc — and the rest of this guide — anchors on **v0.8.30**, the current release. If a doc page elsewhere disagrees, prefer the v0.8.30 release README, and confirm with `graphify --help` on your installed version.
- **`[google]` Workspace connector + `--backend ollama`/`bedrock` env vars** (`OLLAMA_BASE_URL`, `gws auth login`, `--google-workspace`, etc.) are all in the v0.8.30 README. See [04-ingesting-docs-knowledge.md](04-ingesting-docs-knowledge.md) and [05-local-models-and-backends.md](05-local-models-and-backends.md) for exact syntax.
- **Some extra names** (`pdf`, `neo4j`, `anthropic`, `all`) come from the PyPI listing; `[office]`, `[video]`, `[mcp]` are README-confirmed. Confirm others before relying on them.
- **`graphify --help`** as the post-install sanity check is inferred convention, not explicitly documented. `[unverified claim]`
- **Output directory** is `graphify-out/` relative to the run target per the README's literal tree; whether it can be relocated is not documented here.