210 lines
13 KiB
Markdown
210 lines
13 KiB
Markdown
---
|
|
summary: How to ingest non-code content (docs, PDFs, images, video, Google Workspace) into a Graphify knowledge graph, including format support, backends, chunking, and personal KB organization patterns.
|
|
tags:
|
|
- type/howto
|
|
- tool/graphify
|
|
- scope/global
|
|
- domain/knowledge-graphs
|
|
source: cc-os
|
|
date: 2026-06-08
|
|
---
|
|
# 04 — Ingesting Documents & a Personal Knowledge Base
|
|
|
|
How to turn notes, PDFs, spreadsheets, images, audio/video, and cloud docs into a queryable Graphify graph. Unlike code (free, local AST), documents need an LLM or local SLM to extract meaning — so this is the part of Graphify that costs tokens (or GPU time), and where organization and chunking matter most.
|
|
|
|
> **Provenance:** Claims are tagged `[github]` (verified against the v8 repo README/docs on 2026-06-03), `[interview]` (from the creator interview — treat as intent/sales claims), `[community](url)`, or `[unverified claim]`. Never assume a flag exists because it sounds plausible — every command below was copied from the verified README unless tagged otherwise.
|
|
|
|
Related docs: [code/AST ingest](03-ingesting-code-ast.md) · [local models & backends](05-local-models-and-backends.md) · [querying & god nodes](06-querying-and-god-nodes.md) · [token economics & updates](07-token-economics-and-updates.md) · [workflows](08-workflows-and-use-cases.md).
|
|
|
|
---
|
|
|
|
## The one rule that governs cost
|
|
|
|
Graphify runs a multi-pass pipeline. The split that matters for your wallet: **code is extracted locally with no API calls (AST via tree-sitter); everything else goes through your AI assistant's model API.** `[github]`
|
|
|
|
- **Code files** — processed locally via tree-sitter. Nothing leaves your machine, no tokens spent. `[github]` (see [03-ingesting-code-ast.md](03-ingesting-code-ast.md))
|
|
- **Video / audio** — transcribed locally with faster-whisper. Nothing leaves your machine. `[github]`
|
|
- **Docs, PDFs, images** — sent to your AI assistant for semantic extraction. `[github]`
|
|
|
|
In the creator's words: *"You have to call the LLM backend only for your documents… [for code] there's no API call included."* `[interview]` So the practical takeaway is: code is free to graph; **documents always need a backend** (a cloud LLM or a local SLM via Ollama). Pick the backend deliberately — that decision and its cost trade-offs live in [05-local-models-and-backends.md](05-local-models-and-backends.md) and [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
|
|
|
|
---
|
|
|
|
## Supported non-code formats
|
|
|
|
Verbatim from the v8 format table `[github]`:
|
|
|
|
| Type | Extensions | Notes |
|
|
|------|-----------|-------|
|
|
| Docs | `.md .mdx .qmd .html .txt .rst .yaml .yml` | Plain text / markup |
|
|
| Office | `.docx .xlsx` | requires `uv tool install "graphifyy[office]"` |
|
|
| Google Workspace | `.gdoc .gsheet .gslides` | opt-in; requires `gws` auth + `--google-workspace`; Sheets need `graphifyy[google]` |
|
|
| PDFs | `.pdf` | requires `uv tool install "graphifyy[pdf]"` |
|
|
| Images | `.png .jpg .webp .gif` | vision-based extraction (screenshots, diagrams) `[unverified claim]` on per-language fidelity |
|
|
| Video / Audio | `.mp4 .mov .mp3 .wav` and more | requires `uv tool install "graphifyy[video]"` (faster-whisper + yt-dlp) |
|
|
| YouTube / URLs | any video URL | requires `uv tool install "graphifyy[video]"` |
|
|
|
|
Optional extras are installed once and add format support; install only what you need:
|
|
|
|
```bash
|
|
uv tool install "graphifyy[pdf]" # PDF extraction
|
|
uv tool install "graphifyy[office]" # .docx and .xlsx
|
|
uv tool install "graphifyy[video]" # video/audio transcription (faster-whisper + yt-dlp)
|
|
uv tool install "graphifyy[google]" # Google Sheets table rendering
|
|
```
|
|
|
|
`uv tool install` installs a single tool, so each line above *replaces* the previous install. To get several extras at once, combine them in one bracket (comma-separated) or grab everything:
|
|
|
|
```bash
|
|
uv tool install "graphifyy[pdf,office,video,google]" # combine extras in one install
|
|
uv tool install "graphifyy[all]" # everything
|
|
```
|
|
|
|
> Note the PyPI package is `graphifyy` (double-y) while the command is `graphify`. `[github]` See [02-installation-setup.md](02-installation-setup.md).
|
|
|
|
---
|
|
|
|
## Two ways to run: in-IDE skill vs. headless CLI
|
|
|
|
This trips people up, so be explicit. Graphify has two invocation forms, and they are **not** interchangeable: `[github]`
|
|
|
|
- **In-IDE skill** — `/graphify ...` (slash command). Runs inside your Claude Code / Cursor / Gemini CLI session and uses *whatever model that session runs* for document extraction. This is the path the interview mostly describes.
|
|
- **Headless CLI** — `graphify extract ...`. Runs from a terminal and needs you to choose a backend explicitly (an API key, a running Ollama instance, or the `claude` CLI binary).
|
|
|
|
Examples:
|
|
|
|
```bash
|
|
# In-IDE skill (uses your session's model for docs)
|
|
/graphify ./docs
|
|
/graphify ./docs --update
|
|
|
|
# Headless CLI (you pick the backend)
|
|
graphify extract ./docs --backend ollama
|
|
graphify extract ./docs --backend gemini
|
|
```
|
|
|
|
For everything that follows, the slash form and the `graphify extract` form do the same ingestion — they differ only in *where the model comes from*.
|
|
|
|
---
|
|
|
|
## Audio & video transcription (Whisper)
|
|
|
|
The interview calls it "whisper"; the shipped implementation is **faster-whisper**, run **locally**. `[github]` Audio and video are transcribed on your machine before any text is sent anywhere, so the transcription step itself is private and free of API tokens (the resulting transcript text is then treated like a document and does hit your backend). `[github]`
|
|
|
|
One nice detail: the transcription prompt is seeded with your **top god nodes** (the most-connected concepts in your graph so far), which biases Whisper toward your domain vocabulary. `[github]`
|
|
|
|
```bash
|
|
uv tool install "graphifyy[video]" # one-time: installs faster-whisper + yt-dlp
|
|
/graphify ./media # transcribe local .mp4/.mov/.mp3/.wav, then graph
|
|
```
|
|
|
|
### YouTube / lecture transcripts
|
|
|
|
The interview's headline KB example — *"put that link, get the lecture's transcription, and then you can have the map of the lecture with each section divided into various graphs"* `[interview]` — is shipped. Add a single video by URL: `[github]`
|
|
|
|
```bash
|
|
/graphify add <youtube-url> # transcribe and add a video
|
|
/graphify add https://arxiv.org/abs/1706.03762 # also works for papers/tweets
|
|
```
|
|
|
|
This downloads via `yt-dlp`, transcribes locally with faster-whisper, and folds the transcript into the graph as document nodes. `[github]`
|
|
|
|
---
|
|
|
|
## Google Workspace connector
|
|
|
|
Pull native Google Docs, Sheets, and Slides into a graph. Important gotcha called out in the README: Google Drive for desktop `.gdoc`/`.gsheet`/`.gslides` files are **shortcut pointers, not content** — so you must authenticate the `gws` CLI to fetch the real documents. `[github]`
|
|
|
|
```bash
|
|
uv tool install "graphifyy[google]" # needed for Google Sheets table rendering
|
|
gws auth login -s drive # authenticate the gws CLI (opens a browser, approve scopes)
|
|
graphify extract ./docs --google-workspace
|
|
```
|
|
|
|
Equivalently, set `GRAPHIFY_GOOGLE_WORKSPACE=1` instead of passing the flag. `[github]` Under the hood, Graphify exports the shortcuts into `graphify-out/converted/` as Markdown sidecars, then extracts those. `[github]` The `gws` CLI itself is a separate Google tool ([googleworkspace/cli](https://github.com/googleworkspace/cli)). `[github]`
|
|
|
|
> Because these are documents, the same backend rule applies: a Workspace ingest spends tokens (or local SLM time). `[github]`
|
|
|
|
---
|
|
|
|
## Roadmap connectors — NOT yet shipped
|
|
|
|
The interview previews several connectors. As of the v8 repo (2026-06-03) **none of these are in Graphify** — treat them as roadmap until a release confirms otherwise: `[interview]`
|
|
|
|
- **Slack messages** — *"you can connect your Slack messages… as a map"* `[interview]`. Not in the docs. `[github]` (absent)
|
|
- **OneNote** ("one node" in the transcript, likely OneNote) — *"even your one node"* `[interview]`. Not in the docs. `[github]` (absent)
|
|
- **Meeting transcripts** — *"even your meetings, the meeting transcripts… everything will be in a brain"* `[interview]`. Not in Graphify. Note: the repo mentions **Penpax**, a *separate* always-on product built on top of Graphify that covers "meetings, browser history, emails, files, and code" `[github]` — so meeting capture appears to be a Penpax feature, not core Graphify.
|
|
|
|
If you need any of these today, the interview's own workaround applies: export to a supported document format (e.g. a Slack export or a `.txt`/`.md` transcript) and ingest it as a regular doc. `[interview]`
|
|
|
|
---
|
|
|
|
## Organizing a mixed personal knowledge base
|
|
|
|
A personal KB is usually a pile of notes + PDFs + media + cloud docs. A few practical principles:
|
|
|
|
**1. Group by graph, not by one giant folder.** The interview's strongest cost/quality advice: *"you can split your repository or your documents… and reingest them rather than putting the whole chunk of documents in one shot."* `[interview]` And: *"the more the context… there will be a massive chance of hallucination or dilution in the reasoning."* `[interview]` So build several focused graphs (e.g. `research/`, `meeting-notes/`, `personal-wiki/`) rather than one undifferentiated dump. You can later combine graphs: `graphify merge-graphs a.json b.json`. `[github]`
|
|
|
|
**2. Keep code and docs in separate runs where you can.** Code is free (AST); docs cost. Splitting them makes it obvious where your tokens go and lets you re-run docs without re-touching code.
|
|
|
|
**3. Build incrementally with `--update`.** Don't re-ingest from scratch each time. `[github]` `[interview]`
|
|
|
|
```bash
|
|
/graphify ./docs --update # re-extract only changed files, merge into existing graph
|
|
```
|
|
|
|
Graphify uses SHA-256 hashing + de-duplication so unchanged files are skipped and entities don't collide across documents. `[interview]` `[github]` Details and economics in [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
|
|
|
|
**4. Query god nodes first.** Once a KB is graphed, *"always go to the god nodes first and then start retrieving from there"* `[interview]`. A sane number of god nodes means cohesion is good; an explosion of them often signals a corpus that should have been split. See [06-querying-and-god-nodes.md](06-querying-and-god-nodes.md).
|
|
|
|
### Chunking & splitting large corpora
|
|
|
|
For large documents — and especially when using a **local SLM** with a small context window — tune the chunk size and concurrency rather than throwing the whole corpus at the model: `[github]`
|
|
|
|
```bash
|
|
graphify extract ./docs --token-budget 30000 # smaller semantic chunks for local/small models
|
|
graphify extract ./docs --max-concurrency 2 # fewer parallel LLM calls (useful for local inference)
|
|
GRAPHIFY_OLLAMA_NUM_CTX=8192 graphify extract ./docs --backend ollama --token-budget 4000 # tiny chunks for a constrained Ollama context
|
|
```
|
|
|
|
`--token-budget` controls semantic chunk size; smaller budgets keep each LLM call inside a modest context window. `--max-concurrency` throttles parallel calls so a local model isn't overwhelmed. `[github]` Pair these with the Ollama env knobs (`GRAPHIFY_OLLAMA_NUM_CTX`, `GRAPHIFY_OLLAMA_KEEP_ALIVE`) covered in [05-local-models-and-backends.md](05-local-models-and-backends.md).
|
|
|
|
---
|
|
|
|
## A runnable starter flow for a personal KB
|
|
|
|
```bash
|
|
# 0) one-time: add the format extras you need (combine in ONE bracket)
|
|
uv tool install "graphifyy[pdf,office,video]"
|
|
|
|
# 1) graph your markdown notes + PDFs (docs cost tokens — choose a backend)
|
|
# in-IDE (uses your session model):
|
|
/graphify ./knowledge/notes
|
|
# or headless, fully local:
|
|
graphify extract ./knowledge/notes --backend ollama --token-budget 8000
|
|
|
|
# 2) add a lecture you want to remember
|
|
/graphify add <youtube-url>
|
|
|
|
# 3) pull in cloud docs
|
|
gws auth login -s drive
|
|
graphify extract ./knowledge/gdrive --google-workspace
|
|
|
|
# 4) keep it fresh as you add files
|
|
/graphify ./knowledge/notes --update
|
|
|
|
# 5) explore
|
|
/graphify query "what connects my notes on retrieval to the Stanford lecture?"
|
|
```
|
|
|
|
---
|
|
|
|
## Open questions / unverified
|
|
|
|
- **`gws auth login -s drive` scope flag** — verified in the v8 README `[github]`, but the `gws` CLI is third-party; its exact scope syntax may change. Confirm against [googleworkspace/cli](https://github.com/googleworkspace/cli) before relying on it.
|
|
- **"71.5x token reduction" and similar multipliers** — sales claims from the README/interview, explicitly described by the creator as corpus-dependent with "no ceiling or floor." `[interview]` Do not treat as a guarantee; see [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
|
|
- **Exact full list of "and more" video/audio formats** — README lists `.mp4 .mov .mp3 .wav` plus unspecified others. `[github]` Test your specific format.
|
|
- **Whether `/graphify add` accepts non-YouTube video hosts** — README says "any video URL" via yt-dlp `[github]`; coverage depends on yt-dlp support for that host. `[unverified claim]`
|
|
- **Slack / OneNote / meeting-transcript connectors** — interview-stated roadmap, confirmed absent from the v8 repo. `[interview]` Meeting capture currently appears to belong to the separate **Penpax** product, not core Graphify. `[github]` Re-check release notes before assuming availability.
|
|
- **Image extraction quality for handwriting/dense scans** — README cites vision extraction for "screenshots, diagrams, any language" `[github]`; handwriting fidelity is untested here.
|