SecondBrain/2026-06-08-graphify-ingesting-code-ast.md

---
summary: How Graphify uses tree-sitter ASTs to extract code structure locally for free — 33 supported languages, call-graph pass, incremental updates, and multi-repo global graphs.
tags:
  - type/howto
  - tool/graphify
  - scope/global
  - domain/knowledge-graphs
source: cc-os
date: 2026-06-08
---
# Ingesting Code with the AST (Free, No LLM)

How Graphify turns a codebase into a graph using tree-sitter abstract syntax trees (ASTs) — a deterministic, local, zero-API-cost pass. This is the cheapest thing Graphify does, and the interview's core rule is to lean on it: **use the AST for code, and save the LLM for documents.**

> Related: [01 Overview & Concepts](./01-overview-concepts.md) · [02 Installation & Setup](./02-installation-setup.md) · [04 Ingesting Docs & Knowledge](./04-ingesting-docs-knowledge.md) · [06 Querying & God Nodes](./06-querying-and-god-nodes.md) · [07 Token Economics & Updates](./07-token-economics-and-updates.md)

---

## The core rule: AST for code, LLM for documents

The creator states this plainly, more than once: parsing a codebase does **not** call an LLM, so don't pay for one.

> "The main problem if you are running... an LLM to parse your codebase, that doesn't make sense at all... Abstract syntax trees. It's a free Python library which can connect various parts of a codebase like functions... and create relationships between them in different files even in multiple repository. So it's free of cost." `[interview]`

Verified against the repo: the first pass is a deterministic tree-sitter walk over every code file with **no LLM, no embeddings, no network** — AST nodes are converted directly into graph nodes and edges. `[github]`

What this means in practice:

- Running `graphify .` on a pure code repo costs **nothing** — no API key needed, no token burn, runs entirely on your machine. `[github]`
- You only spend tokens (cloud or local) on **non-code** content: docs, PDFs, transcripts, images. See [04 Ingesting Docs & Knowledge](./04-ingesting-docs-knowledge.md) and [05 Local Models & Backends](./05-local-models-and-backends.md).
- The interview anecdote of "it ran through my whole daily limit" comes from letting an LLM touch the code path. Index code first (free), then decide what documents are worth an LLM call. `[interview]`

---

## How tree-sitter AST extraction works (high level)

[tree-sitter](https://tree-sitter.github.io/tree-sitter/) is a parser library that turns source files into a concrete syntax tree. Graphify walks that tree to pull out the structural facts of your code.

> "Tree-sitter parses your code files and extracts classes, functions, imports, call graphs, and inline comments. This runs locally with no LLM involved." `[github]`

The pipeline, as documented:

1. **Walk every code file** with tree-sitter (deterministic, local). `[github]`
2. **Extract symbols** — classes, functions, imports, inline comments — and turn each into a graph **node** with an identifier and label. `[github]`
3. **Call-graph pass** — a follow-up pass that links the nodes into **edges**: function `calls`, file `imports`, implementations, and other typed relationships across files. `[github]`
4. **Tag every relationship** with a confidence label so you can tell code-derived facts from inferred ones:
   - `EXTRACTED` — found directly in the source (this is what the AST path produces). `[github]`
   - `INFERRED` — model inference, with a confidence score (~0.55–0.95). `[github]`
   - `AMBIGUOUS` — uncertain, flagged for manual review. `[github]`

   For a pure AST run with no LLM, relationships are `EXTRACTED`. `INFERRED`/`AMBIGUOUS` show up when an LLM enriches the graph (deep mode / documents).

### What gets linked into the graph

| Source artifact | Becomes | Linked by |
| --- | --- | --- |
| File | Node | `imports` edges to other files `[github]` |
| Class / function | Node | `calls` edges (call graph), `implements` edges `[github]` |
| Import statement | Edge | connects the importing file/symbol to its target `[github]` |
| Inline comments | Captured with the symbol | provide context for later querying `[github]` |

The most-connected nodes surface as **god nodes** — the densest hubs in the graph, your fastest way to read an unfamiliar architecture. Query them first. Details in [06 Querying & God Nodes](./06-querying-and-god-nodes.md). `[github]` `[interview]`

---

## Supported languages (33, via tree-sitter)

The current release (v8) lists **33 languages**. Verified verbatim from the v8 code-extension table: `[github]`

```
.py .ts .js .jsx .tsx .mjs .go .rs .java .c .cpp .h .hpp .rb .cs .kt
.scala .php .swift .lua .luau .zig .ps1 .ex .exs .m .mm .jl .vue
.svelte .astro .groovy .gradle .dart .v .sv .svh .sql .f .f90 .f95
.f03 .f08 .pas .pp .dpr .dpk .lpr .inc .dfm .lfm .lpk .sh .bash .json
.dm .dme .dmi .dmm .dmf .sln .csproj .fsproj .vbproj .razor .cshtml
```

By language family, that covers (non-exhaustive, all `[github]`):

- **Mainstream:** Python, TypeScript/JavaScript (+ JSX/TSX/MJS), Go, Rust, Java, C/C++, Ruby, C#, Kotlin, Scala, PHP, Swift, Dart.
- **Systems / scientific:** Zig, Lua/Luau, Julia, Fortran (`.f` … `.f08`), MATLAB/Objective-C (`.m`/`.mm`), Verilog/SystemVerilog (`.v`/`.sv`/`.svh`).
- **Web frameworks:** Vue, Svelte, Astro, Razor/cshtml.
- **JVM build / scripting:** Groovy, Gradle.
- **Data / schema:** **SQL** schemas, JSON.
- **Shell:** **`.sh`, `.bash`, and PowerShell `.ps1`** — see the note below.
- **Project files:** `.sln`, `.csproj`, `.fsproj`, `.vbproj`.
- **Niche:** Pascal/Delphi (`.pas`/`.dpr`/…), BYOND DreamMaker (`.dm`/`.dme`/…).

> The repo's own tagline confirms the breadth: it turns "any folder of code, SQL schemas, R scripts, shell scripts, docs, papers, images, or videos into a queryable knowledge graph." `[github]`

### Shell scripts: now supported by AST (interview was outdated)

This is a place where the **GitHub repo supersedes the interview**, so read carefully if you watched the video.

In the interview, the creator said shell scripts were *not* handled by the AST path and prescribed a workaround:

> "Shell scripts are very flat... rather than going with an A[ST] which isn't capab[le] at the moment for shell scripts, what you can do is just put those as documents and then make the LLM do a semantic extraction of a shell script... That's a temporary fix. I'll get it sorted maybe in coming releases." `[interview]`

As of the current release (v8), that fix has shipped: `.sh`, `.bash`, and `.ps1` are in the code-extension table and are parsed via tree-sitter like any other language — locally, no API call. `[github]` So the interview's workaround is **no longer necessary for shell scripts**; just run `graphify .` and they'll be extracted for free.

### Fallback for any unsupported language

The document-ingestion trick the creator described is still useful — just reframe it for **any file whose language isn't in the 33** (or any code so unusual the AST extraction is thin). Instead of forcing it through the AST path, ingest it as a **document** so an LLM does semantic extraction of its structure:

- Point Graphify at it as document content (see [04 Ingesting Docs & Knowledge](./04-ingesting-docs-knowledge.md)) and let a local Ollama model or a cloud model pull out nodes. `[interview]`
- This costs tokens (it's an LLM call), so reserve it for the handful of files the AST genuinely can't read. For the 33 supported languages, always prefer the free AST path. `[interview]`

---

## Indexing a repo: `graphify .`

Run it from the repo root. `[github]`

```bash
# index the current directory (free, AST-only for code)
graphify .

# or a specific folder
graphify ./services/api
```

This writes three artifacts you and your team can query: `[github]`

- `graph.html` — interactive visualization.
- `graph.json` — the queryable graph.
- `GRAPH_REPORT.md` — human-readable audit (includes god nodes and confidence tags).

> Install with `uv tool install graphifyy` (or `pipx install graphifyy`). Full setup, including how to wire it into Claude Code, is in [02 Installation & Setup](./02-installation-setup.md). `[github]`

After indexing, query the graph — and ask for **god nodes first** before drilling in. That keeps token use minimal on the retrieval side. See [06 Querying & God Nodes](./06-querying-and-god-nodes.md). `[interview]`

---

## Re-indexing after changes: `--update`

Always use `--update` when re-ingesting an already-indexed repo. It avoids rebuilding from scratch.

```bash
graphify . --update
```

> "I've used hashing strategy [SHA]256 in here, so it will start from where it left... make sure you use the word update as well, `graphify update`, so that it doesn't start fresh from the beginning." `[interview]`

Verified mechanics: Graphify fingerprints each extracted file with **SHA256 content hashing** and caches under `graphify-out/cache/`. On a re-run with `--update`, unchanged files are skipped entirely; only new/modified files are re-extracted. There are also de-duplication techniques to minimize duplicate entities across files. `[github]` `[interview]`

More on incremental updates and the token math in [07 Token Economics & Updates](./07-token-economics-and-updates.md).

---

## Auto-rebuild on commit: `graphify hook install`

To keep the graph fresh without remembering to re-run it, install the git hooks:

```bash
graphify hook install
```

This sets up **post-commit and post-checkout** hooks that rebuild the graph automatically — and because it's the AST path for code, there's no API cost. `[github]` This directly addresses the interview's "my assets become stale over time" concern: the graph stays current as you commit. `[interview]`

---

## Many repos / a whole enterprise in one graph

The AST path links symbols **across files and across repositories**, so an entire codebase — or a whole company's set of repos — can live in one graph. `[interview]`

> "It can connect... functions and all of those sections of a code all together and create relationships between them in different files even in multiple repository." `[interview]`

The verified mechanism is the **`graphify global`** command family, which maintains a cross-project index at `~/.graphify/global.json`: `[github]`

```bash
# 1) index each repo individually (free, AST)
cd /path/to/repo-a && graphify .
cd /path/to/repo-b && graphify .

# 2) register each repo's graph into the global cross-project graph
graphify global add graphify-out/graph.json repo-a
graphify global add graphify-out/graph.json repo-b

# 3) inspect / manage the global graph
graphify global list          # shows registered repos with node + edge counts
graphify global remove repo-a # unregister a repo
```

This is the "digital twin of your whole enterprise" use case from the interview: a new senior hire can read the architecture in one shot instead of spending days and Slack threads spelunking. `[interview]` See [08 Workflows & Use Cases](./08-workflows-and-use-cases.md).

---

## A concrete first run

```bash
# from a repo you've never indexed
graphify .                       # free AST extraction of all 33 supported languages

# read the architecture: open graph.html, skim GRAPH_REPORT.md,
# then query for the god nodes first (see doc 06)
graphify query "what are the god nodes?"

# keep it fresh
graphify hook install            # auto-rebuild on every commit
# ...or manually after a batch of changes:
graphify . --update
```

---

## Open questions / unverified

- **Exact per-language extraction depth.** The 33-language list is verified, but how thoroughly each language's call graph is resolved (e.g., dynamic dispatch in Ruby/Python, macro-heavy Rust) is not documented. `[unverified claim]`
- **`--update` vs. a bare `graphify global` re-add.** Whether updating a member repo automatically propagates into an already-registered global graph, or requires re-running `graphify global add`, isn't spelled out in the docs I checked. `[unverified claim]`
- **`graphify global path` semantics.** A `graphify global path` command appears in the repo listing, but its exact arguments/behavior for cross-repo path queries weren't documented in the sources read. `[github]` (command exists) / `[unverified claim]` (usage)
- **Branch/version skew.** The `main` README I first fetched showed only ~13 languages and an older `pip install graphifyy && graphify install` flow; the **v8** branch shows the full 33 languages, shell support, and `uv tool install`. This doc follows **v8** (newest). If you're on an older install, your supported-language list and command surface may be narrower — check `graphify --help`. `[github]`
- **R scripts.** The repo tagline advertises "R scripts," but `.r`/`.R` does not appear in the v8 code-extension table I verified — R may be handled via the document path rather than AST. `[unverified claim]`