13 KiB

Raw Blame History

summary

Ingesting Code with the AST (Free, No LLM)

How Graphify turns a codebase into a graph using tree-sitter abstract syntax trees (ASTs) — a deterministic, local, zero-API-cost pass. This is the cheapest thing Graphify does, and the interview's core rule is to lean on it: use the AST for code, and save the LLM for documents.

Related: 01 Overview & Concepts · 02 Installation & Setup · 04 Ingesting Docs & Knowledge · 06 Querying & God Nodes · 07 Token Economics & Updates

The core rule: AST for code, LLM for documents

The creator states this plainly, more than once: parsing a codebase does not call an LLM, so don't pay for one.

"The main problem if you are running... an LLM to parse your codebase, that doesn't make sense at all... Abstract syntax trees. It's a free Python library which can connect various parts of a codebase like functions... and create relationships between them in different files even in multiple repository. So it's free of cost." [interview]

Verified against the repo: the first pass is a deterministic tree-sitter walk over every code file with no LLM, no embeddings, no network — AST nodes are converted directly into graph nodes and edges. [github]

What this means in practice:

Running graphify . on a pure code repo costs nothing — no API key needed, no token burn, runs entirely on your machine. [github]
You only spend tokens (cloud or local) on non-code content: docs, PDFs, transcripts, images. See 04 Ingesting Docs & Knowledge and 05 Local Models & Backends.
The interview anecdote of "it ran through my whole daily limit" comes from letting an LLM touch the code path. Index code first (free), then decide what documents are worth an LLM call. [interview]

How tree-sitter AST extraction works (high level)

tree-sitter is a parser library that turns source files into a concrete syntax tree. Graphify walks that tree to pull out the structural facts of your code.

"Tree-sitter parses your code files and extracts classes, functions, imports, call graphs, and inline comments. This runs locally with no LLM involved." [github]

The pipeline, as documented:

Walk every code file with tree-sitter (deterministic, local). [github]
Extract symbols — classes, functions, imports, inline comments — and turn each into a graph node with an identifier and label. [github]
Call-graph pass — a follow-up pass that links the nodes into edges: function calls, file imports, implementations, and other typed relationships across files. [github]
Tag every relationship with a confidence label so you can tell code-derived facts from inferred ones:
- EXTRACTED — found directly in the source (this is what the AST path produces). [github]
- INFERRED — model inference, with a confidence score (~0.55–0.95). [github]
- AMBIGUOUS — uncertain, flagged for manual review. [github]
For a pure AST run with no LLM, relationships are EXTRACTED. INFERRED/AMBIGUOUS show up when an LLM enriches the graph (deep mode / documents).

What gets linked into the graph

Source artifact	Becomes	Linked by
File	Node	`imports` edges to other files `[github]`
Class / function	Node	`calls` edges (call graph), `implements` edges `[github]`
Import statement	Edge	connects the importing file/symbol to its target `[github]`
Inline comments	Captured with the symbol	provide context for later querying `[github]`

The most-connected nodes surface as god nodes — the densest hubs in the graph, your fastest way to read an unfamiliar architecture. Query them first. Details in 06 Querying & God Nodes. [github] [interview]

Supported languages (33, via tree-sitter)

The current release (v8) lists 33 languages. Verified verbatim from the v8 code-extension table: [github]

.py .ts .js .jsx .tsx .mjs .go .rs .java .c .cpp .h .hpp .rb .cs .kt
.scala .php .swift .lua .luau .zig .ps1 .ex .exs .m .mm .jl .vue
.svelte .astro .groovy .gradle .dart .v .sv .svh .sql .f .f90 .f95
.f03 .f08 .pas .pp .dpr .dpk .lpr .inc .dfm .lfm .lpk .sh .bash .json
.dm .dme .dmi .dmm .dmf .sln .csproj .fsproj .vbproj .razor .cshtml

By language family, that covers (non-exhaustive, all [github]):

Mainstream: Python, TypeScript/JavaScript (+ JSX/TSX/MJS), Go, Rust, Java, C/C++, Ruby, C#, Kotlin, Scala, PHP, Swift, Dart.
Systems / scientific: Zig, Lua/Luau, Julia, Fortran (.f … .f08), MATLAB/Objective-C (.m/.mm), Verilog/SystemVerilog (.v/.sv/.svh).
Web frameworks: Vue, Svelte, Astro, Razor/cshtml.
JVM build / scripting: Groovy, Gradle.
Data / schema: SQL schemas, JSON.
Shell: .sh, .bash, and PowerShell .ps1 — see the note below.
Project files: .sln, .csproj, .fsproj, .vbproj.
Niche: Pascal/Delphi (.pas/.dpr/…), BYOND DreamMaker (.dm/.dme/…).

The repo's own tagline confirms the breadth: it turns "any folder of code, SQL schemas, R scripts, shell scripts, docs, papers, images, or videos into a queryable knowledge graph." [github]

Shell scripts: now supported by AST (interview was outdated)

This is a place where the GitHub repo supersedes the interview, so read carefully if you watched the video.

In the interview, the creator said shell scripts were not handled by the AST path and prescribed a workaround:

"Shell scripts are very flat... rather than going with an A[ST] which isn't capab[le] at the moment for shell scripts, what you can do is just put those as documents and then make the LLM do a semantic extraction of a shell script... That's a temporary fix. I'll get it sorted maybe in coming releases." [interview]

As of the current release (v8), that fix has shipped: .sh, .bash, and .ps1 are in the code-extension table and are parsed via tree-sitter like any other language — locally, no API call. [github] So the interview's workaround is no longer necessary for shell scripts; just run graphify . and they'll be extracted for free.

Fallback for any unsupported language

The document-ingestion trick the creator described is still useful — just reframe it for any file whose language isn't in the 33 (or any code so unusual the AST extraction is thin). Instead of forcing it through the AST path, ingest it as a document so an LLM does semantic extraction of its structure:

Point Graphify at it as document content (see 04 Ingesting Docs & Knowledge) and let a local Ollama model or a cloud model pull out nodes. [interview]
This costs tokens (it's an LLM call), so reserve it for the handful of files the AST genuinely can't read. For the 33 supported languages, always prefer the free AST path. [interview]

Indexing a repo: `graphify .`

Run it from the repo root. [github]

# index the current directory (free, AST-only for code)
graphify .

# or a specific folder
graphify ./services/api

This writes three artifacts you and your team can query: [github]

graph.html — interactive visualization.
graph.json — the queryable graph.
GRAPH_REPORT.md — human-readable audit (includes god nodes and confidence tags).

Install with uv tool install graphifyy (or pipx install graphifyy). Full setup, including how to wire it into Claude Code, is in 02 Installation & Setup. [github]

After indexing, query the graph — and ask for god nodes first before drilling in. That keeps token use minimal on the retrieval side. See 06 Querying & God Nodes. [interview]

Re-indexing after changes: `--update`

Always use --update when re-ingesting an already-indexed repo. It avoids rebuilding from scratch.

graphify . --update

"I've used hashing strategy [SHA]256 in here, so it will start from where it left... make sure you use the word update as well, graphify update, so that it doesn't start fresh from the beginning." [interview]

Verified mechanics: Graphify fingerprints each extracted file with SHA256 content hashing and caches under graphify-out/cache/. On a re-run with --update, unchanged files are skipped entirely; only new/modified files are re-extracted. There are also de-duplication techniques to minimize duplicate entities across files. [github] [interview]

More on incremental updates and the token math in 07 Token Economics & Updates.

Auto-rebuild on commit: `graphify hook install`

To keep the graph fresh without remembering to re-run it, install the git hooks:

graphify hook install

This sets up post-commit and post-checkout hooks that rebuild the graph automatically — and because it's the AST path for code, there's no API cost. [github] This directly addresses the interview's "my assets become stale over time" concern: the graph stays current as you commit. [interview]

Many repos / a whole enterprise in one graph

The AST path links symbols across files and across repositories, so an entire codebase — or a whole company's set of repos — can live in one graph. [interview]

"It can connect... functions and all of those sections of a code all together and create relationships between them in different files even in multiple repository." [interview]

The verified mechanism is the graphify global command family, which maintains a cross-project index at ~/.graphify/global.json: [github]

# 1) index each repo individually (free, AST)
cd /path/to/repo-a && graphify .
cd /path/to/repo-b && graphify .

# 2) register each repo's graph into the global cross-project graph
graphify global add graphify-out/graph.json repo-a
graphify global add graphify-out/graph.json repo-b

# 3) inspect / manage the global graph
graphify global list          # shows registered repos with node + edge counts
graphify global remove repo-a # unregister a repo

This is the "digital twin of your whole enterprise" use case from the interview: a new senior hire can read the architecture in one shot instead of spending days and Slack threads spelunking. [interview] See 08 Workflows & Use Cases.

A concrete first run

# from a repo you've never indexed
graphify .                       # free AST extraction of all 33 supported languages

# read the architecture: open graph.html, skim GRAPH_REPORT.md,
# then query for the god nodes first (see doc 06)
graphify query "what are the god nodes?"

# keep it fresh
graphify hook install            # auto-rebuild on every commit
# ...or manually after a batch of changes:
graphify . --update

Open questions / unverified

Exact per-language extraction depth. The 33-language list is verified, but how thoroughly each language's call graph is resolved (e.g., dynamic dispatch in Ruby/Python, macro-heavy Rust) is not documented. [unverified claim]
--update vs. a bare graphify global re-add. Whether updating a member repo automatically propagates into an already-registered global graph, or requires re-running graphify global add, isn't spelled out in the docs I checked. [unverified claim]
graphify global path semantics. A graphify global path command appears in the repo listing, but its exact arguments/behavior for cross-repo path queries weren't documented in the sources read. [github] (command exists) / [unverified claim] (usage)
Branch/version skew. The main README I first fetched showed only ~13 languages and an older pip install graphifyy && graphify install flow; the v8 branch shows the full 33 languages, shell support, and uv tool install. This doc follows v8 (newest). If you're on an older install, your supported-language list and command surface may be narrower — check graphify --help. [github]
R scripts. The repo tagline advertises "R scripts," but .r/.R does not appear in the v8 code-extension table I verified — R may be handled via the document path rather than AST. [unverified claim]

13 KiB Raw Blame History Unescape Escape