Build a codebase map with Prism that goes in an agent's prompt #931

New issue

Open

opened 2026-10-04 16:52:44 +00:00 by jared · 0 comments

jared commented

2026-10-04 16:52:44 +00:00

Owner

Problem

A senior engineer does not query the codebase to learn its shape; the shape is already in their head. So the map must be in the prompt, and the query tools are the second step. (Jared, 2026-10-04: "Should be the ethos for this tool.")

The agents in the ticket-takedown harness (researcher, designers, critic, reviser) see only the small part of the codebase that the first research step reported. They design the smallest patch at that spot. In round A4 (2026-10-04, 5 ruled cases, 10 complete runs) no run produced the approved approach.

Query tools do not correct this. An agent does not ask about code that it does not know exists.

The purpose of the tool: give each agent that reads code a view of the whole codebase as a long-time senior engineer holds it. That view gives more entry points, shows behavior that already exists (DRY), and shows the conventions. Each thing that goes into the map must serve that purpose.

Observed (optional)

In the round 4 smoke run of #906 the critic had graphify query, graphify explain, and graphify path. It finished in 4 turns and 11 seconds.

bin/takedown-run 906

**Missing approach:** Searched item.rb, ListedPaths, and the refactorer agent file for a better owner or approach. The tag parser is the single owner, and a case-insensitive regex is the minimal fix. I found nothing better. (gain none, rating nil)

Measurements on a scratch copy, 2026-10-04:

graphify update . on the whole repository: 11 s, 14,161 nodes, report 300 KB. A query about the #907 defect returned mostly docs/crc/ nodes and did not return Repo#stage.
graphify update lib in plugins/os-sdlc: 2.6 s, 3,645 nodes. The same query returned Commit, commit!, ListedPaths, and Repo#stage. The report names its communities "Community 0" to "Community 359".
A 25-line Prism visitor (each file, its modules and classes with superclass, its public methods with parameters) over the 458 files of plugins/os-sdlc/lib: under 1 s, 3,499 lines, about 25,000 tokens.

Reproduce (optional)

n/a: this is a new tool, not a defect.

Approach

Jared chose the custom Prism route on 2026-10-04. The design is open; he wants to define how to build the map for this purpose before it is built. This conversation gives the entry point and the requirements.

Requirements:

The map goes in the prompt. The query tools (graphify, ast-grep, lsp-probe) are the second step, not a replacement.
No model builds the map. The build is deterministic and fast.
Prism builds it. Prism is part of Ruby, so there is no other dependency.
The scope is the code directories of the target only (lib/ and bin/; eval/ is excluded for now). No documents.
The map does not go to the private-method level. The point of view is adjustable, so that the map shows what is useful.
The map is small enough for several prompts of one run. 25,000 tokens in each of 5 steps is too large. Two candidates: a shorter form (one line for each constant, method names with no parameters; estimate 10,000 to 15,000 tokens, not measured), or two levels (directories and constants in the prompt, the methods of a directory on request).
The tool is small and reusable. It works on any Ruby codebase, so that os-sdlc agents in general can use it. The first consumer is ticket-takedown only. Later it probably becomes a Ruby gem.

Starting point: the Prism visitor from the session. Example output for adapter/git/repo.rb: class Repo with stage(paths), changed_paths(glob: nil, base: nil), staged_paths, commit(message, paths).

Open design questions (what the outline alone cannot show):

Relationships between constants: which constant refers to which. Prism gives constant references by name. ruby-lsp gives resolved references, which also permits a rank by use.
Conventions that are DSL calls (Thor method_option, Sequel associations), superclass lines, and include lines.
Recorded decisions linked to constants. A link by path misses: ADR-0171 lists runner/implementation/commit.rb in affected-paths, and the file is now command/implementation/commit.rb. Candidates: a link by affected-components, or by the constant names in the ADR text.
The tests that pin each constant (mirrored test paths).

Measures when it is built:

Do the designs and the critic use code that the map showed? Compare the grades on the 5 ruled cases with round A4 (860: 1,1; 890: 0,0; 893: 0,1; 906: 1,1; 907: 2,1).
Turns and tokens for each step with and without the map. Jared's hypothesis: less exploration, or the same usage with better answers that keep the code DRY.

Required work: none beyond the acceptance criteria.

Rejected: graphify on the whole repository — the queries return document noise, and the report is too large for a prompt.
Rejected: query tools only — the agent must know what to ask; the #906 smoke run shows that it does not ask.

Non-goals (optional)

Runtime facts (for example git index behavior, or a Zeitwerk load error inside a test). No static tool shows them.
Off-the-shelf tools with a similar aim were named and not tested: the aider repo map, repomix --compress, universal-ctags.

Origin

Trigger: round A4 eval of the ticket-takedown harness and the #906 smoke run of the round 4 harness (results/runs/906-20261004-122012), then Jared's direct request for this ticket on 2026-10-04.
Improvised this session: none
Chain: no run produces the approved approach ← each agent sees only the defect site that the research reported ← the harness gives the agents no view of the whole codebase ← DESIGN (docs/handoffs/2026-10-04-takedown-round-4.md, sections "Round A4 results" and "Smoke run")
Root candidate: this ticket
Where: plugins/os-sdlc/eval/ticket-takedown/ (first consumer); the tool itself has no home yet
Session: 47ae25fb-bfb4-4f60-a6fb-77a098c53319
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/47ae25fb-bfb4-4f60-a6fb-77a098c53319.jsonl

## Problem A senior engineer does not query the codebase to learn its shape; the shape is already in their head. So the map must be in the prompt, and the query tools are the second step. (Jared, 2026-10-04: "Should be the ethos for this tool.") The agents in the ticket-takedown harness (researcher, designers, critic, reviser) see only the small part of the codebase that the first research step reported. They design the smallest patch at that spot. In round A4 (2026-10-04, 5 ruled cases, 10 complete runs) no run produced the approved approach. Query tools do not correct this. An agent does not ask about code that it does not know exists. The purpose of the tool: give each agent that reads code a view of the whole codebase as a long-time senior engineer holds it. That view gives more entry points, shows behavior that already exists (DRY), and shows the conventions. Each thing that goes into the map must serve that purpose. ## Observed (optional) In the round 4 smoke run of #906 the critic had `graphify query`, `graphify explain`, and `graphify path`. It finished in 4 turns and 11 seconds. ``` bin/takedown-run 906 ``` ``` **Missing approach:** Searched item.rb, ListedPaths, and the refactorer agent file for a better owner or approach. The tag parser is the single owner, and a case-insensitive regex is the minimal fix. I found nothing better. (gain none, rating nil) ``` Measurements on a scratch copy, 2026-10-04: - `graphify update .` on the whole repository: 11 s, 14,161 nodes, report 300 KB. A query about the #907 defect returned mostly `docs/crc/` nodes and did not return `Repo#stage`. - `graphify update lib` in `plugins/os-sdlc`: 2.6 s, 3,645 nodes. The same query returned `Commit`, `commit!`, `ListedPaths`, and `Repo#stage`. The report names its communities "Community 0" to "Community 359". - A 25-line Prism visitor (each file, its modules and classes with superclass, its public methods with parameters) over the 458 files of `plugins/os-sdlc/lib`: under 1 s, 3,499 lines, about 25,000 tokens. ## Reproduce (optional) n/a: this is a new tool, not a defect. ## Approach Jared chose the custom Prism route on 2026-10-04. The design is open; he wants to define how to build the map for this purpose before it is built. This conversation gives the entry point and the requirements. Requirements: - The map goes in the prompt. The query tools (graphify, ast-grep, `lsp-probe`) are the second step, not a replacement. - No model builds the map. The build is deterministic and fast. - Prism builds it. Prism is part of Ruby, so there is no other dependency. - The scope is the code directories of the target only (`lib/` and `bin/`; `eval/` is excluded for now). No documents. - The map does not go to the private-method level. The point of view is adjustable, so that the map shows what is useful. - The map is small enough for several prompts of one run. 25,000 tokens in each of 5 steps is too large. Two candidates: a shorter form (one line for each constant, method names with no parameters; estimate 10,000 to 15,000 tokens, not measured), or two levels (directories and constants in the prompt, the methods of a directory on request). - The tool is small and reusable. It works on any Ruby codebase, so that os-sdlc agents in general can use it. The first consumer is ticket-takedown only. Later it probably becomes a Ruby gem. Starting point: the Prism visitor from the session. Example output for `adapter/git/repo.rb`: `class Repo` with `stage(paths)`, `changed_paths(glob: nil, base: nil)`, `staged_paths`, `commit(message, paths)`. Open design questions (what the outline alone cannot show): - Relationships between constants: which constant refers to which. Prism gives constant references by name. ruby-lsp gives resolved references, which also permits a rank by use. - Conventions that are DSL calls (Thor `method_option`, Sequel associations), superclass lines, and `include` lines. - Recorded decisions linked to constants. A link by path misses: ADR-0171 lists `runner/implementation/commit.rb` in `affected-paths`, and the file is now `command/implementation/commit.rb`. Candidates: a link by `affected-components`, or by the constant names in the ADR text. - The tests that pin each constant (mirrored test paths). Measures when it is built: - Do the designs and the critic use code that the map showed? Compare the grades on the 5 ruled cases with round A4 (860: 1,1; 890: 0,0; 893: 0,1; 906: 1,1; 907: 2,1). - Turns and tokens for each step with and without the map. Jared's hypothesis: less exploration, or the same usage with better answers that keep the code DRY. Required work: none beyond the acceptance criteria. Rejected: graphify on the whole repository — the queries return document noise, and the report is too large for a prompt. Rejected: query tools only — the agent must know what to ask; the #906 smoke run shows that it does not ask. ## Non-goals (optional) Runtime facts (for example git index behavior, or a Zeitwerk load error inside a test). No static tool shows them. Off-the-shelf tools with a similar aim were named and not tested: the aider repo map, repomix `--compress`, universal-ctags. ## Origin - Trigger: round A4 eval of the ticket-takedown harness and the #906 smoke run of the round 4 harness (`results/runs/906-20261004-122012`), then Jared's direct request for this ticket on 2026-10-04. - Improvised this session: none - Chain: no run produces the approved approach ← each agent sees only the defect site that the research reported ← the harness gives the agents no view of the whole codebase ← DESIGN (`docs/handoffs/2026-10-04-takedown-round-4.md`, sections "Round A4 results" and "Smoke run") - Root candidate: this ticket - Where: `plugins/os-sdlc/eval/ticket-takedown/` (first consumer); the tool itself has no home yet - Session: 47ae25fb-bfb4-4f60-a6fb-77a098c53319 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/47ae25fb-bfb4-4f60-a6fb-77a098c53319.jsonl

jared added the

plugin/os-sdlc

filed-by/user

labels

2026-10-04 16:52:44 +00:00

jared added the

category/owner-ruling

label

2026-10-10 13:31:48 +00:00