Add vault content: notes, journal, templates, conventions
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
b40f907f5d
commit
9bf99676b8
|
|
@ -0,0 +1 @@
|
|||
{}
|
||||
|
|
@ -0,0 +1 @@
|
|||
{}
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
{
|
||||
"file-explorer": true,
|
||||
"global-search": true,
|
||||
"switcher": true,
|
||||
"graph": true,
|
||||
"backlink": true,
|
||||
"canvas": true,
|
||||
"outgoing-link": true,
|
||||
"tag-pane": true,
|
||||
"footnotes": false,
|
||||
"properties": true,
|
||||
"page-preview": true,
|
||||
"daily-notes": true,
|
||||
"templates": true,
|
||||
"note-composer": true,
|
||||
"command-palette": true,
|
||||
"slash-command": false,
|
||||
"editor-status": true,
|
||||
"bookmarks": true,
|
||||
"markdown-importer": false,
|
||||
"zk-prefixer": false,
|
||||
"random-note": false,
|
||||
"outline": true,
|
||||
"word-count": true,
|
||||
"slides": false,
|
||||
"audio-recorder": false,
|
||||
"workspaces": false,
|
||||
"file-recovery": true,
|
||||
"publish": false,
|
||||
"sync": true,
|
||||
"bases": true,
|
||||
"webviewer": false
|
||||
}
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
{
|
||||
"collapse-filter": false,
|
||||
"search": "",
|
||||
"showTags": false,
|
||||
"showAttachments": false,
|
||||
"hideUnresolved": false,
|
||||
"showOrphans": true,
|
||||
"collapse-color-groups": true,
|
||||
"colorGroups": [],
|
||||
"collapse-display": true,
|
||||
"showArrow": false,
|
||||
"textFadeMultiplier": 0,
|
||||
"nodeSizeMultiplier": 1,
|
||||
"lineSizeMultiplier": 1,
|
||||
"collapse-forces": true,
|
||||
"centerStrength": 0.518713248970312,
|
||||
"repelStrength": 10,
|
||||
"linkStrength": 1,
|
||||
"linkDistance": 250,
|
||||
"scale": 1.0519895055086428,
|
||||
"close": true
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
{"folder": "_templates"}
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
This is your new *vault*.
|
||||
|
||||
Make a note of something, [[create a link]], or try [the Importer](https://help.obsidian.md/Plugins/Importer)!
|
||||
|
||||
When you're ready, delete this note and make the vault your own.
|
||||
|
|
@ -0,0 +1,107 @@
|
|||
---
|
||||
source: "hyperthrive_dev"
|
||||
date: "2026-03-13"
|
||||
tags: [research, oo-principles, 99-bottles, tdd, software-design, process, shameless-green, open-closed, refactoring]
|
||||
---
|
||||
|
||||
# 99 Bottles OOP — Full Software Design Process Map
|
||||
|
||||
A complete map of the software design lifecycle described in *99 Bottles of OOP* by Sandi Metz and Katrina Owen. Documented during a session exploring how to better encode OO principles into AI coding conventions.
|
||||
|
||||
## Overview
|
||||
|
||||
The process is an infinite loop with four distinct phases: initial development, a waiting period, a refactoring loop triggered by new requirements, and implementation. The key insight is that refactoring and feature addition are deliberately separated.
|
||||
|
||||
## Phase 1: Initial Development (Shameless Green)
|
||||
|
||||
Goal: get to working, understandable, thoroughly tested code as fast as possible. Do NOT invent requirements or speculate about the future.
|
||||
|
||||
1. Sketch a high-level public API for the problem
|
||||
2. Write the first test targeting the simplest, most thoroughly understood piece of that API
|
||||
3. Red → Green (write only enough simple code to pass)
|
||||
4. Write the next test proving existing code is incomplete
|
||||
5. Write the simplest, most concrete code to pass — tolerate duplication
|
||||
6. Repeat horizontally until all variations are handled
|
||||
7. Stop. You have reached Shameless Green: easy to understand, fully tested, fulfills current requirements
|
||||
|
||||
Shameless Green optimizes for understandability, not changeability. If the code never changes, you stop here.
|
||||
|
||||
## Phase 2: The Waiting Period
|
||||
|
||||
Do nothing proactively. Wait for a new requirement. Two voluntary exceptions:
|
||||
|
||||
- **Exception A (Purify Tests):** Clarify test names, remove echoes of implementation details, ensure every class has its own unit test
|
||||
- **Exception B (Aesthetic Improvements):** Fix Law of Demeter violations, remove hard-coded class names, push object creation to the edges
|
||||
|
||||
Do NOT build abstractions speculatively. Save time and money.
|
||||
|
||||
## Phase 3: The Refactoring Loop (Open/Closed Principle)
|
||||
|
||||
Triggered when a new requirement arrives. Separate refactoring from feature addition.
|
||||
|
||||
**Decision gate:**
|
||||
1. Is the code "open" to the new requirement? (Can you implement it by adding code, not modifying existing code?)
|
||||
- YES → go to Phase 4
|
||||
- NO → continue
|
||||
2. Do you know how to make it open?
|
||||
- YES → refactor → return to step 1
|
||||
- NO → continue
|
||||
3. Identify the best-understood code smell and remove it using a mechanical recipe:
|
||||
- Need to unearth an abstraction → **Flocking Rules**
|
||||
- Conditionals supplying behavior → **Replace Conditional with Polymorphism**
|
||||
- Need to choose which polymorphic class to use → **Factory**
|
||||
- Objects tightly coupled → **Dependency Inversion** (push object creation to edges, inject dependencies)
|
||||
4. Enforce contracts: check LSP (objects return trustworthy types) and LoD (no chaining collaborator messages)
|
||||
5. Return to step 1
|
||||
|
||||
## Phase 4: Implementation
|
||||
|
||||
Code is open. Make the change — usually as simple as creating a new polymorphic class and registering it in the factory. Then loop back to Phase 2.
|
||||
|
||||
## Code Smells and Mechanical Recipes
|
||||
|
||||
**Switch Statement / Conditionals supplying behavior** → Replace Conditional with Polymorphism (or State/Strategy)
|
||||
|
||||
**Primitive Obsession** → Extract Class (give the primitive a domain object)
|
||||
|
||||
**Duplicated Code** → Extract to a single method; apply Flocking Rules to find the abstraction
|
||||
|
||||
**Large Class** → Extract Class (divide responsibilities)
|
||||
|
||||
**Data Clump** → Extract Class or consolidating method
|
||||
|
||||
**Law of Demeter Violation** (chained sends: `a.b.c`) → Delegation / message forwarding, or redesign from the sender's point of view
|
||||
|
||||
**LSP Violation** (returns unexpected types) → Perform type conversion inside the method
|
||||
|
||||
**Temporary Variable** (used only once) → Inline Temp
|
||||
|
||||
**Blank Line within a method** → SRP violation; separate responsibilities
|
||||
|
||||
## The Flocking Rules (Unearthing Abstractions)
|
||||
|
||||
Apply mechanically when you don't understand the abstraction yet:
|
||||
|
||||
1. Select the things that are most alike
|
||||
2. Find the smallest difference between them
|
||||
3. Make the simplest change that removes that difference
|
||||
|
||||
Apply in four microscopic steps: (a) parse the new code, (b) parse and execute it, (c) parse, execute, and use its result, (d) delete unused code. Make one line change at a time. Run tests after every change. Undo immediately on failure.
|
||||
|
||||
## Replace Conditional with Polymorphism (Recipe)
|
||||
|
||||
1. Create a subclass for the value you switch on
|
||||
2. Copy one switching method into the subclass; keep only the true branch
|
||||
3. Create a factory if none exists; register the subclass
|
||||
4. In the superclass, remove everything but the false branch
|
||||
5. Repeat for all switching methods
|
||||
6. Iterate until a subclass exists for every switched value
|
||||
|
||||
## The Ultimate Goal
|
||||
|
||||
Build applications out of trustworthy, loosely-coupled, polymorphic objects that can survive an unknown future. The programming aesthetic: fall in love with polymorphism.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[AI Coding Conventions Organization — External Research Synthesis]] — how practitioners encode process and principles into AI context
|
||||
- [[OO Principles Plugin Concept — Design Recommendations]] — how to build an AI plugin based on this lifecycle
|
||||
|
|
@ -0,0 +1,100 @@
|
|||
---
|
||||
source: "hyperthrive_dev"
|
||||
date: "2026-03-13"
|
||||
tags: [research, ai-conventions, context-engineering, claude-md, progressive-disclosure, token-efficiency, cursor-rules, codified-context, three-tier-architecture]
|
||||
---
|
||||
|
||||
# AI Coding Conventions Organization — External Research Synthesis
|
||||
|
||||
Research findings on how practitioners organize opinionated coding conventions and standards to guide AI coding assistants (Claude, Cursor, Copilot, etc.) toward consistently high-quality output. Gathered during a session exploring improvements to this workspace's OO principles conventions.
|
||||
|
||||
## Core Finding
|
||||
|
||||
Reference documentation with code examples outperforms process flowcharts for AI coding behavior. LLMs already know the concepts — what they need is project-specific patterns, failure modes, and examples. However, process *does* matter at the routing layer: trigger tables and decision gates in a "constitution" document tell the AI what context to load and when.
|
||||
|
||||
## The Three-Tier Architecture (Codified Context, arXiv 2602.20478)
|
||||
|
||||
The most empirically rigorous approach, documented across 283 sessions on a 108,000-line C# codebase. Context infrastructure totaled 24.2% of codebase size — treated as load-bearing infrastructure.
|
||||
|
||||
**Tier 1 — Constitution (~660 lines, always loaded)**
|
||||
Coding conventions, naming rules, build commands, architectural summaries, orchestration protocols, and trigger tables that route tasks to domain specialists. Must stay concise; details live downstream.
|
||||
|
||||
**Tier 2 — Domain Specialist Agents (300–700 lines each, on-demand)**
|
||||
Over half the content of each agent is project-domain knowledge (patterns, formulas, known failure modes), not behavioral instructions. Invoked by trigger table based on which files are being modified.
|
||||
|
||||
**Tier 3 — Knowledge Base (on-demand, served via MCP)**
|
||||
Deep specification documents (~16k lines total), formatted for machine consumption.
|
||||
|
||||
**Quantitative result:** 2,801 human prompts generated 16,522 autonomous agent turns.
|
||||
|
||||
## Token Efficiency Strategies
|
||||
|
||||
- **Keep always-loaded context small** — under 200 lines for root files. Frontier models follow ~150–200 instructions before compliance degrades. Claude's own system prompt consumes ~50 slots.
|
||||
- **Just-in-time retrieval** — maintain lightweight identifiers (paths, doc titles) and load content only when needed
|
||||
- **Linters/formatters replace style rules** — if a tool can enforce it deterministically, don't document it
|
||||
- **Automated backpressure** — instruct agent to run `lint:fix`/`typecheck` after changes; the output IS the feedback loop
|
||||
- **Examples over enumeration** — one canonical worked example outperforms exhaustive lists of minimal-variation cases
|
||||
- **Positive framing only** — negative instructions prime the model toward the forbidden pattern
|
||||
|
||||
## Organizational Patterns
|
||||
|
||||
**Root file (CLAUDE.md, .cursorrules, AGENTS.md):** Always loaded, under 200–300 lines. Technology stack, essential commands, domain terminology, pointers to deeper docs, non-negotiable guardrails.
|
||||
|
||||
**Hierarchical scoping (AGENTS.md / Cursor .mdc):** Nested files override parents. Subproject-level instructions without polluting global context. The AGENTS.md spec: "The closest AGENTS.md to the edited file wins." OpenAI's main repo carries 88 AGENTS.md files.
|
||||
|
||||
**Cursor rule types:**
|
||||
- `alwaysApply: true` — loaded every session
|
||||
- Auto-attached — loaded when matching glob patterns are opened
|
||||
- Agent-requested — agent self-selects based on description
|
||||
- Manual (`@ruleName`) — developer explicitly invokes
|
||||
|
||||
**Feedback loop documentation (the /learn pattern):** Mistakes → conversation analysis → persistent docs → auto-loaded next session. Converts errors into permanent constraints without model retraining.
|
||||
|
||||
## Process vs. Reference Design
|
||||
|
||||
- **Process encoding works best as trigger tables** in the constitution (what context to load when), not as step-by-step workflows in content documents
|
||||
- **Reference docs with examples** are what specialist agents and knowledge base contain
|
||||
- **Decision trees / flowcharts** work when framed as routing logic at the constitution layer, not as concept documentation
|
||||
|
||||
## Context Linking Patterns
|
||||
|
||||
- Root file as map; deeper docs as territory (file path references)
|
||||
- Trigger tables: `modified files in X domain → consult specialist agent Y`
|
||||
- IMPORTANT directive pattern: "Before starting any task, identify which documentation is relevant and read it first"
|
||||
- Hierarchical override: subproject files override workspace-level files
|
||||
|
||||
## Enforcement Mechanisms (strongest to weakest)
|
||||
|
||||
1. Automated tools (linters, formatters, type checkers) — deterministic
|
||||
2. Scoped loading (auto-attach only relevant context) — structural
|
||||
3. Trigger tables in constitution — routing enforcement
|
||||
4. Feedback loop documentation — accumulated institutional memory
|
||||
5. Instruction placement (priority directives at top of always-loaded files)
|
||||
6. Conciseness — shorter instruction sets are better enforced; exceeding 150–200 instructions causes unpredictable selective ignoring
|
||||
|
||||
## Key Tradeoffs
|
||||
|
||||
**Comprehensiveness vs. compliance:** More instructions degrade compliance beyond ~200. Solution: move detailed instructions to on-demand layers, not always-loaded context.
|
||||
|
||||
**Specificity vs. generality:** "Write idiomatic code" is useless; "Use `stem` not `slug` in Nuxt Content queries" is actionable. Over-specification creates rigidity.
|
||||
|
||||
**Process vs. flexibility:** Rigid workflows reduce variance for routine tasks but fail on edge cases. Trigger tables for routing + flexibility within domains is the balance.
|
||||
|
||||
**Documentation overhead vs. velocity:** High upfront cost, compounds over time. Only worthwhile for large, long-horizon projects.
|
||||
|
||||
**Single file vs. distributed system:** Simple to maintain vs. more powerful but fragile. Don't adopt distributed context until single-file demonstrably falls short. "Agents trust documentation absolutely" — a wrong spec produces confidently incorrect code.
|
||||
|
||||
## Notable Public References
|
||||
|
||||
- **Codified Context paper:** arxiv.org/abs/2602.20478 — most rigorous empirical treatment
|
||||
- **github.com/arisvas4/codified-context-infrastructure** — companion templates and examples
|
||||
- **Anthropic Effective Context Engineering:** anthropic.com/engineering/effective-context-engineering-for-ai-agents
|
||||
- **PatrickJS/awesome-cursorrules** — large collection of real-world examples
|
||||
- **alexop.dev — Stop Bloating Your CLAUDE.md** — practical progressive disclosure implementation
|
||||
- **mbleigh.dev — Rules for Rules** — writing docs for LLMs
|
||||
- **agents.md** — AGENTS.md open standard
|
||||
|
||||
## See Also
|
||||
|
||||
- [[99 Bottles OOP — Full Software Design Process Map]] — the OO design lifecycle this workspace is encoding
|
||||
- [[OO Principles Plugin Concept — Design Recommendations]] — how to apply these findings to a plugin
|
||||
|
|
@ -0,0 +1,93 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-13"
|
||||
tags: [research, pest-control, pest-control-spring-2026, niche-automation-prospecting, market-data, lead-response, field-service, source-claude]
|
||||
---
|
||||
|
||||
# Hidden Economics of Missed Calls in Urgent Home Services
|
||||
|
||||
Economic analysis of missed calls in pest control, plumbing, HVAC, and locksmith businesses. Covers speed-to-lead compression, emergency caller behavior, interim engagement research, CAC/LTV by vertical, and true cost of unanswered leads. Source: Claude Compass artifact.
|
||||
|
||||
## Speed-to-Lead: The Window Has Compressed
|
||||
|
||||
Historical MIT/InsideSales benchmark (2007): contact within 5 min vs 30 min = 100× more likely; qualify = 21×. HBR (2011, 2.24M leads): 78% of sales go to first responder; firms responding within 1 hour = 7× more likely to qualify.
|
||||
|
||||
More recent evidence shows the window has compressed further:
|
||||
|
||||
Driven Results (2025) — most trade-specific data available (2,847 contractor leads, 38 businesses):
|
||||
| Response time | Booking conversion | vs. baseline |
|
||||
|---|---|---|
|
||||
| <60 seconds | 47% | baseline |
|
||||
| 2–5 minutes | 31% | −34% |
|
||||
| 10–30 minutes | 11% | −77% |
|
||||
| 30+ minutes | 4% | −91% |
|
||||
|
||||
LeadConnect (2023): optimal window for home services has shrunk from 5 minutes to **90 seconds**.
|
||||
|
||||
Velocify (3.5M leads): responding within 1 minute boosts conversion by 391%; that lift is cut by more than half at the 2-minute mark.
|
||||
|
||||
## After-Hours Is Where the Economics Are Most Punishing
|
||||
|
||||
Housecall Pro: 41% of online bookings arrive after business hours; significant demand between 1 AM and 4 AM.
|
||||
|
||||
Driven Results: 67% of home services leads arrive outside 9–5; only 12% of service businesses can respond instantly.
|
||||
|
||||
Evening leads (5–9 PM): 3.2× higher purchase intent than midday. Late-night leads (9 PM–midnight): 4.1× higher intent; 73% conversion when answered instantly. Weekend leads left until Monday: 87% likely already booked a competitor.
|
||||
|
||||
Industry actual performance: 95% of home service companies fail to respond within 5 min; 55% fail within a full day. Average response time by trade: HVAC 4.2h, plumbing 5.1h, electrical 6.3h. Top 10% of HVAC businesses respond in <5 minutes; top 10% of plumbers in <3 minutes.
|
||||
|
||||
## Emergency Callers Don't Shop — They Book Whoever Answers
|
||||
|
||||
85% of callers who can't reach someone immediately move on to the next listing (NextPhone/Suzee AI, plumbing call data). 66% immediately move to another contractor if the first call goes unanswered (Angi).
|
||||
|
||||
Emergency calls: <10% contact more than 3 companies. Average companies contacted:
|
||||
- Emergency plumbing/HVAC: 1–2
|
||||
- Emergency locksmith: 1
|
||||
- Active pest infestation: 1–3
|
||||
- Planned services: 2–3
|
||||
|
||||
Emergency services convert 73% higher than routine maintenance inquiries.
|
||||
|
||||
## Interim Engagement: "While You Wait, Do X"
|
||||
|
||||
No home-services-specific A/B tests exist, but convergent evidence is strong:
|
||||
|
||||
Harvard Business School (Ryan Buell, 2025 field experiment, 393,036 customers): operational transparency → 20.5% lower cancellation rate + 9.9% higher monthly spending.
|
||||
|
||||
Healthcare: no-show rate dropped from 8% to 2.3% with pre-appointment instructions (71% reduction, Eisenhower Health). Systematic review of 26 studies: reminded patients 23% more likely to attend.
|
||||
|
||||
Six mechanisms: occupied time perception, pre-process to in-process conversion, anxiety reduction, reciprocity activation, commitment escalation, operational transparency. Conservative estimate for home services: 15–25% cancellation reduction.
|
||||
|
||||
Existing automated appointment reminders already show 50–60% reduction in missed appointments. Adding actionable preparation instructions on top is the under-studied, high-upside move.
|
||||
|
||||
## CAC and LTV by Vertical
|
||||
|
||||
| Vertical | CAC | CPL (LSA) | CPL (PPC) | LTV |
|
||||
|---|---|---|---|---|
|
||||
| HVAC | $296–$350 | $45–$85 | $100–$250 | $15,340 |
|
||||
| Plumbing | $100–$300 | $40–$75 | $76–$100 | $3,000–$5,000 |
|
||||
| Pest control | $100–$250 | $20–$70 | $30–$98 | $2,500–$3,600 |
|
||||
| Locksmith | $130–$200 | $20–$30 | ~$66 | $500–$1,500 |
|
||||
|
||||
LSA leads convert at 31% vs 5–12% for traditional PPC (Home Service Direct, 2026).
|
||||
|
||||
## True Cost of a Missed After-Hours Call
|
||||
|
||||
Using conservative model (30% conversion probability, 50% uncertainty discount on future value):
|
||||
|
||||
| Vertical | Lead cost wasted | Realistic total loss/call | Annual impact |
|
||||
|---|---|---|---|
|
||||
| HVAC | $56–$85 | ~$2,400 | $50,000+ |
|
||||
| Plumbing | $56–$75 | ~$710 | $50,000–$60,000 |
|
||||
| Pest control | $30–$50 | ~$520 | $25,000–$40,000 |
|
||||
| Locksmith | $20–$66 | ~$220 | $15,000–$25,000 |
|
||||
|
||||
Suzee AI analysis (2025): typical plumber misses 168 calls/month → ~143 lost prospects → ~43 lost jobs → ~$50,000–$60,000/year in lost revenue.
|
||||
|
||||
ServiceTitan: 5% improvement in booking rate ≈ $100,000 in additional annual revenue for a typical contractor. Industry average booking rate: 42%.
|
||||
|
||||
## Key Novel Findings (Combined)
|
||||
|
||||
1. Speed-to-lead window is 90 seconds for home services — not the commonly cited 5 minutes — and conversion drops from 47% to 4% between instant response and 30-minute delay
|
||||
2. Emergency callers functionally don't shop: <10% contact 3+ companies — first-response capability is the single highest-leverage investment
|
||||
3. "While you wait" interim engagement is an untapped gap — no home services company appears to be systematically deploying this despite strong evidence from healthcare (71% no-show reduction) and behavioral science (20.5% cancellation reduction)
|
||||
|
|
@ -0,0 +1,77 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-13"
|
||||
tags: [research, pest-control, pest-control-spring-2026, niche-automation-prospecting, market-data, lead-response, field-service, source-chatgpt]
|
||||
---
|
||||
|
||||
# Lead Response Benchmarks Urgent Home Services Verticals
|
||||
|
||||
Comprehensive research report on speed-to-lead benchmarks, conversion curves, CAC, homeowner shopping behavior, and interim engagement strategy across urgent home services verticals (pest control, plumbing, HVAC, locksmith). Source: ChatGPT deep research with cited 2023–2026 studies.
|
||||
|
||||
## Speed-to-Lead Benchmarks
|
||||
|
||||
2024 independent mystery-shop study (466 U.S. home service companies):
|
||||
- 40% never responded within 5 days
|
||||
- 95% did not respond within 5 minutes
|
||||
- 71% did not respond within 1 hour
|
||||
|
||||
MIT/InsideSales seminal study (15,000+ web leads): contacting within 5 min vs 30 min = 100× more likely to make contact; 21× more likely to qualify.
|
||||
|
||||
## Booking Conversion Curve (Home Services)
|
||||
|
||||
Best-estimate curve calibrated from recent home-services bin data (Driven Results) — 5–10 and 10–30 ranges are interpolated:
|
||||
- 0–5 min: ~29–58%
|
||||
- 5–10 min: ~28%
|
||||
- 10–30 min: ~21–22%
|
||||
- 30–60 min: ~14–15%
|
||||
- >60 min: ~9–10%
|
||||
|
||||
After-hours share: 40–62% of inquiries arrive outside 9–5 (varies by source/platform).
|
||||
|
||||
## Homeowner Multi-Provider Shopping Behavior
|
||||
|
||||
Emergency services: <10% contact more than 3 companies. The search stops at the first confirmed next step (arrival time or verbal commitment).
|
||||
|
||||
Non-emergency/planned services: 20–30% contact 3+ providers.
|
||||
|
||||
Key behavioral signal: when callers don't get through, 23% immediately call another company, 34% search for another online, 28% fill out forms for multiple companies — only 15% wait until morning (Driven Results, 10,847-inquiry dataset).
|
||||
|
||||
## CPL Benchmarks by Vertical (2023–2026)
|
||||
|
||||
From LocaliQ (3,200+ campaigns, April 2024–March 2025) and Coalmarch pest control benchmark:
|
||||
- HVAC (AC install/repair): ~$127.74
|
||||
- Heating/furnaces: ~$129.02
|
||||
- Plumbing: ~$129.02
|
||||
- Pest control — total paid CPL: ~$111; LSA: ~$96; Search ads: ~$148
|
||||
|
||||
## Interim Engagement — "While You Wait" Instructions
|
||||
|
||||
No home-services-specific A/B tests exist, but convergent evidence from adjacent fields:
|
||||
|
||||
Harvard Business School (Ryan Buell, 2025 field experiment, 393,036 customers): operational transparency → 20.5% lower cancellation rate + 9.9% higher monthly spending.
|
||||
|
||||
Healthcare: pre-appointment instructions reduced no-show rates from 8% to 2.3% at Eisenhower Health (71% reduction, generating ~$158,596 net revenue in one month). Systematic review of 26 studies: reminded patients 23% more likely to attend.
|
||||
|
||||
SaaS onboarding: proactive outreach for stalling users → 40% higher activation, 50% better 90-day retention.
|
||||
|
||||
Six mechanisms: occupied time perception, pre-process to in-process conversion, anxiety reduction, reciprocity activation, commitment escalation, operational transparency.
|
||||
|
||||
## Vertical Nuance
|
||||
|
||||
- Locksmith: highest urgency + scam risk; first trusted responder dominates; callers move on in <15 minutes
|
||||
- Plumbing: "damage clock" (leaks, backups) drives very short response windows; strong competition
|
||||
- HVAC: seasonal urgency; industry is among slowest responders despite high expectations — largest gap/opportunity
|
||||
- Pest control: urgency varies by pest type; high at time of search; conversion curve somewhat less steep than locksmith/plumbing for non-emergency pests
|
||||
|
||||
## CAC Estimates (Model-Based)
|
||||
|
||||
- HVAC: ~$231–$516 per booked job
|
||||
- Plumbing: ~$258–$477
|
||||
- Pest control: ~$274–$987 (wide range due to channel mix and lead quality variation)
|
||||
- Locksmith: ~$67–$400 (high uncertainty; limited primary data)
|
||||
|
||||
## Lost Value Formula
|
||||
|
||||
Expected lost value of slow response ≈ CPL + (p_fast − p_slow) × GP
|
||||
|
||||
Using home-services curve delta (~0.20 between 0–5 min and >60 min): a $50 lead with $500 GP = ~$150 expected value lost per lead from slow response. Scales to $65–$1,000+ per lead depending on CPL and gross profit.
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
---
|
||||
source: "hyperthrive_dev"
|
||||
date: "2026-03-13"
|
||||
tags: [plan, plugin, oo-principles, ai-conventions, rails, process-driven, refactoring, conventions-architecture]
|
||||
---
|
||||
|
||||
# OO Principles Plugin Concept — Design Recommendations
|
||||
|
||||
Gap analysis of the current OO principles conventions structure and architectural recommendations for evolving it into a standalone plugin. Based on a session combining study of 99 Bottles of OOP with external research on AI coding conventions practices.
|
||||
|
||||
The intent is to build the plugin separately and validate it in a new project before applying to this workspace. Primary tools: this workspace's conventions setup as the baseline, and the NotebookLM notebook at https://notebooklm.google.com/notebook/7e69f896-972e-4f5d-ad2c-152259efa62a
|
||||
|
||||
## Current State Assessment
|
||||
|
||||
The existing `conventions/oo-principles/` structure is **accurate but inert**. It captures individual principles correctly (TDD, Shameless Green, Flocking Rules, SRP, LoD, DI, etc.) but provides no decision-tree guidance for when to apply them. An AI agent reading these docs understands *what* each principle is, but not *when to use it*, *in what order*, or *how to transition between development phases*.
|
||||
|
||||
**Structural inventory:**
|
||||
- `QUICK_REFERENCE.md` — one-pager overview, lean and token-efficient
|
||||
- `cards/` — individual principle cards (~25 lines each), accurate and focused
|
||||
- `bundles/` — role-based concept bundles (developer, dev-lead, architect)
|
||||
|
||||
**What the AI loses without process guidance:**
|
||||
- No criteria for Flocking Rules vs. Replace Conditional with Polymorphism
|
||||
- "Refactor later" (Shameless Green) is undefined — when is "later"?
|
||||
- No principle prioritization when multiple smells coexist
|
||||
- The 4-phase development lifecycle doesn't exist anywhere in the codebase
|
||||
|
||||
## The Gap: Concrete Examples
|
||||
|
||||
**Example 1 — Flocking vs. Polymorphism:** An AI sees a `case` statement. The docs say it violates Open/Closed but don't say: "Use Flocking Rules for 2–3 cases; use Replace Conditional with Polymorphism when cases will grow beyond 3 or when a new requirement adds another branch."
|
||||
|
||||
**Example 2 — When to refactor:** Shameless Green says "tolerate duplication, refactor later." No definition of "later." Should the AI refactor after the next test? After the next requirement? Only when a requirement forces it?
|
||||
|
||||
**Example 3 — Which smell first:** Multiple smells coexist (SRP + LoD + DI). No prioritization order. The correct order is: DI first (is the dependency injected?), then LoD (are we chaining?), then SRP (is this class doing too much?).
|
||||
|
||||
**Example 4 — Phase detection:** An AI getting a new requirement can't tell if it's in Phase 1 (keep building Shameless Green) or Phase 3 (stop, check if code is open, refactor before implementing).
|
||||
|
||||
## Recommended Architecture: Two-Layer Hybrid
|
||||
|
||||
Based on external research (Codified Context paper) and brainstorming, the recommended structure is:
|
||||
|
||||
**Layer 1 — Process/Routing Layer** (~400–600 tokens, loaded at role-bundle level):
|
||||
- `PROCESS.md` encoding the 4-phase lifecycle as decision gates
|
||||
- Phase entry conditions ("This applies when...")
|
||||
- Key branch points: "Is the code open?", "Which smell to fix first?", "When to use Flocking vs. Polymorphism?"
|
||||
- An escape hatch: "If no phase fits, document your reasoning"
|
||||
- References Layer 2 but does not include it
|
||||
|
||||
**Layer 2 — Mechanic Layer** (on-demand, 400–800 tokens per file):
|
||||
- One file per executable recipe: flocking-rules-recipe.md, replace-conditional-recipe.md, factory-recipe.md, extract-class-recipe.md, etc.
|
||||
- Self-contained, readable in isolation
|
||||
- Includes worked examples (one canonical example > three paragraphs of description)
|
||||
- Separate from theory — concept cards remain as Layer 3 "why" depth
|
||||
|
||||
**Layer 3 — Theory Layer** (rarely loaded):
|
||||
- Existing concept cards, retained as-is or lightly reorganized
|
||||
- Loaded when AI needs to understand the rationale
|
||||
|
||||
## On Opportunistic Refactoring (Phase 2)
|
||||
|
||||
The book's conservative "wait for a requirement" stance was written for human teams where refactoring has real cost. AI changes the calculus: the mechanical recipes are exactly the kind of structured, low-ambiguity work AI excels at. The cost of refactoring itself approaches zero.
|
||||
|
||||
**The risk that remains:** Premature *abstraction* (not smell-fixing) still locks in wrong designs. Three similar lines of code is safer than a premature abstraction that guesses wrong.
|
||||
|
||||
**Recommended Phase 2 scope for AI:**
|
||||
- Law of Demeter violations (chained message sends)
|
||||
- Hard-coded class names
|
||||
- Push object creation to edges / dependency injection improvements
|
||||
- **Gate:** Only when tests are fully green AND the diff is reviewable as a single coherent unit
|
||||
- Do NOT leave "fix code smells" open-ended — AI will interpret it too broadly
|
||||
|
||||
## Four Architectural Approaches (Brainstorm)
|
||||
|
||||
Ranked from least to most disruptive:
|
||||
|
||||
**Approach 1: Process-First Entry Point** (recommended first move)
|
||||
Add `PROCESS.md` as new primary entry point. Existing concept cards remain, linked from the process doc. Minimal restructuring, additive change. Risk: if PROCESS.md is poorly written, the AI is misled — single point of failure.
|
||||
|
||||
**Approach 2: Situational Trigger Files**
|
||||
`situations/when-requirement-arrives.md`, `situations/when-code-is-not-open.md`, etc. Very low per-file token cost; AI narrates which decision node it's at. Risk: AI must correctly self-diagnose situation; link chains can go stale.
|
||||
|
||||
**Approach 3: Phase-Bundled Context Packs**
|
||||
One self-contained file per phase. Clean narrative, maps to how teams talk about work. Risk: content duplication across phases; Phase 3 complexity is hard to flatten; phase misidentification is a failure mode.
|
||||
|
||||
**Approach 4: Annotated Process Graph** (good as drafting exercise only)
|
||||
Single PROCESS.md with everything inline. Zero navigation overhead. Risk: 1500–2500 tokens paid upfront on every load; single document is hard to update surgically; resists progressive disclosure.
|
||||
|
||||
**Recommended sequence:**
|
||||
1. Draft Approach 4 once to validate coherence of the process
|
||||
2. Decompose into Approach 1 (PROCESS.md entry point + layered cards)
|
||||
3. Audit concept cards: split recipe content into `mechanics/`, keep theory in `concepts/`
|
||||
4. Add explicit Phase 2 gates
|
||||
|
||||
## Plugin Design Notes
|
||||
|
||||
When building as a standalone plugin (not modifying this workspace):
|
||||
- Model the plugin on the three-tier architecture: constitution (routing) + specialist agents (mechanic recipes + role bundles) + knowledge base (theory cards)
|
||||
- The plugin's entry point should be role-aware: a developer loads differently than an architect
|
||||
- Include the 4-phase process as a loadable context pack, not as always-loaded content
|
||||
- The NotebookLM notebook can serve as the knowledge base layer for deep reference during plugin development
|
||||
- Validate on a new project first before applying to this workspace
|
||||
|
||||
## Current Workspace Observation
|
||||
|
||||
This workspace already has the correct three-tier architecture: `CLAUDE.md` → `.CLAUDE.md` files → concept cards. The architecture is sound. The gap is content format (concept lists instead of mechanic recipes with worked examples) and missing process routing layer.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[99 Bottles OOP — Full Software Design Process Map]] — the lifecycle being encoded
|
||||
- [[AI Coding Conventions Organization — External Research Synthesis]] — external practitioner patterns this plugin should incorporate
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-13"
|
||||
tags: [research, pest-control, pest-control-spring-2026, niche-automation-prospecting, market-data, after-hours-sms, source-chatgpt]
|
||||
---
|
||||
|
||||
# Pest Control ACV Seasonality After-Hours Pain Points
|
||||
|
||||
Deep research report on ACV estimates, seasonal demand concentration, and after-hours contact failure patterns for $1M–$5M residential pest control companies. Source: ChatGPT deep research.
|
||||
|
||||
## ACV Estimates
|
||||
|
||||
Industry-standard ACV for residential pest control customers: $450–$840/year on renewal, $600–$1,100 in year one (higher due to initial treatment premium). Sources: NPMA, IBISWorld, Rollins/Rentokil public filings, review analysis.
|
||||
|
||||
Customer count models by revenue tier:
|
||||
- $1M company: ~1,700–2,000 active agreements
|
||||
- $3M company: ~5,000–6,000
|
||||
- $5M company: ~8,000–8,300
|
||||
|
||||
Revenue per technician benchmarks: $150,000–$175,000/year. Gross margins 50–55%; EBITDA target 20%+.
|
||||
|
||||
## After-Hours Complaint Patterns
|
||||
|
||||
Dominant complaint themes from Yelp, Google, BBB review analysis:
|
||||
- "No one answered" / "no one will answer the local number"
|
||||
- "Voicemail full" — signal of operational overwhelm to callers
|
||||
- Slow or absent callbacks — by the time the company reaches out, the customer has hired a competitor
|
||||
- Offshore/untrained call center agents who lack pest-specific knowledge
|
||||
- Weekend and holiday unavailability
|
||||
|
||||
Harvard Business Review research cited: lead conversion drops precipitously after 5 minutes; average callback time is 4 hours. By then, the customer has typically hired a competitor.
|
||||
|
||||
## Seasonality
|
||||
|
||||
Q2 + Q3 = 54% of annual demand. Quarter-by-quarter breakdown:
|
||||
- Jan–Feb: 5–8% (rodents, cockroaches; slowest period; optimal window to sell automation solutions)
|
||||
- Mar–May: explosive ramp — termite swarming season; ant, tick, stinging insect activation
|
||||
- Jun–Aug: absolute peak — mosquitoes, wasps/yellowjackets, cockroaches, bed bugs; 40% of annual marketing budget recommended for Q2 alone
|
||||
- Sep–Nov: first freeze triggers rodent migration; yellowjackets aggressive as colonies decline; high-value rodent exclusion jobs
|
||||
- Dec: 5–7%
|
||||
|
||||
Pest-specific seasonal triggers:
|
||||
- Termites: March–May (Southeast starts January, spreads northward through June)
|
||||
- Mosquitoes: May–November (peak July–August)
|
||||
- Yellowjackets: late summer (most aggressive as colonies decline in fall)
|
||||
- Rodents: colder months (especially within 24–48 hours of first freeze)
|
||||
|
||||
## Recommended Outreach Timing
|
||||
|
||||
Peak outreach windows align with Q2 (spring rush) and pre-fall rodent season (August–September). Companies overwhelmed during June–August are poor targets; the sell happens when they have capacity to evaluate new tools — January–February and August–September.
|
||||
|
||||
## Operational Priorities Identified
|
||||
|
||||
1. Human response loops — eliminate the dead ends that send callers to competitors
|
||||
2. Eliminate voicemail failure — 80–85% of callers won't leave a message; they move on
|
||||
3. Reduce call-center ping-pong — offshore agents who can't resolve issues drive reviews and churn
|
||||
|
|
@ -0,0 +1,103 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-13"
|
||||
tags:
|
||||
- research
|
||||
- pest-control
|
||||
- pest-control-spring-2026
|
||||
- niche-automation-prospecting
|
||||
- after-hours-sms
|
||||
- lead-capture
|
||||
- field-service
|
||||
- source-claude
|
||||
---
|
||||
|
||||
# Pest Control After-Hours SMS Lead Capture — Market Research & Pitch Data
|
||||
|
||||
Deep market research supporting the pitch for an After-Hours SMS Lead Capture & Qualification automation product targeting SMB pest control companies. Compiled from high-authority benchmarks, unit economics, and competitive intelligence. Generated by Claude deep research (March 2026).
|
||||
|
||||
## Core argument
|
||||
|
||||
Pest control SMBs are hemorrhaging revenue after hours through a channel (voicemail) that fewer than 3% of callers will use, in an industry where 85% of revenue is recurring and customer LTV reaches $1,500–$3,000. The competitive landscape is wide open — roughly 60% of companies have no real after-hours solution, and only 5–10% use automated SMS. The regulatory environment is simultaneously punishing opaque AI and creating a moat for transparent automation.
|
||||
|
||||
## How fast leads die
|
||||
|
||||
The MIT/InsideSales.com Lead Response Management Study found contacting a lead within 5 minutes is 100× more likely to result in contact and 21× more likely to qualify than waiting 30 minutes. Home-services-specific data (DrivenResults.co, 2,847 contractor leads):
|
||||
|
||||
- Within 60 seconds (text): 47% appointment booking rate
|
||||
- 2–5 minutes: 31% (−34%)
|
||||
- 10–30 minutes: 11% (−77%)
|
||||
- After 30 minutes: 4% (−91%)
|
||||
|
||||
The average HVAC contractor responds in 4.2 hours; plumbers average 5.1 hours. 80% of callers sent to voicemail hang up without leaving a message. Invoca home-services platform data: fewer than 3% of home services callers leave a voicemail. 85% of callers whose calls aren't answered will never call back; 62% immediately call a competitor.
|
||||
|
||||
## Wasted marketing spend
|
||||
|
||||
Pest control CPL by channel: Google LSA ~$35, Google PPC ~$50 (WordStream: $45.60), HomeAdvisor/Angi ~$50, blended average ~$45. A company missing 5 after-hours calls/week loses 260 leads/year — at $45 CPL, that's $11,700/year in wasted marketing spend before counting lost revenue.
|
||||
|
||||
## SMS vs. voicemail
|
||||
|
||||
SMS open rates: 98%, with 95% of texts read within 3 minutes (Gartner). Voicemail listen rates for unknown numbers: ~18% (some sources as low as 1%). EZ Texting 2024: text messaging (35%) surpassed email (31%) and phone (29%) as consumers' preferred customer support channel. 63% of consumers would switch to a company offering text messaging (Avochato).
|
||||
|
||||
## Pest control unit economics
|
||||
|
||||
- One-time general pest treatment: national average $171 (HomeAdvisor/Angi 2025), range $100–$300
|
||||
- Annual contracts: ~$500/year average (Coalmarch); 85.2% of residential revenue is recurring (NPMA)
|
||||
- Customer LTV: $1,500–$3,000 (3–5 year retention at 70–87% annual retention)
|
||||
- Specialized: termite avg $558–$694; bed bugs $1,000–$2,500; rodent exclusion $200–$600
|
||||
|
||||
Lead close rates: inbound phone calls 35–50% (Coalmarch ~50%, Invoca 37% cross-industry); web form leads 5–15%.
|
||||
|
||||
## Revenue recovery formula
|
||||
|
||||
Annual Lost Revenue = Weekly Missed Calls × 52 × Close Rate × Customer LTV
|
||||
|
||||
| Company size | Missed calls/week | Annual lost revenue (conservative: 35% × $1,500) |
|
||||
|---|---|---|
|
||||
| Small | 5 | $136,500 |
|
||||
| Medium | 10 | $273,000 |
|
||||
| Larger | 20 | $546,000 |
|
||||
|
||||
With SMS capture rate factored in (conservative 70%): 260 × 0.70 × 0.35 × $1,500 = **$95,550 recoverable annual revenue** for a 5-missed-calls/week company. Product price point $99–$199/month ($1,200–$2,400/year) — recovering 3 customers/month at $1,500 LTV = $54,000/year = 22–45× ROI.
|
||||
|
||||
## Competitive landscape
|
||||
|
||||
- ~55–65% of pest control SMBs use voicemail or don't answer (aligns with 411 Locals: only 37.8% of SMB calls answered live)
|
||||
- ~15–20% use live answering services ($300–$1,000+/month)
|
||||
- ~5–10% use AI-powered voice or SMS
|
||||
- ~32,720+ active pest control companies in the US, ~$26.1B revenue, two-thirds are single-location operators averaging $400K/year
|
||||
|
||||
## Transparent automation advantage
|
||||
|
||||
Gartner 2024 (5,728 customers): 64% prefer companies not use AI for customer service; 53% would switch upon discovering undisclosed AI use. Key insight: customers don't hate AI — they hate being deceived by it.
|
||||
|
||||
Regulatory environment punishing opaque AI:
|
||||
- FCC February 2024: AI-generated voices require prior express consent under TCPA
|
||||
- FTC "Operation AI Comply" (September 2024): $5M+ fines for deceptive AI
|
||||
- California BOT Act (SB 1001): fines up to $2,500/violation for undisclosed bots
|
||||
- Colorado AI Act: up to $20,000/violation
|
||||
- Federal preemption standards expected 2026
|
||||
|
||||
An SMS product positioned as transparent automation ("This is an automated message from [Company Name]") is pre-compliant and avoids hallucination liability (ref: Air Canada chatbot tribunal ruling, February 2024).
|
||||
|
||||
## 5 sales collateral stats
|
||||
|
||||
1. "Missing 5 after-hours calls a week costs $136,500/year in lost revenue" — 260 leads × 35% close × $1,500 LTV (pest-control-specific inputs from Coalmarch, Invoca, NPMA)
|
||||
2. "78% of customers hire whoever responds first — and 85% never call back if you don't answer" — Lead Connect + Message Direct/PATLive
|
||||
3. "Your voicemail captures fewer than 3% of after-hours callers. An SMS reaches 98% — a 33× improvement" — Invoca home-services data + Gartner
|
||||
4. "You're burning $11,700/year in marketing budget on leads that ring into voicemail" — 260 missed leads × $45 blended CPL (WordStream, Cube Creative)
|
||||
5. "53% of customers would switch providers if they discovered hidden AI — but 63% would switch *to* a company offering texting" — Gartner + Avochato
|
||||
|
||||
## Source quality notes
|
||||
|
||||
- CPL data is pest-control-specific for LSA and PPC (WordStream, Cube Creative); Angi/Thumbtack figures are broader home services
|
||||
- Anxiety gap data is general service psychology, not pest-control-specific
|
||||
- SMS preference data is general consumer behavior
|
||||
- Invoca "fewer than 3% leave voicemail" is home-services-specific
|
||||
- Close rates are pest-control-specific (Coalmarch, Soleo)
|
||||
- LTV figures are pest-control-specific (multiple sources)
|
||||
|
||||
## Related
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research-stats]] (Claude)
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research-pitch-economics]] (ChatGPT)
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research]] (Gemini)
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-13"
|
||||
tags:
|
||||
- research
|
||||
- pest-control
|
||||
- pest-control-spring-2026
|
||||
- niche-automation-prospecting
|
||||
- after-hours-sms
|
||||
- lead-capture
|
||||
- field-service
|
||||
- source-chatgpt
|
||||
---
|
||||
|
||||
# Pest Control After-Hours SMS Lead Capture — Market Research & Pitch Economics
|
||||
|
||||
Deep research report (ChatGPT, March 2026) benchmarking the market opportunity for an "After-Hours SMS Lead Capture & Qualification" automation product pitched to pest control SMBs.
|
||||
|
||||
## Key Findings
|
||||
|
||||
**Speed-to-lead decay:** Conversion rates are 8x higher within 5 minutes vs 6+ minutes. Responding in 30 min vs 5 min makes contact 100x less likely and qualification 21x less likely. Over half of homeowners decide on a pest control provider within 4 hours of reaching out — "next morning follow-up" often means the lead is already decided.
|
||||
|
||||
**After-hours volume:** 41% of home service jobs booked online come in after hours (Housecall Pro). Home services companies miss 27% of inbound calls (Invoca). 74% of consumers don't answer calls from unknown numbers, making "we'll call you back" an unreliable recovery strategy.
|
||||
|
||||
**Cost per lead benchmarks (pest control):** Operator-reported median CPL: $38; mean: $50.30. Home services paid media benchmark: $66.69–$70.11. High-competition Google Ads reality (2025): ~$98.12/lead.
|
||||
|
||||
**Waste metric — 5 missed after-hours calls/week:** 260 missed calls/year × $38–$98 CPL = $9.9k–$25.5k in wasted ad spend annually.
|
||||
|
||||
**Revenue loss formula:** 260 missed calls × 29–42% close rate × $2.7k–$6.5k LTV = $203k–$709k/year in long-term revenue at risk.
|
||||
|
||||
**Unit economics:** One-time treatment AOV: $171–$550. Annual recurring contract: $300–$900/yr. Specialized jobs: termites avg $1,100; bed bugs avg $967. LTV at 82–87% retention: ~$2.7k–$6.5k.
|
||||
|
||||
**SMS market adoption:** 74% of pest control operators already use SMS to communicate with customers; 43% expect it to be their primary channel within 3 years. 58% already respond to service requests via SMS. The offer is not teaching a new behavior — it's making existing behavior always-on and after-hours capable.
|
||||
|
||||
**Transparent automation positioning:** 48% of consumers don't trust AI completely handling customer service. California law requires bot disclosure. Framing as "transparent automation" (the bot identifies itself) is a trust and legal risk differentiator vs competitors deploying undisclosed AI.
|
||||
|
||||
## Pitch-Ready Cheat Sheet
|
||||
|
||||
- 8x conversion lift within 5 minutes
|
||||
- 4-hour homeowner decision window
|
||||
- 52% of pest control shoppers compare providers; availability is a top reason competitors lose
|
||||
- 5 missed after-hours calls/week = ~$10k–$25k wasted ad spend/year
|
||||
- Long-term revenue at risk: $203k–$709k/year
|
||||
- 82% of consumers check texts within 5 minutes; 57% expect a response within 15 minutes
|
||||
|
||||
## Related
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research-stats]] (Claude)
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research-pitch-data]] (Claude)
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research]] (Gemini)
|
||||
|
|
@ -0,0 +1,121 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-13"
|
||||
tags:
|
||||
- research
|
||||
- pest-control
|
||||
- pest-control-spring-2026
|
||||
- niche-automation-prospecting
|
||||
- after-hours-sms
|
||||
- lead-capture
|
||||
- field-service
|
||||
- source-claude
|
||||
---
|
||||
|
||||
# Pest Control After-Hours SMS Lead Capture — Market Research Stats
|
||||
|
||||
Deep market research supporting the pitch for an After-Hours SMS Lead Capture & Qualification automation product sold to pest control companies. Source-verified statistics across seven categories, each traced to its original study or industry report. Generated by Claude deep research.
|
||||
|
||||
The core finding: speed kills in lead conversion and almost no small service company is fast enough. A pest control owner paying $50/lead watches most of that spend evaporate because nobody answers after 7 PM.
|
||||
|
||||
## Lead Response Time and Conversion Decay
|
||||
|
||||
Three landmark studies form the foundation:
|
||||
|
||||
**MIT/InsideSales.com Lead Response Management Study (2007)** — 15,000+ leads, 100,000+ call attempts, 6 companies over 3 years:
|
||||
- Contact within 5 min vs. 30 min: odds of contact drop 100x; qualification odds drop 21x
|
||||
- After 1 hour: contact odds decline 10x+; qualification odds 6x+
|
||||
|
||||
**Harvard Business Review (March 2011)** — 2,241 U.S. companies, 1.25 million sales leads:
|
||||
- Firms responding within 1 hour were 7x more likely to qualify vs. waiting one hour longer; 60x more likely vs. 24+ hours
|
||||
- Average first response time: 42 hours. 23% of companies never responded. Only 37% responded within one hour.
|
||||
|
||||
**Velocify "Ultimate Contact Strategy" Study (2012)** — 3.5 million leads, 400+ companies (minute-by-minute decay curve):
|
||||
- 1 minute: +391% conversion; 2 min: +160%; 3 min: +98%; 30 min: +62%; 1 hour: +36%; 24 hours: +17%
|
||||
|
||||
**InsideSales.com 2021** (55M sales activities, 5.7M inbound leads): conversion 8x higher when contacted within 5 min vs. 5 min–24 hours. Only 0.1% of inbound leads engaged under 5 minutes. 57.1% of first call attempts happened after more than a week.
|
||||
|
||||
## After-Hours Call Volume
|
||||
|
||||
- 47% of all home service calls occur outside traditional business hours (HomeAdvisor)
|
||||
- 58% of home service calls are emergency-related (Angi)
|
||||
- Call volume peaks 5–8 PM — exactly when most businesses stop answering (CallRail)
|
||||
- 85% of callers won't try again if unanswered the first time (PATLive)
|
||||
- 80% of callers sent to voicemail hang up without leaving a message (Forbes; SellCell 2024)
|
||||
- 62–67% of unanswered callers immediately call a competitor (PATLive; Angi)
|
||||
- Weekend leads sitting until Monday are 87% likely to have already booked someone else (Driven Results, 2,847 contractor leads, 2025)
|
||||
- 82% of consumers expect a response within 10 minutes (HubSpot 2023)
|
||||
|
||||
## Cost Per Lead and Waste
|
||||
|
||||
| Channel | Cost per lead |
|
||||
|---|---|
|
||||
| Google Ads (PPC) | $45.60 average |
|
||||
| Google Local Services Ads | $20–$70 |
|
||||
| HomeAdvisor / Angi | $25–$120 (shared with 3–5 contractors) |
|
||||
| Thumbtack | $10–$75 (shared) |
|
||||
| Industry working average | ~$50/lead (Coalmarch) |
|
||||
|
||||
- 71% of internet leads are wasted due to poor/slow follow-up (Forbes/InsideSales) → company spending $3,000/month on lead gen burns ~$2,100/month (~$25,000/year)
|
||||
- 95% of home services companies did NOT respond within 5 minutes (Valve+Meter, 466 companies, secret shopper)
|
||||
- 40% of home services companies never responded at all within 5 days (same study)
|
||||
- Each missed call costs home service companies avg $1,200 in lost revenue (Invoca, 2024)
|
||||
|
||||
## Pest Control Unit Economics
|
||||
|
||||
- One-time visit: $135–$250; general treatment ~$170 nationally
|
||||
- Recurring contracts: $35–$75/month ($420–$900/year)
|
||||
- Industry planning average: ~$500/year per customer (Coalmarch)
|
||||
- LTV: $1,500–$3,600 (3–5 year retention at 70–90% retention rate)
|
||||
- 77% of pest control customers never switch companies (PPMA survey)
|
||||
- ~75% of industry revenue comes from recurring services (Scorpion)
|
||||
- Average gross margin: 58% (NPMA & PCO Bookkeepers 2025 Cost Study, 246 firms, $584M combined revenue — gold standard source)
|
||||
- Net profit margins: 10–20%, average ~13.7%
|
||||
- U.S. market: ~$24–26 billion (2025), ~5% CAGR, ~33,000 businesses nationally
|
||||
|
||||
**The math for copy:** At $50/lead, $500 annual customer value, $2,000+ LTV — every unanswered lead is $2,000+ in lifetime revenue walking to a competitor. 3 missed leads/week = ~$300,000+ in lost LTV/year.
|
||||
|
||||
## SMS vs. Phone Effectiveness
|
||||
|
||||
- SMS open rates: up to 98% vs. ~20% email (Gartner 2016 — ceiling figure; realistic range 90–98%)
|
||||
- SMS response rate: 45% vs. email 6% (same Gartner report)
|
||||
- 90% of texts read within 3 minutes; avg SMS response time 90 seconds vs. 90 minutes for email (D7 Networks; CTIA)
|
||||
- 80% of all calls go to voicemail (SellCell 2024)
|
||||
- 74% of consumers don't answer calls from unknown numbers — assume spam (TransUnion, 1,556 adults, August 2024)
|
||||
- 69% of consumers prefer an unfamiliar company contact them via text rather than phone (Avochato, 1,000 adults, 2019)
|
||||
- 63% would switch to a company that offered text messaging as a channel (same study)
|
||||
- Velocify/Leads360 (3.5M leads, 2013): prospects receiving texts convert 40% higher; 3+ texts after phone contact = 328% higher conversion
|
||||
|
||||
**Important nuance:** 83% of homeowners still prefer phone calls for initial contact (HomeAdvisor) — urgency drives this. But they respond far better to texts for follow-up and scheduling. The play is catching leads the phone misses, not replacing the phone.
|
||||
|
||||
## Industry Adoption Gap (Competitive Opportunity)
|
||||
|
||||
- Only ~20% of pest control companies have adopted advanced software/automation tools; 80% rely on manual processes (Briostack, via Cube Creative Design, 2024)
|
||||
- 64% of contractors still rely on phone calls as dominant channel; only 7% prioritize text messaging (ServiceTitan 2025, 1,000+ contractors)
|
||||
- Average response times by trade: HVAC 4.2 hrs, plumbing 5.1 hrs, electrical 6.3 hrs, roofing 8.7 hrs (Driven Results, 2025)
|
||||
- A pest control company auto-responding via text within 60 seconds is operating in the top 5% of the entire home services industry for responsiveness
|
||||
|
||||
## Outcome Data (Companies That Fixed It)
|
||||
|
||||
- Text responses under 60 seconds: 73% response rate, 47% lead-to-booked-appointment conversion. After 30 min: dropped to 4% (Driven Results, 2,847 contractor leads, 2025)
|
||||
- Point Loma Electric & Plumbing: 15% booking rate boost; 20–30 extra weekend appointments from bots alone (Hatch case study)
|
||||
- Shafer Services (HVAC): 80% more leads booked with AI text/voice follow-up (Hatch)
|
||||
- Aruza Pest Control: routing time cut from 10 hrs/week to 30 min/month; cost-per-lead reduced 38% (Cube Creative Design case study)
|
||||
- ServiceTitan data: every 5% increase in call booking rate = ~$100,000 additional revenue for mid-size companies
|
||||
|
||||
## Strongest Stats for Cold Email Copy (Ranked)
|
||||
|
||||
1. **5-minute window**: leads contacted within 5 min are 21x more likely to convert than at 30 min (HBR/MIT — 1.25M leads, gold-standard)
|
||||
2. **Voicemail death spiral**: 80% don't leave a message; 74% won't answer callback from unknown number
|
||||
3. **Competitor defection**: 62–67% of unanswered callers immediately call a competitor
|
||||
4. **After-hours gap**: 47% of home service calls come outside business hours
|
||||
5. **LTV math**: missed after-hours lead = $2,000+ loss, not $50
|
||||
6. **SMS advantage**: 98% open rate, 90-sec response time, 45% response rate vs. 5% voicemail callback
|
||||
7. **Industry gap**: 95% of home services fail the 5-min window; only 20% of pest control use any automation
|
||||
|
||||
Use pest control unit economics ($500/year customer value, $2,000+ LTV, 58% gross margins from NPMA) to translate abstract percentages into concrete dollar amounts.
|
||||
|
||||
## Related
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research-pitch-data]] (Claude)
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research-pitch-economics]] (ChatGPT)
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research]] (Gemini)
|
||||
|
|
@ -0,0 +1,88 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-13"
|
||||
tags:
|
||||
- research
|
||||
- pest-control
|
||||
- pest-control-spring-2026
|
||||
- niche-automation-prospecting
|
||||
- after-hours-sms
|
||||
- lead-capture
|
||||
- field-service
|
||||
- source-gemini
|
||||
---
|
||||
|
||||
# Pest Control After-Hours SMS Lead Capture — Market Research
|
||||
|
||||
Deep market research generated by Gemini on the economics of after-hours SMS lead capture and qualification for pest control companies. This research supports pitching an automation product that captures and qualifies inbound leads outside business hours — the core pain being that high-intent pest inquiries decay in minutes, not hours.
|
||||
|
||||
## Core Finding: Lead Decay is Severe and Quantified
|
||||
|
||||
Pest control leads are perishable. The probability of qualifying a lead drops off a cliff within the first 30 minutes:
|
||||
|
||||
- Responding within 5 minutes = 21x more likely to qualify vs. 30-minute response
|
||||
- Responding within 60 seconds = up to 391% conversion lift
|
||||
- After 4 hours, a lead is 50% less likely to convert
|
||||
- After-hours leads sitting 12–14 hours lose 80–90% of their conversion probability
|
||||
|
||||
78% of homeowners buy from the first company to respond. 60% move to the next listing if the first call goes unanswered. The friction to switch is nearly zero in a mobile-first search environment.
|
||||
|
||||
## Why SMS Is the Right Channel
|
||||
|
||||
SMS open rate is 98% vs. under 20% for email. 89% of consumers prefer text over phone calls for business interactions. 80% of consumers block or ignore unknown numbers, making outbound callback ineffective. The channel match for after-hours capture is strong.
|
||||
|
||||
## The Anxiety Gap
|
||||
|
||||
Pest discovery creates an acute psychological state — fear of structural damage, disease, financial ruin. Immediate SMS acknowledgment closes the "Anxiety Gap" not by solving the problem, but by eliminating uncertainty about whether help is coming. 90% of customers rate immediate response as essential when they have a service question.
|
||||
|
||||
## Financial Model: The Waste Metric
|
||||
|
||||
At a conservative $50 CPL, missing 5 after-hours leads per week = $13,000/year in marketing spend wasted (the lead cost, not the job revenue). In competitive markets at $150 CPL, that rises to $39,000/year. This is money already spent, generating zero return.
|
||||
|
||||
## Revenue Recovery Formula
|
||||
|
||||
Using 260 missed leads/year × 40% close rate × $3,000 LTV = $312,000 in annual lost revenue potential. Even recovering 20% of those leads = $62,400/year in pure margin — the marketing cost is already sunk.
|
||||
|
||||
## Unit Economics Benchmarks (2025–2026)
|
||||
|
||||
- General pest one-time service AOV: $150–$400
|
||||
- Annual contract recurring LTV: $2,000–$3,600 (4–6 year retention)
|
||||
- Termite treatment LTV: $3,000–$5,000+
|
||||
- Blended recurring LTV used in modeling: ~$3,000
|
||||
- Residential retention rate: 82–87%; commercial: 94%+
|
||||
- CAC: $200–$400; LTV:CAC target ratio: >3:1
|
||||
- Gross profit margin: 50–55%
|
||||
- Close rate on inbound leads: 30–50%
|
||||
|
||||
## Cost Per Lead Benchmarks
|
||||
|
||||
- Google LSA: $20–$70 (pay-per-qualified-lead)
|
||||
- Google PPC: $40–$120+ ($25–$40 cost-per-click)
|
||||
- Facebook/Instagram: $50–$150 (mid-to-low intent)
|
||||
- Termite/specialty keywords: $180–$400+
|
||||
- Metro markets (Atlanta, Houston, Phoenix): $250–$400
|
||||
|
||||
## Competitive Landscape and Automation Adoption
|
||||
|
||||
- 60% of pest control companies use some form of AI for customer interactions as of late 2025 (CAGR 20% through 2029)
|
||||
- 27% have adopted AI-powered lead capture/SMS specifically
|
||||
- 30% use 24/7 live answering services
|
||||
- 50% use automated SMS reminders
|
||||
- 45% of operators rank recruiting/retention as their single biggest threat — labor pressure is forcing office function automation
|
||||
|
||||
## The Uncanny Valley Risk
|
||||
|
||||
Human-mimicking bots create trust damage in an industry where technicians enter homes. The recommended positioning is "Transparent Automation" — clearly identified as a digital assistant, not a human. Sets realistic expectations, avoids deception complaints, still delivers 24/7 responsiveness. Key failure modes to avoid: deceptive human personas, looping inflexibility, hallucinated pricing, and robotic apologies for serious failures.
|
||||
|
||||
## Abandonment Rate by Response Method
|
||||
|
||||
- Voicemail: 80–90% abandonment
|
||||
- Offshore call center: 25–40%
|
||||
- On-hold live agent: 40% after 30 seconds
|
||||
- Automated SMS: <2%
|
||||
- Instant AI voice/SMS (under 5 seconds): <5%, 80%+ satisfaction
|
||||
|
||||
## Related
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research-stats]] (Claude)
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research-pitch-data]] (Claude)
|
||||
- [[2026-03-13-pest-control-after-hours-sms-lead-capture-market-research-pitch-economics]] (ChatGPT)
|
||||
|
|
@ -0,0 +1,84 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-13"
|
||||
tags: [research, pest-control, pest-control-spring-2026, niche-automation-prospecting, market-data, after-hours-sms, unit-economics, source-claude]
|
||||
---
|
||||
|
||||
# Pest Control Business Model Decoded for After-Hours Automation
|
||||
|
||||
Complete breakdown of the pest control business model, after-hours complaint patterns, seasonal dynamics, and the financial case for after-hours call handling automation — framed specifically for a $1M–$5M residential pest control operator. Source: Claude Compass artifact.
|
||||
|
||||
## Business Model Snapshot
|
||||
|
||||
ACV: $480–$600/year for general pest contracts. With termite protection ($175–$500/year), seasonal mosquito/tick packages ($350–$700/season), and wildlife/rodent exclusion ($500–$2,000+), blended ACV can reach $700+.
|
||||
|
||||
Customer count by revenue tier:
|
||||
- $1M: ~1,500–2,000 customers, 5–7 trucks
|
||||
- $3M: ~4,500–6,000, 18–21 trucks
|
||||
- $5M: ~8,000–10,000, 28–35 trucks
|
||||
|
||||
Revenue per technician: $150,000–$175,000/year.
|
||||
|
||||
Recurring revenue share target: 70–80%+. Companies hitting 80%+ command highest M&A multiples (3–4x EBITDA).
|
||||
|
||||
Customer retention: 80–85% annually. Average customer lifespan: 4–7 years. LTV: $2,500–$3,000.
|
||||
|
||||
CAC: $150–$350. LTV:CAC target: 3:1. Gross margin: 50–55%. EBITDA target: 20%+.
|
||||
|
||||
Key stat: 91% of cancellations are preventable. #1 reason customers leave (62%): feeling the company "no longer cares." This is a communication problem, not a treatment problem.
|
||||
|
||||
## After-Hours Complaint Patterns
|
||||
|
||||
37% of 1-star reviews for home service businesses cite failure to answer the phone or poor communication as the primary complaint — not treatment quality.
|
||||
|
||||
- 80–85% of callers will not leave a voicemail when dealing with a pest issue; they hang up and call the next Google listing
|
||||
- Only 3% of callers pushed to voicemail leave a message (Invoca platform data)
|
||||
- Average callback time is 4 hours — by which point the customer has hired a competitor
|
||||
|
||||
Specific complaint themes from Yelp/Google/BBB: "called 10 times, no one answers"; "voicemail full"; routed to offshore agents who don't know pest control; no-show + no follow-up communication.
|
||||
|
||||
## After-Hours Emergency Pest Types (in urgency order)
|
||||
|
||||
1. Wasps/hornets/yellowjackets — most-cited emergency; nests near doorways/playgrounds; July–September peak
|
||||
2. Rodents — nighttime noise (scratching in walls/attics) or kitchen droppings; nocturnal triggers evening/overnight calls
|
||||
3. Bed bugs — discovered when going to bed or waking with bites; panic response is immediate
|
||||
4. Wildlife intrusion — raccoons, bats, squirrels; nocturnal activity; highest-value tickets ($200–$600/call)
|
||||
|
||||
## Seasonal Demand
|
||||
|
||||
| Season | Share of annual | Primary pests | After-hours intensity |
|
||||
|---|---|---|---|
|
||||
| Jan–Feb | 5–8% | Rodents, cockroaches | Low |
|
||||
| Mar–May | 25–30% | Termites, ants, early wasps | Rising fast |
|
||||
| Jun–Aug | 35–40% | Wasps, mosquitoes, roaches, bed bugs | Highest |
|
||||
| Sep–Nov | 20–25% | Rodents, stinging insects, spiders | Moderate-high |
|
||||
| Dec | 5–7% | Rodents | Low-moderate |
|
||||
|
||||
25% of pest control sales calls happen after hours and on weekends (Slingshot data). Home services overall miss 27% of inbound calls (Invoca).
|
||||
|
||||
After-hours call timing patterns:
|
||||
- Weekday evenings 5–10 PM (homeowners return, discover evidence)
|
||||
- Saturday mornings (yard work, outdoor cleaning)
|
||||
- Sunday evenings (pre-work-week anxiety)
|
||||
- 10 PM–2 AM (highest urgency: bed bugs, rodent noise, roach sightings)
|
||||
- Monday mornings surge (weekend discoverers who couldn't reach anyone)
|
||||
|
||||
Four biology-driven after-hours spike events:
|
||||
1. Formosan subterranean termite swarms: 8–11 PM, Gulf Coast states, April–June
|
||||
2. Summer weekend stinging insect encounters: July–September
|
||||
3. First freeze rodent migration: within 24–48 hours of first cold snap, Sep–Nov
|
||||
4. Post-holiday bed bug discoveries: Nov–December
|
||||
|
||||
## Financial Case for After-Hours Automation
|
||||
|
||||
A company missing 15 after-hours calls/month at $150–$250 average service value loses $1,500–$3,000/month in immediate revenue ($18,000–$36,000/year). During peak season (June–August), when volume doubles, losses reach $4,000–$6,000/month.
|
||||
|
||||
True cost understated: each lost new customer represents $2,500–$3,000 LTV. Each lost customer eliminates 2–3 referrals. Each missed call wastes the $200–$350 CAC already spent (Google Ads, SEO, truck wraps, direct mail).
|
||||
|
||||
9× conversion lift: leads responded to within 5 minutes are 9× more likely to convert than those waiting 30 minutes.
|
||||
|
||||
## Three Selling Points for This Market
|
||||
|
||||
1. **Timing matters:** Sell January–February (before spring rush) or August–September (before fall rodent season) — windows when operators have capacity to evaluate tools and are acutely aware of coming demand
|
||||
2. **Frame in LTV, not per-call revenue:** a missed after-hours call doesn't cost $200 — it costs $2,500+ in LTV plus the $200+ CAC already spent
|
||||
3. **Demonstrate pest-specific intelligence:** the #1 complaint about existing answering services is that agents "don't know the first thing about pest control." A solution that distinguishes a termite swarm from carpenter ants, or schedules wasp removal vs. routine quarterly treatment, solves exactly what answering services have failed to deliver for decades
|
||||
|
|
@ -0,0 +1,97 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-13"
|
||||
tags: [research, pest-control, pest-control-spring-2026, niche-automation-prospecting, market-data, unit-economics, source-gemini]
|
||||
---
|
||||
|
||||
# Pest Control Enterprise Revenue Architecture and Seasonality
|
||||
|
||||
Strategic analysis of the $1M–$5M pest control enterprise: revenue architecture, ACV tiering, communication failure patterns, climatologically-driven lead seasonality, operational unit economics, and M&A context. Source: Gemini deep research.
|
||||
|
||||
## Revenue Architecture
|
||||
|
||||
85.2% of residential revenue is recurring — the core financial thesis. Recurring revenue shields the business from volatility and determines M&A valuation.
|
||||
|
||||
ACV tiers:
|
||||
| Program | Frequency | Price/visit | ACV |
|
||||
|---|---|---|---|
|
||||
| Monthly maintenance | 12/year | $40–$70 | $480–$840 |
|
||||
| Bi-monthly | 6/year | $50–$85 | $300–$510 |
|
||||
| Tri-annual | 3/year | $100–$300 | $300–$900 |
|
||||
| Quarterly | 4/year | $45–$75 | $180–$300 |
|
||||
| Premium/Gold bundled | variable | $58–$119/mo | $696–$1,428 |
|
||||
| Termite warranty renewal | annual | $150–$400 | $150–$400 |
|
||||
|
||||
Initial service fee: 3–4× monthly rate ($150–$300). Designed to offset CAC and labor-intensive "flush-out" treatment. Inflates first-year ACV vs. renewal years — must be accounted for in churn and LTV modeling.
|
||||
|
||||
Termite warranty renewals: 70%+ net margin after initial installation. High-value financial floor for mid-market companies.
|
||||
|
||||
Specialty upsells: termite monitoring/baiting $500–$1,200 initial; mosquito seasonal $350–$1,000; rodent exclusion $250–$1,200; wasp/hornet removal $50–$200; bed bug remediation $1,000–$4,000+.
|
||||
|
||||
## Communication Failure Patterns
|
||||
|
||||
Four recurring complaint types from review analysis of $1M–$5M operators:
|
||||
|
||||
1. **"No one answered"** — most prevalent during peak-season transition. In markets where CPL reaches $400, every missed call is a catastrophic marketing capital waste.
|
||||
2. **"Voicemail full"** — perceived as sign the company is overwhelmed/unprofessional. Psychological dead-end for callers; they immediately move to the next listing.
|
||||
3. **"Dropped calls and ghosting"** — supervisor callback promised and never delivered. Common in mid-sized firms without a dedicated dispatch layer.
|
||||
4. **Offshore agent disconnect** — agents who can't deviate from scripts, misidentify pests (e.g., carpenter ants vs. termites), and can't resolve billing issues. Specific to $2M–$5M firms that outsource customer service to cut margins.
|
||||
|
||||
Churn economics: 15–20% annual churn is standard. Reducing to 10% through better communication = $15,000 revenue advantage for every 100–150 customers. Alternatively: 5% churn reduction = 15% marketing efficiency gain.
|
||||
|
||||
## Climatologically-Driven Lead Seasonality
|
||||
|
||||
Pest control demand is triggered by specific biological and meteorological events — not calendar quarters.
|
||||
|
||||
**Spring: Termite Swarming**
|
||||
- Most significant lead-generation event of the year. Subterranean termites swarm on warm days with calm winds following soaking rainfall.
|
||||
- Humidity trigger: wet/warm March → massive simultaneous lead spike across region
|
||||
- Search volume spikes: termite-related terms up 215% from February to May
|
||||
- Growing Degree Days (GDD) used by operators to predict emergence; marketing teams increase PPC spend when GDD thresholds approach
|
||||
|
||||
**Summer: Heatwave-Driven Migrations**
|
||||
- First heatwave (3+ consecutive days >90°F) triggers: ant indoor migration for water/cool-microclimate; yellowjacket/hornet colony aggression; mosquito population explosion
|
||||
- After-hours call volume peaks June–August; this is the most expensive window to miss calls
|
||||
|
||||
**Fall: First Freeze Rodent Push**
|
||||
- Night-time temps <40°F → rodents begin intensive search for overwintering sites
|
||||
- Call spikes occur during the freeze AND the thaw that follows (scratching in walls, pantry droppings)
|
||||
- Successful firms launch rodent exclusion campaigns 2–3 weeks before historical first freeze date for their region
|
||||
|
||||
**Regional variation:**
|
||||
| Region | Primary triggers | Peak window | After-hours spikes |
|
||||
|---|---|---|---|
|
||||
| Northeast | Termites, rodents, ticks | Apr–Jun; Oct–Nov | First 80°F day; first freeze |
|
||||
| Mid-Atlantic | Subterranean termites, rats | Mar–Jul | Post-rain warm days; heatwaves |
|
||||
| Southeast | Mosquitoes, fire ants, Formosans | Year-round; spring peak | High humidity nights |
|
||||
| Southwest | Scorpions, desert termites | Spring and fall | Extreme heat >100°F migrations |
|
||||
| Northwest | Carpenter ants, moisture pests | Summer and fall | Sustained rain patterns |
|
||||
|
||||
## Operational Unit Economics
|
||||
|
||||
Revenue per technician: $250,000–$300,000/year when supported by dense routes and automated scheduling. Stops per day: 10–15 residential. Each additional job on an existing route shift: +8–10% contribution margin (fixed costs already covered).
|
||||
|
||||
COGS structure (% of revenue):
|
||||
- Direct labor: 25.8%
|
||||
- Materials/chemicals: 7.8% (chemicals <10% of revenue — unusually low for scaling)
|
||||
- Marketing/lead gen: 6.6–12%
|
||||
- Vehicle ops: 5–8%
|
||||
- Admin/management: 15–20%
|
||||
- Target EBITDA: 15–25%
|
||||
|
||||
Marketing spend as % of revenue decreases from ~12% to ~8% as companies grow from $1M to $5M — referral density and organic reputation take over from paid CPL.
|
||||
|
||||
Marketing allocation model: 60% PPC (immediate capture during peak triggers), 30% SEO (long-term CPL reduction), 10% retention (referral bonuses, off-season re-engagement).
|
||||
|
||||
## Commercial Pivot
|
||||
|
||||
Commercial = 30% of U.S. market. Commercial ACV: restaurant/office $1,200–$6,000/year; large industrial $15,000–$75,000. Non-negotiable for many clients (health codes, regulatory compliance) → multi-year contracts, low churn. Commercial work often off-hours, maximizing fleet utilization across a 24-hour cycle.
|
||||
|
||||
## M&A Context
|
||||
|
||||
Industry consolidating — "Big Four" (Rollins, Rentokil, Ecolab, Terminix) control ~40% of market. Mid-market operators have lucrative exit potential:
|
||||
- Revenue multiples: 0.84–1.1×
|
||||
- SDE multiples: 2.16–2.6×
|
||||
- Premium drivers: 40%+ recurring revenue, professional management layer (owner not "the business"), documented SOPs — these add 1–2 turns to the multiple
|
||||
|
||||
Acquirers are paying for route density and customer retention, not chemicals or equipment.
|
||||
|
|
@ -0,0 +1,78 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-13"
|
||||
tags: [research, pest-control, pest-control-spring-2026, niche-automation-prospecting, market-data, lead-response, field-service, source-gemini]
|
||||
---
|
||||
|
||||
# Velocity of Response Lead Dynamics Urgent Home Services
|
||||
|
||||
Synthesis of 2023–2026 data on speed-to-lead benchmarks, consumer psychology, economic consequences of friction, and structural barriers to responsiveness across HVAC, plumbing, pest control, and locksmith verticals. Source: Gemini deep research.
|
||||
|
||||
## The Platinum Minute and Golden Window
|
||||
|
||||
Responding within 60 seconds: +391% conversion rate — the "Platinum Minute." By 2026, the 5-minute window is considered the absolute maximum before significant lead leakage; 21× more likely to qualify at <5 min vs 30 min; 100× more likely to connect.
|
||||
|
||||
| Response time | Impact |
|
||||
|---|---|
|
||||
| <1 minute | +391% conversion |
|
||||
| <5 minutes | 21× more likely to qualify |
|
||||
| <5 minutes | 100× more likely to connect |
|
||||
| 5 vs 10 min | 80% drop in qualification odds |
|
||||
| <1 hour | 7× more likely to qualify vs >1 hour |
|
||||
| >24 hours | 60× less likely to qualify |
|
||||
|
||||
63% of companies never respond to inquiries at all. Average B2B lead response time: 42–47 hours.
|
||||
|
||||
## First-Responder Dynamics
|
||||
|
||||
78% of customers buy from the business that responds first. 50%+ hire the first company to respond even if more expensive than competitors.
|
||||
|
||||
Locksmith-specific: 78% of local mobile searches result in a purchase within 24 hours. Locksmith customers rarely wait more than 15 minutes before moving to the next provider.
|
||||
|
||||
The search ends only at a "confirmed next step" — scheduled arrival time or verbal commitment from a technician. Without it, the homeowner continues feeling the unresolved problem and keeps calling.
|
||||
|
||||
## After-Hours Volume and Intent
|
||||
|
||||
41% of online bookings arrive after business hours; significant demand 1 AM–4 AM. Triage instructions are especially critical here — they restore a sense of agency to the homeowner in distress, pause the search behavior, and build trust.
|
||||
|
||||
Key pattern: homeowners in crisis trust the first competent voice, regardless of price or years in business.
|
||||
|
||||
## Economic Benchmarks
|
||||
|
||||
| Industry | Avg CPL | Avg CAC | Avg ticket / LTV |
|
||||
|---|---|---|---|
|
||||
| HVAC | $153 | $296–$350 | $350–$600 ticket / $15,340 LTV |
|
||||
| Plumbing | $167 | $300–$450 | $2,208 avg ticket |
|
||||
| Pest control | $20–$70 (LSA) / $150–$250 (blended) | $300–$500 | $1,200–$3,000 LTV |
|
||||
| Locksmith | $66 | $150–$200 | $150–$300 emergency / $50 routine |
|
||||
|
||||
CAC has risen 222% over the last 8 years. Small businesses lose avg $126,000/year to unanswered calls. Cost of a missed home services call: $100–$1,200 depending on trade.
|
||||
|
||||
Lost Opportunity Value formula: N_missed × % viable leads (56%) × reach rate (70–90%) × conversion rate (35–60%) × avg job value × margin.
|
||||
|
||||
Using this framework: a business missing 42 calls/month with $1,200 avg job value is potentially walking away from $30,000+/month in revenue.
|
||||
|
||||
## Pest Control Notes
|
||||
|
||||
85.2% of residential revenue from ongoing contracts. Peak demand April–September; CPC increases 40–60% during peak. Failing to respond in minutes during peak wastes the most expensive, highest-intent traffic of the year.
|
||||
|
||||
## AI vs. Human Response Infrastructure
|
||||
|
||||
62.5% of AI-using companies meet <15-min response standard vs 39.1% manual.
|
||||
|
||||
| Metric | Traditional | AI-integrated |
|
||||
|---|---|---|
|
||||
| Call answer rate | ~38% | >99% |
|
||||
| Response time | 2–3 min | <60 sec |
|
||||
| Monthly operating cost | $500–$2,000+ | $29–$199 |
|
||||
| Customer retention | baseline | +24% improvement |
|
||||
| Lead leakage | 69.1% | ~54% |
|
||||
|
||||
AI platform cost: $0.50–$5 per interaction vs $5–$25 for traditional answering services.
|
||||
|
||||
## Core Conclusions
|
||||
|
||||
1. Eliminate the five-minute barrier — in this market it is now a point of catastrophic failure; elite standard is <60 seconds
|
||||
2. Institutionalize triage — every initial contact provides an immediate stabilizing task to reduce cognitive load and increase booking stickiness
|
||||
3. Audit after-hours leads as revenue protection, not operational expense — 62% of calls go unanswered; 85% of those callers won't try again
|
||||
4. Focus on unit economics over lead volume — a 5% boost in retention can increase profits by 25–95%
|
||||
|
|
@ -0,0 +1,118 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-14"
|
||||
tags: [research, methodology, cold-email, ab-test, copy-testing, experiment-design]
|
||||
---
|
||||
|
||||
# Blind Email A/B/C Test — Experiment Methodology and Replication Guide
|
||||
|
||||
## Purpose
|
||||
|
||||
Tests which of three email versions performs best with actual buyers — and whether business-consultant gatekeepers can reliably predict buyer preference. Blind comparison removes version-label bias: judges rank emails without knowing which version they're reading or who wrote them. This isolates copy quality from author credibility.
|
||||
|
||||
The experiment also benchmarks gatekeeper accuracy, which determines whether consultants/advisors can serve as a proxy for buyer panels in future tests where finding real owner-operators is expensive or slow.
|
||||
|
||||
## What Was Compared
|
||||
|
||||
Three email versions written for the pest-control-spring-2026 campaign, all targeting the same pain (after-hours lead loss):
|
||||
|
||||
- **Draft** — Golden 4-Sentence Framework structure: pain hook, stat, implication, single question close
|
||||
- **Revised** — Rubric-critique rewrite of draft: tightened framing, removed escape hatches, strengthened closes
|
||||
- **Copywriter** — 10 distinct copy strategies, one per group (scenario contrast, empathetic review framing, one-liner pattern interrupt, disarming self-aware opener, etc.)
|
||||
|
||||
The 30 total emails were organized into 10 positionally matched groups (group 1 = all three versions on the same hook/angle, group 2 = all three on a different hook, etc.). Each group contained exactly one draft, one revised, and one copywriter version covering the same underlying angle — with some exceptions noted under Limitations.
|
||||
|
||||
## Two-Panel Design
|
||||
|
||||
### Gatekeepers (5 judges)
|
||||
|
||||
Business consultants and advisors acting as proxies for buyers. Each was given the gatekeeper ballot — 10 groups × 3 emails — and asked to rank each group 1st/2nd/3rd as if deciding which email would most likely get a response from a pest control owner-operator. Persona framing was provided: they were told to evaluate from the perspective of a skeptical, time-pressed SMB owner who receives cold email frequently. Judges did not know about the owner panel, the version labels (draft/revised/copywriter), or each other's responses.
|
||||
|
||||
### Owners (3 judges)
|
||||
|
||||
Actual pest control owner-operators, each with differentiated profiles:
|
||||
|
||||
- **Mike Deluca** — 22-year owner-operator, Ohio, 12 trucks, GorillaDesk user, tech-skeptical
|
||||
- **Sandra Kowalski** — 6-year owner-operator, Texas, 8 trucks (growth-mode operator)
|
||||
- **Ray Tanner** — 14-year owner-operator, Georgia, 7 trucks
|
||||
|
||||
Each was given the owner ballot — independently scrambled from the gatekeeper ballot — and asked to rank each group as if the emails had arrived in their actual inbox. Owners did not know about the gatekeeper panel, the version labels, or each other.
|
||||
|
||||
**Blind guarantee:** Neither panel knew about the experiment structure, the other panel, the version labels, or each other. Each received a ballot with UUID-identified emails only — no author or version metadata.
|
||||
|
||||
## Ballot Construction
|
||||
|
||||
The `build_ballots.py` script produced four output files from hardcoded email data:
|
||||
|
||||
1. **ballot-data.json** — raw structured data (not blind; version labels intact) for reference
|
||||
2. **gatekeeper-ballot.json** — scrambled ballot for the gatekeeper panel
|
||||
3. **owner-ballot.json** — independently scrambled ballot for the owner panel
|
||||
4. **ballot-mapping.json** — unscrambling key mapping every UUID back to `{group_num, version}`
|
||||
|
||||
**UUID scheme:**
|
||||
- Gatekeeper ballot IDs: `gk-{uuid4}` (e.g., `gk-9b5ca2c2-cea3-44a3-8fd7-0d0fcdaa2df7`)
|
||||
- Owner ballot IDs: `ow-{uuid4}` (e.g., `ow-47962807-35bf-4c70-bbe5-9c0cb2d5327d`)
|
||||
- The prefix makes it immediately clear which ballot an ID belongs to without consulting the mapping file
|
||||
|
||||
**Scramble logic:** For each group, the three versions were independently shuffled using Python's `random.shuffle()` with `random.seed()` (true random, no fixed seed). Gatekeeper and owner ballots received separate shuffles — even within the same group, the three emails appear in a different order on each ballot. This prevents cross-contamination if a judge somehow saw both ballots.
|
||||
|
||||
**Mapping file structure:** A flat JSON object keyed by UUID. Each value is `{"group_num": N, "version": "draft|revised|copywriter"}`. Used post-scoring to decode which version received which rank.
|
||||
|
||||
## Scoring System
|
||||
|
||||
Ranked-choice scoring within each group:
|
||||
- 1st place = 3 points
|
||||
- 2nd place = 2 points
|
||||
- 3rd place = 1 point
|
||||
|
||||
Scores were aggregated within each panel (gatekeeper total = sum of all 5 judges × 10 groups = max 150 points per version; owner total = sum of all 3 judges × 10 groups = max 90 points). A combined 8-judge score was also computed for overall version ranking.
|
||||
|
||||
Tie-breaking: when two versions tied on weighted score within a group, 1st-place vote count was used as tiebreaker.
|
||||
|
||||
## Gatekeeper Accuracy Metric
|
||||
|
||||
For each of the 10 groups, the majority gatekeeper pick (version with highest gatekeeper weighted score) was compared against the majority owner pick. Agreement = 1, disagreement = 0.
|
||||
|
||||
**Result: 60% accuracy (6/10 groups).**
|
||||
|
||||
**Systematic bias pattern observed:** Gatekeepers consistently over-indexed on research signals and analytical depth relative to what owners actually responded to. In groups 8 and 10, gatekeepers chose versions that demonstrated more business knowledge; owners in both cases chose shorter, less analytical formats. Gatekeepers also underweighted emotional recognition — the instant "that's my life" reaction that drove owner behavior in groups 7 and 8. A secondary bias: gatekeepers were more tolerant of competitive claims and slightly underestimated how much growth-mode operators respond to market-position arguments (group 6).
|
||||
|
||||
## Files Generated
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `build_ballots.py` | Ballot construction script — generates all 4 JSON artifacts |
|
||||
| `ballot-data.json` | Raw email data with version labels (not blind) |
|
||||
| `gatekeeper-ballot.json` | Scrambled ballot delivered to gatekeeper panel |
|
||||
| `owner-ballot.json` | Independently scrambled ballot delivered to owner panel |
|
||||
| `ballot-mapping.json` | UUID → {group_num, version} decoding key |
|
||||
| `gatekeeper-1-results.json` through `gatekeeper-5-results.json` | Individual gatekeeper rankings with rationale |
|
||||
| `gatekeeper-aggregate.json` | Aggregated gatekeeper scores per version per group |
|
||||
| `gatekeeper-aggregate-summary.md` | Narrative summary of gatekeeper results, patterns, themes |
|
||||
| `owner-1-results.json` through `owner-3-results.json` | Individual owner rankings with rationale |
|
||||
| `owner-1-elaboration.md` through `owner-3-elaboration.md` | Extended owner commentary per group |
|
||||
| `owner-aggregate.json` | Aggregated owner scores per version per group |
|
||||
| `owner-aggregate-summary.md` | Narrative summary of owner results |
|
||||
| `final-analysis.md` | Combined analysis: gatekeeper accuracy, version performance, consensus wins, divergence breakdown |
|
||||
|
||||
## Replication Notes
|
||||
|
||||
**Adapting persona specs for a different buyer profile:** Replace the three owner personas with profiles representative of the target niche. Key differentiators to specify: years in business, company size (employees or trucks), software stack (signals tech sophistication), and operator mode (growth vs. stability). The more differentiated the three personas, the more useful the disagreements between them are.
|
||||
|
||||
**Minimum judge count:** 5 gatekeepers is adequate for detecting systematic bias; 3 owners is the minimum viable panel. With only 3 owners, a single outlier can swing a group result — 5 owners would be more reliable. Consider 3 owners + 2 "challenger" owners with atypical profiles (very small, very large, different geography) to stress-test findings.
|
||||
|
||||
**Positional mismatch:** If the three versions don't cover the same angle within a group, the comparison becomes angle vs. angle rather than execution vs. execution. Audit for this before running the experiment: each group should share a core hook. Mismatched groups can still be scored but results are less actionable — note them explicitly in the analysis.
|
||||
|
||||
**Extending to more than 3 versions:** The ranked-choice system extends naturally. With 4 versions, scoring becomes 1st=4pts, 2nd=3pts, 3rd=2pts, 4th=1pt. With more than 5 versions per group, judge fatigue becomes a real concern — consider splitting into elimination rounds (top 3 per preliminary group, then finals) rather than asking for full rankings across 6+ options.
|
||||
|
||||
**Script reuse:** `build_ballots.py` is self-contained. Swap in new email data in the `EMAIL_DATA` dict, adjust group count, and rerun. The UUID prefix convention (`gk-` / `ow-`) and independent shuffle logic should be preserved for all future runs.
|
||||
|
||||
## Limitations
|
||||
|
||||
**Positional grouping issue:** Groups 3 and 6 had cross-angle comparisons. In group 3, the draft and revised versions covered weekend call capture (wasp nest Saturday scenario); the copywriter version opened with the 47% after-hours stat, a different hook. In group 6, draft and revised covered competitive first-mover framing; the copywriter version pivoted to 1-star review / communication framing. Judges in these groups were comparing both angle and execution simultaneously. The group 3 revised version winning unanimously may partly reflect angle advantage (concrete scenario) rather than pure execution quality. Group 6 results should be read with the same caveat.
|
||||
|
||||
**Panel size:** 3 owners is the minimum viable panel. Results are directionally useful but a single strong-preference outlier (as seen in groups 4 and 10 with Ray Tanner) can materially shift aggregate scores. Treat per-group owner results as signals, not verdicts.
|
||||
|
||||
**Judge selection bias:** Gatekeepers were business consultants/advisors — likely more analytically oriented than average cold email recipients. This may explain their systematic preference for research-signaling emails. Replicating with a gatekeeper panel of sales practitioners rather than business advisors could produce different accuracy numbers.
|
||||
|
||||
**Single campaign context:** All emails address the same pain (after-hours lead loss) in the same niche (pest control). The gatekeeper accuracy finding (60%) and the systematic biases observed may not generalize to campaigns with different pain categories, different buyer sophistication levels, or different email structures.
|
||||
|
|
@ -0,0 +1,74 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-14"
|
||||
tags: [research, cold-email, pest-control, pest-control-spring-2026, copy-testing, ab-test, hub]
|
||||
---
|
||||
|
||||
# Pest Control Email A/B/C Test — Experiment Hub
|
||||
|
||||
## What This Was
|
||||
|
||||
A blind ranked-choice comparison of 3 cold email drafts — draft (Golden 4-Sentence Framework), revised (rubric-critique rewrite), and copywriter (10 distinct copy strategies) — tested across 10 email groups covering different hooks and angles for the pest-control-spring-2026 campaign. Two panels judged independently: 5 gatekeeper personas (business consultants/advisors) and 3 pest control owner-operators (Mike Deluca / Ohio / 12 trucks, Sandra Kowalski / Texas / 8 trucks, Ray Tanner / Georgia / 7 trucks). Results were compiled 2026-03-14.
|
||||
|
||||
## Key Numbers
|
||||
|
||||
- Emails compared: 30 (10 groups × 3 versions)
|
||||
- Judges: 5 gatekeepers + 3 owner personas = 8 total
|
||||
- Gatekeeper accuracy: 60% (6/10 groups correctly predicted owner preference)
|
||||
- Overall winner: Revised (175 weighted points across all judges — 108 GK + 67 owner)
|
||||
- Runner-up: Copywriter (163 total — 99 GK + 64 owner)
|
||||
- Draft: distant third (142 total — 93 GK + 49 owner)
|
||||
- Unanimous sweeps: 2 — Group 3 Revised (wasp nest Saturday), Group 7 Copywriter (Scenario A vs. B)
|
||||
- Owner unanimous groups: 5 of 10 (groups 2, 3, 5, 7, 8)
|
||||
|
||||
## Executive Findings
|
||||
|
||||
- **Revised wins overall but copywriter leads in raw first-place owner votes (15 vs 13)** — revised's consistency across more groups pushed the weighted score higher; copywriter is high-variance (dominant when it works, last-place when it doesn't).
|
||||
- **Gatekeepers are a useful but incomplete proxy at 60% accuracy** — they reliably catch obvious losers (ROI math, review-shaming, escape hatches) but cannot predict which of two strong versions will resonate with a specific operator profile.
|
||||
- **Core gatekeeper blind spot: they over-index on research signals and analytical depth vs. emotional recognition** — in Group 8, gatekeepers ranked the GorillaDesk/P&L email first; all three owners unanimously chose the one-sentence Saturday text scenario.
|
||||
- **Owners reject fabricated revenue math universally** — the $136,500 ROI projection (Group 5 copywriter) earned zero second-place votes from any owner; no ROI projections belong in Email 1.
|
||||
- **Review-shaming openers trigger defensive shutdown, not curiosity** — any opener framed as "I noticed something wrong with your business" lost to scenario-based openers regardless of how it was softened.
|
||||
- **Two unanimous sweeps emerged** — Group 3 Revised (wasp-nest Saturday scenario + Driven Results citation) swept both panels; Group 7 Copywriter (Scenario A vs. B two-format structure) earned identical positive reactions from all three owner personas across different profiles.
|
||||
- **The LSA-anchor version (Group 4) requires audience segmentation** — strong for paid-acquisition operators (owners 1 and 2), immediately rejected by referral-only operator (owner 3) from sentence one; do not use as default for mixed lists.
|
||||
- **5 emails recommended for live sequence** — covering weekend call capture, speed-to-lead, voicemail reframe, after-hours scenario, and termite-season urgency; together they represent the clearest consensus across both panels.
|
||||
|
||||
## 5 Recommended Emails for Live Sequence
|
||||
|
||||
| Priority | Group | Version | Hook |
|
||||
|---|---|---|---|
|
||||
| 1 | Group 3 | Revised | Saturday wasp nest scenario |
|
||||
| 2 | Group 7 | Copywriter | Scenario A vs. B two-format |
|
||||
| 3 | Group 9 | Revised | "Lead disappearance problem" reframe |
|
||||
| 4 | Group 2 | Copywriter | Bed bugs at 11pm scenario |
|
||||
| 5 | Group 5 | Revised | Termite LTV urgency ("first callback wins") |
|
||||
|
||||
## Structural Patterns to Adopt
|
||||
|
||||
- **Two-scenario contrast format** (Group 7): "Scenario A: [current]. Scenario B: [proposed]. Same [situation]. Completely different outcome." — the only format to earn identical reactions from all 3 owner personas.
|
||||
- **Scenario first, stat second**: open with a specific named operational moment the owner recognizes; follow with a verifiable citation to prove the point.
|
||||
- **Reframe before solution**: "That's not a voicemail problem. It's a lead disappearance problem." creates a named mental category that stops owners mid-read.
|
||||
- **Single closing question with an honest out**: "Have you managed to stay ahead of it?" outperforms closes that imply the problem is certain.
|
||||
|
||||
## Patterns to Retire
|
||||
|
||||
- Fabricated revenue math in Email 1 (no baselines, no LTV multiplication before engagement)
|
||||
- Review-shaming openers ("I noticed your reviews show X")
|
||||
- Escape-hatch closes ("or are you not running paid ads right now?")
|
||||
- Guessed specificity (truck-count assumptions, coverage-area merge tokens)
|
||||
- Self-aware meta-openers ("You probably get emails like this. I'm not doing that.") — experienced operators recognize it as a trope
|
||||
|
||||
## Related Notes
|
||||
|
||||
- [[2026-03-14-pest-control-email-a-b-c-test-per-group-copy-and-scores]] — Full email copy + scores for all 10 groups
|
||||
- [[2026-03-14-what-pest-control-owners-respond-to-and-reject-in-cold-email]] — Owner psychology guide + copy rubric input
|
||||
- [[2026-03-14-blind-email-a-b-c-test-experiment-methodology-and-replication-guide]] — How to replicate this experiment
|
||||
|
||||
## Campaign Files
|
||||
|
||||
Local experiment artifacts (may be archived/deleted after vault notes are saved):
|
||||
`go-to-market/campaigns/pest-control-spring-2026/experiments/email-version-comparison/`
|
||||
|
||||
Email sequence files:
|
||||
- `sequences/v4-golden-framework-draft.md`
|
||||
- `sequences/v4-golden-framework-revised.md`
|
||||
- `sequences/v4-copywriter-draft.md`
|
||||
|
|
@ -0,0 +1,443 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-14"
|
||||
tags: [research, cold-email, pest-control, pest-control-spring-2026, copy-testing, ab-test]
|
||||
---
|
||||
|
||||
# Pest Control Email A/B/C Test — Per-Group Copy and Scores
|
||||
|
||||
Blind ranked-choice comparison of 3 email versions across 10 groups. 8 total judges: 5 gatekeepers (business consultants/advisors) + 3 owner-operators. Versions tested: draft (Golden 4-Sentence Framework), revised (rubric-critique rewrite), copywriter (10 distinct copy strategies). Date: 2026-03-14.
|
||||
|
||||
**Overall result:** Revised wins on total first-place votes (36 vs 34 for copywriter, 9 for draft) and total weighted score (175 vs 163 vs 142). Gatekeeper accuracy vs. owner picks: 6/10 (60%). Draft is the safest "avoid" at scale — received last-place finishes in groups 2, 4, 6, 7, 8 from owners.
|
||||
|
||||
---
|
||||
|
||||
## Group 1 — After-Hours Call Capture, Spring Volume
|
||||
|
||||
| Version | GK 1st-Place | GK Score | Owner 1st-Place | Owner Score | Overall Rank |
|
||||
|---|---|---|---|---|---|
|
||||
| draft | 1 | 8 | 0 | 4 | 3 |
|
||||
| revised | 2 | 10 | 1 | 6 | 2 |
|
||||
| copywriter | 2 | 12 | 2 | 8 | 1 |
|
||||
|
||||
**Winner:** copywriter (GK) / copywriter (Owners) — agree
|
||||
|
||||
**Draft subject:** after-hours calls this spring
|
||||
**Draft body:** {{first_name}}, you're about to hit your busiest stretch of the year — and a big chunk of those calls are landing after 5pm when nobody picks up.
|
||||
|
||||
85% of callers who hit voicemail never call back. (PATLive) They just dial the next company on Yelp.
|
||||
|
||||
Is that something you've been eating, or have you figured out a way to handle it?
|
||||
|
||||
**Revised subject:** after-hours calls this spring
|
||||
**Revised body:** {{first_name}}, spring is your highest-volume stretch — and based on {{company_name}}'s coverage area, a meaningful chunk of those calls are landing after 5pm when the phones go dark.
|
||||
|
||||
85% of callers who hit voicemail never call back. (PATLive) They just dial the next company on Yelp.
|
||||
|
||||
Is that something you've been absorbing, or have you found a way to handle it?
|
||||
|
||||
**Copywriter subject:** how do you handle after-hours calls?
|
||||
**Copywriter body:** {{first_name}}, quick question —
|
||||
|
||||
When someone calls {{company_name}} at 10pm and hits voicemail, what happens next?
|
||||
|
||||
Most pest control owners I ask assume the homeowner leaves a message and waits. The data says something different: 85% of callers who hit voicemail never call back, and 62% call a competitor within minutes.
|
||||
|
||||
Worth knowing whether that's happening to you?
|
||||
|
||||
**Why copywriter won:** Conversational "quick question" framing and absence of fake-personalization tokens was preferred; revised's `{{company_name}}'s coverage area` token was flagged as hollow and signaling fake personalization by gatekeepers and owners alike.
|
||||
|
||||
---
|
||||
|
||||
## Group 2 — 1-Star Reviews / Review-as-Signal
|
||||
|
||||
| Version | GK 1st-Place | GK Score | Owner 1st-Place | Owner Score | Overall Rank |
|
||||
|---|---|---|---|---|---|
|
||||
| draft | 1 | 10 | 0 | 6 | 2 |
|
||||
| revised | 0 | 6 | 0 | 3 | 3 |
|
||||
| copywriter | 4 | 14 | 3 | 9 | 1 |
|
||||
|
||||
**Winner:** copywriter (GK) / copywriter (Owners) — agree
|
||||
|
||||
**Draft subject:** your 1-star reviews
|
||||
**Draft body:** {{first_name}}, I noticed {{company_name}} has a handful of 1-star reviews — and my guess is they're not about a bad treatment.
|
||||
|
||||
37% of 1-star reviews in pest control cite failure to answer or poor communication. (industry research) One missed after-hours call, no callback, and the review writes itself.
|
||||
|
||||
Worth a look at what's actually driving those — or are the reviews not a concern for you?
|
||||
|
||||
**Revised subject:** your 1-star reviews
|
||||
**Revised body:** {{first_name}}, I noticed {{company_name}} has a few 1-star reviews — and I'd bet most of them aren't about the treatment itself.
|
||||
|
||||
37% of 1-star reviews in home services cite communication failure — no callback, no response, no follow-up. (BrightLocal, 2024) One missed after-hours call and the review writes itself.
|
||||
|
||||
Are you seeing that pattern in your reviews, or is it something else driving them?
|
||||
|
||||
**Copywriter subject:** 11pm, bed bugs, yelp open
|
||||
**Copywriter body:** {{first_name}},
|
||||
|
||||
Picture the homeowner who pulls back the sheets at 11pm and finds bed bugs. She's not going to sleep. She's on her phone, she's panicking, and she's calling the first pest control company that comes up on Yelp.
|
||||
|
||||
If that's {{company_name}} and she hits voicemail, she calls the next one.
|
||||
|
||||
That second call takes under 30 seconds. And 78% of customers hire whoever responds first.
|
||||
|
||||
Is there anything catching those calls right now?
|
||||
|
||||
**Why copywriter won:** The bed bug scenario (11pm, panicking homeowner) created visceral recognition; review-shaming openers in draft/revised triggered immediate defensive response in all owners — the angle itself is rejected, not just the phrasing.
|
||||
|
||||
---
|
||||
|
||||
## Group 3 — Weekend Calls Going Unanswered
|
||||
|
||||
| Version | GK 1st-Place | GK Score | Owner 1st-Place | Owner Score | Overall Rank |
|
||||
|---|---|---|---|---|---|
|
||||
| draft | 0 | 10 | 0 | 6 | 2 |
|
||||
| revised | 5 | 15 | 3 | 9 | 1 |
|
||||
| copywriter | 0 | 5 | 0 | 3 | 3 |
|
||||
|
||||
**Winner:** revised (GK) / revised (Owners) — agree (unanimous sweep both panels)
|
||||
|
||||
**Draft subject:** weekend calls sitting until Monday
|
||||
**Draft body:** {{first_name}}, pest problems don't respect weekends — but most pest control companies don't have anyone responding until Monday morning.
|
||||
|
||||
By then, 87% of those leads have already booked a competitor. (Driven Results, 2,847 leads)
|
||||
|
||||
I'm curious — do your weekend calls get any kind of response, or are they just sitting in a voicemail queue until someone checks in?
|
||||
|
||||
**Revised subject:** weekend calls sitting until Monday
|
||||
**Revised body:** {{first_name}}, if a homeowner calls {{company_name}} Saturday evening about a wasp nest in their garage, what actually happens to that call?
|
||||
|
||||
By Monday morning, 87% of those weekend leads have already booked a competitor. (Driven Results, 2,847 leads)
|
||||
|
||||
Do your weekend calls get any kind of live response, or are they sitting until someone checks the queue?
|
||||
|
||||
**Copywriter subject:** 47% of your calls
|
||||
**Copywriter body:** {{first_name}},
|
||||
|
||||
47% of home service calls come in outside business hours — evenings, weekends, holidays.
|
||||
|
||||
For a company running 10–15 trucks, that's a meaningful slice of inbound that hits voicemail or rings out. Most of that doesn't come back.
|
||||
|
||||
I'm curious how {{company_name}} is handling that gap. Want me to send a short breakdown of what a text-back system looks like in practice?
|
||||
|
||||
**Why revised won:** "Saturday evening about a wasp nest in their garage" — specific pest, specific timing, specific scenario. Copywriter's truck-count guess ("10–15 trucks") was actively penalized by all five gatekeepers and all three owners.
|
||||
|
||||
---
|
||||
|
||||
## Group 4 — Ad Spend / Google LSA Lead Waste
|
||||
|
||||
| Version | GK 1st-Place | GK Score | Owner 1st-Place | Owner Score | Overall Rank |
|
||||
|---|---|---|---|---|---|
|
||||
| draft | 0 | 6 | 0 | 3 | 3 |
|
||||
| revised | 2 | 12 | 2 | 8 | 1 |
|
||||
| copywriter | 3 | 12 | 1 | 7 | 1 (GK tie) |
|
||||
|
||||
**Winner:** copywriter (GK, tie broken by 1st-place votes) / revised (Owners) — disagree
|
||||
|
||||
**Draft subject:** what happens to your ad spend after hours
|
||||
**Draft body:** {{first_name}}, if you're running Google Local Services ads or paying for any kind of lead gen, you're spending money to drive calls — and a portion of those calls are hitting voicemail when the office is closed.
|
||||
|
||||
62% of unanswered callers immediately dial a competitor. (Angi) The ad spend is gone either way.
|
||||
|
||||
Is this something that's shown up on your radar, or are you not running paid ads right now?
|
||||
|
||||
**Revised subject:** what happens to your ad spend after hours
|
||||
**Revised body:** {{first_name}}, if {{company_name}} is running Google Local Services ads, you're paying for calls — including the ones that hit voicemail at 7pm when the office is closed.
|
||||
|
||||
62% of unanswered callers immediately dial a competitor. (Angi) The ad spend is gone either way.
|
||||
|
||||
Is that something you've noticed, or are you not running paid lead gen right now?
|
||||
|
||||
**Copywriter subject:** leads from friday night
|
||||
**Copywriter body:** {{first_name}},
|
||||
|
||||
Pest calls that come in Friday evening don't get answered until Monday morning. By Monday, 87% of those homeowners have already booked someone else.
|
||||
|
||||
That's not a follow-up problem. That's a response problem — the window closes in hours, not days.
|
||||
|
||||
Does {{company_name}} have any coverage on weekend evenings, or does that go straight to voicemail?
|
||||
|
||||
**Why revised won (owners):** Owners running paid ads strongly preferred revised's Google Local Services anchor. Owner 3 (referral-only) rejected it entirely from sentence one — this email should be segmented to paid-acquisition operators only.
|
||||
|
||||
---
|
||||
|
||||
## Group 5 — Termite Call Urgency + LTV
|
||||
|
||||
| Version | GK 1st-Place | GK Score | Owner 1st-Place | Owner Score | Overall Rank |
|
||||
|---|---|---|---|---|---|
|
||||
| draft | 1 | 11 | 0 | 6 | 2 |
|
||||
| revised | 4 | 14 | 3 | 9 | 1 |
|
||||
| copywriter | 0 | 5 | 0 | 3 | 3 |
|
||||
|
||||
**Winner:** revised (GK) / revised (Owners) — agree (unanimous owner sweep)
|
||||
|
||||
**Draft subject:** termite calls you're not catching
|
||||
**Draft body:** {{first_name}}, termite season is starting to ramp up — and homeowners who think they've got an infestation aren't patient callers. They want someone on the phone now.
|
||||
|
||||
78% of customers hire whoever responds first, and more than half will pay more to get the company that picked up first. (industry data)
|
||||
|
||||
Are you catching most of those calls, or are some slipping through after hours?
|
||||
|
||||
**Revised subject:** termite calls you're not catching
|
||||
**Revised body:** {{first_name}}, a homeowner who thinks they have termites isn't a patient caller. They want confirmation today — and if {{company_name}} doesn't answer, they're calling the next company on the list.
|
||||
|
||||
78% of customers in home services hire whoever responds first. (Driven Results) Termite jobs are high-LTV, low-hesitation — the first callback wins.
|
||||
|
||||
Are your after-hours termite inquiries getting any kind of response, or are they going to voicemail?
|
||||
|
||||
**Copywriter subject:** marketing spend going to voicemail
|
||||
**Copywriter body:** {{first_name}},
|
||||
|
||||
Every Google ad, every Yelp listing, every door hanger that drives an after-hours call — if it hits voicemail, the acquisition cost is sunk with zero return.
|
||||
|
||||
5 missed after-hours calls per week adds up to roughly $136,500 a year in lost revenue at standard pest control LTV.
|
||||
|
||||
I'm not saying that's your number. But if you're spending on lead gen and a chunk of it is going to voicemail after 6pm, that's worth a look.
|
||||
|
||||
Want me to walk through how the math usually lands for a company your size?
|
||||
|
||||
**Why revised won:** "Termite jobs are high-LTV, low-hesitation — the first callback wins" was rated sharper industry insight than generic response-time stats. Copywriter's $136,500 ROI math earned zero second-place votes from any owner — all five gatekeepers also last-placed it.
|
||||
|
||||
---
|
||||
|
||||
## Group 6 — Competitive After-Hours Positioning
|
||||
|
||||
| Version | GK 1st-Place | GK Score | Owner 1st-Place | Owner Score | Overall Rank |
|
||||
|---|---|---|---|---|---|
|
||||
| draft | 0 | 6 | 0 | 5 | 3 |
|
||||
| revised | 2 | 12 | 2 | 8 | 1 |
|
||||
| copywriter | 3 | 12 | 1 | 5 | 1 (GK tie) |
|
||||
|
||||
**Winner:** copywriter (GK, tie broken by 1st-place votes) / revised (Owners) — disagree
|
||||
|
||||
**Draft subject:** what your competitors are doing after hours
|
||||
**Draft body:** {{first_name}}, the pest control company down the road from you is probably not answering after-hours calls either — but if one of them installs an instant text response, they start winning every lead that comes in after 5pm.
|
||||
|
||||
47% of home service calls come in outside business hours. (HomeAdvisor) That's not a small slice.
|
||||
|
||||
Just curious — is after-hours lead capture something you've thought about, or not on the list yet?
|
||||
|
||||
**Revised subject:** what your competitors are doing after hours
|
||||
**Revised body:** {{first_name}}, right now none of the pest control companies in your market are answering after-hours calls. That's a shared vulnerability — but the first one to fix it captures the entire pool.
|
||||
|
||||
47% of home service calls come in after business hours. (HomeAdvisor) That's nearly half your potential volume going to whoever responds.
|
||||
|
||||
Is after-hours lead capture something {{company_name}} has looked at, or not a priority yet?
|
||||
|
||||
**Copywriter subject:** 1-star reviews and communication
|
||||
**Copywriter body:** {{first_name}},
|
||||
|
||||
37% of 1-star pest control reviews aren't about treatment quality — they're about communication. Slow to respond, hard to reach, no one followed up.
|
||||
|
||||
For a company that does good work, that's a hard pill.
|
||||
|
||||
Most of those reviews come from the gap between when someone tried to reach you and when they actually heard back. Worth looking at?
|
||||
|
||||
**Why revised won (owners):** Growth-mode owners strongly preferred the competitive first-mover framing ("the first one to fix it captures the entire pool"). Gatekeepers underestimated how much competitive-threat angles resonate with growth-oriented operators.
|
||||
|
||||
---
|
||||
|
||||
## Group 7 — Speed-to-Lead / 60-Second Response
|
||||
|
||||
| Version | GK 1st-Place | GK Score | Owner 1st-Place | Owner Score | Overall Rank |
|
||||
|---|---|---|---|---|---|
|
||||
| draft | 1 | 11 | 0 | 3 | 2 |
|
||||
| revised | 0 | 6 | 0 | 6 | 3 |
|
||||
| copywriter | 4 | 13 | 3 | 9 | 1 |
|
||||
|
||||
**Winner:** copywriter (GK) / copywriter (Owners) — agree (unanimous owner sweep)
|
||||
|
||||
**Draft subject:** how fast are you calling leads back
|
||||
**Draft body:** {{first_name}}, when a homeowner submits a pest inquiry at 8pm, what happens on your end?
|
||||
|
||||
Conversion rate at 60 seconds is 47%. After 30 minutes, it drops to 4%. (Driven Results, 2,847 contractor leads) By morning, you're competing against three companies who already called them back.
|
||||
|
||||
Is speed-to-lead something you've looked at for {{company_name}}, or is it more of a staffing constraint?
|
||||
|
||||
**Revised subject:** how fast are you calling leads back
|
||||
**Revised body:** {{first_name}}, when a homeowner submits a pest inquiry at 8pm, what actually happens on your end?
|
||||
|
||||
Conversion rate at 60 seconds is 47%. After 30 minutes, it drops to 4%. (Driven Results, 2,847 contractor leads) By morning, you're competing against companies who already called them back.
|
||||
|
||||
Is speed-to-lead something {{company_name}} has looked at, or is it not on the radar yet?
|
||||
|
||||
**Copywriter subject:** voicemail vs. a text back
|
||||
**Copywriter body:** {{first_name}},
|
||||
|
||||
Two scenarios, same missed call:
|
||||
|
||||
A homeowner calls at 9pm about rodents in the walls. Scenario A: voicemail. Fewer than 3% of callers leave one. Scenario B: they get a text back in under 60 seconds acknowledging the call, asking one qualifying question, and confirming someone will reach out in the morning.
|
||||
|
||||
Same call. Completely different outcome.
|
||||
|
||||
Is that something {{company_name}} has looked at building?
|
||||
|
||||
**Why copywriter won:** The Scenario A/B structure shows the alternative instead of naming the problem. The one email all three owners described with identical positive reactions — "the first email in this whole batch that describes something real," "actually answers the 'so what' before I have to ask," "doesn't lecture me — it just shows me the difference." Revised (speed-to-lead stat without the "three companies" specificity) was last-placed by all five gatekeepers.
|
||||
|
||||
---
|
||||
|
||||
## Group 8 — Invisible Leads / P&L Visibility
|
||||
|
||||
| Version | GK 1st-Place | GK Score | Owner 1st-Place | Owner Score | Overall Rank |
|
||||
|---|---|---|---|---|---|
|
||||
| draft | 3 | 12 | 0 | 4 | 1 (GK) / 3 (Owner) |
|
||||
| revised | 0 | 6 | 0 | 5 | 3 (GK) / 2 (Owner) |
|
||||
| copywriter | 2 | 12 | 3 | 9 | 1 (GK tie) / 1 (Owner) |
|
||||
|
||||
**Winner:** draft (GK, tie broken by 1st-place votes) / copywriter (Owners) — disagree (sharpest miss of the study)
|
||||
|
||||
**Draft subject:** leads you're not seeing on the P&L
|
||||
**Draft body:** {{first_name}}, the leads you lose after hours don't show up anywhere — no record in GorillaDesk, no entry in QuickBooks, just a homeowner who called someone else.
|
||||
|
||||
5 missed after-hours calls a week is 260 leads a year — and at a pest control LTV of $1,500+, that's a real number that's not visible until you look for it. (PPMA retention data)
|
||||
|
||||
Is that math close for {{company_name}}, or is your after-hours volume lower than that?
|
||||
|
||||
**Revised subject:** leads you're not seeing on the P&L
|
||||
**Revised body:** {{first_name}}, the leads you lose after hours don't show up anywhere — no record in your FSM, no entry in QuickBooks, just a homeowner who called someone else.
|
||||
|
||||
5 missed after-hours calls a week is 260 leads a year. At a pest control LTV of $1,500+, that's real money that's invisible until you go looking. (PPMA retention data)
|
||||
|
||||
Is that math close for {{company_name}}, or is your after-hours volume lower than that?
|
||||
|
||||
**Copywriter subject:** quick question
|
||||
**Copywriter body:** {{first_name}} —
|
||||
|
||||
Are you the kind of operation where someone occasionally texts you on a Saturday saying "we missed a call last night — any idea who that was?"
|
||||
|
||||
**Why copywriter won (owners):** Emotional recognition beat analytical depth across all three owners unanimously. The Saturday-text one-liner demonstrates that the sender knows what it feels like to run the operation — owners reacted to "this describes a moment I recognize." Gatekeepers screened for research-signaling depth (GorillaDesk name-drop); owners did not reward it.
|
||||
|
||||
---
|
||||
|
||||
## Group 9 — Voicemail Behavior / Lead Disappearance
|
||||
|
||||
| Version | GK 1st-Place | GK Score | Owner 1st-Place | Owner Score | Overall Rank |
|
||||
|---|---|---|---|---|---|
|
||||
| draft | 0 | 8 | 0 | 5 | 2 |
|
||||
| revised | 5 | 15 | 2 | 7 | 1 |
|
||||
| copywriter | 0 | 7 | 1 | 6 | 3 |
|
||||
|
||||
**Winner:** revised (GK) / revised (Owners) — agree (unanimous gatekeeper sweep)
|
||||
|
||||
**Draft subject:** why homeowners text instead of leave voicemails
|
||||
**Draft body:** {{first_name}}, fewer than 3% of callers leave a voicemail when they don't get a live person. (Invoca) The rest hang up and move on.
|
||||
|
||||
But 69% of consumers say they'd prefer a business they've never heard of to text them rather than call. (Avochato, 1,000 adults) That's a pretty direct signal about what homeowners actually want.
|
||||
|
||||
Are your after-hours callers getting any kind of text response right now, or is it just voicemail?
|
||||
|
||||
**Revised subject:** why homeowners don't leave voicemails
|
||||
**Revised body:** {{first_name}}, here's a number that's worth sitting with: fewer than 3% of home service callers leave a voicemail when they don't get a live answer. (Invoca) The other 97% hang up and move on.
|
||||
|
||||
That's not a voicemail problem. It's a lead disappearance problem — and most pest control owners have no visibility into how many calls {{company_name}} is losing this way.
|
||||
|
||||
How are after-hours inquiries being handled right now?
|
||||
|
||||
**Copywriter subject:** after-hours coverage
|
||||
**Copywriter body:** {{first_name}},
|
||||
|
||||
Is there any discussion internally at {{company_name}} about what happens to after-hours calls?
|
||||
|
||||
Not asking whether you have a solution — just whether it's on anyone's radar. A study mystery-shopped 466 home service companies in 2024. 95% didn't respond to an inquiry within 5 minutes. 40% didn't respond within 5 days at all.
|
||||
|
||||
If it's a live topic, I have something worth sharing. If it's not, no worries.
|
||||
|
||||
**Why revised won:** "That's not a voicemail problem. It's a lead disappearance problem." — reframe creating a new mental category. Inverting the stat to 97% (instead of 3%) was specifically called out as the most visceral presentation in the ballot. Both lines stop the owner mid-read.
|
||||
|
||||
---
|
||||
|
||||
## Group 10 — Reputation Risk / Non-Response Review
|
||||
|
||||
| Version | GK 1st-Place | GK Score | Owner 1st-Place | Owner Score | Overall Rank |
|
||||
|---|---|---|---|---|---|
|
||||
| draft | 1 | 11 | 2 | 7 | 2 (GK) / 1 (Owner) |
|
||||
| revised | 3 | 12 | 0 | 6 | 1 (GK) / 2 (Owner) |
|
||||
| copywriter | 1 | 7 | 1 | 5 | 3 |
|
||||
|
||||
**Winner:** revised (GK) / draft (Owners) — disagree
|
||||
|
||||
**Draft subject:** not responding is the review
|
||||
**Draft body:** {{first_name}}, 40% of home service companies in a recent mystery shop never responded to an inquiry within 5 days. (466 companies surveyed) Not slowly — never.
|
||||
|
||||
Most pest control owners assume their biggest reputation risk is a botched treatment. It's usually a missed call.
|
||||
|
||||
Is that showing up in your reviews, or have you managed to stay ahead of it?
|
||||
|
||||
**Revised subject:** not responding is the review
|
||||
**Revised body:** {{first_name}}, in a mystery shop of 466 home service companies, 40% never responded to an inquiry within 5 days. Not slowly — never.
|
||||
|
||||
Most pest control owners think their biggest reputation risk is a botched treatment. It's usually a missed call.
|
||||
|
||||
Is that pattern showing up in {{company_name}}'s reviews?
|
||||
|
||||
**Copywriter subject:** you probably get emails like this
|
||||
**Copywriter body:** {{first_name}},
|
||||
|
||||
You probably get emails promising you more leads. I'm not doing that.
|
||||
|
||||
What I'm looking at is the leads you're already getting — specifically the ones that call after 6pm, hit voicemail, and never call back. 95% of home service companies don't respond to an inquiry within 5 minutes. The ones that do close at nearly 12x the rate of the ones that don't.
|
||||
|
||||
That gap is where I work. Want me to show you what closing it looks like for a pest control operation?
|
||||
|
||||
**Why draft won (owners):** Draft's close ("have you managed to stay ahead of it?") gave operators an honest out and felt less accusatory than revised's company-name interrogative ("is that pattern showing up in {{company_name}}'s reviews?"). Owners 1 and 3 experienced the company-name close as implying an answer already assumed. The self-aware copywriter opener ("you probably get emails like this") was called a cliche by two of three owners.
|
||||
|
||||
---
|
||||
|
||||
## Overall Totals
|
||||
|
||||
### Gatekeeper totals (5 judges)
|
||||
|
||||
| Version | Total 1st-Place | Total Weighted Score |
|
||||
|---|---|---|
|
||||
| revised | 23 | 108 |
|
||||
| copywriter | 19 | 99 |
|
||||
| draft | 7 | 93 |
|
||||
|
||||
### Owner totals (3 judges)
|
||||
|
||||
| Version | Total 1st-Place | Total Weighted Score |
|
||||
|---|---|---|
|
||||
| copywriter | 15 | 64 |
|
||||
| revised | 13 | 67 |
|
||||
| draft | 2 | 49 |
|
||||
|
||||
### Combined totals (8 judges)
|
||||
|
||||
| Version | Total 1st-Place | Total Weighted Score |
|
||||
|---|---|---|
|
||||
| revised | 36 | 175 |
|
||||
| copywriter | 34 | 163 |
|
||||
| draft | 9 | 142 |
|
||||
|
||||
### Group consensus summary
|
||||
|
||||
| Group | GK Winner | Owner Winner | Consensus |
|
||||
|---|---|---|---|
|
||||
| 1 | copywriter | copywriter | copywriter |
|
||||
| 2 | copywriter | copywriter | copywriter |
|
||||
| 3 | revised | revised | revised |
|
||||
| 4 | copywriter | revised | split |
|
||||
| 5 | revised | revised | revised |
|
||||
| 6 | copywriter | revised | split |
|
||||
| 7 | copywriter | copywriter | copywriter |
|
||||
| 8 | draft | copywriter | copywriter (owner sweep) |
|
||||
| 9 | revised | revised | revised |
|
||||
| 10 | revised | draft | split |
|
||||
|
||||
### Key structural findings
|
||||
|
||||
**Patterns owners responded to:**
|
||||
- Specific visualizable scenarios ("Saturday evening about a wasp nest in their garage", "pulls back the sheets at 11pm and finds bed bugs", "someone texts you on a Saturday saying 'we missed a call'")
|
||||
- Two-scenario contrast structure: Scenario A (current state) / Scenario B (proposed state) / "Same call. Completely different outcome."
|
||||
- Reframes that create new mental categories: "That's not a voicemail problem. It's a lead disappearance problem."
|
||||
- Single closing question that gives an honest out: "Have you managed to stay ahead of it?"
|
||||
|
||||
**Patterns owners rejected permanently:**
|
||||
- Fabricated revenue math in email 1 ($136,500, "260 leads a year") — zero first-place owner votes across entire study
|
||||
- Review-shaming openers ("I noticed {{company_name}} has a handful of 1-star reviews") — immediate defensive shutdown
|
||||
- Escape-hatch closes ("or are you not running paid ads right now?") — signals sender doesn't know the prospect
|
||||
- Guessed specificity (truck-count assumptions, coverage-area tokens)
|
||||
- Self-aware meta-opener ("You probably get emails like this") — recognized as a cliche
|
||||
|
||||
**Gatekeeper accuracy:** 60% (6/10). Systematic over-indexing on research signals and analytical depth. Blind spots: emotional recognition, skeptical operator sensitivity, assumed-context tolerance.
|
||||
|
|
@ -0,0 +1,138 @@
|
|||
---
|
||||
source: "niche-automation-prospecting"
|
||||
date: "2026-03-14"
|
||||
tags: [research, cold-email, pest-control, pest-control-spring-2026, copy-rubric, owner-psychology, field-service]
|
||||
---
|
||||
|
||||
# What Pest Control Owners Respond To and Reject in Cold Email
|
||||
|
||||
## What This Is
|
||||
|
||||
This note is derived from elaboration interviews with 3 simulated pest control owner-operator personas (Mike Deluca / 22yr / Ohio / 12 trucks; Sandra Kowalski / 6yr / Texas / 8 trucks; Ray Tanner / 14yr / Georgia / 7 trucks) who participated in a blind ranked-choice experiment rating 30 cold emails across 10 groups. The findings are empirically grounded in stated owner reactions, not theoretical copy principles.
|
||||
|
||||
## Instant Delete Triggers
|
||||
|
||||
**Review-shaming openers ("I noticed you have a few 1-star reviews")**
|
||||
|
||||
All three owners reacted defensively and immediately. Mike: "Who asked you?" and "It feels aggressive in a passive-aggressive way — like someone walking up to you at a chamber event and opening with 'I see your trucks have a few dents.'" Sandra: "They opened by walking up to me, a stranger, and pointing at something I'm already sensitive about. That's not how you start a conversation. That's how you pick a fight." Ray: "Who are you to walk up to my front door and tell me my house looks dirty?" The rejection is not a softening problem — the angle itself is structurally rejected. The premise implies "I found something wrong with your business," which triggers defensiveness before the useful content can land. Sandra noted she believed the stat cited (37% of bad reviews are about communication) but "couldn't get past the opener to care about it."
|
||||
|
||||
What to do instead: Open with a scenario that lets the owner arrive at the reputation risk themselves. The bed bug scenario (homeowner at 11pm, panicking, checking Yelp) covered identical ground about reputation risk without touching reviews — and Sandra read it twice.
|
||||
|
||||
**Fabricated revenue math ("$136,500 a year in lost revenue")**
|
||||
|
||||
Earned zero second-place votes across the entire owner panel — the worst-performing single element in the study. The failure mechanism: owners immediately reverse-engineer the baseline assumptions and find them unverifiable. Sandra: "My exact reaction was: okay, where did you get that... I'm doing the reverse math. 5 missed calls a week times 52 weeks is 260 leads. At what conversion rate? What LTV are they assuming? Do they know we do mostly residential quarterly plans at $150 a quarter?" Ray: "That number is built on assumptions the sender made about my business without knowing anything about my business." Mike: "Nobody looked at my business before they wrote that number." The escape-hatch disclaimer ("I'm not saying that's your number") made things worse, not better — owners read it as an admission the number was invented.
|
||||
|
||||
What to do instead: Use verifiable industry stats tied to named sources ("Driven Results, 2,847 leads" or "Invoca"). If dollar framing is needed, frame it as a question with the owner's numbers — "is that math close for your operation?" — not a projection.
|
||||
|
||||
**Wrong operational assumptions (truck count, wrong software)**
|
||||
|
||||
"For a company running 10–15 trucks" earned unanimous last place in its group even though Owner 1 (Mike, 12 trucks) was technically in range. Sandra (8 trucks): "The moment I see a number that doesn't describe me, I know it's a segment, not a person." One owner had GorillaDesk mentioned incorrectly: "We use something else. Instantly out. Even if the email is right about everything else, the wrong tool reference tells me they guessed. And if they guessed on that, they guessed on everything."
|
||||
|
||||
What to do instead: Use specificity that is verifiable and industry-level (FSM, QuickBooks, missed call logs) rather than specificity that is assumed at the operator level (truck count, software name, coverage area tokens).
|
||||
|
||||
**Self-aware cold email openers ("You probably get emails like this. I'm not doing that.")**
|
||||
|
||||
Split the panel — Owner 2 liked it; Owners 1 and 3 rejected it. Mike: "The self-awareness doesn't make me trust you more — it makes me think you know your email is weak and you're trying to get ahead of that. Write a better email." Ray called it a cliche: "the cold email equivalent of saying I'll keep this brief before writing six paragraphs." May have worked when fresh, but experienced operators recognize it as a trope.
|
||||
|
||||
**Escape-hatch closes ("or are you not running paid ads right now?")**
|
||||
|
||||
Both versions with this structure were penalized. The double-option close hands the prospect a reason to disengage before they've processed the pitch. It also signals the sender doesn't know the prospect's situation, undermining any credibility built in the body.
|
||||
|
||||
What to do instead: Commit to the premise, or choose a premise that applies broadly. Closes that assume the owner may have already solved the problem ("Have you managed to stay ahead of it?") outperform closes that imply uncertainty about whether the problem applies.
|
||||
|
||||
**Double-question closes**
|
||||
|
||||
Multiple owners flagged closing with two questions as diluting the ask. Sandra: "One problem, one question." Mike: "Three short paragraphs and a question." Any close with compound structure — "is this something you've noticed? and if so, would you be open to a quick conversation?" — reads as a pitch inside a question.
|
||||
|
||||
## What Makes Them Stop and Read
|
||||
|
||||
**Specific scenarios they've lived**
|
||||
|
||||
The highest-performing emails opened with a named moment — not a statistic, not a general observation, but a visualizable situation. Mike on the wasp nest scenario: "Not because it was clever writing. Because I've lived that call... The difference is texture. Generic emails describe a category of problem. The scenario emails described a moment." Ray on bed bugs at 11pm: "I've had that call. Or I've had the version of it where the voicemail sits until Monday and by then they've already called Orkin." The scenario earns attention by letting the owner react to something they know is real before they have to evaluate whether they trust the sender.
|
||||
|
||||
Key structure: open with the customer's experience, not the operator's failure. Mike: the wasp nest email "put me on the customer's side of the call before it put me on my own. I wasn't reading about my operation failing — I was reading about a homeowner who's in a bind."
|
||||
|
||||
**Conceptual reframes that name an unnamed problem**
|
||||
|
||||
The two most universally praised lines were both reframes that created new mental categories:
|
||||
- "That's not a voicemail problem. It's a lead disappearance problem."
|
||||
- "Most pest control owners think their biggest reputation risk is a botched treatment. It's usually a missed call."
|
||||
|
||||
Mike on the response/follow-up reframe: "That's the distinction I've been trying to make to myself but haven't found the words for. I've been thinking about this as a staffing problem, a coverage problem. Calling it a response problem with a defined window is a more useful frame." The mechanism: owners recognize the truth immediately because the problem was always there but unlabeled. These lines stop owners mid-read.
|
||||
|
||||
**Questions with uncomfortable honest answers**
|
||||
|
||||
Ray on "Quick question — if someone calls your company on a Saturday about a wasp problem, what actually happens to that call?": "My honest reaction was: that's a good question. And I don't say that about many cold emails... The reason that email worked is because it asked me to describe my own process, and the process has a gap in it that I already know about. It didn't tell me I had a problem. It asked me a question that made me realize it." The mechanism: the question doesn't accuse — it invites the owner to surface a gap they've been glossing over.
|
||||
|
||||
**Stats that confirm what they've already sensed**
|
||||
|
||||
Mike: "The stats that work are the ones that describe behavior I've already observed and just didn't have a number for. 'Fewer than 3% of callers leave a voicemail' — that's exactly what I've noticed. The voicemail box is never as full as the calls we missed. Now I have a name for it." Verifiable sources matter: "If you cite a source I can look up, you earn more credibility... 'Invoca' or 'Driven Results, 2,847 leads' — I can search that. 'Industry data' tells me nothing."
|
||||
|
||||
**The two-scenario contrast structure**
|
||||
|
||||
The only format to earn identical positive reactions from all three owners across different personas. Scenario A: voicemail, fewer than 3% leave one. Scenario B: text back in under 60 seconds, one qualifying question, confirmation of morning callback. Same call. Completely different outcome. Owner 1: "the first email in this whole batch that describes something real." Owner 2: "actually answers the 'so what' before I have to ask." Owner 3: "doesn't lecture me — it just shows me the difference." This structure earns cross-persona buy-in by showing the alternative instead of arguing for it.
|
||||
|
||||
## The Review-Shaming Finding (Deeper)
|
||||
|
||||
All three owners elaborated on this unprompted. The emotional reaction is consistent: defensive, not curious. But the underlying mechanism varies slightly across owners.
|
||||
|
||||
Mike frames it as presumptuous knowledge: "The assumption underneath that opener is wrong. The email assumed I didn't know why those reviews existed, and it was going to educate me." Ray frames it as a manipulation tactic: "They found something that might make me feel defensive and they led with it on purpose, hoping that insecurity would make me keep reading. That's a manipulation tactic. I recognize it because I've seen it from bad salespeople my whole career." Sandra frames it as an earned-right problem: "I'd want to hear it from someone who's earned the right to say it — a consultant I've hired, a peer in the industry, someone I asked for an opinion. Not from a cold email."
|
||||
|
||||
What the opener signals about the sender: that they know how to type your business name into Google (Ray: "That's not homework. That's ten seconds."); that they're using your reputation as a lever; that they've guessed at why the reviews exist and are presenting that guess as insight ("'I'd bet most of them aren't about the treatment itself.' You'd bet. Based on what?" — Ray).
|
||||
|
||||
The lesson for reputation-angle emails: the insight about reputation risk is valid and owners accept it — but the angle must arrive through the customer's experience, not through the owner's review profile. Describe the scenario that creates the review (missed call → no callback → 1-star), not the review itself. Let the owner connect the dots.
|
||||
|
||||
## Format Rules (From Owner Feedback)
|
||||
|
||||
**Email length:** Three short paragraphs maximum. Mike: "I read email on my phone between jobs. If I have to scroll, I'm already half-checked-out. Three short paragraphs and a question. That's the format." Sandra on the best email in the batch: "three sentences. Four at most." Ray: "if you know what you're saying, you don't need six paragraphs to say it. The people who write long first emails are usually trying to cover for a weak premise with volume."
|
||||
|
||||
**Number of asks:** One. Sandra: "One problem, one question. The emails I kept reading were the ones that identified one specific problem and then asked me one honest question about it." Any email trying to cover multiple angles (review risk + missed revenue + ad spend + competitor advantage) lost all three owners.
|
||||
|
||||
**Stats:** One stat, sourced, placed after the scenario — not before. Mike: "Use one stat. One. Make sure it's real and make sure I can look it up. Don't stack three data points in a three-paragraph email." Stats that open the email without scenario context are processed as headlines and filed away, not felt.
|
||||
|
||||
**How to open:** With a specific operational moment the owner will recognize — not a statistic, not a company name reference, not a review citation. The scenario earns two more sentences of attention. Mike: "If you can write one sentence that makes me think 'yeah, that happens here' — you have my attention for the next two."
|
||||
|
||||
**How to close:** Single direct question, give them an honest out. Best performing closes assumed the owner may have already handled the problem. Worst performing closes assumed the problem was certain or implied an answer. Never end with an offer to send materials or book a call before the owner has expressed interest.
|
||||
|
||||
## What Earns a Reply (Even Without a Sale)
|
||||
|
||||
Ray said it directly: the bar for a reply is low — he just needs to not be annoyed. "Most cold emails fail that test in the first two sentences. An email that gets past the first two sentences without annoying me is already in the top ten percent... If it makes it to the end without doing anything that makes me distrust it, I'll probably reply — if only to say 'not something I'm looking at right now.' I've done that before. When someone writes something real, they deserve a real answer."
|
||||
|
||||
What "real" means to these owners:
|
||||
|
||||
It asks a question the owner actually has to think about to answer. Ray: "It would have to ask me something I don't already know the answer to, or something where my answer would reveal a real gap." The Saturday wasp call question worked because "when I actually answered it in my head, the answer exposed something I'd been glossing over."
|
||||
|
||||
It doesn't explain the solution in email 1. Ray: "If it just asks me a real question, I might reply because I'm curious where it's going... if it explains the solution, I'm probably less likely to reply, because now it's a pitch."
|
||||
|
||||
It proves the sender has been inside a service business. Sandra: "I need to know you understand how companies like mine work. That FSM-QuickBooks-whiteboard-group-text observation landed because it's accurate. Show me you've been around field service before." Mike: "Start with something that tells me you know what this business actually feels like. Not demographics. Not 'pest control companies like yours.' A moment."
|
||||
|
||||
It doesn't manufacture urgency. Ray: "Nothing turns me off faster than manufactured urgency — 'spring is your busiest season and you're about to miss this window.' I know when my busy season is. I don't need you to tell me."
|
||||
|
||||
## Rubric Application Notes
|
||||
|
||||
**Hard to detect programmatically:**
|
||||
- Whether a scenario is specific enough to be visceral vs. generic enough to feel made up — requires understanding of actual pest control operations (what a Saturday in Central Texas looks like vs. a made-up scenario that "sounds" like pest control)
|
||||
- Whether a stat feels like it matches the owner's lived experience vs. describes an aggregate — depends on context and framing, not the number itself
|
||||
- Tone register mismatch ("too polished signals nobody in the room has ever run a service route" — Sandra) — catching this requires sensitivity to formality markers
|
||||
- Whether a question is genuinely open vs. rhetorical/leading — requires parsing intent from sentence structure
|
||||
|
||||
**Easy to detect programmatically:**
|
||||
- Revenue projection math: any dollar figure derived from missed-call-count × LTV × conversion in email 1
|
||||
- Truck count or size assumptions: specific fleet-size claims not tied to verified data
|
||||
- Review references: any opener mentioning star ratings, reviews, or reputation deficiencies by name
|
||||
- Escape-hatch closes: double-option structure with an opt-out conditional
|
||||
- Multiple CTAs or questions in the close
|
||||
- Word count over a threshold (3 short paragraphs = ~120–180 words maximum for the body)
|
||||
- Vague source attribution: "industry research," "studies show," "data suggests" without a named source
|
||||
- Self-aware meta-openers: "you probably get emails like this" or "I know you're busy"
|
||||
- Hedge language on competitive claims: "probably," "I'd bet," "likely" — commit fully or omit
|
||||
|
||||
**Questions a rubric should ask about any email:**
|
||||
1. Does the opening contain a specific, named operational scenario — or a statistic, a review reference, or a generic industry claim?
|
||||
2. Does the email contain fabricated revenue math (missed calls × LTV × conversion)?
|
||||
3. Does the email contain assumed specifics about the prospect's operation that are not verifiably sourced?
|
||||
4. Is there exactly one closing question, and does it give the owner a genuine "no" option?
|
||||
5. Is every statistic cited with a named, searchable source?
|
||||
6. Does the email stay under ~180 words?
|
||||
7. Does the email try to solve the problem or pitch a solution — or does it only surface the problem?
|
||||
8. Does the close offer to send materials or book a call before the owner has expressed interest?
|
||||
|
|
@ -0,0 +1,85 @@
|
|||
---
|
||||
tags:
|
||||
- reference
|
||||
- sms
|
||||
- pest-control
|
||||
- pest-control-spring-2026
|
||||
- niche-automation-prospecting
|
||||
source: niche-automation-prospecting
|
||||
date: 2026-03-31
|
||||
---
|
||||
|
||||
# 10DLC ISV Setup Guide for OnCadence
|
||||
|
||||
**Date:** 2026-03-31
|
||||
**Context:** Practical guide for managing 10DLC registration on behalf of pest control clients via Twilio ISV architecture.
|
||||
|
||||
## Architecture
|
||||
- OnCadence = primary Twilio account (ISV/CSP)
|
||||
- Each pest control client = Twilio subaccount
|
||||
- Each subaccount gets: Secondary Customer Profile, Brand registration, Campaign registration, dedicated phone number(s)
|
||||
- If one client gets flagged, others are unaffected
|
||||
|
||||
## What OnCadence Submits (via TrustHub API)
|
||||
- ISV creates Primary Business Profile once
|
||||
- Per client: Secondary Customer Profile with client's identity data
|
||||
- Brand registration + Campaign registration per client
|
||||
|
||||
## What Each Client Provides (5-field intake form)
|
||||
1. Legal business name (must match IRS EIN records exactly)
|
||||
2. EIN
|
||||
3. Business address (no P.O. boxes)
|
||||
4. Website URL (Facebook Business page acceptable if no website)
|
||||
5. Owner name, email, mobile number
|
||||
|
||||
## No-Website Edge Case
|
||||
- Facebook Business page accepted by some carriers
|
||||
- Google Business Profile URL sometimes works
|
||||
- Page must "bear some relationship" to business name
|
||||
- Companies with zero web presence = genuine friction point
|
||||
|
||||
## Sole Proprietors (No EIN)
|
||||
- Lower throughput path: 1,000 msgs/day on T-Mobile
|
||||
- Limited to one campaign, one number
|
||||
- Requires mobile phone OTP verification
|
||||
- Adequate for single-tech pest control shops
|
||||
|
||||
## Cost Per Client
|
||||
|
||||
| Item | Cost |
|
||||
|---|---|
|
||||
| Brand registration | $4.50 one-time |
|
||||
| Campaign vetting | $15.00 one-time |
|
||||
| Authentication verification | $12.50 one-time |
|
||||
| **Total one-time** | **~$32** |
|
||||
| Campaign monthly fee | $1.50-10/mo |
|
||||
| Twilio number rental | ~$1.15/mo |
|
||||
| Carrier surcharge/SMS | $0.003-0.005/msg |
|
||||
| **Practical monthly (500 SMS)** | **~$10-20/mo** |
|
||||
|
||||
## Timeline
|
||||
- Twilio announced direct carrier connections March 24, 2026 — registration now "days not weeks"
|
||||
- Realistic: 5-10 business days for clean submission
|
||||
- Tell clients 2 weeks, aim for 1
|
||||
|
||||
## Common Rejection Causes
|
||||
- Legal name doesn't match IRS records (most common)
|
||||
- Website down, parked, or doesn't mention business
|
||||
- Vague campaign description
|
||||
- Missing opt-out language in sample messages
|
||||
|
||||
## OnCadence Implementation
|
||||
1. Build standardized intake form (5 fields)
|
||||
2. Pre-write one campaign template for pest control: "Appointment Reminders / Customer Care" use case
|
||||
3. Include sample messages with STOP opt-out language
|
||||
4. Submit via Twilio TrustHub API using subaccount-per-client architecture
|
||||
5. Budget ~$32 into setup fee (trivial at $3,500-5,000 implementation price)
|
||||
|
||||
## Key Rules
|
||||
- Cannot use OnCadence's identity for client registrations — must use client's actual EIN/name
|
||||
- Cannot share one number across multiple client brands (prohibited since Summer 2023)
|
||||
- Each client gets dedicated Twilio long-code number registered to their brand
|
||||
- Forward callbacks to client's actual business line
|
||||
|
||||
## ISV Legal
|
||||
Yes, OnCadence can submit registrations on behalf of clients. This is the designed ISV path in Twilio's system. The registration attests the sending business identity, which must be the client's — but the ISV handles the submission process.
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
---
|
||||
tags:
|
||||
- research
|
||||
- pest-control
|
||||
- pest-control-spring-2026
|
||||
- competitive-analysis
|
||||
- niche-automation-prospecting
|
||||
source: niche-automation-prospecting
|
||||
date: 2026-03-31
|
||||
---
|
||||
|
||||
# RingCentral AIR + VoIP AI Receptionist Competitive Analysis
|
||||
|
||||
**Date:** 2026-03-31
|
||||
**Context:** Assessing whether VoIP providers' built-in AI features compete with OnCadence's done-for-you lead capture system.
|
||||
|
||||
## RingCentral AI Receptionist (AIR)
|
||||
- Conversational voice AI that answers calls, qualifies with configurable questions, books appointments (Google/Outlook), sends one SMS follow-up
|
||||
- **Pricing:** $39/mo add-on (requires RingEX) or $59/mo standalone (AIR Everywhere, any phone system). 100 min/mo included, ~$0.50/min overage
|
||||
- **Real cost for 300-call/month SMB:** $150-200/mo after overages
|
||||
|
||||
### What AIR Can Do
|
||||
- Answer inbound calls with NLP
|
||||
- FAQ resolution from knowledge base
|
||||
- Lead capture questionnaire (customizable questions)
|
||||
- Book into Google Calendar or Outlook
|
||||
- Send ONE SMS follow-up
|
||||
- Route to live agent with context
|
||||
- Sync to HubSpot, Zoho, Salesforce
|
||||
|
||||
### What AIR Cannot Do
|
||||
- Multi-step SMS sequences (sends one text only)
|
||||
- Conditional branching in qualification (flat questionnaire, not decision tree)
|
||||
- FSM integration (no Jobber, ServiceTitan, Housecall Pro, GorillaDesk)
|
||||
- Post-call nurture/drip sequences
|
||||
- MMS/media in SMS
|
||||
- Complex qualification (job scope, budget triage, urgency scoring)
|
||||
|
||||
### Reviewer quote
|
||||
"For businesses with specific intake processes — contractors that need to qualify job scope before scheduling — this lack of customization is a problem."
|
||||
|
||||
## Other VoIP AI Features
|
||||
- **JustCall AI Voice Agent:** Most dangerous competitor. Drag-and-drop workflow builder, multi-step SMS sequences post-call, 100+ CRM integrations. $29+/user/mo, AI agent is add-on tier.
|
||||
- **Dialpad:** Static auto-reply SMS only. Not a conversational flow.
|
||||
- **OpenPhone/Quo:** AI voicemail transcription only. No booking or qualification agent.
|
||||
|
||||
## Purpose-Built Competitor: LeadTruffle
|
||||
- Missed call → voicemail transcription → SMS within 60-120 sec → AI qualification sequence
|
||||
- Integrates with Google LSA, Yelp, Angi, Thumbtack
|
||||
- Targets HVAC, plumbing, electrical, roofing, landscaping
|
||||
- **$399-629/month recurring** + $299 onboarding
|
||||
|
||||
## OnCadence Differentiation
|
||||
1. **Workflow depth:** Full branching qualification → nurture sequences → booking → job creation → reminders → review requests (not just one SMS)
|
||||
2. **FSM integration:** Connects to actual dispatch/job management layer
|
||||
3. **Niche calibration:** Pre-built sequences for pest control service types
|
||||
4. **Economics:** $3,500-5,000 one-time vs LeadTruffle $4,788+/year recurring
|
||||
5. **Setup ownership:** Done-for-you installation, not self-serve configuration
|
||||
|
||||
## Competitive Positioning
|
||||
|
||||
| Layer | Who Solves It | Price |
|
||||
|---|---|---|
|
||||
| Missed call → text back | RingCentral AIR, JustCall | $39-59/mo |
|
||||
| Multi-step qualification + booking | LeadTruffle, GoHighLevel agencies | $399-629/mo |
|
||||
| Full intake → dispatch → nurture → review pipeline | OnCadence | $3,500-5,000 setup |
|
||||
|
||||
**Key insight:** "Missed call → text back" is now a $39 commodity. The pitch must be the complete lead-to-job pipeline, not the trigger.
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
---
|
||||
tags: [research, pest-control, pest-control-spring-2026, sms, lead-capture]
|
||||
source: niche-automation-prospecting
|
||||
date: 2026-03-31
|
||||
---
|
||||
|
||||
# VoIP Provider API Research — Pest Control Lead Capture Integration
|
||||
|
||||
**Date:** 2026-03-31
|
||||
**Context:** Evaluating whether OnCadence (Rails app) can replace n8n+Airtable+Twilio bundle by integrating directly with customers' VoIP providers for missed call detection + SMS response.
|
||||
|
||||
## Key Finding
|
||||
Most pest control SMBs (5-15 employees, QuickBooks+whiteboard segment) are on Google Voice, Grasshopper, or personal cell — ALL have zero API. Phone system migration is part of the product for most customers.
|
||||
|
||||
## Provider Tiers
|
||||
|
||||
### Tier 1 — Build Adapters First
|
||||
- **JustCall** ($49/user/mo Pro): Only provider with explicit "auto-reply to missed calls via API" documentation. Dedicated webhook events for missed calls AND voicemails separately. Best documentation for this exact use case.
|
||||
- **RingCentral** ($20/user/mo Core): Most mature API, included on all plans, widest install base at 20+ employees. Webhook subscriptions have TTL that must be renewed programmatically. Also offers built-in "AI Receptionist" with appointment booking + lead capture + SMS follow-up as add-on — potential competitor.
|
||||
- **Dialpad** ($15/user/mo Standard): Best price-to-API ratio. Full missed call + voicemail transcription + SMS. OAuth approval process required.
|
||||
|
||||
### Tier 2 — Viable with Caveats
|
||||
- **OpenPhone/Quo** ($35/user/mo Scale): API locked to highest tier. Webhook reliability concerns reported.
|
||||
- **Aircall** ($50/user/mo Pro, 3-user minimum = $150/mo floor): Excellent developer docs, purpose-built SMS deflection for missed calls.
|
||||
- **GoTo Connect**: Solid API, thin developer community.
|
||||
|
||||
### Hard Disqualifications (No API)
|
||||
Google Voice, Grasshopper, Ooma Office, MagicJack, personal cell, landline+ATA
|
||||
|
||||
## 10DLC / A2P SMS Compliance
|
||||
- Every customer = separate 10DLC brand + campaign registration
|
||||
- 1-3 weeks per client before SMS goes live
|
||||
- As of Feb 2025, US carriers BLOCK SMS from unregistered local numbers
|
||||
- VoIP provider facilitates registration but business must provide: EIN, legal name, website, opt-in language
|
||||
- ISV/CSP model available via Twilio for managing multiple client registrations
|
||||
|
||||
## Adoption Path for Target Segment
|
||||
1. Personal cell → Google Voice (free) → Dedicated VoIP
|
||||
2. The 10DLC enforcement is pushing Google Voice users off free tier — active migration moment
|
||||
3. Companies at 15-50 employees with dispatcher step up to multi-seat systems
|
||||
4. FSM-native phone (ServiceTitan Phones Pro, Voice for Pest) only relevant above FSM adoption floor
|
||||
|
||||
## Number Porting
|
||||
- FCC-guaranteed (LNP). Landline→VoIP: 5-7 days. Cell→VoIP: 7-10 days.
|
||||
- Real objections: fear of downtime, 20-year number anxiety, contract ETFs, internet reliability concerns
|
||||
- Common gotcha: number registered to individual not business
|
||||
|
||||
## Strategic Implications for OnCadence
|
||||
- Phone system migration is part of the product for most customers
|
||||
- Email-first default; SMS is Phase 2 after phone situation confirmed
|
||||
- Discovery call qualifier: "What phone system are you using?"
|
||||
- JustCall or Dialpad as recommended migration targets
|
||||
- 10DLC registration built into onboarding timeline (1-3 weeks)
|
||||
|
||||
## Providers in Decline
|
||||
- Vonage: Acquired by Ericsson, strategic shift to enterprise Network APIs, not SMB
|
||||
- 8x8: Declining revenue, cost-cutting mode
|
||||
- Grasshopper: Functionally stagnant under GoTo ownership
|
||||
|
|
@ -0,0 +1,138 @@
|
|||
---
|
||||
summary: A scannable do/don't checklist, command quick-reference card, multi-project setup guide, and common-mistakes list for using Graphify v0.8.30 correctly and efficiently.
|
||||
tags:
|
||||
- type/reference
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
- domain/llm
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
|
||||
# 09 — Best-Practices Checklist & Quick Reference
|
||||
|
||||
The page to keep open while using Graphify (v0.8.30): a scannable do/don't list, a command quick-reference, a "many projects" setup mini-guide, and the common mistakes to avoid. Detail lives in the sibling docs — this card cross-links to them.
|
||||
|
||||
> Provenance tags: `[interview]` (creator's intent/sales claims), `[github]` (verified repo facts), `[community]` (third-party), `[unverified claim]`.
|
||||
|
||||
---
|
||||
|
||||
## Do / Don't
|
||||
|
||||
**Do**
|
||||
|
||||
- **Index code freely — it's local AST, no tokens.** Code (33 languages, tree-sitter) parses on your machine with no model calls. Run `graphify .` on any repo without fear. `[github]` `[interview]` — see [03-ingesting-code-ast.md](03-ingesting-code-ast.md)
|
||||
- **Reserve the model (tokens) for documents/images/audio/video only.** This split is the #1 cost rule. `[interview]` — see [04-ingesting-docs-knowledge.md](04-ingesting-docs-knowledge.md)
|
||||
- **Use a local backend (`--backend ollama`) for documents** to keep extraction free and private. `[interview]` — see [05-local-models-and-backends.md](05-local-models-and-backends.md)
|
||||
- **Ask for god nodes FIRST, then scalpel down.** Read the top of `GRAPH_REPORT.md` for the architecture spine, then `explain` / `path` / bounded `query` into specifics. `[interview]` — see [06-querying-and-god-nodes.md](06-querying-and-god-nodes.md)
|
||||
- **Prompt the graph, don't make the LLM read the corpus.** The graph IS the memory; have the assistant *extract from* it, not traverse it. `[interview]`
|
||||
- **Always `--update` when adding data.** SHA-256 hashing + dedup re-process only changed files and merge in, instead of rebuilding. `[interview]` `[github]` — see [07-token-economics-and-updates.md](07-token-economics-and-updates.md)
|
||||
- **Split large corpora and re-ingest in pieces** rather than dumping one giant chunk — cheaper and less hallucination/dilution. `[interview]`
|
||||
- **Tune for local/small models:** smaller `--token-budget` (chunk size), lower `--max-concurrency`, raise `--api-timeout` if it stalls. `[github]`
|
||||
- **Automate freshness for code:** `graphify hook install` (commit hook) or `--watch` — both AST-only, free. `[github]`
|
||||
- **Measure your own savings with `graphify benchmark`** instead of trusting any headline multiplier. `[github]`
|
||||
|
||||
**Don't**
|
||||
|
||||
- **Don't quote a token multiplier as a promise.** "70x / 90x / 71.5x" are corpus-dependent marketing figures; independent testing shows ~1–49x, and net-*negative* on small repos (<100 files ≈ 1–3x; some users measure *more* tokens). Say "measure with `graphify benchmark`." `[unverified claim]` `[community]` — see [07](07-token-economics-and-updates.md) and [external-tips.md](external-tips.md)
|
||||
- **Don't run your first build on a big doc pile with a cloud backend** — that's the way to burn your daily limit (one user burned theirs + $25–30 extra). Start on code, then add docs locally. `[interview]`
|
||||
- **Don't expect `graphify query "god nodes"` to return the ranking** — `query` is semantic search; the structural god-node list lives at the top of `GRAPH_REPORT.md`. `[github]`
|
||||
- **Don't confuse `--budget` with `--token-budget`.** `--budget N` caps a **query answer**; `--token-budget N` sets the **extraction chunk size**. Different stages. `[github]`
|
||||
- **Don't expect the commit hook (or `--watch`) to refresh docs** — they handle **code only**. Re-run `--update` on docs manually (they cost tokens). `[github]`
|
||||
- **Don't rebuild from scratch to add data** — use `--update`. Use `--force` only to overwrite/clear ghost nodes after deletions. `[github]`
|
||||
- **Don't treat Obsidian as a substitute.** It's a nice visual pairing, but it can't cluster or surface cross-community links the way Graphify does. `[interview]` — see [01-overview-concepts.md](01-overview-concepts.md)
|
||||
- **Don't get the package name wrong** — it's `graphifyy` (double-y), the binary is `graphify`. The single most common install mistake. `[community]`
|
||||
|
||||
---
|
||||
|
||||
## Command quick-reference (v0.8.30)
|
||||
|
||||
> **Invocation forms.** In-IDE uses the slash form (`/graphify ./path --update`) and borrows your session's model for docs. Headless uses `graphify extract ./path` and needs an explicit `--backend`. The `query` / `path` / `explain` verbs work both ways (drop the slash in a terminal; always use the no-slash form on Windows PowerShell). `[github]`
|
||||
|
||||
**Install & register**
|
||||
|
||||
```bash
|
||||
uv tool install graphifyy # recommended (or: pip install graphifyy)
|
||||
graphify install # register the skill with your assistant
|
||||
graphify claude install # optional: make Claude consult the graph automatically
|
||||
```
|
||||
|
||||
**Build / update**
|
||||
|
||||
| Goal | Command | Cost |
|
||||
|---|---|---|
|
||||
| First build | `/graphify ./path` — or headless `graphify extract ./path --backend ollama` | code free; docs/images cost tokens |
|
||||
| Add / changed data | `/graphify ./path --update` | only changed files re-processed |
|
||||
| Rebuild, clear ghost nodes after deletions | `graphify extract . --force` (or `GRAPHIFY_FORCE=1`) | as a full rebuild |
|
||||
| Richer inference | `/graphify ./path --mode deep` | higher (more LLM passes) |
|
||||
| Auto-rebuild on commit (code only) | `graphify hook install` | free (AST) |
|
||||
| Live auto-rebuild (code only) | `/graphify ./path --watch` | free (AST) |
|
||||
| Measure savings | `graphify benchmark` | free |
|
||||
|
||||
**Query (start at god nodes in `GRAPH_REPORT.md`, then drill down)**
|
||||
|
||||
| Goal | Command |
|
||||
|---|---|
|
||||
| Semantic search, bounded answer | `graphify query "..." --dfs --budget 1500` |
|
||||
| Connect two named nodes | `graphify path "A" "B"` |
|
||||
| Deep-dive one node | `graphify explain "Node"` |
|
||||
| Visual call-flow | `graphify export callflow-html` |
|
||||
| Re-cluster (free, no re-extract) | `graphify cluster-only . --resolution 1.5` |
|
||||
| De-noise god-node ranking | `... --exclude-hubs 99` |
|
||||
| PR impact / review queue | `graphify prs` (`prs 42`, `--triage`, `--conflicts`) |
|
||||
|
||||
**Local-model tuning flags** (extraction stage; see [05](05-local-models-and-backends.md))
|
||||
|
||||
```bash
|
||||
graphify extract ./docs --backend ollama \
|
||||
--token-budget 4000 --max-concurrency 2 --api-timeout 900
|
||||
# Ollama env: OLLAMA_BASE_URL, OLLAMA_MODEL,
|
||||
# GRAPHIFY_OLLAMA_NUM_CTX, GRAPHIFY_OLLAMA_KEEP_ALIVE
|
||||
```
|
||||
|
||||
**Outputs** land in `graphify-out/`: `graph.html`, `GRAPH_REPORT.md`, `graph.json`, `cache/`. Backends: `ollama` (local), `bedrock`, `claude`, `gemini`, `openai`, `deepseek`, `kimi`.
|
||||
|
||||
---
|
||||
|
||||
## Setting up across many projects (code + docs + KB)
|
||||
|
||||
The pattern: **one graph per project, folded into one global view.** Keeps each `--update` cheap and isolated while giving you a single place to query across everything. Full playbook in [08-workflows-and-use-cases.md](08-workflows-and-use-cases.md) (Playbook 4) and the global-graph mechanics in [03-ingesting-code-ast.md](03-ingesting-code-ast.md).
|
||||
|
||||
1. **Build a graph in each repo / doc folder / KB vault.** Code is free (AST); route documents through a local backend to keep them free and private (see [05-local-models-and-backends.md](05-local-models-and-backends.md)):
|
||||
```bash
|
||||
/graphify . # in a code repo (free)
|
||||
graphify extract ./notes --backend ollama # in a docs/KB folder (local model)
|
||||
```
|
||||
2. **Register each into the global graph** (lives at `~/.graphify/global.json`):
|
||||
```bash
|
||||
graphify global add graphify-out/graph.json my-api
|
||||
graphify global add graphify-out/graph.json my-notes
|
||||
graphify global list # node/edge counts per repo
|
||||
graphify global remove my-api # unregister
|
||||
```
|
||||
3. **Set a freshness cadence by content type:**
|
||||
- **Code → automate:** `graphify hook install` per repo (rebuilds on commit, free). `[github]`
|
||||
- **Docs/KB → on demand:** `--update` deliberately after adding/editing material (costs tokens; the hook won't touch docs). `[github]`
|
||||
- **Re-register after a doc refresh** so the global view picks up new nodes: `graphify global add graphify-out/graph.json my-notes`.
|
||||
4. **Pull in external knowledge** (papers, lectures — videos transcribed locally):
|
||||
```bash
|
||||
/graphify add https://arxiv.org/abs/1706.03762
|
||||
/graphify add <youtube-url>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common mistakes
|
||||
|
||||
- **Wrong package name.** Install `graphifyy` (double-y); the binary is `graphify`. `[community]`
|
||||
- **First run = big docs + cloud backend → token burn.** Start on code (free), add docs with a local Ollama backend. `[interview]`
|
||||
- **Expecting `query` to return god nodes.** It's semantic search — read the ranking at the top of `GRAPH_REPORT.md`. `[github]`
|
||||
- **Mixing up `--budget` (query answer) and `--token-budget` (extraction chunk).** `[github]`
|
||||
- **Expecting the commit hook / `--watch` to refresh docs.** Code only; re-run `--update` on docs yourself. `[github]`
|
||||
- **Stale-node drift.** `--update` is known to leak stale nodes (removed symbols aren't always pruned); use `--force` to rebuild clean when the graph drifts. `[community]` `[github]`
|
||||
- **Quoting a savings multiplier.** Corpus-dependent — measure your own with `graphify benchmark`. `[unverified claim]`
|
||||
|
||||
---
|
||||
|
||||
Index: [00-README.md](00-README.md) · Concepts: [01-overview-concepts.md](01-overview-concepts.md) · Install: [02-installation-setup.md](02-installation-setup.md) · Community/skeptical notes: [external-tips.md](external-tips.md)
|
||||
|
|
@ -0,0 +1,108 @@
|
|||
---
|
||||
summary: A provenance-tagged handbook overview of Graphify — what it is, how the multi-file guide was built, what sources to trust, and the one-paragraph mental model for using it across codebases and personal knowledge bases.
|
||||
tags:
|
||||
- type/reference
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
- domain/llm
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
|
||||
# Graphify — Setup & Best-Practices Guide
|
||||
|
||||
A practical handbook for setting up and using **Graphify** ([safishamsi/graphify](https://github.com/safishamsi/graphify),
|
||||
PyPI package `graphifyy`) across many projects at once — code, documents, and a personal knowledge base.
|
||||
Graphify turns a folder of mixed content into a queryable **knowledge graph** that your AI coding assistant
|
||||
reads instead of re-reading the whole corpus every session.
|
||||
|
||||
The guide is distilled from a creator interview (a marketing/influencer piece — see provenance below),
|
||||
then verified and corrected against the official GitHub repository and supplemented with independent sources.
|
||||
|
||||
## Read in this order
|
||||
|
||||
1. **[01-overview-concepts.md](01-overview-concepts.md)** — What Graphify is and the mental model:
|
||||
knowledge-graph-of-everything, god nodes, confidence tags, neuro-symbolic framing, why a graph beats
|
||||
plain file search and Obsidian. **Start here.**
|
||||
2. **[02-installation-setup.md](02-installation-setup.md)** — Install (`uv tool install graphifyy`),
|
||||
register with your assistant, first `graphify .` run, and how to avoid a first-run token blowout.
|
||||
3. **[03-ingesting-code-ast.md](03-ingesting-code-ast.md)** — Indexing **code** with tree-sitter AST:
|
||||
free, no LLM, 33 languages, multi-repo. The core rule: *AST for code, save the model for documents.*
|
||||
4. **[04-ingesting-docs-knowledge.md](04-ingesting-docs-knowledge.md)** — Indexing **documents & a personal
|
||||
KB**: PDF/docx/xlsx/images, audio/video transcription, YouTube, Google Workspace; when documents need a model.
|
||||
5. **[05-local-models-and-backends.md](05-local-models-and-backends.md)** — **Backends**: local Ollama/SLM
|
||||
vs. cloud (Bedrock, Claude, Gemini, OpenAI, …), env vars, privacy, and choosing on cost/quality/privacy.
|
||||
6. **[06-querying-and-god-nodes.md](06-querying-and-god-nodes.md)** — The highest-leverage skill:
|
||||
ask for **god nodes first**, then scalpel down; *prompt the graph, don't make the LLM read the corpus.*
|
||||
7. **[07-token-economics-and-updates.md](07-token-economics-and-updates.md)** — Where savings really come
|
||||
from (honestly), cost levers, and keeping the graph fresh with `--update` (SHA-256 + dedup) and hooks.
|
||||
8. **[08-workflows-and-use-cases.md](08-workflows-and-use-cases.md)** — End-to-end playbooks: onboarding,
|
||||
bug tracing, AI-slop audits, the cross-project "second brain," PR impact analysis.
|
||||
9. **[09-best-practices-checklist.md](09-best-practices-checklist.md)** — The do/don't reference card +
|
||||
command quick-reference + setting-up-across-many-projects mini-guide. **Keep this one open while working.**
|
||||
10. **[external-tips.md](external-tips.md)** — Independent/community tips, gotchas with issue links, and an
|
||||
even-handed look at the token-savings debate.
|
||||
11. **[10-extraction-model-options.md](10-extraction-model-options.md)** — Why Graphify uses a general
|
||||
structured-output LLM (not a purpose-built KG extractor), the architecture constraints that make
|
||||
drop-in specialist models (Triplex, GLiNER, REBEL) non-starters, and an honest assessment of whether
|
||||
the Triplex adapter route is worth experimenting with.
|
||||
|
||||
## One-paragraph summary
|
||||
|
||||
Install with `uv tool install graphifyy` (the package is `graphifyy` — double-y, a temporary name — but the
|
||||
command is `graphify`), then `graphify install` to register it with Claude Code (or Codex/Cursor/Gemini/OpenCode).
|
||||
Run `graphify .` on a folder; it writes `graphify-out/` containing an interactive `graph.html`, a `GRAPH_REPORT.md`
|
||||
(whose top lists the **god nodes** — the most-connected concepts), and a `graph.json`. **Code** is parsed locally
|
||||
with tree-sitter AST — free, no tokens, 33 languages. **Documents, images, and media** need a model backend for
|
||||
semantic extraction; point that at **local Ollama** to keep it free and private, or at a cloud model. The retrieval
|
||||
discipline that delivers the savings: ask for god nodes first, then query the graph (`graphify query`/`path`/`explain`)
|
||||
rather than dumping the whole corpus into context. Always `graphify ... --update` when adding data so it merges
|
||||
(SHA-256 hashing + dedup) instead of rebuilding. Fold many repos into one graph with `graphify global add`.
|
||||
|
||||
## How this guide was built — provenance & honesty
|
||||
|
||||
This matters because you're going to *run* these commands. Each substantive claim in the docs is tagged inline:
|
||||
|
||||
| Tag | Meaning |
|
||||
|-----|---------|
|
||||
| `[github]` | Verified against the official repository README (see version note below) — **most trustworthy** |
|
||||
| `[interview]` | Stated only in the creator interview — unconfirmed framing or claim |
|
||||
| `[community](url)` | From an independent third-party source, with link |
|
||||
| `[site]` / `[pypi]` | From graphifylabs.ai or the PyPI listing |
|
||||
| `[unverified claim]` | Asserted (often marketing) but not confirmed against a primary source |
|
||||
|
||||
**Sources, and how much to trust them:**
|
||||
|
||||
- **The interview** (`graphify-interview` in the repo root) is a hype/marketing influencer piece with the creator
|
||||
(Safi Shamsi) and noisy auto-transcription. It's a good source of *intent and workflow advice* but a poor source
|
||||
of *facts* — names, commands, and numbers are garbled. We treated it as a starting hypothesis, not ground truth.
|
||||
- **The GitHub repo** is the authority. We anchored on the **v0.8.30 release README** (published 2026-06-02, the
|
||||
latest published release as of 2026-06-03; a `v1.0.0` tag also exists but isn't a published release). Disputed
|
||||
flags were settled by grepping the *raw* README bytes, not summaries.
|
||||
- **The official site** (https://graphifylabs.ai/) returned **HTTP 403** to automated fetches, so nothing was
|
||||
verified directly from it; site-only claims are flagged.
|
||||
|
||||
**Corrections we made to the interview's claims (and why they're in the docs):**
|
||||
|
||||
- **"AST for code, LLM for documents"** holds — but the interview's claim that **shell scripts aren't AST-supported**
|
||||
is outdated; v0.8.30 lists `.sh`/`.bash`/`.ps1` among its 33 tree-sitter languages.
|
||||
- **Package name** is `graphifyy` (double-y), not `graphify`, on PyPI.
|
||||
- **Flag set is version-dependent.** `--token-budget`, `--max-concurrency`, `--api-timeout`, `--budget`, and
|
||||
`--force` are all real in **v0.8.30** (grep-verified) but absent from the shorter `main`-branch README. If your
|
||||
build differs, confirm with `graphify --help`. Note: `--budget` caps *query answer* size; `--token-budget` sets
|
||||
*extraction* chunk size — different flags, different stages.
|
||||
- **"Meetings / Slack / OneNote connectors"** are roadmap or belong to a separate product, **not** shipped in
|
||||
Graphify today.
|
||||
|
||||
**On the headline token-savings numbers (70x / 90x / 71.5x):** treat them as corpus-dependent marketing, not a
|
||||
guarantee. Independent testing (roborhythms) reproduced large-monorepo wins but measured a realistic **~1–49x**,
|
||||
with **net-negative** results on small repos (the mandatory "read the graph first" preamble can cost more than it
|
||||
saves on <100-file projects — see GitHub issue #580). The creator himself says there's "no floor, no ceiling."
|
||||
**Measure your own with `graphify benchmark`.** (Separately, the repo's *popularity* claims under-sold reality:
|
||||
third-party trackers showed ~58K stars and ~1.28M downloads as of 2026-06-03 — bigger than the interview's figures.)
|
||||
|
||||
**Caveat — Graphify moves fast.** Multiple releases shipped *per day* during early June 2026. Commands and flags
|
||||
here reflect v0.8.30; if something doesn't match, run `graphify --help` and prefer the latest release README.
|
||||
|
||||
_Last updated: 2026-06-03 · anchored to Graphify v0.8.30_
|
||||
|
|
@ -0,0 +1,142 @@
|
|||
---
|
||||
summary: Why Graphify uses a general structured-output LLM (not a purpose-built KG model), the fixed prompt/schema contract, the qwen2.5-coder:7b default and its num_ctx gotcha, and an honest assessment of the Triplex adapter route.
|
||||
tags:
|
||||
- type/reference
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
- domain/llm
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
# 10 — Extraction Model Options: Why a General Structured-Output LLM, and Is a Purpose-Built KG Model Worth It?
|
||||
|
||||
_Last updated: 2026-06-05_ | _Status: reference / decision record_
|
||||
|
||||
This document explains why Graphify's document extraction uses a general instruction-following / structured-output LLM rather than a purpose-built knowledge-graph extraction model (e.g. Triplex, GLiNER, REBEL), and honestly assesses whether the purpose-built route is worth experimenting with for this project.
|
||||
|
||||
See also: [05 — Local models & backends](05-local-models-and-backends.md) · [Benchmark results (2026-06-04)](../memory-system/benchmark/scoring-results-2026-06-04.md)
|
||||
|
||||
---
|
||||
|
||||
## The architecture constraint (this is the crux)
|
||||
|
||||
Graphify owns **both** the prompt and the output contract. It is not a thin wrapper you can swap a model into — the extraction pipeline is opinionated end-to-end:
|
||||
|
||||
**Fixed system prompt.** For document/markdown extraction, Graphify sends its own system prompt `_EXTRACTION_SYSTEM` (`llm.py:218-232`). `[github]` There is no user hook to supply a custom extraction prompt; the prompt is hardcoded.
|
||||
|
||||
**Fixed output schema.** The model must return exactly this JSON structure — no approximation accepted:
|
||||
|
||||
```json
|
||||
{
|
||||
"nodes": [{"id": "stem_entity", "label": "...", "file_type": "code|document|paper|image|rationale|concept",
|
||||
"source_file": "...", "source_location": null, "source_url": null,
|
||||
"captured_at": null, "author": null, "contributor": null}],
|
||||
"edges": [{"source": "node_id", "target": "node_id",
|
||||
"relation": "calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to",
|
||||
"confidence": "EXTRACTED|INFERRED|AMBIGUOUS",
|
||||
"confidence_score": 1.0, "source_file": "...", "source_location": null, "weight": 1.0}],
|
||||
"hyperedges": [], "input_tokens": 0, "output_tokens": 0
|
||||
}
|
||||
```
|
||||
|
||||
The `relation` field is constrained to a fixed vocabulary of seven values. The `confidence` field is a three-value enum (`EXTRACTED`/`INFERRED`/`AMBIGUOUS`). `[github]` (`llm.py:218-232`)
|
||||
|
||||
**No JSON-mode / grammar enforcement.** For the Ollama backend, Graphify passes only `{"options": {"num_ctx": ...}, "keep_alive": ...}` as extra parameters (`llm.py:506`). `[github]` There is no `response_format`, grammar constraint, or structured-output enforcement. Schema compliance rests entirely on the model's instruction-following ability.
|
||||
|
||||
**Lenient but not magic parser.** The response goes through `_parse_llm_json` (`llm.py:269-303`), which strips optional markdown fences and then tries `json.loads`. If that fails, it scans for the first balanced JSON object in the text. A model that emits its own schema — triples, spans, BIO tags — will produce an empty or nonsense graph, not a graceful fallback. `[github]`
|
||||
|
||||
**Temperature forced to 0 for Ollama.** The Ollama backend config sets `"temperature": 0` (`llm.py:65-73`). `[github]`
|
||||
|
||||
**Deep mode.** `--mode deep` appends `_DEEP_EXTRACTION_SUFFIX` to the same prompt, asking for extra `INFERRED` edges with a more conservative framing (`llm.py:234-247`). `[github]` The schema stays identical.
|
||||
|
||||
---
|
||||
|
||||
## The shipped default: why a coder model?
|
||||
|
||||
The Ollama fallback default is `qwen2.5-coder:7b` (`llm.py:67`): `[github]`
|
||||
|
||||
```python
|
||||
"default_model": os.environ.get("OLLAMA_MODEL", "qwen2.5-coder:7b"),
|
||||
```
|
||||
|
||||
Even though Graphify sends *documents* (not code) for extraction, the maintainer chose a coder/structured-output-tuned model. This is a revealed preference: the binding requirement is **structured-JSON instruction-following discipline**, not domain entity recognition. A coder/instruct model is the right axis; a NER specialist is the wrong one; frontier reasoning is overkill.
|
||||
|
||||
**Config knobs for the Ollama backend** (all `[github]` from README env-var table unless noted):
|
||||
|
||||
| Knob | How to set | What it does |
|
||||
|---|---|---|
|
||||
| `OLLAMA_MODEL` | env var | Override the `qwen2.5-coder:7b` default |
|
||||
| `OLLAMA_BASE_URL` | env var | Must end in `/v1` (e.g. `http://127.0.0.1:11434/v1`) — any other suffix produces 404s on every call |
|
||||
| `GRAPHIFY_OLLAMA_NUM_CTX` | env var | **See GOTCHA below** |
|
||||
| `GRAPHIFY_OLLAMA_KEEP_ALIVE` | env var | Minutes to keep model resident; default `30m`; set `0` to unload after each chunk |
|
||||
| `--token-budget` | CLI flag | Per-chunk input cap (tokens); pack multiple files per chunk |
|
||||
| `--max-concurrency` | CLI flag | Set 1–2 for a single local GPU |
|
||||
| `--mode deep` | CLI flag | Appends deep-inference suffix; elicits more INFERRED edges |
|
||||
|
||||
**GOTCHA — num_ctx does not propagate through graphify (verified 2026-06-04 on this project).** `[github]` Graphify posts to Ollama's OpenAI-compatible `/v1/chat/completions` endpoint. That endpoint **silently ignores** the per-request `options.num_ctx` that Graphify sends (`llm.py:506`). Proven by A/B: a POST to `/v1/chat/completions` with `num_ctx=8192` left `ollama ps` showing CONTEXT=4096; the same value through the native `/api/chat` endpoint was honoured. Therefore `GRAPHIFY_OLLAMA_NUM_CTX` has **no effect** through Graphify — context pins at Ollama's `/v1` default of 4096. At 4096 tokens, Graphify's extraction output JSON is truncated mid-response (`finish_reason=length`) and the chunk is **discarded**, producing an empty graph.
|
||||
|
||||
**Workaround (validated):** bake context into a Modelfile variant:
|
||||
|
||||
```bash
|
||||
# Create a 16 k-context variant of qwen2.5-coder:7b:
|
||||
cat <<'EOF' | ollama create qwen25-coder-7b-16k -f -
|
||||
FROM qwen2.5-coder:7b
|
||||
PARAMETER num_ctx 16384
|
||||
EOF
|
||||
OLLAMA_MODEL=qwen25-coder-7b-16k graphify extract ./docs --backend ollama
|
||||
```
|
||||
|
||||
The `/v1` endpoint honours the model's baked-in default. Non-invasive: no sudo, no systemd restart; reversible with `ollama rm`. Full investigation: `docs/memory-system/benchmark/scoring-results-2026-06-04.md`.
|
||||
|
||||
**Local patch — thinking mode disabled (applied 2026-06-04):** `reasoning_effort: "none"` was added at `llm.py:71` in the installed binary to suppress thinking output for any thinking-capable model. The original is backed up at `/tmp/graphify-bench/llm.py.orig`. This patch is a **no-op for `qwen2.5-coder:7b`** (no thinking mode) but will be **lost on `pip install --upgrade`** and would need re-applying for any future thinking-capable candidate. Needs a production path: upstream PR or a maintained local wrapper. `[github]`
|
||||
|
||||
---
|
||||
|
||||
## Purpose-built KG / entity / relation models — and why they don't drop in
|
||||
|
||||
Every purpose-built extraction model assumes **it** owns the prompt and output contract. Graphify owns both. That's the structural incompatibility.
|
||||
|
||||
| Model | Architecture | Ollama-pullable? | Follows Graphify's fixed prompt + schema? | Verdict |
|
||||
|---|---|---|---|---|
|
||||
| **SciPhi/Triplex** (Phi-3-3.8B, fine-tuned for KG triples) | Autoregressive causal LM | Yes (`ollama pull sciphi/triplex`) | No — locked to its own `{entity_types}/{predicates}` input template; emits subject–predicate–object triples, not Graphify's nodes/edges JSON | Can't drop in as-is |
|
||||
| **GLiNER** (lightweight NER) | BERT-like bidirectional encoder | No | No — NER-only (span outputs); cannot emit arbitrary JSON or follow a system prompt | Wrong architecture |
|
||||
| **REBEL / Relik** (relation extraction) | Seq2seq BART with special-token triples | No | No — emits `<triplet>/<subj>/<obj>` tokens; custom parsing required | Wrong architecture |
|
||||
|
||||
Sources: HuggingFace SciPhi/Triplex model card, `ollama.com/library/sciphi/triplex`, GitHub `urchade/GLiNER`, GitHub `Babelscape/rebel`. `[unverified claim]` — line numbers in those repos were not pinned.
|
||||
|
||||
---
|
||||
|
||||
## Is the purpose-built route worth experimenting with?
|
||||
|
||||
**User hypothesis:** if the JSON-shape mismatch is the only blocker, write a shim converting Triplex's triple output into Graphify's schema with a custom prompt. Triplex is marketed as very low-cost/fast (small 3.8B Phi-3 model).
|
||||
|
||||
What it would actually take `[speculative]`:
|
||||
|
||||
1. **A model-specific prompt path.** Triplex requires its `{entity_types}/{predicates}` template, not `_EXTRACTION_SYSTEM`. Graphify currently hard-codes one system prompt; a Triplex adapter needs to inject a different one for that model (or backend).
|
||||
2. **A model-specific output parser.** Convert subject–predicate–object triples into Graphify's `nodes` + `edges` JSON: assign `file_type` to synthesized nodes, map predicates onto Graphify's fixed relation vocabulary (or accept free-text), synthesize `confidence` since Triplex emits no `EXTRACTED`/`INFERRED`/`AMBIGUOUS` enum, and reconstruct per-file provenance fields (`source_file`, `source_location`).
|
||||
3. **A new backend/model branch in `llm.py`.** This is not a one-line patch — it is a backend adapter (prompt + parser) of moderate size.
|
||||
|
||||
**Architecturally consistent.** Graphify already has model-specific branches: the Kimi/moonshot backend sets `extra_body={"thinking": {"type": "disabled"}}` (`llm.py:461-462`) `[github]` to suppress reasoning output. A Triplex adapter follows the same pattern and could in principle be an upstream PR.
|
||||
|
||||
**Quality unknowns `[speculative]`.** Triplex targets generic NER + triples. Graphify's design wants typed nodes, a fixed relation vocabulary, EXTRACTED/INFERRED/AMBIGUOUS confidence, and per-file provenance. None of those would be native outputs from Triplex — they'd be synthesized, lowering fidelity compared to a model that follows Graphify's schema directly.
|
||||
|
||||
**Cost/speed `[speculative]`.** Triplex is plausibly faster per token (3.8B vs 7B), but `qwen2.5-coder:7b` at Q4 on a 12GB GPU is already rapid. The adapter effort isn't justified unless the default model proves too slow or too heavy for the freshness budget.
|
||||
|
||||
**Verdict `[speculative]`:** Realistic but moderate effort. Worth it **only if**: (a) `qwen2.5-coder:7b` proves too slow or too heavy for the project's freshness budget on the full vault, **and** (b) a quick Triplex spike shows acceptable triple quality on the vault fixtures. Neither condition is known yet. **PARKED — revisit after the qwen2.5-coder:7b benchmark result is in hand.**
|
||||
|
||||
**Cheap first probe:** `ollama run sciphi/triplex` on one fixture note using Triplex's native `{entity_types}/{predicates}` template. If the triples are coherent and well-typed, the adapter is worth scoping. If they are sparse or incorrectly typed, don't bother.
|
||||
|
||||
---
|
||||
|
||||
## Why qwen2.5-coder:7b is the right axis, not a domain specialist
|
||||
|
||||
To be concrete: the extractors that do better than `qwen2.5-coder:7b` at Graphify's actual task are models that are **better at structured-JSON instruction following**, not models that are better at entity recognition or triple extraction in isolation. Relevant axis: instruction-following + JSON discipline. Irrelevant axis: NER F1, triple-completeness on academic benchmarks. `[speculative]` (Based on observed failure modes: models that failed in the 2026-06-04 benchmark failed by producing invalid JSON or consuming the output budget with thinking tokens — not by extracting the wrong entities.)
|
||||
|
||||
---
|
||||
|
||||
## Pointers
|
||||
|
||||
- Full local-model investigation + the num_ctx/thinking findings: `docs/memory-system/benchmark/scoring-results-2026-06-04.md`
|
||||
- Backend config details and Ollama setup: `docs/graphify/05-local-models-and-backends.md`
|
||||
- Graphify source (installed): `~/.local/lib/python3.14/site-packages/graphify/llm.py` — anchored to 0.8.31
|
||||
|
|
@ -0,0 +1,212 @@
|
|||
---
|
||||
summary: How Graphify uses tree-sitter ASTs to extract code structure locally for free — 33 supported languages, call-graph pass, incremental updates, and multi-repo global graphs.
|
||||
tags:
|
||||
- type/howto
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
# Ingesting Code with the AST (Free, No LLM)
|
||||
|
||||
How Graphify turns a codebase into a graph using tree-sitter abstract syntax trees (ASTs) — a deterministic, local, zero-API-cost pass. This is the cheapest thing Graphify does, and the interview's core rule is to lean on it: **use the AST for code, and save the LLM for documents.**
|
||||
|
||||
> Related: [01 Overview & Concepts](./01-overview-concepts.md) · [02 Installation & Setup](./02-installation-setup.md) · [04 Ingesting Docs & Knowledge](./04-ingesting-docs-knowledge.md) · [06 Querying & God Nodes](./06-querying-and-god-nodes.md) · [07 Token Economics & Updates](./07-token-economics-and-updates.md)
|
||||
|
||||
---
|
||||
|
||||
## The core rule: AST for code, LLM for documents
|
||||
|
||||
The creator states this plainly, more than once: parsing a codebase does **not** call an LLM, so don't pay for one.
|
||||
|
||||
> "The main problem if you are running... an LLM to parse your codebase, that doesn't make sense at all... Abstract syntax trees. It's a free Python library which can connect various parts of a codebase like functions... and create relationships between them in different files even in multiple repository. So it's free of cost." `[interview]`
|
||||
|
||||
Verified against the repo: the first pass is a deterministic tree-sitter walk over every code file with **no LLM, no embeddings, no network** — AST nodes are converted directly into graph nodes and edges. `[github]`
|
||||
|
||||
What this means in practice:
|
||||
|
||||
- Running `graphify .` on a pure code repo costs **nothing** — no API key needed, no token burn, runs entirely on your machine. `[github]`
|
||||
- You only spend tokens (cloud or local) on **non-code** content: docs, PDFs, transcripts, images. See [04 Ingesting Docs & Knowledge](./04-ingesting-docs-knowledge.md) and [05 Local Models & Backends](./05-local-models-and-backends.md).
|
||||
- The interview anecdote of "it ran through my whole daily limit" comes from letting an LLM touch the code path. Index code first (free), then decide what documents are worth an LLM call. `[interview]`
|
||||
|
||||
---
|
||||
|
||||
## How tree-sitter AST extraction works (high level)
|
||||
|
||||
[tree-sitter](https://tree-sitter.github.io/tree-sitter/) is a parser library that turns source files into a concrete syntax tree. Graphify walks that tree to pull out the structural facts of your code.
|
||||
|
||||
> "Tree-sitter parses your code files and extracts classes, functions, imports, call graphs, and inline comments. This runs locally with no LLM involved." `[github]`
|
||||
|
||||
The pipeline, as documented:
|
||||
|
||||
1. **Walk every code file** with tree-sitter (deterministic, local). `[github]`
|
||||
2. **Extract symbols** — classes, functions, imports, inline comments — and turn each into a graph **node** with an identifier and label. `[github]`
|
||||
3. **Call-graph pass** — a follow-up pass that links the nodes into **edges**: function `calls`, file `imports`, implementations, and other typed relationships across files. `[github]`
|
||||
4. **Tag every relationship** with a confidence label so you can tell code-derived facts from inferred ones:
|
||||
- `EXTRACTED` — found directly in the source (this is what the AST path produces). `[github]`
|
||||
- `INFERRED` — model inference, with a confidence score (~0.55–0.95). `[github]`
|
||||
- `AMBIGUOUS` — uncertain, flagged for manual review. `[github]`
|
||||
|
||||
For a pure AST run with no LLM, relationships are `EXTRACTED`. `INFERRED`/`AMBIGUOUS` show up when an LLM enriches the graph (deep mode / documents).
|
||||
|
||||
### What gets linked into the graph
|
||||
|
||||
| Source artifact | Becomes | Linked by |
|
||||
| --- | --- | --- |
|
||||
| File | Node | `imports` edges to other files `[github]` |
|
||||
| Class / function | Node | `calls` edges (call graph), `implements` edges `[github]` |
|
||||
| Import statement | Edge | connects the importing file/symbol to its target `[github]` |
|
||||
| Inline comments | Captured with the symbol | provide context for later querying `[github]` |
|
||||
|
||||
The most-connected nodes surface as **god nodes** — the densest hubs in the graph, your fastest way to read an unfamiliar architecture. Query them first. Details in [06 Querying & God Nodes](./06-querying-and-god-nodes.md). `[github]` `[interview]`
|
||||
|
||||
---
|
||||
|
||||
## Supported languages (33, via tree-sitter)
|
||||
|
||||
The current release (v8) lists **33 languages**. Verified verbatim from the v8 code-extension table: `[github]`
|
||||
|
||||
```
|
||||
.py .ts .js .jsx .tsx .mjs .go .rs .java .c .cpp .h .hpp .rb .cs .kt
|
||||
.scala .php .swift .lua .luau .zig .ps1 .ex .exs .m .mm .jl .vue
|
||||
.svelte .astro .groovy .gradle .dart .v .sv .svh .sql .f .f90 .f95
|
||||
.f03 .f08 .pas .pp .dpr .dpk .lpr .inc .dfm .lfm .lpk .sh .bash .json
|
||||
.dm .dme .dmi .dmm .dmf .sln .csproj .fsproj .vbproj .razor .cshtml
|
||||
```
|
||||
|
||||
By language family, that covers (non-exhaustive, all `[github]`):
|
||||
|
||||
- **Mainstream:** Python, TypeScript/JavaScript (+ JSX/TSX/MJS), Go, Rust, Java, C/C++, Ruby, C#, Kotlin, Scala, PHP, Swift, Dart.
|
||||
- **Systems / scientific:** Zig, Lua/Luau, Julia, Fortran (`.f` … `.f08`), MATLAB/Objective-C (`.m`/`.mm`), Verilog/SystemVerilog (`.v`/`.sv`/`.svh`).
|
||||
- **Web frameworks:** Vue, Svelte, Astro, Razor/cshtml.
|
||||
- **JVM build / scripting:** Groovy, Gradle.
|
||||
- **Data / schema:** **SQL** schemas, JSON.
|
||||
- **Shell:** **`.sh`, `.bash`, and PowerShell `.ps1`** — see the note below.
|
||||
- **Project files:** `.sln`, `.csproj`, `.fsproj`, `.vbproj`.
|
||||
- **Niche:** Pascal/Delphi (`.pas`/`.dpr`/…), BYOND DreamMaker (`.dm`/`.dme`/…).
|
||||
|
||||
> The repo's own tagline confirms the breadth: it turns "any folder of code, SQL schemas, R scripts, shell scripts, docs, papers, images, or videos into a queryable knowledge graph." `[github]`
|
||||
|
||||
### Shell scripts: now supported by AST (interview was outdated)
|
||||
|
||||
This is a place where the **GitHub repo supersedes the interview**, so read carefully if you watched the video.
|
||||
|
||||
In the interview, the creator said shell scripts were *not* handled by the AST path and prescribed a workaround:
|
||||
|
||||
> "Shell scripts are very flat... rather than going with an A[ST] which isn't capab[le] at the moment for shell scripts, what you can do is just put those as documents and then make the LLM do a semantic extraction of a shell script... That's a temporary fix. I'll get it sorted maybe in coming releases." `[interview]`
|
||||
|
||||
As of the current release (v8), that fix has shipped: `.sh`, `.bash`, and `.ps1` are in the code-extension table and are parsed via tree-sitter like any other language — locally, no API call. `[github]` So the interview's workaround is **no longer necessary for shell scripts**; just run `graphify .` and they'll be extracted for free.
|
||||
|
||||
### Fallback for any unsupported language
|
||||
|
||||
The document-ingestion trick the creator described is still useful — just reframe it for **any file whose language isn't in the 33** (or any code so unusual the AST extraction is thin). Instead of forcing it through the AST path, ingest it as a **document** so an LLM does semantic extraction of its structure:
|
||||
|
||||
- Point Graphify at it as document content (see [04 Ingesting Docs & Knowledge](./04-ingesting-docs-knowledge.md)) and let a local Ollama model or a cloud model pull out nodes. `[interview]`
|
||||
- This costs tokens (it's an LLM call), so reserve it for the handful of files the AST genuinely can't read. For the 33 supported languages, always prefer the free AST path. `[interview]`
|
||||
|
||||
---
|
||||
|
||||
## Indexing a repo: `graphify .`
|
||||
|
||||
Run it from the repo root. `[github]`
|
||||
|
||||
```bash
|
||||
# index the current directory (free, AST-only for code)
|
||||
graphify .
|
||||
|
||||
# or a specific folder
|
||||
graphify ./services/api
|
||||
```
|
||||
|
||||
This writes three artifacts you and your team can query: `[github]`
|
||||
|
||||
- `graph.html` — interactive visualization.
|
||||
- `graph.json` — the queryable graph.
|
||||
- `GRAPH_REPORT.md` — human-readable audit (includes god nodes and confidence tags).
|
||||
|
||||
> Install with `uv tool install graphifyy` (or `pipx install graphifyy`). Full setup, including how to wire it into Claude Code, is in [02 Installation & Setup](./02-installation-setup.md). `[github]`
|
||||
|
||||
After indexing, query the graph — and ask for **god nodes first** before drilling in. That keeps token use minimal on the retrieval side. See [06 Querying & God Nodes](./06-querying-and-god-nodes.md). `[interview]`
|
||||
|
||||
---
|
||||
|
||||
## Re-indexing after changes: `--update`
|
||||
|
||||
Always use `--update` when re-ingesting an already-indexed repo. It avoids rebuilding from scratch.
|
||||
|
||||
```bash
|
||||
graphify . --update
|
||||
```
|
||||
|
||||
> "I've used hashing strategy [SHA]256 in here, so it will start from where it left... make sure you use the word update as well, `graphify update`, so that it doesn't start fresh from the beginning." `[interview]`
|
||||
|
||||
Verified mechanics: Graphify fingerprints each extracted file with **SHA256 content hashing** and caches under `graphify-out/cache/`. On a re-run with `--update`, unchanged files are skipped entirely; only new/modified files are re-extracted. There are also de-duplication techniques to minimize duplicate entities across files. `[github]` `[interview]`
|
||||
|
||||
More on incremental updates and the token math in [07 Token Economics & Updates](./07-token-economics-and-updates.md).
|
||||
|
||||
---
|
||||
|
||||
## Auto-rebuild on commit: `graphify hook install`
|
||||
|
||||
To keep the graph fresh without remembering to re-run it, install the git hooks:
|
||||
|
||||
```bash
|
||||
graphify hook install
|
||||
```
|
||||
|
||||
This sets up **post-commit and post-checkout** hooks that rebuild the graph automatically — and because it's the AST path for code, there's no API cost. `[github]` This directly addresses the interview's "my assets become stale over time" concern: the graph stays current as you commit. `[interview]`
|
||||
|
||||
---
|
||||
|
||||
## Many repos / a whole enterprise in one graph
|
||||
|
||||
The AST path links symbols **across files and across repositories**, so an entire codebase — or a whole company's set of repos — can live in one graph. `[interview]`
|
||||
|
||||
> "It can connect... functions and all of those sections of a code all together and create relationships between them in different files even in multiple repository." `[interview]`
|
||||
|
||||
The verified mechanism is the **`graphify global`** command family, which maintains a cross-project index at `~/.graphify/global.json`: `[github]`
|
||||
|
||||
```bash
|
||||
# 1) index each repo individually (free, AST)
|
||||
cd /path/to/repo-a && graphify .
|
||||
cd /path/to/repo-b && graphify .
|
||||
|
||||
# 2) register each repo's graph into the global cross-project graph
|
||||
graphify global add graphify-out/graph.json repo-a
|
||||
graphify global add graphify-out/graph.json repo-b
|
||||
|
||||
# 3) inspect / manage the global graph
|
||||
graphify global list # shows registered repos with node + edge counts
|
||||
graphify global remove repo-a # unregister a repo
|
||||
```
|
||||
|
||||
This is the "digital twin of your whole enterprise" use case from the interview: a new senior hire can read the architecture in one shot instead of spending days and Slack threads spelunking. `[interview]` See [08 Workflows & Use Cases](./08-workflows-and-use-cases.md).
|
||||
|
||||
---
|
||||
|
||||
## A concrete first run
|
||||
|
||||
```bash
|
||||
# from a repo you've never indexed
|
||||
graphify . # free AST extraction of all 33 supported languages
|
||||
|
||||
# read the architecture: open graph.html, skim GRAPH_REPORT.md,
|
||||
# then query for the god nodes first (see doc 06)
|
||||
graphify query "what are the god nodes?"
|
||||
|
||||
# keep it fresh
|
||||
graphify hook install # auto-rebuild on every commit
|
||||
# ...or manually after a batch of changes:
|
||||
graphify . --update
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Open questions / unverified
|
||||
|
||||
- **Exact per-language extraction depth.** The 33-language list is verified, but how thoroughly each language's call graph is resolved (e.g., dynamic dispatch in Ruby/Python, macro-heavy Rust) is not documented. `[unverified claim]`
|
||||
- **`--update` vs. a bare `graphify global` re-add.** Whether updating a member repo automatically propagates into an already-registered global graph, or requires re-running `graphify global add`, isn't spelled out in the docs I checked. `[unverified claim]`
|
||||
- **`graphify global path` semantics.** A `graphify global path` command appears in the repo listing, but its exact arguments/behavior for cross-repo path queries weren't documented in the sources read. `[github]` (command exists) / `[unverified claim]` (usage)
|
||||
- **Branch/version skew.** The `main` README I first fetched showed only ~13 languages and an older `pip install graphifyy && graphify install` flow; the **v8** branch shows the full 33 languages, shell support, and `uv tool install`. This doc follows **v8** (newest). If you're on an older install, your supported-language list and command surface may be narrower — check `graphify --help`. `[github]`
|
||||
- **R scripts.** The repo tagline advertises "R scripts," but `.r`/`.R` does not appear in the v8 code-extension table I verified — R may be handled via the document path rather than AST. `[unverified claim]`
|
||||
|
|
@ -0,0 +1,209 @@
|
|||
---
|
||||
summary: How to ingest non-code content (docs, PDFs, images, video, Google Workspace) into a Graphify knowledge graph, including format support, backends, chunking, and personal KB organization patterns.
|
||||
tags:
|
||||
- type/howto
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
# 04 — Ingesting Documents & a Personal Knowledge Base
|
||||
|
||||
How to turn notes, PDFs, spreadsheets, images, audio/video, and cloud docs into a queryable Graphify graph. Unlike code (free, local AST), documents need an LLM or local SLM to extract meaning — so this is the part of Graphify that costs tokens (or GPU time), and where organization and chunking matter most.
|
||||
|
||||
> **Provenance:** Claims are tagged `[github]` (verified against the v8 repo README/docs on 2026-06-03), `[interview]` (from the creator interview — treat as intent/sales claims), `[community](url)`, or `[unverified claim]`. Never assume a flag exists because it sounds plausible — every command below was copied from the verified README unless tagged otherwise.
|
||||
|
||||
Related docs: [code/AST ingest](03-ingesting-code-ast.md) · [local models & backends](05-local-models-and-backends.md) · [querying & god nodes](06-querying-and-god-nodes.md) · [token economics & updates](07-token-economics-and-updates.md) · [workflows](08-workflows-and-use-cases.md).
|
||||
|
||||
---
|
||||
|
||||
## The one rule that governs cost
|
||||
|
||||
Graphify runs a multi-pass pipeline. The split that matters for your wallet: **code is extracted locally with no API calls (AST via tree-sitter); everything else goes through your AI assistant's model API.** `[github]`
|
||||
|
||||
- **Code files** — processed locally via tree-sitter. Nothing leaves your machine, no tokens spent. `[github]` (see [03-ingesting-code-ast.md](03-ingesting-code-ast.md))
|
||||
- **Video / audio** — transcribed locally with faster-whisper. Nothing leaves your machine. `[github]`
|
||||
- **Docs, PDFs, images** — sent to your AI assistant for semantic extraction. `[github]`
|
||||
|
||||
In the creator's words: *"You have to call the LLM backend only for your documents… [for code] there's no API call included."* `[interview]` So the practical takeaway is: code is free to graph; **documents always need a backend** (a cloud LLM or a local SLM via Ollama). Pick the backend deliberately — that decision and its cost trade-offs live in [05-local-models-and-backends.md](05-local-models-and-backends.md) and [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
|
||||
|
||||
---
|
||||
|
||||
## Supported non-code formats
|
||||
|
||||
Verbatim from the v8 format table `[github]`:
|
||||
|
||||
| Type | Extensions | Notes |
|
||||
|------|-----------|-------|
|
||||
| Docs | `.md .mdx .qmd .html .txt .rst .yaml .yml` | Plain text / markup |
|
||||
| Office | `.docx .xlsx` | requires `uv tool install "graphifyy[office]"` |
|
||||
| Google Workspace | `.gdoc .gsheet .gslides` | opt-in; requires `gws` auth + `--google-workspace`; Sheets need `graphifyy[google]` |
|
||||
| PDFs | `.pdf` | requires `uv tool install "graphifyy[pdf]"` |
|
||||
| Images | `.png .jpg .webp .gif` | vision-based extraction (screenshots, diagrams) `[unverified claim]` on per-language fidelity |
|
||||
| Video / Audio | `.mp4 .mov .mp3 .wav` and more | requires `uv tool install "graphifyy[video]"` (faster-whisper + yt-dlp) |
|
||||
| YouTube / URLs | any video URL | requires `uv tool install "graphifyy[video]"` |
|
||||
|
||||
Optional extras are installed once and add format support; install only what you need:
|
||||
|
||||
```bash
|
||||
uv tool install "graphifyy[pdf]" # PDF extraction
|
||||
uv tool install "graphifyy[office]" # .docx and .xlsx
|
||||
uv tool install "graphifyy[video]" # video/audio transcription (faster-whisper + yt-dlp)
|
||||
uv tool install "graphifyy[google]" # Google Sheets table rendering
|
||||
```
|
||||
|
||||
`uv tool install` installs a single tool, so each line above *replaces* the previous install. To get several extras at once, combine them in one bracket (comma-separated) or grab everything:
|
||||
|
||||
```bash
|
||||
uv tool install "graphifyy[pdf,office,video,google]" # combine extras in one install
|
||||
uv tool install "graphifyy[all]" # everything
|
||||
```
|
||||
|
||||
> Note the PyPI package is `graphifyy` (double-y) while the command is `graphify`. `[github]` See [02-installation-setup.md](02-installation-setup.md).
|
||||
|
||||
---
|
||||
|
||||
## Two ways to run: in-IDE skill vs. headless CLI
|
||||
|
||||
This trips people up, so be explicit. Graphify has two invocation forms, and they are **not** interchangeable: `[github]`
|
||||
|
||||
- **In-IDE skill** — `/graphify ...` (slash command). Runs inside your Claude Code / Cursor / Gemini CLI session and uses *whatever model that session runs* for document extraction. This is the path the interview mostly describes.
|
||||
- **Headless CLI** — `graphify extract ...`. Runs from a terminal and needs you to choose a backend explicitly (an API key, a running Ollama instance, or the `claude` CLI binary).
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
# In-IDE skill (uses your session's model for docs)
|
||||
/graphify ./docs
|
||||
/graphify ./docs --update
|
||||
|
||||
# Headless CLI (you pick the backend)
|
||||
graphify extract ./docs --backend ollama
|
||||
graphify extract ./docs --backend gemini
|
||||
```
|
||||
|
||||
For everything that follows, the slash form and the `graphify extract` form do the same ingestion — they differ only in *where the model comes from*.
|
||||
|
||||
---
|
||||
|
||||
## Audio & video transcription (Whisper)
|
||||
|
||||
The interview calls it "whisper"; the shipped implementation is **faster-whisper**, run **locally**. `[github]` Audio and video are transcribed on your machine before any text is sent anywhere, so the transcription step itself is private and free of API tokens (the resulting transcript text is then treated like a document and does hit your backend). `[github]`
|
||||
|
||||
One nice detail: the transcription prompt is seeded with your **top god nodes** (the most-connected concepts in your graph so far), which biases Whisper toward your domain vocabulary. `[github]`
|
||||
|
||||
```bash
|
||||
uv tool install "graphifyy[video]" # one-time: installs faster-whisper + yt-dlp
|
||||
/graphify ./media # transcribe local .mp4/.mov/.mp3/.wav, then graph
|
||||
```
|
||||
|
||||
### YouTube / lecture transcripts
|
||||
|
||||
The interview's headline KB example — *"put that link, get the lecture's transcription, and then you can have the map of the lecture with each section divided into various graphs"* `[interview]` — is shipped. Add a single video by URL: `[github]`
|
||||
|
||||
```bash
|
||||
/graphify add <youtube-url> # transcribe and add a video
|
||||
/graphify add https://arxiv.org/abs/1706.03762 # also works for papers/tweets
|
||||
```
|
||||
|
||||
This downloads via `yt-dlp`, transcribes locally with faster-whisper, and folds the transcript into the graph as document nodes. `[github]`
|
||||
|
||||
---
|
||||
|
||||
## Google Workspace connector
|
||||
|
||||
Pull native Google Docs, Sheets, and Slides into a graph. Important gotcha called out in the README: Google Drive for desktop `.gdoc`/`.gsheet`/`.gslides` files are **shortcut pointers, not content** — so you must authenticate the `gws` CLI to fetch the real documents. `[github]`
|
||||
|
||||
```bash
|
||||
uv tool install "graphifyy[google]" # needed for Google Sheets table rendering
|
||||
gws auth login -s drive # authenticate the gws CLI (opens a browser, approve scopes)
|
||||
graphify extract ./docs --google-workspace
|
||||
```
|
||||
|
||||
Equivalently, set `GRAPHIFY_GOOGLE_WORKSPACE=1` instead of passing the flag. `[github]` Under the hood, Graphify exports the shortcuts into `graphify-out/converted/` as Markdown sidecars, then extracts those. `[github]` The `gws` CLI itself is a separate Google tool ([googleworkspace/cli](https://github.com/googleworkspace/cli)). `[github]`
|
||||
|
||||
> Because these are documents, the same backend rule applies: a Workspace ingest spends tokens (or local SLM time). `[github]`
|
||||
|
||||
---
|
||||
|
||||
## Roadmap connectors — NOT yet shipped
|
||||
|
||||
The interview previews several connectors. As of the v8 repo (2026-06-03) **none of these are in Graphify** — treat them as roadmap until a release confirms otherwise: `[interview]`
|
||||
|
||||
- **Slack messages** — *"you can connect your Slack messages… as a map"* `[interview]`. Not in the docs. `[github]` (absent)
|
||||
- **OneNote** ("one node" in the transcript, likely OneNote) — *"even your one node"* `[interview]`. Not in the docs. `[github]` (absent)
|
||||
- **Meeting transcripts** — *"even your meetings, the meeting transcripts… everything will be in a brain"* `[interview]`. Not in Graphify. Note: the repo mentions **Penpax**, a *separate* always-on product built on top of Graphify that covers "meetings, browser history, emails, files, and code" `[github]` — so meeting capture appears to be a Penpax feature, not core Graphify.
|
||||
|
||||
If you need any of these today, the interview's own workaround applies: export to a supported document format (e.g. a Slack export or a `.txt`/`.md` transcript) and ingest it as a regular doc. `[interview]`
|
||||
|
||||
---
|
||||
|
||||
## Organizing a mixed personal knowledge base
|
||||
|
||||
A personal KB is usually a pile of notes + PDFs + media + cloud docs. A few practical principles:
|
||||
|
||||
**1. Group by graph, not by one giant folder.** The interview's strongest cost/quality advice: *"you can split your repository or your documents… and reingest them rather than putting the whole chunk of documents in one shot."* `[interview]` And: *"the more the context… there will be a massive chance of hallucination or dilution in the reasoning."* `[interview]` So build several focused graphs (e.g. `research/`, `meeting-notes/`, `personal-wiki/`) rather than one undifferentiated dump. You can later combine graphs: `graphify merge-graphs a.json b.json`. `[github]`
|
||||
|
||||
**2. Keep code and docs in separate runs where you can.** Code is free (AST); docs cost. Splitting them makes it obvious where your tokens go and lets you re-run docs without re-touching code.
|
||||
|
||||
**3. Build incrementally with `--update`.** Don't re-ingest from scratch each time. `[github]` `[interview]`
|
||||
|
||||
```bash
|
||||
/graphify ./docs --update # re-extract only changed files, merge into existing graph
|
||||
```
|
||||
|
||||
Graphify uses SHA-256 hashing + de-duplication so unchanged files are skipped and entities don't collide across documents. `[interview]` `[github]` Details and economics in [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
|
||||
|
||||
**4. Query god nodes first.** Once a KB is graphed, *"always go to the god nodes first and then start retrieving from there"* `[interview]`. A sane number of god nodes means cohesion is good; an explosion of them often signals a corpus that should have been split. See [06-querying-and-god-nodes.md](06-querying-and-god-nodes.md).
|
||||
|
||||
### Chunking & splitting large corpora
|
||||
|
||||
For large documents — and especially when using a **local SLM** with a small context window — tune the chunk size and concurrency rather than throwing the whole corpus at the model: `[github]`
|
||||
|
||||
```bash
|
||||
graphify extract ./docs --token-budget 30000 # smaller semantic chunks for local/small models
|
||||
graphify extract ./docs --max-concurrency 2 # fewer parallel LLM calls (useful for local inference)
|
||||
GRAPHIFY_OLLAMA_NUM_CTX=8192 graphify extract ./docs --backend ollama --token-budget 4000 # tiny chunks for a constrained Ollama context
|
||||
```
|
||||
|
||||
`--token-budget` controls semantic chunk size; smaller budgets keep each LLM call inside a modest context window. `--max-concurrency` throttles parallel calls so a local model isn't overwhelmed. `[github]` Pair these with the Ollama env knobs (`GRAPHIFY_OLLAMA_NUM_CTX`, `GRAPHIFY_OLLAMA_KEEP_ALIVE`) covered in [05-local-models-and-backends.md](05-local-models-and-backends.md).
|
||||
|
||||
---
|
||||
|
||||
## A runnable starter flow for a personal KB
|
||||
|
||||
```bash
|
||||
# 0) one-time: add the format extras you need (combine in ONE bracket)
|
||||
uv tool install "graphifyy[pdf,office,video]"
|
||||
|
||||
# 1) graph your markdown notes + PDFs (docs cost tokens — choose a backend)
|
||||
# in-IDE (uses your session model):
|
||||
/graphify ./knowledge/notes
|
||||
# or headless, fully local:
|
||||
graphify extract ./knowledge/notes --backend ollama --token-budget 8000
|
||||
|
||||
# 2) add a lecture you want to remember
|
||||
/graphify add <youtube-url>
|
||||
|
||||
# 3) pull in cloud docs
|
||||
gws auth login -s drive
|
||||
graphify extract ./knowledge/gdrive --google-workspace
|
||||
|
||||
# 4) keep it fresh as you add files
|
||||
/graphify ./knowledge/notes --update
|
||||
|
||||
# 5) explore
|
||||
/graphify query "what connects my notes on retrieval to the Stanford lecture?"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Open questions / unverified
|
||||
|
||||
- **`gws auth login -s drive` scope flag** — verified in the v8 README `[github]`, but the `gws` CLI is third-party; its exact scope syntax may change. Confirm against [googleworkspace/cli](https://github.com/googleworkspace/cli) before relying on it.
|
||||
- **"71.5x token reduction" and similar multipliers** — sales claims from the README/interview, explicitly described by the creator as corpus-dependent with "no ceiling or floor." `[interview]` Do not treat as a guarantee; see [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
|
||||
- **Exact full list of "and more" video/audio formats** — README lists `.mp4 .mov .mp3 .wav` plus unspecified others. `[github]` Test your specific format.
|
||||
- **Whether `/graphify add` accepts non-YouTube video hosts** — README says "any video URL" via yt-dlp `[github]`; coverage depends on yt-dlp support for that host. `[unverified claim]`
|
||||
- **Slack / OneNote / meeting-transcript connectors** — interview-stated roadmap, confirmed absent from the v8 repo. `[interview]` Meeting capture currently appears to belong to the separate **Penpax** product, not core Graphify. `[github]` Re-check release notes before assuming availability.
|
||||
- **Image extraction quality for handwriting/dense scans** — README cites vision extraction for "screenshots, diagrams, any language" `[github]`; handwriting fidelity is untested here.
|
||||
|
|
@ -0,0 +1,236 @@
|
|||
---
|
||||
summary: Step-by-step guide to installing Graphify, registering it with Claude Code and other assistants, running a first graph safely, and avoiding the token-burn trap on a first document ingestion run.
|
||||
tags:
|
||||
- type/howto
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
- domain/llm
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
|
||||
# Installation & First Run
|
||||
|
||||
How to install Graphify, register it with Claude Code (and other assistants), run your first graph, and confirm it worked — without accidentally burning through your token/usage limit on the first try.
|
||||
|
||||
> Provenance tags used throughout: `[github]` = official GitHub README (anchored to the **v0.8.30** release README, the comprehensive current version — fetched 2026-06-03; the shorter `main`-branch README omits some flags), `[pypi]` = PyPI page, `[site]` = graphifylabs.ai, `[interview]` = creator interview (sales/marketing context, treat as claims), `[community]` = third-party source, `[unverified claim]` = stated somewhere but not confirmed against a primary source.
|
||||
|
||||
---
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
| Requirement | Notes |
|
||||
|---|---|
|
||||
| **Python 3.10+** | Required. Check with `python --version`. `[github][pypi]` |
|
||||
| **An AI coding assistant** | Claude Code is the primary target; Codex, OpenCode, Cursor, and Gemini CLI are also supported. `[github]` |
|
||||
| **`uv` or `pipx`** (recommended) | Either gives you isolated tool installs and puts `graphify` on your PATH automatically. `[github]` |
|
||||
| **Ollama** (optional) | Only needed if you want a local LLM backend for *documents*. See [05-local-models-and-backends.md](05-local-models-and-backends.md). `[github]` (verified facts) |
|
||||
| **`[google]` extra + Google auth** (optional) | Only for the Google Workspace connector. `[github]` (verified facts) |
|
||||
| **AWS Bedrock access** (optional) | Only if you want a cloud LLM backend via `--backend bedrock`. `[github]` (verified facts) |
|
||||
|
||||
> **Code ingestion is done locally and needs no LLM.** Code files are parsed with a tree-sitter AST (abstract syntax tree) pass — no file contents leave your machine, no API calls. `[github]` Semantic extraction of *docs, papers, and images* is what calls a model (your assistant's provider, or a local backend). `[github]` Video/audio is transcribed locally with Whisper. `[github]` This split is the single most important thing to understand before your first run (see §6).
|
||||
|
||||
---
|
||||
|
||||
## 2. Install
|
||||
|
||||
The PyPI package is named **`graphifyy`** (double `y`) — the plain `graphify` name on PyPI is an unrelated package. The CLI command you type is still **`graphify`**. `[github]`
|
||||
|
||||
Install and register in one line. The README leads with `uv`:
|
||||
|
||||
```bash
|
||||
# Recommended — works on Mac and Linux with no PATH setup needed
|
||||
uv tool install graphifyy && graphify install
|
||||
# or with pipx
|
||||
pipx install graphifyy && graphify install
|
||||
# or plain pip
|
||||
pip install graphifyy && graphify install
|
||||
```
|
||||
|
||||
`[github]`
|
||||
|
||||
**Why `uv` (or `pipx`) is recommended:** both put the CLI in a managed location that's automatically on your PATH, so you avoid the `graphify: command not found` problem. With plain `pip` you may have to add `~/.local/bin` (Linux) or `~/Library/Python/3.x/bin` (Mac) to your PATH yourself, or run `python -m graphify` instead. `[github]`
|
||||
|
||||
### Optional extras
|
||||
|
||||
Install feature groups with bracket syntax. Confirmed in the README's file-type table:
|
||||
|
||||
```bash
|
||||
pip install "graphifyy[office]" # .docx / .xlsx ingestion
|
||||
pip install "graphifyy[video]" # video/audio transcription (faster-whisper + yt-dlp)
|
||||
pip install "graphifyy[mcp]" # MCP stdio server (graphify.serve)
|
||||
```
|
||||
|
||||
`[github]` PDFs and images are supported out of the box (no extra needed). `[github]`
|
||||
|
||||
The Google Workspace connector uses a `[google]` extra (`uv tool install "graphifyy[google]"`, then `gws auth login`, run with `--google-workspace`), verified in the v0.8.30 README and detailed in [04-ingesting-docs-knowledge.md](04-ingesting-docs-knowledge.md). `[github]` Other extra names seen on the PyPI listing (`pdf`, `neo4j`, `anthropic`, `all`, …) should be confirmed against the PyPI page or `graphify --help` before you rely on them. `[pypi]`/`[unverified claim]`
|
||||
|
||||
---
|
||||
|
||||
## 3. Register with your assistant
|
||||
|
||||
`graphify install` (already chained into the install command above) drops the platform-specific skill manifest into your assistant's config directory so you can invoke `/graphify` from inside the assistant. For Claude Code on Mac/Linux this is all you need. `[github]`
|
||||
|
||||
Target a specific assistant with `--platform` (or a per-platform subcommand). From the README's platform table `[github]`:
|
||||
|
||||
| Platform | Install command |
|
||||
|---|---|
|
||||
| Claude Code (Linux/Mac) | `graphify install` |
|
||||
| Claude Code (Windows) | `graphify install` (auto-detected) or `graphify install --platform windows` |
|
||||
| Codex | `graphify install --platform codex` |
|
||||
| OpenCode | `graphify install --platform opencode` |
|
||||
| Gemini CLI | `graphify install --platform gemini` |
|
||||
| GitHub Copilot CLI | `graphify install --platform copilot` |
|
||||
| Cursor | `graphify cursor install` |
|
||||
| Aider | `graphify install --platform aider` |
|
||||
|
||||
(Full list in the README also covers VS Code Copilot Chat, Trae, Kiro, Hermes, Factory Droid, OpenClaw, and Google Antigravity.) `[github]`
|
||||
|
||||
> Codex calls skills with `$` instead of `/`, so type `$graphify .` there. Codex also needs `multi_agent = true` under `[features]` in `~/.codex/config.toml` for parallel extraction. `[github]`
|
||||
|
||||
### Make the assistant *always* use the graph (recommended)
|
||||
|
||||
Registering the skill lets you call `/graphify` on demand. To make your assistant consult the graph automatically before grepping files, run the always-on installer **after** you've built a graph in a project:
|
||||
|
||||
```bash
|
||||
graphify claude install # Claude Code: writes a CLAUDE.md section + a PreToolUse hook
|
||||
```
|
||||
|
||||
For Claude Code this adds a `CLAUDE.md` note telling Claude to read `graphify-out/GRAPH_REPORT.md` before architecture questions, plus a PreToolUse hook (in `settings.json`) that fires before Glob/Grep and reminds Claude to navigate via the graph. Equivalent per-platform commands exist (`graphify codex install`, `graphify gemini install`, `graphify cursor install`, etc.); uninstall with the matching `... uninstall`. `[github]`
|
||||
|
||||
### Manual install (no package manager)
|
||||
|
||||
The README documents a curl-based fallback that copies the skill into `~/.claude/skills/graphify/SKILL.md` and adds a trigger line to `~/.claude/CLAUDE.md`. Use this only if you can't use `uv`/`pipx`. `[github]`
|
||||
|
||||
---
|
||||
|
||||
## 4. First run
|
||||
|
||||
From inside your assistant (Claude Code, etc.):
|
||||
|
||||
```
|
||||
/graphify .
|
||||
```
|
||||
|
||||
Or from a plain shell (on **Windows/PowerShell**, drop the slash — `/` is read as a path separator):
|
||||
|
||||
```bash
|
||||
graphify .
|
||||
```
|
||||
|
||||
This reads the files in the target folder, builds the knowledge graph, and writes everything into a `graphify-out/` directory. `[github]` You can point it at any folder — a codebase, a notes directory, a folder of papers. `[github]`
|
||||
|
||||
> **Three invocation contexts — know which one you're in** `[github]`:
|
||||
> - **Inside your AI assistant:** `/graphify .` — semantic extraction uses your IDE session's model (no API key needed).
|
||||
> - **Windows / plain shell:** `graphify .` (no leading slash) — same skill.
|
||||
> - **Headless / CI:** `graphify extract ./path` (build) and `graphify update ./path` (incremental) — these run standalone and need a backend: an API key (`ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, …) **or** a local `--backend ollama`. Code (AST) is always free; only documents need the backend. See [05-local-models-and-backends.md](05-local-models-and-backends.md).
|
||||
|
||||
Useful first-run variants `[github]`:
|
||||
|
||||
```bash
|
||||
graphify ./docs # run on a specific folder
|
||||
graphify . --update # re-extract only changed files, merge into existing graph
|
||||
graphify . --no-viz # skip the HTML, just produce the report + JSON
|
||||
graphify . --mode deep # more aggressive INFERRED edge extraction
|
||||
```
|
||||
|
||||
> A `--force` flag (or `GRAPHIFY_FORCE=1`) forces a full rebuild and overwrites the graph even when it ends up with fewer nodes — use it after a refactor that deleted files, to clear lingering "ghost" nodes. Verified in the v0.8.30 README. `[github]` See [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
|
||||
|
||||
> Add a `.graphifyignore` file (same syntax as `.gitignore`) at your repo root to exclude folders like `node_modules/`, `dist/`, or generated files from the graph. `[github]`
|
||||
|
||||
---
|
||||
|
||||
## 5. What you get and where it lands
|
||||
|
||||
Everything is written under **`graphify-out/`** in the directory you ran against. From the README's literal output tree `[github]`:
|
||||
|
||||
```
|
||||
graphify-out/
|
||||
├── graph.html interactive graph — open in any browser, click nodes, search, filter by community
|
||||
├── GRAPH_REPORT.md god nodes, surprising connections, suggested questions
|
||||
├── graph.json persistent graph — query weeks later without re-reading files
|
||||
└── cache/ SHA256 cache — re-runs only process changed files
|
||||
```
|
||||
|
||||
The three headline outputs map to the verified "interactive HTML graph, markdown report, JSON graph file": **`graph.html`**, **`GRAPH_REPORT.md`**, **`graph.json`**. `[github]`
|
||||
|
||||
Optional extra outputs are opt-in via flags, not produced by default: `--obsidian` writes an Obsidian vault, `--wiki` writes `index.md` + per-community articles, `--svg` writes `graph.svg`, `--graphml` writes `graph.graphml`, `--neo4j` writes `cypher.txt`. Video/audio runs also create `graphify-out/transcripts/`. `[github]`
|
||||
|
||||
### Confirming it worked
|
||||
|
||||
1. **Check the directory:** `graphify-out/` exists and contains `graph.html`, `GRAPH_REPORT.md`, and `graph.json`. `[github]`
|
||||
2. **Open the report:** `GRAPH_REPORT.md` lists the **god nodes** (highest-degree concepts), surprising connections, and 4–5 suggested questions. If you see god nodes, extraction succeeded. `[github]` See [06-querying-and-god-nodes.md](06-querying-and-god-nodes.md).
|
||||
3. **Watch the token benchmark:** Graphify prints a token benchmark automatically after every run, showing tokens-per-query vs. reading the raw files. `[github]`
|
||||
4. **Open the graph:** open `graph.html` in a browser to click through nodes and communities. `[github]`
|
||||
5. **Sanity-check the CLI:** `graphify --help` confirms the binary is on your PATH and lists available commands/flags. `[unverified claim]`
|
||||
|
||||
> On a tiny corpus (a handful of files) the token reduction may be ~1x — that's expected. At that size the value is structural clarity, not compression; reductions scale with corpus size. `[github]` More on this in [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
|
||||
|
||||
---
|
||||
|
||||
## 6. Avoid blowing your token limit on the first run
|
||||
|
||||
This is the #1 new-user mistake. In the interview, a user described running Graphify cold and burning through their entire daily limit plus ~$25–30 extra on top of a $100/mo Claude plan. `[interview]`
|
||||
|
||||
The cause: pointing Graphify at a big pile of *documents* and letting it call a **cloud model** (your assistant's provider) to extract every chunk. The fix is to understand what actually costs tokens:
|
||||
|
||||
- **Code → local AST, free.** Code parsing uses tree-sitter AST locally with **no model calls** and no file contents leaving your machine. Run it on codebases freely. `[github][interview]`
|
||||
- **Docs / papers / images → model, costs tokens.** Only these go to a model for semantic extraction. `[github]` To make that free too, point the document backend at **Ollama** (a local small language model). `[interview]` (verified facts: `--backend ollama`) See [05-local-models-and-backends.md](05-local-models-and-backends.md).
|
||||
- **The first build costs tokens; queries are where you save.** The initial extraction pays the token cost; every later query reads the compact graph instead of raw files, and the SHA256 cache means re-runs only re-process changed files. `[github]`
|
||||
- **Use `--update`, not full rebuilds.** When adding files later, re-run with `--update` so the cache reuses prior work instead of starting fresh. `[interview][github]` See [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
|
||||
- **Split large doc sets** and re-ingest in pieces rather than dumping one giant chunk in a single run. `[interview]`
|
||||
|
||||
**Practical first run:** start on a code folder (free), confirm the graph looks right, then add documents with a local Ollama backend before scaling up.
|
||||
|
||||
---
|
||||
|
||||
## 7. Platform notes
|
||||
|
||||
**macOS / Linux:** `uv tool install graphifyy` or `pipx install graphifyy` both put `graphify` on your PATH with no setup. If you used plain `pip` and get `graphify: command not found`, add `~/.local/bin` (Linux) or `~/Library/Python/3.x/bin` (Mac) to your PATH, or run `python -m graphify`. `[github]`
|
||||
|
||||
**Windows:** Plain-`pip` scripts land in `%APPDATA%\Python\PythonXY\Scripts` — add that to your PATH, or just use `uv`/`pipx`. For Claude Code on Windows, `graphify install` auto-detects the platform (or use `--platform windows`). `[github]`
|
||||
|
||||
**WSL / Ubuntu:** Ubuntu ships `python3`, not `python`. For the MCP server path, install into a project venv to avoid PEP 668 conflicts: `python3 -m venv .venv && .venv/bin/pip install "graphifyy[mcp]"`. `[github]`
|
||||
|
||||
---
|
||||
|
||||
## 8. Copy-pasteable quickstart
|
||||
|
||||
```bash
|
||||
# 1. Prereqs: Python 3.10+ and uv (or pipx). Check Python:
|
||||
python --version
|
||||
|
||||
# 2. Install Graphify + register the skill (package is "graphifyy" with double y;
|
||||
# command stays "graphify"). Recommended — no PATH setup on Mac/Linux:
|
||||
uv tool install graphifyy && graphify install
|
||||
# or: pipx install graphifyy && graphify install
|
||||
# other assistants: graphify install --platform codex (Cursor: graphify cursor install)
|
||||
|
||||
# 3. (Optional) make the assistant always consult the graph before grepping:
|
||||
graphify claude install
|
||||
|
||||
# 4. First run — start with a CODE folder (AST parsing is free, no LLM):
|
||||
graphify .
|
||||
|
||||
# 5. Confirm it worked:
|
||||
ls graphify-out/ # expect graph.html, GRAPH_REPORT.md, graph.json
|
||||
# open graphify-out/GRAPH_REPORT.md -> check the god nodes
|
||||
# open graphify-out/graph.html -> explore the graph
|
||||
|
||||
# 6. For DOCUMENTS, use a local model so you don't burn tokens — see doc 05.
|
||||
# Then add incrementally:
|
||||
graphify ./docs --update
|
||||
```
|
||||
|
||||
Next: [05-local-models-and-backends.md](05-local-models-and-backends.md) (keep document ingestion free with Ollama) and [07-token-economics-and-updates.md](07-token-economics-and-updates.md) (incremental updates and where the savings come from). Concept refresher: [01-overview-concepts.md](01-overview-concepts.md).
|
||||
|
||||
---
|
||||
|
||||
## Open questions / unverified
|
||||
|
||||
- **README version skew (resolved).** The repo ships two READMEs: a short `main`-branch one (leads with `pip install`, omits several flags) and the comprehensive **v0.8.30 release** README (leads with `uv tool install graphifyy && graphify install`, documents the full flag set). This doc — and the rest of this guide — anchors on **v0.8.30**, the current release. If a doc page elsewhere disagrees, prefer the v0.8.30 release README, and confirm with `graphify --help` on your installed version.
|
||||
- **`[google]` Workspace connector + `--backend ollama`/`bedrock` env vars** (`OLLAMA_BASE_URL`, `gws auth login`, `--google-workspace`, etc.) are all in the v0.8.30 README. See [04-ingesting-docs-knowledge.md](04-ingesting-docs-knowledge.md) and [05-local-models-and-backends.md](05-local-models-and-backends.md) for exact syntax.
|
||||
- **Some extra names** (`pdf`, `neo4j`, `anthropic`, `all`) come from the PyPI listing; `[office]`, `[video]`, `[mcp]` are README-confirmed. Confirm others before relying on them.
|
||||
- **`graphify --help`** as the post-install sanity check is inferred convention, not explicitly documented. `[unverified claim]`
|
||||
- **Output directory** is `graphify-out/` relative to the run target per the README's literal tree; whether it can be relocated is not documented here.
|
||||
|
|
@ -0,0 +1,216 @@
|
|||
---
|
||||
summary: Reference for Graphify's backend menu (Ollama, Bedrock, Claude, OpenAI, etc.), Ollama setup and tuning knobs, and the code-free / docs-cost split that drives backend selection.
|
||||
tags:
|
||||
- type/reference
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
- domain/llm
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
# 05 — Local Models & Backends
|
||||
|
||||
How Graphify decides what (if anything) talks to an LLM, the full menu of backends, and how to run document extraction fully local with Ollama. The short version: **code never needs a model; only documents do** — so backend choice is purely a documents/knowledge-base question.
|
||||
|
||||
See also: [03 — Ingesting code (AST)](03-ingesting-code-ast.md) · [04 — Ingesting docs & knowledge](04-ingesting-docs-knowledge.md) · [07 — Token economics & updates](07-token-economics-and-updates.md).
|
||||
|
||||
---
|
||||
|
||||
## The one rule that drives everything: code is free, docs cost a model
|
||||
|
||||
Graphify splits its inputs in two:
|
||||
|
||||
- **Code files — processed locally via tree-sitter (AST). No API calls, nothing leaves your machine.** `[github]` (README: *"Code is extracted locally with no API calls (AST via tree-sitter)"* and *"Code files — processed locally via tree-sitter. Nothing leaves your machine."*) This is also the creator's repeated point in the interview: *"The a call is free of cost. There's no API call included for an LLM... You have to call the LLM back end only for your documents."* `[interview]`
|
||||
- **Docs, PDFs, images — sent to a model for semantic extraction.** `[github]` (README: *"Docs, PDFs, images — sent to your AI assistant for semantic extraction"*).
|
||||
|
||||
So if you only ingest code, **you never pick a backend and never spend a token on extraction.** Backend selection below matters *only* for documents and knowledge bases.
|
||||
|
||||
---
|
||||
|
||||
## SLM vs LLM — for a non-expert
|
||||
|
||||
The interview leans hard on **SLMs (small language models)** as the local-first future. `[interview]` In plain terms:
|
||||
|
||||
- An **LLM** (large language model — e.g. Claude, GPT, Gemini) is huge. It generally runs in the cloud because it needs a lot of memory / GPU to hold.
|
||||
- An **SLM** is a smaller model (a few billion parameters) that **fits in ordinary RAM, or in a consumer GPU's VRAM, and runs on your own machine.** Because it runs locally, **your files never go to a cloud provider.** `[interview]`
|
||||
|
||||
The creator's framing: *"large language models do not fit in smaller RAM size. So you need larger RAM or ... GPUs ... to fit them in. So rather than ... LLMs, you can just go for SLMs ... so that their data isn't shared to cloud-based LLMs."* `[interview]` He describes Graphify's direction as a *"local-first AI memory system"* where *"none of your file will be shared with cloud-based LLMs."* `[interview]`
|
||||
|
||||
Reality check: that "as good as a frontier model" claim is the creator's **aspiration**, not a measured fact. `[unverified claim]` For document extraction (chunk → entities/relationships), a competent local instruction-following model is usually *good enough*; for the highest-quality graphs on messy corpora, a cloud model still tends to win. Treat local as the **privacy/cost** choice, cloud as the **max-quality** choice.
|
||||
|
||||
---
|
||||
|
||||
## The backend menu
|
||||
|
||||
All set per run with `--backend <name>` on `graphify extract`. `[github]` (README backend list, verbatim: *"gemini, kimi, claude, openai, deepseek, ollama, bedrock, or claude-cli"*.) Each non-local backend reads its credential from an env var:
|
||||
|
||||
| Backend | Flag | Auth | Where data goes |
|
||||
|---|---|---|---|
|
||||
| **Ollama (local)** | `--backend ollama` | none for loopback | **Your machine only** |
|
||||
| **AWS Bedrock** | `--backend bedrock` | IAM via standard AWS credential chain (`AWS_*` / `~/.aws/credentials`) — **no API key** | AWS account |
|
||||
| **Claude (API)** | `--backend claude` | `ANTHROPIC_API_KEY` | Anthropic |
|
||||
| **Claude Code CLI** | `--backend claude-cli` | none — uses your Claude subscription | Anthropic |
|
||||
| **Gemini** | `--backend gemini` | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | Google |
|
||||
| **OpenAI / compatible** | `--backend openai` | `OPENAI_API_KEY` | OpenAI (or your compatible endpoint) |
|
||||
| **DeepSeek** | `--backend deepseek` | `DEEPSEEK_API_KEY` | DeepSeek |
|
||||
| **Kimi (Moonshot)** | `--backend kimi` | `MOONSHOT_API_KEY` | **Moonshot AI servers in China** `[github]` |
|
||||
|
||||
All flag values, env-var names, and auth notes above are quoted from the v8 README env-var table and backend list. `[github]`
|
||||
|
||||
**In-platform vs headless.** When you run Graphify *inside* a coding assistant via the `/graphify` skill, document extraction uses **whatever model your IDE session already runs** (Claude / Gemini / etc.) — no separate key needed. `[github]` (README: *"using whatever model your IDE session runs"*.) Headless `graphify extract` is where you must supply a backend + credential.
|
||||
|
||||
**Auto-detect priority.** If you don't pass `--backend`, headless `graphify extract` picks one based on which key is set, in this order: **Gemini → Kimi → Claude → OpenAI → DeepSeek → Bedrock → Ollama.** `[github]` Pass `--backend` explicitly if you care which one runs (you usually do, for privacy).
|
||||
|
||||
### How to choose
|
||||
|
||||
- **Privacy / offline / zero per-token cost → `--backend ollama`.** Nothing leaves the machine. The interview's recommended default for documents when you want to *"save cost."* `[interview]`
|
||||
- **Enterprise cloud, no API keys to manage → `--backend bedrock`** (uses your existing IAM). `[github]` `[interview]`
|
||||
- **Already paying for a coding-assistant subscription → in-platform `/graphify`, or `--backend claude-cli`** to reuse that subscription with no extra key. `[github]`
|
||||
- **Max extraction quality, simplest setup → a frontier cloud backend** (`claude` / `gemini` / `openai`) with the matching key.
|
||||
|
||||
---
|
||||
|
||||
## Setting up Ollama (runnable)
|
||||
|
||||
### 1. Install Ollama and pull a model
|
||||
|
||||
[Ollama](https://ollama.com) is a separate program that runs models locally and exposes an HTTP API on `http://localhost:11434`. Install it, then pull an instruction-following model:
|
||||
|
||||
```bash
|
||||
# Install Ollama: see https://ollama.com/download
|
||||
# Then pull a model that fits your RAM/VRAM (see "Which model?" below):
|
||||
ollama pull qwen2.5:7b # example only — pick one that fits your hardware
|
||||
ollama serve # if not already running as a service
|
||||
```
|
||||
|
||||
### 2. Install Graphify with the Ollama extra
|
||||
|
||||
```bash
|
||||
uv tool install "graphifyy[ollama]"
|
||||
```
|
||||
|
||||
> Note the package name is **`graphifyy`** (double-y) and the optional extra is `[ollama]`. `[github]` (README install table.)
|
||||
|
||||
### 3. Run document extraction against Ollama
|
||||
|
||||
```bash
|
||||
# Local Ollama — no API key needed for loopback:
|
||||
graphify extract ./docs --backend ollama
|
||||
```
|
||||
|
||||
That's it for code, too — but remember, **code uses AST regardless**, so the backend only kicks in for the documents in `./docs`.
|
||||
|
||||
### Ollama environment variables
|
||||
|
||||
All four below are confirmed verbatim in the v8 README env-var table. `[github]`
|
||||
|
||||
| Env var | Purpose | Default |
|
||||
|---|---|---|
|
||||
| `OLLAMA_BASE_URL` | Ollama server URL | `http://localhost:11434` |
|
||||
| `OLLAMA_MODEL` | Model name to use | auto-detect |
|
||||
| `GRAPHIFY_OLLAMA_NUM_CTX` | Override Ollama KV-cache (context) window size | auto-sized |
|
||||
| `GRAPHIFY_OLLAMA_KEEP_ALIVE` | Minutes to keep the model loaded in memory; **set `0` to unload after each chunk** | (model stays loaded) |
|
||||
|
||||
```bash
|
||||
# Point at a remote Ollama box and pin a specific model:
|
||||
OLLAMA_BASE_URL=http://192.168.1.50:11434 \
|
||||
OLLAMA_MODEL=qwen2.5:7b \
|
||||
graphify extract ./docs --backend ollama
|
||||
```
|
||||
|
||||
### Tuning `GRAPHIFY_OLLAMA_NUM_CTX` and `KEEP_ALIVE`
|
||||
|
||||
These two are the knobs the README's own troubleshooting section reaches for when *"Ollama runs out of VRAM / context window exceeded."* `[github]`
|
||||
|
||||
**`GRAPHIFY_OLLAMA_NUM_CTX`** — the KV-cache window. Auto-sized by default; override it down if you hit VRAM limits, or up if your chunks are large. Both directions appear verbatim in the README:
|
||||
|
||||
```bash
|
||||
# Shrink the context window to survive a small GPU (from the VRAM troubleshooting section):
|
||||
GRAPHIFY_OLLAMA_NUM_CTX=8192 graphify extract ./docs --backend ollama --token-budget 4000
|
||||
|
||||
# Or raise it for big chunks on a big GPU:
|
||||
GRAPHIFY_OLLAMA_NUM_CTX=32768 graphify extract ./docs --backend ollama
|
||||
```
|
||||
|
||||
**`GRAPHIFY_OLLAMA_KEEP_ALIVE=0`** — unload the model between chunks. Frees VRAM on small GPUs at the cost of reload time per chunk:
|
||||
|
||||
```bash
|
||||
GRAPHIFY_OLLAMA_KEEP_ALIVE=0 graphify extract ./docs --backend ollama # saves VRAM on small GPUs
|
||||
```
|
||||
|
||||
**Slow local models timing out?** Raise the HTTP timeout (default 600s) — README confirms both the env var `GRAPHIFY_API_TIMEOUT` and the `--api-timeout` flag:
|
||||
|
||||
```bash
|
||||
graphify extract ./docs --backend ollama --api-timeout 900 # 15-minute timeout
|
||||
```
|
||||
|
||||
> **Known sharp edge:** community bug reports flag context-window saturation across consecutive chunks on the Ollama backend (the model's session not resetting between chunks, exhausting VRAM after a few chunks). If you see degrading or "hollow" responses partway through a large run, lower `GRAPHIFY_OLLAMA_NUM_CTX` and/or set `GRAPHIFY_OLLAMA_KEEP_ALIVE=0`. `[community](https://github.com/safishamsi/graphify/issues/798)`
|
||||
|
||||
### Which Ollama model?
|
||||
|
||||
**There is no official Graphify-recommended/tested model list in the README or on the site.** Neither the v8 README nor the site publishes one (verified — see Open questions). So follow the rule the task itself sets and the interview implies: **pick an instruction-following model that fits your RAM/VRAM.** `[interview]` Document extraction is "read this chunk, emit structured entities/relationships," which is an instruction-following + structured-output job, not deep reasoning.
|
||||
|
||||
> **Correction — there IS a shipped code default (2026-06-05).** Although the README/site publish no model recommendation, installed graphify 0.8.31 **hardcodes** a fallback in `llm.py:67`: `"default_model": os.environ.get("OLLAMA_MODEL", "qwen2.5-coder:7b")`. `[github]` So if you run `graphify extract --backend ollama` without setting `OLLAMA_MODEL`, the binary silently uses `qwen2.5-coder:7b`. The README's silence on a recommended model is accurate as a documentation statement; it is not accurate as a claim that "no default exists." The choice of a *coder* model for document extraction is a revealed preference — the binding requirement is structured-JSON instruction-following discipline, not domain NER. See [10-extraction-model-options.md](10-extraction-model-options.md) for the full analysis.
|
||||
|
||||
Rough sizing rule of thumb: a model needs roughly its parameter count in **GB of (V)RAM** at common 4-bit quantization (a 7B model ≈ ~5 GB; a 30B+ model wants 24 GB+ VRAM). Leave headroom for the context window.
|
||||
|
||||
Community signals (not Graphify-verified):
|
||||
|
||||
- A user feature-request reports running **`gemma3:27b`-class models on a 24 GB+ VRAM GPU** with custom `num_ctx`/concurrency settings for high GPU saturation. Treat the specific tag and numbers as one user's setup, not a spec. `[community](https://github.com/safishamsi/graphify/issues/792)`
|
||||
- General Ollama guidance points to **Qwen2.5 (e.g. `qwen2.5:7b`)** and **Phi-4 (`phi4:14b`)** as strong instruction-following / structured-output models in their size classes. Good *starting points* to test, not Graphify endorsements. `[community](https://ollama.com/library)`
|
||||
|
||||
Start with the largest instruction-following model that comfortably fits your hardware, run a small `extract`, then check the [god nodes](06-querying-and-god-nodes.md) to judge whether the graph quality is acceptable before committing to a big run.
|
||||
|
||||
---
|
||||
|
||||
## Controlling spend & local load: `--token-budget` and `--max-concurrency`
|
||||
|
||||
These two govern *how* document chunks are sent to whatever backend you chose. Both are confirmed in the v8 README. `[github]`
|
||||
|
||||
- **`--token-budget`** — size of each semantic chunk sent to the model. **Smaller budget = smaller chunks**, which is the recommended setting for local/small models (less context per call, fits small windows). `[github]` (README: *"smaller semantic chunks for local/small models"*.)
|
||||
- **`--max-concurrency`** — number of parallel LLM calls. **Lower it for local inference** so you don't overwhelm a single GPU/CPU. `[github]` (README: *"fewer parallel LLM calls (useful for local inference)"*.)
|
||||
|
||||
```bash
|
||||
# Gentle on a single local GPU: small chunks, low concurrency, generous timeout
|
||||
GRAPHIFY_OLLAMA_NUM_CTX=8192 \
|
||||
graphify extract ./docs \
|
||||
--backend ollama \
|
||||
--token-budget 4000 \
|
||||
--max-concurrency 2 \
|
||||
--api-timeout 900
|
||||
```
|
||||
|
||||
For **cloud** backends, the same two flags are your cost dials: smaller `--token-budget` and the [`graphify update`](07-token-economics-and-updates.md) incremental flow keep token spend down. The interview's repeated cost advice: extract **code with AST (free)**, only spend model tokens on **documents**, and always **`graphify update`** rather than re-ingesting from scratch. `[interview]`
|
||||
|
||||
---
|
||||
|
||||
## Quick reference
|
||||
|
||||
```bash
|
||||
# Fully local document extraction (privacy / zero per-token cost)
|
||||
graphify extract ./docs --backend ollama
|
||||
|
||||
# Enterprise cloud via existing AWS IAM (no API key)
|
||||
graphify extract ./docs --backend bedrock
|
||||
|
||||
# Reuse your Claude subscription, no extra key
|
||||
graphify extract ./docs --backend claude-cli
|
||||
|
||||
# Explicit cloud key
|
||||
ANTHROPIC_API_KEY=sk-... graphify extract ./docs --backend claude
|
||||
|
||||
# Code only → no backend needed at all (AST is free)
|
||||
graphify extract ./src
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Open questions / unverified
|
||||
|
||||
- **No official Ollama model recommendation exists in the README**, but the installed binary hardcodes `qwen2.5-coder:7b` as the Ollama fallback (`llm.py:67`) — see the Correction callout above and `10-extraction-model-options.md`. `[github]`
|
||||
- **`GRAPHIFY_OLLAMA_KEEP_ALIVE` default value** is documented by *behavior* ("minutes to keep loaded; `0` to unload after each chunk") but the README does not state the numeric default. `[github]`
|
||||
- **`gemma3:27b` exact tag / VRAM numbers** come from a single user issue (#792), not Graphify docs. The original ASR transcript said "Gemma 4 31b," which does not match a shipped tag; treat the community `gemma3:27b`-class figure as illustrative, not authoritative. `[community](https://github.com/safishamsi/graphify/issues/792)`
|
||||
- **The "SLMs as good as frontier models soon" vision** is the creator's aspiration, not a benchmark. `[unverified claim]` / `[interview]`
|
||||
- **The 70x/90x token-savings headline numbers** are corpus-dependent sales claims (the creator himself says *"there is no ceiling or floor"*); not used as a basis for any guidance here. `[unverified claim]` / `[interview]`
|
||||
- Could not confirm any `graphify.yaml` config-file equivalents for these knobs against the raw README; the `graphify.yaml` keys mentioned in issue #792 are a **feature request**, not shipped config. `[community](https://github.com/safishamsi/graphify/issues/792)`
|
||||
|
|
@ -0,0 +1,128 @@
|
|||
---
|
||||
summary: Mental model for Graphify's knowledge-graph approach — god nodes, community detection, confidence tags, and how the three output artifacts fit together.
|
||||
tags:
|
||||
- type/reference
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
# Graphify: Overview & Mental Model
|
||||
|
||||
Graphify turns any folder — code, documents, and media — into a single queryable **knowledge graph** that your AI assistant reads instead of blindly re-reading raw files. This page is the mental model you need *before* deciding whether to adopt it; the how-to lives in the rest of this set.
|
||||
|
||||
---
|
||||
|
||||
## The one-line concept
|
||||
|
||||
> "Graphify transforms project folders into queryable knowledge graphs ... it maps your entire codebase — code, documentation, PDFs, images, and videos — into a structured representation you can search and explore." `[github]`
|
||||
|
||||
The pitch is a **knowledge-graph-of-everything**: source code, docs, PDFs, images, and transcribed audio/video all land in *one* graph you can query, rather than living in separate silos that your assistant has to grep through one at a time. The creator (Safi Shamsi) frames it as a "digital brain" / "digital twin" of a codebase or enterprise that you can recall at any time `[interview]`.
|
||||
|
||||
See [02-installation-setup.md](02-installation-setup.md) to get it running and [08-workflows-and-use-cases.md](08-workflows-and-use-cases.md) for concrete scenarios.
|
||||
|
||||
---
|
||||
|
||||
## Why a graph beats plain file search
|
||||
|
||||
Plain AI coding assistants work on flat files:
|
||||
|
||||
> "AI coding assistants operate on flat-file context. They read files, sometimes many at once, but they have no map of how concepts relate across your codebase." `[community]`([augmentcode](https://www.augmentcode.com/learn/graphify-knowledge-graphs-ai-coding))
|
||||
|
||||
A graph pre-computes the **relationships** — which function calls which, which doc explains which module, which concepts recur across files. The verified payoff over grep/file-search is *cross-file structure*:
|
||||
|
||||
> "Unlike grep or file search, Graphify understands *relationships* across your codebase ... links between things that live in different files or modules. Ranked by how unexpected they are." `[github]`
|
||||
|
||||
So instead of "find the file named `auth`," you can ask "what connects auth to the database?" and get a path through the graph. Query mechanics are covered in [06-querying-and-god-nodes.md](06-querying-and-god-nodes.md).
|
||||
|
||||
---
|
||||
|
||||
## Why a graph beats Obsidian
|
||||
|
||||
People reasonably ask: *isn't this just an Obsidian vault graph?* The creator's argument is that Obsidian only *visualizes* links, while Graphify does real graph analysis on top `[interview]`:
|
||||
|
||||
- Obsidian "can't do clustering for you" — it won't group related notes into communities. `[interview]`
|
||||
- Obsidian "can't do cross-community interaction" — it won't surface links *between* clusters. `[interview]`
|
||||
- Obsidian's graph "looks pretty much well but there is nothing credible you can take from there." `[interview]`
|
||||
|
||||
Treat those three claims as the creator's framing (unconfirmed), not measured fact. The *verified* version of the same point is milder: Graphify computes ranked "surprising connections" across files/modules `[github]`, which a visualization-only tool doesn't do. In practice Obsidian remains a fine **visual pairing** — the interview itself calls it "a decent recommended pairing because it's visual" `[interview]`.
|
||||
|
||||
---
|
||||
|
||||
## God nodes
|
||||
|
||||
**God nodes** are the most important hubs in your graph:
|
||||
|
||||
> "the most-connected concepts in your project. Everything flows through these." `[github]`
|
||||
|
||||
Mental model: god nodes are the load-bearing entities — the architecture's spine. The recommended habit is to **ask for god nodes first**, get the high-level map in one shot, then "scalpel" down into specific nodes only where needed (this is the core token-saving move) `[interview]`. The creator also offers a rough diagnostic: an unexpectedly *large* number of god nodes can signal poor cohesion in a codebase `[interview]` (unverified heuristic). Details and example queries: [06-querying-and-god-nodes.md](06-querying-and-god-nodes.md).
|
||||
|
||||
---
|
||||
|
||||
## Community / cluster detection
|
||||
|
||||
Graphify runs clustering to group related entities into **communities**, with adjustable granularity:
|
||||
|
||||
> "The tool runs clustering to group related entities. You can rerun clustering on existing graph and adjust granularity with resolution parameters." `[github]`
|
||||
|
||||
This is what powers the two things Obsidian reportedly can't do: grouping nodes into communities, and detecting **cross-community links** (relationships that bridge otherwise-separate clusters). Conceptually this is standard graph theory (community detection via clustering algorithms) `[interview]`. The relevant flags (e.g. cluster-only runs and a resolution setting) are documented in [06-querying-and-god-nodes.md](06-querying-and-god-nodes.md) — don't guess at them from here.
|
||||
|
||||
---
|
||||
|
||||
## Confidence tags: EXTRACTED / INFERRED / AMBIGUOUS
|
||||
|
||||
Every relationship in the graph carries a confidence label, so you know how much to trust it:
|
||||
|
||||
- `EXTRACTED` — "directly found in source" `[github]`
|
||||
- `INFERRED` — "logically deduced" `[github]`
|
||||
- `AMBIGUOUS` — "uncertain connections" `[github]`
|
||||
|
||||
Corroborated by community write-ups: "Every relationship gets tagged as `EXTRACTED`, `INFERRED`, or `AMBIGUOUS`, so developers know which connections came from code versus model inference." `[community]`([augmentcode](https://www.augmentcode.com/learn/graphify-knowledge-graphs-ai-coding))
|
||||
|
||||
The mental-model takeaway: `EXTRACTED` edges come from deterministic parsing (the AST — see [03-ingesting-code-ast.md](03-ingesting-code-ast.md)); `INFERRED`/`AMBIGUOUS` edges come from a language model reading prose. Weight them accordingly when you act on a query result.
|
||||
|
||||
---
|
||||
|
||||
## The "neuro-symbolic" framing
|
||||
|
||||
The creator positions Graphify as more than a RAG store — as a step toward **neuro-symbolic AI**, where the graph acts as a layer of *symbols* that ground the *neural* network:
|
||||
|
||||
> Graphify "isn't just supporting neural networks, it's giving rise to neuro-symbolic AI systems where you have a map or symbols to support the neural networks to come up with a response." `[interview]`
|
||||
|
||||
The stated motivation: neural networks (LLMs) hallucinate and lose context, especially as you cram more into the context window; a symbolic graph constrains and grounds retrieval, reducing that drift `[interview]`. Neuro-symbolic AI is a genuine research area, but the specific claim that *Graphify* meaningfully reduces hallucination is the creator's pitch — not independently verified here. Treat it as a useful intuition, not a measured result.
|
||||
|
||||
---
|
||||
|
||||
## The three outputs
|
||||
|
||||
Every run produces three artifacts `[github]`:
|
||||
|
||||
| File | What it is |
|
||||
|------|-----------|
|
||||
| `graph.html` | Interactive visualization — "open in any browser — click nodes, filter, search" `[github]` |
|
||||
| `GRAPH_REPORT.md` | Markdown report — "the highlights: key concepts, surprising connections, suggested questions" `[github]` |
|
||||
| `graph.json` | The full graph — "query it anytime without re-reading your files" `[github]` |
|
||||
|
||||
The `graph.json` is the durable, reusable asset: it's what lets later queries stay cheap. That cost story (and the incremental-`update` mechanism) is in [07-token-economics-and-updates.md](07-token-economics-and-updates.md). Local-vs-cloud extraction backends are in [05-local-models-and-backends.md](05-local-models-and-backends.md).
|
||||
|
||||
---
|
||||
|
||||
## How the pieces fit (decision summary)
|
||||
|
||||
- **Code** is parsed locally via tree-sitter ASTs — no API calls, no tokens spent `[github]` `[interview]`. See [03-ingesting-code-ast.md](03-ingesting-code-ast.md).
|
||||
- **Docs / media** need a language model to extract meaning, so they cost tokens (or run on a local model) `[interview]`. See [04-ingesting-docs-knowledge.md](04-ingesting-docs-knowledge.md) and [05-local-models-and-backends.md](05-local-models-and-backends.md).
|
||||
- The graph is the **memory/context**; the discipline is "ask the graph, don't make the LLM re-read the corpus" `[interview]`.
|
||||
- Start from **god nodes**, then drill down — that's where the savings come from `[interview]`.
|
||||
|
||||
For an end-to-end adoption checklist, jump to [09-best-practices-checklist.md](09-best-practices-checklist.md). Index: [00-README.md](00-README.md).
|
||||
|
||||
---
|
||||
|
||||
## Open questions / unverified
|
||||
|
||||
- **Headline numbers are marketing.** Token savings of "70x"/"90x", "500K downloads," "43K stars" are sales claims and they conflict across sources (e.g. one community post cites 58.3K stars). `[unverified claim]` Do not treat any of these as fact.
|
||||
- **Neuro-symbolic = less hallucination** for Graphify specifically is the creator's framing `[interview]`, not independently measured.
|
||||
- **The Obsidian critique** (can't cluster, no cross-community links, "nothing credible") is `[interview]` only; the verified contrast is the narrower "ranked cross-file connections" claim `[github]`.
|
||||
- **"Large god-node count signals poor cohesion"** is an unverified diagnostic heuristic from the interview.
|
||||
- **graphifylabs.ai** could not be fetched (HTTP 403 bot protection), so its exact tagline/claims are not directly verified here.
|
||||
|
|
@ -0,0 +1,243 @@
|
|||
---
|
||||
summary: The retrieval playbook for Graphify — how to surface god nodes, use query/path/explain commands, read confidence tags, tune community clustering, and extract answers from the graph without re-reading the corpus.
|
||||
tags:
|
||||
- type/howto
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
- domain/llm
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
|
||||
# Querying the Graph & God Nodes
|
||||
|
||||
Querying is the highest-leverage skill in Graphify: a good query makes the graph *do the reading for you* so your assistant spends tokens on an answer, not on re-grepping the corpus. This page is the retrieval playbook — start at god nodes, scalpel down, and trust the graph as your memory.
|
||||
|
||||
Prerequisite: you've built a graph (`graphify-out/graph.json`, `GRAPH_REPORT.md`, `graph.html`). See [03-ingesting-code-ast.md](03-ingesting-code-ast.md) and [04-ingesting-docs-knowledge.md](04-ingesting-docs-knowledge.md). The cost story behind "ask the graph, not the corpus" lives in [07-token-economics-and-updates.md](07-token-economics-and-updates.md).
|
||||
|
||||
---
|
||||
|
||||
## The core principle: prompt the graph, don't make the LLM read the corpus
|
||||
|
||||
The single most important habit, in the creator's words:
|
||||
|
||||
> "When you are prompting the graph, don't ask the LLM to go through the graph — rather ask the LLM to extract from the graph. So the graph is the memory, the graph is the context." `[interview]`
|
||||
|
||||
The graph **is** the context/memory. You want the assistant to *extract from* it (pull the specific nodes/edges that answer the question), not *traverse* the whole thing (which just re-spends the tokens you built the graph to save) `[interview]`.
|
||||
|
||||
This is backed by how the tool installs itself. The persistent install config tells your assistant to:
|
||||
|
||||
> "consult the knowledge graph for codebase questions — preferring scoped queries like `graphify query "<question>"` over reading the full report or grepping raw files." `[github]`
|
||||
|
||||
On Claude Code and Gemini CLI a hook even fires *before* search-style tool calls (and before reading source files one-by-one) to nudge the assistant onto the graph path `[github]`. So the discipline is partly automated — but you still drive it with good queries.
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — Always ask for the god nodes first
|
||||
|
||||
**God nodes** are the most-connected concepts — the load-bearing spine of the project.
|
||||
|
||||
> "God nodes — the most-connected concepts in your project. Everything flows through these." `[github]`
|
||||
|
||||
The retrieval playbook starts here every time:
|
||||
|
||||
> "First of all you prompt to give you the god nodes from the whole corpus first of all ... from god nodes you can easily see where you have to prompt and write to minimize as much tokens as possible." `[interview]`
|
||||
|
||||
Get the high-level architecture in one shot, *then* "scalpel" down into the specific nodes you actually need `[interview]`. That ordering — map first, drill second — is where the token savings come from.
|
||||
|
||||
### How to actually surface god nodes (important)
|
||||
|
||||
There is **no** dedicated `graphify god-nodes` command or `--god-nodes` flag. God nodes are surfaced in two honest ways:
|
||||
|
||||
1. **Read the top of `GRAPH_REPORT.md`.** God nodes are the first thing in the report `[github]`:
|
||||
> "What's in the report: **God nodes** — the most-connected concepts ... Everything flows through these." `[github]`
|
||||
2. **Ask your assistant** for the high-level architecture; with the graph installed it reads the graph rather than the files `[github]`.
|
||||
|
||||
Do *not* expect `graphify query "what are the god nodes?"` to return the structural ranking — `query` is *semantic search* over content, while the god-node ranking is a structural (degree-based) result that lives in the report `[github]`. You can ask a semantic question to get oriented, but the authoritative hub list is the report.
|
||||
|
||||
To tune that ranking, see `--exclude-hubs` under [clusters](#step-4--community--cluster-detection) below.
|
||||
|
||||
---
|
||||
|
||||
## Step 2 — Scalpel down: query / path / explain
|
||||
|
||||
Three verified commands cover the drill-down. (Inside an IDE skill they take a `/` prefix, e.g. `/graphify query`; from a plain terminal drop the slash, e.g. `graphify query`. Same command, two entry points `[github]`. On Windows PowerShell, always use the no-slash form `[github]`.)
|
||||
|
||||
| Command | What it does | Use it to |
|
||||
|---|---|---|
|
||||
| `graphify query "<question>"` | Semantic search over the graph `[github]` | Find the nodes relevant to a question |
|
||||
| `graphify path "Node1" "Node2"` | Find connections between two nodes `[github]` | Trace how two things relate |
|
||||
| `graphify explain "Node"` | Single-node deep-dive `[github]` | Scalpel into one specific node |
|
||||
|
||||
```bash
|
||||
# semantic search — "extract from the graph"
|
||||
graphify query "what connects auth to the database?" # [github]
|
||||
graphify query "show the auth flow" # [github]
|
||||
|
||||
# point at a specific graph file if not in the default location
|
||||
graphify query "what connects DigestAuth to Response?" --graph graphify-out/graph.json # [github]
|
||||
|
||||
# bound the search to keep the answer (and tokens) tight
|
||||
graphify query "..." --dfs --budget 1500 # [github]
|
||||
|
||||
# connection between two named nodes
|
||||
graphify path "UserService" "DatabasePool" # [github]
|
||||
graphify path "DigestAuth" "Response" # [github]
|
||||
|
||||
# deep-dive a single node — the literal "scalpel"
|
||||
graphify explain "RateLimiter" # [github]
|
||||
```
|
||||
|
||||
The `--dfs --budget` flags are the direct, runnable expression of "minimize tokens": you cap the traversal/budget so the assistant extracts a bounded slice instead of wandering the whole graph `[github]`.
|
||||
|
||||
### Architecture / call-flow diagram
|
||||
|
||||
For a visual map of how the system fits together:
|
||||
|
||||
```bash
|
||||
graphify export callflow-html # Mermaid architecture/call-flow HTML [github]
|
||||
graphify export callflow-html --max-sections 8 # cap the number of generated sections [github]
|
||||
graphify export callflow-html --output docs/arch.html
|
||||
graphify export callflow-html ./some-repo/graphify-out
|
||||
```
|
||||
|
||||
If you installed the git hook, this auto-regenerates on every commit `[github]`. This pairs well with the god-node read: the report tells you *which* nodes matter, the call-flow HTML shows you *how* they connect.
|
||||
|
||||
---
|
||||
|
||||
## Step 3 — Read the confidence tags
|
||||
|
||||
Every **inferred** relationship is labeled so you know how much to trust it:
|
||||
|
||||
> "Confidence tags — every inferred relationship is marked `EXTRACTED`, `INFERRED`, or `AMBIGUOUS`. You always know what was found vs guessed." `[github]`
|
||||
|
||||
| Tag | Meaning `[github]` (overview doc) | How much to trust |
|
||||
|---|---|---|
|
||||
| `EXTRACTED` | "directly found in source" | High — deterministic; for code this is the AST `[github]` |
|
||||
| `INFERRED` | "logically deduced" | Medium — a model reasoned it out |
|
||||
| `AMBIGUOUS` | "uncertain connections" | Low — verify before acting |
|
||||
|
||||
Rule of thumb: code edges are `EXTRACTED` from the tree-sitter AST with no model involved `[github]`, so they're solid; `INFERRED`/`AMBIGUOUS` edges came from a language model reading prose `[github]` — weight them down and confirm before you act. (Confidence-tag concept also covered in [01-overview-concepts.md](01-overview-concepts.md).)
|
||||
|
||||
---
|
||||
|
||||
## Step 4 — Community / cluster detection
|
||||
|
||||
Graphify groups related entities into **communities**, and you can rerun clustering at adjustable granularity *without* re-extracting (so it's free — no API calls):
|
||||
|
||||
```bash
|
||||
graphify cluster-only ./my-project # rerun clustering on existing graph [github]
|
||||
graphify cluster-only ./my-project --resolution 1.5 # more, smaller communities [github]
|
||||
graphify cluster-only ./my-project --exclude-hubs 99 # exclude p99-degree nodes from partitioning [github]
|
||||
```
|
||||
|
||||
The same controls exist as flags on a build run:
|
||||
|
||||
```bash
|
||||
/graphify . --cluster-only # rerun clustering, no re-extract [github]
|
||||
/graphify . --cluster-only --resolution 1.5 # more granular communities [github]
|
||||
/graphify . --cluster-only --exclude-hubs 99 # suppress utility super-hubs from god-node rankings [github]
|
||||
```
|
||||
|
||||
- **`--resolution`** raises granularity: a higher value (e.g. `1.5`) yields *more, smaller* communities; lower values yield fewer, larger ones `[github]`. Sweep it to find the topic-cluster scale that matches how you think about the project.
|
||||
- **`--exclude-hubs 99`** drops the very highest-degree utility nodes (e.g. a logger imported everywhere) from the partitioning / god-node ranking so they don't swamp the real structure `[github]`.
|
||||
|
||||
**Cross-community links** — relationships that bridge otherwise-separate clusters — are the thing the creator says plain visualizers (e.g. Obsidian) can't surface `[interview]`. There is **no** dedicated flag for them; they appear in the report as **"surprising connections"**:
|
||||
|
||||
> "Surprising connections — links between things that live in different files or modules. Ranked by how unexpected they are." `[github]`
|
||||
|
||||
So: clusters come from `cluster-only`/`--resolution`; the bridges between them are the ranked "surprising connections" in `GRAPH_REPORT.md`.
|
||||
|
||||
---
|
||||
|
||||
## Step 5 — The god-node diagnostic heuristic
|
||||
|
||||
A rough health signal from the creator:
|
||||
|
||||
> "If you have large amount of god nodes that means that something has gone wrong with the cohesion or maybe with your codebase ... if you have a decent number of god nodes ... things have worked fine." `[interview]`
|
||||
|
||||
Treat this as an `[interview]` heuristic, not a measured threshold — there's no defined "too many." `[unverified claim]` But it's testable: run `--exclude-hubs 99` to strip out the utility super-hubs and see whether a sane architecture remains underneath `[github]`. If a huge, sprawling god-node set survives even after excluding the obvious utility hubs, that's a soft signal of poor cohesion worth a closer look. (`--exclude-hubs` adjusts the *ranking*; it does not *fix* cohesion.)
|
||||
|
||||
---
|
||||
|
||||
## Putting it together: inspecting a fresh graph
|
||||
|
||||
A concrete sequence for a graph you've never seen (e.g. a new hire on an unfamiliar codebase — the interview's headline scenario `[interview]`):
|
||||
|
||||
```bash
|
||||
# 1. Build the graph (AST for code = free, no API calls) [github][interview]
|
||||
/graphify .
|
||||
|
||||
# 2. Read the high-level map FIRST: god nodes + surprising
|
||||
# connections are at the top of the report [github][interview]
|
||||
# -> open graphify-out/GRAPH_REPORT.md
|
||||
|
||||
# 3. See how the spine connects, visually
|
||||
graphify export callflow-html # [github]
|
||||
|
||||
# 4. Sanity-check cohesion: strip utility super-hubs, re-look
|
||||
graphify cluster-only . --exclude-hubs 99 # [github]
|
||||
# too many god nodes left? possible cohesion problem [interview / unverified]
|
||||
|
||||
# 5. Scalpel into specifics — extract from the graph, bounded
|
||||
graphify query "what connects auth to the database?" --dfs --budget 1500 # [github]
|
||||
graphify path "UserService" "DatabasePool" # [github]
|
||||
graphify explain "RateLimiter" # [github]
|
||||
|
||||
# 6. Tune cluster granularity if topics feel too coarse/fine
|
||||
graphify cluster-only . --resolution 1.5 # [github]
|
||||
```
|
||||
|
||||
Throughout: read the **confidence tag** on any edge before you act on it — `EXTRACTED` is trustworthy, `INFERRED`/`AMBIGUOUS` needs a second look `[github]`.
|
||||
|
||||
### Example queries to copy
|
||||
|
||||
```bash
|
||||
graphify query "what connects attention to the optimizer?" # [github]
|
||||
graphify query "show the auth flow" # [github]
|
||||
graphify path "DigestAuth" "Response" # [github]
|
||||
graphify explain "SwinTransformer" # [github]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Repeated / programmatic access (MCP)
|
||||
|
||||
For an assistant that hits the graph many times in a session, expose it as an MCP server instead of one-off CLI calls:
|
||||
|
||||
```bash
|
||||
python -m graphify.serve graphify-out/graph.json # [github]
|
||||
```
|
||||
|
||||
This gives structured tools: `query_graph`, `get_node`, `get_neighbors`, `shortest_path`, `list_prs`, `get_pr_impact`, `triage_prs` `[github]`. Note `get_neighbors` / `shortest_path` are the programmatic equivalents of `explain` / `path`. (PR-impact tooling — `graphify prs` — is covered in [08-workflows-and-use-cases.md](08-workflows-and-use-cases.md).)
|
||||
|
||||
---
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Goal | Do this |
|
||||
|---|---|
|
||||
| See the architecture spine | Read god nodes at top of `GRAPH_REPORT.md` `[github]` |
|
||||
| Visual call-flow | `graphify export callflow-html` `[github]` |
|
||||
| Find relevant nodes (semantic) | `graphify query "..."` (add `--dfs --budget N`) `[github]` |
|
||||
| Connect two named nodes | `graphify path "A" "B"` `[github]` |
|
||||
| Deep-dive one node | `graphify explain "Node"` `[github]` |
|
||||
| Re-cluster (free) | `graphify cluster-only . --resolution 1.5` `[github]` |
|
||||
| De-noise god-node ranking | `--exclude-hubs 99` `[github]` |
|
||||
| Bridges between clusters | "surprising connections" in the report `[github]` |
|
||||
| Judge trust of an edge | Read its `EXTRACTED` / `INFERRED` / `AMBIGUOUS` tag `[github]` |
|
||||
|
||||
Index: [00-README.md](00-README.md) · Next: [07-token-economics-and-updates.md](07-token-economics-and-updates.md) · Workflows: [08-workflows-and-use-cases.md](08-workflows-and-use-cases.md)
|
||||
|
||||
---
|
||||
|
||||
## Open questions / unverified
|
||||
|
||||
- **No `god-nodes` command exists.** God nodes are surfaced via `GRAPH_REPORT.md` and the assistant, not a dedicated CLI verb. `graphify query` is *semantic search*, not the structural hub ranking — don't conflate them. `[github]`
|
||||
- **"Too many god nodes = poor cohesion"** is an `[interview]` heuristic with no defined threshold. `[unverified claim]` `--exclude-hubs` lets you *test* it but does not fix cohesion.
|
||||
- **Cross-community links have no flag.** They map to the report's ranked "surprising connections" `[github]`; the interview's framing that Obsidian "can't do cross-community interaction" `[interview]` is the creator's claim, not independently verified.
|
||||
- **`--resolution` direction** (higher = more, smaller communities) is from the README's inline comments `[github]`; exact algorithm/parameter semantics aren't documented in the README.
|
||||
- **`--dfs --budget`** appears in the README command reference `[github]` but the exact budget units (tokens? nodes? hops?) and the `--budget` default are not spelled out there. `[unverified claim]`
|
||||
- **graphifylabs.ai** returns HTTP 403 to plain fetch, so any query syntax documented only on the official site is not directly verified here.
|
||||
- **Token-savings figures** ("70x"/"90x"/"20x") are sales claims handled in [07-token-economics-and-updates.md](07-token-economics-and-updates.md), not retrieval facts. `[unverified claim]`
|
||||
|
|
@ -0,0 +1,184 @@
|
|||
---
|
||||
summary: Where Graphify's token savings actually come from, how honest the headline numbers are, practical cost levers (local backends, incremental updates, budget flags), and how to keep the graph fresh.
|
||||
tags:
|
||||
- type/reference
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
- domain/llm
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
# 07 — Token Economics & Keeping the Graph Fresh
|
||||
|
||||
Where Graphify's token savings actually come from, how honest the headline numbers are, the practical levers you control, and how to keep the graph current with incremental updates instead of rebuilding from scratch.
|
||||
|
||||
> **Provenance:** Claims are tagged `[github]` (verified on 2026-06-03 by reading the raw `skill.md`, `__main__.py`, and `README.md` at the release line this guide anchors on — the `v0.8.x` line, which the brief refers to as "v8"; the disputed flags below were also checked against the `v1.0.0` tag and are absent there too), `[interview]` (creator interview — treat as intent/sales claims), `[community](url)`, or `[unverified claim]`. Several flags people repeat online do **not** exist in the shipped CLI; those are flagged explicitly below. Never assume a flag exists because it sounds plausible — check `--help` or `skill.md`.
|
||||
|
||||
Related docs: [overview](01-overview-concepts.md) · [code/AST ingest](03-ingesting-code-ast.md) · [docs ingest](04-ingesting-docs-knowledge.md) · [local models & backends](05-local-models-and-backends.md) · [querying & god nodes](06-querying-and-god-nodes.md) · [workflows](08-workflows-and-use-cases.md) · [best practices](09-best-practices-checklist.md).
|
||||
|
||||
---
|
||||
|
||||
## 1. Where the savings actually come from
|
||||
|
||||
Graphify does not "compress" your tokens. The savings come from three mechanics. Understanding them tells you exactly when savings will be large and when they'll be near zero.
|
||||
|
||||
### a) Code extraction is free (local AST)
|
||||
Code is parsed locally with tree-sitter into an abstract syntax tree. No API calls, no tokens, no cost — just CPU. `[github]` In the creator's words: *"If you are calling an LLM to parse your codebase, that doesn't make sense at all… the AST call is free of cost. There's no API call included."* `[interview]`
|
||||
|
||||
So for a pure-code corpus, **building** the graph is effectively free. Only **documents, papers, and images** go through a model backend (cloud LLM or a local SLM). `[github]` That single split — free code vs. paid docs — is the biggest cost lever you have. (See [04-ingesting-docs-knowledge.md](04-ingesting-docs-knowledge.md).)
|
||||
|
||||
### b) Querying the graph instead of dumping the corpus
|
||||
The real recurring savings are at **query time**. Instead of letting your assistant read dozens of raw files to orient itself, you query the pre-built graph and pull back only the relevant subgraph. `[github]` The skeptical-but-fair review puts it well: the savings come *"from precision — not from compression — allowing you to pay for relevant tokens, not orientation tokens."* [community](https://www.roborhythms.com/graphify-review/)
|
||||
|
||||
The creator's rule: *"Don't ask the LLM to go through the graph; ask the LLM to extract from the graph. The graph is the memory, the graph is the context."* `[interview]`
|
||||
|
||||
### c) God-nodes-first retrieval
|
||||
Query the **god nodes** (the highest-connectivity hub nodes / `GRAPH_REPORT.md`) first to get the architecture in one shot, then "scalpel" down into the specific subgraph you need. `[interview]` This avoids the expensive pattern of repeatedly reading files to reconstruct structure the graph already encodes. See [06-querying-and-god-nodes.md](06-querying-and-god-nodes.md).
|
||||
|
||||
---
|
||||
|
||||
## 2. How honest are the savings numbers?
|
||||
|
||||
**Treat every multiplier as corpus-dependent, not a guarantee.**
|
||||
|
||||
- The README/benchmark headline is **71.5x fewer tokens per query** — but that is from **one specific run**: a mixed corpus of Karpathy repos + 5 papers + 4 images. `[github]` The benchmark is printed automatically after a run via `graphify benchmark`. `[github]`
|
||||
- The creator is explicit that this is not a floor or a ceiling: *"There is no ceiling or floor in token savings here… 71.5 in my readme is for the repository I tried on. I've seen people getting 90x… and people getting 20x. It's totally corpus dependent."* `[interview]`
|
||||
- He also acknowledges the backlash directly: *"I have a lot of hate in my comments… 'Oh, 70 times, are you out of your mind?'"* `[interview]`
|
||||
|
||||
**Independent / skeptical takes (read these before quoting a number):**
|
||||
- An independent review reproduced the mechanism but found the realistic range is roughly **6.8x to 49x**, with the 71x figure being *"the upper bound on a curve… the tail of the distribution, not the middle."* It reports near-**1x–3x savings under ~100 files** and large multipliers only on very large repos. [community](https://www.roborhythms.com/graphify-review/)
|
||||
- Community criticism also targets non-technical claims — GitHub star-count growth tactics and missing credit for early PR contributors — separate from whether the tool works. [community](https://www.mindstudio.ai/blog/graphify-claude-code-knowledge-graph-large-codebase-70x)
|
||||
|
||||
**Bottom line:** savings scale with corpus size and how disciplined your querying is. Small corpora may save little or nothing (graph-building overhead isn't amortized). Don't promise a multiplier to anyone — measure your own with `graphify benchmark`. `[github]`
|
||||
|
||||
> **The download/star numbers** ("500K+ PyPI downloads", "43K stars") are creator sales claims from the interview. `[interview]` Third-party write-ups cite different figures (e.g. ~55K stars / ~450K downloads). [community](https://www.mindstudio.ai/blog/graphify-claude-code-knowledge-graph-large-codebase-70x) Treat all of these as **unverified and time-sensitive.** `[unverified claim]`
|
||||
|
||||
---
|
||||
|
||||
## 3. Practical cost levers
|
||||
|
||||
### Lever 1 — Use a local backend (Ollama) for documents
|
||||
Code is free regardless. The cost is documents. Route document extraction through a **local small language model via Ollama** instead of a paid cloud model. `[interview]` The creator's own workflow: *"local LLM for the extraction… free of cost as well rather than going for a cloud-based LLM."* `[interview]` Backends are installed as optional extras (e.g. `uv tool install "graphifyy[anthropic]"` for the cloud path). `[github]`
|
||||
|
||||
> Note: the package on PyPI is **`graphifyy`** (double *y*); the CLI command is **`graphify`**. `[github]` Exact Ollama/SLM setup steps live in [05-local-models-and-backends.md](05-local-models-and-backends.md). The *intent* (Ollama for docs to cut cost) is well-supported; treat specific Ollama flag syntax as `[unverified claim]` until confirmed against your installed `--help`.
|
||||
|
||||
### Lever 2 — Split large corpora and re-ingest in pieces
|
||||
Don't dump one giant corpus in a single shot. *"You can split your repository or your documents and then reingest them… rather than putting the whole chunk in one shot."* `[interview]` More context per pass also raises hallucination/dilution risk, so smaller, merged passes are both cheaper and more reliable. `[interview]` Each re-ingest should use `--update` (Section 4) so pieces merge instead of rebuilding.
|
||||
|
||||
### Lever 3 — Cap query answer size with `--budget`
|
||||
You can cap how many tokens a **query answer** returns:
|
||||
|
||||
```bash
|
||||
/graphify query "how does auth talk to the database?" --budget 1500 # cap answer at ~1500 tokens
|
||||
```
|
||||
|
||||
Default budget is ~2000 tokens; results are ranked by relevance and truncated at the budget. `[github]`
|
||||
|
||||
> **`--budget` vs `--token-budget` — two different flags for two different stages** (both verified in the v0.8.30 README): `--budget N` caps **query answer** size (retrieval), while **`--token-budget N`** controls the **semantic chunk size sent to the model during extraction** (smaller = better for local/small models). `[github]` Don't confuse them — `--budget` is for `query`, `--token-budget` is for `extract`. See [05-local-models-and-backends.md](05-local-models-and-backends.md) for `--token-budget` tuning.
|
||||
|
||||
### Lever 4 — Tune extraction cost with `--token-budget` and `--max-concurrency`
|
||||
For the **extraction** stage (where document tokens are actually spent), v0.8.30 exposes two real flags: `[github]`
|
||||
|
||||
```bash
|
||||
graphify extract ./docs --token-budget 30000 # smaller semantic chunks for local/small models
|
||||
graphify extract ./docs --max-concurrency 2 # fewer parallel LLM calls (useful for local inference)
|
||||
graphify extract ./docs --api-timeout 900 # longer HTTP timeout for slow local models (default 600s)
|
||||
```
|
||||
|
||||
- **`--token-budget N`** — size of each semantic chunk sent to the model. Smaller chunks fit small/local model context windows and reduce tokens per call. `[github]`
|
||||
- **`--max-concurrency N`** — number of parallel LLM calls; lower it for local inference so you don't overwhelm a single GPU/CPU. `[github]`
|
||||
- **`--api-timeout` / `GRAPHIFY_API_TIMEOUT`** — HTTP timeout in seconds (default 600). `[github]`
|
||||
|
||||
Under the hood the skill also parallelizes automatically (semantic-extraction subagents over chunks, AST in parallel), so these flags tune — rather than introduce — concurrency. `[github]` Exact Ollama setup and a worked example live in [05-local-models-and-backends.md](05-local-models-and-backends.md).
|
||||
|
||||
---
|
||||
|
||||
## 4. Incremental updates — always `--update` when adding data
|
||||
|
||||
**The single most important habit.** When you add new files, run `--update` so Graphify re-extracts only what changed and merges into the existing graph, instead of rebuilding from scratch:
|
||||
|
||||
```bash
|
||||
/graphify ./raw --update # re-extract only new/changed files, merge into existing graph
|
||||
```
|
||||
|
||||
The creator hammered this twice: *"When you are ingesting more data, make sure you use the word update — `graphify update` — so it doesn't start fresh from the beginning."* and *"Always make sure you put `graphify update` when you are ingesting new files or codebases or documents, so it doesn't have to go through the whole code once more."* `[interview]`
|
||||
|
||||
**How `--update` knows what changed:**
|
||||
- **SHA-256 content hashing** of each file. Unchanged files (same hash) are skipped; only changed/new files are re-processed. The cache lives in `graphify-out/cache/`. `[github]` *"I've used hashing strategy SHA-256 in here so it will start from where it left."* `[interview]`
|
||||
- **Deduplication** of entities so the same node isn't created twice across documents/passes. `[github]` *"I've also got some deduplication techniques so there will be very few collisions of the entities."* `[interview]`
|
||||
|
||||
This is what makes Lever 2 (split + re-ingest) safe: each piece merges cleanly via hash + dedup rather than producing a duplicated or reset graph.
|
||||
|
||||
### "Overwrite" / `--force` semantics — read this carefully
|
||||
A re-run of the full pipeline (without `--update`) rebuilds the graph rather than merging incrementally. `[github]`
|
||||
|
||||
> **`--force`:** The `--force` flag **does exist** in v0.8.30 (equivalently set `GRAPHIFY_FORCE=1`). It overwrites `graph.json` **even when the rebuild produces fewer nodes**, which is exactly what you want after a refactor that deleted files — otherwise the old "ghost" nodes linger. `[github]`
|
||||
>
|
||||
> ```bash
|
||||
> graphify extract . --force # rebuild, overwrite even if fewer nodes (clears ghost nodes after deletions)
|
||||
> graphify update ./src --force # same, while still merging changed files
|
||||
> ```
|
||||
>
|
||||
> Mental model: **`--update` = merge changed files; plain re-run = rebuild; `--force` = rebuild and overwrite even if the graph shrinks.** `[github]`
|
||||
|
||||
---
|
||||
|
||||
## 5. Auto-rebuild: git hooks and `--watch`
|
||||
|
||||
To avoid forgetting to update, automate it.
|
||||
|
||||
### Git commit hook (recommended for code)
|
||||
```bash
|
||||
graphify hook install # post-commit hook: re-runs AST on changed files, rebuilds graph after every commit
|
||||
graphify hook status # check if installed
|
||||
graphify hook uninstall # remove it
|
||||
```
|
||||
After each commit the hook detects changed **code** files (via `git diff HEAD~1`), re-runs AST extraction on just those, and rebuilds `graph.json` + `GRAPH_REPORT.md`. `[github]`
|
||||
|
||||
> **Important limitation:** the hook handles **code only**. Doc/image changes are *ignored* by the hook — you must run `/graphify ./raw --update` manually after editing docs, because those need the (paid) LLM semantic re-pass. `[github]`
|
||||
|
||||
### Watch mode (for active/agentic sessions)
|
||||
```bash
|
||||
/graphify <path> --watch # watch folder, auto-rebuild on code changes (no LLM)
|
||||
```
|
||||
Code saves trigger an instant AST-only rebuild (free, debounced ~3s). Doc/image changes write a `graphify-out/needs_update` flag and notify you to run `--update`. `[github]` Useful when parallel agents are writing code in waves. `[github]`
|
||||
|
||||
---
|
||||
|
||||
## 6. The staleness problem (and how the above mitigates it)
|
||||
|
||||
The interview's most candid admission is the operator's real pain point: assets go stale and re-grooming them is a chore. The interviewer: *"My assets become stale over time, so the process of having to remember to groom your data… becomes a bit of a constraint."* `[interview]` The creator's framing is a *"digital twin / enterprise brain that gets smarter with time"* via incremental updates `[interview]` — but that only holds if the graph is actually kept current.
|
||||
|
||||
**How to keep it from going stale (all verified):**
|
||||
- **Code:** install the commit hook (`graphify hook install`) or run `--watch` during active work — code rebuilds are free and automatic. `[github]`
|
||||
- **Docs/knowledge:** the hook and watch mode will **not** auto-update these (they cost tokens). Build a habit (or a scheduled job) of running `/graphify ./raw --update` after adding/editing documents. `[github]`
|
||||
- **Merge, don't reset:** always `--update` so SHA-256 + dedup re-process only what changed and merge it in. `[github]`
|
||||
|
||||
A realistic personal cadence: hook on for every code repo; a periodic `--update` on your docs/notes folder (manually or via cron) so the knowledge base never drifts far from reality.
|
||||
|
||||
---
|
||||
|
||||
## Quick reference (verified against v0.8.30)
|
||||
|
||||
| Goal | Command | Cost |
|
||||
|------|---------|------|
|
||||
| First build of a corpus | `/graphify <path>` | code free; docs/images cost backend tokens |
|
||||
| Add/changed data later | `/graphify <path> --update` | only changed files re-processed |
|
||||
| Richer inference | `/graphify <path> --mode deep` | higher (more LLM passes) |
|
||||
| Auto-rebuild on commit (code) | `graphify hook install` | free (AST only) |
|
||||
| Live auto-rebuild (code) | `/graphify <path> --watch` | free (AST only) |
|
||||
| Query, capped answer | `/graphify query "..." --budget 1500` | small (subgraph only) |
|
||||
| Smaller chunks for local models | `graphify extract <path> --token-budget 30000` | lowers tokens/call |
|
||||
| Fewer parallel model calls | `graphify extract <path> --max-concurrency 2` | eases local GPU/CPU load |
|
||||
| Rebuild, overwrite even if smaller | `graphify extract <path> --force` | clears ghost nodes after deletions |
|
||||
| Measure your own savings | `graphify benchmark` | free |
|
||||
|
||||
---
|
||||
|
||||
## Open questions / unverified
|
||||
|
||||
- **Flag set is version-dependent.** All of `--budget`, `--token-budget`, `--max-concurrency`, `--api-timeout`, and `--force` are documented in the **v0.8.30** release README (verified by grepping the raw file). `[github]` The shorter `main`-branch README omits them, so if you're on a different build, confirm with `graphify --help`.
|
||||
- **Ollama backend specifics** — the *intent* (local SLM for docs to save cost) is well-supported `[interview]`, and backends install as extras `[github]`, but exact Ollama setup/flags were not confirmed from the raw repo here. See [05-local-models-and-backends.md](05-local-models-and-backends.md) and confirm with `--help`. `[unverified claim]`
|
||||
- **Savings multipliers** — 71.5x is one Karpathy-corpus benchmark; independent testing suggests ~6.8x–49x realistically, ~1x–3x under ~100 files. Always measure your own. [community](https://www.roborhythms.com/graphify-review/)
|
||||
- **Download/star counts** — creator figures (500K downloads, 43K stars) differ from third-party figures; all are time-sensitive. `[unverified claim]`
|
||||
|
|
@ -0,0 +1,231 @@
|
|||
---
|
||||
summary: End-to-end playbooks for using Graphify across code onboarding, bug tracing, AI-slop auditing, persistent digital-twin setup, PR impact analysis, and the creator's own daily workflow.
|
||||
tags:
|
||||
- type/howto
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
- domain/knowledge-graphs
|
||||
source: cc-os
|
||||
date: 2026-06-08
|
||||
---
|
||||
# Workflows & Use Cases
|
||||
|
||||
End-to-end playbooks for running Graphify as a one-person operation across code, documents, and a personal knowledge base. Every command below is verbatim from the v8 README; concepts are covered in the sibling docs and cross-linked rather than re-explained.
|
||||
|
||||
> **Provenance:** commands are tagged `[github]` (verified from the [v8 README](https://raw.githubusercontent.com/safishamsi/graphify/v8/README.md)). Claims from the creator interview are tagged `[interview]` and headline savings numbers are **sales claims**, not benchmarks. See [00-README](00-README.md) for the tagging scheme.
|
||||
|
||||
> **Skill vs. CLI prefix:** commands written `/graphify ...` run through the IDE skill (your assistant's model provides the LLM calls for documents — no separate API key). Commands written `graphify ...` are the plain CLI. `graphify extract ...` is the headless form used in CI or when you want to pin a backend explicitly. `[github]`
|
||||
|
||||
---
|
||||
|
||||
## Playbook 1 — Onboard fast to an unfamiliar codebase
|
||||
|
||||
For a new hire, contractor, or your own first hour in a repo you inherited. The interview frames this as the flagship use case: replace days of code-spelunking and Slack pings with one map. `[interview]`
|
||||
|
||||
1. Map the whole repo. Code is parsed locally via AST — no API calls, no token cost. `[github]`
|
||||
```bash
|
||||
/graphify .
|
||||
```
|
||||
This writes `graphify-out/graph.html`, `graphify-out/GRAPH_REPORT.md`, and `graphify-out/graph.json`. `[github]`
|
||||
2. Read the report first, not the source. Open `graphify-out/GRAPH_REPORT.md` for architectural highlights. `[github]`
|
||||
3. **God nodes first.** Ask the graph for its most-connected concepts before reading any file — these are the architectural chokepoints "everything flows through." `[github]` See [06-querying-and-god-nodes](06-querying-and-god-nodes.md) for the god-node querying pattern.
|
||||
```bash
|
||||
/graphify query "what are the god nodes / most connected components?"
|
||||
```
|
||||
Heuristic from the creator: a *huge* pile of god nodes signals weak cohesion or a structural mess; a *modest* set means the architecture is sane. `[interview]`
|
||||
4. Trace the architecture from those hubs outward.
|
||||
```bash
|
||||
/graphify explain "AuthService"
|
||||
/graphify query "what connects auth to the database?"
|
||||
```
|
||||
5. Generate a readable architecture page with Mermaid call-flow diagrams to skim the system visually.
|
||||
```bash
|
||||
graphify export callflow-html
|
||||
```
|
||||
`[github]`
|
||||
|
||||
**ROI framing (sales claim):** the creator pitches this as replacing a $150/hr engineer's multi-day ramp-up with a few free minutes. Treat the dollar figure as illustrative. `[interview]`
|
||||
|
||||
---
|
||||
|
||||
## Playbook 2 — Fix a bug / ticket by tracing dependencies
|
||||
|
||||
You have a ticket ("users hitting an error on the backend") and need the blast radius before you touch code.
|
||||
|
||||
1. Make sure the graph reflects current `HEAD` (see [Playbook 4](#playbook-4--build-a-persistent-digital-twin--second-brain) for keeping it fresh):
|
||||
```bash
|
||||
/graphify --update
|
||||
```
|
||||
`[github]`
|
||||
2. Locate the suspect component via god nodes / a targeted query, then expand its neighborhood:
|
||||
```bash
|
||||
/graphify explain "RateLimiter"
|
||||
```
|
||||
`[github]`
|
||||
3. Trace the dependency chain between the failing area and its likely root using shortest path:
|
||||
```bash
|
||||
/graphify path "UserService" "DatabasePool"
|
||||
```
|
||||
This returns the shortest connection between two entities — the chain of files/functions the bug can propagate through. `[github]`
|
||||
4. Confirm the blast radius before editing:
|
||||
```bash
|
||||
/graphify query "what depends on DatabasePool?"
|
||||
```
|
||||
5. Query the graph, don't dump the corpus. Ask the assistant to **extract from the graph** rather than read every file — the graph is your context. `[interview]` Token mechanics in [07-token-economics-and-updates](07-token-economics-and-updates.md).
|
||||
|
||||
---
|
||||
|
||||
## Playbook 3 — Audit AI-generated "slop" / junk code
|
||||
|
||||
The creator's pitch for non-experts shipping AI-written code fast: map it, then let the graph surface what's wrong. `[interview]`
|
||||
|
||||
1. Map the suspect tree:
|
||||
```bash
|
||||
/graphify .
|
||||
```
|
||||
2. Look at god-node count as a smell test. An unusually large or sprawling set of god nodes points at poor cohesion — a hallmark of slop. `[interview]`
|
||||
3. Hunt for orphans and dead ends — nodes with no meaningful connections often mean dead or duplicated code:
|
||||
```bash
|
||||
/graphify query "which components are disconnected or have no dependents?"
|
||||
```
|
||||
`[github]`
|
||||
4. Use relationship confidence tags. Every inferred edge is marked `EXTRACTED`, `INFERRED`, or `AMBIGUOUS`, so you can tell what was found in the code vs. guessed — `AMBIGUOUS` clusters are good places to look for confusion. `[github]`
|
||||
5. For shell scripts (no AST support yet): ingest them as **documents** so the model does a semantic extraction instead of AST parsing. `[interview]` The creator calls shell scripts "flat" and treats document-mode as the temporary workaround. Note `.sh`/`.bash` *do* appear in the v8 supported-extensions list `[github]`, so behavior may have shifted since the interview — verify on your version.
|
||||
|
||||
---
|
||||
|
||||
## Playbook 4 — Build a persistent "digital twin" / second brain
|
||||
|
||||
The cross-project setup you actually want: one queryable memory spanning your code repos, documents, and a personal KB, kept fresh over time. The creator's framing is a "digital twin" that "gets smarter with time" via incremental updates. `[interview]`
|
||||
|
||||
### Recommended structure: many local graphs, one global view
|
||||
|
||||
Don't try to cram everything into a single monolithic extraction. **Build a graph per project, then fold them into one cross-project global graph.** This keeps each project's `--update` cheap and isolated while still giving you one place to query across everything. `[github]`
|
||||
|
||||
1. In each repo / doc folder / KB vault, build its own graph:
|
||||
```bash
|
||||
/graphify . # in a code repo
|
||||
/graphify ./notes # in your personal KB / Obsidian vault
|
||||
```
|
||||
Code is free/local; documents and notes go through a model — use a **local backend** to keep this free and private (see [Playbook 6](#playbook-6--the-creators-day-to-day-workflow) and [05-local-models-and-backends](05-local-models-and-backends.md)). `[github]` `[interview]`
|
||||
2. Register each project's graph into the global graph with a memorable name:
|
||||
```bash
|
||||
graphify global add graphify-out/graph.json my-api
|
||||
graphify global add graphify-out/graph.json my-notes
|
||||
graphify global list # see all registered repos + node/edge counts
|
||||
```
|
||||
The global graph lives at `~/.graphify/global.json`. `[github]`
|
||||
3. (Alternative one-step form) Extract and register in a single command:
|
||||
```bash
|
||||
graphify extract ./notes --global --as my-notes
|
||||
```
|
||||
`[github]`
|
||||
4. Query across the whole twin via your normal `query` / `path` / `explain` calls, leaning on god nodes to navigate. `[github]` `[interview]`
|
||||
|
||||
### Ingest cadence (keep the twin from going stale)
|
||||
|
||||
The user's real concern is assets going stale. Split the cadence by content type:
|
||||
|
||||
- **Code — automate it.** Install the git hook once per repo so the graph rebuilds on every commit (AST only, no API cost), and `graph.json` auto-merges instead of conflicting:
|
||||
```bash
|
||||
graphify hook install
|
||||
```
|
||||
`[github]`
|
||||
- **Docs / KB / papers — refresh on demand.** These need a model call, so run them deliberately after you add or change material. `--update` re-extracts only changed files using SHA-256 hashing, so it resumes where it left off rather than rebuilding: `[interview]` `[github]`
|
||||
```bash
|
||||
/graphify --update # or: /graphify ./notes --update
|
||||
```
|
||||
- **Re-register after a doc refresh** so the global view picks up the new nodes:
|
||||
```bash
|
||||
graphify global add graphify-out/graph.json my-notes
|
||||
```
|
||||
- **Ingest in splits, not one giant dump.** The creator recommends splitting a large repo or document set and re-ingesting in pieces rather than pushing the whole corpus in one shot — it merges in more cleanly and avoids context dilution. `[interview]`
|
||||
|
||||
### Pulling external knowledge into the brain
|
||||
|
||||
Add papers and videos straight into a graph (videos transcribed locally via faster-whisper): `[github]`
|
||||
```bash
|
||||
/graphify add https://arxiv.org/abs/1706.03762
|
||||
/graphify add <youtube-url>
|
||||
/graphify add https://... --author "Name" --contributor "Name"
|
||||
```
|
||||
`[interview]` describes this as turning a Stanford lecture or any video/audio into a queryable section-by-section map instead of re-watching it.
|
||||
|
||||
---
|
||||
|
||||
## Playbook 5 — PR impact analysis
|
||||
|
||||
For reviewing your own or a team's pull requests with graph-aware impact. Requires the GitHub CLI auth your repo already uses.
|
||||
|
||||
1. See the dashboard — every open PR with CI state, review status, worktree mapping, and graph impact:
|
||||
```bash
|
||||
graphify prs
|
||||
```
|
||||
2. Deep-dive a specific PR's blast radius:
|
||||
```bash
|
||||
graphify prs 42
|
||||
```
|
||||
3. Let the graph rank your review queue (auto-detects backend from env):
|
||||
```bash
|
||||
graphify prs --triage
|
||||
```
|
||||
4. Spot merge-order risk — PRs touching the same graph communities are likely to conflict:
|
||||
```bash
|
||||
graphify prs --conflicts
|
||||
```
|
||||
5. Scope it when you have many branches:
|
||||
```bash
|
||||
graphify prs --base main # only PRs targeting main
|
||||
graphify prs --worktrees # worktree → branch → PR mapping
|
||||
graphify prs --repo owner/repo # a different repo
|
||||
```
|
||||
All `[github]`. To pin the triage backend (e.g. keep it local/cheap):
|
||||
```bash
|
||||
GRAPHIFY_TRIAGE_BACKEND=ollama graphify prs --triage
|
||||
```
|
||||
`[github]`
|
||||
|
||||
---
|
||||
|
||||
## Playbook 6 — The creator's own day-to-day workflow
|
||||
|
||||
The distilled habit the creator described for himself. `[interview]`
|
||||
|
||||
1. **Local LLM for documents.** Extract docs/notes with Ollama so it's free and stays on your machine — no cloud, no per-token cost: `[interview]` `[github]`
|
||||
```bash
|
||||
graphify extract ./docs --backend ollama
|
||||
```
|
||||
Code never needs this (it's local AST already). Tuning for small GPUs:
|
||||
```bash
|
||||
GRAPHIFY_OLLAMA_NUM_CTX=8192 graphify extract ./docs --backend ollama
|
||||
GRAPHIFY_OLLAMA_KEEP_ALIVE=0 graphify extract ./docs --backend ollama # unload after each chunk
|
||||
graphify extract ./docs --backend ollama --token-budget 4000 # smaller chunks for local models
|
||||
```
|
||||
`[github]` Full backend setup in [05-local-models-and-backends](05-local-models-and-backends.md).
|
||||
2. **God nodes first.** Always open with the most-connected concepts to get the map before drilling in. `[interview]`
|
||||
3. **Query the graph, not the corpus.** Prompt the assistant to *extract from the graph* — the graph is the context/memory. Never ask the LLM to read the whole corpus again. `[interview]`
|
||||
4. **Always `--update` when ingesting new material** so a later query doesn't force a full re-read. `[interview]`
|
||||
```bash
|
||||
/graphify --update
|
||||
```
|
||||
5. **Lock the query-first behavior in.** Register Graphify with your assistant so it consults the graph before grepping/reading files:
|
||||
```bash
|
||||
graphify install # or: graphify claude install
|
||||
```
|
||||
`[github]` The creator notes he's working toward hard-wiring "go to the graph first" into the agent's behavior. `[interview]`
|
||||
|
||||
---
|
||||
|
||||
## Token-savings note
|
||||
|
||||
The interview's headline figures — "71.5x" in the README, with reports of "20x to 90x" from users — are **sales claims** and explicitly described as corpus-dependent with "no ceiling or floor." Do not treat them as guarantees. `[interview]` See [07-token-economics-and-updates](07-token-economics-and-updates.md).
|
||||
|
||||
---
|
||||
|
||||
## Open questions / unverified
|
||||
|
||||
- **Shell-script handling drift.** The interview says shell scripts aren't AST-supported and must be ingested as documents, but the v8 supported-extensions list includes `.sh`/`.bash`. `[github]` The document-mode workaround in Playbook 3 may be obsolete on current versions — confirm against your installed build. `[unverified claim]`
|
||||
- **`graphify-out/` reuse across projects in the global graph.** Each `graphify .` writes to a local `graphify-out/`; the exact path you register with `graphify global add` matters. Confirm whether re-running `global add` with the same name *replaces* or *appends* — not stated in the README. `[unverified claim]`
|
||||
- **`/graphify add` for KB sources** (arxiv/youtube) is documented `[github]`, but whether added external sources are automatically included when you fold the project into the global graph is unverified. `[unverified claim]`
|
||||
- **Self-learning / domain-adapting "smarter over time" layer** and **Slack/Notion/meeting connectors** are described by the creator as in-progress/future, not shipped in v8. `[interview]` Do not build a workflow that depends on them yet.
|
||||
- **Official site** (graphifylabs.ai) blocks plain fetch (403); documented workflows here are anchored on the v8 README rather than the marketing site.
|
||||
|
|
@ -0,0 +1,180 @@
|
|||
---
|
||||
summary: Architecture Decision Records for the Claude Code personal memory system — 14 decisions covering memory-type split, tooling choices, vault structure, freshness model, and graph connectivity.
|
||||
tags:
|
||||
- type/adr
|
||||
- domain/knowledge-graphs
|
||||
- domain/llm
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
source: cc-os
|
||||
date: 2026-06-09
|
||||
---
|
||||
|
||||
# Claude Code Memory System — Architecture Decision Records
|
||||
|
||||
A running log of decisions and *why*. Format per entry: Context · Decision · Rationale · Alternatives rejected · Status. Newest decisions extend the log; supersede rather than delete.
|
||||
|
||||
---
|
||||
|
||||
## ADR-001 — Two memory types, kept as separate systems
|
||||
|
||||
- **Context**: Earlier attempts to make one tool serve both "what happened" and "how do we do X" felt forced (e.g. trying to make memsearch filter knowledge by tags).
|
||||
- **Decision**: Model **episodic** memory and **semantic/knowledge** memory as two separate systems with different tools.
|
||||
- **Rationale**: They have different lifecycles (episodic accretes and decays; knowledge is deliberately maintained), different write paths (auto-captured vs curated with guardrails), and different query patterns ("when did we…" vs "how do we…"). Separation dissolves the earlier integration tension entirely.
|
||||
- **Alternatives rejected**: One unified store (memsearch-for-everything, or a single `thoughts` table) — conflates the two and forces awkward filtering.
|
||||
- **Status**: Accepted.
|
||||
|
||||
## ADR-002 — memsearch for the episodic layer
|
||||
|
||||
- **Context**: Need timeline/"what happened" memory that's NL-queryable and lazy.
|
||||
- **Decision**: Adopt **memsearch** (Zilliz) off-the-shelf for episodic memory.
|
||||
- **Rationale**: It already implements the daily-notes + "dreaming" pattern and the markdown-as-truth / disposable-shadow-index philosophy. Embedded **Milvus Lite** (single file), hybrid BM25+vector+RRF search, local ONNX embeddings (no API key/cost), a FileWatcher that handles deletions — **no Docker, no server**. Two-line install.
|
||||
- **Alternatives rejected**: claude-mem (MCP-based — Claude must actively call search; opaque blobs vs readable markdown; overkill features). Hand-building daily notes + dreaming ourselves (reinventing a solved tool).
|
||||
- **Status**: Accepted.
|
||||
|
||||
## ADR-003 — Flat vault with namespaced tags, not folders
|
||||
|
||||
- **Context**: Want a flat Obsidian vault with tags as virtual indexes, and cross-cutting filters (client × tool × convention).
|
||||
- **Decision**: One **flat markdown vault**; organize via **namespaced, nested tags** (`tool/`, `client/`, `domain/`, `convention/`, `scope/`). Slashes are valid Obsidian nested tags, so `#tool` matches all children.
|
||||
- **Rationale**: A note can carry several namespaces at once (`tool/semrush` + `client/clientx` + `convention/react-ts`) — folders can't express that. Enables "filter by client+tool to narrow the index." Enumerable virtual indexes ("what clients/tools exist").
|
||||
- **Alternatives rejected**: Folder hierarchy (single-axis; can't do cross-cutting filters). Pure-prefix path filtering via memsearch `source_prefix` (would force directories back in).
|
||||
- **Trade-off accepted**: Tags give the human/Obsidian free filtering, but the AI gets nothing for free from tags — they must be materialized into a queryable index.
|
||||
- **Status**: **Refined by ADR-011** (type/ and project/ namespaces added; hierarchy-vs-facets clarified). Core decision — flat vault, namespaced tags — stands.
|
||||
|
||||
## ADR-004 — SQLite + Sequel (Ruby) tag index as the knowledge-layer cache
|
||||
|
||||
- **Context**: The AI can't use Obsidian tags directly; tag filtering needs a machine-queryable index.
|
||||
- **Decision**: A small **Ruby program using the Sequel ORM over SQLite**, exposed as a **CLI**. Schema: `files(path, mtime, summary, scope)`, `tags(name)`, `files_tags` join (`many_to_many`). The summary is a **column on `files`** (an attribute), not a join.
|
||||
- **Rationale**: Normalized `tags` table makes enumerating the vocabulary a first-class cheap query. The `summary` column is what turns the index from a *finder* into a *router* — the AI sees enough to pick a file without opening it (progressive disclosure, low tokens). **Failure-mode guard**: markdown is always authoritative; the SQLite file is a disposable cache that is never synced and can be rebuilt from frontmatter anytime.
|
||||
- **Alternatives rejected**: Plain-markdown generated `INDEX.md` (must regenerate; grep-at-scale is token-heavy). Frontmatter grep on demand (scales badly). Milvus/Postgres for knowledge (overkill).
|
||||
- **Query output**: returns **path + summary + matched tags** — tags are cheap and show *why* a result matched.
|
||||
- **Status**: **Superseded by ADR-010** (Graphify replaces the Ruby/SQLite tag index). The `summary` + namespaced-tag frontmatter is **retained as note metadata**; only the bespoke Ruby/SQLite index and CLI are dropped.
|
||||
|
||||
## ADR-005 — Structured-first; semantic search over the vault deferred
|
||||
|
||||
- **Context**: Tag filtering may miss notes whose wording doesn't match the query.
|
||||
- **Decision**: Ship the knowledge layer **structured-only** (tags + summaries). **Defer** meaning-based search until it demonstrably bites.
|
||||
- **Rationale**: Structured tagging is the lightweight/fast thing; follow the "only level up when it bites" principle.
|
||||
- **Status**: **Superseded by ADR-010.** Graphify makes the knowledge layer a graph from day one, giving structured *and* connection-based recall together. The "only level up when it bites" instinct carries forward to whether a *vector* layer is ever needed on top of the graph.
|
||||
|
||||
## ADR-006 — QMD as the (deferred) semantic-over-knowledge layer
|
||||
|
||||
- **Context**: When ADR-005's structured-only proves insufficient, want a set-and-forget semantic layer over the vault, local and Docker-free.
|
||||
- **Decision**: Earmark **QMD** (github.com/tobi/qmd) for that role; do **not** install yet.
|
||||
- **Rationale**: Local markdown search using **SQLite + FTS5/BM25 + local vector embeddings (EmbeddingGemma-300M GGUF) + LLM rerank**; CLI + optional MCP server; no Docker, no API keys. Complements the tag index (QMD filters by path/collection context, not first-class frontmatter tags).
|
||||
- **Alternatives rejected**: Pointing memsearch at the vault (mixes episodic and knowledge corpora; its filtering is path-prefix not tags). A bespoke embedding index (reinvents QMD).
|
||||
- **Status**: **Superseded by ADR-010.** Graphify's knowledge graph fills the semantic-recall role (traversal/`explain` over connections) without a separate vector system, so QMD is no longer earmarked. Revisit a vector layer only if graph traversal demonstrably misses cases where embedding similarity would win.
|
||||
|
||||
## ADR-007 — Lazy freshness: write-hook + session-start reconcile, no daemon/cron
|
||||
|
||||
- **Context**: The cache must reflect new/edited/deleted/renamed notes without becoming a resource hog or going stale.
|
||||
- **Decision**: **Option A (lazy).** A `PostToolUse` hook updates the index on **AI** writes (single-file, prunes on delete). **Manual** edits are caught by a **session-start reconcile** (`index update --since` + prune of vanished paths). **No daemon, no cron.**
|
||||
- **Rationale**: The AI is the primary writer, so write-time hooks give event-driven freshness with no polling. The user rarely edits the vault by hand, so a session-start reconcile is enough; a continuous `inotify` daemon would add an always-on process to manage for negligible benefit.
|
||||
- **Alternatives rejected**: `inotify`/`listen` daemon (live freshness, but always-on process to manage — unnecessary). Cron reconcile (session-start covers it).
|
||||
- **Status**: Accepted.
|
||||
|
||||
## ADR-008 — Markdown-as-truth; sync the vault, not the indexes
|
||||
|
||||
- **Context**: Must be accessible on a VPS / multiple machines but run local-fast.
|
||||
- **Decision**: Sync the **markdown vault** to a remote machine via **git or Syncthing** (choice deferred to build time). **Graphs/indexes (Milvus Lite, Graphify `graphify-out/`) are rebuilt per machine and never synced.**
|
||||
- **Rationale**: Markdown is plain text — git/Syncthing sync it trivially; lazy (hourly or continuous-async) is enough. Indexes are disposable caches; syncing binary DBs invites conflicts for no gain. Local reads stay fast; ownership and portability stay with the user.
|
||||
- **Alternatives rejected**: Hosted DBs (always-remote, adds per-query latency and monthly cost, conflicts with local-fast; ownership weaker). Only worth it for real-time cross-tool memory, which was deemed overkill.
|
||||
- **Status**: Accepted.
|
||||
|
||||
## ADR-009 — Package as a global Claude Code plugin with skills
|
||||
|
||||
- **Context**: Every project, on every machine, should know how to use the vault — write conventions, query patterns, the hooks, and the CLI — without per-project setup.
|
||||
- **Decision**: Ship hooks + scripts + CRUD know-how as a **global Claude Code plugin with skills**, installed at the user level.
|
||||
- **Rationale**: Skills carry the "when to write / what conventions / how & when to query" guidance to the model; the plugin registers the session-start / session-end / PostToolUse hooks and wires up Graphify. Global install = consistent behavior everywhere; single source of truth for the conventions themselves.
|
||||
- **Status**: Accepted.
|
||||
|
||||
## ADR-010 — Graphify knowledge graph as the knowledge layer (supersedes ADR-004/005/006)
|
||||
|
||||
- **Context**: ADR-004 specced a hand-built Ruby/Sequel/SQLite tag index (+ CLI) as the machine-queryable layer over the vault, with ADR-005/006 deferring meaning-based recall to a future QMD vector layer. Before building any of it, **Graphify** (`graphify`, PyPI `graphifyy`) was evaluated — a tool that turns a folder into a queryable knowledge graph (local tree-sitter AST for code, local-SLM entity/relationship extraction for docs).
|
||||
- **Decision**: Use **Graphify as the knowledge-layer engine** over the vault, with a **local Ollama** backend for doc extraction and free AST for per-project code graphs. **Drop** the Ruby/SQLite tag-index CLI (ADR-004) and the earmarked QMD layer (ADR-006); **retain** the `summary` + namespaced-tag frontmatter from ADR-003/004 as note metadata.
|
||||
- **Rationale**: One off-the-shelf tool delivers both what the tag index was for (structured retrieval) and what QMD was deferred for (connection/meaning-based recall via graph traversal + `explain`) — without writing or maintaining a bespoke index, and without a vector store. Code graphs come free. Keeps the markdown-as-truth, no-Docker, no-API-key, local-first properties.
|
||||
- **What's retained / changed**: `summary` stays the human-written router hint Graphify does not generate; namespaced tags stay useful for Obsidian filtering and as node attributes. How tightly metadata feeds graph queries is a **build-time refinement**, not settled here.
|
||||
- **Trade-off accepted**: Graphify's `--update` doesn't prune deleted nodes (stale-node drift) — mitigated by a periodic `--force` rebuild on the session-start staleness check (ADR-007's lazy model still applies). Graphify also moves fast (flags are version-dependent; anchor to the installed version) and its headline token-savings numbers are corpus-dependent — benchmark your own.
|
||||
- **Alternatives rejected**: Building the Ruby/SQLite index as originally planned (more code to own; no semantic recall); adding QMD as a second system on top (two stores where one graph suffices).
|
||||
- **Status**: Accepted.
|
||||
|
||||
## ADR-011 — Faceted tag taxonomy: six independent namespaces (refines ADR-003)
|
||||
|
||||
_Date: 2026-06-04_
|
||||
|
||||
- **Context**: ADR-003 introduced five namespaces (`tool/`, `client/`, `domain/`, `convention/`, `scope/`). During vault-reuse assessment it became clear that (1) the existing vault uses a de-facto first-tag convention for note kind that should be made explicit and machine-queryable, and (2) for a freelancer working many projects per client, project identity deserves a first-class namespace.
|
||||
- **Decision**: Knowledge-vault notes are classified by **six independent, flat tag facets** that sit side-by-side, never nested into one another:
|
||||
- `type/` — note kind: `research`, `howto`, `adr`, `hub`, `plan`, `log`, `clip`, etc.
|
||||
- `client/` — which client
|
||||
- `project/` — which project (first-class; a freelancer's projects are the primary unit of work)
|
||||
- `domain/` — knowledge domain / topic area
|
||||
- `tool/` — tool-specific knowledge
|
||||
- `convention/` — conventions
|
||||
- …plus `scope/global` or `scope/project`
|
||||
|
||||
Hierarchy and relationships are expressed via **hub notes** (`type/hub`), **wikilinks**, and **Graphify knowledge-graph edges** — NOT via nested tag paths.
|
||||
|
||||
By convention `type/` is listed **first** in frontmatter.
|
||||
- **Rationale**: The vault is flat — hierarchy is not expressed through folder paths or tag nesting. A many-to-many reality (many projects per client, knowledge domains spanning clients) maps badly onto a single-parent tree. A project hub note links out to both its `client/` and relevant `domain/` tags rather than being buried under either. Per-type `_templates` for **core types only** (research, howto, adr, hub); the long tail stays freeform. Consistent per-type structure also improves Graphify's local-SLM extraction reliability.
|
||||
- **Alternatives rejected**: Hierarchical nesting in the style of `domain/{product}/{project}.md` folder structure. Rejected because: (1) the vault is flat; (2) the many-to-many reality maps badly onto a single-parent folder tree; (3) nesting one facet through another creates traversal coupling. Faceted parallel tags are the flat-vault analogue of what the Graphify graph already does with edges, so they compose naturally with the knowledge layer.
|
||||
- **Status**: Accepted (supersedes the namespace list in ADR-003; core flat-vault + namespaced-tags decision stands).
|
||||
|
||||
## ADR-012 — Reuse an existing vault rather than creating a new one
|
||||
|
||||
_Date: 2026-06-04_
|
||||
|
||||
- **Context**: The design called for a flat markdown vault as the semantic knowledge layer. The question was whether to stand up a new vault from scratch or adopt an existing one.
|
||||
- **Decision**: **Adopt an existing vault** rather than creating a new vault.
|
||||
- **Rationale**: An existing vault that is already flat (all notes at root, only a `_templates/` exception), already articulates the correct "durable knowledge, not working memory" role, and contains real notes is worth preserving. Two patterns the existing vault may already include that improve on a bare-spec design:
|
||||
1. An "act without being asked" section specifying *when* the AI should proactively query the vault — a behavioral spec.
|
||||
2. Project-config hub notes with a tag-inference table (auto-tag by path pattern) that operationalizes *how* to tag a note from a given project.
|
||||
- **Adaptations typically required (migration cost)**:
|
||||
- Add `summary:` frontmatter to existing notes.
|
||||
- Migrate flat unnamespaced tags to the six-facet namespaced form (per ADR-011).
|
||||
- Add `scope/global` or `scope/project` to each note.
|
||||
- Initialize git in the vault (required by ADR-008's sync strategy).
|
||||
- Replace any legacy vault-search script references with `graphify query` (ADR-010).
|
||||
These are mechanical schema migrations, not structural rework.
|
||||
- **Alternatives rejected**: Starting fresh. Rejected because the hardest design decision — flat structure, durable-knowledge-only role, governance philosophy — is already made and practiced in an existing vault. The improved behavioral patterns and the existing notes are worth preserving.
|
||||
- **Status**: Accepted.
|
||||
|
||||
## ADR-013 — Build-first / migrate-incrementally (build-order inversion)
|
||||
|
||||
_Date: 2026-06-04_
|
||||
|
||||
- **Context**: The original build order front-loaded bulk vault migration as Step 1 — migrating all existing notes and all projects to the ADR-011 six-facet taxonomy before the system existed to validate them. This committed to a schema and workflow before any end-to-end path had been exercised. The risk: locking in an approach that fails at scale, with no feedback loop until the entire vault has been touched.
|
||||
- **Decision**: **Invert the build order.** The full system is built and validated against a small **5–10 note fixture set** first. Bulk vault migration is deferred to the final stage. The first real-data validation uses **one small project that contains both code AND documents**, exercising both the local-SLM doc-extraction path and the tree-sitter code path in the same run. After that single project validates end-to-end, remaining projects are onboarded **one at a time** with an observe-and-adjust step between each.
|
||||
- **Rationale**: Validates the ADR-011 taxonomy and vault conventions against the real Graphify extraction pipeline before the entire vault is committed. The first mixed code+docs project surfaces both extraction paths (SLM for docs, tree-sitter for code) early, when corrections are cheap. Per-project rollout keeps the blast radius of any schema or workflow correction small.
|
||||
- **Alternatives rejected**:
|
||||
- **Keep migration-first**: Front-loads all notes and all projects before any end-to-end validation exists.
|
||||
- **Big-bang migrate everything after build**: Build against fixtures, then migrate all notes and all projects in one batch at the end. Per-project rollout with intermediate checkpoints is strictly safer.
|
||||
- **Status**: Accepted.
|
||||
|
||||
## ADR-014 — Graph connectivity comes from authored structure; migration scaffolding is a first-class prerequisite
|
||||
|
||||
_Date: 2026-06-05_
|
||||
|
||||
- **Context**: ADR-011 specified hub notes + wikilinks + Graphify graph edges as the mechanism for expressing hierarchy and cross-note relationships, with ADR-013 deferring bulk vault migration. Before build began, an empirical test compared a cached-replay graph (per-fixture isolated extractions) against a clean single-pass deep extraction against a real vault under Graphify 0.8.31 + qwen2.5-coder:7b. [primary/measured — 2026-06-05]
|
||||
- **Decision**: **The connective spine of the knowledge graph must be author-provided.** Hub notes and wikilinks are not optional scaffolding to add "someday" — they are the mechanism by which Graphify connects thematically related notes, and they must be authored **as part of the migration step**, not deferred to bulk import. Migration scaffolding (hub notes + wikilinks for key concepts) is a **first-class build deliverable**.
|
||||
- **Rationale**: The empirical test found that Graphify is a **structure extractor, not a topic clusterer**. Even at `--mode deep --token-budget 8000`, no emergent shared-topic hub nodes appeared. All cross-note edges observed came from explicit references, wikilinks, or document-level semantic similarity — not from shared thematic identity. A practical test query ("how do we do niche prospecting outreach for pest control?") returned 3 starting notes and traversal could not reach related notes in separate communities (no connecting edges). This confirms that useful retrieval is gated on migration scaffolding, not on Graphify's extraction power.
|
||||
- **Relationship to ADR-011**: Validates the hub-notes + wikilinks half of ADR-011 empirically. The facet-tag half is **not yet validated**: no edge was observed to arise from shared frontmatter facet tags alone. Whether `client/X` or `tool/Y` tags create graph connectivity is an **open question** — do not assume facet tags contribute to graph traversal retrieval until tested.
|
||||
- **Relationship to ADR-013**: Refines the migrate-incrementally stage. "Migration" must include hub note authoring and wikilink addition for key concepts, not just frontmatter schema migration.
|
||||
- **Alternatives rejected**: Relying on the SLM to auto-cluster topics and synthesize hub entities — **empirically does not happen** at 7B model size with `--mode deep`. The design already intended human-authored hub notes; the test confirms that intent was correct and the fallback assumption ("maybe the LLM will do it") is false.
|
||||
- **Deferred**:
|
||||
1. **Facet-tag-to-graph-edge question**: Do shared frontmatter facet tags (`client/`, `tool/`, `domain/`, etc.) cause Graphify to create edges between notes, or does graph connectivity come only from explicit wikilinks/references and semantic similarity? This was NOT tested. Resolve before designing graph-traversal retrieval skills.
|
||||
2. **Larger extraction model**: Whether a substantially larger SLM (14B, 30B) would synthesize emergent topic-hub nodes is untested.
|
||||
3. **`reasoning_effort:"none"` patch**: The clean run required a local patch to `graphify/llm.py`. Track the upstream Graphify issue tracker for an official fix; treat the installed version as pinned until resolved.
|
||||
- **Status**: Accepted. Refines ADR-013 (migrate-incrementally phase scope) and empirically validates the hub-notes/wikilinks mechanism of ADR-011 while flagging its facet-tag half as an open question.
|
||||
|
||||
---
|
||||
|
||||
## Rejected tools (summary)
|
||||
|
||||
| Tool | Why rejected |
|
||||
|------|--------------------------|
|
||||
| MemPalace | Storage not readable markdown; isolated drawers (knowledge not interconnected); fights self-managing + cross-linking goals |
|
||||
| Recall / LightRAG | Content knowledge bases / deep research, not operational memory; Recall = hosted, you don't own data; LightRAG = enterprise overkill |
|
||||
| OpenBrain / Mem0 | Always-remote DB → latency + cost; conflicts with local-fast lazy-sync; only pays off for real-time cross-tool memory (overkill) |
|
||||
| Postgres / Milvus server | Unnecessary — Graphify's local graph (knowledge) + Milvus Lite (memsearch episodic) cover everything locally with no Docker |
|
||||
| claude-mem | MCP-based (Claude must call search); opaque blobs vs readable markdown; feature overkill |
|
||||
| Ruby/SQLite tag index CLI; QMD vector layer | Superseded by Graphify before build — one knowledge graph replaces both the structured index and the deferred semantic layer (ADR-010) |
|
||||
|
|
@ -0,0 +1,125 @@
|
|||
---
|
||||
summary: Architecture for a two-layer personal memory system for Claude Code — episodic (memsearch) and semantic/knowledge (Obsidian vault + Graphify), hook-injected and on-demand.
|
||||
tags:
|
||||
- type/reference
|
||||
- domain/knowledge-graphs
|
||||
- domain/llm
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
source: cc-os
|
||||
date: 2026-06-09
|
||||
---
|
||||
|
||||
# Claude Code Memory System — Architecture Design
|
||||
|
||||
## Core principle: two memory types, kept separate
|
||||
|
||||
| Type | Question | Lifecycle | Write path | Tool |
|
||||
|------|----------|-----------|-----------|------|
|
||||
| **Episodic** | "What happened, when?" | accretes & decays | auto-captured | **memsearch** |
|
||||
| **Semantic / knowledge** | "How do we…?" | deliberately maintained | curated by AI/author | **Obsidian vault + Graphify knowledge graph** |
|
||||
|
||||
The episodic vs. semantic split is the key architectural decision. They have different lifecycles, write paths, and query patterns. Forcing one tool to do both is what makes every unified-memory design feel forced.
|
||||
|
||||
## Goals the architecture must satisfy
|
||||
|
||||
1. **Thin projects** — keep as little AI context inside each project repo as possible. Knowledge is pulled in on demand or injected by hooks.
|
||||
2. **Cross-project / cross-client knowledge** — learn something once (e.g. a tool's API) and reference it from anywhere. Two scopes: **global** (broadly useful) and **project/client-specific** — both globally reachable.
|
||||
3. **Timeline awareness** — lightweight awareness of recent activity ("what was I doing yesterday"), with the ability to drill deeper.
|
||||
4. **Remote, local-fast** — accessible anywhere but runs local-fast; lazy sync (minutes/hourly) is fine; real-time is overkill.
|
||||
|
||||
Desired properties: **lightweight** (low tokens), **fast** (out of the way), **flexible** (cross project/client), **self-evolving** (AI maintains it under clear rules), **semi-structured** (organization that can evolve).
|
||||
|
||||
Both layers are **local-first, markdown-as-truth, no Docker, no server, no API keys** (Graphify extraction runs against a local Ollama model).
|
||||
|
||||
## Layer 1 — Episodic (memsearch)
|
||||
|
||||
- **What it is**: A Claude Code plugin (by Zilliz) that auto-captures session notes as daily markdown, chunks them, and stores a **shadow index** in **Milvus Lite** (a single embedded file — no server, no Docker). Hybrid search = BM25 + dense vectors + RRF, local ONNX embeddings (`bge-m3`, no API key/cost). A FileWatcher (1500ms debounce) handles updates and deletions.
|
||||
- **Why off-the-shelf**: It already implements the daily-notes + "dreaming" pattern and the markdown-as-truth / disposable-shadow-index philosophy.
|
||||
- **Role**: Satisfies timeline goal. The AI queries it in natural language ("what was decided about X last week"). Does **not** filter by knowledge tags — it owns the episodic corpus only.
|
||||
|
||||
## Layer 2 — Knowledge (vault + Graphify knowledge graph)
|
||||
|
||||
### Vault structure
|
||||
|
||||
- **Flat markdown directory**, single source of truth — reuses an existing Obsidian vault (`~/Documents/SecondBrain`). Browsable in Obsidian as a viewer.
|
||||
- Replaces project-local documentation: instead of docs scattered per repo, knowledge lives once in the vault and is pulled into any project on demand.
|
||||
|
||||
### Frontmatter contract (every note)
|
||||
|
||||
```yaml
|
||||
---
|
||||
summary: One line, written at creation. The router shows this so the AI can pick a file without opening it.
|
||||
tags:
|
||||
- type/reference # listed first by convention; e.g. type/hub, type/how-to
|
||||
- client/clientname
|
||||
- project/project-name
|
||||
- domain/seo
|
||||
- tool/semrush
|
||||
- convention/api-style
|
||||
- scope/project # or scope/global
|
||||
---
|
||||
```
|
||||
|
||||
- **Six flat facets**: `type/`, `client/`, `project/`, `domain/`, `tool/`, `convention/` — plus `scope/`. Each facet is independent and parallel (never nested into each other). `#tool` matches all `tool/*` values — native Obsidian prefix filtering, no folders needed.
|
||||
- **Hierarchy and relationships** are expressed via **hub notes** (`type/hub`), **wikilinks**, and **Graphify graph edges** — NOT via nested tag paths.
|
||||
- **Two knowledge scopes** via `scope/global` vs `scope/project` (+ a `client/` tag): global = broadly useful tool/domain knowledge; project = how a specific client uses it. Both are globally queryable.
|
||||
|
||||
### Knowledge graph (Graphify)
|
||||
|
||||
Graphify turns the vault into a queryable **knowledge graph** — a disposable, rebuildable structure over the markdown. It replaces a bespoke tag index CLI *and* a deferred vector layer: one graph gives both structured and semantic retrieval, without vectors.
|
||||
|
||||
**Extraction:**
|
||||
- **Vault docs** → a **local Ollama SLM** extracts entities + typed relationships from each note (confidence-tagged `EXTRACTED` / `INFERRED` / `AMBIGUOUS`). Local model = no API cost, no data leaving the machine.
|
||||
- **Project code** → free **tree-sitter AST** (`--no-docs`), no model, no token cost. Kept as separate per-project graphs, not merged with the vault graph.
|
||||
|
||||
**What it produces**: `graphify-out/` with `graph.json`, an interactive `graph.html`, and a `GRAPH_REPORT.md` whose top lists the **god nodes** (the most-connected concepts — highest-value entry points).
|
||||
|
||||
**Query** (via CLI and MCP server exposing `query_graph` / `get_node` / `shortest_path`): ask for **god nodes first**, then scalpel down with `graphify query` / `path` / `explain`. Prompt the graph; don't dump the corpus into context.
|
||||
|
||||
**Metadata still matters**: the `summary` + six facet tags remain first-class note attributes — `summary` is the human-written router hint Graphify does **not** generate. Facet namespaces stay useful for Obsidian filtering and as node attributes.
|
||||
|
||||
**Source of truth rule**: markdown is authoritative; `graphify-out/` is a rebuildable artifact that is **never synced** and can be deleted/rebuilt anytime (`graphify ... --force`).
|
||||
|
||||
### Freshness (lazy)
|
||||
|
||||
- **AI writes** → a `PostToolUse` hook on `Write`/`Edit` targeting vault `.md` files runs `graphify ... --update` to merge the changed note into the vault graph. Event-driven, no polling.
|
||||
- **Stale-node caveat**: Graphify's `--update` merges (SHA-256 + dedup) but does **not** prune deleted notes — ghost nodes accumulate. A periodic `--force` rebuild clears them, triggered by the **session-start reconcile** when a rebuild stamp is older than N days. **No daemon, no cron.**
|
||||
|
||||
### Retrieval (hook-injected + on-demand)
|
||||
|
||||
- **Session-start hook** injects: (a) a compact overview — the vault graph's **god nodes** as the map of what's known, (b) the current project's declared `convention/*` notes resolved to their summaries (coding conventions auto-pull; a convention edit propagates to every project using that tag), (c) a pointer to recent episodic journal.
|
||||
- **On demand**: the AI runs `graphify query` / `path` / `explain` (or the MCP tools) to pull specific knowledge into context only when the task needs it. Projects stay thin — their CLAUDE.md holds **tags/pointers**, not content.
|
||||
|
||||
## Semantic recall — covered by Graphify
|
||||
|
||||
An earlier design earmarked a separate vector layer for "when structured tag filtering misses a note whose wording doesn't match the query." Graphify's knowledge graph covers that need without a second system or vectors: relationship traversal and `explain` surface notes by *connection*, not just exact tag match. Revisit a vector layer only if graph traversal demonstrably misses cases where embedding similarity would clearly win.
|
||||
|
||||
## Timeline details
|
||||
|
||||
A **session-end hook** appends a daily journal note (one file per date) with pointers to the project/knowledge files touched. memsearch indexes these; today+yesterday are cheap to load, older entries are reachable by query for drill-down.
|
||||
|
||||
## Self-evolution guardrails
|
||||
|
||||
- The AI **writes only to the vault**, never silently into project repos.
|
||||
- **Required frontmatter schema** (summary + namespaced tags) is enforced so the index stays queryable.
|
||||
- **Daily notes are append-only**; consolidation/reorg is a **separate, reviewable step** run in plan mode.
|
||||
- **Promotion to `scope/global`** requires a rule (e.g. a fact recurring N times) — not every stray note gets promoted.
|
||||
|
||||
## Sync
|
||||
|
||||
- The **vault** syncs to a remote machine via **git** (versioned history) or **Syncthing** (continuous, zero-thought).
|
||||
- **Graphs/indexes are never synced** — the Milvus Lite episodic index and Graphify `graphify-out/` graphs are rebuilt per machine. Sync only the markdown.
|
||||
|
||||
## Packaging
|
||||
|
||||
Ships as a **global Claude Code plugin with skills** (hooks + scripts + CRUD know-how) so every project, on every machine, knows how to use the vault effectively.
|
||||
|
||||
## How each goal is met
|
||||
|
||||
| Goal | Met by |
|
||||
|------|--------|
|
||||
| 1. Thin projects | Knowledge in the vault, not repos; CLAUDE.md holds tags/pointers; on-demand graph query |
|
||||
| 2. Cross-project/client knowledge | Vault + six-facet tags + Graphify knowledge graph (god nodes + traversal) |
|
||||
| 3. Timeline | memsearch episodic layer + session-end journal hook |
|
||||
| 4. Remote, local-fast | Markdown vault synced via git/Syncthing; disposable per-machine graphs/indexes |
|
||||
|
|
@ -0,0 +1,135 @@
|
|||
---
|
||||
summary: Evaluation of Graphify as the knowledge-layer engine for a personal AI memory system — fit by layer, rationale for adoption, empirical findings, and limitations discovered.
|
||||
tags:
|
||||
- type/reference
|
||||
- domain/knowledge-graphs
|
||||
- domain/llm
|
||||
- tool/graphify
|
||||
- scope/global
|
||||
source: cc-os
|
||||
date: 2026-06-09
|
||||
---
|
||||
|
||||
# Graphify Evaluation: Fit as a Knowledge-Layer Engine
|
||||
|
||||
> Evaluated: 2026-06-03. Source docs: Graphify GitHub + handbook.
|
||||
|
||||
---
|
||||
|
||||
## Summary Verdict
|
||||
|
||||
**Graphify is a strong fit for a knowledge layer over a markdown vault and a natural fit for project-file code analysis. It is NOT the right tool for episodic/session logs, and it does not "connect" multiple sources on its own — that bridge still needs custom glue.**
|
||||
|
||||
---
|
||||
|
||||
## What Graphify Actually Is
|
||||
|
||||
Graphify builds a knowledge graph from code and documents:
|
||||
|
||||
- **Nodes**: entities (functions, classes, concepts, topics)
|
||||
- **Edges**: typed relationships with confidence tags (`EXTRACTED` = deterministic AST, `INFERRED` = LLM, `AMBIGUOUS` = flagged)
|
||||
- **Storage**: `graph.json` + optional exports (Obsidian sidecar, GraphML, Cypher)
|
||||
- **Query**: semantic graph traversal (`query`, `path`, `explain`) — not vector/embedding search
|
||||
- **MCP server**: `graphify.serve` exposes `query_graph`, `get_node`, `shortest_path` as Claude tools
|
||||
|
||||
The core cost split:
|
||||
- **Code** → tree-sitter AST, 33 languages, deterministic, **zero LLM cost**
|
||||
- **Docs** → LLM entity extraction, can use local SLMs via Ollama (`--backend ollama`)
|
||||
|
||||
---
|
||||
|
||||
## Fit by Layer
|
||||
|
||||
### Layer 1: Episodic ("what happened") — NOT a fit for Graphify
|
||||
|
||||
Episodic queries are time-anchored semantic lookups: "what was I working on last Tuesday?", "what client did we discuss the Stripe integration for?" Vector similarity over session logs is well-matched to this. A knowledge graph of session entities would add build overhead without improving recall for timeline queries. **Keep memsearch (Milvus Lite).**
|
||||
|
||||
### Layer 2: Knowledge ("how do we…") — strong fit
|
||||
|
||||
**What Graphify replaces**: A hand-built tag-index CLI and a deferred vector semantic layer. Instead of manually adding tags to every note, a local SLM (Ollama + Qwen2.5 7B or Phi-4 14B) extracts entities automatically. You get entity nodes and relationship edges from the vault without requiring exhaustive frontmatter authoring.
|
||||
|
||||
**What it adds that tags cannot**: Inferred cross-note relationships. Tags only filter ("give me notes tagged tool/semrush"). A graph can say "tool/semrush is connected to client/X via 3 project notes, and both reference convention/seo-workflow." That's a graph query, not a tag query.
|
||||
|
||||
**What it does NOT replace**: The `summary` field in the frontmatter schema. Graphify extracts entities, not prose summaries. If summaries are the primary token-efficiency mechanism (the AI picks a file by reading its summary without opening it), they must remain author-controlled.
|
||||
|
||||
**Known limitation — stale node drift**: Graphify's `--update` does not prune deleted symbols/notes — you must `--force` rebuild to clear ghost nodes. The graph is a snapshot with known drift. Mitigate by scheduling a periodic `--force` rebuild (e.g., triggered by a session-start staleness check when the rebuild stamp is older than N days).
|
||||
|
||||
### Project Files (code) — no-brainer add
|
||||
|
||||
Project-file code analysis via AST is free (no LLM, no API key needed at runtime), deterministic, and produces exactly the call-graph / symbol map that "what does this codebase do?" queries need. Every project gets a `graph.json` at zero ongoing cost. Query via the MCP server.
|
||||
|
||||
**Limitation** [github]: `graphify extract` may demand a backend credential even for pure-code extraction. Workaround: set a dummy key or use `--backend ollama` with a running (or not) Ollama instance — confirm at install time.
|
||||
|
||||
---
|
||||
|
||||
## The "Multiple Sources Connected" Question
|
||||
|
||||
Graphify does **not** natively connect three separate graphs. It builds one graph per ingestion run. To connect project files, vault, and session logs in one queryable surface, you'd need:
|
||||
|
||||
1. **A combined ingestion run** that points at all three source trees, OR
|
||||
2. **Three separate graphs** queried via three MCP tool instances and synthesized at the Claude layer
|
||||
|
||||
Option 1 risks cross-contamination of unrelated client projects. Option 2 is the safer model: each source has its own graph, and Claude stitches results from multiple MCP calls.
|
||||
|
||||
The more honest framing: Graphify replaces tags as the **vault entity index**, adds a **free code graph per project**, and leaves episodic (memsearch) alone. That's a meaningful simplification — you no longer need a bespoke tag index CLI — but it's a layer replacement, not a unified connector.
|
||||
|
||||
---
|
||||
|
||||
## What Changes vs. a Tag-Index Design
|
||||
|
||||
| Prior design | With Graphify |
|
||||
|---|---|
|
||||
| Bespoke tag index CLI (e.g., Ruby/SQLite) | Replaced by `graphify query` + MCP server |
|
||||
| Manual tag-discipline on every note | Auto-extracted by local SLM on vault ingestion |
|
||||
| `summary` frontmatter (author-written, router hint) | Unchanged — Graphify doesn't generate these |
|
||||
| Deferred vector/semantic layer (e.g., QMD) | Graphify fills this role without vectors |
|
||||
| memsearch (episodic) | Unchanged |
|
||||
| Project files (unindexed) | Free code graph via AST |
|
||||
|
||||
**What's saved**: The entire bespoke index build (a significant chunk of critical-path work) can be skipped. Significant scope reduction.
|
||||
|
||||
**What's added**: A local Ollama SLM running on this machine for vault doc extraction. This is a new infrastructure dependency. On low-RAM machines, a 7B model at 4-bit takes ~5 GB VRAM/RAM.
|
||||
|
||||
---
|
||||
|
||||
## Empirical Findings (Graphify v0.8.31 + qwen2.5-coder:7b) [primary/measured — 2026-06-05]
|
||||
|
||||
A discriminating test compared a cached-replay graph against a clean single-pass deep extraction (`--mode deep --token-budget 8000`) against a real vault.
|
||||
|
||||
**Key finding — Graphify is a structure extractor, not a topic clusterer.** Even at `--mode deep`, no emergent shared-topic hub nodes appeared (no auto-synthesized "Pest Control" node, no "Niche Prospecting" node). All cross-note edges observed came from explicit references, wikilinks, or document-level semantic similarity — not from shared thematic identity.
|
||||
|
||||
**Practical impact**: A test query ("how do we do niche prospecting outreach for pest control?") returned 3 starting notes, and traversal could not reach thematically related notes in separate communities because there were no connecting edges. This confirms that useful retrieval is gated on **migration scaffolding** (hub notes + wikilinks), not on Graphify's extraction power.
|
||||
|
||||
**Build artifact caveat**: The cached graph was partially a build artifact — cross-note edges rose from 41% to 78% in a single-pass run. The structural finding (no emergent hub nodes) held in both runs.
|
||||
|
||||
**Open empirical question**: Whether shared frontmatter facet tags (`client/`, `tool/`, `domain/`, etc.) cause Graphify to create edges between notes was NOT tested. Do not assume facet tags contribute to graph traversal retrieval until this is verified.
|
||||
|
||||
**`reasoning_effort:"none"` issue**: The clean run required a local patch to `graphify/llm.py`. Treat the installed version as pinned until an official fix appears upstream.
|
||||
|
||||
---
|
||||
|
||||
## Design Conclusions from Evaluation
|
||||
|
||||
1. **Hub notes and wikilinks are mandatory, not optional**: The connective spine of the knowledge graph must be author-provided as part of any vault migration. Deferring this to "someday" means graph traversal retrieval will fail for cross-topic queries.
|
||||
|
||||
2. **Migration means more than frontmatter**: Adding `summary:` and six-facet tags to existing notes is necessary but not sufficient. Hub notes and wikilinks for key concepts must be authored alongside the frontmatter migration.
|
||||
|
||||
3. **Summary frontmatter survives the pivot**: Even though Graphify auto-extracts entities, the human-written `summary` field remains the token-efficient router hint that lets the AI pick a file without opening it.
|
||||
|
||||
4. **Stale-node drift is real and must be scheduled**: Rely on the lazy freshness model (write-hook `--update` + periodic session-start `--force` rebuild), not continuous re-extraction.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Rebuild cadence**: How often do you rebuild the vault graph? On every session-start (expensive), on vault git commit (right cadence), or manually? The `--update` flag helps but stale-node drift means periodic `--force` rebuilds are needed.
|
||||
|
||||
2. **Model choice for vault extraction**: Test Qwen2.5 7B and Phi-4 14B against a sample of vault notes and review god-node quality before committing. [unverified claim: no official recommendation exists]
|
||||
|
||||
3. **Facet-tag-to-graph-edge question**: Do shared frontmatter facet tags cause Graphify to create edges between notes, or does graph connectivity come only from explicit wikilinks/references and semantic similarity? Resolve before designing graph-traversal retrieval skills.
|
||||
|
||||
4. **Larger extraction model**: Whether a substantially larger SLM (14B, 30B) would synthesize emergent topic-hub nodes is untested. Secondary — the design does not depend on it.
|
||||
|
||||
5. **Cross-client isolation**: Project code graphs should be per-client-project, not merged. Separate `graphify-out/` per project is the safe default.
|
||||
|
||||
6. **MCP server management**: Running `graphify.serve` for vault + N project graphs means N+1 MCP server instances. Manageable for small N; a single meta-server may be needed at scale.
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
# SecondBrain
|
||||
|
||||
A personal knowledge base. Notes arrive from any source — Claude Code projects,
|
||||
web clips, iOS Shortcuts, n8n automations, manual entry. All are equal citizens.
|
||||
|
||||
This vault is NOT working memory. It stores completed, durable knowledge:
|
||||
finished plans, decisions, research, how-tos, and project learnings.
|
||||
|
||||
## Note format
|
||||
|
||||
```yaml
|
||||
---
|
||||
source: systems-admin # origin: repo name, "web-clipper", "ios-shortcut", etc.
|
||||
date: YYYY-MM-DD
|
||||
tags: [plan, ruby, obsidian] # type tag first, then content tags
|
||||
---
|
||||
```
|
||||
|
||||
## Tag conventions
|
||||
|
||||
Type tags (use exactly one): plan, adr, research, howto, log, clip
|
||||
|
||||
Content tags: anything that aids future discovery. kebab-case for multi-word: claude-code
|
||||
|
||||
## Filename convention
|
||||
|
||||
YYYY-MM-DD-descriptive-slug.md — generated by vault_archive.rb, do not rename manually.
|
||||
|
||||
Full conventions and tag vocabulary: [[CONVENTIONS]]
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
---
|
||||
tags:
|
||||
- project-config
|
||||
- {{project-slug}}
|
||||
source: {{source}}
|
||||
date: {{date}}
|
||||
---
|
||||
|
||||
# Project Config — {{project-slug}}
|
||||
|
||||
One sentence: what is this project and what does this hub note cover?
|
||||
|
||||
Project repo: `{{repo-path}}`
|
||||
|
||||
## Navigation
|
||||
|
||||
| I want to... | Do this |
|
||||
|---|---|
|
||||
| Know which tags to apply when archiving | See Tag Inference Rules below |
|
||||
| Find all notes from this project | Run inventory query below |
|
||||
| Understand vault-wide naming and frontmatter rules | Read [[vault-conventions]] |
|
||||
|
||||
## Inventory Query
|
||||
|
||||
```bash
|
||||
~/.claude/scripts/vault_search.rb "{{project-slug}}"
|
||||
```
|
||||
|
||||
## Tag Inference Rules
|
||||
|
||||
| Path pattern | Tags added |
|
||||
|---|---|
|
||||
| Any file from this repo | `{{project-slug}}` |
|
||||
| `campaigns/{{campaign-slug}}/` | `{{campaign-slug}}` |
|
||||
|
||||
## Campaign → Automation Product Mapping
|
||||
|
||||
| Campaign | Tags |
|
||||
|---|---|
|
||||
| `{{campaign-slug}}` | `{{automation-product-tags}}` |
|
||||
|
||||
## Active Campaigns
|
||||
|
||||
- `{{campaign-slug}}`
|
||||
|
||||
## Related
|
||||
|
||||
- [[vault-conventions]]
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
sort:
|
||||
- property: date
|
||||
direction: desc
|
||||
filters:
|
||||
and:
|
||||
- file.tags:contains("niche-automation-prospecting")
|
||||
views:
|
||||
- type: table
|
||||
name: All Notes
|
||||
order:
|
||||
- file.name
|
||||
- date
|
||||
- tags
|
||||
- source
|
||||
|
|
@ -0,0 +1,242 @@
|
|||
---
|
||||
summary: Daily session log
|
||||
tags: [scope/global, type/log]
|
||||
---
|
||||
|
||||
## Session — 2026-06-05T17:31:37Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** exit
|
||||
**Vault notes touched:**
|
||||
/home/jared/Documents/SecondBrain/test1.md
|
||||
/home/jared/Documents/SecondBrain/test2.md
|
||||
|
||||
## Session — 2026-06-05T17:31:40Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** clear
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T17:31:43Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T17:59:42Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T18:21:35Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** exit
|
||||
**Vault notes touched:**
|
||||
/path/to/vault/test-note.md
|
||||
|
||||
## Session — 2026-06-05T18:21:38Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T18:21:43Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T18:21:43Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T18:56:41Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T19:33:40Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T19:52:45Z
|
||||
|
||||
**Project:** /home/jared
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T19:58:05Z
|
||||
|
||||
**Project:** /home/jared/dev/websites/hyperthrive
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T19:59:21Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:01:08Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:02:24Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:03:18Z
|
||||
|
||||
**Project:** /home/jared/dev/websites/hyperthrive
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:04:17Z
|
||||
|
||||
**Project:** /home/jared/dev/llf-schema
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:09:55Z
|
||||
|
||||
**Project:** /home/jared/dev/websites/hyperthrive
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:15:18Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:15:20Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:15:24Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:15:26Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:15:30Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:15:34Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:33:22Z
|
||||
|
||||
**Project:** /home/jared/dev/websites/hyperthrive
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
---
|
||||
|
||||
## 2026-06-05 — Memory plugin complete
|
||||
|
||||
Memory plugin installed globally at `~/.claude/plugins/memory/`. Three hooks running:
|
||||
|
||||
- `session-start.sh` — triggers background graphify rebuild on SessionStart
|
||||
- `session-context.sh` — injects vault graph map via UserPromptSubmit
|
||||
- `post-tool-use-write.sh` — session log + dirty-file tracking on PostToolUse
|
||||
|
||||
**Key fix discovered:** SessionStart hook has undocumented last-write-wins semantics — when the memory plugin and superpowers plugin both set `additionalContext` on SessionStart, only one wins. Moving vault context injection to UserPromptSubmit resolved this; UserPromptSubmit explicitly concatenates multiple hooks' `additionalContext` values.
|
||||
|
||||
Verified working: vault graph map (366 nodes, 346 edges) injected into every new session.
|
||||
|
||||
**Graph runs today:**
|
||||
- SecondBrain vault: 333 nodes, 333 edges, 21 communities
|
||||
- cc-os project: 575 nodes, 511 edges, 64 communities
|
||||
|
||||
**ADR-014 locked:** Graphify is a structure extractor, not a topic clusterer. No emergent hub nodes appear even at `--mode deep`. Hub notes + wikilinks must be author-provided during migration — migration scaffolding is now a first-class build deliverable.
|
||||
|
||||
## Session — 2026-06-05T20:44:20Z
|
||||
|
||||
**Project:** /home/jared/dev/websites/hyperthrive
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:47:37Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:58:27Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T20:59:55Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T21:01:16Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-05T21:01:20Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
---
|
||||
summary: Daily session log
|
||||
tags: [scope/global, type/log]
|
||||
---
|
||||
|
||||
## Session — 2026-06-08T16:26:55Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-08T16:29:06Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-08T17:17:09Z
|
||||
|
||||
**Project:** /home/jared/dev/verona-vocab
|
||||
**Reason:** clear
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-08T17:24:17Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** clear
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-08T17:31:39Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** clear
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-08T17:31:41Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-08T17:33:21Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-08T17:42:53Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
/home/jared/Documents/SecondBrain/CONVENTIONS.md
|
||||
/home/jared/Documents/SecondBrain/CLAUDE.md
|
||||
|
||||
## Session — 2026-06-08T17:43:47Z
|
||||
|
||||
**Project:** /home/jared/clients
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
|
@ -0,0 +1,178 @@
|
|||
---
|
||||
summary: Daily session log
|
||||
tags: [scope/global, type/log]
|
||||
---
|
||||
|
||||
## Session — 2026-06-09T12:48:08Z
|
||||
|
||||
**Project:** /home/jared/dev/verona-vocab
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T12:48:12Z
|
||||
|
||||
**Project:** /home/jared/dev/verona-vocab
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T13:13:57Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T13:28:52Z
|
||||
|
||||
**Project:** /home/jared/systems-admin
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T17:37:49Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T17:37:57Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-conceptual-model.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-installation-setup.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-querying-god-nodes.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-best-practices-checklist.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-overview-concepts.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-ingesting-code-ast.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-ingesting-docs-knowledge.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-local-models-backends.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-token-economics-updates.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-workflows-use-cases.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-08-graphify-extraction-model-options.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-09-claude-code-memory-system-design.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-09-claude-code-memory-system-adrs.md
|
||||
/home/jared/Documents/SecondBrain/2026-06-09-graphify-evaluation-as-knowledge-layer.md
|
||||
|
||||
## Session — 2026-06-09T17:38:00Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:05:19Z
|
||||
|
||||
**Project:** /home/jared
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:15:40Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:17:20Z
|
||||
|
||||
**Project:** /home/jared
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:19:21Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:20:03Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:31:51Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:32:19Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:36:00Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:36:06Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:36:28Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:38:27Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T18:46:16Z
|
||||
|
||||
**Project:** /home/jared
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T19:13:42Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T19:14:33Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T19:26:38Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-09T19:33:49Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
---
|
||||
summary: Daily session log
|
||||
tags: [scope/global, type/log]
|
||||
---
|
||||
|
||||
## Session — 2026-06-10T15:49:33Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-10T16:34:20Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-10T16:34:48Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-10T17:38:17Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
summary: Daily session log
|
||||
tags: [scope/global, type/log]
|
||||
---
|
||||
|
||||
## Session — 2026-06-11T13:18:50Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** other
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-11T19:37:11Z
|
||||
|
||||
**Project:** /home/jared/dev/websites/hyperthrive
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-11T19:58:21Z
|
||||
|
||||
**Project:** /home/jared/systems-admin
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-11T20:10:13Z
|
||||
|
||||
**Project:** /home/jared
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-11T20:21:46Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
|
@ -0,0 +1,67 @@
|
|||
---
|
||||
summary: Daily session log
|
||||
tags: [scope/global, type/log]
|
||||
---
|
||||
|
||||
## Session — 2026-06-12T14:40:23Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-12T14:44:34Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-12T15:15:47Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-12T15:50:35Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-12T16:27:38Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-12T18:56:45Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-12T18:58:17Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-12T19:06:51Z
|
||||
|
||||
**Project:** /home/jared/clients/philly-search-engine-marketing
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
||||
## Session — 2026-06-12T19:47:27Z
|
||||
|
||||
**Project:** /home/jared/dev/viking-warrior-training-log
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
---
|
||||
summary: Daily session log
|
||||
tags: [scope/global, type/log]
|
||||
---
|
||||
|
||||
## Session — 2026-06-15T18:40:29Z
|
||||
|
||||
**Project:** /home/jared/dev/cc-os
|
||||
**Reason:** prompt_input_exit
|
||||
**Vault notes touched:**
|
||||
(none)
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
---
|
||||
tags:
|
||||
- project-config
|
||||
- niche-automation-prospecting
|
||||
source: niche-automation-prospecting
|
||||
date: 2026-03-13
|
||||
---
|
||||
|
||||
# Project Config — niche-automation-prospecting
|
||||
|
||||
Hub note for the `niche-automation-prospecting` project. Start here before archiving any note or looking up project research.
|
||||
|
||||
Project repo: `/home/jared/projects/niche-automation-prospecting`
|
||||
|
||||
## Navigation
|
||||
|
||||
| I want to... | Do this |
|
||||
|---|---|
|
||||
| Know which tags to apply when archiving | See Tag Inference Rules below |
|
||||
| Find all notes from this project | Run inventory query below |
|
||||
| See what research exists on pest control | Run `vault_search.rb "pest-control"` |
|
||||
| Understand vault-wide naming and frontmatter rules | Read [[vault-conventions]] |
|
||||
| Find notes about a specific campaign | Run `vault_search.rb "pest-control-spring-2026"` |
|
||||
|
||||
## Inventory Query
|
||||
|
||||
```bash
|
||||
~/.claude/scripts/vault_search.rb "niche-automation-prospecting"
|
||||
```
|
||||
|
||||
For campaign-scoped inventory:
|
||||
```bash
|
||||
~/.claude/scripts/vault_search.rb "pest-control-spring-2026"
|
||||
```
|
||||
|
||||
The file `inventory-niche-automation-prospecting.base` renders this as a sortable table in Obsidian.
|
||||
|
||||
## Tag Inference Rules
|
||||
|
||||
When archiving any file from this project, apply tags automatically based on file path:
|
||||
|
||||
| Path pattern | Tags added |
|
||||
|---|---|
|
||||
| Any file from this repo | `niche-automation-prospecting` |
|
||||
| `campaigns/pest-control-spring-2026/` | `pest-control` · `pest-control-spring-2026` |
|
||||
| `*/research/` | `research` (type tag) |
|
||||
| `logs/` or `*/increments/` | `log` (type tag) |
|
||||
| Filename ends `-chatgpt` | `source-chatgpt` |
|
||||
| Filename ends `-gemini` | `source-gemini` |
|
||||
| Filename ends `-claude` | `source-claude` |
|
||||
|
||||
## Campaign → Automation Product Mapping
|
||||
|
||||
| Campaign | Tags |
|
||||
|---|---|
|
||||
| `pest-control-spring-2026` | `after-hours-sms` · `lead-capture` · `field-service` |
|
||||
|
||||
## Active Campaigns
|
||||
|
||||
- `pest-control-spring-2026`
|
||||
|
||||
## Related
|
||||
|
||||
- [[vault-conventions]]
|
||||
- [[inventory-niche-automation-prospecting]]
|
||||
|
|
@ -0,0 +1,134 @@
|
|||
---
|
||||
summary: Universal conventions for all notes in this vault — naming, frontmatter, tag taxonomy, hub notes, and Graphify awareness.
|
||||
tags:
|
||||
- type/meta
|
||||
- scope/global
|
||||
source: SecondBrain
|
||||
date: 2026-06-08
|
||||
---
|
||||
|
||||
# Vault Conventions
|
||||
|
||||
Universal conventions for all notes in this vault. One-stop reference for naming, frontmatter,
|
||||
note types, hub notes, and tag vocabulary. Project-specific rules live in `project-config-*` notes.
|
||||
|
||||
This vault stores **durable, evergreen knowledge** — finished plans, decisions, research, how-tos,
|
||||
and project learnings. It is NOT working memory. Working notes that don't yet have a permanent home
|
||||
should not be archived here until they have something stable to say.
|
||||
|
||||
## Navigation
|
||||
|
||||
| I want to... | Do this |
|
||||
|---|---|
|
||||
| Know the note types | See Note Types below |
|
||||
| Look up valid tag facets | See Tag Taxonomy below |
|
||||
| Know file naming rules | See File Naming below |
|
||||
| Understand hub notes | See Hub Notes below |
|
||||
| Find project-specific context | Read the relevant `project-config-*` note |
|
||||
|
||||
## File Naming
|
||||
|
||||
`YYYY-MM-DD-descriptive-slug.md`
|
||||
|
||||
- Date is creation date
|
||||
- Slug is lowercase kebab-case
|
||||
- Source platform suffix when applicable: `-chatgpt`, `-gemini`, `-claude`
|
||||
- Generated by tooling — do not rename manually
|
||||
|
||||
## Frontmatter Contract
|
||||
|
||||
Every note requires these fields. No exceptions; defer writing the `summary` for nothing.
|
||||
|
||||
```yaml
|
||||
---
|
||||
summary: One sentence, human-written at creation. Used by the Graphify graph and SessionStart context injection.
|
||||
tags:
|
||||
- type/reference # facet-namespaced; type/ listed first; at least one type/ required
|
||||
- client/acme # omit if not client-specific
|
||||
- project/website-redesign # omit if not project-specific
|
||||
- domain/seo # topic area
|
||||
- tool/semrush # tool referenced, if any
|
||||
- convention/api-style # cross-project convention, if applicable
|
||||
- scope/global # or scope/project — always include one
|
||||
source: <origin: repo name, "web-clipper", "ios-shortcut", "manual", etc.>
|
||||
date: YYYY-MM-DD
|
||||
---
|
||||
```
|
||||
|
||||
**`summary` field rules:**
|
||||
- One sentence, written by the human (or agent) at creation time.
|
||||
- Describes what the note *is about*, not what it contains.
|
||||
- Used by Graphify when building the knowledge graph and by the memory plugin for SessionStart injection.
|
||||
- Never defer it. A placeholder summary defeats the purpose.
|
||||
|
||||
## Note Types (`type/` facet)
|
||||
|
||||
Use exactly one `type/` tag per note.
|
||||
|
||||
| Type tag | Use for |
|
||||
|---|---|
|
||||
| `type/reference` | Reference material: market research, competitive intel, research dumps |
|
||||
| `type/plan` | Implementation plans, strategy docs |
|
||||
| `type/log` | Session logs, increment logs, decisions |
|
||||
| `type/adr` | Architecture decision records |
|
||||
| `type/howto` | Process docs, step-by-step guides |
|
||||
| `type/hub` | Hub notes: index/navigation notes linking 3+ related notes (see Hub Notes) |
|
||||
| `type/clip` | Web clips, saved articles |
|
||||
| `type/project-config` | Per-project tag inference rules and conventions |
|
||||
| `type/meta` | Vault governance (this file, CLAUDE.md) |
|
||||
|
||||
## Tag Taxonomy
|
||||
|
||||
Tags are **flat and parallel** — no nested paths, no hierarchy inside a facet. Six facets plus `scope/`:
|
||||
|
||||
| Facet | Purpose | Example |
|
||||
|---|---|---|
|
||||
| `type/` | Note type (see table above) | `type/howto` |
|
||||
| `client/` | Client or company this pertains to | `client/acme` |
|
||||
| `project/` | Specific project or engagement | `project/website-redesign` |
|
||||
| `domain/` | Topic area or discipline | `domain/seo`, `domain/cold-email` |
|
||||
| `tool/` | Tool, library, or platform referenced | `tool/graphify`, `tool/semrush` |
|
||||
| `convention/` | Cross-project convention or pattern | `convention/api-style` |
|
||||
| `scope/` | Applicability breadth | `scope/global` or `scope/project` |
|
||||
|
||||
**`scope/` values:**
|
||||
- `scope/global` — applies across all work; not tied to a specific client or project
|
||||
- `scope/project` — specific to one client/project context; pair with `client/` or `project/`
|
||||
|
||||
**Rules:**
|
||||
- Always include `type/` and `scope/`. All other facets are optional.
|
||||
- Use multiple tags within a facet when accurate (e.g., `domain/seo` and `domain/cold-email`).
|
||||
- Facets are not nested: `domain/cold-email` is correct; `domain/outbound/cold-email` is not.
|
||||
|
||||
## Hub Notes
|
||||
|
||||
When a topic has **3 or more related notes**, create a hub note (`type/hub`) that wikilinks to all
|
||||
of them. Hub notes are the primary navigation mechanism — hierarchy and relationships live here,
|
||||
not in nested folders or tag nesting.
|
||||
|
||||
A hub note should:
|
||||
- Have a descriptive `summary` explaining what this cluster is about
|
||||
- List wikilinks to all related notes with a one-line description of each
|
||||
- Use `type/hub` as the type tag
|
||||
- Be linked back to from the notes it indexes (add a `## Related` section)
|
||||
|
||||
Hierarchy lives in hub notes + wikilinks, not in folder structure or tag nesting.
|
||||
|
||||
## Graphify Awareness
|
||||
|
||||
The `graphify-out/` directory at the vault root is **generated by Graphify** — do not author files
|
||||
there. It is disposable and fully rebuilt by running `graphify` against the vault. The graph
|
||||
uses `summary` fields and wikilinks as primary signal.
|
||||
|
||||
Never edit files in `graphify-out/` by hand.
|
||||
|
||||
## Migration Compatibility
|
||||
|
||||
Pre-migration notes use **legacy flat tags** (`plan`, `research`, `log`, etc.) — both legacy and
|
||||
facet-namespaced forms coexist during incremental migration (ADR-013). Do not "fix" old notes on
|
||||
sight without a migration plan. New notes must use facet-namespaced tags from creation.
|
||||
|
||||
## Related
|
||||
|
||||
- [[CLAUDE]]
|
||||
- [[project-config-niche-automation-prospecting]]
|
||||
Loading…
Reference in New Issue