jared
/
cc-os
1
0
Fork 0
Code Issues 18 Pull Requests Packages Projects Releases Wiki Activity

Execute Step 2c: Claude reference-set benchmark — 18 gold-standard outputs 

Completed the benchmark run that gates Ollama model selection. Generated 18
reference fragments (6 cross-domain fixtures × 3 Claude tiers) and tightened
extraction spec rules based on first-run learnings. Updated implementation
status to reflect that reference set is complete and Ollama scoring is
unblocked. Populated fixture list in dispatch-prompt with final selection.
This commit is contained in:
jared 2026-06-04 12:49:22 -04:00
parent b408a62136
commit 77355b4b44
23 changed files with 2642 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/memory-system/04-build-plan.md
View File

@ -104,6 +104,10 @@ Opus output is the rubric. **Deferred later step:** local Ollama models (gemma4:
gemma4:e4b) are timed AND scored against these references — that scoring run is where speed
re-enters and the final model is chosen. Do not hardcode a model before that run completes.
**Status (2026-06-04): EXECUTED.** 6 cross-domain fixtures × 3 Claude tiers = 18 reference
fragments generated in `benchmark/reference-outputs/`. Run as-is (no vault frontmatter
modification); verified clean. Fixtures listed in `benchmark/dispatch-prompt.md`.
Authoritative detail lives in `docs/memory-system/05-implementation-process.md` §2c.
#### 2d — Initial fixture graph build (ADR-013: small-first)

19
docs/memory-system/05-implementation-process.md
View File

@ -1,6 +1,6 @@
# Implementation Process
_Last updated: 2026-06-04_ | _Status: Ready to plan — awaiting Step 2c reference set_
_Last updated: 2026-06-04_ | _Status: Step 2c executed — reference set complete; awaiting Ollama scoring_
This document distills `04-build-plan.md` into a concrete, staged build process and folds in
two locked decisions: **ADR-011** (faceted six-namespace taxonomy) and **ADR-012** (reuse the
@ -46,6 +46,10 @@ references — happens after the system is validated end-to-end.
### 1a — Select fixture notes
**Status (2026-06-04): DONE.** 6 cross-domain fixtures selected and listed in
`benchmark/dispatch-prompt.md`. Run as-is (no vault frontmatter modification per the
methodology decision — fixtures used exactly as they exist in the vault).
Choose 5–10 notes from `~/Documents/SecondBrain` with deliberate variety. The selection must
include at minimum:
@ -152,6 +156,10 @@ Verify context allocation after the first extraction call: `ollama ps` shows all
### 2c — Claude reference-set benchmark (THE GATE)
**Status (2026-06-04): EXECUTED.** 6 cross-domain fixtures × 3 Claude tiers = 18 reference
fragments generated in `benchmark/reference-outputs/`. Run as-is (no vault frontmatter
modification); verified clean. Gate is passed — Ollama model scoring is now unblocked.
Produce a reference set of Graphify-shaped extraction outputs before committing to any local
Ollama model. Use the 5–10 fixture notes selected in Step 1a as the input set.
@ -332,11 +340,10 @@ and Ollama models can be scored against it.
summaries. The human (or AI at note-creation time) must write `summary:` frontmatter. Confirm
this holds in practice and add a lint/reminder to the memory-write skill if it drifts.
6. **Step 2c reference set (Claude gold-standard outputs)** — Step 2c produces the scoring
rubric (Opus output as gold standard, Haiku/Sonnet as lighter-tier references), not the final
model choice. Local Ollama model selection is deferred until those references exist and Ollama
models can be timed and quality-scored against them. Do not hardcode `gemma4:e2b` — the Ollama
scoring step decides. The Step 2c outcome is the reference set, not the chosen model.
6. **Step 2c reference set (Claude gold-standard outputs)** — **RESOLVED (2026-06-04):** The
reference set was generated: 18 fragments (6 fixtures × 3 tiers) in `benchmark/reference-
outputs/`. Ollama model selection is now unblocked — score local models against these
references when Ollama is available. Do not hardcode `gemma4:e2b` until that scoring runs.
7. **memsearch + journal integration** — does memsearch index SessionEnd journal notes or only
its own auto-capture? How does the journal pointer injected at SessionStart reference the

13
docs/memory-system/benchmark/dispatch-prompt.md
View File

@ -34,12 +34,15 @@ Assign each note a **kebab-case slug** (used in output filenames). Populate the
below before dispatching:
```
# FIXTURES — populate before running
# Format: <note-slug>:<absolute-path-to-note>
# FIXTURES — selected 2026-06-04; benchmark run same day (no vault frontmatter modification)
# Format: <note-slug>:<vault-relative-path> (vault root: ~/Documents/SecondBrain)
FIXTURES=(
# <note-slug>:<path>
# <note-slug>:<path>
# ... (5-10 entries, one per selected note)
10dlc-isv-setup-guide:2026-03-31-10dlc-isv-setup-guide-oncadence.md
pest-control-enterprise-revenue:2026-03-13-pest-control-enterprise-revenue-architecture-and-seasonality.md
ai-coding-conventions-synthesis:2026-03-13-ai-coding-conventions-organization-external-research-synthesis.md
pest-control-sms-market-research-stats:2026-03-13-pest-control-after-hours-sms-lead-capture-market-research-stats.md
oo-principles-plugin-concept:2026-03-13-oo-principles-plugin-concept-design-recommendations.md
pest-control-email-abc-hub:2026-03-14-pest-control-email-a-b-c-test-experiment-hub.md
)
```

13
docs/memory-system/benchmark/extraction-spec.md
View File

@ -1,6 +1,6 @@
# Graphify Document-Extraction Benchmark — Shared Extraction Spec
_Last updated: 2026-06-04_
_Last updated: 2026-06-04 — rules tightened after first benchmark run (bare-slash facets and undeclared relationship entities)_
_Status: active — used as the shared schema for the Claude-tier reference-set benchmark_
## Purpose
@ -38,6 +38,12 @@ Notes in the SecondBrain vault carry six flat, namespaced facets plus one scope
An entity should carry a `facet` value only when the note text directly supports mapping it to one
of the above. Absence of a `facet` field is correct when no mapping is warranted.
**Sub-value required (STRICT):** A `facet` value MUST include a non-empty sub-value after the
namespace slash — e.g. `tool/cursor`, `convention/tdd`, `domain/distributed-systems`. A bare
namespace with nothing after the slash (e.g. `tool/`, `domain/`) is **invalid** and must never
appear in output. If a suitable sub-value cannot be identified from the note text, omit the
`facet` field entirely.
---
## Output Schema
@ -82,3 +88,8 @@ relationships:
- **No extra keys:** do not add summaries, scores, embeddings, or metadata fields.
- **One block per note:** if processing multiple fixture notes, emit one `note_slug:` block per note,
separated by `---`.
- **Referential integrity (STRICT):** every relationship `source` and `target` MUST exactly match
the `name` of an entity declared in the `entities` list of the same block. Do not reference
undeclared concept strings, free-form text, or the note's own `note_slug` as a `source` or
`target`. If a relationship needs a concept that is not yet in the entity list, declare it as an
entity first (with an appropriate `confidence` tag if it is inferred), then reference it.

88
docs/memory-system/benchmark/reference-outputs/10dlc-isv-setup-guide.haiku.md Normal file
View File

@ -0,0 +1,88 @@
note_slug: 10dlc-isv-setup-guide
entities:
- name: OnCadence
type: Project
facet: project/oncadence
- name: Twilio
type: Tool
facet: tool/twilio
- name: TrustHub API
type: Tool
facet: tool/twilio
- name: 10DLC
type: Domain
facet: domain/sms
- name: ISV architecture
type: Convention
facet: convention/isv-integration
- name: Secondary Customer Profile
type: Concept
- name: Brand registration
type: Concept
- name: Campaign registration
type: Concept
- name: Primary Business Profile
type: Concept
- name: pest control
type: Domain
facet: domain/pest-control
- name: opt-out language
type: Convention
- name: EIN
type: Concept
- name: subaccount
type: Concept
- name: T-Mobile
type: Tool
- name: Facebook Business page
type: Convention
facet: convention/web-presence
- name: Google Business Profile
type: Convention
facet: convention/web-presence
- name: Sole Proprietors
type: Concept
- name: long-code number
type: Concept
- name: SMS carrier surcharge
type: Concept
relationships:
- source: OnCadence
type: uses
target: Twilio
- source: OnCadence
type: implements
target: ISV architecture
- source: Twilio
type: provides
target: TrustHub API
- source: OnCadence
type: targets
target: pest control
- source: TrustHub API
type: manages
target: Brand registration
- source: TrustHub API
type: manages
target: Campaign registration
- source: Secondary Customer Profile
type: requires
target: EIN
- source: Brand registration
type: precedes
target: Campaign registration
- source: subaccount
type: carries
target: Secondary Customer Profile
- source: opt-out language
type: required_in
target: Campaign registration
- source: Sole Proprietors
type: limited_to
target: T-Mobile
- source: long-code number
type: isolated_per
target: subaccount
confidence: INFERRED

106
docs/memory-system/benchmark/reference-outputs/10dlc-isv-setup-guide.opus.md Normal file
View File

@ -0,0 +1,106 @@
# --- Graphify extraction fragment ---
note_slug: 10dlc-isv-setup-guide
entities:
- name: OnCadence
type: Project
facet: project/oncadence
- name: 10DLC
type: Concept
facet: domain/10dlc
- name: Twilio
type: Tool
facet: tool/twilio
- name: TrustHub API
type: Tool
facet: tool/twilio-trusthub
- name: ISV
type: Concept
- name: CSP
type: Concept
- name: pest control clients
type: Client
facet: client/pest-control
- name: Secondary Customer Profile
type: Concept
- name: Primary Business Profile
type: Concept
- name: Brand registration
type: Concept
- name: Campaign registration
type: Concept
- name: EIN
type: Concept
- name: IRS
type: Concept
- name: Facebook Business page
type: Concept
- name: Google Business Profile
type: Concept
- name: T-Mobile
type: Concept
- name: Sole Proprietors
type: Concept
- name: long-code number
type: Concept
- name: STOP opt-out language
type: Convention
facet: convention/sms-opt-out
- name: subaccount-per-client architecture
type: Concept
facet: convention/subaccount-per-client
relationships:
- source: OnCadence
type: is_primary_account_of
target: Twilio
- source: OnCadence
type: acts_as
target: ISV
- source: OnCadence
type: acts_as
target: CSP
- source: OnCadence
type: manages_10dlc_for
target: pest control clients
- source: OnCadence
type: submits_via
target: TrustHub API
- source: OnCadence
type: creates_once
target: Primary Business Profile
- source: pest control clients
type: each_assigned
target: Secondary Customer Profile
- source: pest control clients
type: each_assigned
target: Brand registration
- source: pest control clients
type: each_assigned
target: Campaign registration
- source: Brand registration
type: must_match
target: EIN
- source: EIN
type: issued_by
target: IRS
confidence: INFERRED
- source: Sole Proprietors
type: lack
target: EIN
- source: Sole Proprietors
type: limited_to_throughput_on
target: T-Mobile
- source: Facebook Business page
type: accepted_for
target: Brand registration
confidence: INFERRED
- source: Campaign registration
type: must_include
target: STOP opt-out language
- source: OnCadence
type: implements
target: subaccount-per-client architecture
- source: pest control clients
type: each_assigned
target: long-code number

66
docs/memory-system/benchmark/reference-outputs/10dlc-isv-setup-guide.sonnet.md Normal file
View File

@ -0,0 +1,66 @@
# --- Graphify extraction fragment ---
note_slug: 10dlc-isv-setup-guide
entities:
- name: OnCadence
type: Project
facet: project/OnCadence
- name: Twilio
type: Tool
facet: tool/Twilio
- name: 10DLC
type: Concept
facet: domain/SMS
- name: TrustHub API
type: Tool
facet: tool/TrustHub API
- name: ISV
type: Concept
facet: domain/SMS
- name: Secondary Customer Profile
type: Concept
- name: Brand registration
type: Concept
- name: Campaign registration
type: Concept
- name: Primary Business Profile
type: Concept
- name: T-Mobile
type: Client
- name: Sole Proprietor
type: Concept
- name: pest control client
type: Client
facet: client/pest-control
relationships:
- source: OnCadence
type: acts as ISV/CSP on
target: Twilio
- source: OnCadence
type: submits registrations on behalf of
target: pest control client
- source: OnCadence
type: uses
target: TrustHub API
- source: pest control client
type: registered as
target: Secondary Customer Profile
- source: pest control client
type: requires
target: Brand registration
- source: pest control client
type: requires
target: Campaign registration
- source: OnCadence
type: creates once
target: Primary Business Profile
- source: Sole Proprietor
type: has lower throughput on
target: T-Mobile
- source: 10DLC
type: governs
target: Campaign registration
- source: TrustHub API
type: is part of
target: Twilio

149
docs/memory-system/benchmark/reference-outputs/ai-coding-conventions-synthesis.haiku.md Normal file
View File

@ -0,0 +1,149 @@
note_slug: ai-coding-conventions-synthesis
entities:
- name: Claude
type: tool
facet: tool/claude
- name: Cursor
type: tool
facet: tool/cursor
- name: Copilot
type: tool
facet: tool/copilot
- name: CLAUDE.md
type: document_type
facet: convention/root-file
- name: .cursorrules
type: document_type
facet: convention/root-file
- name: AGENTS.md
type: document_type
facet: convention/hierarchical-scoping
- name: Three-Tier Architecture
type: framework
facet: convention/context-architecture
- name: Constitution
type: framework_component
facet: convention/constitution-pattern
- name: Domain Specialist Agents
type: framework_component
facet: convention/specialist-agents
- name: Knowledge Base
type: framework_component
- name: Codified Context
type: research_paper
facet: domain/context-engineering
- name: arXiv 2602.20478
type: publication
- name: Nuxt Content
type: tool
facet: tool/nuxt
- name: MCP
type: tool
facet: tool/mcp
- name: Token Efficiency Strategies
type: practice_set
facet: convention/token-optimization
- name: Just-in-time retrieval
type: practice
facet: convention/context-loading
- name: Linters
type: tool_category
facet: tool/linting
- name: Formatters
type: tool_category
facet: tool/formatting
- name: Feedback loop documentation
type: practice
facet: convention/feedback-loop
- name: Process encoding
type: practice
facet: convention/process-encoding
- name: Trigger tables
type: mechanism
facet: convention/routing-mechanism
- name: Reference documentation
type: practice
facet: convention/documentation-strategy
- name: Hierarchical override
type: mechanism
facet: convention/context-scoping
- name: github.com/arisvas4/codified-context-infrastructure
type: reference
- name: Anthropic Effective Context Engineering
type: reference
- name: PatrickJS/awesome-cursorrules
type: reference
- name: alexop.dev
type: reference
- name: mbleigh.dev
type: reference
- name: agents.md specification
type: standard
- name: 99 Bottles OOP — Full Software Design Process Map
type: reference
- name: OO Principles Plugin Concept — Design Recommendations
type: reference
relationships:
- source: Three-Tier Architecture
type: comprises
target: Constitution
- source: Three-Tier Architecture
type: comprises
target: Domain Specialist Agents
- source: Three-Tier Architecture
type: comprises
target: Knowledge Base
- source: Constitution
type: routes to
target: Domain Specialist Agents
- source: Domain Specialist Agents
type: contains
target: Knowledge Base
confidence: INFERRED
- source: Knowledge Base
type: served via
target: MCP
- source: CLAUDE.md
type: implements
target: Constitution
- source: .cursorrules
type: implements
target: Constitution
- source: AGENTS.md
type: implements
target: Hierarchical override
- source: Token Efficiency Strategies
type: includes
target: Just-in-time retrieval
- source: Token Efficiency Strategies
type: includes
target: Linters
- source: Token Efficiency Strategies
type: includes
target: Formatters
- source: Token Efficiency Strategies
type: includes
target: Feedback loop documentation
- source: Linters
type: replaces
target: Reference documentation
confidence: INFERRED
- source: Formatters
type: replaces
target: Reference documentation
confidence: INFERRED
- source: Process encoding
type: encodes
target: Trigger tables
- source: Trigger tables
type: enforces
target: Constitution
confidence: INFERRED
- source: Codified Context
type: describes
target: Three-Tier Architecture
- source: Codified Context
type: published at
target: arXiv 2602.20478

156
docs/memory-system/benchmark/reference-outputs/ai-coding-conventions-synthesis.opus.md Normal file
View File

@ -0,0 +1,156 @@
# --- Graphify extraction fragment ---
note_slug: ai-coding-conventions-synthesis
entities:
- name: Claude
type: Tool
facet: tool/claude
- name: Cursor
type: Tool
facet: tool/cursor
- name: Copilot
type: Tool
facet: tool/copilot
- name: Codified Context
type: Concept
- name: Three-Tier Architecture
type: Concept
- name: Constitution
type: Concept
- name: Domain Specialist Agents
type: Concept
- name: Knowledge Base
type: Concept
- name: MCP
type: Tool
facet: tool/mcp
- name: Just-in-time retrieval
type: Convention
facet: convention/just-in-time-retrieval
- name: Trigger tables
type: Concept
- name: Progressive disclosure
type: Convention
facet: convention/progressive-disclosure
- name: Positive framing only
type: Convention
facet: convention/positive-framing
- name: /learn pattern
type: Convention
facet: convention/feedback-loop-documentation
- name: CLAUDE.md
type: Convention
facet: convention/root-file
- name: .cursorrules
type: Convention
facet: convention/root-file
- name: AGENTS.md
type: Convention
facet: convention/root-file
- name: Cursor .mdc
type: Tool
- name: alwaysApply
type: Convention
- name: Auto-attached rules
type: Convention
- name: Agent-requested rules
type: Convention
- name: Manual rules
type: Convention
- name: Hierarchical scoping
type: Convention
facet: convention/hierarchical-scoping
- name: Nuxt Content
type: Tool
facet: tool/nuxt-content
- name: Codified Context paper
type: Concept
- name: github.com/arisvas4/codified-context-infrastructure
type: Project
- name: Anthropic Effective Context Engineering
type: Concept
- name: PatrickJS/awesome-cursorrules
type: Project
- name: alexop.dev — Stop Bloating Your CLAUDE.md
type: Concept
- name: mbleigh.dev — Rules for Rules
type: Concept
- name: agents.md
type: Project
- name: OpenAI
type: Client
- name: Anthropic
type: Client
- name: Context engineering
type: Domain
facet: domain/context-engineering
- name: AI coding conventions
type: Domain
facet: domain/ai-coding-conventions
- name: 99 Bottles OOP — Full Software Design Process Map
type: Concept
- name: OO Principles Plugin Concept — Design Recommendations
type: Concept
- name: Linters
type: Tool
- name: Formatters
type: Tool
- name: Type checkers
type: Tool
relationships:
- source: Three-Tier Architecture
type: comprises
target: Constitution
- source: Three-Tier Architecture
type: comprises
target: Domain Specialist Agents
- source: Three-Tier Architecture
type: comprises
target: Knowledge Base
- source: Codified Context
type: documents
target: Three-Tier Architecture
- source: Constitution
type: contains
target: Trigger tables
- source: Knowledge Base
type: served_via
target: MCP
- source: Codified Context paper
type: described_in
target: github.com/arisvas4/codified-context-infrastructure
confidence: INFERRED
- source: Constitution
type: routes_tasks_to
target: Domain Specialist Agents
- source: OpenAI
type: uses
target: AGENTS.md
- source: Anthropic
type: published
target: Anthropic Effective Context Engineering
confidence: INFERRED
- source: Cursor
type: uses
target: .cursorrules
confidence: INFERRED
- source: Cursor .mdc
type: supports
target: alwaysApply
- source: Cursor .mdc
type: supports
target: Auto-attached rules
- source: Cursor .mdc
type: supports
target: Agent-requested rules
- source: Cursor .mdc
type: supports
target: Manual rules
- source: Linters
type: replace
target: alwaysApply
confidence: INFERRED
- source: Trigger tables
type: route_to
target: Domain Specialist Agents

135
docs/memory-system/benchmark/reference-outputs/ai-coding-conventions-synthesis.sonnet.md Normal file
View File

@ -0,0 +1,135 @@
# --- Graphify extraction fragment ---
note_slug: ai-coding-conventions-synthesis
entities:
- name: Codified Context
type: Concept
facet: convention/codified-context
confidence: INFERRED
- name: Three-Tier Architecture
type: Concept
facet: convention/three-tier-architecture
- name: Constitution
type: Concept
facet: convention/constitution
- name: Domain Specialist Agents
type: Concept
facet: convention/domain-specialist-agents
- name: Knowledge Base
type: Concept
facet: convention/knowledge-base
- name: CLAUDE.md
type: Convention
facet: tool/claude-md
- name: AGENTS.md
type: Convention
facet: tool/agents-md
- name: Cursor
type: Tool
facet: tool/cursor
- name: Claude
type: Tool
facet: tool/claude
- name: Copilot
type: Tool
facet: tool/copilot
- name: MCP
type: Tool
facet: tool/mcp
- name: arXiv 2602.20478
type: Reference
- name: github.com/arisvas4/codified-context-infrastructure
type: Reference
- name: Anthropic Effective Context Engineering
type: Reference
- name: PatrickJS/awesome-cursorrules
type: Reference
- name: alexop.dev — Stop Bloating Your CLAUDE.md
type: Reference
- name: mbleigh.dev — Rules for Rules
type: Reference
- name: agents.md
type: Reference
- name: progressive-disclosure
type: Concept
facet: convention/progressive-disclosure
- name: trigger tables
type: Concept
facet: convention/trigger-tables
- name: feedback loop documentation
type: Concept
facet: convention/feedback-loop-documentation
- name: hierarchical scoping
type: Concept
facet: convention/hierarchical-scoping
- name: token efficiency
type: Concept
facet: convention/token-efficiency
- name: just-in-time retrieval
type: Concept
facet: convention/just-in-time-retrieval
- name: cursor rules
type: Convention
facet: tool/cursor-rules
- name: hyperthrive_dev
type: Person
confidence: AMBIGUOUS
relationships:
- source: Three-Tier Architecture
type: documented_in
target: arXiv 2602.20478
- source: Three-Tier Architecture
type: consists_of
target: Constitution
- source: Three-Tier Architecture
type: consists_of
target: Domain Specialist Agents
- source: Three-Tier Architecture
type: consists_of
target: Knowledge Base
- source: Constitution
type: uses
target: trigger tables
- source: Domain Specialist Agents
type: invoked_by
target: trigger tables
- source: Knowledge Base
type: served_via
target: MCP
- source: CLAUDE.md
type: implements
target: Constitution
confidence: INFERRED
- source: AGENTS.md
type: implements
target: Constitution
confidence: INFERRED
- source: cursor rules
type: implements
target: hierarchical scoping
- source: Three-Tier Architecture
type: enables
target: token efficiency
- source: progressive-disclosure
type: supports
target: token efficiency
- source: just-in-time retrieval
type: supports
target: token efficiency
- source: feedback loop documentation
type: converts_errors_into
target: Constitution
confidence: INFERRED
- source: Codified Context
type: described_by
target: arXiv 2602.20478
- source: Codified Context
type: exemplified_by
target: github.com/arisvas4/codified-context-infrastructure
- source: Claude
type: guided_by
target: CLAUDE.md
- source: Cursor
type: guided_by
target: cursor rules

150
docs/memory-system/benchmark/reference-outputs/oo-principles-plugin-concept.haiku.md Normal file
View File

@ -0,0 +1,150 @@
note_slug: oo-principles-plugin-concept
entities:
- name: OO Principles Plugin
type: plugin
facet: type/plugin
- name: 99 Bottles of OOP
type: reference-book
facet: domain/oo-design
- name: Shameless Green
type: principle
facet: convention/shameless-green
- name: Flocking Rules
type: principle
facet: convention/flocking-rules
- name: Replace Conditional with Polymorphism
type: refactoring-technique
facet: convention/polymorphism-refactoring
- name: Single Responsibility Principle
type: principle
facet: convention/srp
- name: Law of Demeter
type: principle
facet: convention/law-of-demeter
- name: Dependency Injection
type: principle
facet: convention/dependency-injection
- name: Open/Closed Principle
type: principle
facet: convention/open-closed
- name: Process/Routing Layer
type: architectural-component
facet: type/architecture
- name: Mechanic Layer
type: architectural-component
facet: type/architecture
- name: Theory Layer
type: architectural-component
facet: type/architecture
- name: PROCESS.md
type: documentation-file
facet: convention/process-routing
- name: concept cards
type: documentation-format
- name: NotebookLM notebook
type: knowledge-base
facet: tool/notebooklm
- name: Phase 1
type: development-phase
- name: Phase 2
type: development-phase
- name: Phase 3
type: development-phase
- name: 4-phase development lifecycle
type: process-model
- name: Approach 1: Process-First Entry Point
type: architectural-approach
- name: Approach 2: Situational Trigger Files
type: architectural-approach
- name: Approach 3: Phase-Bundled Context Packs
type: architectural-approach
- name: Approach 4: Annotated Process Graph
type: architectural-approach
- name: conventions/oo-principles/
type: workspace-structure
facet: project/hyperthrive
- name: hyperthrive_dev
type: project
facet: project/hyperthrive
- name: Codified Context paper
type: research-reference
- name: TDD
type: principle
facet: convention/tdd
- name: QUICK_REFERENCE.md
type: documentation-file
- name: role bundles
type: documentation-format
- name: AI Coding Conventions Organization — External Research Synthesis
type: reference-document
- name: 99 Bottles OOP — Full Software Design Process Map
type: reference-document
- name: CLAUDE.md
type: documentation-file
facet: tool/claude-code
relationships:
- source: OO Principles Plugin
type: is designed to encode
target: 4-phase development lifecycle
- source: OO Principles Plugin
type: models architecture on
target: Process/Routing Layer
- source: OO Principles Plugin
type: models architecture on
target: Mechanic Layer
- source: OO Principles Plugin
type: models architecture on
target: Theory Layer
- source: Mechanic Layer
type: should contain recipes for
target: Flocking Rules
- source: Mechanic Layer
type: should contain recipes for
target: Replace Conditional with Polymorphism
- source: Theory Layer
type: contains explanatory material for
target: concept cards
- source: Process/Routing Layer
type: references
target: Mechanic Layer
- source: Flocking Rules
type: competes with
target: Replace Conditional with Polymorphism
- source: Dependency Injection
type: should be checked before
target: Law of Demeter
- source: Law of Demeter
type: should be checked before
target: Single Responsibility Principle
- source: Phase 2
type: includes opportunistic refactoring guided by
target: Law of Demeter
- source: Phase 2
type: includes opportunistic refactoring guided by
target: Dependency Injection
- source: Approach 1: Process-First Entry Point
type: is recommended for
target: OO Principles Plugin
- source: NotebookLM notebook
type: serves as knowledge base for
target: OO Principles Plugin
- source: conventions/oo-principles/
type: is baseline for
target: OO Principles Plugin
- source: hyperthrive_dev
type: is source context for
target: OO Principles Plugin
- source: OO Principles Plugin
type: based on
target: 99 Bottles of OOP
- source: Process/Routing Layer
type: encodes
target: 4-phase development lifecycle
- source: Phase 1
type: applies
target: Shameless Green
- source: Phase 3
type: requires checking
target: Open/Closed Principle

159
docs/memory-system/benchmark/reference-outputs/oo-principles-plugin-concept.opus.md Normal file
View File

@ -0,0 +1,159 @@
# --- Graphify extraction fragment ---
note_slug: oo-principles-plugin-concept
entities:
- name: OO Principles Plugin
type: plugin
facet: type/plugin
- name: oo-principles conventions structure
type: conventions structure
facet: convention/oo-principles
- name: 99 Bottles of OOP
type: book
facet: domain/object-oriented-design
- name: NotebookLM notebook
type: knowledge base resource
facet: tool/notebooklm
- name: Codified Context paper
type: research paper
- name: TDD
type: principle
facet: convention/tdd
- name: Shameless Green
type: principle
facet: convention/shameless-green
- name: Flocking Rules
type: principle
facet: convention/flocking-rules
- name: SRP
type: principle
facet: convention/srp
- name: LoD
type: principle
facet: convention/law-of-demeter
- name: DI
type: principle
facet: convention/dependency-injection
- name: Replace Conditional with Polymorphism
type: principle
facet: convention/replace-conditional-with-polymorphism
- name: Open/Closed
type: principle
facet: convention/open-closed
- name: 4-phase development lifecycle
type: process model
- name: QUICK_REFERENCE.md
type: file
- name: cards/
type: directory
- name: bundles/
type: directory
- name: Two-Layer Hybrid
type: architecture
- name: Process/Routing Layer
type: layer
- name: Mechanic Layer
type: layer
- name: Theory Layer
type: layer
- name: PROCESS.md
type: file
- name: Approach 1 - Process-First Entry Point
type: architectural approach
- name: Approach 2 - Situational Trigger Files
type: architectural approach
- name: Approach 3 - Phase-Bundled Context Packs
type: architectural approach
- name: Approach 4 - Annotated Process Graph
type: architectural approach
- name: Phase 2
type: development phase
- name: three-tier architecture
type: architecture
- name: CLAUDE.md
type: file
- name: 99 Bottles OOP — Full Software Design Process Map
type: note
- name: AI Coding Conventions Organization — External Research Synthesis
type: note
relationships:
- source: OO Principles Plugin
type: evolves from
target: oo-principles conventions structure
- source: OO Principles Plugin
type: based on study of
target: 99 Bottles of OOP
- source: OO Principles Plugin
type: informed by research on
target: Codified Context paper
- source: OO Principles Plugin
type: can use as knowledge base layer
target: NotebookLM notebook
- source: oo-principles conventions structure
type: contains
target: QUICK_REFERENCE.md
- source: oo-principles conventions structure
type: contains
target: cards/
- source: oo-principles conventions structure
type: contains
target: bundles/
- source: Two-Layer Hybrid
type: includes
target: Process/Routing Layer
- source: Two-Layer Hybrid
type: includes
target: Mechanic Layer
- source: Two-Layer Hybrid
type: includes
target: Theory Layer
- source: Process/Routing Layer
type: encodes
target: 4-phase development lifecycle
- source: Process/Routing Layer
type: contains
target: PROCESS.md
- source: Process/Routing Layer
type: references
target: Mechanic Layer
- source: Theory Layer
type: retains
target: cards/
confidence: INFERRED
- source: Approach 1 - Process-First Entry Point
type: adds as entry point
target: PROCESS.md
- source: 99 Bottles of OOP
type: defines
target: Shameless Green
confidence: INFERRED
- source: 99 Bottles of OOP
type: defines
target: Flocking Rules
confidence: INFERRED
- source: Phase 2
type: addresses
target: LoD
- source: Phase 2
type: addresses
target: DI
- source: Replace Conditional with Polymorphism
type: alternative to
target: Flocking Rules
- source: 4-phase development lifecycle
type: included as loadable context pack in
target: OO Principles Plugin
- source: OO Principles Plugin
type: modeled on
target: three-tier architecture
- source: three-tier architecture
type: routing entry point is
target: CLAUDE.md
confidence: INFERRED
- source: oo-principles conventions structure
type: see also
target: 99 Bottles OOP — Full Software Design Process Map
- source: oo-principles conventions structure
type: see also
target: AI Coding Conventions Organization — External Research Synthesis

126
docs/memory-system/benchmark/reference-outputs/oo-principles-plugin-concept.sonnet.md Normal file
View File

@ -0,0 +1,126 @@
# --- Graphify extraction fragment ---
note_slug: oo-principles-plugin-concept
entities:
- name: OO Principles Plugin
type: project
facet: project/oo-principles-plugin
- name: conventions/oo-principles
type: directory-structure
facet: convention/oo-principles
- name: QUICK_REFERENCE.md
type: file
- name: concept cards
type: document-type
- name: role bundles
type: document-type
- name: PROCESS.md
type: file
- name: 99 Bottles of OOP
type: book
facet: domain/software-design
- name: Codified Context paper
type: external-reference
- name: NotebookLM
type: tool
facet: tool/notebooklm
- name: Shameless Green
type: development-principle
facet: convention/shameless-green
- name: Flocking Rules
type: development-principle
facet: convention/flocking-rules
- name: Replace Conditional with Polymorphism
type: development-principle
facet: convention/replace-conditional-with-polymorphism
- name: TDD
type: development-principle
facet: convention/tdd
- name: SRP
type: development-principle
facet: convention/srp
- name: Law of Demeter
type: development-principle
facet: convention/law-of-demeter
- name: Dependency Injection
type: development-principle
facet: convention/dependency-injection
- name: 4-phase development lifecycle
type: concept
facet: convention/4-phase-lifecycle
- name: Process-First Entry Point
type: architectural-approach
- name: Situational Trigger Files
type: architectural-approach
- name: Phase-Bundled Context Packs
type: architectural-approach
- name: Annotated Process Graph
type: architectural-approach
- name: Two-Layer Hybrid architecture
type: architectural-pattern
- name: hyperthrive_dev
type: client
facet: client/hyperthrive-dev
- name: Rails
type: tool
facet: tool/rails
- name: CLAUDE.md
type: file
- name: AI Coding Conventions Organization — External Research Synthesis
type: note
confidence: INFERRED
- name: 99 Bottles OOP — Full Software Design Process Map
type: note
confidence: INFERRED
relationships:
- source: OO Principles Plugin
type: evolved from
target: conventions/oo-principles
- source: OO Principles Plugin
type: uses architecture
target: Two-Layer Hybrid architecture
- source: Two-Layer Hybrid architecture
type: layer 1 is
target: PROCESS.md
- source: Two-Layer Hybrid architecture
type: layer 3 is
target: concept cards
- source: OO Principles Plugin
type: informed by
target: 99 Bottles of OOP
- source: OO Principles Plugin
type: informed by
target: Codified Context paper
- source: PROCESS.md
type: encodes
target: 4-phase development lifecycle
- source: Flocking Rules
type: contrasted with
target: Replace Conditional with Polymorphism
- source: Shameless Green
type: part of
target: 4-phase development lifecycle
- source: Process-First Entry Point
type: recommended over
target: Situational Trigger Files
- source: Process-First Entry Point
type: recommended over
target: Phase-Bundled Context Packs
- source: Process-First Entry Point
type: recommended over
target: Annotated Process Graph
- source: NotebookLM
type: serves as knowledge base for
target: OO Principles Plugin
- source: hyperthrive_dev
type: source workspace for
target: conventions/oo-principles
- source: CLAUDE.md
type: entry point for
target: conventions/oo-principles
confidence: INFERRED
- source: OO Principles Plugin
type: links to
target: 99 Bottles OOP — Full Software Design Process Map
- source: OO Principles Plugin
type: links to
target: AI Coding Conventions Organization — External Research Synthesis

177
docs/memory-system/benchmark/reference-outputs/pest-control-email-abc-hub.haiku.md Normal file
View File

@ -0,0 +1,177 @@
note_slug: pest-control-email-abc-hub
entities:
- name: Pest Control Email A/B/C Test
type: experiment
- name: Golden 4-Sentence Framework
type: email draft variant
- name: Revised
type: email draft variant
- name: Copywriter
type: email draft variant
- name: pest-control-spring-2026
type: campaign
facet: project/pest-control-spring-2026
- name: Mike Deluca
type: persona
facet: client/pest-control-owner
confidence: INFERRED
- name: Sandra Kowalski
type: persona
facet: client/pest-control-owner
confidence: INFERRED
- name: Ray Tanner
type: persona
facet: client/pest-control-owner
confidence: INFERRED
- name: gatekeeper personas
type: judge group
- name: pest control owner-operators
type: judge group
- name: Group 3 Revised
type: email group variant
- name: Group 7 Copywriter
type: email group variant
- name: Group 9 Revised
type: email group variant
- name: Group 2 Copywriter
type: email group variant
- name: Group 5 Revised
type: email group variant
- name: Two-scenario contrast format
type: copy pattern
facet: convention/email-copy-pattern
- name: Scenario first, stat second
type: structural pattern
facet: convention/email-copy-pattern
- name: Reframe before solution
type: structural pattern
facet: convention/email-copy-pattern
- name: Single closing question with honest out
type: structural pattern
facet: convention/email-copy-pattern
- name: Review-shaming openers
type: anti-pattern
facet: convention/email-copy-pattern
- name: Fabricated revenue math
type: anti-pattern
facet: convention/email-copy-pattern
- name: Escape-hatch closes
type: anti-pattern
facet: convention/email-copy-pattern
- name: Self-aware meta-openers
type: anti-pattern
facet: convention/email-copy-pattern
- name: Saturday wasp nest scenario
type: email hook
- name: Scenario A vs. B two-format
type: email hook
- name: Lead disappearance problem reframe
type: email hook
- name: Bed bugs at 11pm scenario
type: email hook
- name: Termite LTV urgency
type: email hook
- name: GorillaDesk/P&L email
type: referenced email version
confidence: INFERRED
- name: LSA-anchor version
type: email variant
- name: Driven Results
type: citation source
confidence: INFERRED
relationships:
- source: Pest Control Email A/B/C Test
type: compares
target: Golden 4-Sentence Framework
- source: Pest Control Email A/B/C Test
type: compares
target: Revised
- source: Pest Control Email A/B/C Test
type: compares
target: Copywriter
- source: Pest Control Email A/B/C Test
type: part of campaign
target: pest-control-spring-2026
- source: Pest Control Email A/B/C Test
type: judged by
target: gatekeeper personas
- source: Pest Control Email A/B/C Test
type: judged by
target: pest control owner-operators
- source: Mike Deluca
type: participates in
target: Pest Control Email A/B/C Test
- source: Sandra Kowalski
type: participates in
target: Pest Control Email A/B/C Test
- source: Ray Tanner
type: participates in
target: Pest Control Email A/B/C Test
- source: Revised
type: wins
target: Pest Control Email A/B/C Test
- source: Copywriter
type: runner-up in
target: Pest Control Email A/B/C Test
- source: Group 3 Revised
type: unanimous sweep in
target: Pest Control Email A/B/C Test
- source: Group 7 Copywriter
type: unanimous sweep in
target: Pest Control Email A/B/C Test
- source: Two-scenario contrast format
type: recommended for adoption
target: pest-control-spring-2026
- source: Scenario first, stat second
type: recommended for adoption
target: pest-control-spring-2026
- source: Reframe before solution
type: recommended for adoption
target: pest-control-spring-2026
- source: Single closing question with honest out
type: recommended for adoption
target: pest-control-spring-2026
- source: Review-shaming openers
type: recommended to retire
target: pest-control-spring-2026
- source: Fabricated revenue math
type: recommended to retire
target: pest-control-spring-2026
- source: Escape-hatch closes
type: recommended to retire
target: pest-control-spring-2026
- source: Self-aware meta-openers
type: recommended to retire
target: pest-control-spring-2026
- source: Group 3 Revised
type: uses hook
target: Saturday wasp nest scenario
- source: Group 7 Copywriter
type: uses hook
target: Scenario A vs. B two-format
- source: Group 9 Revised
type: uses hook
target: Lead disappearance problem reframe
- source: Group 2 Copywriter
type: uses hook
target: Bed bugs at 11pm scenario
- source: Group 5 Revised
type: uses hook
target: Termite LTV urgency
- source: gatekeeper personas
type: predicted preference for
target: Revised
confidence: AMBIGUOUS
- source: pest control owner-operators
type: prefer
target: Revised
- source: LSA-anchor version
type: works for
target: paid-acquisition operators
confidence: INFERRED
- source: LSA-anchor version
type: rejected by
target: referral-only operator
confidence: INFERRED

192
docs/memory-system/benchmark/reference-outputs/pest-control-email-abc-hub.opus.md Normal file
View File

@ -0,0 +1,192 @@
# --- Graphify extraction fragment ---
note_slug: pest-control-email-abc-hub
entities:
- name: Pest Control Email A/B/C Test
type: experiment
facet: type/experiment
- name: pest-control-spring-2026
type: campaign
facet: project/pest-control-spring-2026
- name: Golden 4-Sentence Framework
type: copy framework
facet: convention/golden-4-sentence-framework
- name: draft
type: email version
- name: revised
type: email version
- name: copywriter
type: email version
- name: Mike Deluca
type: owner-operator persona
- name: Sandra Kowalski
type: owner-operator persona
- name: Ray Tanner
type: owner-operator persona
- name: gatekeeper panel
type: judging panel
- name: owner panel
type: judging panel
- name: Group 3
type: email group
- name: Group 7
type: email group
- name: Group 2
type: email group
- name: Group 4
type: email group
- name: Group 5
type: email group
- name: Group 8
type: email group
- name: Group 9
type: email group
- name: Saturday wasp nest scenario
type: email hook
- name: Scenario A vs. B two-format
type: email hook
- name: Lead disappearance problem reframe
type: email hook
- name: Bed bugs at 11pm scenario
type: email hook
- name: Termite LTV urgency
type: email hook
- name: Two-scenario contrast format
type: structural pattern
facet: convention/two-scenario-contrast-format
- name: Scenario first, stat second
type: structural pattern
facet: convention/scenario-first-stat-second
- name: Reframe before solution
type: structural pattern
facet: convention/reframe-before-solution
- name: Single closing question with an honest out
type: structural pattern
facet: convention/single-closing-question-honest-out
- name: Fabricated revenue math
type: retired pattern
- name: Review-shaming openers
type: retired pattern
- name: Escape-hatch closes
type: retired pattern
- name: Guessed specificity
type: retired pattern
- name: Self-aware meta-openers
type: retired pattern
- name: GorillaDesk/P&L email
type: email
- name: Driven Results citation
type: citation
- name: LSA-anchor version
type: email version
- name: $136,500 ROI projection
type: copy claim
relationships:
- source: Pest Control Email A/B/C Test
type: tested for campaign
target: pest-control-spring-2026
- source: Pest Control Email A/B/C Test
type: compared
target: draft
- source: Pest Control Email A/B/C Test
type: compared
target: revised
- source: Pest Control Email A/B/C Test
type: compared
target: copywriter
- source: draft
type: built on framework
target: Golden 4-Sentence Framework
- source: gatekeeper panel
type: judged
target: Pest Control Email A/B/C Test
- source: owner panel
type: judged
target: Pest Control Email A/B/C Test
- source: owner panel
type: includes persona
target: Mike Deluca
- source: owner panel
type: includes persona
target: Sandra Kowalski
- source: owner panel
type: includes persona
target: Ray Tanner
- source: revised
type: won overall
target: Pest Control Email A/B/C Test
- source: copywriter
type: led in raw first-place owner votes
target: revised
- source: Group 3
type: swept by version
target: revised
- source: Group 7
type: swept by version
target: copywriter
- source: Group 3
type: uses hook
target: Saturday wasp nest scenario
- source: Group 3
type: cites
target: Driven Results citation
- source: Group 7
type: uses hook
target: Scenario A vs. B two-format
- source: Group 9
type: uses hook
target: Lead disappearance problem reframe
- source: Group 2
type: uses hook
target: Bed bugs at 11pm scenario
- source: Group 5
type: uses hook
target: Termite LTV urgency
- source: Group 8
type: gatekeepers ranked first
target: GorillaDesk/P&L email
- source: Group 5
type: contains copy claim
target: $136,500 ROI projection
confidence: INFERRED
- source: LSA-anchor version
type: appears in
target: Group 4
- source: Group 7
type: exemplifies pattern
target: Two-scenario contrast format
- source: Group 9
type: exemplifies pattern
target: Reframe before solution
confidence: INFERRED
- source: Pest Control Email A/B/C Test
type: recommends for live sequence
target: Group 3
- source: Pest Control Email A/B/C Test
type: recommends for live sequence
target: Group 7
- source: Pest Control Email A/B/C Test
type: recommends for live sequence
target: Group 9
- source: Pest Control Email A/B/C Test
type: recommends for live sequence
target: Group 2
- source: Pest Control Email A/B/C Test
type: recommends for live sequence
target: Group 5
- source: Pest Control Email A/B/C Test
type: recommends retiring
target: Fabricated revenue math
- source: Pest Control Email A/B/C Test
type: recommends retiring
target: Review-shaming openers
- source: Pest Control Email A/B/C Test
type: recommends retiring
target: Escape-hatch closes
- source: Pest Control Email A/B/C Test
type: recommends retiring
target: Guessed specificity
- source: Pest Control Email A/B/C Test
type: recommends retiring
target: Self-aware meta-openers

104
docs/memory-system/benchmark/reference-outputs/pest-control-email-abc-hub.sonnet.md Normal file
View File

@ -0,0 +1,104 @@
# --- Graphify extraction fragment ---
note_slug: pest-control-email-abc-hub
entities:
- name: pest-control-spring-2026
type: campaign
facet: project/pest-control-spring-2026
- name: Golden 4-Sentence Framework
type: email framework
facet: convention/golden-4-sentence-framework
- name: Revised
type: email version
- name: Copywriter
type: email version
- name: Draft
type: email version
- name: Mike Deluca
type: owner persona
- name: Sandra Kowalski
type: owner persona
- name: Ray Tanner
type: owner persona
- name: Driven Results
type: organization
- name: GorillaDesk
type: tool
facet: tool/gorillaDesk
- name: Two-scenario contrast format
type: copy pattern
facet: convention/two-scenario-contrast-format
- name: Saturday wasp nest scenario
type: email hook
- name: Scenario A vs. B two-format
type: email structure
facet: convention/scenario-a-vs-b-two-format
- name: lead disappearance problem reframe
type: copy pattern
facet: convention/lead-disappearance-reframe
- name: LSA-anchor version
type: email version
- name: $136,500 ROI projection
type: copy element
- name: review-shaming opener
type: copy pattern
- name: escape-hatch close
type: copy pattern
- name: cold email
type: domain
facet: domain/cold-email
relationships:
- source: Revised
type: won overall in
target: pest-control-spring-2026
- source: Copywriter
type: runner-up in
target: pest-control-spring-2026
- source: Draft
type: placed third in
target: pest-control-spring-2026
- source: Mike Deluca
type: judged as owner persona in
target: pest-control-spring-2026
- source: Sandra Kowalski
type: judged as owner persona in
target: pest-control-spring-2026
- source: Ray Tanner
type: judged as owner persona in
target: pest-control-spring-2026
- source: Saturday wasp nest scenario
type: achieved unanimous sweep in
target: pest-control-spring-2026
- source: Scenario A vs. B two-format
type: achieved unanimous sweep in
target: pest-control-spring-2026
- source: Two-scenario contrast format
type: earned identical reactions from all owner personas in
target: pest-control-spring-2026
- source: LSA-anchor version
type: requires audience segmentation for
target: pest-control-spring-2026
- source: $136,500 ROI projection
type: recommended to retire from
target: pest-control-spring-2026
- source: review-shaming opener
type: recommended to retire from
target: pest-control-spring-2026
- source: escape-hatch close
type: recommended to retire from
target: pest-control-spring-2026
- source: Revised
type: uses citation from
target: Driven Results
- source: lead disappearance problem reframe
type: appears in
target: Revised
confidence: INFERRED
- source: GorillaDesk
type: referenced in email version
target: Copywriter
confidence: INFERRED
- source: Golden 4-Sentence Framework
type: basis for
target: Draft

124
docs/memory-system/benchmark/reference-outputs/pest-control-enterprise-revenue.haiku.md Normal file
View File

@ -0,0 +1,124 @@
note_slug: pest-control-enterprise-revenue
entities:
- name: Gemini
type: Tool
facet: tool/gemini
- name: Pest Control Enterprise
type: Concept
facet: domain/pest-control
- name: Recurring Revenue
type: Concept
- name: Termite Swarming
type: Concept
facet: domain/pest-control
- name: Growing Degree Days
type: Convention
facet: convention/gdd-forecasting
- name: Heatwave-Driven Migrations
type: Concept
facet: domain/pest-control
- name: First Freeze Rodent Push
type: Concept
facet: domain/pest-control
- name: PPC
type: Tool
facet: tool/ppc
- name: SEO
type: Tool
facet: tool/seo
- name: Customer Acquisition Cost
type: Concept
- name: Route Density
type: Concept
facet: domain/pest-control
- name: Rollins
type: Client
- name: Rentokil
type: Client
- name: Ecolab
type: Client
- name: Terminix
type: Client
- name: Churn Rate
type: Concept
- name: Communication Failure Patterns
type: Concept
facet: domain/pest-control
- name: Northeast Region
type: Concept
facet: scope/geographic
- name: Mid-Atlantic Region
type: Concept
facet: scope/geographic
- name: Southeast Region
type: Concept
facet: scope/geographic
- name: Southwest Region
type: Concept
facet: scope/geographic
- name: Northwest Region
type: Concept
facet: scope/geographic
- name: Commercial Pest Control
type: Concept
facet: domain/pest-control
- name: Offshore Customer Service
type: Concept
confidence: INFERRED
- name: M&A Valuation
type: Concept
facet: domain/pest-control
relationships:
- source: Pest Control Enterprise
type: achieves recurring revenue through
target: Recurring Revenue
- source: Pest Control Enterprise
type: experiences lead generation spike during
target: Termite Swarming
- source: Growing Degree Days
type: predicts onset of
target: Termite Swarming
- source: Pest Control Enterprise
type: experiences demand spike during
target: Heatwave-Driven Migrations
- source: Pest Control Enterprise
type: experiences demand spike during
target: First Freeze Rodent Push
- source: PPC
type: captures demand during
target: Termite Swarming
- source: SEO
type: reduces long-term CAC for
target: Pest Control Enterprise
- source: Route Density
type: drives revenue per technician in
target: Pest Control Enterprise
- source: Rollins
type: consolidates market share in
target: Pest Control Enterprise
- source: Rentokil
type: consolidates market share in
target: Pest Control Enterprise
- source: Ecolab
type: consolidates market share in
target: Pest Control Enterprise
- source: Terminix
type: consolidates market share in
target: Pest Control Enterprise
- source: Communication Failure Patterns
type: drives churn in
target: Pest Control Enterprise
- source: Churn Rate
type: correlates with customer satisfaction in
target: Communication Failure Patterns
- source: Offshore Customer Service
type: contributes to
target: Communication Failure Patterns
- source: Commercial Pest Control
type: represents market segment of
target: Pest Control Enterprise
- source: Gemini
type: source for
target: Pest Control Enterprise

224
docs/memory-system/benchmark/reference-outputs/pest-control-enterprise-revenue.opus.md Normal file
View File

@ -0,0 +1,224 @@
# --- Graphify extraction fragment ---
note_slug: pest-control-enterprise-revenue
entities:
- name: Pest Control Enterprise
type: Domain
facet: domain/pest-control
- name: Recurring Revenue
type: Concept
- name: ACV
type: Concept
- name: Monthly maintenance
type: Concept
- name: Bi-monthly
type: Concept
- name: Tri-annual
type: Concept
- name: Quarterly
type: Concept
- name: Premium/Gold bundled
type: Concept
- name: Termite warranty renewal
type: Concept
- name: Initial service fee
type: Concept
- name: CAC
type: Concept
- name: LTV
type: Concept
- name: Termite monitoring/baiting
type: Concept
- name: Mosquito seasonal
type: Concept
- name: Rodent exclusion
type: Concept
- name: Wasp/hornet removal
type: Concept
- name: Bed bug remediation
type: Concept
- name: Communication Failure Patterns
type: Concept
- name: '"No one answered"'
type: Concept
- name: '"Voicemail full"'
type: Concept
- name: '"Dropped calls and ghosting"'
type: Concept
- name: Offshore agent disconnect
type: Concept
- name: CPL
type: Concept
- name: Churn
type: Concept
- name: Climatologically-Driven Lead Seasonality
type: Concept
- name: Termite Swarming
type: Concept
- name: Subterranean termites
type: Concept
- name: Growing Degree Days (GDD)
type: Concept
- name: PPC
type: Concept
- name: Heatwave-Driven Migrations
type: Concept
- name: First Freeze Rodent Push
type: Concept
- name: Northeast
type: Concept
- name: Mid-Atlantic
type: Concept
- name: Southeast
type: Concept
- name: Southwest
type: Concept
- name: Northwest
type: Concept
- name: Operational Unit Economics
type: Concept
- name: Revenue per technician
type: Concept
- name: COGS
type: Concept
- name: Direct labor
type: Concept
- name: Materials/chemicals
type: Concept
- name: Marketing/lead gen
type: Concept
- name: Vehicle ops
type: Concept
- name: Admin/management
type: Concept
- name: EBITDA
type: Concept
- name: SEO
type: Concept
- name: Marketing allocation model
type: Concept
- name: Commercial
type: Concept
- name: M&A
type: Concept
- name: Big Four
type: Concept
- name: Rollins
type: Client
- name: Rentokil
type: Client
- name: Ecolab
type: Client
- name: Terminix
type: Client
- name: Revenue multiples
type: Concept
- name: SDE multiples
type: Concept
- name: Route density
type: Concept
- name: Gemini
type: Tool
- name: SOPs
type: Concept
relationships:
- source: Recurring Revenue
type: comprises_share_of
target: Pest Control Enterprise
- source: Recurring Revenue
type: determines
target: M&A
- source: Initial service fee
type: offsets
target: CAC
- source: Initial service fee
type: inflates
target: ACV
- source: Termite warranty renewal
type: is_a
target: ACV
- source: Monthly maintenance
type: is_a
target: ACV
- source: Communication Failure Patterns
type: drives
target: Churn
- source: '"No one answered"'
type: wastes
target: CPL
- source: Offshore agent disconnect
type: misidentifies
target: Subterranean termites
- source: Termite Swarming
type: is_event_of
target: Climatologically-Driven Lead Seasonality
- source: Subterranean termites
type: cause
target: Termite Swarming
- source: Growing Degree Days (GDD)
type: predicts
target: Termite Swarming
- source: PPC
type: increases_with
target: Growing Degree Days (GDD)
- source: Heatwave-Driven Migrations
type: is_event_of
target: Climatologically-Driven Lead Seasonality
- source: First Freeze Rodent Push
type: is_event_of
target: Climatologically-Driven Lead Seasonality
- source: First Freeze Rodent Push
type: triggers
target: Rodent exclusion
- source: Northeast
type: experiences
target: Subterranean termites
- source: Mid-Atlantic
type: experiences
target: Subterranean termites
- source: Direct labor
type: component_of
target: COGS
- source: Materials/chemicals
type: component_of
target: COGS
- source: Marketing/lead gen
type: component_of
target: COGS
- source: Vehicle ops
type: component_of
target: COGS
- source: Admin/management
type: component_of
target: COGS
- source: Marketing allocation model
type: allocates_to
target: PPC
- source: Marketing allocation model
type: allocates_to
target: SEO
- source: Big Four
type: includes
target: Rollins
- source: Big Four
type: includes
target: Rentokil
- source: Big Four
type: includes
target: Ecolab
- source: Big Four
type: includes
target: Terminix
- source: Big Four
type: consolidates
target: Pest Control Enterprise
- source: SOPs
type: increases
target: SDE multiples
- source: Route density
type: drives
target: M&A
- source: Gemini
type: sourced
target: Pest Control Enterprise

206
docs/memory-system/benchmark/reference-outputs/pest-control-enterprise-revenue.sonnet.md Normal file
View File

@ -0,0 +1,206 @@
# --- Graphify extraction fragment ---
note_slug: pest-control-enterprise-revenue
entities:
- name: Pest Control Enterprise
type: business-segment
facet: domain/pest-control
- name: Recurring Revenue
type: financial-concept
facet: domain/pest-control
- name: Monthly Maintenance Program
type: service-program
facet: domain/pest-control
- name: Bi-monthly Program
type: service-program
facet: domain/pest-control
- name: Tri-annual Program
type: service-program
facet: domain/pest-control
- name: Quarterly Program
type: service-program
facet: domain/pest-control
- name: Premium/Gold Bundled Program
type: service-program
facet: domain/pest-control
- name: Termite Warranty Renewal
type: service-program
facet: domain/pest-control
- name: Initial Service Fee
type: pricing-mechanism
facet: domain/pest-control
- name: Termite Monitoring/Baiting
type: specialty-service
facet: domain/pest-control
- name: Mosquito Seasonal Service
type: specialty-service
facet: domain/pest-control
- name: Rodent Exclusion
type: specialty-service
facet: domain/pest-control
- name: Wasp/Hornet Removal
type: specialty-service
facet: domain/pest-control
- name: Bed Bug Remediation
type: specialty-service
facet: domain/pest-control
- name: Missed Call Failure
type: communication-failure-pattern
facet: domain/pest-control
- name: Voicemail Full Failure
type: communication-failure-pattern
facet: domain/pest-control
- name: Dropped Calls and Ghosting
type: communication-failure-pattern
facet: domain/pest-control
- name: Offshore Agent Disconnect
type: communication-failure-pattern
facet: domain/pest-control
- name: Annual Churn
type: operational-metric
facet: domain/pest-control
- name: Termite Swarming Season
type: seasonal-lead-trigger
facet: domain/pest-control
- name: Heatwave-Driven Migrations
type: seasonal-lead-trigger
facet: domain/pest-control
- name: First Freeze Rodent Push
type: seasonal-lead-trigger
facet: domain/pest-control
- name: Growing Degree Days
type: forecasting-concept
facet: domain/pest-control
- name: Revenue per Technician
type: operational-metric
facet: domain/pest-control
- name: COGS Structure
type: financial-concept
facet: domain/pest-control
- name: Marketing Allocation Model
type: operational-framework
facet: domain/pest-control
- name: Commercial Pest Control
type: business-segment
facet: domain/pest-control
- name: Rollins
type: company
facet: domain/pest-control
- name: Rentokil
type: company
facet: domain/pest-control
- name: Ecolab
type: company
facet: domain/pest-control
- name: Terminix
type: company
facet: domain/pest-control
- name: Big Four
type: company-group
facet: domain/pest-control
- name: M&A Valuation
type: financial-concept
facet: domain/pest-control
- name: SDE Multiple
type: financial-metric
facet: domain/pest-control
- name: Revenue Multiple
type: financial-metric
facet: domain/pest-control
- name: Route Density
type: operational-metric
facet: domain/pest-control
- name: Customer Retention
type: operational-metric
facet: domain/pest-control
relationships:
- source: Pest Control Enterprise
type: generates
target: Recurring Revenue
- source: Recurring Revenue
type: determines
target: M&A Valuation
- source: Monthly Maintenance Program
type: is-tier-of
target: Recurring Revenue
- source: Bi-monthly Program
type: is-tier-of
target: Recurring Revenue
- source: Tri-annual Program
type: is-tier-of
target: Recurring Revenue
- source: Quarterly Program
type: is-tier-of
target: Recurring Revenue
- source: Premium/Gold Bundled Program
type: is-tier-of
target: Recurring Revenue
- source: Termite Warranty Renewal
type: is-tier-of
target: Recurring Revenue
- source: Initial Service Fee
type: offsets-CAC-for
target: Pest Control Enterprise
- source: Missed Call Failure
type: increases
target: Annual Churn
confidence: INFERRED
- source: Voicemail Full Failure
type: increases
target: Annual Churn
confidence: INFERRED
- source: Dropped Calls and Ghosting
type: increases
target: Annual Churn
confidence: INFERRED
- source: Offshore Agent Disconnect
type: causes
target: Dropped Calls and Ghosting
confidence: INFERRED
- source: Termite Swarming Season
type: drives-demand-for
target: Termite Monitoring/Baiting
- source: Heatwave-Driven Migrations
type: drives-demand-for
target: Mosquito Seasonal Service
- source: First Freeze Rodent Push
type: drives-demand-for
target: Rodent Exclusion
- source: Growing Degree Days
type: used-to-predict
target: Termite Swarming Season
- source: Marketing Allocation Model
type: allocates-spend-toward
target: Termite Swarming Season
- source: Rollins
type: member-of
target: Big Four
- source: Rentokil
type: member-of
target: Big Four
- source: Ecolab
type: member-of
target: Big Four
- source: Terminix
type: member-of
target: Big Four
- source: Big Four
type: consolidates
target: Pest Control Enterprise
- source: M&A Valuation
type: measured-by
target: SDE Multiple
- source: M&A Valuation
type: measured-by
target: Revenue Multiple
- source: Route Density
type: premium-driver-for
target: M&A Valuation
- source: Customer Retention
type: premium-driver-for
target: M&A Valuation
- source: Commercial Pest Control
type: segment-of
target: Pest Control Enterprise
- source: Annual Churn
type: inversely-affects
target: Customer Retention

136
docs/memory-system/benchmark/reference-outputs/pest-control-sms-market-research-stats.haiku.md Normal file
View File

@ -0,0 +1,136 @@
note_slug: pest-control-sms-market-research-stats
entities:
- name: MIT/InsideSales.com Lead Response Management Study
type: research study
- name: Harvard Business Review
type: publication
facet: domain/sales-operations
- name: Velocify Ultimate Contact Strategy Study
type: research study
- name: InsideSales.com 2021
type: research study
- name: HomeAdvisor
type: company
facet: tool/homeadvisor
- name: Angi
type: company
facet: tool/angi
- name: CallRail
type: company
facet: tool/callrail
- name: PATLive
type: company
- name: Forbes
type: publication
- name: SellCell
type: publication
- name: Driven Results
type: research organization
- name: HubSpot
type: company
facet: tool/hubspot
- name: Google Ads
type: tool/google-ads
facet: tool/google-ads
- name: Google Local Services Ads
type: tool/google-local-services-ads
facet: tool/google-local-services-ads
- name: Thumbtack
type: company
facet: tool/thumbtack
- name: Coalmarch
type: company
- name: Valve+Meter
type: research organization
- name: Invoca
type: company
- name: PPMA
type: organization
- name: Scorpion
type: company
- name: NPMA
type: organization
- name: PCO Bookkeepers
type: organization
- name: Gartner
type: company
- name: D7 Networks
type: company
- name: CTIA
type: organization
- name: TransUnion
type: company
- name: Avochato
type: company
- name: Leads360
type: company
- name: Briostack
type: company
- name: Cube Creative Design
type: company
- name: ServiceTitan
type: company
facet: tool/servicetitan
- name: Point Loma Electric & Plumbing
type: company
- name: Hatch
type: company
- name: Shafer Services
type: company
- name: Aruza Pest Control
type: company
- name: After-Hours SMS Lead Capture
type: product concept
facet: project/after-hours-sms-lead-capture
- name: pest control industry
type: market segment
facet: domain/pest-control
relationships:
- source: MIT/InsideSales.com Lead Response Management Study
type: documents lead response time decay
target: After-Hours SMS Lead Capture
confidence: INFERRED
- source: Harvard Business Review
type: documents lead qualification speed
target: After-Hours SMS Lead Capture
confidence: INFERRED
- source: Velocify Ultimate Contact Strategy Study
type: documents conversion decay curve
target: After-Hours SMS Lead Capture
confidence: INFERRED
- source: HomeAdvisor
type: shows after-hours call volume in
target: pest control industry
- source: Angi
type: shows after-hours call volume in
target: pest control industry
- source: CallRail
type: documents call volume peaks in
target: pest control industry
- source: PATLive
type: measures caller abandonment in
target: pest control industry
- source: Driven Results
type: measured real-world conversion outcomes for
target: After-Hours SMS Lead Capture
confidence: INFERRED
- source: ServiceTitan
type: tracks adoption gap in
target: pest control industry
- source: NPMA
type: published gross margin data for
target: pest control industry
- source: Point Loma Electric & Plumbing
type: achieved booking rate boost with
target: After-Hours SMS Lead Capture
confidence: INFERRED
- source: Shafer Services
type: achieved lead booking increase with
target: After-Hours SMS Lead Capture
confidence: INFERRED
- source: Aruza Pest Control
type: achieved cost reduction with
target: After-Hours SMS Lead Capture
confidence: INFERRED

134
docs/memory-system/benchmark/reference-outputs/pest-control-sms-market-research-stats.opus.md Normal file
View File

@ -0,0 +1,134 @@
# --- Graphify extraction fragment ---
note_slug: pest-control-sms-market-research-stats
entities:
- name: After-Hours SMS Lead Capture & Qualification
type: product
facet: project/niche-automation-prospecting
- name: pest control companies
type: industry
facet: domain/pest-control
- name: niche-automation-prospecting
type: initiative
facet: project/niche-automation-prospecting
- name: pest-control-spring-2026
type: campaign
facet: project/pest-control-spring-2026
- name: MIT/InsideSales.com Lead Response Management Study (2007)
type: study
- name: Harvard Business Review (March 2011)
type: study
- name: Velocify "Ultimate Contact Strategy" Study (2012)
type: study
- name: InsideSales.com 2021
type: study
- name: HomeAdvisor
type: company
- name: Angi
type: company
- name: CallRail
type: company
- name: PATLive
type: company
- name: Forbes
type: publication
- name: SellCell 2024
type: study
- name: Driven Results
type: company
- name: HubSpot 2023
type: study
- name: Google Ads (PPC)
type: channel
facet: tool/google-ads
- name: Google Local Services Ads
type: channel
facet: tool/google-local-services-ads
- name: Thumbtack
type: company
- name: Coalmarch
type: company
- name: Valve+Meter
type: company
- name: Invoca
type: company
- name: NPMA & PCO Bookkeepers 2025 Cost Study
type: study
- name: PPMA survey
type: study
- name: Scorpion
type: company
- name: Gartner 2016
type: study
- name: D7 Networks
type: company
- name: CTIA
type: organization
- name: TransUnion
type: company
- name: Avochato
type: company
- name: Velocify/Leads360
type: study
- name: Briostack
type: company
- name: Cube Creative Design
type: company
- name: ServiceTitan 2025
type: study
- name: Point Loma Electric & Plumbing
type: company
- name: Hatch
type: company
- name: Shafer Services
type: company
- name: Aruza Pest Control
type: company
- name: SMS
type: channel
facet: domain/sms
- name: NPMA
type: organization
- name: ServiceTitan
type: company
relationships:
- source: After-Hours SMS Lead Capture & Qualification
type: is sold to
target: pest control companies
- source: After-Hours SMS Lead Capture & Qualification
type: uses channel
target: SMS
- source: MIT/InsideSales.com Lead Response Management Study (2007)
type: measured conversion decay for
target: pest control companies
confidence: INFERRED
- source: Velocify/Leads360
type: found texted prospects convert 40% higher for
target: SMS
confidence: INFERRED
- source: Aruza Pest Control
type: is a customer outcome case studied by
target: Cube Creative Design
- source: Point Loma Electric & Plumbing
type: is a case study by
target: Hatch
- source: Shafer Services
type: is a case study by
target: Hatch
- source: NPMA & PCO Bookkeepers 2025 Cost Study
type: reports gross margin for
target: pest control companies
- source: Coalmarch
type: provides cost-per-lead average for
target: pest control companies
- source: ServiceTitan 2025
type: surveyed
target: ServiceTitan
confidence: AMBIGUOUS
- source: Briostack
type: reports automation adoption gap for
target: pest control companies
- source: Gartner 2016
type: reports open and response rates for
target: SMS

142
docs/memory-system/benchmark/reference-outputs/pest-control-sms-market-research-stats.sonnet.md Normal file
View File

@ -0,0 +1,142 @@
# --- Graphify extraction fragment ---
note_slug: pest-control-sms-market-research-stats
entities:
- name: MIT/InsideSales.com Lead Response Management Study
type: research-study
facet: domain/lead-response
- name: Harvard Business Review Lead Response Study
type: research-study
facet: domain/lead-response
- name: Velocify Ultimate Contact Strategy Study
type: research-study
facet: domain/lead-response
- name: InsideSales.com 2021 Study
type: research-study
facet: domain/lead-response
- name: HomeAdvisor
type: organization
- name: Angi
type: organization
- name: CallRail
type: organization
- name: PATLive
type: organization
- name: Driven Results
type: organization
- name: HubSpot
type: organization
- name: Valve+Meter
type: organization
- name: Invoca
type: organization
- name: Coalmarch
type: organization
- name: NPMA
type: organization
- name: PCO Bookkeepers
type: organization
- name: PPMA
type: organization
- name: Scorpion
type: organization
- name: Briostack
type: organization
- name: Cube Creative Design
type: organization
- name: ServiceTitan
type: organization
- name: Gartner
type: organization
- name: TransUnion
type: organization
- name: Avochato
type: organization
- name: Hatch
type: organization
- name: Google Ads
type: tool
facet: tool/paid-search
- name: Google Local Services Ads
type: tool
facet: tool/paid-search
- name: Thumbtack
type: tool
facet: tool/lead-marketplace
- name: Point Loma Electric & Plumbing
type: organization
- name: Shafer Services
type: organization
- name: Aruza Pest Control
type: organization
- name: After-Hours SMS Lead Capture & Qualification
type: product-concept
facet: project/niche-automation-prospecting
- name: pest control
type: industry
facet: domain/pest-control
- name: home services
type: industry
facet: domain/field-service
- name: lead response time
type: concept
facet: domain/lead-response
- name: SMS marketing
type: concept
facet: domain/lead-capture
- name: lead conversion
type: concept
facet: domain/lead-response
- name: after-hours call handling
type: concept
facet: domain/field-service
- name: niche-automation-prospecting
type: project
facet: project/niche-automation-prospecting
relationships:
- source: MIT/InsideSales.com Lead Response Management Study
type: measures decay of
target: lead conversion
- source: Harvard Business Review Lead Response Study
type: measures decay of
target: lead conversion
- source: Velocify Ultimate Contact Strategy Study
type: measures decay of
target: lead conversion
- source: InsideSales.com 2021 Study
type: measures decay of
target: lead conversion
- source: After-Hours SMS Lead Capture & Qualification
type: targets industry
target: pest control
- source: After-Hours SMS Lead Capture & Qualification
type: addresses problem of
target: after-hours call handling
- source: After-Hours SMS Lead Capture & Qualification
type: leverages
target: SMS marketing
- source: Aruza Pest Control
type: is customer in case study by
target: Cube Creative Design
- source: Point Loma Electric & Plumbing
type: is customer in case study by
target: Hatch
- source: Shafer Services
type: is customer in case study by
target: Hatch
- source: Driven Results
type: sourced after-hours data for
target: home services
- source: Valve+Meter
type: conducted secret-shopper study of
target: home services
- source: NPMA
type: co-published cost study with
target: PCO Bookkeepers
- source: ServiceTitan
type: published industry survey on
target: home services
- source: niche-automation-prospecting
type: contains research for
target: After-Hours SMS Lead Capture & Qualification

31
docs/memory-system/benchmark/run-findings-2026-06-04.md Normal file
View File

@ -0,0 +1,31 @@
# Benchmark Run Findings — 2026-06-04 (Step 2c reference set)
_Run: 2026-06-04 — reference set generated, verified, gate satisfied._
This document records the outcomes, surprises, and lessons from the Step 2c reference-extraction benchmark run. It is the authoritative findings record for this run date and serves as input to future benchmark comparisons when the Graphify/Ollama production pipeline is evaluated.
## What was run
- 6 cross-domain fixture notes hand-selected from ~/Documents/SecondBrain (4 pest-control / niche-automation-prospecting, 2 software-design), covering all five fixture categories (tool, client/project, convention, domain, relationship-dense). Fixtures are listed in dispatch-prompt.md.
- Each note extracted by 3 Claude tiers (claude-haiku-4-5, claude-sonnet-4-6, claude-opus-4-8) via subagent dispatch with model overrides = 18 reference fragments in reference-outputs/.
- Notes were run AS-IS (no vault frontmatter modification); the SecondBrain vault was backed up first to ~/Documents/SecondBrain-backup-20260604-114749.
- The .opus.md files are the gold-standard scoring rubric; haiku/sonnet are kept to characterize the tier quality gradient.
## Key finding: fairness-contract contamination (the gate working as designed)
- The first-pass haiku extraction of the `ai-coding-conventions-synthesis` note leaked three entities — `Graphify`, `Milvus Lite`, `SecondBrain vault` — that do NOT appear in the note text. They were injected from the cc-os CLAUDE.md / project context that Claude Code auto-loads into subagents. Sonnet and opus did not leak on the same note.
- Lesson 1: the soft "ignore any injected context" instruction in the fairness contract is NOT reliable on the haiku tier — re-running clean required an explicit forbidden-class hint (do not introduce knowledge-graph/vector-db/vault tooling unless it appears in the note text).
- Lesson 2 (important, prevents a wrong conclusion): this is largely a HARNESS ARTIFACT, not a property of the production pipeline. In the real Graphify→Ollama pipeline, the SLM receives only the prompt Graphify constructs — there is no CLAUDE.md injection path. So the benchmark's contamination exposure is HIGHER than production; the cleaned reference set is conservative. Do NOT conclude "haiku extracts poorly," and do NOT carry the anti-contamination hint into the production Graphify extraction prompt — it is unneeded there.
## Spec gaps found and fixed
- The original extraction-spec.md did not require a sub-value after a facet namespace, producing degenerate bare-slash facets (`domain/`, `tool/`) in several sonnet/haiku outputs. It also did not state strict referential integrity, producing relationships pointing at undeclared entities.
- Both rules were added to extraction-spec.md on 2026-06-04, and the affected files regenerated so the whole 18-file set conforms to the final spec.
## Tier richness gradient (observed)
- Opus is reliably the richest extractor (e.g. the enterprise-revenue note: opus ~59-61 entities vs haiku ~25). Sonnet is sometimes terser than haiku. On stat-dense notes all three tiers saturate to similar counts. Treat entity/relationship counts as indicative, not exact (extractor self-counts diverged from verified counts).
## Status
- All 18 files verified: valid YAML, conformant schema, referential integrity, no contamination. Step 2c gate satisfied 2026-06-04.