Add new doc docs/memory-system/benchmark/local-llm-findings-2026-06-04.md
consolidating the local-LLM doc-extraction gut-check run. Key findings:
gemma4:e4b is the best installed model and adequate for the extraction
role (below frontier haiku but acceptable, catching ~45–60% of Opus's
high-value entities); GPU went unused due to NVIDIA driver version
mismatch (fix = reboot, pending); and Graphify owns the ollama call
(HTTP API, prompt/chunking/context/parsing internal) so raw-ollama
tuning is not production config. Add progressive-disclosure pointers
from 04-build-plan.md and 05-implementation-process.md.
Completed the benchmark run that gates Ollama model selection. Generated 18
reference fragments (6 cross-domain fixtures × 3 Claude tiers) and tightened
extraction spec rules based on first-run learnings. Updated implementation
status to reflect that reference set is complete and Ollama scoring is
unblocked. Populated fixture list in dispatch-prompt with final selection.
Invert the build order to build-first / migrate-incrementally and
redesign Step 2c as a Claude reference-set benchmark.
- ADR-013 records the build-order inversion; CLAUDE.md locked-decisions
pointer updated
- New benchmark deliverable under docs/memory-system/benchmark/:
shared extraction spec + copy/paste dispatch prompt + reference-outputs
- Runbook (05): Step 1 = fixture selection with bulk migration deferred;
Step 2c = Claude-tiers-only reference set, quality-only metrics, Opus
as gold-standard rubric (produces references, not a model choice)
- 04-build-plan reconciled to the new build order and benchmark design
- Sync incremental-migration + reference-extraction-benchmark specs into
openspec/specs/; archive the completed change
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Update CLAUDE.md: clarify the knowledge layer now specifies the
~/Documents/SecondBrain vault, six-facet taxonomy, and reference the
locked decisions (ADR-011, ADR-012).
Update 02-system-design.md: align with faceted taxonomy (type/
client/project/domain/tool/convention/ plus scope/), express hierarchy
via hubs + wikilinks + Graphify edges rather than nested paths, name
the vault explicitly, update build-plan timestamp.
Add 05-implementation-process.md: concrete build runbook integrating the
locked decisions — seven steps bottom-up from vault migration through
plugin packaging, with the Graphify + Ollama model benchmark (Step 2c)
marked as the critical gate. Open questions deferred or defaulted; most
are non-blocking.
Design milestone: tag taxonomy and vault location locked.
jared