Archive graphify-ollama-setup and sync new main specs
Move the completed graphify-ollama-setup change to changes/archive/2026-06-05-graphify-ollama-setup/ and promote its two delta specs into the stable spec set: local-model-selection and vault-graph-build. Verification passed with no critical issues; 16/16 tasks done. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
3e38398500
commit
6a5b09f253
|
|
@ -0,0 +1,59 @@
|
||||||
|
# Spec: local-model-selection
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Install Graphify, configure Ollama for extraction (flash attention, context window), score candidate models against Claude-Opus references (entity correctness, relationship typing, confidence-tag accuracy, speed), and select the extraction model by evidence. As of 2026-06-04, `qwen2.5-coder:7b` is the selected model.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
### Requirement: Extraction toolchain is installed and verified
|
||||||
|
|
||||||
|
The Graphify CLI SHALL be installed from the `graphifyy` PyPI package and verified to run before any extraction is attempted, and a running Ollama with at least one pulled candidate model SHALL be available.
|
||||||
|
|
||||||
|
#### Scenario: Graphify is callable
|
||||||
|
|
||||||
|
- **WHEN** the toolchain setup completes
|
||||||
|
- **THEN** `graphify --version` returns a version without error
|
||||||
|
- **AND** at least one candidate Ollama model is pulled and listed by `ollama list`
|
||||||
|
|
||||||
|
### Requirement: Ollama runtime is configured for extraction
|
||||||
|
|
||||||
|
Ollama SHALL be configured with the settings the extraction run depends on: flash attention enabled (`OLLAMA_FLASH_ATTENTION=1`) and a context window sufficient for vault notes (`GRAPHIFY_OLLAMA_NUM_CTX=8192`), and the allocated context SHALL be verified after the first extraction call.
|
||||||
|
|
||||||
|
#### Scenario: Configuration is in effect
|
||||||
|
|
||||||
|
- **WHEN** the first extraction call is made
|
||||||
|
- **THEN** flash attention is enabled and the context size is 8192
|
||||||
|
- **AND** `ollama ps` shows the expected allocated context for the loaded model
|
||||||
|
|
||||||
|
### Requirement: Candidate models are scored against the gold-standard reference set
|
||||||
|
|
||||||
|
The extraction model SHALL be selected by scoring candidate Ollama models against the existing Step 2c reference set (the 18 fragments in `docs/memory-system/benchmark/reference-outputs/`) over the same 6 fixture notes. Each candidate's Graphify-shaped output SHALL be compared to the `claude-opus-4-8` gold-standard output for entity correctness, relationship plausibility and typing, and `INFERRED`/`AMBIGUOUS` confidence-tag accuracy, and wall-clock extraction speed SHALL be measured per candidate.
|
||||||
|
|
||||||
|
#### Scenario: Each candidate is scored on quality and speed
|
||||||
|
|
||||||
|
- **WHEN** a candidate model is run over the 6 fixtures
|
||||||
|
- **THEN** its output is scored against the Opus reference on entity correctness, relationship typing, and confidence-tag accuracy
|
||||||
|
- **AND** its wall-clock extraction speed is recorded
|
||||||
|
|
||||||
|
#### Scenario: Reference benchmark is consumed, not re-created
|
||||||
|
|
||||||
|
- **WHEN** scoring is performed
|
||||||
|
- **THEN** it reads the existing reference fragments produced by the `reference-extraction-benchmark` capability
|
||||||
|
- **AND** it does not regenerate or modify the Claude reference set
|
||||||
|
|
||||||
|
### Requirement: Model is selected by evidence, not hardcoded
|
||||||
|
|
||||||
|
The chosen extraction model SHALL be the one justified by the scoring run's recorded results, and no model SHALL be hardcoded as the selection before scoring completes. `gemma4:e4b` MAY be the front-runner candidate, but its selection SHALL depend on its scored quality, not its feasibility gut-check alone.
|
||||||
|
|
||||||
|
#### Scenario: Selection records its rationale
|
||||||
|
|
||||||
|
- **WHEN** a model is selected
|
||||||
|
- **THEN** a result artifact records the chosen model, its quality scores against the Opus rubric, and its measured speed
|
||||||
|
- **AND** the rationale references the scoring evidence rather than asserting a pre-chosen model
|
||||||
|
|
||||||
|
#### Scenario: No model is locked before scoring
|
||||||
|
|
||||||
|
- **WHEN** scoring has not yet run
|
||||||
|
- **THEN** no model is committed as the selection
|
||||||
|
- **AND** the front-runner candidate is treated as unconfirmed until scored
|
||||||
|
|
@ -0,0 +1,43 @@
|
||||||
|
# Spec: vault-graph-build
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Build the initial knowledge graph over the `~/Documents/SecondBrain` vault using the selected Ollama model via Graphify; verify god-node plausibility; treat graph artifacts as disposable/rebuildable, not source-of-truth (ADR-008).
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
### Requirement: Initial vault graph is built with the selected model
|
||||||
|
|
||||||
|
The initial knowledge graph SHALL be built over the `~/Documents/SecondBrain` vault using the model selected by the `local-model-selection` capability, via Graphify's Ollama document-extraction backend, and SHALL NOT begin until a model has been selected.
|
||||||
|
|
||||||
|
#### Scenario: Build uses the selected model
|
||||||
|
|
||||||
|
- **WHEN** the initial vault graph is built
|
||||||
|
- **THEN** Graphify extracts over `~/Documents/SecondBrain` using the selected Ollama model
|
||||||
|
- **AND** the build does not run before model selection is complete
|
||||||
|
|
||||||
|
#### Scenario: Build scope is the vault only
|
||||||
|
|
||||||
|
- **WHEN** the initial graph is built
|
||||||
|
- **THEN** the extraction target is the vault, not any project code repository
|
||||||
|
- **AND** per-project code graphs are not built as part of this change
|
||||||
|
|
||||||
|
### Requirement: Graph sanity is verified via god-nodes
|
||||||
|
|
||||||
|
After the build, `GRAPH_REPORT.md` SHALL be reviewed to confirm the most-connected nodes (god-nodes) are the tools, clients, and domain concepts that the vault's content would predict, so an obviously wrong extraction is caught before downstream steps depend on it.
|
||||||
|
|
||||||
|
#### Scenario: God-nodes match vault content
|
||||||
|
|
||||||
|
- **WHEN** `GRAPH_REPORT.md` is reviewed after the build
|
||||||
|
- **THEN** the top god-nodes are recognizable high-traffic tools, clients, or domains from the vault
|
||||||
|
- **AND** an implausible god-node distribution is flagged rather than accepted
|
||||||
|
|
||||||
|
### Requirement: Graph artifacts are treated as disposable
|
||||||
|
|
||||||
|
The produced graph artifacts (`graphify-out/`, `GRAPH_REPORT.md`, any index caches) SHALL be treated as rebuildable from the markdown vault and SHALL NOT be synced as source of truth, consistent with the markdown-as-truth decision (ADR-008).
|
||||||
|
|
||||||
|
#### Scenario: Graph output is not the source of truth
|
||||||
|
|
||||||
|
- **WHEN** the graph artifacts are produced
|
||||||
|
- **THEN** they are rebuildable from the vault markdown
|
||||||
|
- **AND** they are excluded from vault sync rather than treated as authoritative
|
||||||
Loading…
Reference in New Issue