# Spec: local-model-selection

## Purpose

Install Graphify, configure Ollama for extraction (flash attention, context window), score candidate models against Claude-Opus references (entity correctness, relationship typing, confidence-tag accuracy, speed), and select the extraction model by evidence. As of 2026-06-04, `qwen2.5-coder:7b` is the selected model.

## Requirements

### Requirement: Extraction toolchain is installed and verified

The Graphify CLI SHALL be installed from the `graphifyy` PyPI package and verified to run before any extraction is attempted, and a running Ollama with at least one pulled candidate model SHALL be available.

#### Scenario: Graphify is callable

- **WHEN** the toolchain setup completes
- **THEN** `graphify --version` returns a version without error
- **AND** at least one candidate Ollama model is pulled and listed by `ollama list`

### Requirement: Ollama runtime is configured for extraction

Ollama SHALL be configured with the settings the extraction run depends on: flash attention enabled (`OLLAMA_FLASH_ATTENTION=1`) and a context window sufficient for vault notes (`GRAPHIFY_OLLAMA_NUM_CTX=8192`), and the allocated context SHALL be verified after the first extraction call.

#### Scenario: Configuration is in effect

- **WHEN** the first extraction call is made
- **THEN** flash attention is enabled and the context size is 8192
- **AND** `ollama ps` shows the expected allocated context for the loaded model

### Requirement: Candidate models are scored against the gold-standard reference set

The extraction model SHALL be selected by scoring candidate Ollama models against the existing Step 2c reference set (the 18 fragments in `docs/memory-system/benchmark/reference-outputs/`) over the same 6 fixture notes. Each candidate's Graphify-shaped output SHALL be compared to the `claude-opus-4-8` gold-standard output for entity correctness, relationship plausibility and typing, and `INFERRED`/`AMBIGUOUS` confidence-tag accuracy, and wall-clock extraction speed SHALL be measured per candidate.

#### Scenario: Each candidate is scored on quality and speed

- **WHEN** a candidate model is run over the 6 fixtures
- **THEN** its output is scored against the Opus reference on entity correctness, relationship typing, and confidence-tag accuracy
- **AND** its wall-clock extraction speed is recorded

#### Scenario: Reference benchmark is consumed, not re-created

- **WHEN** scoring is performed
- **THEN** it reads the existing reference fragments produced by the `reference-extraction-benchmark` capability
- **AND** it does not regenerate or modify the Claude reference set

### Requirement: Model is selected by evidence, not hardcoded

The chosen extraction model SHALL be the one justified by the scoring run's recorded results, and no model SHALL be hardcoded as the selection before scoring completes. `gemma4:e4b` MAY be the front-runner candidate, but its selection SHALL depend on its scored quality, not its feasibility gut-check alone.

#### Scenario: Selection records its rationale

- **WHEN** a model is selected
- **THEN** a result artifact records the chosen model, its quality scores against the Opus rubric, and its measured speed
- **AND** the rationale references the scoring evidence rather than asserting a pre-chosen model

#### Scenario: No model is locked before scoring

- **WHEN** scoring has not yet run
- **THEN** no model is committed as the selection
- **AND** the front-runner candidate is treated as unconfirmed until scored
