diff --git a/docs/memory-system/benchmark/local-llm-findings-2026-06-04.md b/docs/memory-system/benchmark/local-llm-findings-2026-06-04.md index 31a93ad..f628880 100644 --- a/docs/memory-system/benchmark/local-llm-findings-2026-06-04.md +++ b/docs/memory-system/benchmark/local-llm-findings-2026-06-04.md @@ -1,6 +1,6 @@ # Local-LLM Doc-Extraction Gut-Check — 2026-06-04 -_Last updated: 2026-06-04_ | _Status: gut-check complete; gemma4:e4b is the candidate; GPU fix pending reboot; Graphify validation is the next step._ +_Last updated: 2026-06-04_ | _Status: gut-check complete; gemma4:e4b is the candidate; GPU confirmed post-reboot (~74 tok/s, see appendix); Graphify validation is the next step._ This document consolidates the local-LLM doc-extraction gut-check run on 2026-06-04. It is the authoritative findings record for that session and feeds forward into the Graphify-ollama @@ -232,6 +232,26 @@ time curl -s http://127.0.0.1:11434/api/generate \ This measures raw model tok/s via the hand-rolled harness — it does NOT measure Graphify's production extraction performance. +#### Measured results (run 2026-06-04, post-reboot) + +GPU is now fully active and resident — the driver mismatch diagnosed in section C is +**RESOLVED**. `nvidia-smi` is healthy (Driver 580.159.03, RTX 3060, no NVML error). The +`gemma4:e4b` model loaded fully GPU-resident: `/api/ps` reports `size_vram` 3.29 GB == +`size` 3.29 GB (no CPU fallback), with 4462 MiB used on the card. + +| Pass | eval tok/s | eval_count | total wall (s) | load (s) | prompt_eval (s) | prompt_eval_count | +|---|---|---|---|---|---|---| +| Cold | 74.27 | 701 | 76.04 | 38.15 | 28.44 | 905 | +| Warm | 74.25 | 717 | 10.13 | 0.43 | 0.04 | — | + +Conclusion: steady-state generation is **~74 tok/s** — at or above the section C estimate of +40–60 tok/s. Warm extraction lands at **~10 s/note**; the cold pass is ~76 s, dominated by the +one-time 38 s model load plus a 28 s cold prompt-eval. This confirms section C's "well under a +minute once the model is resident" prediction. + +> **Caveat retained:** these are raw model tok/s from the hand-rolled curl harness, NOT +> Graphify production extraction performance (see Deferred follow-up §1). + --- ## Deferred follow-ups @@ -246,8 +266,9 @@ production extraction performance. after `uv tool install "graphifyy[ollama]"`, read the installed client source under the tool's site-packages. This does not require the GPU — a cheap deferred follow-up. -3. **Confirm GPU timing post-reboot** using the curl command in the appendix above. The - ~40–60 tok/s estimate in section C is currently unverified. +3. **Confirm GPU timing post-reboot** — **RESOLVED (2026-06-04).** GPU re-timing ran post-reboot; + measured ~74 tok/s steady-state (at/above the section C estimate). See the "Measured results" + table under the Post-reboot GPU re-timing appendix above. 4. **Raw per-model benchmark outputs from this run** were throwaway scratch (wrong harness, hand-rolled prompt) and are not committed.