jared
/
cc-os
1
0
Fork 0
Code Issues 18 Pull Requests Packages Projects Releases Wiki Activity

Record post-reboot GPU timing and mark deferred #3 resolved 

GPU is fully resident and functional post-reboot (~74 tok/s steady-state).
Driver mismatch is resolved; gemma4:e4b confirmed at/above section C's
estimate. Deferred follow-up #3 (Confirm GPU timing post-reboot) marked
RESOLVED.
This commit is contained in:
jared 2026-06-04 14:52:21 -04:00
parent 65595c3995
commit 3823bb7c91
1 changed files with 24 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
docs/memory-system/benchmark/local-llm-findings-2026-06-04.md
View File

@ -1,6 +1,6 @@
# Local-LLM Doc-Extraction Gut-Check — 2026-06-04 # Local-LLM Doc-Extraction Gut-Check — 2026-06-04
_Last updated: 2026-06-04_ | _Status: gut-check complete; gemma4:e4b is the candidate; GPU fix pending reboot; Graphify validation is the next step._ _Last updated: 2026-06-04_ | _Status: gut-check complete; gemma4:e4b is the candidate; GPU confirmed post-reboot (~74 tok/s, see appendix); Graphify validation is the next step._
This document consolidates the local-LLM doc-extraction gut-check run on 2026-06-04. It is This document consolidates the local-LLM doc-extraction gut-check run on 2026-06-04. It is
the authoritative findings record for that session and feeds forward into the Graphify-ollama the authoritative findings record for that session and feeds forward into the Graphify-ollama
@ -232,6 +232,26 @@ time curl -s http://127.0.0.1:11434/api/generate \
This measures raw model tok/s via the hand-rolled harness — it does NOT measure Graphify's This measures raw model tok/s via the hand-rolled harness — it does NOT measure Graphify's
production extraction performance. production extraction performance.
#### Measured results (run 2026-06-04, post-reboot)
GPU is now fully active and resident — the driver mismatch diagnosed in section C is
**RESOLVED**. `nvidia-smi` is healthy (Driver 580.159.03, RTX 3060, no NVML error). The
`gemma4:e4b` model loaded fully GPU-resident: `/api/ps` reports `size_vram` 3.29 GB ==
`size` 3.29 GB (no CPU fallback), with 4462 MiB used on the card.
| Pass | eval tok/s | eval_count | total wall (s) | load (s) | prompt_eval (s) | prompt_eval_count |
|---|---|---|---|---|---|---|
| Cold | 74.27 | 701 | 76.04 | 38.15 | 28.44 | 905 |
| Warm | 74.25 | 717 | 10.13 | 0.43 | 0.04 | — |
Conclusion: steady-state generation is **~74 tok/s** — at or above the section C estimate of
40–60 tok/s. Warm extraction lands at **~10 s/note**; the cold pass is ~76 s, dominated by the
one-time 38 s model load plus a 28 s cold prompt-eval. This confirms section C's "well under a
minute once the model is resident" prediction.
> **Caveat retained:** these are raw model tok/s from the hand-rolled curl harness, NOT
> Graphify production extraction performance (see Deferred follow-up §1).
--- ---
## Deferred follow-ups ## Deferred follow-ups
@ -246,8 +266,9 @@ production extraction performance.
after `uv tool install "graphifyy[ollama]"`, read the installed client source under the after `uv tool install "graphifyy[ollama]"`, read the installed client source under the
tool's site-packages. This does not require the GPU — a cheap deferred follow-up. tool's site-packages. This does not require the GPU — a cheap deferred follow-up.
3. **Confirm GPU timing post-reboot** using the curl command in the appendix above. The 3. **Confirm GPU timing post-reboot** — **RESOLVED (2026-06-04).** GPU re-timing ran post-reboot;
~40–60 tok/s estimate in section C is currently unverified. measured ~74 tok/s steady-state (at/above the section C estimate). See the "Measured results"
table under the Post-reboot GPU re-timing appendix above.
4. **Raw per-model benchmark outputs from this run** were throwaway scratch (wrong harness, 4. **Raw per-model benchmark outputs from this run** were throwaway scratch (wrong harness,
hand-rolled prompt) and are not committed. hand-rolled prompt) and are not committed.