Switch the os-vault rebuild hook to Unsloth 9B V4 on llama.cpp #1576

Closed
opened 2026-11-05 18:31:40 +00:00 by jared · 7 comments
Owner

Problem

The os-vault rebuild hook runs Ollama qwen3.5:9b. Jared decided that Unsloth Qwen3.5-9B on llama.cpp is the default (ruling 23: config V4). A rebuild with the old model replaces the re-extraction results. The hook starts this rebuild when a stamp is older than 7 days. The hook also starts it in each session when no stamp and no lock exist.

Observed

plugins/os-vault/hooks/session_start.py hardcodes --backend ollama. On 2026-11-05 at 12:53 the hook started a rebuild on Ollama, and session 47ffa777 stopped it. The rebuild gate regex misses the output still hollow and still truncated. #1556 covers the gate of the eval, with no numeric limit.

Reproduce

n/a: a rebuild takes over one hour on the GPU.

Approach

The hook runs the V4 config on llama.cpp and re-extracts only changed notes. llm-stack has no CLI and no process control, so os-vault owns a small readiness script and the router unit file.

Required work:

  1. plugins/os-vault/config.yaml is the one place that names the choice. Replace ollama_model with graphify_backend: llamacpp-qwen35, graphify_model: qwen3.5-9b (the id graphify sends to the router), and llm_model: qwen3-5-9b (the llm-stack catalog name). Remove GRAPHIFY_OLLAMA_NUM_CTX, GRAPHIFY_OLLAMA_KEEP_ALIVE, and OLLAMA_BASE_URL. Set GRAPHIFY_MAX_OUTPUT_TOKENS=10000, GRAPHIFY_MAX_RETRY_DEPTH=1, GRAPHIFY_MAX_RETRIES=0, GRAPHIFY_API_TIMEOUT=900, LLAMACPP_KEY=local. Keep GRAPHIFY_DISABLE_THINKING. check: present llamacpp-qwen35 in plugins/os-vault/config.yaml
  2. hooks/config.py reads the three new keys and loses its fallback model copy. A missing key loads as an empty value; main starts no rebuild and logs the reason. check: absent qwen3.5:9b in plugins/os-vault/hooks/config.py
  3. Add the router unit and preset as source. The service command and the V4 preset are in llm-stack docs/research/llamacpp-router-spike.md. The unit runs llama serve --host 127.0.0.1 --port 8087 --models-preset <preset> --models-max 1 --sleep-idle-seconds 30. The hook does not start or stop the service. check: present --sleep-idle-seconds in plugins/os-vault/llama-router/llama-router.service
  4. Add the preset file next to the unit, with the model path /home/jared/.cache/huggingface/hub/models--unsloth--Qwen3.5-9B-GGUF/snapshots/3885219b6810b007914f3a7950a8d1b469d598a5/Qwen3.5-9B-UD-Q4_K_XL.gguf. check: present qwen3.5-9b in plugins/os-vault/llama-router/preset.ini
  5. Add plugins/os-vault/bin/vault-model-ready, a Ruby script, and plugins/os-vault/Gemfile with gem 'llm-stack' by path in the same form as plugins/os-adr/Gemfile. It takes the catalog name, calls LLM.models.find(name, provider: "llamacpp"), exits 0 when ready? is true, and otherwise prints reason and exits 1. _REBUILD_SCRIPT calls it as ruby "$READY_SCRIPT" "$LLM_MODEL", so a test can place a ruby shim on PATH. check: present LLM.models.find in plugins/os-vault/bin/vault-model-ready
  6. Add plugins/os-vault/hooks/changed_notes.py, run with $GRAPHIFY_PYTHON as stub_markdown_pages.py is. It calls graphify.detect.detect_incremental(root, manifest_path=<root>/graphify-out/manifest.json, kind="semantic"). The default manifest path is relative to the working directory, so the script must pass the path. Exit 0 when new_total is above 0, exit 1 when it is 0, and any other exit on error. check: present detect_incremental in plugins/os-vault/hooks/changed_notes.py
  7. _REBUILD_SCRIPT runs changed_notes.py first. With changes: run vault-model-ready; if it fails, the run fails. Then run graphify extract "$VAULT_PATH" --backend "$BACKEND" --model "$MODEL" --token-budget 4000 --max-concurrency 1 --allow-partial, then graphify cluster-only with the same --backend and --model. With no changes: skip readiness, extract, and cluster-only. In both cases run the stub step and the two graphify update --force runs. Extract stays incremental and never uses --force. check: present --allow-partial in plugins/os-vault/hooks/session_start.py
  8. Add still hollow and still truncated to the failure check next to chunk N/M failed. check: present still truncated in plugins/os-vault/hooks/session_start.py
  9. Update stale names. check: present llamacpp-qwen35 in docs/implementation-status/os-vault.md
  10. In the README, describe the llama.cpp setup and remove the Ollama settings. check: absent GRAPHIFY_OLLAMA_NUM_CTX in plugins/os-vault/README.md
  11. Update the fixture config. check: absent ollama_model in plugins/os-vault/tests/harness.sh
  12. Update the fixture config. check: absent ollama_model in plugins/os-vault/tests/generate-fixtures.sh

Acceptance criteria

  1. Given one changed note and a ready model, when the rebuild runs, then graphify extract receives --backend llamacpp-qwen35 --model qwen3.5-9b --allow-partial, and no argument names ollama.
  2. Given no changed notes, when the rebuild runs, then the readiness script, extract, and cluster-only do not run, and the stub step and both update runs do.
  3. Given run output that contains still hollow or still truncated, when the rebuild ends, then it writes no stamp and it writes the failed marker.
  4. Given a changed note and a readiness script that exits 1, when the rebuild runs, then extract does not run, it writes no stamp, and it writes the failed marker.
  5. Given a config with no graphify_model, when the hook runs, then it starts no rebuild and logs the reason.
  6. The rebuild script passes no --force to extract.

After merge

The orchestrator does these by hand, because they sit outside the repo. They are not gates of this pipeline.

  • Install the unit under ~/.config/systemd/user, enable it, and confirm that port 8087 answers.
  • Set base_url in ~/.graphify/providers.json to port 8087, because 8089 was the eval logging proxy. Update ~/Documents/SecondBrain/howto/running-qwen35-9b-gguf-behind-graphify-with-llama-cpp.md to match.
  • With Jared's consent and a stated run time, prove a one-note change: V4 runs through llama.cpp, it re-extracts that note only, no Ollama model loads, and after the run the model sleeps with the service up. A sleeping model keeps 122 MiB of VRAM until POST /models/unload.

Owner sketch

_REBUILD_SCRIPT owns the run steps. config.yaml owns the choice. changed_notes.py owns the question "did a note change", and vault-model-ready owns the question "can the model run". Both are thin scripts with no new domain object.

plugins/os-vault/tests/test_session_start_rebuild.py is retired. The pipeline deleted it from the worktree at open, so the Retired tests section is gone: the reroute check refuses an entry whose file no longer exists. The new tests replace it.

Out of scope

  • onboard-project skill still passes --backend ollama (SKILL.md:45-51,86).
  • eval/graph-extraction/bin/run_extract.sh stays on Ollama.
  • Full rebuilds wait for Jared.

Blocking edges

None. jared/llm-stack#22 and #23 are closed, and both are on main.

Origin

  • Trigger: ruling 23 and handoff 2026-11-05-1807-unsloth-llamacpp-default-switch.md
  • Improvised this session: recreated ~/.cache/graphify/vault-rebuild.lock as a hold, which ends 2026-11-06 12:55.
  • Chain: hook starts the old model ← --backend ollama in plugins/os-vault/hooks/session_start.py ← DESIGN
  • Root candidate: none found; searched llama.cpp, unsloth, backend ollama, session_start, rebuild hook
  • Where: plugins/os-vault/hooks/session_start.py
  • Session: 47ffa777-a9a8-4ac8-808a-b9b19489944a
  • Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/47ffa777-a9a8-4ac8-808a-b9b19489944a.jsonl

VERDICT: CREATE

## Problem The os-vault rebuild hook runs Ollama `qwen3.5:9b`. Jared decided that Unsloth Qwen3.5-9B on llama.cpp is the default (ruling 23: config V4). A rebuild with the old model replaces the re-extraction results. The hook starts this rebuild when a stamp is older than 7 days. The hook also starts it in each session when no stamp and no lock exist. ## Observed `plugins/os-vault/hooks/session_start.py` hardcodes `--backend ollama`. On 2026-11-05 at 12:53 the hook started a rebuild on Ollama, and session 47ffa777 stopped it. The rebuild gate regex misses the output `still hollow` and `still truncated`. #1556 covers the gate of the eval, with no numeric limit. ## Reproduce n/a: a rebuild takes over one hour on the GPU. ## Approach The hook runs the V4 config on llama.cpp and re-extracts only changed notes. llm-stack has no CLI and no process control, so os-vault owns a small readiness script and the router unit file. Required work: 1. `plugins/os-vault/config.yaml` is the one place that names the choice. Replace `ollama_model` with `graphify_backend: llamacpp-qwen35`, `graphify_model: qwen3.5-9b` (the id graphify sends to the router), and `llm_model: qwen3-5-9b` (the llm-stack catalog name). Remove `GRAPHIFY_OLLAMA_NUM_CTX`, `GRAPHIFY_OLLAMA_KEEP_ALIVE`, and `OLLAMA_BASE_URL`. Set `GRAPHIFY_MAX_OUTPUT_TOKENS=10000`, `GRAPHIFY_MAX_RETRY_DEPTH=1`, `GRAPHIFY_MAX_RETRIES=0`, `GRAPHIFY_API_TIMEOUT=900`, `LLAMACPP_KEY=local`. Keep `GRAPHIFY_DISABLE_THINKING`. check: present `llamacpp-qwen35` in `plugins/os-vault/config.yaml` 2. `hooks/config.py` reads the three new keys and loses its fallback model copy. A missing key loads as an empty value; `main` starts no rebuild and logs the reason. check: absent `qwen3.5:9b` in `plugins/os-vault/hooks/config.py` 3. Add the router unit and preset as source. The service command and the V4 preset are in `llm-stack docs/research/llamacpp-router-spike.md`. The unit runs `llama serve --host 127.0.0.1 --port 8087 --models-preset <preset> --models-max 1 --sleep-idle-seconds 30`. The hook does not start or stop the service. check: present `--sleep-idle-seconds` in `plugins/os-vault/llama-router/llama-router.service` 4. Add the preset file next to the unit, with the model path `/home/jared/.cache/huggingface/hub/models--unsloth--Qwen3.5-9B-GGUF/snapshots/3885219b6810b007914f3a7950a8d1b469d598a5/Qwen3.5-9B-UD-Q4_K_XL.gguf`. check: present `qwen3.5-9b` in `plugins/os-vault/llama-router/preset.ini` 5. Add `plugins/os-vault/bin/vault-model-ready`, a Ruby script, and `plugins/os-vault/Gemfile` with `gem 'llm-stack'` by path in the same form as `plugins/os-adr/Gemfile`. It takes the catalog name, calls `LLM.models.find(name, provider: "llamacpp")`, exits 0 when `ready?` is true, and otherwise prints `reason` and exits 1. `_REBUILD_SCRIPT` calls it as `ruby "$READY_SCRIPT" "$LLM_MODEL"`, so a test can place a `ruby` shim on PATH. check: present `LLM.models.find` in `plugins/os-vault/bin/vault-model-ready` 6. Add `plugins/os-vault/hooks/changed_notes.py`, run with `$GRAPHIFY_PYTHON` as `stub_markdown_pages.py` is. It calls `graphify.detect.detect_incremental(root, manifest_path=<root>/graphify-out/manifest.json, kind="semantic")`. The default manifest path is relative to the working directory, so the script must pass the path. Exit 0 when `new_total` is above 0, exit 1 when it is 0, and any other exit on error. check: present `detect_incremental` in `plugins/os-vault/hooks/changed_notes.py` 7. `_REBUILD_SCRIPT` runs `changed_notes.py` first. With changes: run `vault-model-ready`; if it fails, the run fails. Then run `graphify extract "$VAULT_PATH" --backend "$BACKEND" --model "$MODEL" --token-budget 4000 --max-concurrency 1 --allow-partial`, then `graphify cluster-only` with the same `--backend` and `--model`. With no changes: skip readiness, extract, and cluster-only. In both cases run the stub step and the two `graphify update --force` runs. Extract stays incremental and never uses `--force`. check: present `--allow-partial` in `plugins/os-vault/hooks/session_start.py` 8. Add `still hollow` and `still truncated` to the failure check next to `chunk N/M failed`. check: present `still truncated` in `plugins/os-vault/hooks/session_start.py` 9. Update stale names. check: present `llamacpp-qwen35` in `docs/implementation-status/os-vault.md` 10. In the README, describe the llama.cpp setup and remove the Ollama settings. check: absent `GRAPHIFY_OLLAMA_NUM_CTX` in `plugins/os-vault/README.md` 11. Update the fixture config. check: absent `ollama_model` in `plugins/os-vault/tests/harness.sh` 12. Update the fixture config. check: absent `ollama_model` in `plugins/os-vault/tests/generate-fixtures.sh` ## Acceptance criteria 1. Given one changed note and a ready model, when the rebuild runs, then graphify extract receives `--backend llamacpp-qwen35 --model qwen3.5-9b --allow-partial`, and no argument names ollama. 2. Given no changed notes, when the rebuild runs, then the readiness script, extract, and cluster-only do not run, and the stub step and both update runs do. 3. Given run output that contains `still hollow` or `still truncated`, when the rebuild ends, then it writes no stamp and it writes the failed marker. 4. Given a changed note and a readiness script that exits 1, when the rebuild runs, then extract does not run, it writes no stamp, and it writes the failed marker. 5. Given a config with no `graphify_model`, when the hook runs, then it starts no rebuild and logs the reason. 6. The rebuild script passes no `--force` to extract. ## After merge The orchestrator does these by hand, because they sit outside the repo. They are not gates of this pipeline. - Install the unit under `~/.config/systemd/user`, enable it, and confirm that port 8087 answers. - Set `base_url` in `~/.graphify/providers.json` to port 8087, because 8089 was the eval logging proxy. Update `~/Documents/SecondBrain/howto/running-qwen35-9b-gguf-behind-graphify-with-llama-cpp.md` to match. - With Jared's consent and a stated run time, prove a one-note change: V4 runs through llama.cpp, it re-extracts that note only, no Ollama model loads, and after the run the model sleeps with the service up. A sleeping model keeps 122 MiB of VRAM until `POST /models/unload`. ## Owner sketch `_REBUILD_SCRIPT` owns the run steps. `config.yaml` owns the choice. `changed_notes.py` owns the question "did a note change", and `vault-model-ready` owns the question "can the model run". Both are thin scripts with no new domain object. plugins/os-vault/tests/test_session_start_rebuild.py is retired. The pipeline deleted it from the worktree at open, so the Retired tests section is gone: the reroute check refuses an entry whose file no longer exists. The new tests replace it. ## Out of scope - `onboard-project` skill still passes `--backend ollama` (`SKILL.md:45-51,86`). - `eval/graph-extraction/bin/run_extract.sh` stays on Ollama. - Full rebuilds wait for Jared. ## Blocking edges None. jared/llm-stack#22 and #23 are closed, and both are on main. ## Origin - Trigger: ruling 23 and handoff 2026-11-05-1807-unsloth-llamacpp-default-switch.md - Improvised this session: recreated `~/.cache/graphify/vault-rebuild.lock` as a hold, which ends 2026-11-06 12:55. - Chain: hook starts the old model ← `--backend ollama` in `plugins/os-vault/hooks/session_start.py` ← DESIGN - Root candidate: none found; searched llama.cpp, unsloth, backend ollama, session_start, rebuild hook - Where: plugins/os-vault/hooks/session_start.py - Session: 47ffa777-a9a8-4ac8-808a-b9b19489944a - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/47ffa777-a9a8-4ac8-808a-b9b19489944a.jsonl VERDICT: CREATE
Author
Owner

Date limit: the stamp was written 2026-11-05, so it passes 7 days on 2026-11-12. The hook then starts a rebuild on Ollama qwen3.5:9b unless this ticket ships. The V4 full run also showed that extract needs --allow-partial whenever it replaces older notes, or the shrink guard refuses the save.

Date limit: the stamp was written 2026-11-05, so it passes 7 days on 2026-11-12. The hook then starts a rebuild on Ollama `qwen3.5:9b` unless this ticket ships. The V4 full run also showed that extract needs `--allow-partial` whenever it replaces older notes, or the shrink guard refuses the save.
Author
Owner

Blocked by jared/llm-stack#22. The 2026-11-12 date in the earlier comment is not a real limit: any vault note write deletes the stamp, and each run is incremental. Jared, 2026-11-06: no full rebuild until the setup is proven; incremental updates continue.

Blocked by jared/llm-stack#22. The 2026-11-12 date in the earlier comment is not a real limit: any vault note write deletes the stamp, and each run is incremental. Jared, 2026-11-06: no full rebuild until the setup is proven; incremental updates continue.
Author
Owner

Ticket revised for jared/llm-stack#23. llm-stack run is dropped: the router runs as a systemd user service on port 8087, and the unit is not installed yet. Revised: item 3, item 4, criteria 2 and 4, blocking edges. Readiness: LLM.models.find("qwen3-5-9b", provider: "llamacpp").ready?. Note: a sleeping model keeps 122 MiB of VRAM until POST /models/unload. Router spike evidence: llm-stack docs/research/llamacpp-router-spike.md.

Ticket revised for jared/llm-stack#23. llm-stack run is dropped: the router runs as a systemd user service on port 8087, and the unit is not installed yet. Revised: item 3, item 4, criteria 2 and 4, blocking edges. Readiness: LLM.models.find("qwen3-5-9b", provider: "llamacpp").ready?. Note: a sleeping model keeps 122 MiB of VRAM until POST /models/unload. Router spike evidence: llm-stack docs/research/llamacpp-router-spike.md.
Author
Owner

Resumed. jared/llm-stack#22 and #23 are closed and on main (b7af097, 4b3d459). The revised body matches the shipped interface: LLM.models.find with provider llamacpp, port 8087, no llm-stack run. Removed the waiting label.

Resumed. jared/llm-stack#22 and #23 are closed and on main (b7af097, 4b3d459). The revised body matches the shipped interface: LLM.models.find with provider llamacpp, port 8087, no llm-stack run. Removed the waiting label.
Author
Owner

Work started on branch ticket-1576.

Work started on branch ticket-1576.
Author
Owner

Follow-up dropped by ticket-skeptic (depth-1 limit): reroute intake re-runs missing_retired_tests after Implementation::Open has already deleted the retired files (ticket_shape.rb:103, open.rb:58). Any reroute of a ticket with a Retired tests section is refused.

Follow-up dropped by ticket-skeptic (depth-1 limit): reroute intake re-runs missing_retired_tests after Implementation::Open has already deleted the retired files (ticket_shape.rb:103, open.rb:58). Any reroute of a ticket with a Retired tests section is refused.
Author
Owner

Resolution

Done: The os-vault rebuild hook now runs Unsloth Qwen3.5-9B V4 through llamacpp-qwen35 on changed notes only. No changed notes means no router request. vault-model-ready gates extract. changed_notes.py fails the run on any error or a missing manifest or graph, so it cannot start a full re-extraction. The failure scan covers hollow, truncated, and incomplete output. The router runs as llama-router.service on port 8087, and providers.json and the howto point at it.

Evidence: main d02eb380 (39eacd90, df4f36e7). pytest plugins/os-vault/tests: 35 passed. rubocop clean. Each required-work check verified by a script. One opus design audit: 9 of 10 findings fixed or answered by code, finding 10 answered with no change. Live run at 2026-11-06 15:22 through the real hook: 50 s, 3 re-extracted, 339 unchanged, 0 chunks failed, stamp written, no failed marker, /api/ps empty on Ollama. The model slept about 30 s after the last request: 146 MiB VRAM, service active. Hand-finished through the escalation procedure because the diff-test gate cannot run pytest. AC 2, the no-change branch, is proved by tests, not by a live run.

Follow-ups: #1625 diff-test gate for pytest targets. #1626 onboard-project and reorganize skill text. Dropped by ticket-skeptic and noted: reroute retired-tests refusal (comment on #1576); harness.sh repair (comment on #1527).

## Resolution **Done:** The os-vault rebuild hook now runs Unsloth Qwen3.5-9B V4 through llamacpp-qwen35 on changed notes only. No changed notes means no router request. vault-model-ready gates extract. changed_notes.py fails the run on any error or a missing manifest or graph, so it cannot start a full re-extraction. The failure scan covers hollow, truncated, and incomplete output. The router runs as llama-router.service on port 8087, and providers.json and the howto point at it. **Evidence:** main d02eb380 (39eacd90, df4f36e7). pytest plugins/os-vault/tests: 35 passed. rubocop clean. Each required-work check verified by a script. One opus design audit: 9 of 10 findings fixed or answered by code, finding 10 answered with no change. Live run at 2026-11-06 15:22 through the real hook: 50 s, 3 re-extracted, 339 unchanged, 0 chunks failed, stamp written, no failed marker, /api/ps empty on Ollama. The model slept about 30 s after the last request: 146 MiB VRAM, service active. Hand-finished through the escalation procedure because the diff-test gate cannot run pytest. AC 2, the no-change branch, is proved by tests, not by a live run. **Follow-ups:** #1625 diff-test gate for pytest targets. #1626 onboard-project and reorganize skill text. Dropped by ticket-skeptic and noted: reroute retired-tests refusal (comment on #1576); harness.sh repair (comment on #1527).
jared closed this issue 2026-11-06 20:25:06 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
jared/cc-os#1576
No description provided.