jared/cc-os

Watch

Fork 0

os-adr: move Adr::Judge::ClaudeCommand onto the llm-stack gem #1437

New issue

Closed

opened 2026-10-27 18:11:52 +00:00 by jared · 2 comments

jared commented

2026-10-27 18:11:52 +00:00

Owner

Problem

Adr::Judge::ClaudeCommand in plugins/os-adr/lib/adr/judge/claude_command.rb shells out to claude -p with its own Open3 call, argv list, and timeout.
Several cc-os plugins repeat this shell-out.
ADR-0301 makes the llm-stack gem the one place that wraps the claude binary.
This ticket is the first caller to move (spec section 6, step 2).
It proves the gem against a real caller before the other plugins follow.
plugins/os-adr has no Gemfile today, so the ticket also adds one.

Reproduce

n/a: this is a feature, not a defect.

Approach

Required work:

Create plugins/os-adr/Gemfile with gem "llm-stack", git: "ssh://git@forgejo.swansoncloud.com:2222/jared/llm-stack.git", require: "llm", plus zeitwerk and minitest in the test group, and commit plugins/os-adr/Gemfile.lock (ADR-0166) — check: present llm-stack in plugins/os-adr/Gemfile.lock
Delete plugins/os-adr/lib/adr/judge/claude_command.rb and add plugins/os-adr/lib/adr/judge/model_chat.rb with class Adr::Judge::ModelChat — check: absent Open3 in plugins/os-adr/lib/adr/judge/*.rb
Make ModelChat build its own LLM::Client with one profile of provider claude_code, the judge model, and these settings: cwd is the repo root, tools is empty, budget_usd is 0.5, timeout is 180 — check: present budget_usd in plugins/os-adr/lib/adr/judge/model_chat.rb
Make ModelChat#run(prompt) call client.chat(profile:, prompt:, system: card, schema:) and return Response#text, which is the verdict JSON text — check: present schema: in plugins/os-adr/lib/adr/judge/model_chat.rb
Change Adr::Judge to take chat: in place of runner:, to call JSON.parse on the text, and to pass the hash to Verdict.parse; remove the reply['is_error'] and structured_output envelope handling, because the gem now owns the envelope — check: absent structured_output in plugins/os-adr/lib/adr/judge.rb
Map gem errors to Adr::Judge::Unavailable and keep the meaning of the three old messages: LLM::TimedOut gives "claude did not answer within 180 seconds"; LLM::ProviderError gives "claude failed: " plus the first 200 characters of the error message, which also covers a missing executable and a non-zero exit; unparsable text gives "claude reply is not JSON: " plus the first 200 characters — check: present LLM::TimedOut in plugins/os-adr/lib/adr/judge/model_chat.rb
Rewrite plugins/os-adr/tests/judge_test.rb to use a fake claude script, set through the profile executable setting, in the style of test/support/bin/claude in the llm-stack gem; no test runs the real binary — check: absent RecordingRunner in plugins/os-adr/tests/judge_test.rb
Make the plugins/os-adr/bin/* scripts load the bundle before lib/adr — check: present bundler/setup in plugins/os-adr/bin/*

Orchestrator work: run bin/refresh-plugins, then update the os-adr and llm-stack leaves under docs/implementation-status/ with one History row each, and mark spec section 6 step 2 as migrated.

Acceptance criteria

Given a fake claude that returns a valid verdict, when Adr::Judge#call runs in create mode, then it returns a Verdict with the fake's verdict.
Given a fake claude that sleeps past the timeout, when Adr::Judge#call runs, then it raises Unavailable with the 180 second message.
Given a fake claude that exits non-zero, when Adr::Judge#call runs, then it raises Unavailable that starts with "claude failed".
Given an executable setting that names no file, when Adr::Judge#call runs, then it raises Unavailable.
Given a fake claude that prints text that is not JSON, when Adr::Judge#call runs, then it raises Unavailable with "claude reply is not JSON".
Given the fake claude records its argv, when Adr::Judge#call runs, then the argv holds the model, an empty tool list, the card as the system prompt, the schema, and a 0.5 dollar budget cap, and the prompt arrives on stdin.
Given the plugin tree after the change, when you search plugins/os-adr/lib for Open3, then nothing matches.

Owner sketch

module Adr
  class Judge
    class ModelChat
      PROFILE = :adr_judge
      TIMEOUT_SECONDS = 180
      BUDGET_USD = 0.5

      def initialize(model:, card:, schema:, root:, executable: 'claude')
      end

      def run(prompt)
        # LLM::Client#chat; LLM::TimedOut and LLM::ProviderError become Unavailable
      end
    end
  end
end

Non-goals

Other callers of claude -p: they have their own tickets.
Change to the judge card, the verdict schema, or the Verdict class.
A global LLM.configure call in the plugin.
A live call to the real claude binary in any test.

Origin

Trigger: Spec docs/llm-stack/spec.md section 6 step 2 names Adr::Judge::ClaudeCommand as the first migration.
Improvised this session: none
Chain: Adr::Judge::ClaudeCommand owns its own claude -p shell-out ← ADR-0166 and ADR-0279 forbid a shared model layer outside a pinned gem ← DESIGN (docs/adr/0301-llm-stack-gem-is-the-shared-llm-layer-pinned-per-plugin-from-forgejo.md)
Root candidate: none found; searched llm-stack, ClaudeCommand, claude_command, Adr::Judge, claude -p, Open3, ModelChat
Where: plugins/os-adr/lib/adr/judge/claude_command.rb
Session: 375f16d0-a830-4297-b9f5-3acb6090a73f
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/375f16d0-a830-4297-b9f5-3acb6090a73f.jsonl
VERDICT: CREATE
REASON: First migration in the human-ruled ADR-0301 / spec section 6 plan; no open ticket covers it.

## Problem `Adr::Judge::ClaudeCommand` in `plugins/os-adr/lib/adr/judge/claude_command.rb` shells out to `claude -p` with its own Open3 call, argv list, and timeout. Several cc-os plugins repeat this shell-out. ADR-0301 makes the `llm-stack` gem the one place that wraps the `claude` binary. This ticket is the first caller to move (spec section 6, step 2). It proves the gem against a real caller before the other plugins follow. `plugins/os-adr` has no Gemfile today, so the ticket also adds one. ## Reproduce n/a: this is a feature, not a defect. ## Approach Required work: 1. Create `plugins/os-adr/Gemfile` with `gem "llm-stack", git: "ssh://git@forgejo.swansoncloud.com:2222/jared/llm-stack.git", require: "llm"`, plus `zeitwerk` and `minitest` in the test group, and commit `plugins/os-adr/Gemfile.lock` (ADR-0166) — check: present `llm-stack` in `plugins/os-adr/Gemfile.lock` 2. Delete `plugins/os-adr/lib/adr/judge/claude_command.rb` and add `plugins/os-adr/lib/adr/judge/model_chat.rb` with class `Adr::Judge::ModelChat` — check: absent `Open3` in `plugins/os-adr/lib/adr/judge/*.rb` 3. Make `ModelChat` build its own `LLM::Client` with one profile of provider `claude_code`, the judge model, and these settings: `cwd` is the repo root, `tools` is empty, `budget_usd` is 0.5, `timeout` is 180 — check: present `budget_usd` in `plugins/os-adr/lib/adr/judge/model_chat.rb` 4. Make `ModelChat#run(prompt)` call `client.chat(profile:, prompt:, system: card, schema:)` and return `Response#text`, which is the verdict JSON text — check: present `schema:` in `plugins/os-adr/lib/adr/judge/model_chat.rb` 5. Change `Adr::Judge` to take `chat:` in place of `runner:`, to call `JSON.parse` on the text, and to pass the hash to `Verdict.parse`; remove the `reply['is_error']` and `structured_output` envelope handling, because the gem now owns the envelope — check: absent `structured_output` in `plugins/os-adr/lib/adr/judge.rb` 6. Map gem errors to `Adr::Judge::Unavailable` and keep the meaning of the three old messages: `LLM::TimedOut` gives "claude did not answer within 180 seconds"; `LLM::ProviderError` gives "claude failed: " plus the first 200 characters of the error message, which also covers a missing executable and a non-zero exit; unparsable text gives "claude reply is not JSON: " plus the first 200 characters — check: present `LLM::TimedOut` in `plugins/os-adr/lib/adr/judge/model_chat.rb` 7. Rewrite `plugins/os-adr/tests/judge_test.rb` to use a fake `claude` script, set through the profile `executable` setting, in the style of `test/support/bin/claude` in the llm-stack gem; no test runs the real binary — check: absent `RecordingRunner` in `plugins/os-adr/tests/judge_test.rb` 8. Make the `plugins/os-adr/bin/*` scripts load the bundle before `lib/adr` — check: present `bundler/setup` in `plugins/os-adr/bin/*` Orchestrator work: run `bin/refresh-plugins`, then update the os-adr and llm-stack leaves under `docs/implementation-status/` with one History row each, and mark spec section 6 step 2 as migrated. ## Acceptance criteria 1. Given a fake `claude` that returns a valid verdict, when `Adr::Judge#call` runs in `create` mode, then it returns a `Verdict` with the fake's verdict. 2. Given a fake `claude` that sleeps past the timeout, when `Adr::Judge#call` runs, then it raises `Unavailable` with the 180 second message. 3. Given a fake `claude` that exits non-zero, when `Adr::Judge#call` runs, then it raises `Unavailable` that starts with "claude failed". 4. Given an `executable` setting that names no file, when `Adr::Judge#call` runs, then it raises `Unavailable`. 5. Given a fake `claude` that prints text that is not JSON, when `Adr::Judge#call` runs, then it raises `Unavailable` with "claude reply is not JSON". 6. Given the fake `claude` records its argv, when `Adr::Judge#call` runs, then the argv holds the model, an empty tool list, the card as the system prompt, the schema, and a 0.5 dollar budget cap, and the prompt arrives on stdin. 7. Given the plugin tree after the change, when you search `plugins/os-adr/lib` for `Open3`, then nothing matches. ## Owner sketch ```ruby module Adr class Judge class ModelChat PROFILE = :adr_judge TIMEOUT_SECONDS = 180 BUDGET_USD = 0.5 def initialize(model:, card:, schema:, root:, executable: 'claude') end def run(prompt) # LLM::Client#chat; LLM::TimedOut and LLM::ProviderError become Unavailable end end end end ``` ## Non-goals - Other callers of `claude -p`: they have their own tickets. - Change to the judge card, the verdict schema, or the `Verdict` class. - A global `LLM.configure` call in the plugin. - A live call to the real `claude` binary in any test. ## Origin - Trigger: Spec docs/llm-stack/spec.md section 6 step 2 names Adr::Judge::ClaudeCommand as the first migration. - Improvised this session: none - Chain: Adr::Judge::ClaudeCommand owns its own claude -p shell-out ← ADR-0166 and ADR-0279 forbid a shared model layer outside a pinned gem ← DESIGN (docs/adr/0301-llm-stack-gem-is-the-shared-llm-layer-pinned-per-plugin-from-forgejo.md) - Root candidate: none found; searched llm-stack, ClaudeCommand, claude_command, Adr::Judge, claude -p, Open3, ModelChat - Where: plugins/os-adr/lib/adr/judge/claude_command.rb - Session: 375f16d0-a830-4297-b9f5-3acb6090a73f - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/375f16d0-a830-4297-b9f5-3acb6090a73f.jsonl VERDICT: CREATE REASON: First migration in the human-ruled ADR-0301 / spec section 6 plan; no open ticket covers it.

jared added the

labels

2026-10-27 18:11:52 +00:00

jared referenced this issue

2026-10-27 18:12:03 +00:00

os-me: move HeadlessRunner onto llm-stack #1438

jared referenced this issue

2026-10-27 18:12:04 +00:00

os-sdlc takedown: move the claude -p path onto llm-stack #1439

jared commented

2026-10-27 18:12:28 +00:00

Author

Owner

Work started on branch ticket-1437 via /os-sdlc:implement (session 375f16d0).

jared referenced this issue from a commit

2026-10-27 18:30:10 +00:00

feat(os-adr): judge calls the llm-stack gem instead of shelling out to claude -p (#1437)

jared commented

2026-10-27 18:30:45 +00:00

Author

Owner

Resolution

Done: Adr::Judge::ModelChat calls the llm-stack gem's LLM::Client#chat on a claude_code profile; ClaudeCommand and Open3 removed; plugins/os-adr has a Gemfile and lockfile pinned to llm-stack from the Forgejo git source; Judge takes a chat: seam.

Evidence: Commit 3ef516f5 merged to main at 0faed2ff. os-adr suite 114 runs, 365 assertions, 0 failures. Root rubocop clean on changed files. Design audit (contract-auditor, opus): one fix-required (bin require order) applied; rest answered acceptable. bin/adr-find and bin/adr-detect run under the new Gemfile.

Follow-ups: Dropped: a shared contract test between CannedChat and ModelChat. Reason: the judge argv tests already drive the real ModelChat end to end against a fake claude executable, so the double is pinned by the same acceptance behavior. No other follow-ups.

## Resolution **Done:** Adr::Judge::ModelChat calls the llm-stack gem's LLM::Client#chat on a claude_code profile; ClaudeCommand and Open3 removed; plugins/os-adr has a Gemfile and lockfile pinned to llm-stack from the Forgejo git source; Judge takes a chat: seam. **Evidence:** Commit 3ef516f5 merged to main at 0faed2ff. os-adr suite 114 runs, 365 assertions, 0 failures. Root rubocop clean on changed files. Design audit (contract-auditor, opus): one fix-required (bin require order) applied; rest answered acceptable. bin/adr-find and bin/adr-detect run under the new Gemfile. **Follow-ups:** Dropped: a shared contract test between CannedChat and ModelChat. Reason: the judge argv tests already drive the real ModelChat end to end against a fake claude executable, so the double is pinned by the same acceptance behavior. No other follow-ups.

jared closed this issue

2026-10-27 18:30:45 +00:00

jared referenced this issue from a commit

2026-10-27 18:31:12 +00:00

docs: History row for #1437; spec build order marks step 2 done and names #1438 #1439

jared referenced this issue

2026-10-27 22:59:47 +00:00

Sdlc/Structural/DiscardedErrorCode: carve out predicate conditions and per-branch exception classes; state the fix #1353