jared/cc-os

Watch

Fork 0

os-sdlc takedown: move the claude -p path onto llm-stack #1439

New issue

Closed

opened 2026-10-27 18:11:58 +00:00 by jared · 2 comments

jared commented

2026-10-27 18:11:58 +00:00

Owner

Problem

The takedown eval harness runs claude -p through four objects under plugins/os-sdlc/eval/ticket-takedown/lib/takedown/.
Takedown::ModelRequest builds the argv.
Takedown::ProcessRunner runs the process, feeds stdin, and kills the process group on timeout.
Takedown::ClaudeExecutor joins the two.
Takedown::ClaudeReply parses the JSON reply for cost, turns, and token usage.
The llm-stack gem now holds the same work in its claude_code provider (ADR-0301).
Was blocked by #1437 and #1438, both merged 2026-10-27.

Reproduce

n/a: this is a feature, not a defect.

Approach

Required work:

Add gem "llm-stack", git: "ssh://git@forgejo.swansoncloud.com:2222/jared/llm-stack.git", require: "llm" to plugins/os-sdlc/Gemfile (the takedown harness has no Gemfile of its own) and commit plugins/os-sdlc/Gemfile.lock (ADR-0166) — check: present llm-stack in plugins/os-sdlc/Gemfile.lock
Make ClaudeExecutor#call build a LLM::Client per call with a claude_code profile from the ModelRequest fields and call LLM::Client#chat with system from card, the prompt, and the schema; the profile carries model, cwd, tools, allowed_tools, disallowed_tools, budget_usd from cap, persist_session, resume, settings_json from the constructor, executable from the constructor, and a timeout of 900 seconds; the client uses an empty LLM::PriceTable so the CLI-reported cost is kept — check: present chat( in plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/claude_executor.rb
Stop ClaudeExecutor from using ProcessRunner; keep ProcessRunner itself, because SuiteRunner, LintRun, ScratchRepo, and RunEnvironment still use it for git, suite, and lint commands — check: absent runner in plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/claude_executor.rb
Delete ModelRequest#claude_argv and its helper methods; keep ModelRequest as the value that holds the step inputs — check: absent claude_argv in plugins/os-sdlc/eval/ticket-takedown/lib/**/*.rb
Replace ClaudeReply.parse with a mapping from the gem Response: token counts from Response#usage, cost from Response#cost, raw from meta["raw"], session_id from meta["session_id"], turns from meta["num_turns"], models from meta["models"], and structured from parsing Response#text; keep StepOutcome fields unchanged — check: present Response in plugins/os-sdlc/eval/ticket-takedown/lib/takedown/parser/claude_reply.rb
Rescue LLM::TimedOut and LLM::ProviderError in ClaudeExecutor and build the failed reply from e.message (reported <subtype>: <result> or exit N: <stderr>), so the CLI result text still reaches note; claude_error_note_test may change its expected text but must keep asserting that result text reaches the note — check: present LLM::TimedOut in plugins/os-sdlc/eval/ticket-takedown/lib/**/*.rb
Keep every recorded fixture and case file as is; the tests use a fake claude through the profile executable setting — check: absent FakeRunner.new in plugins/os-sdlc/eval/ticket-takedown/test/adapter/claude_executor_test.rb

Orchestrator work: compare one takedown run before and after the change for the same case, run bin/refresh-plugins, and add one History row to the os-sdlc leaf under docs/implementation-status/.

Acceptance criteria

Given a fake claude that returns usage and cost, when ClaudeExecutor#call runs, then the reply holds the token counts and the cost from the gem Response.
Given a fake claude that never answers, when ClaudeExecutor#call runs, then the reply is a failure with a timeout note.
Given a ModelRequest with resume set, when ClaudeExecutor#call runs, then the fake claude records the resume session ID.
Given a ModelRequest with hidden paths, when ClaudeExecutor#call runs, then the fake claude records the matching disallowed-tool rules.
Given the recorded fixtures, when the takedown test suite runs, then every fixture test passes without change to a fixture file.

Owner sketch

module Takedown
  class ClaudeExecutor
    TIMEOUT = 900

    def initialize(executable: "claude", settings_json:)
    end

    def call(request)
      # LLM::Client#chat on a claude_code profile built from request; answers ClaudeReply
    end
  end
end

Non-goals

Change to the takedown cards, cases, results, or scoring.
The StepOutcome shape.
A live call to the real claude binary in any test.
Other plugins that call claude -p.
Takedown-specific isolation flags: the gem's ISOLATION flags replace them, and tests that pinned those exact flags are updated to the gem's.

Origin

Trigger: Spec docs/llm-stack/spec.md section 6 step 3 lists the takedown ModelRequest, ProcessRunner, and ClaudeReply.
Improvised this session: none
Chain: Takedown builds its own claude argv, runs its own process, and parses its own reply ← ADR-0166 and ADR-0279 forbid a shared model layer outside a pinned gem ← DESIGN (docs/adr/0301-llm-stack-gem-is-the-shared-llm-layer-pinned-per-plugin-from-forgejo.md)
Root candidate: none found; searched llm-stack, claude -p, ClaudeExecutor, ProcessRunner, ClaudeReply, ModelRequest, takedown
Where: plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/claude_executor.rb
Session: 375f16d0-a830-4297-b9f5-3acb6090a73f
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/375f16d0-a830-4297-b9f5-3acb6090a73f.jsonl
VERDICT: CREATE
REASON: Implements human-ruled ADR-0301 and spec step 3 for the takedown caller; no open ticket covers it.

## Problem The takedown eval harness runs `claude -p` through four objects under `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/`. `Takedown::ModelRequest` builds the argv. `Takedown::ProcessRunner` runs the process, feeds stdin, and kills the process group on timeout. `Takedown::ClaudeExecutor` joins the two. `Takedown::ClaudeReply` parses the JSON reply for cost, turns, and token usage. The `llm-stack` gem now holds the same work in its `claude_code` provider (ADR-0301). Was blocked by #1437 and #1438, both merged 2026-10-27. ## Reproduce n/a: this is a feature, not a defect. ## Approach Required work: 1. Add `gem "llm-stack", git: "ssh://git@forgejo.swansoncloud.com:2222/jared/llm-stack.git", require: "llm"` to `plugins/os-sdlc/Gemfile` (the takedown harness has no Gemfile of its own) and commit `plugins/os-sdlc/Gemfile.lock` (ADR-0166) — check: present `llm-stack` in `plugins/os-sdlc/Gemfile.lock` 2. Make `ClaudeExecutor#call` build a `LLM::Client` per call with a `claude_code` profile from the `ModelRequest` fields and call `LLM::Client#chat` with `system` from `card`, the `prompt`, and the `schema`; the profile carries `model`, `cwd`, `tools`, `allowed_tools`, `disallowed_tools`, `budget_usd` from `cap`, `persist_session`, `resume`, `settings_json` from the constructor, `executable` from the constructor, and a timeout of 900 seconds; the client uses an empty `LLM::PriceTable` so the CLI-reported cost is kept — check: present `chat(` in `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/claude_executor.rb` 3. Stop `ClaudeExecutor` from using `ProcessRunner`; keep `ProcessRunner` itself, because `SuiteRunner`, `LintRun`, `ScratchRepo`, and `RunEnvironment` still use it for git, suite, and lint commands — check: absent `runner` in `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/claude_executor.rb` 4. Delete `ModelRequest#claude_argv` and its helper methods; keep `ModelRequest` as the value that holds the step inputs — check: absent `claude_argv` in `plugins/os-sdlc/eval/ticket-takedown/lib/**/*.rb` 5. Replace `ClaudeReply.parse` with a mapping from the gem `Response`: token counts from `Response#usage`, cost from `Response#cost`, `raw` from `meta["raw"]`, `session_id` from `meta["session_id"]`, `turns` from `meta["num_turns"]`, `models` from `meta["models"]`, and `structured` from parsing `Response#text`; keep `StepOutcome` fields unchanged — check: present `Response` in `plugins/os-sdlc/eval/ticket-takedown/lib/takedown/parser/claude_reply.rb` 6. Rescue `LLM::TimedOut` and `LLM::ProviderError` in `ClaudeExecutor` and build the failed reply from `e.message` (`reported <subtype>: <result>` or `exit N: <stderr>`), so the CLI result text still reaches `note`; `claude_error_note_test` may change its expected text but must keep asserting that result text reaches the note — check: present `LLM::TimedOut` in `plugins/os-sdlc/eval/ticket-takedown/lib/**/*.rb` 7. Keep every recorded fixture and case file as is; the tests use a fake `claude` through the profile `executable` setting — check: absent `FakeRunner.new` in `plugins/os-sdlc/eval/ticket-takedown/test/adapter/claude_executor_test.rb` Orchestrator work: compare one takedown run before and after the change for the same case, run `bin/refresh-plugins`, and add one History row to the os-sdlc leaf under `docs/implementation-status/`. ## Acceptance criteria 1. Given a fake `claude` that returns usage and cost, when `ClaudeExecutor#call` runs, then the reply holds the token counts and the cost from the gem `Response`. 2. Given a fake `claude` that never answers, when `ClaudeExecutor#call` runs, then the reply is a failure with a timeout note. 3. Given a `ModelRequest` with `resume` set, when `ClaudeExecutor#call` runs, then the fake `claude` records the resume session ID. 4. Given a `ModelRequest` with `hidden` paths, when `ClaudeExecutor#call` runs, then the fake `claude` records the matching disallowed-tool rules. 5. Given the recorded fixtures, when the takedown test suite runs, then every fixture test passes without change to a fixture file. ## Owner sketch ```ruby module Takedown class ClaudeExecutor TIMEOUT = 900 def initialize(executable: "claude", settings_json:) end def call(request) # LLM::Client#chat on a claude_code profile built from request; answers ClaudeReply end end end ``` ## Non-goals - Change to the takedown cards, cases, results, or scoring. - The `StepOutcome` shape. - A live call to the real `claude` binary in any test. - Other plugins that call `claude -p`. - Takedown-specific isolation flags: the gem's ISOLATION flags replace them, and tests that pinned those exact flags are updated to the gem's. ## Origin - Trigger: Spec docs/llm-stack/spec.md section 6 step 3 lists the takedown ModelRequest, ProcessRunner, and ClaudeReply. - Improvised this session: none - Chain: Takedown builds its own claude argv, runs its own process, and parses its own reply ← ADR-0166 and ADR-0279 forbid a shared model layer outside a pinned gem ← DESIGN (docs/adr/0301-llm-stack-gem-is-the-shared-llm-layer-pinned-per-plugin-from-forgejo.md) - Root candidate: none found; searched llm-stack, claude -p, ClaudeExecutor, ProcessRunner, ClaudeReply, ModelRequest, takedown - Where: plugins/os-sdlc/eval/ticket-takedown/lib/takedown/adapter/claude_executor.rb - Session: 375f16d0-a830-4297-b9f5-3acb6090a73f - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/375f16d0-a830-4297-b9f5-3acb6090a73f.jsonl VERDICT: CREATE REASON: Implements human-ruled ADR-0301 and spec step 3 for the takedown caller; no open ticket covers it.

jared added the

plugin/os-sdlc

filed-by/agent

category/code-change

labels

2026-10-27 18:11:58 +00:00

jared referenced this issue from a commit

2026-10-27 18:31:12 +00:00

docs: History row for #1437; spec build order marks step 2 done and names #1438 #1439

jared added the

ready-for-agent

label

2026-10-27 19:10:29 +00:00

jared commented

2026-10-27 19:11:10 +00:00

Author

Owner

Work started on branch ticket-1439 via /os-sdlc:implement (session 375f16d0). Gem prerequisites shipped: llm-stack a708188 (Response#meta, settings_json).

jared commented

2026-10-27 19:31:58 +00:00

Author

Owner

Resolution

Done: Takedown ClaudeExecutor calls LLM::Client#chat on a claude_code profile. ModelRequest#settings holds the step inputs. ClaudeReply.from_response reads text, usage, cost and meta from LLM::Response. The claude -p argv builder and JSON reply parser are deleted. plugins/os-sdlc/Gemfile pins llm-stack from Forgejo with a committed lockfile.

Evidence: Implementation 198 on branch ticket-1439, commit d6c7f1e7: change-auditor, seam-designer, test-writer, test-reviewer, programmer, behavior-verifier, suite-check, contract-auditor and refactorer all passed. Required-work checks green. Lens lines challenged: one code defect fixed by the refactorer, one heuristic holds.

Follow-ups: none. Gem follow-ups already filed as llm-stack #1-#4.

## Resolution **Done:** Takedown ClaudeExecutor calls LLM::Client#chat on a claude_code profile. ModelRequest#settings holds the step inputs. ClaudeReply.from_response reads text, usage, cost and meta from LLM::Response. The claude -p argv builder and JSON reply parser are deleted. plugins/os-sdlc/Gemfile pins llm-stack from Forgejo with a committed lockfile. **Evidence:** Implementation 198 on branch ticket-1439, commit d6c7f1e7: change-auditor, seam-designer, test-writer, test-reviewer, programmer, behavior-verifier, suite-check, contract-auditor and refactorer all passed. Required-work checks green. Lens lines challenged: one code defect fixed by the refactorer, one heuristic holds. **Follow-ups:** none. Gem follow-ups already filed as llm-stack #1-#4.

jared closed this issue

2026-10-27 19:31:58 +00:00

jared referenced this issue from a commit

2026-10-27 19:36:24 +00:00

feat(takedown): Audited POODR boundaries for the llm-stack ClaudeExecutor change; all seven WORK items present, contracts protected. (#1439)

jared referenced this issue from a commit

2026-10-27 19:36:24 +00:00

docs: History row and spec step 3 for #1439 takedown move onto llm-stack

jared referenced this issue from a commit

2026-10-27 19:36:24 +00:00

chore(os-sdlc): A10 lens signals from #1439 challenge