jared/cc-os

Watch

Fork 0

Use default price table in os-me headless runner and read meta[reported_cost] #1477

New issue

Closed

opened 2026-10-30 21:21:24 +00:00 by jared · 2 comments

jared commented

2026-10-30 21:21:24 +00:00

Owner

OsMe::HeadlessRunner passes an empty price table so that response.cost keeps the claude -p CLI's API-rate estimate. Use the default price table and read the estimate from meta["reported_cost"] instead.

Problem

Price table	`response.cost`	`response.meta["reported_cost"]`	`HeadlessResult#cost_usd`
Now: empty `PriceTable.new({})`	`0.0123` (CLI estimate)	not there	`0.0123`
Default table, no other change	`0.0` (subscription rate)	`0.0123`	`0.0` (estimate lost)
Default table + read `meta` (this ticket)	`0.0`	`0.0123`	`0.0123`

Do not stop at the middle row. If only the empty table goes, cost_usd becomes 0.0 and Report#cost_usd (plugins/os-me/lib/os_me/report.rb:35) and ScoreLog (plugins/os-me/lib/os_me/score_log.rb:16) record 0 for every run.

Evidence

plugins/os-me/lib/os_me/headless_runner.rb:6   REPORTED_COSTS = LLM::PriceTable.new({})
plugins/os-me/lib/os_me/headless_runner.rb:23  credentials: {}, prices: REPORTED_COSTS)
plugins/os-me/lib/os_me/headless_runner.rb:27  response => LLM::Response[..., cost:]
plugins/os-me/lib/os_me/headless_runner.rb:29  cost_usd: cost, ...

llm-stack 963bf56 (lib/llm/client.rb) keeps a replaced provider cost in meta:

# before
return result.with(cost: rate.cost(result.usage)) if rate
# after
return result.with(cost: rate.cost(result.usage), meta: with_reported(result.meta, result.cost)) if rate

Approach

Required work:

Bump llm-stack in plugins/os-me to a revision that includes 963bf56. The current lock pin, 88b5f17 (plugins/os-me/Gemfile.lock:11), does not include it (checked with git merge-base --is-ancestor).
Remove REPORTED_COSTS and the prices: argument, so the client uses the default price table.
In result_from, take cost_usd from meta.fetch('reported_cost'). Use fetch, not []: when the CLI omits total_cost_usd the key is absent, and the run must fail at this call, not later in Report#cost_usd's sum.
Move OsMe::OllamaEmbedder (plugins/os-me/lib/os_me/ollama_embedder.rb) onto the one-profile client. Every llm-stack revision that includes 963bf56 also carries llm-stack #6: LLM::Client.new takes profile: (one LLM::Profile), not profiles:, and the gem has no LLM.configure or LLM.embed. The embedder holds an LLM::Client built from LLM::Profile.find(provider: 'ollama', model: 'nomic-embed-text') (the gem catalog has this model) and calls embed(texts:) on it. The test stub plugins/os-me/test/llm_embed_stub.rb replaces LLM.embed, so it changes with the embedder.

Owner sketch

OsMe::HeadlessRunner stays the owner. It names the value it wants (meta["reported_cost"]) instead of a pricing side effect.

def client(request)
  LLM::Client.new(profiles: { PROFILE => request.profile(executable: @executable, timeout: @timeout) },
                  credentials: {})
end

def result_from(response, schema)
  response => LLM::Response[text:, usage: { input:, output:, cache_read:, cache_write: }, meta:]
  HeadlessResult.new(text: text, input_tokens: input + cache_read + cache_write, output_tokens: output,
                     cost_usd: meta.fetch('reported_cost'), structured_output: structured_from(text, schema))
end

module OsMe
  class OllamaEmbedder
    def self.standard = new(client: LLM::Client.new(profile: LLM::Profile.find(provider: 'ollama', model: 'nomic-embed-text')))

    def initialize(client:)
      @client = client
    end

    def embed(texts) = @client.embed(texts: texts).vectors
  end
end

Acceptance criteria

os-me's llm-stack lock pin includes 963bf56.
headless_runner.rb has no PriceTable reference.
A test shows HeadlessResult#cost_usd equals the CLI's total_cost_usd under subscription billing, where response.cost is 0.0.
A test shows a reply without total_cost_usd raises KeyError from the runner.
OsMe::OllamaEmbedder embeds through an LLM::Client built with profile:; os-me has no LLM.configure or LLM.embed reference.
The full os-me suite passes on the bumped llm-stack.

Discoverer: /home/jared/dev/ruby-gems/llm-stack, session 45db010a-a574-43ed-b4ff-502b3da5677d, 2026-10-30. Found while working llm-stack#14.

`OsMe::HeadlessRunner` passes an empty price table so that `response.cost` keeps the `claude -p` CLI's API-rate estimate. Use the default price table and read the estimate from `meta["reported_cost"]` instead. ## Problem | Price table | `response.cost` | `response.meta["reported_cost"]` | `HeadlessResult#cost_usd` | | --- | --- | --- | --- | | Now: empty `PriceTable.new({})` | `0.0123` (CLI estimate) | not there | `0.0123` | | Default table, no other change | `0.0` (subscription rate) | `0.0123` | **`0.0`** (estimate lost) | | Default table + read `meta` (this ticket) | `0.0` | `0.0123` | `0.0123` | Do not stop at the middle row. If only the empty table goes, `cost_usd` becomes 0.0 and `Report#cost_usd` (`plugins/os-me/lib/os_me/report.rb:35`) and `ScoreLog` (`plugins/os-me/lib/os_me/score_log.rb:16`) record 0 for every run. ## Evidence ``` plugins/os-me/lib/os_me/headless_runner.rb:6 REPORTED_COSTS = LLM::PriceTable.new({}) plugins/os-me/lib/os_me/headless_runner.rb:23 credentials: {}, prices: REPORTED_COSTS) plugins/os-me/lib/os_me/headless_runner.rb:27 response => LLM::Response[..., cost:] plugins/os-me/lib/os_me/headless_runner.rb:29 cost_usd: cost, ... ``` llm-stack `963bf56` (`lib/llm/client.rb`) keeps a replaced provider cost in `meta`: ```ruby # before return result.with(cost: rate.cost(result.usage)) if rate # after return result.with(cost: rate.cost(result.usage), meta: with_reported(result.meta, result.cost)) if rate ``` ## Approach Required work: 1. Bump llm-stack in `plugins/os-me` to a revision that includes `963bf56`. The current lock pin, `88b5f17` (`plugins/os-me/Gemfile.lock:11`), does not include it (checked with `git merge-base --is-ancestor`). 2. Remove `REPORTED_COSTS` and the `prices:` argument, so the client uses the default price table. 3. In `result_from`, take `cost_usd` from `meta.fetch('reported_cost')`. Use `fetch`, not `[]`: when the CLI omits `total_cost_usd` the key is absent, and the run must fail at this call, not later in `Report#cost_usd`'s `sum`. 4. Move `OsMe::OllamaEmbedder` (`plugins/os-me/lib/os_me/ollama_embedder.rb`) onto the one-profile client. Every llm-stack revision that includes `963bf56` also carries llm-stack #6: `LLM::Client.new` takes `profile:` (one `LLM::Profile`), not `profiles:`, and the gem has no `LLM.configure` or `LLM.embed`. The embedder holds an `LLM::Client` built from `LLM::Profile.find(provider: 'ollama', model: 'nomic-embed-text')` (the gem catalog has this model) and calls `embed(texts:)` on it. The test stub `plugins/os-me/test/llm_embed_stub.rb` replaces `LLM.embed`, so it changes with the embedder. ## Owner sketch `OsMe::HeadlessRunner` stays the owner. It names the value it wants (`meta["reported_cost"]`) instead of a pricing side effect. ```ruby def client(request) LLM::Client.new(profiles: { PROFILE => request.profile(executable: @executable, timeout: @timeout) }, credentials: {}) end def result_from(response, schema) response => LLM::Response[text:, usage: { input:, output:, cache_read:, cache_write: }, meta:] HeadlessResult.new(text: text, input_tokens: input + cache_read + cache_write, output_tokens: output, cost_usd: meta.fetch('reported_cost'), structured_output: structured_from(text, schema)) end ``` ```ruby module OsMe class OllamaEmbedder def self.standard = new(client: LLM::Client.new(profile: LLM::Profile.find(provider: 'ollama', model: 'nomic-embed-text'))) def initialize(client:) @client = client end def embed(texts) = @client.embed(texts: texts).vectors end end ``` ## Acceptance criteria - [ ] os-me's llm-stack lock pin includes `963bf56`. - [ ] `headless_runner.rb` has no `PriceTable` reference. - [ ] A test shows `HeadlessResult#cost_usd` equals the CLI's `total_cost_usd` under subscription billing, where `response.cost` is 0.0. - [ ] A test shows a reply without `total_cost_usd` raises `KeyError` from the runner. - [ ] `OsMe::OllamaEmbedder` embeds through an `LLM::Client` built with `profile:`; os-me has no `LLM.configure` or `LLM.embed` reference. - [ ] The full os-me suite passes on the bumped llm-stack. -------- **Discoverer:** /home/jared/dev/ruby-gems/llm-stack, session 45db010a-a574-43ed-b4ff-502b3da5677d, 2026-10-30. Found while working llm-stack#14.

jared added the

ready-for-agent

label

2026-10-30 21:28:01 +00:00

jared commented

2026-10-30 21:31:09 +00:00

Author

Owner

Work started on branch ticket-1477 through the os-sdlc pipeline (poodr-ticket-implementation).

Work started on branch `ticket-1477` through the os-sdlc pipeline (poodr-ticket-implementation).

jared added the

category/code-change

label

2026-10-30 21:31:26 +00:00

jared commented

2026-10-30 21:50:15 +00:00

Author

Owner

Resolution

Done: HeadlessRunner uses the default price table and reads cost_usd from meta.fetch('reported_cost'); OllamaEmbedder holds a one-profile LLM::Client (llm-stack #6 interface); os-me pins llm-stack 3109af7, which includes 963bf56.

Evidence: Commits f6100a56 (pipeline implementation 216, all gates pass, os-me suite green) and e1f9b03d (status leaf) on branch ticket-1477, merged to main. Tests: headless_runner_cost_test.rb (cost_usd = CLI total_cost_usd under subscription billing; KeyError without total_cost_usd), ollama_embedder_test.rb (client built with the nomic profile).

Follow-ups: none

## Resolution **Done:** HeadlessRunner uses the default price table and reads cost_usd from meta.fetch('reported_cost'); OllamaEmbedder holds a one-profile LLM::Client (llm-stack #6 interface); os-me pins llm-stack 3109af7, which includes 963bf56. **Evidence:** Commits f6100a56 (pipeline implementation 216, all gates pass, os-me suite green) and e1f9b03d (status leaf) on branch ticket-1477, merged to main. Tests: headless_runner_cost_test.rb (cost_usd = CLI total_cost_usd under subscription billing; KeyError without total_cost_usd), ollama_embedder_test.rb (client built with the nomic profile). **Follow-ups:** none

jared closed this issue

2026-10-30 21:50:16 +00:00

jared referenced this issue from a commit

2026-10-30 21:51:28 +00:00

feat(os-me): Audited POODR boundaries for #1477; HeadlessRunner reads meta reported_cost and OllamaEmbedder holds a one-profile LLM::Client; all WORK items present; pass. (#1477)

jared referenced this issue from a commit

2026-10-30 21:51:28 +00:00

docs(os-me): record #1477 (llm-stack 3109af7, reported_cost, one-profile embedder) in status leaf