jared/cc-os

Watch

Fork

You've already forked cc-os

Code Issues 147 Pull requests Projects Releases Packages Wiki Activity

os-sdlc: run summary carries model and token usage read from dispatch transcripts #553

New issue

Closed

opened 2026-09-11 14:17:56 +00:00 by jared · 3 comments

jared commented

2026-09-11 14:17:56 +00:00

Owner

Copy link

Context

Follow-on to #552 (terminal payload run summary, merged 61995ac). The user wants model and tokens per dispatch in the end-of-run table. The runner never receives these: the SubagentStop hook event carries no usage, and the database has no model or token columns. Each dispatch belongs to an agent row (dispatches.agent_id), and the agent row owns transcript_path. The transcript JSONL holds assistant entries with message.model and message.usage (input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens).
Design decided with the user on 2026-09-11 (option C plus a report command): the model belongs to the agent, the tokens belong to the dispatch, both persisted. Stamping happens in a new implementation-report CLI command that the orchestrator runs after the last agent has returned, so every transcript is complete. The terminal payload from subagent-stop and next stays as shipped in #552. Same domain modeling as #552: objects named by what they summarize, each model responsible for its own value, no code comments.

Observed

Dispatch entries in the run summary carry id, agent_type, elapsed_s, verdict, stalled, gate_steps. No model, no tokens. No report command exists; the Thor CLI has implementation-open, next, subagent-start, subagent-stop, gate, violations.

Reproduce

n/a: feature ticket

Expected

os-sdlc-runner implementation-report SESSION_ID stamps usage on every dispatch of the session's latest implementation and prints the summary JSON: each dispatch entry gains model: and tokens: {in:, out:, cache_read:, cache_creation:}; totals gains tokens: with the same four keys summed. A dispatch whose transcript is missing or unreadable carries model: nil, tokens: nil and the command still succeeds. Running the command twice returns identical output.

Tasks

Schema 23 in db.rb: agents.model (String); dispatches.input_tokens, output_tokens, cache_read_tokens, cache_creation_tokens (Integer, nullable).
Add lib/os_sdlc/runner/transcript_usage.rb: class TranscriptUsage with Totals = Data.define(:model, :input, :output, :cache_read, :cache_creation); self.read(path) returns nil unless the path exists, else new(File.foreach(path)).totals; totals walks lines, parses JSON, skips lines that fail to parse, takes message from entries whose type is assistant, takes the first non-nil model, and sums the four usage keys.
Agent#usage returns TranscriptUsage.read(transcript_path); Agent#record_model!(model) updates the column when model is non-nil.
Dispatch#record_usage!: returns early if input_tokens is set; reads agent.usage; returns if nil; calls agent.record_model!; updates the four token columns.
Implementation#record_usage! calls record_usage! on each dispatch.
New Thor command implementation-report SESSION_ID in cli.rb: finds the session's latest implementation the same way next does, calls record_usage!, then emits Summary.for(Models.for(db).implementation[id]) as JSON. Follow the existing command-object pattern in lib/os_sdlc/runner/ (one class per command).
DispatchSummary#entry_for adds model: dispatch.agent.model and tokens: { in:, out:, cache_read:, cache_creation: } from the dispatch columns (nil when unstamped). DispatchSummary#token_totals sums the four across dispatches, nil-safe. Summary#totals adds tokens: @dispatch_summary.token_totals.
Tests: tests/runner/transcript_usage_test.rb with a fixture JSONL under tests/fixtures/ (two assistant messages with known usage, one malformed line, one non-assistant entry): sums, first model, missing file returns nil. A dispatch test: record_usage! stamps tokens and the agent model; a second call does not re-read. tests/runner/implementation/summary_test.rb: totals tokens sums two dispatches. A CLI test: implementation-report output carries model and tokens for a dispatch whose agent has a fixture transcript path, and is byte-identical on a repeated call.
skills/implement/SKILL.md: replace the #552 render line with this one unwrapped line: On any terminal payload, run \os-sdlc-runner implementation-report SESSION_ID` and render its `dispatches` as a table: one row per dispatch, followed by the `totals` line. If you observed workflow issues, list them below the table.`
New ADR via /os-adr:create: usage persisted on agent and dispatch rather than read lazily; stamped by the report command after the last agent returns rather than at subagent-stop or at terminal transition; transcript format knowledge confined to TranscriptUsage.
History row in docs/implementation-status/os-sdlc.md; refresh the plugin cache.

Acceptance criteria

A toy-map run to completion followed by implementation-report prints model and token figures per dispatch and a token total when transcripts exist.
A run whose transcript paths are absent still reports, with nil model and tokens.
Repeated implementation-report output is byte-identical.
Full suite green; rubocop clean on changed files.

Out of scope

Reload-after-stamp for CURRENT_TIMESTAMP columns (separate ticket). A node column on dispatches.

Blocking edges

Blocked by #552 (closed 2026-09-11).

Origin

Session: 5fdd0aa3-8a9c-4c31-a5b3-d575d8e709ab
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/5fdd0aa3-8a9c-4c31-a5b3-d575d8e709ab.jsonl
Where: plugins/os-sdlc/lib/os_sdlc/runner/cli.rb; plugins/os-sdlc/lib/os_sdlc/runner/implementation/dispatch_summary.rb; plugins/os-sdlc/lib/os_sdlc/runner/agent.rb; plugins/os-sdlc/lib/os_sdlc/runner/dispatch.rb
Filed by: user request, 2026-09-11. Body rewritten 2026-09-11 after design review.

## Context Follow-on to #552 (terminal payload run summary, merged 61995ac). The user wants model and tokens per dispatch in the end-of-run table. The runner never receives these: the SubagentStop hook event carries no usage, and the database has no model or token columns. Each dispatch belongs to an agent row (`dispatches.agent_id`), and the agent row owns `transcript_path`. The transcript JSONL holds assistant entries with `message.model` and `message.usage` (`input_tokens`, `output_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens`). Design decided with the user on 2026-09-11 (option C plus a report command): the model belongs to the agent, the tokens belong to the dispatch, both persisted. Stamping happens in a new `implementation-report` CLI command that the orchestrator runs after the last agent has returned, so every transcript is complete. The terminal payload from `subagent-stop` and `next` stays as shipped in #552. Same domain modeling as #552: objects named by what they summarize, each model responsible for its own value, no code comments. ## Observed Dispatch entries in the run summary carry id, agent_type, elapsed_s, verdict, stalled, gate_steps. No model, no tokens. No report command exists; the Thor CLI has implementation-open, next, subagent-start, subagent-stop, gate, violations. ## Reproduce n/a: feature ticket ## Expected `os-sdlc-runner implementation-report SESSION_ID` stamps usage on every dispatch of the session's latest implementation and prints the summary JSON: each dispatch entry gains `model:` and `tokens: {in:, out:, cache_read:, cache_creation:}`; `totals` gains `tokens:` with the same four keys summed. A dispatch whose transcript is missing or unreadable carries `model: nil, tokens: nil` and the command still succeeds. Running the command twice returns identical output. ## Tasks - [ ] Schema 23 in `db.rb`: `agents.model` (String); `dispatches.input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_creation_tokens` (Integer, nullable). - [ ] Add `lib/os_sdlc/runner/transcript_usage.rb`: class `TranscriptUsage` with `Totals = Data.define(:model, :input, :output, :cache_read, :cache_creation)`; `self.read(path)` returns nil unless the path exists, else `new(File.foreach(path)).totals`; `totals` walks lines, parses JSON, skips lines that fail to parse, takes `message` from entries whose `type` is `assistant`, takes the first non-nil `model`, and sums the four usage keys. - [ ] `Agent#usage` returns `TranscriptUsage.read(transcript_path)`; `Agent#record_model!(model)` updates the column when model is non-nil. - [ ] `Dispatch#record_usage!`: returns early if `input_tokens` is set; reads `agent.usage`; returns if nil; calls `agent.record_model!`; updates the four token columns. - [ ] `Implementation#record_usage!` calls `record_usage!` on each dispatch. - [ ] New Thor command `implementation-report SESSION_ID` in `cli.rb`: finds the session's latest implementation the same way `next` does, calls `record_usage!`, then emits `Summary.for(Models.for(db).implementation[id])` as JSON. Follow the existing command-object pattern in `lib/os_sdlc/runner/` (one class per command). - [ ] `DispatchSummary#entry_for` adds `model: dispatch.agent.model` and `tokens: { in:, out:, cache_read:, cache_creation: }` from the dispatch columns (nil when unstamped). `DispatchSummary#token_totals` sums the four across dispatches, nil-safe. `Summary#totals` adds `tokens: @dispatch_summary.token_totals`. - [ ] Tests: `tests/runner/transcript_usage_test.rb` with a fixture JSONL under `tests/fixtures/` (two assistant messages with known usage, one malformed line, one non-assistant entry): sums, first model, missing file returns nil. A dispatch test: `record_usage!` stamps tokens and the agent model; a second call does not re-read. `tests/runner/implementation/summary_test.rb`: totals tokens sums two dispatches. A CLI test: `implementation-report` output carries `model` and `tokens` for a dispatch whose agent has a fixture transcript path, and is byte-identical on a repeated call. - [ ] `skills/implement/SKILL.md`: replace the #552 render line with this one unwrapped line: `On any terminal payload, run \`os-sdlc-runner implementation-report SESSION_ID\` and render its \`dispatches\` as a table: one row per dispatch, followed by the \`totals\` line. If you observed workflow issues, list them below the table.` - [ ] New ADR via `/os-adr:create`: usage persisted on agent and dispatch rather than read lazily; stamped by the report command after the last agent returns rather than at subagent-stop or at terminal transition; transcript format knowledge confined to TranscriptUsage. - [ ] History row in `docs/implementation-status/os-sdlc.md`; refresh the plugin cache. ## Acceptance criteria - [ ] A toy-map run to completion followed by `implementation-report` prints model and token figures per dispatch and a token total when transcripts exist. - [ ] A run whose transcript paths are absent still reports, with nil model and tokens. - [ ] Repeated `implementation-report` output is byte-identical. - [ ] Full suite green; rubocop clean on changed files. ## Out of scope Reload-after-stamp for CURRENT_TIMESTAMP columns (separate ticket). A node column on dispatches. ## Blocking edges Blocked by #552 (closed 2026-09-11). ## Origin Session: 5fdd0aa3-8a9c-4c31-a5b3-d575d8e709ab Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/5fdd0aa3-8a9c-4c31-a5b3-d575d8e709ab.jsonl Where: plugins/os-sdlc/lib/os_sdlc/runner/cli.rb; plugins/os-sdlc/lib/os_sdlc/runner/implementation/dispatch_summary.rb; plugins/os-sdlc/lib/os_sdlc/runner/agent.rb; plugins/os-sdlc/lib/os_sdlc/runner/dispatch.rb Filed by: user request, 2026-09-11. Body rewritten 2026-09-11 after design review.

jared added the

plugin/os-sdlc

filed-by/user

labels

2026-09-11 14:17:56 +00:00

jared referenced this issue

2026-09-11 14:49:30 +00:00

os-sdlc: terminal payload carries a deterministic run summary #552

jared commented

2026-09-11 15:43:00 +00:00

Author

Owner

Copy link

Work started via /os-sdlc:implement on branch ticket-553 (session 5fdd0aa3). Body rewritten to the option C + implementation-report design first.

jared referenced this issue

2026-09-11 16:16:14 +00:00

os-sdlc: behavior-verifier judges from ticket_brief and code_changes only; Bash-only; complains loudly on missing evidence #555

jared commented

2026-09-11 17:01:56 +00:00

Author

Owner

Copy link

Resolution

Done: Model persisted on agents.model_id, tokens on four dispatch columns (schema 23); TranscriptUsage reads the transcript; implementation-report CLI stamps usage once after the last agent returns and prints the summary with model, tokens, and a tokens total; SKILL.md updated; ADR-0174; history row.

Evidence: Commit 9252987 on branch ticket-553; full os-sdlc suite 1187 runs, 0 failures, 0 errors; rubocop clean on changed files. Pipeline: implementation 3 ended on a test-reviewer error verdict (commented on #555), implementation 4 ran four red-green increments then escalated with unexpected_pass; docs, model_id rename (Sequel collision), DispatchEntry fold, and a no-re-read test finished by hand under the escalation procedure. Live check of the report command against the shared DB wiped it (schema bump, ADR-0129); lesson recorded in memory.

Follow-ups: #557 reload-after-stamp; test-reviewer error-verdict route noted on #555; none other

## Resolution **Done:** Model persisted on agents.model_id, tokens on four dispatch columns (schema 23); TranscriptUsage reads the transcript; implementation-report CLI stamps usage once after the last agent returns and prints the summary with model, tokens, and a tokens total; SKILL.md updated; ADR-0174; history row. **Evidence:** Commit 9252987 on branch ticket-553; full os-sdlc suite 1187 runs, 0 failures, 0 errors; rubocop clean on changed files. Pipeline: implementation 3 ended on a test-reviewer error verdict (commented on #555), implementation 4 ran four red-green increments then escalated with unexpected_pass; docs, model_id rename (Sequel collision), DispatchEntry fold, and a no-re-read test finished by hand under the escalation procedure. Live check of the report command against the shared DB wiped it (schema bump, ADR-0129); lesson recorded in memory. **Follow-ups:** #557 reload-after-stamp; test-reviewer error-verdict route noted on #555; none other

jared closed this issue

2026-09-11 17:01:56 +00:00

jared commented

2026-09-11 17:06:13 +00:00

Author

Owner

Copy link

Merged to main at 69fc891 (fast-forward via ticket-553 merge). Correction to the close comment: the ADR is ADR-0175, renumbered after a collision with #558's ADR-0174.

Merged to main at 69fc891 (fast-forward via ticket-553 merge). Correction to the close comment: the ADR is ADR-0175, renumbered after a collision with #558's ADR-0174.

jared referenced this issue from a commit

2026-09-11 17:41:24 +00:00

feat(os-sdlc): terminal payload carries a deterministic run summary (#552)

jared referenced this issue from a commit

2026-09-11 17:41:24 +00:00

feat(os-sdlc): run summary carries model and tokens via implementation-report (#553)

jared referenced this issue from a commit

2026-09-11 17:41:24 +00:00

docs(adr): renumber #553 usage ADR to 0175 after collision with #558

jared referenced this issue

2026-09-14 12:08:10 +00:00

implementation-report records 0 tokens for some implementations (impl 2 and 3 of the #568 batch) #569

No Branch/Tag specified

main

No results found.

Labels

Clear labels

Priority 0 — drop everything

Ticket creates something new

delete

Ticket removes something

enhancement

New feature or improvement

filed-by/agent

filed-by/user

frozen

Pre-reset backlog freeze 2026-08-16; do not work unless rediscovered by a live run

lint-rule

Ticket proposes/modifies a lint rule or cop

needs-info

Waiting on reporter for more information

needs-triage

Maintainer needs to evaluate this issue

Human-curated next up — AI never applies or removes

plugin/cc-architect

Scope: the cc-architect plugin

plugin/os

Scope: the os plugin

plugin/os-adr

Scope: the os-adr plugin

plugin/os-aidd-lint

Scope: the os-aidd-lint plugin

plugin/os-backlog

Scope: the os-backlog plugin

plugin/os-context

Scope: the os-context plugin

plugin/os-doc-hygiene

Scope: the os-doc-hygiene plugin

plugin/os-sdlc

Scope: the os-sdlc plugin

plugin/os-vault

Scope: the os-vault plugin

project/cc-os

Parent-scope: cc-os repo-wide, not specific to one plugin

ready-for-agent

Triage complete; an agent can pick this up

ready-for-human

Human-owned; never picked up autonomously

recurring

Recurring flavor: occurrence outcomes commented, never closed by AI

review

Shipped semi work awaiting human sign-off

update

Ticket updates something existing

waiting

Blocked; blocker named in a comment

wayfinder map child: task

wontfix

Will not be actioned

worklist/deviations

Deviation incidents from skill/pipeline runs — evidence pile for correcting the owning artifact

worklist/lint-rule

Proposed os-sdlc or os-aidd-lint rule

worklist/new-implement-build

Updated implementation harness (ast-grep + stepper)

No labels

plugin/os-doc-hygiene

worklist/new-implement-build

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

jared/cc-os#553

Reference in a new issue

Repository

jared/cc-os

Title

Body

No description provided.

Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?

Rows
Columns