Spec: os-sdlc rebuild phase 2 — hook-driven runner tracer-bullet skeleton (ADR-0108/0109) #327

Closed
opened 2026-08-07 17:27:29 +00:00 by jared · 4 comments
Owner

Spec: os-sdlc rebuild phase 2 — hook-driven runner tracer-bullet skeleton

Governing decisions: ADR-0108 (hook-driven runner), ADR-0109 (SQLite+Sequel state, briefs by path). Design: docs/os-sdlc-rebuild/ (architecture.md, runner.md, dispatch-hook.md, dr-port-map.md). Delta Refinery (~/dev/delta-refinery @ 4511438) is the template per the port map — its PORT/ADAPT/SKIP verdicts are binding. This spec supersedes the earlier PRD text of this issue; it integrates the accepted findings of the 2026-08-07 Codex second-opinion review (dispatch correlation; honest DB rationale; one stale doc claim). Rejected review items (interface freeze "premature", hook timeout sizing, named recovery CLI) stay rejected as speculative hardening.

Problem Statement

The old implement pipeline burned ~20 orchestrator rounds per ticket relaying actions between an LLM and a stepper CLI, with token cost compounding per round. Phase 1 deleted it. There is currently no implement pipeline, and the replacement's load-bearing mechanism — a PostToolUse hook on the Agent tool injecting the next action as additionalContext — has never been run by anyone.

Solution

A walking skeleton with the production interfaces and a two-state toy pipeline: the orchestrator runs intake <ticket>, dispatches test-writer, the hook fires on the subagent's return, the Runner confirms the return matches the dispatch it is waiting on, records state in SQLite, writes the programmer brief file, and injects the next dispatch; programmer returns the same way; the pipeline reaches done. One live session proves the channel; every later state is added behind interfaces that never change again.

User Stories

  1. As the user, I want the orchestrator's whole job to be "dispatch the named agent with the named brief path", so that a haiku-tier session can run the pipeline without burning frontier-model tokens on relay work.
  2. As the user, I want the next pipeline action to arrive via hook-injected additionalContext when a subagent returns, so that no orchestrator round is spent polling or reporting completions.
  3. As the user, I want the Runner to act only when a subagent return matches the dispatch it recorded, so that an unrelated Agent call (an Explore lookup, a second ticket, a parallel dispatch) can never advance pipeline state.
  4. As the user, I want the Runner to execute gates itself and route on exit codes, so that no routing decision ever depends on LLM free text.
  5. As the user, I want pipeline state in a per-project SQLite database, so that state survives the process boundary between hook invocations and dispatch correlation has a durable home.
  6. As the user, I want every transition recorded as an append-only timestamped row, so that history and the eventual done-report timeline fall out of the schema instead of bespoke code.
  7. As the user, I want briefs written as markdown files dispatched by path, so that ticket text never transits orchestrator context.
  8. As the user, I want the implement SKILL.md written in the house pseudoskill style with steps so plain a bright eleven-year-old could follow them, so that any model tier can orchestrate without interpretation.
  9. As the user, I want a stall to be loud (a raised NoProgressError, a visibly stuck session), so that a hook failure is diagnosable rather than silently absorbed.
  10. As the user, I want the Runner testable at the hook contract (event JSON in, additionalContext JSON out), so that the whole pipeline — including the ignore-unmatched-event path — is exercisable without a live Claude session.
  11. As a future session, I want the transition table as plain validated data with a gate column, so that adding the real states (red-assert, lint, review) is adding rows, not code.
  12. As a future session, I want thin Sequel wrappers over the state rows, so that fetch/create of pipeline data is one obvious call.
  13. As the user, I want the skeleton to reuse Delta Refinery's ported StateMachine and test patterns, so that we do not reinvent tested code.

Implementation Decisions

  • New runner lives in the os-sdlc plugin alongside the kept gate CLI; the kept gate entrypoints and their 653-run suite are untouched by this phase.

  • Dispatch correlation (from the Codex review — a guaranteed breakage point, not hardening). PostToolUse fires on every successful Agent tool call, pipeline-related or not. When the Runner emits a dispatch it records the expected return as a row (a dispatches table: ticket, agent name, brief path, status pending/returned, timestamps — this is the handoffs table ADR-0109 deferred; the correlation requirement earns it now). The hook advances state only when the completed Agent call matches the pending dispatch row (match on the strongest identity the hook payload provides — tool_use_id if present, else agent type — determined empirically during the build and recorded in dispatch-hook.md); any non-matching event, and any event when zero or more-than-one pending dispatch row exists, is a silent no-op: no state change, no rows, no injected action. A lookup that can return the wrong row is a correctness bug; refusing to act on ambiguity is the KISS totality rule, not speculation.

  • StateMachine: ported near-verbatim from DR — constructor validation, transition! raising on illegal edges, fail! restricted to terminal states, history.

  • Transition table: plain data adapted from DR's LevelConfig shape — parallel hashes plus constructor validate!, a gate/command column per row, single success terminal done, no level plural machinery. Skeleton table is two rows: test-writer → programmer → done, with stub gates. Retry routes and their validate! rule land with the first real retry state (next phase) — not built into a table that has no retry row.

  • Runner: one iteration per hook invocation (inverted from DR's blocking loop). Body: confirm the event matches the pending dispatch → execute the current state's gate(s) via Open3.capture3, route on status.exitstatus → record the transition row → write the next brief file → record the new pending dispatch → emit exactly one action. assert_progress! ported (raise if a matched agent return produces no state change).

  • Persistence: per-project SQLite via Sequel; schema tickets, transitions (append-only, DB timestamps), dispatches, and retry_counts (added with the first retry state; counters are per-(state, cause), reset only on forward exit, never keyed on object identity — DR wart). The transitions table IS StateMachine#history persisted. Gems sequel + sqlite3 are accepted plugin dependencies. The honest justification for the DB is re-entrant process-boundary state plus dispatch correlation, not file-rewrite cost (ADR-0109 rationale amended per review).

  • Runner locates its ticket by the single non-terminal ticket row for the cwd; zero or multiple rows → no-op (see correlation decision). No active-marker file, no fallback instructions, no bespoke timeout handling (rejected per ADR-0108).

  • Hook: PostToolUse matched on the Agent tool; contract is hook-event JSON on stdin → additionalContext JSON on stdout (or no output for the no-op path). The hook is the completion report. Registered in the plugin's hook manifest; bin/refresh-plugins after landing.

  • Intake: intake <ticket> creates the ticket row, writes the first brief, records the first pending dispatch, prints the first action.

  • Briefs: markdown files under the ticket's briefs directory in .sdlc/; dispatch actions carry {agent, brief_path, optional output_file}. Rule: structured multi-writer state → rows; prose with one LLM reader → files referenced by path from rows.

  • Action vocabulary frozen at three shapes: dispatch {agent, brief_path, optional output_file}, ask_human {question, skippable}, done {report path}. The freeze is deliberate interface-first design; the live shakedown validates it before anything leans on it.

  • SKILL.md: production interface, written now in the pseudoskill style (vault: howto/writing-pseudocode-style-skills.md — numbered phases, intent bullets with command sub-bullets, declared loop exit, branch per action type). One structural update to the style: there is no report-and-fetch tail call, because the orchestrator never fetches — actions arrive by injection. The skill body is, in essence (snippet encodes the decision; final wording at build time):

    ## 1. Start
    - Get the ticket number
      - none given -> stop and ask the user
    - Start the pipeline
      - run `os-sdlc intake <ticket>`
      - it prints the first action
    
    ## 2. Loop until the action is done
    **Switch on the action type:**
    - If the action is dispatch:
      - call the Agent tool with the named agent and the brief path as its prompt
      - when it finishes, the next action appears in your context by itself — do not run anything to fetch it
    - If the action is ask_human:
      - ask the user the question, then follow the answer instruction the action carries
    - If the action is done:
      - tell the user the report path. You are finished.
    
    You never run tests, never read briefs, never judge agent output.
    
  • No recovery analyst, no free-text route derivation, no JSON brief representation (SKIP verdicts in the port map are binding).

  • Doc corrections landing with this phase: dispatch-hook.md's "a code loop can't lose its place" idempotency claim (describes the deleted in-process design) rewritten for the re-entrant reality; ADR-0109 amended for the dispatches table + honest rationale.

Testing Decisions

Good tests assert external behavior at the highest seam: given a hook event and a DB state, the emitted action and the resulting rows — never Runner internals.

  • Primary seam — the hook contract: fake PostToolUse Agent-tool event JSON on stdin; assert the additionalContext action emitted and the rows written. The full skeleton path (intake → dispatch → hook → dispatch → hook → done) runs as a test at this seam.
  • The correlation no-op paths are tested at the same seam: a non-matching agent event, an event with no pending dispatch, and an event with two non-terminal tickets each produce no output and no row changes.
  • Gates in tests are stub scripts with controlled exit codes; routing-on-exitstatus asserted without a real project.
  • DB per test is in-memory SQLite; assertions read rows.
  • Unit seam for StateMachine + table: DR's ported tests, one fixture table per test class, every loop test wrapped in Timeout.timeout.
  • Harness patterns ported from DR: stdout capture in test_helper, named fakes (fake gate results, spy dispatch sink).
  • Prior art: the kept CLI-subcommand suite in the plugin's tests/ directory.
  • A state-divergence regression test (DB state vs on-disk brief reality) is written at skeleton time, per the DR persistence-bug lesson.
  • The live-session shakedown is the phase exit criterion, performed manually once: the injected dispatch reaching the orchestrator, an unrelated Explore dispatch mid-pipeline NOT advancing state, and done surfacing. It is the only behavior the harness cannot reach.

Out of Scope

  • The real pipeline states — red-assert, lint/autocorrect/lint-fixer loop, reviewer verdict, escalation caps — are the follow-up spec; internally they will land in risk order (red/green vertical with retry first, then lint loop, then reviewer, then escalation/teardown).
  • Retry routes, retry_counts, and their validate! rule (land with the first retry state).
  • PostToolUseFailure handling (Agent call fails rather than completes) — recorded as an Open item in dispatch-hook.md; built only if the shakedown or a live run shows it.
  • Sharding, the claims table, parallel dispatch (deferred per ADR-0108/0109).
  • Migrating the kept gate CLI or Artifact::StepperState onto the new schema (bridge stays until the tickets table replaces artifact serialization).
  • Brief bodies in the database.
  • Stale stepper references in plugins/os-sdlc/CLAUDE.md and old planning docs (separate doc pass).

Further Notes

  • Style: TDD, minitest, Sandi Metz OOP Ruby. Skeleton first as a vertical slice; nothing built ahead of a row that needs it.
  • Port verdicts and file-level guidance: docs/os-sdlc-rebuild/dr-port-map.md; confirmed DR warts (object_id retry keys, fail-open verdict parsing, #call/#run mismatch) must not survive the port.
  • After any plugin source edit lands, bin/refresh-plugins must run before a live session can see it.
# Spec: os-sdlc rebuild phase 2 — hook-driven runner tracer-bullet skeleton Governing decisions: ADR-0108 (hook-driven runner), ADR-0109 (SQLite+Sequel state, briefs by path). Design: `docs/os-sdlc-rebuild/` (architecture.md, runner.md, dispatch-hook.md, dr-port-map.md). Delta Refinery (`~/dev/delta-refinery` @ `4511438`) is the template per the port map — its PORT/ADAPT/SKIP verdicts are binding. This spec supersedes the earlier PRD text of this issue; it integrates the accepted findings of the 2026-08-07 Codex second-opinion review (dispatch correlation; honest DB rationale; one stale doc claim). Rejected review items (interface freeze "premature", hook timeout sizing, named recovery CLI) stay rejected as speculative hardening. ## Problem Statement The old implement pipeline burned ~20 orchestrator rounds per ticket relaying actions between an LLM and a stepper CLI, with token cost compounding per round. Phase 1 deleted it. There is currently no implement pipeline, and the replacement's load-bearing mechanism — a PostToolUse hook on the Agent tool injecting the next action as additionalContext — has never been run by anyone. ## Solution A walking skeleton with the production interfaces and a two-state toy pipeline: the orchestrator runs `intake <ticket>`, dispatches test-writer, the hook fires on the subagent's return, the Runner confirms the return matches the dispatch it is waiting on, records state in SQLite, writes the programmer brief file, and injects the next dispatch; programmer returns the same way; the pipeline reaches `done`. One live session proves the channel; every later state is added behind interfaces that never change again. ## User Stories 1. As the user, I want the orchestrator's whole job to be "dispatch the named agent with the named brief path", so that a haiku-tier session can run the pipeline without burning frontier-model tokens on relay work. 2. As the user, I want the next pipeline action to arrive via hook-injected additionalContext when a subagent returns, so that no orchestrator round is spent polling or reporting completions. 3. As the user, I want the Runner to act only when a subagent return matches the dispatch it recorded, so that an unrelated Agent call (an Explore lookup, a second ticket, a parallel dispatch) can never advance pipeline state. 4. As the user, I want the Runner to execute gates itself and route on exit codes, so that no routing decision ever depends on LLM free text. 5. As the user, I want pipeline state in a per-project SQLite database, so that state survives the process boundary between hook invocations and dispatch correlation has a durable home. 6. As the user, I want every transition recorded as an append-only timestamped row, so that history and the eventual done-report timeline fall out of the schema instead of bespoke code. 7. As the user, I want briefs written as markdown files dispatched by path, so that ticket text never transits orchestrator context. 8. As the user, I want the implement SKILL.md written in the house pseudoskill style with steps so plain a bright eleven-year-old could follow them, so that any model tier can orchestrate without interpretation. 9. As the user, I want a stall to be loud (a raised NoProgressError, a visibly stuck session), so that a hook failure is diagnosable rather than silently absorbed. 10. As the user, I want the Runner testable at the hook contract (event JSON in, additionalContext JSON out), so that the whole pipeline — including the ignore-unmatched-event path — is exercisable without a live Claude session. 11. As a future session, I want the transition table as plain validated data with a gate column, so that adding the real states (red-assert, lint, review) is adding rows, not code. 12. As a future session, I want thin Sequel wrappers over the state rows, so that fetch/create of pipeline data is one obvious call. 13. As the user, I want the skeleton to reuse Delta Refinery's ported StateMachine and test patterns, so that we do not reinvent tested code. ## Implementation Decisions - New runner lives in the os-sdlc plugin alongside the kept gate CLI; the kept gate entrypoints and their 653-run suite are untouched by this phase. - **Dispatch correlation (from the Codex review — a guaranteed breakage point, not hardening).** PostToolUse fires on every successful Agent tool call, pipeline-related or not. When the Runner emits a dispatch it records the expected return as a row (a `dispatches` table: ticket, agent name, brief path, status pending/returned, timestamps — this is the `handoffs` table ADR-0109 deferred; the correlation requirement earns it now). The hook advances state only when the completed Agent call matches the pending dispatch row (match on the strongest identity the hook payload provides — tool_use_id if present, else agent type — determined empirically during the build and recorded in dispatch-hook.md); any non-matching event, and any event when zero or more-than-one pending dispatch row exists, is a silent no-op: no state change, no rows, no injected action. A lookup that can return the wrong row is a correctness bug; refusing to act on ambiguity is the KISS totality rule, not speculation. - StateMachine: ported near-verbatim from DR — constructor validation, `transition!` raising on illegal edges, `fail!` restricted to terminal states, `history`. - Transition table: plain data adapted from DR's LevelConfig shape — parallel hashes plus constructor `validate!`, a gate/command column per row, single success terminal `done`, no level plural machinery. Skeleton table is two rows: test-writer → programmer → done, with stub gates. Retry routes and their validate! rule land with the first real retry state (next phase) — not built into a table that has no retry row. - Runner: one iteration per hook invocation (inverted from DR's blocking loop). Body: confirm the event matches the pending dispatch → execute the current state's gate(s) via `Open3.capture3`, route on `status.exitstatus` → record the transition row → write the next brief file → record the new pending dispatch → emit exactly one action. `assert_progress!` ported (raise if a matched agent return produces no state change). - Persistence: per-project SQLite via Sequel; schema `tickets`, `transitions` (append-only, DB timestamps), `dispatches`, and `retry_counts` (added with the first retry state; counters are per-(state, cause), reset only on forward exit, never keyed on object identity — DR wart). The transitions table IS StateMachine#history persisted. Gems `sequel` + `sqlite3` are accepted plugin dependencies. The honest justification for the DB is re-entrant process-boundary state plus dispatch correlation, not file-rewrite cost (ADR-0109 rationale amended per review). - Runner locates its ticket by the single non-terminal ticket row for the cwd; zero or multiple rows → no-op (see correlation decision). No active-marker file, no fallback instructions, no bespoke timeout handling (rejected per ADR-0108). - Hook: PostToolUse matched on the Agent tool; contract is hook-event JSON on stdin → additionalContext JSON on stdout (or no output for the no-op path). The hook is the completion report. Registered in the plugin's hook manifest; `bin/refresh-plugins` after landing. - Intake: `intake <ticket>` creates the ticket row, writes the first brief, records the first pending dispatch, prints the first action. - Briefs: markdown files under the ticket's briefs directory in `.sdlc/`; dispatch actions carry `{agent, brief_path, optional output_file}`. Rule: structured multi-writer state → rows; prose with one LLM reader → files referenced by path from rows. - Action vocabulary frozen at three shapes: `dispatch {agent, brief_path, optional output_file}`, `ask_human {question, skippable}`, `done {report path}`. The freeze is deliberate interface-first design; the live shakedown validates it before anything leans on it. - **SKILL.md: production interface, written now in the pseudoskill style** (vault: howto/writing-pseudocode-style-skills.md — numbered phases, intent bullets with command sub-bullets, declared loop exit, branch per action type). One structural update to the style: there is no report-and-fetch tail call, because the orchestrator never fetches — actions arrive by injection. The skill body is, in essence (snippet encodes the decision; final wording at build time): ``` ## 1. Start - Get the ticket number - none given -> stop and ask the user - Start the pipeline - run `os-sdlc intake <ticket>` - it prints the first action ## 2. Loop until the action is done **Switch on the action type:** - If the action is dispatch: - call the Agent tool with the named agent and the brief path as its prompt - when it finishes, the next action appears in your context by itself — do not run anything to fetch it - If the action is ask_human: - ask the user the question, then follow the answer instruction the action carries - If the action is done: - tell the user the report path. You are finished. You never run tests, never read briefs, never judge agent output. ``` - No recovery analyst, no free-text route derivation, no JSON brief representation (SKIP verdicts in the port map are binding). - Doc corrections landing with this phase: dispatch-hook.md's "a code loop can't lose its place" idempotency claim (describes the deleted in-process design) rewritten for the re-entrant reality; ADR-0109 amended for the dispatches table + honest rationale. ## Testing Decisions Good tests assert external behavior at the highest seam: given a hook event and a DB state, the emitted action and the resulting rows — never Runner internals. - Primary seam — the hook contract: fake PostToolUse Agent-tool event JSON on stdin; assert the additionalContext action emitted and the rows written. The full skeleton path (intake → dispatch → hook → dispatch → hook → done) runs as a test at this seam. - The correlation no-op paths are tested at the same seam: a non-matching agent event, an event with no pending dispatch, and an event with two non-terminal tickets each produce no output and no row changes. - Gates in tests are stub scripts with controlled exit codes; routing-on-exitstatus asserted without a real project. - DB per test is in-memory SQLite; assertions read rows. - Unit seam for StateMachine + table: DR's ported tests, one fixture table per test class, every loop test wrapped in `Timeout.timeout`. - Harness patterns ported from DR: stdout capture in test_helper, named fakes (fake gate results, spy dispatch sink). - Prior art: the kept CLI-subcommand suite in the plugin's `tests/` directory. - A state-divergence regression test (DB state vs on-disk brief reality) is written at skeleton time, per the DR persistence-bug lesson. - The live-session shakedown is the phase exit criterion, performed manually once: the injected dispatch reaching the orchestrator, an unrelated Explore dispatch mid-pipeline NOT advancing state, and `done` surfacing. It is the only behavior the harness cannot reach. ## Out of Scope - The real pipeline states — red-assert, lint/autocorrect/lint-fixer loop, reviewer verdict, escalation caps — are the follow-up spec; internally they will land in risk order (red/green vertical with retry first, then lint loop, then reviewer, then escalation/teardown). - Retry routes, `retry_counts`, and their validate! rule (land with the first retry state). - PostToolUseFailure handling (Agent call fails rather than completes) — recorded as an Open item in dispatch-hook.md; built only if the shakedown or a live run shows it. - Sharding, the claims table, parallel dispatch (deferred per ADR-0108/0109). - Migrating the kept gate CLI or `Artifact::StepperState` onto the new schema (bridge stays until the tickets table replaces artifact serialization). - Brief bodies in the database. - Stale stepper references in `plugins/os-sdlc/CLAUDE.md` and old planning docs (separate doc pass). ## Further Notes - Style: TDD, minitest, Sandi Metz OOP Ruby. Skeleton first as a vertical slice; nothing built ahead of a row that needs it. - Port verdicts and file-level guidance: `docs/os-sdlc-rebuild/dr-port-map.md`; confirmed DR warts (object_id retry keys, fail-open verdict parsing, #call/#run mismatch) must not survive the port. - After any plugin source edit lands, `bin/refresh-plugins` must run before a live session can see it.
jared changed title from PRD: os-sdlc rebuild phase 2 — hook-driven runner tracer-bullet skeleton (ADR-0108/0109) to Spec: os-sdlc rebuild phase 2 — hook-driven runner tracer-bullet skeleton (ADR-0108/0109) 2026-08-07 17:37:32 +00:00
Author
Owner

Body replaced with the /to-spec revision: integrates the Codex review's dispatch-correlation requirement (dispatches table, match-or-no-op hook), moves retry machinery to the first retry state, locks the SKILL.md to pseudoskill style, and records rejected review items. Prior PRD text is in the edit history.

Body replaced with the /to-spec revision: integrates the Codex review's dispatch-correlation requirement (dispatches table, match-or-no-op hook), moves retry machinery to the first retry state, locks the SKILL.md to pseudoskill style, and records rejected review items. Prior PRD text is in the edit history.
Author
Owner

Work started on branch sdlc-rebuild-phase2 (hook-driven runner tracer-bullet skeleton).

Work started on branch sdlc-rebuild-phase2 (hook-driven runner tracer-bullet skeleton).
Author
Owner

Skeleton implemented on branch sdlc-rebuild-phase2 (commit 038368a). 682-run suite green (29 new tests at the hook-contract seam incl. correlation no-op paths + state-divergence regression). Passed opus alignment critique, aidd-lint, and code review; fixes applied incl. namespaced subagent_type correlation and loud-stall hook behavior. Remaining before close: merge + the manual live-session shakedown (phase exit criterion).

Skeleton implemented on branch sdlc-rebuild-phase2 (commit 038368a). 682-run suite green (29 new tests at the hook-contract seam incl. correlation no-op paths + state-divergence regression). Passed opus alignment critique, aidd-lint, and code review; fixes applied incl. namespaced subagent_type correlation and loud-stall hook behavior. Remaining before close: merge + the manual live-session shakedown (phase exit criterion).
Author
Owner

Resolution

Done: Phase 2 tracer-bullet runner skeleton shipped and hardened: SQLite/Sequel runner, two-state skeleton table, PostToolUse hook channel, intake CLI, slim implement SKILL.md; all 8 code-review findings addressed

Evidence: Merged to main (branch sdlc-rebuild-phase2, commits 038368a, c2c5125, b5ef02c); suite 684 runs / 1783 assertions / 0 failures

Follow-ups: Phase 3+ tracked as #328-#334

## Resolution **Done:** Phase 2 tracer-bullet runner skeleton shipped and hardened: SQLite/Sequel runner, two-state skeleton table, PostToolUse hook channel, intake CLI, slim implement SKILL.md; all 8 code-review findings addressed **Evidence:** Merged to main (branch sdlc-rebuild-phase2, commits 038368a, c2c5125, b5ef02c); suite 684 runs / 1783 assertions / 0 failures **Follow-ups:** Phase 3+ tracked as #328-#334
jared closed this issue 2026-08-09 13:15:36 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
jared/cc-os#327
No description provided.