decision-system S9b: the One Thing question at SessionStart and its verdict readback #1278

New issue

Closed

opened 2026-10-23 19:37:30 +00:00 by jared · 2 comments

jared commented

2026-10-23 19:37:30 +00:00

Owner

Problem

S9a writes open lesson candidates to the Store queue/lesson-candidates.jsonl, but nothing shows a candidate to Jared and nothing reads his answer. Review must follow the One Thing rule (D-10): at most one yes/no question per session, at a natural pause, with the answer read later from the transcript as a natural verdict.

Approach

Required work:

Add OsMe::SessionLedger over the Store file queue/one-thing.jsonl: #asked?(session_id), #record(session_id:, transcript_path:, candidate:) (row {session_id, transcript_path, candidate_id, question, asked_at}), #unread (asked rows with no verdict row), and #record_verdict(session_id, verdict). check: present class SessionLedger in plugins/os-me/lib/**/*.rb
Add CandidateLog#open (rows whose latest status is open) and CandidateLog#mark(id, status) (appends a status row; the latest row of an id wins). check: present def mark in plugins/os-me/lib/os_me/candidate_log.rb
Add OsMe::OneThingSelector: #pick(session_id, transcript_path) returns nil for a held-out session (Split) or a session the ledger already holds; else it takes the open candidate with the most moments (ties: the smaller id), records it in the ledger, and returns it. check: present class OneThingSelector in plugins/os-me/lib/**/*.rb
Add OsMe::OneThingPrompt: #context(payload) returns nil when the Store directory is absent or the selector picks nothing; else the SessionStart hook output hash with additionalContext At a natural pause, ask this one question word for word, as the last line of that reply: <question>. check: present natural pause in plugins/os-me/lib/**/*.rb
Add the thin hook script hooks/one_thing.rb: it reads the payload from stdin, calls OneThingPrompt on ~/dev/os-me-store, prints the JSON only when the result is not nil, and exits 0 also on an error (as extract_moments.rb does). Add it to hooks.json under SessionStart with matcher startup and timeout 5. check: present one_thing.rb in plugins/os-me/hooks/hooks.json
Add OsMe::LessonVerdict (model-backed Decision::Model::Question.choice with options confirmed, rejected, unclear, through LabelerClient) and OsMe::VerdictReadback: for each unread ledger row, find the first ExchangeFinder#pairs pair whose AI text holds the question; classify its reply; write the verdict to the ledger; mark the candidate confirmed or rejected (an unclear verdict leaves it open). No pair: no model call and no verdict. check: present class VerdictReadback in plugins/os-me/lib/**/*.rb
Add the os-me-eval one-thing-verdicts command: it runs VerdictReadback on --store with the HeadlessClaude labeler client and prints the number of verdicts read. check: present desc 'one-thing-verdicts' in plugins/os-me/lib/os_me/cli.rb

Notes: O-5 AFK default: the question goes in SessionStart additionalContext. The hook makes no model call and no embedding call; it reads two small JSONL files. The SessionStart matcher is startup, and the ledger still guards against a second question in one session. A held-out session gets no question, so no verdict comes from a held-out session (held-out discipline, AFK rule 8). The hook prints context, so it is a separate hook from extract_moments.rb, which prints nothing (O-10 default). All tests live under plugins/os-me/test; the hook test runs the script with HOME set to a temporary directory, as extract_launcher_test.rb does. Tests use the gem Fixed adapter; no test calls a model. Run bin/refresh-plugins after the hooks.json change. Gate (not part of this ticket): the first three questions are asked in real sessions and their answers are read back as natural verdicts.

Acceptance criteria

Given open candidates with 4 and 7 moments and an empty ledger, when OneThingSelector#pick runs for a train session, then it returns the 7-moment candidate and SessionLedger#asked? is true for that session.
Given a ledger that already holds the session, when OneThingSelector#pick runs for it, then it returns nil and the ledger file gains no row.
Given a session id that Split holds out, when OneThingSelector#pick runs for it, then it returns nil and the ledger file gains no row.
Given a candidate log whose only candidates are marked rejected or confirmed, when OneThingSelector#pick runs, then it returns nil.
Given a temporary HOME whose dev/os-me-store holds one open candidate, when hooks/one_thing.rb runs with a SessionStart startup payload on stdin, then it exits 0 and stdout is JSON whose hookSpecificOutput.hookEventName is SessionStart and whose additionalContext holds At a natural pause, ask this one question word for word and the candidate question.
Given a temporary HOME with no dev/os-me-store directory, when hooks/one_thing.rb runs with a SessionStart payload, then it exits 0 and stdout is empty.
Given a ledger row whose transcript holds the question in an assistant message followed by a typed reply, and a fake adapter that answers rejected, when VerdictReadback#run runs, then CandidateLog#open no longer holds the candidate and OneThingSelector#pick for a new train session returns nil.
Given the same transcript and a fake adapter that answers confirmed, when VerdictReadback#run runs, then the candidate's latest status is confirmed.
Given a ledger row whose transcript holds no assistant message with the question, when VerdictReadback#run runs, then the adapter receives no call and the candidate stays open.
Given a ledger row that already has a verdict, when VerdictReadback#run runs again, then the adapter receives no call.

Owner sketch

module OsMe
  class SessionLedger
    FILE_NAME = 'queue/one-thing.jsonl'

    def initialize(path); end
    def asked?(session_id); end
    def record(session_id:, transcript_path:, candidate:); end
    def unread; end
    def record_verdict(session_id, verdict); end
  end

  class OneThingSelector
    def initialize(candidates:, ledger:, split: Split.new); end
    def pick(session_id, transcript_path); end
  end

  class OneThingPrompt
    LINE = 'At a natural pause, ask this one question word for word, as the last line of that reply: '

    def initialize(store:); end
    def context(payload); end
  end

  class LessonVerdict
    CRITERIA = { 'confirmed' => '...', 'rejected' => '...', 'unclear' => '...' }.freeze

    def initialize(client:); end
    def classify(pair); end
  end

  class VerdictReadback
    def initialize(store_path:, verdict:, finder:); end
    def run; end
  end
end

Scope and blocking

Blocked by: #1251
Design is in docs/decision-system/review.md (OneThingSelector, SessionLedger, where the question appears) and docs/decision-system/learning-loops.md (demotion sequence). Related: #1240.

Non-goals: writing a confirmed lesson to its destination (lint lesson, judgment/ file, or tier row); running candidates and one-thing-verdicts from the SessionStart launcher; the calibration factor in the pick order; tuning loops (S10).

Origin

Trigger: User asked for the decision-system build tickets this session
Improvised this session: none
Chain: no ticket for this stage ← the build plan names the stage and the user asked for the tickets. Stop at DESIGN (docs/decision-system/build-plan.md).
Root candidate: none found; related #1240 (design ticket, different mechanism)
Where: docs/decision-system/build-plan.md S9; plugins/os-me/lib/os_me/; plugins/os-me/hooks/
Session: 4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5.jsonl

VERDICT: CREATE (user-approved; backlog search only)

## Problem S9a writes open lesson candidates to the Store `queue/lesson-candidates.jsonl`, but nothing shows a candidate to Jared and nothing reads his answer. Review must follow the One Thing rule (D-10): at most one yes/no question per session, at a natural pause, with the answer read later from the transcript as a natural verdict. ## Approach Required work: 1. Add `OsMe::SessionLedger` over the Store file `queue/one-thing.jsonl`: `#asked?(session_id)`, `#record(session_id:, transcript_path:, candidate:)` (row `{session_id, transcript_path, candidate_id, question, asked_at}`), `#unread` (asked rows with no verdict row), and `#record_verdict(session_id, verdict)`. check: present `class SessionLedger` in `plugins/os-me/lib/**/*.rb` 2. Add `CandidateLog#open` (rows whose latest status is `open`) and `CandidateLog#mark(id, status)` (appends a status row; the latest row of an id wins). check: present `def mark` in `plugins/os-me/lib/os_me/candidate_log.rb` 3. Add `OsMe::OneThingSelector`: `#pick(session_id, transcript_path)` returns nil for a held-out session (`Split`) or a session the ledger already holds; else it takes the open candidate with the most moments (ties: the smaller id), records it in the ledger, and returns it. check: present `class OneThingSelector` in `plugins/os-me/lib/**/*.rb` 4. Add `OsMe::OneThingPrompt`: `#context(payload)` returns nil when the Store directory is absent or the selector picks nothing; else the SessionStart hook output hash with `additionalContext` `At a natural pause, ask this one question word for word, as the last line of that reply: <question>`. check: present `natural pause` in `plugins/os-me/lib/**/*.rb` 5. Add the thin hook script `hooks/one_thing.rb`: it reads the payload from stdin, calls `OneThingPrompt` on `~/dev/os-me-store`, prints the JSON only when the result is not nil, and exits 0 also on an error (as `extract_moments.rb` does). Add it to `hooks.json` under `SessionStart` with matcher `startup` and timeout 5. check: present `one_thing.rb` in `plugins/os-me/hooks/hooks.json` 6. Add `OsMe::LessonVerdict` (model-backed `Decision::Model::Question.choice` with options `confirmed`, `rejected`, `unclear`, through `LabelerClient`) and `OsMe::VerdictReadback`: for each unread ledger row, find the first `ExchangeFinder#pairs` pair whose AI text holds the question; classify its reply; write the verdict to the ledger; mark the candidate `confirmed` or `rejected` (an `unclear` verdict leaves it `open`). No pair: no model call and no verdict. check: present `class VerdictReadback` in `plugins/os-me/lib/**/*.rb` 7. Add the `os-me-eval one-thing-verdicts` command: it runs `VerdictReadback` on `--store` with the `HeadlessClaude` labeler client and prints the number of verdicts read. check: present `desc 'one-thing-verdicts'` in `plugins/os-me/lib/os_me/cli.rb` Notes: O-5 AFK default: the question goes in SessionStart `additionalContext`. The hook makes no model call and no embedding call; it reads two small JSONL files. The SessionStart matcher is `startup`, and the ledger still guards against a second question in one session. A held-out session gets no question, so no verdict comes from a held-out session (held-out discipline, AFK rule 8). The hook prints context, so it is a separate hook from `extract_moments.rb`, which prints nothing (O-10 default). All tests live under `plugins/os-me/test`; the hook test runs the script with `HOME` set to a temporary directory, as `extract_launcher_test.rb` does. Tests use the gem `Fixed` adapter; no test calls a model. Run `bin/refresh-plugins` after the `hooks.json` change. Gate (not part of this ticket): the first three questions are asked in real sessions and their answers are read back as natural verdicts. ## Acceptance criteria 1. Given open candidates with 4 and 7 moments and an empty ledger, when `OneThingSelector#pick` runs for a train session, then it returns the 7-moment candidate and `SessionLedger#asked?` is true for that session. 2. Given a ledger that already holds the session, when `OneThingSelector#pick` runs for it, then it returns nil and the ledger file gains no row. 3. Given a session id that `Split` holds out, when `OneThingSelector#pick` runs for it, then it returns nil and the ledger file gains no row. 4. Given a candidate log whose only candidates are marked `rejected` or `confirmed`, when `OneThingSelector#pick` runs, then it returns nil. 5. Given a temporary `HOME` whose `dev/os-me-store` holds one open candidate, when `hooks/one_thing.rb` runs with a SessionStart `startup` payload on stdin, then it exits 0 and stdout is JSON whose `hookSpecificOutput.hookEventName` is `SessionStart` and whose `additionalContext` holds `At a natural pause, ask this one question word for word` and the candidate question. 6. Given a temporary `HOME` with no `dev/os-me-store` directory, when `hooks/one_thing.rb` runs with a SessionStart payload, then it exits 0 and stdout is empty. 7. Given a ledger row whose transcript holds the question in an assistant message followed by a typed reply, and a fake adapter that answers `rejected`, when `VerdictReadback#run` runs, then `CandidateLog#open` no longer holds the candidate and `OneThingSelector#pick` for a new train session returns nil. 8. Given the same transcript and a fake adapter that answers `confirmed`, when `VerdictReadback#run` runs, then the candidate's latest status is `confirmed`. 9. Given a ledger row whose transcript holds no assistant message with the question, when `VerdictReadback#run` runs, then the adapter receives no call and the candidate stays `open`. 10. Given a ledger row that already has a verdict, when `VerdictReadback#run` runs again, then the adapter receives no call. ## Owner sketch ```ruby module OsMe class SessionLedger FILE_NAME = 'queue/one-thing.jsonl' def initialize(path); end def asked?(session_id); end def record(session_id:, transcript_path:, candidate:); end def unread; end def record_verdict(session_id, verdict); end end class OneThingSelector def initialize(candidates:, ledger:, split: Split.new); end def pick(session_id, transcript_path); end end class OneThingPrompt LINE = 'At a natural pause, ask this one question word for word, as the last line of that reply: ' def initialize(store:); end def context(payload); end end class LessonVerdict CRITERIA = { 'confirmed' => '...', 'rejected' => '...', 'unclear' => '...' }.freeze def initialize(client:); end def classify(pair); end end class VerdictReadback def initialize(store_path:, verdict:, finder:); end def run; end end end ``` ## Scope and blocking Blocked by: #1251 Design is in `docs/decision-system/review.md` (`OneThingSelector`, `SessionLedger`, where the question appears) and `docs/decision-system/learning-loops.md` (demotion sequence). Related: #1240. Non-goals: writing a confirmed lesson to its destination (lint lesson, `judgment/` file, or tier row); running `candidates` and `one-thing-verdicts` from the SessionStart launcher; the calibration factor in the pick order; tuning loops (S10). ## Origin - Trigger: User asked for the decision-system build tickets this session - Improvised this session: none - Chain: no ticket for this stage ← the build plan names the stage and the user asked for the tickets. Stop at `DESIGN` (`docs/decision-system/build-plan.md`). - Root candidate: none found; related #1240 (design ticket, different mechanism) - Where: docs/decision-system/build-plan.md S9; plugins/os-me/lib/os_me/; plugins/os-me/hooks/ - Session: 4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5.jsonl VERDICT: CREATE (user-approved; backlog search only)

jared added the

project/cc-os

filed-by/user

category/code-change

labels

2026-10-23 19:37:30 +00:00

jared referenced this issue

2026-10-23 19:51:53 +00:00

decision-system S9: demotion and the One Thing question #1251

jared commented

2026-10-23 19:53:03 +00:00

Author

Owner

Work started on branch ticket-1278 (os-sdlc implement, AFK decision-system build, S9b).

jared commented

2026-10-23 20:01:35 +00:00

Author

Owner

Resolution

Done: SessionStart asks at most one lesson-candidate question per train session through additionalContext; os-me-eval one-thing-verdicts reads Jared's reply as a natural verdict and marks the candidate confirmed or rejected.

Evidence: Implementation 142: 8 dispatches, 0 retries, 8 min 5 s, implementation_complete. Suite gate green. Records commit on ticket-1278.

Follow-ups: Dropped: running candidates and one-thing-verdicts from the SessionStart launcher (ticket non-goal; the S9 gate runs them by hand first). Dropped: writing a confirmed lesson to its destination (remainder of #1240, already open). Contract-auditor LENS line on the one_thing file prefix has no proposal; revisit if a third one_thing file appears.

## Resolution **Done:** SessionStart asks at most one lesson-candidate question per train session through additionalContext; os-me-eval one-thing-verdicts reads Jared's reply as a natural verdict and marks the candidate confirmed or rejected. **Evidence:** Implementation 142: 8 dispatches, 0 retries, 8 min 5 s, implementation_complete. Suite gate green. Records commit on ticket-1278. **Follow-ups:** Dropped: running candidates and one-thing-verdicts from the SessionStart launcher (ticket non-goal; the S9 gate runs them by hand first). Dropped: writing a confirmed lesson to its destination (remainder of #1240, already open). Contract-auditor LENS line on the one_thing file prefix has no proposal; revisit if a third one_thing file appears.

jared closed this issue

2026-10-23 20:01:35 +00:00

jared referenced this issue from a commit

2026-10-23 20:02:37 +00:00

feat(os-me): Audited POODR boundaries for SessionLedger, CandidateLog#open and #mark, OneThingSelector, OneThingPrompt, LessonVerdict, VerdictReadback, hooks/one_thing.rb and the one-thing-verdicts command. Design and contracts hold; all seven WORK items are present. (#1278)

jared referenced this issue from a commit

2026-10-23 20:02:37 +00:00

docs(decision-system): S9 built, gate open (#1278)