jared/cc-os

Watch

Fork 0

os-me: the Stop hook records only a Prediction shown after the pending invocation, and skips held-out sessions #1762

New issue

Closed

opened 2026-11-13 20:57:00 +00:00 by jared · 3 comments

jared commented

2026-11-13 20:57:00 +00:00

Owner

Problem

The Stop hook of os-me has two faults. It marks a new /os-me:predict invocation as recorded when the transcript holds only an older Prediction, so the new Prediction never gets a row in ~/dev/os-me-store/prediction-log.jsonl. It also writes rows, with text, for held-out sessions, which the Store rules forbid. The code is Ruby, in plugins/os-me/hooks; the tests are Minitest.

Evidence, from the source at commit 43bdc0800 and from the Store on 2026-11-13:

CliAppender#prediction_shown? (plugins/os-me/hooks/cli_appender.rb:11-14) reads all messages of the transcript. PredictionRecorder#stop (plugins/os-me/hooks/prediction_recorder.rb:19-25) uses it as the ready check. When a session has an older Prediction, the first Stop after a new invocation is ready before the new Prediction exists. append reads the older Prediction again, the log drops it as a duplicate, and mark_recorded writes the Stop marker. No later Stop reads the new Prediction.
Queue history of queue/invocations.jsonl: session f8d22c34 got its second invocation at 2026-11-05T14:45:20Z and the Stop marker at 14:45:32Z (12 s). Session 47ffa777 got its second invocation at 2026-11-05T18:25:56Z and the Stop marker at 18:26:05Z (9 s). Each session has 1 log row.
Each invocation entry has an at value (InvocationLog#append, plugins/os-me/hooks/invocation_log.rb:53). Each transcript entry has a timestamp (Entry#timestamp, plugins/os-me/lib/os_me/entry.rb:20). No code compares the two.
No file under plugins/os-me/hooks/ calls Split#held_out?. The log has one row each for the held-out sessions ae4be489 (2026-11-04) and 9d8c2e15 (2026-11-07). Each row has a text value copied from the transcript. ~/dev/os-me-store/CHARTER.md line 31 says "Never read a held-out session". ~/dev/os-me-store/learning/loop.md line 39 says "never copy text from one".

Approach

The owner is OsMe::PredictionRecorder. Its collaborators are InvocationLog, CliAppender, and OsMe::Split.

Ready check. PredictionRecorder#stop calls the ready check with two arguments: the transcript path and the at value of the pending invocation. CliAppender#prediction_shown?(transcript_path, since = nil) is true only when a Prediction is in a transcript entry whose timestamp is later than since. TranscriptReader#prediction_candidates(since: nil) gives only those entries. When the check is false, the hook returns and waits for the next Stop, as it does today.

Entries with no timestamp. A transcript entry with no timestamp counts as later than since. When since is nil, all entries count. Thus prediction_shown?(path) with one argument, and the fixtures of plugins/os-me/test/support/transcript_scenario.rb (which carry no timestamp), behave as today.

Time comparison. The at value is UTC with whole seconds (InvocationLog#append writes utc.iso8601, plugins/os-me/hooks/invocation_log.rb:53; example 2026-11-05T14:45:20Z). The transcript timestamp is UTC with milliseconds (example 2026-11-05T14:47:00.128Z). Compare the two values as times, not as strings. An entry whose timestamp is equal to since counts as later. An entry whose timestamp does not parse counts as later, the same as an entry with no timestamp. Known limit: the at value has no fraction of a second, so a Prediction shown in the same second as the invocation, before it, counts as later. The comparison runs in TranscriptReader, and plugins/os-me/lib/os_me.rb does not load time, so transcript_reader.rb loads it.

Held-out sessions. Ruling of 2026-11-13 (session 2bb44e9c, orchestrator): skip. When OsMe::Split#held_out? is true for the session id of the Stop payload, and the session has a pending invocation, the hook does not call the ready check, does not call the appender, and calls mark_recorded. The marker stops a retry on each later Stop. Do this check before the hook reads the transcript. PredictionRecorder.new gets a split: keyword with the default OsMe::Split.new.

Wiring. plugins/os-me/hooks/record_prediction.rb keeps appender.method(:prediction_shown?) as the ready check; the method now takes the two arguments.

Tests. Thirteen tests give PredictionRecorder a ready lambda with one argument and reach the check: ten in plugins/os-me/hooks/test/prediction_recorder_test.rb, one in plugins/os-me/test/prediction_recorder_provenance_test.rb, and two in plugins/os-me/test/prediction_recorder_payload_provenance_test.rb. The ready check now gets two arguments, so these tests fail; they are under Retired tests. They fail only because of the lambda. The behaviors that they pin stay, and acceptance criteria 8 to 19 state them. The test for a Stop with no predict entry in prediction_recorder_test.rb stays valid, because it does not reach the check. plugins/os-me/hooks/test/cli_appender_test.rb stays valid: it calls prediction_shown?(path) with one argument.

Put the held-out check in PredictionRecorder. Do not put it in AppendRequest, CLI#append, or CliAppender: the CLI test fixtures use the session ids sess-1 and sess-2, and Split#held_out? is true for the two ids.

Required work:

PredictionRecorder#stop gives the at value of the pending invocation to the ready check, and skips a held-out session — check: present held_out? in plugins/os-me/hooks/prediction_recorder.rb
CliAppender#prediction_shown? accepts the time and reads only later entries — check: present since in plugins/os-me/hooks/cli_appender.rb
TranscriptReader#prediction_candidates accepts since: — check: present since in plugins/os-me/lib/os_me/transcript_reader.rb
transcript_reader.rb loads time for the time comparison — check: present require 'time' in plugins/os-me/lib/os_me/transcript_reader.rb

Orchestrator work: update the Stop hook text of docs/implementation-status/os-me.md (the lines "A second Stop appends nothing. An append failure writes no marker, so the next Stop retries.") with the ready check after the invocation time and the held-out skip, and add one History row.

Rejected: a held-out check in AppendRequest or CLI#append — the fixtures sess-1 and sess-2 are held-out ids, so the CLI tests would fail.

Rejected: a ready check that keeps one argument — the check cannot know the invocation time, and only PredictionRecorder holds the pending invocation.

Rejected: a marked log row with no text for a held-out session — the ruling of 2026-11-13 is skip.

Acceptance criteria

Given a session whose transcript holds a Prediction with a timestamp before the at value of a pending invocation, and no later Prediction, when the Stop hook runs, then the appender is not called and the queue gets no Stop marker.
Given the same session after a Prediction appears in an entry with a timestamp later than the at value, when the Stop hook runs again, then the appender is called one time and the queue gets one Stop marker.
Given a transcript whose entries have no timestamp, and a Prediction in it, when the ready check runs with the time of the pending invocation, then the check is true.
Given a held-out session with a pending invocation, when the Stop hook runs, then the appender is not called, the ready check is not called, and the queue gets one Stop marker.
Given the held-out session of criterion 4, when the Stop hook runs a second time, then the queue gets no second Stop marker.
Given a pending invocation and a Prediction in an entry whose timestamp is equal to the at value, when the ready check runs, then the check is true.
Given a pending invocation and a Prediction in an entry whose timestamp does not parse as a time, when the ready check runs, then the check is true.
Given a pending invocation from a UserPromptExpansion event with no arguments, and a true ready check, when the Stop hook runs, then the appender receives the transcript path of the Stop payload, the origin typed, and the autopilot flag false.
Given a pending invocation from a PreToolUse event, and a true ready check, when the Stop hook runs, then the appender receives the origin agent.
Given a pending invocation whose arguments contain the word autopilot, and a true ready check, when the Stop hook runs, then the appender receives the autopilot flag true.
Given a pending invocation whose arguments do not contain the word autopilot, and a true ready check, when the Stop hook runs, then the appender receives the autopilot flag false.
Given a pending queue entry whose event is not PreToolUse and not UserPromptExpansion, and a true ready check, when the Stop hook runs, then the hook fails with a KeyError and the queue gets no Stop marker.
Given a session that is not held out and whose invocation the Stop hook recorded, when the Stop hook runs a second time, then the appender is not called again and the queue gets no second Stop marker.
Given a pending invocation and an appender that fails, when the Stop hook runs, then the queue gets no Stop marker, and the next Stop calls the appender again.
Given the session of criterion 14 after the second Stop, in which the appender does not fail, when a person reads the queue, then the session has exactly one Stop marker.
Given a pending invocation, when the Stop hook runs, then the ready check receives the transcript path of the Stop payload and the at value of the pending invocation.
Given a Stop payload that has a producer value and a model value, and a Store repository, when the Stop hook records the Prediction, then the log row has that producer, that model, the store_rev of the Store, and the brief_rev of the Store.
Given a Stop payload with no model value, and a transcript whose last assistant message has a model, when the Stop hook records the Prediction, then the log row has the model of that message.
Given a Stop payload with no producer value, and a transcript that has an assistant message, when the Stop hook records the Prediction, then the log row has the producer agent.

Owner sketch

class OsMe::PredictionRecorder
  def initialize(log:, appender:, ready:, split: OsMe::Split.new)
  def stop(payload)
end

class OsMe::CliAppender
  def prediction_shown?(transcript_path, since = nil)
end

class OsMe::TranscriptReader
  def prediction_candidates(since: nil)
end

Non-goals

In scope: the ready check of the Stop hook, and the held-out skip in the Stop hook.

Out of scope:

Removal of the two present rows of the held-out sessions ae4be489 and 9d8c2e15. That is a Store data decision for Jared.
The block shapes that the parser reads, and one row for each Prediction of a message (#1758).
A held-out check in os-me-eval append.

Blocking edges: none. This ticket and #1758 can ship in either order.

Retired tests

plugins/os-me/hooks/test/prediction_recorder_test.rb#test_stop_after_typed_entry_records_typed
plugins/os-me/hooks/test/prediction_recorder_test.rb#test_stop_after_agent_entry_records_agent
plugins/os-me/hooks/test/prediction_recorder_test.rb#test_second_stop_appends_nothing
plugins/os-me/hooks/test/prediction_recorder_test.rb#test_failed_append_is_retried_on_the_next_stop
plugins/os-me/hooks/test/prediction_recorder_test.rb#test_failed_append_then_retry_marks_the_session_recorded_once
plugins/os-me/hooks/test/prediction_recorder_test.rb#test_stop_before_the_prediction_is_shown_waits_and_retries_on_the_next_stop
plugins/os-me/hooks/test/prediction_recorder_test.rb#test_ready_check_reads_the_transcript_path_of_the_stop_payload
plugins/os-me/hooks/test/prediction_recorder_test.rb#test_autopilot_word_sets_autopilot_true
plugins/os-me/hooks/test/prediction_recorder_test.rb#test_missing_autopilot_word_sets_false
plugins/os-me/hooks/test/prediction_recorder_test.rb#test_unknown_event_raises
plugins/os-me/test/prediction_recorder_provenance_test.rb#test_stop_writes_a_record_carrying_producer_model_store_rev_and_brief_rev
plugins/os-me/test/prediction_recorder_payload_provenance_test.rb#test_stop_without_model_in_payload_records_the_transcript_model
plugins/os-me/test/prediction_recorder_payload_provenance_test.rb#test_stop_without_producer_in_payload_records_agent

Origin

Trigger: the os-me check of 2026-11-13 in session 2bb44e9c found predict runs with no prediction-log row (#1758). The Decide phase of #1758 on 2026-11-13 split the Stop hook causes (c and e of the first #1758 body) into this ticket.
Improvised this session: none
Chain: a new Prediction has no log row ← the Stop marker is written 9 s to 12 s after the invocation (queue history of f8d22c34 and 47ffa777) ← CliAppender#prediction_shown? reads the whole transcript, so an older Prediction makes the check true (plugins/os-me/hooks/cli_appender.rb:11-14) ← DESIGN (plugins/os-me/hooks/prediction_recorder.rb: the ready check gets only the transcript path). Held-out rows ← no file under plugins/os-me/hooks/ calls Split#held_out? ← DESIGN (plugins/os-me/hooks/prediction_recorder.rb).
Root candidate: none found; searched Stop hook, PredictionRecorder, prediction_shown?, held_out?, held-out, prediction-log, os-me hooks
Where: plugins/os-me/hooks/prediction_recorder.rb; plugins/os-me/hooks/cli_appender.rb; plugins/os-me/lib/os_me/transcript_reader.rb
Session: 2bb44e9c-522c-4c97-8449-bd46bed898e0
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/2bb44e9c-522c-4c97-8449-bd46bed898e0.jsonl

VERDICT: CREATE (ticket-skeptic, 2026-11-13): observed faults; sessions f8d22c34 and 47ffa777 have a Stop marker 9 s to 12 s after a new invocation and one log row; held-out sessions ae4be489 and 9d8c2e15 have rows with copied text. No open ticket covers the ready check or the held-out skip.

## Problem The Stop hook of os-me has two faults. It marks a new `/os-me:predict` invocation as recorded when the transcript holds only an older Prediction, so the new Prediction never gets a row in `~/dev/os-me-store/prediction-log.jsonl`. It also writes rows, with text, for held-out sessions, which the Store rules forbid. The code is Ruby, in `plugins/os-me/hooks`; the tests are Minitest. Evidence, from the source at commit 43bdc0800 and from the Store on 2026-11-13: - `CliAppender#prediction_shown?` (`plugins/os-me/hooks/cli_appender.rb:11-14`) reads all messages of the transcript. `PredictionRecorder#stop` (`plugins/os-me/hooks/prediction_recorder.rb:19-25`) uses it as the `ready` check. When a session has an older Prediction, the first Stop after a new invocation is ready before the new Prediction exists. `append` reads the older Prediction again, the log drops it as a duplicate, and `mark_recorded` writes the Stop marker. No later Stop reads the new Prediction. - Queue history of `queue/invocations.jsonl`: session f8d22c34 got its second invocation at 2026-11-05T14:45:20Z and the Stop marker at 14:45:32Z (12 s). Session 47ffa777 got its second invocation at 2026-11-05T18:25:56Z and the Stop marker at 18:26:05Z (9 s). Each session has 1 log row. - Each invocation entry has an `at` value (`InvocationLog#append`, `plugins/os-me/hooks/invocation_log.rb:53`). Each transcript entry has a timestamp (`Entry#timestamp`, `plugins/os-me/lib/os_me/entry.rb:20`). No code compares the two. - No file under `plugins/os-me/hooks/` calls `Split#held_out?`. The log has one row each for the held-out sessions ae4be489 (2026-11-04) and 9d8c2e15 (2026-11-07). Each row has a `text` value copied from the transcript. `~/dev/os-me-store/CHARTER.md` line 31 says "Never read a held-out session". `~/dev/os-me-store/learning/loop.md` line 39 says "never copy text from one". ## Approach The owner is `OsMe::PredictionRecorder`. Its collaborators are `InvocationLog`, `CliAppender`, and `OsMe::Split`. Ready check. `PredictionRecorder#stop` calls the `ready` check with two arguments: the transcript path and the `at` value of the pending invocation. `CliAppender#prediction_shown?(transcript_path, since = nil)` is true only when a Prediction is in a transcript entry whose timestamp is later than `since`. `TranscriptReader#prediction_candidates(since: nil)` gives only those entries. When the check is false, the hook returns and waits for the next Stop, as it does today. Entries with no timestamp. A transcript entry with no timestamp counts as later than `since`. When `since` is nil, all entries count. Thus `prediction_shown?(path)` with one argument, and the fixtures of `plugins/os-me/test/support/transcript_scenario.rb` (which carry no timestamp), behave as today. Time comparison. The `at` value is UTC with whole seconds (`InvocationLog#append` writes `utc.iso8601`, `plugins/os-me/hooks/invocation_log.rb:53`; example `2026-11-05T14:45:20Z`). The transcript timestamp is UTC with milliseconds (example `2026-11-05T14:47:00.128Z`). Compare the two values as times, not as strings. An entry whose timestamp is equal to `since` counts as later. An entry whose timestamp does not parse counts as later, the same as an entry with no timestamp. Known limit: the `at` value has no fraction of a second, so a Prediction shown in the same second as the invocation, before it, counts as later. The comparison runs in `TranscriptReader`, and `plugins/os-me/lib/os_me.rb` does not load `time`, so `transcript_reader.rb` loads it. Held-out sessions. Ruling of 2026-11-13 (session 2bb44e9c, orchestrator): skip. When `OsMe::Split#held_out?` is true for the session id of the Stop payload, and the session has a pending invocation, the hook does not call the `ready` check, does not call the appender, and calls `mark_recorded`. The marker stops a retry on each later Stop. Do this check before the hook reads the transcript. `PredictionRecorder.new` gets a `split:` keyword with the default `OsMe::Split.new`. Wiring. `plugins/os-me/hooks/record_prediction.rb` keeps `appender.method(:prediction_shown?)` as the `ready` check; the method now takes the two arguments. Tests. Thirteen tests give `PredictionRecorder` a `ready` lambda with one argument and reach the check: ten in `plugins/os-me/hooks/test/prediction_recorder_test.rb`, one in `plugins/os-me/test/prediction_recorder_provenance_test.rb`, and two in `plugins/os-me/test/prediction_recorder_payload_provenance_test.rb`. The `ready` check now gets two arguments, so these tests fail; they are under Retired tests. They fail only because of the lambda. The behaviors that they pin stay, and acceptance criteria 8 to 19 state them. The test for a Stop with no predict entry in `prediction_recorder_test.rb` stays valid, because it does not reach the check. `plugins/os-me/hooks/test/cli_appender_test.rb` stays valid: it calls `prediction_shown?(path)` with one argument. Put the held-out check in `PredictionRecorder`. Do not put it in `AppendRequest`, `CLI#append`, or `CliAppender`: the CLI test fixtures use the session ids `sess-1` and `sess-2`, and `Split#held_out?` is true for the two ids. Required work: 1. `PredictionRecorder#stop` gives the `at` value of the pending invocation to the `ready` check, and skips a held-out session — check: present `held_out?` in `plugins/os-me/hooks/prediction_recorder.rb` 2. `CliAppender#prediction_shown?` accepts the time and reads only later entries — check: present `since` in `plugins/os-me/hooks/cli_appender.rb` 3. `TranscriptReader#prediction_candidates` accepts `since:` — check: present `since` in `plugins/os-me/lib/os_me/transcript_reader.rb` 4. `transcript_reader.rb` loads `time` for the time comparison — check: present `require 'time'` in `plugins/os-me/lib/os_me/transcript_reader.rb` Orchestrator work: update the Stop hook text of `docs/implementation-status/os-me.md` (the lines "A second Stop appends nothing. An append failure writes no marker, so the next Stop retries.") with the ready check after the invocation time and the held-out skip, and add one History row. Rejected: a held-out check in `AppendRequest` or `CLI#append` — the fixtures `sess-1` and `sess-2` are held-out ids, so the CLI tests would fail. Rejected: a `ready` check that keeps one argument — the check cannot know the invocation time, and only `PredictionRecorder` holds the pending invocation. Rejected: a marked log row with no text for a held-out session — the ruling of 2026-11-13 is skip. ## Acceptance criteria 1. Given a session whose transcript holds a Prediction with a timestamp before the `at` value of a pending invocation, and no later Prediction, when the Stop hook runs, then the appender is not called and the queue gets no Stop marker. 2. Given the same session after a Prediction appears in an entry with a timestamp later than the `at` value, when the Stop hook runs again, then the appender is called one time and the queue gets one Stop marker. 3. Given a transcript whose entries have no timestamp, and a Prediction in it, when the ready check runs with the time of the pending invocation, then the check is true. 4. Given a held-out session with a pending invocation, when the Stop hook runs, then the appender is not called, the ready check is not called, and the queue gets one Stop marker. 5. Given the held-out session of criterion 4, when the Stop hook runs a second time, then the queue gets no second Stop marker. 6. Given a pending invocation and a Prediction in an entry whose timestamp is equal to the `at` value, when the ready check runs, then the check is true. 7. Given a pending invocation and a Prediction in an entry whose timestamp does not parse as a time, when the ready check runs, then the check is true. 8. Given a pending invocation from a `UserPromptExpansion` event with no arguments, and a true ready check, when the Stop hook runs, then the appender receives the transcript path of the Stop payload, the origin `typed`, and the autopilot flag false. 9. Given a pending invocation from a `PreToolUse` event, and a true ready check, when the Stop hook runs, then the appender receives the origin `agent`. 10. Given a pending invocation whose arguments contain the word `autopilot`, and a true ready check, when the Stop hook runs, then the appender receives the autopilot flag true. 11. Given a pending invocation whose arguments do not contain the word `autopilot`, and a true ready check, when the Stop hook runs, then the appender receives the autopilot flag false. 12. Given a pending queue entry whose event is not `PreToolUse` and not `UserPromptExpansion`, and a true ready check, when the Stop hook runs, then the hook fails with a `KeyError` and the queue gets no Stop marker. 13. Given a session that is not held out and whose invocation the Stop hook recorded, when the Stop hook runs a second time, then the appender is not called again and the queue gets no second Stop marker. 14. Given a pending invocation and an appender that fails, when the Stop hook runs, then the queue gets no Stop marker, and the next Stop calls the appender again. 15. Given the session of criterion 14 after the second Stop, in which the appender does not fail, when a person reads the queue, then the session has exactly one Stop marker. 16. Given a pending invocation, when the Stop hook runs, then the ready check receives the transcript path of the Stop payload and the `at` value of the pending invocation. 17. Given a Stop payload that has a `producer` value and a `model` value, and a Store repository, when the Stop hook records the Prediction, then the log row has that producer, that model, the `store_rev` of the Store, and the `brief_rev` of the Store. 18. Given a Stop payload with no `model` value, and a transcript whose last assistant message has a model, when the Stop hook records the Prediction, then the log row has the model of that message. 19. Given a Stop payload with no `producer` value, and a transcript that has an assistant message, when the Stop hook records the Prediction, then the log row has the producer `agent`. ## Owner sketch ```ruby class OsMe::PredictionRecorder def initialize(log:, appender:, ready:, split: OsMe::Split.new) def stop(payload) end class OsMe::CliAppender def prediction_shown?(transcript_path, since = nil) end class OsMe::TranscriptReader def prediction_candidates(since: nil) end ``` ## Non-goals In scope: the ready check of the Stop hook, and the held-out skip in the Stop hook. Out of scope: - Removal of the two present rows of the held-out sessions ae4be489 and 9d8c2e15. That is a Store data decision for Jared. - The block shapes that the parser reads, and one row for each Prediction of a message (#1758). - A held-out check in `os-me-eval append`. Blocking edges: none. This ticket and #1758 can ship in either order. ## Retired tests plugins/os-me/hooks/test/prediction_recorder_test.rb#test_stop_after_typed_entry_records_typed plugins/os-me/hooks/test/prediction_recorder_test.rb#test_stop_after_agent_entry_records_agent plugins/os-me/hooks/test/prediction_recorder_test.rb#test_second_stop_appends_nothing plugins/os-me/hooks/test/prediction_recorder_test.rb#test_failed_append_is_retried_on_the_next_stop plugins/os-me/hooks/test/prediction_recorder_test.rb#test_failed_append_then_retry_marks_the_session_recorded_once plugins/os-me/hooks/test/prediction_recorder_test.rb#test_stop_before_the_prediction_is_shown_waits_and_retries_on_the_next_stop plugins/os-me/hooks/test/prediction_recorder_test.rb#test_ready_check_reads_the_transcript_path_of_the_stop_payload plugins/os-me/hooks/test/prediction_recorder_test.rb#test_autopilot_word_sets_autopilot_true plugins/os-me/hooks/test/prediction_recorder_test.rb#test_missing_autopilot_word_sets_false plugins/os-me/hooks/test/prediction_recorder_test.rb#test_unknown_event_raises plugins/os-me/test/prediction_recorder_provenance_test.rb#test_stop_writes_a_record_carrying_producer_model_store_rev_and_brief_rev plugins/os-me/test/prediction_recorder_payload_provenance_test.rb#test_stop_without_model_in_payload_records_the_transcript_model plugins/os-me/test/prediction_recorder_payload_provenance_test.rb#test_stop_without_producer_in_payload_records_agent ## Origin - Trigger: the os-me check of 2026-11-13 in session 2bb44e9c found predict runs with no prediction-log row (#1758). The Decide phase of #1758 on 2026-11-13 split the Stop hook causes (c and e of the first #1758 body) into this ticket. - Improvised this session: none - Chain: a new Prediction has no log row ← the Stop marker is written 9 s to 12 s after the invocation (queue history of f8d22c34 and 47ffa777) ← `CliAppender#prediction_shown?` reads the whole transcript, so an older Prediction makes the check true (`plugins/os-me/hooks/cli_appender.rb:11-14`) ← DESIGN (`plugins/os-me/hooks/prediction_recorder.rb`: the `ready` check gets only the transcript path). Held-out rows ← no file under `plugins/os-me/hooks/` calls `Split#held_out?` ← DESIGN (`plugins/os-me/hooks/prediction_recorder.rb`). - Root candidate: none found; searched Stop hook, PredictionRecorder, prediction_shown?, held_out?, held-out, prediction-log, os-me hooks - Where: plugins/os-me/hooks/prediction_recorder.rb; plugins/os-me/hooks/cli_appender.rb; plugins/os-me/lib/os_me/transcript_reader.rb - Session: 2bb44e9c-522c-4c97-8449-bd46bed898e0 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/2bb44e9c-522c-4c97-8449-bd46bed898e0.jsonl VERDICT: CREATE (ticket-skeptic, 2026-11-13): observed faults; sessions f8d22c34 and 47ffa777 have a Stop marker 9 s to 12 s after a new invocation and one log row; held-out sessions ae4be489 and 9d8c2e15 have rows with copied text. No open ticket covers the ready check or the held-out skip.

jared added the

project/cc-os

filed-by/agent

category/code-change

labels

2026-11-13 20:57:00 +00:00

jared referenced this issue

2026-11-13 20:57:06 +00:00

os-me: the prediction parser reads every Prediction block shape and every block of a message #1758

jared referenced this issue

2026-11-13 20:57:12 +00:00

os-me: the prediction parser reads every Prediction block shape and every block of a message #1758

jared added the

ready-for-agent

label

2026-11-13 21:02:54 +00:00

jared referenced this issue

2026-11-13 21:24:50 +00:00

os-me: the prediction parser makes rows from Prediction blocks that an agent report only quotes #1763

jared referenced this issue

2026-11-13 21:25:10 +00:00

os-me: the prediction parser reads every Prediction block shape and every block of a message #1758

jared commented

2026-11-13 21:26:42 +00:00

Author

Owner

Work started on 2026-11-13 through /os-sdlc:implement. Branch: ticket-1762. #1758 merged first as main c5fd757db; TranscriptReader#prediction_candidates and CliAppender#prediction_shown? are still present after it.

Work started on 2026-11-13 through /os-sdlc:implement. Branch: ticket-1762. #1758 merged first as main c5fd757db; TranscriptReader#prediction_candidates and CliAppender#prediction_shown? are still present after it.

jared referenced this issue

2026-11-13 21:38:10 +00:00

Map tests under a second test folder of an os-sdlc target (os-me hooks/test) #1276

jared commented

2026-11-13 21:38:11 +00:00

Author

Owner

Implementation 293 failed on 2026-11-13 at the test-reviewer step (dispatch 2905), reason verdict:error. Cause: a workflow defect, not a ticket or test defect: the changed tests under plugins/os-me/hooks/test are outside the target's test_path, so the gate did not run them and the reviewer got no diff. Recorded on #1276. The ticket is finished by hand on branch ticket-1762 under the escalation procedure's hand-finish checks: full suite, lint, one design audit that also reviews the hook tests, then commit, merge, close.

jared commented

2026-11-13 21:45:35 +00:00

Author

Owner

Resolution

Done: The Stop hook records only a Prediction shown after the pending invocation. A held-out session with a pending invocation gets one marker and no append. An unparseable invocation time raises ArgumentError. Entry owns the before-time rule (Entry#before?). Finished by hand after implementation 293 failed at test-reviewer (tests under plugins/os-me/hooks/test are outside the target test path; root ticket #1276).

Evidence: Commit 3332c596e on branch ticket-1762, merged to main on 2026-11-13. os-me suite: 466 runs, 837 assertions, 0 failures (bundle exec ruby test/all.rb in plugins/os-me). Rubocop: 11 files, no offenses. One design audit ran on the diff; its 7 findings (2 must-fix, 5 advisory) are all fixed; red proofs exist for the criteria 2, 8 and 16 tests. Plugin cache refreshed by worktree finish.

Follow-ups: #1276 (second occurrence recorded: hook tests outside test_path). #1763 (quoted Prediction blocks in agent reports become rows). Store data for Jared: two held-out records already in prediction-log.jsonl (sessions ae4be489, 9d8c2e15) are not removed. The append step still reads the whole transcript; only the ready check uses the invocation time: dropped, #1758's copy rule and id positions cover it.

## Resolution **Done:** The Stop hook records only a Prediction shown after the pending invocation. A held-out session with a pending invocation gets one marker and no append. An unparseable invocation time raises ArgumentError. Entry owns the before-time rule (Entry#before?). Finished by hand after implementation 293 failed at test-reviewer (tests under plugins/os-me/hooks/test are outside the target test path; root ticket #1276). **Evidence:** Commit 3332c596e on branch ticket-1762, merged to main on 2026-11-13. os-me suite: 466 runs, 837 assertions, 0 failures (bundle exec ruby test/all.rb in plugins/os-me). Rubocop: 11 files, no offenses. One design audit ran on the diff; its 7 findings (2 must-fix, 5 advisory) are all fixed; red proofs exist for the criteria 2, 8 and 16 tests. Plugin cache refreshed by worktree finish. **Follow-ups:** #1276 (second occurrence recorded: hook tests outside test_path). #1763 (quoted Prediction blocks in agent reports become rows). Store data for Jared: two held-out records already in prediction-log.jsonl (sessions ae4be489, 9d8c2e15) are not removed. The append step still reads the whole transcript; only the ready check uses the invocation time: dropped, #1758's copy rule and id positions cover it.

jared closed this issue

2026-11-13 21:45:35 +00:00

jared referenced this issue from a commit

2026-11-13 21:52:39 +00:00

fix(os-me): Record only fresh predictions at Stop

jared referenced this issue from a commit

2026-11-13 21:52:39 +00:00

docs(os-me): Record 2026-11-13 Outcome Engine work