jared/cc-os

Watch

Fork 0

os-me: the prediction parser reads every Prediction block shape and every block of a message #1758

New issue

Closed

opened 2026-11-13 20:10:58 +00:00 by jared · 3 comments

jared commented

2026-11-13 20:10:58 +00:00

Owner

Problem

Some /os-me:predict runs leave no row in ~/dev/os-me-store/prediction-log.jsonl. A run with no row can never get a score, and bin/os-me-report counts too few predictions. The parser causes most of the loss: it reads one block shape, one block of a message, and one message of a transcript. The code is Ruby, in plugins/os-me; the tests are Minitest.

This ticket was an investigation ticket. The body was rewritten on 2026-11-13 after the causes were found, and again in the Decide phase of 2026-11-13, which moved the Stop hook causes to #1762.

Evidence, from the source at commit 43bdc0800 (#1233) and from non-held-out transcripts, checked on 2026-11-13:

PredictionParser::YAML_BLOCK (plugins/os-me/lib/os_me/prediction_parser.rb:7) needs prediction: on the first line after the yaml fence. It rejects four shapes that sessions write (shapes 1 to 4 below). For these blocks prediction? is false and the session gets no row. Sessions: c2e6868b, 9e888d99, 208fbb3f, bbe5ea91 (shape 1); 19f78576, a48658d5 (shape 2); a48658d5 (shapes 3 and 4).
PredictionParser#yaml_block (prediction_parser.rb:46) takes the last block of a message. Sessions f8d22c34 and b9b5b401 each have a message with three prediction: blocks. The log has one row for each session, and that row holds the third block.
PredictionParser#last_positioned_prediction (prediction_parser.rb:18-21) returns only the last message that has a Prediction. AppendRequest#append (plugins/os-me/lib/os_me/cli/append_request.rb:11-18) records only that message.
A transcript holds most Predictions two times. A subagent makes the Prediction, and the session shows the result again (plugins/os-me/skills/predict/SKILL.md step 2). The first copy is in a task-notification result. The second copy is in the next assistant message that has a Prediction. last_positioned_prediction counts the two copies as positions 1 and 2. With the code of 43bdc0800, os-me-eval append over the transcript of session 6098efae (one Prediction, recorded as session:1 in the Store log) gives the id session:2:1 on a scratch log. Sessions 974224e0, b9b5b401, bbe5ea91, and 5aee9255 give session:2:1 also.
In each pair of copies examined (sessions 19f78576, a48658d5, f8d22c34, 47ffa777, 6098efae, 974224e0, b9b5b401, bbe5ea91, 01fe836c, 5aee9255), the two messages have the same confidence values in the same sequence. The toward text is not always equal (a48658d5, f8d22c34, 47ffa777).
Session c2e6868b has two different Predictions in two assistant messages, each with confidence 80, and no task-notification result between them.
Session 01fe836c has a task-notification result with a prediction: block that does not load (MalformedPrediction: prediction YAML does not load). The copy in the next assistant message loads.

Reproduce (always pass a scratch --log; never write test rows to the Store log):

cd ~/dev/cc-os/plugins/os-me
bundle exec bin/os-me-eval append ~/.claude/projects/-home-jared-dev-cc-os/c2e6868b-f4fd-4728-baf0-9f28d44f19c4.jsonl --log /tmp/scratch-log.jsonl; echo $?

Today this prints "no Prediction with a Confidence line in " and exits 1.

Approach

The owner is OsMe::PredictionParser. OsMe::CLI::AppendRequest is its caller. One change: the parser returns every shown Prediction of a transcript as positioned rows, and AppendRequest#append gives all of them to PredictionLog#append. The dedupe of PredictionLog#append on [id, session_id] does not change.

Block shapes. A Prediction block is a fenced yaml block whose first key is prediction:, predictions:, or decision_<n>:, in one of the shapes 0 to 4. Comment lines (# ...) can come between the fence and the first key. The text in these examples is invented.

Shape 0, read today. One block is one Prediction:

prediction:
  action: accept
  toward: "Use option A."
  confidence: 80

Shape 1, comment lines before the first key. One Prediction:

# model: example-model
prediction:
  action: accept
  toward: "Use option A."
  confidence: 80

Shape 2, a predictions: list. Each item has a prediction: mapping. Each item is one Prediction:

predictions:
  - decision: "First question"
    prediction:
      action: accept
      toward: "Use option A."
      confidence: 80
  - decision: "Second question"
    prediction:
      action: correct
      toward: "Use option B."
      confidence: 70

Shape 3, decision_<n>: keys at the top level. Each value has a prediction: mapping. Each key is one Prediction:

decision_1:
  prediction:
    action: accept
    toward: "Use option A."
    confidence: 80
decision_2:
  prediction:
    action: correct
    toward: "Use option B."
    confidence: 70

Shape 4, a predictions: mapping with decision_<n>: keys. Each value holds the fields directly. Each key is one Prediction:

predictions:
  decision_1:
    choice: a
    action: accept
    toward: "Use option A."
    confidence: 80
  decision_2:
    choice: b
    action: correct
    toward: "Use option B."
    confidence: 70

In each real example of shapes 2, 3, and 4, each Prediction has its own action, toward, and confidence. So the row of each Prediction comes from its field mapping with the rules of YamlPrediction#row (plugins/os-me/lib/os_me/prediction_parser/yaml_prediction.rb:24-27): the same text, confidence check, triggers, and lever keys. The keys decision, choice, and ruling are not row fields. Today YamlPrediction#initialize loads the block body and reads its prediction key (yaml_prediction.rb:8-13). After the change YamlPrediction.new(fields) takes one field mapping, and a new PredictionParser::YamlBlock loads the body, finds the shape, and returns one YamlPrediction for each field mapping from YamlBlock#predictions. YamlBlock raises the two present MalformedPrediction errors: the YAML does not load, and the Prediction is not a mapping.

Shape 5 is not a Prediction block. Its decision_<n>: values have no prediction: mapping and no action, so a row has no action text:

decision_1:
  choice: a
  toward: "Use option A."
  confidence: 80

The one real example (session a48658d5) is the assistant copy of a shape 4 block in a task-notification result, so its Predictions get their rows from shape 4. prediction? is false for shape 5, and shape 5 gives no rows and no error.

A yaml block whose first key is some other key, for example plan:, is not a Prediction block, with or without comment lines before the key.

Messages. AppendRequest#append gives the parser all prediction candidates of the transcript in transcript sequence, each with its source: assistant text or task-notification result. Entry#assistant_text and Entry#notification_text (plugins/os-me/lib/os_me/entry.rb:24-38) already separate the two. TranscriptReader#prediction_messages returns [source, text] pairs, where the source is :assistant or :notification. TranscriptReader#prediction_candidates does not change, because CliAppender#prediction_shown? uses it.

Predictions of a message. A message gives its Predictions in the sequence shown: each Prediction block from first to last, and in a block each list item or decision_<n>: key from first to last. A message with a Confidence line and no Prediction block is one Prediction with one row for each ruling, as today (LegacyPrediction#rows).

Copies. An assistant message is a copy when the nearest earlier message that gives rows is a task-notification result, and the rows of the two messages have the same confidence values in the same sequence. A copy gives no rows and takes no position. Two assistant messages are never copies of each other.

Blocks that do not load. A Prediction block that raises MalformedPrediction in a task-notification result gives no rows, and that block is not an error. A Prediction block that raises MalformedPrediction in an assistant message raises, as today. This rule is in positioned_rows only, because only positioned_rows knows the source. prediction? stays true for a block that does not load, and parse still raises for it (plugins/os-me/test/prediction_yaml_test.rb:62-66 pins the two results). The Stop hook path does not change for a task-notification result with a block that does not load and no assistant copy yet: append fails, the hook writes no Stop marker, and the next Stop tries again.

Id scheme. Number the Predictions of the transcript 1, 2, 3, and so on, in the sequence shown. Copies have no number. Prediction 1 keeps the id that YamlPrediction#row and LegacyPrediction#rows give today: session:1, or session:<letter> for each lettered ruling. Prediction n, for n above 1, gets session:<n>:1, or session:<n>:<letter>. This is at_position(ruling, n) (prediction_parser.rb:40-44) with no change. PredictionParser#parse(text, position:) stays.

Why this scheme keeps the present ids: each of the 84 session:<n> rows of the Store log has the id session:1 (count of 2026-11-13; the other 111 rows have ids of one capital letter, from before the hook). No row has the form session:<n>:<m>. A transcript with one Prediction, with or without its copy, gives session:1. A later Prediction never changes the number of an earlier one, so a second run gives the same ids, and the dedupe adds nothing.

AppendRequest#append raises the present Thor::Error ("no Prediction with a Confidence line in ") only when the transcript gives no rows.

PredictionParser#toward and PredictionParser#stated_confidence read the last Prediction of the text: the last Prediction block, and in it the last item or key. After the change they read shapes 1 to 4 for each caller: plugins/os-me/lib/os_me/confidence_extractor.rb:10 (used by moment_builder.rb and confidence_score.rb) and plugins/os-me/lib/os_me/moment_topic.rb:31,35. Shapes 1 to 4 behave for these callers as shape 0 does today. A block that does not load raises MalformedPrediction from the two methods. A confidence that is not an integer 0 to 100 raises MalformedPrediction from stated_confidence; toward does not examine the confidence. The callers do not rescue the error today, and they do not rescue it after the change. A good shape 2, 3, or 4 block gives the callers a value and no error.

These tests pin the present behavior of the parser and its callers and must stay green: test/confidence_extractor_yaml_test.rb, test/confidence_extractor_test.rb, test/draft_extraction_test.rb, test/candidate_refresh_agent_toward_test.rb, test/candidate_refresh_topic_test.rb, test/prediction_parser_test.rb, test/lever_fields_test.rb, and test/prediction_fields_test.rb (paths from plugins/os-me).

Required work:

PredictionParser#prediction? and the block reader accept shapes 1 to 4 and refuse shape 5 and other first keys.
PredictionParser#positioned_rows(messages) returns the rows of every Prediction of the transcript, with copies removed and the ids of the id scheme — check: present def positioned_rows in plugins/os-me/lib/os_me/prediction_parser.rb
TranscriptReader#prediction_messages returns the candidates with their source — check: present def prediction_messages in plugins/os-me/lib/os_me/transcript_reader.rb
AppendRequest#append appends all rows of positioned_rows; last_positioned_prediction has no caller and is removed — check: absent last_positioned_prediction in plugins/os-me/lib/os_me/**/*.rb
PredictionParser::YamlBlock loads a block body and returns one YamlPrediction for each field mapping — check: present class YamlBlock in plugins/os-me/lib/os_me/prediction_parser/yaml_block.rb

Orchestrator work: add one History row to docs/implementation-status/os-me.md (the text "reads the last fenced prediction: yaml block" is in the History row of 2026-10-23, which stays as a record), and update the OsMe::PredictionParser card in docs/decision-system/current-state.md.

Rejected: positions that count messages, as last_positioned_prediction does — each copy takes a position, so the recorded transcript of 6098efae gets session:2:1 in place of session:1.

Rejected: a copy rule that compares the toward text — the copies of a48658d5, f8d22c34, and 47ffa777 change the text.

Rejected: a copy rule that compares only confidence values, without the source — session c2e6868b has two different Predictions with confidence 80.

Rejected: rows for shape 5 — it has no action, and its one real example is a copy.

Acceptance criteria

Given a transcript with one assistant message that has a shape 1 block (a comment line, then prediction:), when os-me-eval append runs, then the log gets one row with the id session:1 and the confidence of the block.
Given a transcript with one message that has a shape 2 block with two items, when os-me-eval append runs, then the log gets two rows with the ids session:1 and session:2:1, and each row has the confidence and the text of its item.
Given a transcript with one message that has a shape 3 block with two decision_<n>: keys, when os-me-eval append runs, then the log gets two rows with the ids session:1 and session:2:1.
Given a transcript with one message that has a shape 4 block with two decision_<n>: keys, when os-me-eval append runs, then the log gets two rows with the ids session:1 and session:2:1.
Given a transcript with a shape 4 block in a task-notification result and a shape 5 block in the next assistant message, when os-me-eval append runs, then the log gets rows only for the Predictions of the shape 4 block, and the command exits 0.
Given a transcript with one message that has three prediction: blocks, when os-me-eval append runs, then the log gets three rows with the ids session:1, session:2:1, and session:3:1, in block sequence.
Given a transcript with a prediction: block in a task-notification result and a block with the same confidence in the next assistant message, when os-me-eval append runs, then the log gets one row with the id session:1.
Given a log that already has a row with the id session:1 for the session, and the transcript of criterion 7, when os-me-eval append runs, then the log still has one row for the session.
Given a transcript with two assistant messages, each with one prediction: block with the same confidence, and no task-notification result between them, when os-me-eval append runs, then the log gets two rows with the ids session:1 and session:2:1.
Given a transcript with a task-notification result whose prediction: block does not load, and a prediction: block that loads in the next assistant message, when os-me-eval append runs, then the log gets one row with the id session:1, and the command exits 0.
Given a transcript with two task-notification results, each followed by its assistant copy, and the first result has three prediction: blocks, when os-me-eval append runs two times, then the log has four rows after the first run and the same four rows after the second run.
Given a transcript whose only yaml block has a comment line and then plan:, when os-me-eval append runs, then the log gets no row and the command prints "no Prediction with a Confidence line".
Given a text whose last yaml block is a shape 2 block, when the parser gives the stated confidence of the text, then the result is the confidence of the last item, and no error occurs.
Given a text whose last yaml block is a shape 2 block, and the confidence of its last item is not an integer 0 to 100, when the confidence extractor reads the text, then it fails with a MalformedPrediction error that names the confidence, as it does today for a shape 0 block.

Owner sketch

class OsMe::PredictionParser
  def positioned_rows(messages)
  def prediction?(text)
  def toward(text)
  def stated_confidence(text)
  def parse(text, position: 1)
end

class OsMe::PredictionParser::YamlBlock
  def initialize(body)
  def predictions
end

class OsMe::PredictionParser::YamlPrediction
  def initialize(fields)
  def row
end

class OsMe::TranscriptReader
  def prediction_messages
end

class OsMe::CLI::AppendRequest
  def append(transcript_path)
end

Non-goals

In scope: the block shapes that the parser reads; one row for each Prediction of a transcript; the copy rule; the id scheme; os-me-eval append.

Out of scope:

The ready check of the Stop hook and the held-out check of the Stop hook (#1762).
Verdict scoring of the recovered rows (#1753).
Removal or repair of rows that are in the log now. This includes the session:1 rows of f8d22c34 and b9b5b401, which hold the third Prediction of a message with three blocks. For these two sessions a new run drops Prediction 1 (the id session:1 is taken) and adds Prediction 3 again as session:3:1. That repair is a Store data decision for Jared.
A new run of append over the sessions that are in the Store log now is a Store data decision for Jared. This ticket adds no migration.
The Store script ~/dev/os-me-store/bin/rederive-wait-records. It is a one-time repair script of 2026-10-22 that was used; this ticket does not update it.
An option of os-me-eval append that names one invocation or one block (cause d of the first body). It is not necessary when one run records every Prediction.
Rows for shape 5.

Blocking edges: none. This ticket and #1762 can ship in either order.

Origin

Trigger: the os-me check of 2026-11-13 in session 2bb44e9c. Jared asked how the os-me outcome function works and said "We definitely need to get this back on track. /os-me:predict desired outcome, how to get there, then do it!" The applied Prediction (docs/decisions/rulings.tsv row 161) sends this work to a ticket.
Improvised this session: none
Chain: predict run has no log row ← the Stop hook returns with no message when the parser finds no Prediction, and marks an invocation as recorded from an older Prediction ← the parser accepts one block shape and one block per message, and no step limits the read to the invocation ← DESIGN (#1079 append dedupe; #1081 id assignment; #1292 fenced YAML prediction block)
Root candidate: none found; searched prediction_recorder, record_prediction, prediction-log, PredictionRecorder, Stop hook, no row
Where: plugins/os-me/lib/os_me/prediction_parser.rb; plugins/os-me/hooks/prediction_recorder.rb; plugins/os-me/hooks/cli_appender.rb; plugins/os-me/lib/os_me/cli/append_request.rb
Session: 2bb44e9c-522c-4c97-8449-bd46bed898e0
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/2bb44e9c-522c-4c97-8449-bd46bed898e0.jsonl

VERDICT: CREATE (ticket-skeptic, 2026-11-13): six one-invocation sessions have no prediction-log row; #1233 covers a different fault. Causes found and body rewritten on 2026-11-13.

## Problem Some `/os-me:predict` runs leave no row in `~/dev/os-me-store/prediction-log.jsonl`. A run with no row can never get a score, and `bin/os-me-report` counts too few predictions. The parser causes most of the loss: it reads one block shape, one block of a message, and one message of a transcript. The code is Ruby, in `plugins/os-me`; the tests are Minitest. This ticket was an investigation ticket. The body was rewritten on 2026-11-13 after the causes were found, and again in the Decide phase of 2026-11-13, which moved the Stop hook causes to #1762. Evidence, from the source at commit 43bdc0800 (#1233) and from non-held-out transcripts, checked on 2026-11-13: - `PredictionParser::YAML_BLOCK` (`plugins/os-me/lib/os_me/prediction_parser.rb:7`) needs `prediction:` on the first line after the yaml fence. It rejects four shapes that sessions write (shapes 1 to 4 below). For these blocks `prediction?` is false and the session gets no row. Sessions: c2e6868b, 9e888d99, 208fbb3f, bbe5ea91 (shape 1); 19f78576, a48658d5 (shape 2); a48658d5 (shapes 3 and 4). - `PredictionParser#yaml_block` (`prediction_parser.rb:46`) takes the last block of a message. Sessions f8d22c34 and b9b5b401 each have a message with three `prediction:` blocks. The log has one row for each session, and that row holds the third block. - `PredictionParser#last_positioned_prediction` (`prediction_parser.rb:18-21`) returns only the last message that has a Prediction. `AppendRequest#append` (`plugins/os-me/lib/os_me/cli/append_request.rb:11-18`) records only that message. - A transcript holds most Predictions two times. A subagent makes the Prediction, and the session shows the result again (`plugins/os-me/skills/predict/SKILL.md` step 2). The first copy is in a task-notification result. The second copy is in the next assistant message that has a Prediction. `last_positioned_prediction` counts the two copies as positions 1 and 2. With the code of 43bdc0800, `os-me-eval append` over the transcript of session 6098efae (one Prediction, recorded as `session:1` in the Store log) gives the id `session:2:1` on a scratch log. Sessions 974224e0, b9b5b401, bbe5ea91, and 5aee9255 give `session:2:1` also. - In each pair of copies examined (sessions 19f78576, a48658d5, f8d22c34, 47ffa777, 6098efae, 974224e0, b9b5b401, bbe5ea91, 01fe836c, 5aee9255), the two messages have the same `confidence` values in the same sequence. The `toward` text is not always equal (a48658d5, f8d22c34, 47ffa777). - Session c2e6868b has two different Predictions in two assistant messages, each with confidence 80, and no task-notification result between them. - Session 01fe836c has a task-notification result with a `prediction:` block that does not load (`MalformedPrediction: prediction YAML does not load`). The copy in the next assistant message loads. Reproduce (always pass a scratch `--log`; never write test rows to the Store log): ``` cd ~/dev/cc-os/plugins/os-me bundle exec bin/os-me-eval append ~/.claude/projects/-home-jared-dev-cc-os/c2e6868b-f4fd-4728-baf0-9f28d44f19c4.jsonl --log /tmp/scratch-log.jsonl; echo $? ``` Today this prints "no Prediction with a Confidence line in <path>" and exits 1. ## Approach The owner is `OsMe::PredictionParser`. `OsMe::CLI::AppendRequest` is its caller. One change: the parser returns every shown Prediction of a transcript as positioned rows, and `AppendRequest#append` gives all of them to `PredictionLog#append`. The dedupe of `PredictionLog#append` on `[id, session_id]` does not change. Block shapes. A Prediction block is a fenced yaml block whose first key is `prediction:`, `predictions:`, or `decision_<n>:`, in one of the shapes 0 to 4. Comment lines (`# ...`) can come between the fence and the first key. The text in these examples is invented. Shape 0, read today. One block is one Prediction: ```yaml prediction: action: accept toward: "Use option A." confidence: 80 ``` Shape 1, comment lines before the first key. One Prediction: ```yaml # model: example-model prediction: action: accept toward: "Use option A." confidence: 80 ``` Shape 2, a `predictions:` list. Each item has a `prediction:` mapping. Each item is one Prediction: ```yaml predictions: - decision: "First question" prediction: action: accept toward: "Use option A." confidence: 80 - decision: "Second question" prediction: action: correct toward: "Use option B." confidence: 70 ``` Shape 3, `decision_<n>:` keys at the top level. Each value has a `prediction:` mapping. Each key is one Prediction: ```yaml decision_1: prediction: action: accept toward: "Use option A." confidence: 80 decision_2: prediction: action: correct toward: "Use option B." confidence: 70 ``` Shape 4, a `predictions:` mapping with `decision_<n>:` keys. Each value holds the fields directly. Each key is one Prediction: ```yaml predictions: decision_1: choice: a action: accept toward: "Use option A." confidence: 80 decision_2: choice: b action: correct toward: "Use option B." confidence: 70 ``` In each real example of shapes 2, 3, and 4, each Prediction has its own `action`, `toward`, and `confidence`. So the row of each Prediction comes from its field mapping with the rules of `YamlPrediction#row` (`plugins/os-me/lib/os_me/prediction_parser/yaml_prediction.rb:24-27`): the same text, confidence check, triggers, and lever keys. The keys `decision`, `choice`, and `ruling` are not row fields. Today `YamlPrediction#initialize` loads the block body and reads its `prediction` key (`yaml_prediction.rb:8-13`). After the change `YamlPrediction.new(fields)` takes one field mapping, and a new `PredictionParser::YamlBlock` loads the body, finds the shape, and returns one `YamlPrediction` for each field mapping from `YamlBlock#predictions`. `YamlBlock` raises the two present `MalformedPrediction` errors: the YAML does not load, and the Prediction is not a mapping. Shape 5 is not a Prediction block. Its `decision_<n>:` values have no `prediction:` mapping and no `action`, so a row has no action text: ```yaml decision_1: choice: a toward: "Use option A." confidence: 80 ``` The one real example (session a48658d5) is the assistant copy of a shape 4 block in a task-notification result, so its Predictions get their rows from shape 4. `prediction?` is false for shape 5, and shape 5 gives no rows and no error. A yaml block whose first key is some other key, for example `plan:`, is not a Prediction block, with or without comment lines before the key. Messages. `AppendRequest#append` gives the parser all prediction candidates of the transcript in transcript sequence, each with its source: assistant text or task-notification result. `Entry#assistant_text` and `Entry#notification_text` (`plugins/os-me/lib/os_me/entry.rb:24-38`) already separate the two. `TranscriptReader#prediction_messages` returns `[source, text]` pairs, where the source is `:assistant` or `:notification`. `TranscriptReader#prediction_candidates` does not change, because `CliAppender#prediction_shown?` uses it. Predictions of a message. A message gives its Predictions in the sequence shown: each Prediction block from first to last, and in a block each list item or `decision_<n>:` key from first to last. A message with a Confidence line and no Prediction block is one Prediction with one row for each ruling, as today (`LegacyPrediction#rows`). Copies. An assistant message is a copy when the nearest earlier message that gives rows is a task-notification result, and the rows of the two messages have the same `confidence` values in the same sequence. A copy gives no rows and takes no position. Two assistant messages are never copies of each other. Blocks that do not load. A Prediction block that raises `MalformedPrediction` in a task-notification result gives no rows, and that block is not an error. A Prediction block that raises `MalformedPrediction` in an assistant message raises, as today. This rule is in `positioned_rows` only, because only `positioned_rows` knows the source. `prediction?` stays true for a block that does not load, and `parse` still raises for it (`plugins/os-me/test/prediction_yaml_test.rb:62-66` pins the two results). The Stop hook path does not change for a task-notification result with a block that does not load and no assistant copy yet: `append` fails, the hook writes no Stop marker, and the next Stop tries again. Id scheme. Number the Predictions of the transcript 1, 2, 3, and so on, in the sequence shown. Copies have no number. Prediction 1 keeps the id that `YamlPrediction#row` and `LegacyPrediction#rows` give today: `session:1`, or `session:<letter>` for each lettered ruling. Prediction n, for n above 1, gets `session:<n>:1`, or `session:<n>:<letter>`. This is `at_position(ruling, n)` (`prediction_parser.rb:40-44`) with no change. `PredictionParser#parse(text, position:)` stays. Why this scheme keeps the present ids: each of the 84 `session:<n>` rows of the Store log has the id `session:1` (count of 2026-11-13; the other 111 rows have ids of one capital letter, from before the hook). No row has the form `session:<n>:<m>`. A transcript with one Prediction, with or without its copy, gives `session:1`. A later Prediction never changes the number of an earlier one, so a second run gives the same ids, and the dedupe adds nothing. `AppendRequest#append` raises the present `Thor::Error` ("no Prediction with a Confidence line in <path>") only when the transcript gives no rows. `PredictionParser#toward` and `PredictionParser#stated_confidence` read the last Prediction of the text: the last Prediction block, and in it the last item or key. After the change they read shapes 1 to 4 for each caller: `plugins/os-me/lib/os_me/confidence_extractor.rb:10` (used by `moment_builder.rb` and `confidence_score.rb`) and `plugins/os-me/lib/os_me/moment_topic.rb:31,35`. Shapes 1 to 4 behave for these callers as shape 0 does today. A block that does not load raises `MalformedPrediction` from the two methods. A confidence that is not an integer 0 to 100 raises `MalformedPrediction` from `stated_confidence`; `toward` does not examine the confidence. The callers do not rescue the error today, and they do not rescue it after the change. A good shape 2, 3, or 4 block gives the callers a value and no error. These tests pin the present behavior of the parser and its callers and must stay green: `test/confidence_extractor_yaml_test.rb`, `test/confidence_extractor_test.rb`, `test/draft_extraction_test.rb`, `test/candidate_refresh_agent_toward_test.rb`, `test/candidate_refresh_topic_test.rb`, `test/prediction_parser_test.rb`, `test/lever_fields_test.rb`, and `test/prediction_fields_test.rb` (paths from `plugins/os-me`). Required work: 1. `PredictionParser#prediction?` and the block reader accept shapes 1 to 4 and refuse shape 5 and other first keys. 2. `PredictionParser#positioned_rows(messages)` returns the rows of every Prediction of the transcript, with copies removed and the ids of the id scheme — check: present `def positioned_rows` in `plugins/os-me/lib/os_me/prediction_parser.rb` 3. `TranscriptReader#prediction_messages` returns the candidates with their source — check: present `def prediction_messages` in `plugins/os-me/lib/os_me/transcript_reader.rb` 4. `AppendRequest#append` appends all rows of `positioned_rows`; `last_positioned_prediction` has no caller and is removed — check: absent `last_positioned_prediction` in `plugins/os-me/lib/os_me/**/*.rb` 5. `PredictionParser::YamlBlock` loads a block body and returns one `YamlPrediction` for each field mapping — check: present `class YamlBlock` in `plugins/os-me/lib/os_me/prediction_parser/yaml_block.rb` Orchestrator work: add one History row to `docs/implementation-status/os-me.md` (the text "reads the last fenced `prediction:` yaml block" is in the History row of 2026-10-23, which stays as a record), and update the `OsMe::PredictionParser` card in `docs/decision-system/current-state.md`. Rejected: positions that count messages, as `last_positioned_prediction` does — each copy takes a position, so the recorded transcript of 6098efae gets `session:2:1` in place of `session:1`. Rejected: a copy rule that compares the `toward` text — the copies of a48658d5, f8d22c34, and 47ffa777 change the text. Rejected: a copy rule that compares only `confidence` values, without the source — session c2e6868b has two different Predictions with confidence 80. Rejected: rows for shape 5 — it has no `action`, and its one real example is a copy. ## Acceptance criteria 1. Given a transcript with one assistant message that has a shape 1 block (a comment line, then `prediction:`), when `os-me-eval append` runs, then the log gets one row with the id `session:1` and the confidence of the block. 2. Given a transcript with one message that has a shape 2 block with two items, when `os-me-eval append` runs, then the log gets two rows with the ids `session:1` and `session:2:1`, and each row has the confidence and the text of its item. 3. Given a transcript with one message that has a shape 3 block with two `decision_<n>:` keys, when `os-me-eval append` runs, then the log gets two rows with the ids `session:1` and `session:2:1`. 4. Given a transcript with one message that has a shape 4 block with two `decision_<n>:` keys, when `os-me-eval append` runs, then the log gets two rows with the ids `session:1` and `session:2:1`. 5. Given a transcript with a shape 4 block in a task-notification result and a shape 5 block in the next assistant message, when `os-me-eval append` runs, then the log gets rows only for the Predictions of the shape 4 block, and the command exits 0. 6. Given a transcript with one message that has three `prediction:` blocks, when `os-me-eval append` runs, then the log gets three rows with the ids `session:1`, `session:2:1`, and `session:3:1`, in block sequence. 7. Given a transcript with a `prediction:` block in a task-notification result and a block with the same confidence in the next assistant message, when `os-me-eval append` runs, then the log gets one row with the id `session:1`. 8. Given a log that already has a row with the id `session:1` for the session, and the transcript of criterion 7, when `os-me-eval append` runs, then the log still has one row for the session. 9. Given a transcript with two assistant messages, each with one `prediction:` block with the same confidence, and no task-notification result between them, when `os-me-eval append` runs, then the log gets two rows with the ids `session:1` and `session:2:1`. 10. Given a transcript with a task-notification result whose `prediction:` block does not load, and a `prediction:` block that loads in the next assistant message, when `os-me-eval append` runs, then the log gets one row with the id `session:1`, and the command exits 0. 11. Given a transcript with two task-notification results, each followed by its assistant copy, and the first result has three `prediction:` blocks, when `os-me-eval append` runs two times, then the log has four rows after the first run and the same four rows after the second run. 12. Given a transcript whose only yaml block has a comment line and then `plan:`, when `os-me-eval append` runs, then the log gets no row and the command prints "no Prediction with a Confidence line". 13. Given a text whose last yaml block is a shape 2 block, when the parser gives the stated confidence of the text, then the result is the confidence of the last item, and no error occurs. 14. Given a text whose last yaml block is a shape 2 block, and the confidence of its last item is not an integer 0 to 100, when the confidence extractor reads the text, then it fails with a `MalformedPrediction` error that names the confidence, as it does today for a shape 0 block. ## Owner sketch ```ruby class OsMe::PredictionParser def positioned_rows(messages) def prediction?(text) def toward(text) def stated_confidence(text) def parse(text, position: 1) end class OsMe::PredictionParser::YamlBlock def initialize(body) def predictions end class OsMe::PredictionParser::YamlPrediction def initialize(fields) def row end class OsMe::TranscriptReader def prediction_messages end class OsMe::CLI::AppendRequest def append(transcript_path) end ``` ## Non-goals In scope: the block shapes that the parser reads; one row for each Prediction of a transcript; the copy rule; the id scheme; `os-me-eval append`. Out of scope: - The ready check of the Stop hook and the held-out check of the Stop hook (#1762). - Verdict scoring of the recovered rows (#1753). - Removal or repair of rows that are in the log now. This includes the `session:1` rows of f8d22c34 and b9b5b401, which hold the third Prediction of a message with three blocks. For these two sessions a new run drops Prediction 1 (the id `session:1` is taken) and adds Prediction 3 again as `session:3:1`. That repair is a Store data decision for Jared. - A new run of `append` over the sessions that are in the Store log now is a Store data decision for Jared. This ticket adds no migration. - The Store script `~/dev/os-me-store/bin/rederive-wait-records`. It is a one-time repair script of 2026-10-22 that was used; this ticket does not update it. - An option of `os-me-eval append` that names one invocation or one block (cause d of the first body). It is not necessary when one run records every Prediction. - Rows for shape 5. Blocking edges: none. This ticket and #1762 can ship in either order. ## Origin - Trigger: the os-me check of 2026-11-13 in session 2bb44e9c. Jared asked how the os-me outcome function works and said "We definitely need to get this back on track. /os-me:predict desired outcome, how to get there, then do it!" The applied Prediction (docs/decisions/rulings.tsv row 161) sends this work to a ticket. - Improvised this session: none - Chain: predict run has no log row ← the Stop hook returns with no message when the parser finds no Prediction, and marks an invocation as recorded from an older Prediction ← the parser accepts one block shape and one block per message, and no step limits the read to the invocation ← DESIGN (#1079 append dedupe; #1081 id assignment; #1292 fenced YAML prediction block) - Root candidate: none found; searched prediction_recorder, record_prediction, prediction-log, PredictionRecorder, Stop hook, no row - Where: plugins/os-me/lib/os_me/prediction_parser.rb; plugins/os-me/hooks/prediction_recorder.rb; plugins/os-me/hooks/cli_appender.rb; plugins/os-me/lib/os_me/cli/append_request.rb - Session: 2bb44e9c-522c-4c97-8449-bd46bed898e0 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/2bb44e9c-522c-4c97-8449-bd46bed898e0.jsonl VERDICT: CREATE (ticket-skeptic, 2026-11-13): six one-invocation sessions have no prediction-log row; #1233 covers a different fault. Causes found and body rewritten on 2026-11-13.

jared added the

project/cc-os

filed-by/agent

category/investigation

labels

2026-11-13 20:10:58 +00:00

jared changed title from ~~os-me: find why predict runs leave no prediction-log row~~ to os-me: record every shown Prediction; fix the parser and Stop hook faults that lose prediction-log rows

2026-11-13 20:24:45 +00:00

jared referenced this issue

2026-11-13 20:24:48 +00:00

Record every Prediction in a session, not only the first #1233

jared added

category/code-change

and removed

category/investigation

labels

2026-11-13 20:24:53 +00:00

jared referenced this issue

2026-11-13 20:25:50 +00:00

Record every Prediction in a session, not only the first #1233

jared referenced this issue

2026-11-13 20:43:26 +00:00

Record every Prediction in a session, not only the first #1233

jared commented

2026-11-13 20:44:39 +00:00

Author

Owner

Work started on 2026-11-13 through /os-sdlc:implement. Branch: ticket-1758. #1233 (positioned ids) merged first as main 43bdc0800.

Work started on 2026-11-13 through /os-sdlc:implement. Branch: ticket-1758. #1233 (positioned ids) merged first as main 43bdc0800.

jared referenced this issue from a commit

2026-11-13 20:49:46 +00:00

fix(os-context): Define outcome node project name

jared referenced this issue

2026-11-13 20:57:00 +00:00

os-me: the Stop hook records only a Prediction shown after the pending invocation, and skips held-out sessions #1762

jared changed title from ~~os-me: record every shown Prediction; fix the parser and Stop hook faults that lose prediction-log rows~~ to os-me: the prediction parser reads every Prediction block shape and every block of a message

2026-11-13 20:57:06 +00:00

jared commented

2026-11-13 20:57:12 +00:00

Author

Owner

Decide phase 2026-11-13: one change = the parser returns every shown Prediction as positioned rows; owner = PredictionParser. Hook scope moved to #1762. Rulings: held-out sessions are skipped; removal of the two present held-out rows waits for Jared.

jared added the

ready-for-agent

label

2026-11-13 21:02:52 +00:00

jared referenced this issue

2026-11-13 21:24:50 +00:00

os-me: the prediction parser makes rows from Prediction blocks that an agent report only quotes #1763

jared commented

2026-11-13 21:25:10 +00:00

Author

Owner

Resolution

Done: The prediction parser reads four yaml block shapes (comment line before prediction:, predictions: list, decision_N: wrapper, predictions: mapping with decision_N: keys), yields every block of a message, numbers Predictions and not messages, and gives no row to an assistant copy of a task-notification Prediction. os-me-eval append records every Prediction of a transcript; a second run adds nothing. This also removes the #1233 regression that gave the first Prediction of a normal session the id session:2:1.

Evidence: os-sdlc implementation 292, 2026-11-13: 11 dispatches, suite gate green, survivor gate green, contract audit pass; plugin suite 456 runs, 0 failures. Real-data check on 14 non-held-out sessions with a scratch log: 13 correct (4 recorded sessions keep id session:1 and the Store confidence; 9 sessions that lost rows give 37 rows that match an independent count; a second run adds no row). Branch ticket-1758.

Follow-ups: #1763 (filed, skeptic verdict CREATE): the parser makes rows from Prediction blocks that an agent report only quotes; found on session 2bb44e9c in the same check. #1762 (already filed): Stop hook ready check and held-out skip. Store data decision for Jared, no ticket: sessions f8d22c34 and b9b5b401 hold Prediction 3 under session:1; bbe5ea91 has a second Prediction that the Store lacks. Docs (os-me leaf History row, PredictionParser card in current-state.md) are done in the same session as orchestrator work.

## Resolution **Done:** The prediction parser reads four yaml block shapes (comment line before prediction:, predictions: list, decision_N: wrapper, predictions: mapping with decision_N: keys), yields every block of a message, numbers Predictions and not messages, and gives no row to an assistant copy of a task-notification Prediction. os-me-eval append records every Prediction of a transcript; a second run adds nothing. This also removes the #1233 regression that gave the first Prediction of a normal session the id session:2:1. **Evidence:** os-sdlc implementation 292, 2026-11-13: 11 dispatches, suite gate green, survivor gate green, contract audit pass; plugin suite 456 runs, 0 failures. Real-data check on 14 non-held-out sessions with a scratch log: 13 correct (4 recorded sessions keep id session:1 and the Store confidence; 9 sessions that lost rows give 37 rows that match an independent count; a second run adds no row). Branch ticket-1758. **Follow-ups:** #1763 (filed, skeptic verdict CREATE): the parser makes rows from Prediction blocks that an agent report only quotes; found on session 2bb44e9c in the same check. #1762 (already filed): Stop hook ready check and held-out skip. Store data decision for Jared, no ticket: sessions f8d22c34 and b9b5b401 hold Prediction 3 under session:1; bbe5ea91 has a second Prediction that the Store lacks. Docs (os-me leaf History row, PredictionParser card in current-state.md) are done in the same session as orchestrator work.

jared closed this issue

2026-11-13 21:25:10 +00:00

jared referenced this issue

2026-11-13 21:26:42 +00:00

os-me: the Stop hook records only a Prediction shown after the pending invocation, and skips held-out sessions #1762

jared referenced this issue

2026-11-13 21:38:10 +00:00

Map tests under a second test folder of an os-sdlc target (os-me hooks/test) #1276

jared referenced this issue

2026-11-13 21:45:35 +00:00

os-me: the Stop hook records only a Prediction shown after the pending invocation, and skips held-out sessions #1762

jared referenced this issue from a commit