os-me: the predictor returns YAML; PredictionParser reads YAML and the old line format #1292
Labels
No labels
P0
P1
P2
P3
bug
category/code-change
category/cross-repo
category/eval-run
category/investigation
category/no-test-target
category/owner-ruling
category/prose-artifact
category/recurring-chore
category/structure
category/umbrella
crc-refactor
create
delete
enhancement
filed-by/agent
filed-by/user
frozen
lint-rule
needs-info
needs-triage
next
plugin/cc-architect
plugin/os
plugin/os-adr
plugin/os-aidd-lint
plugin/os-backlog
plugin/os-context
plugin/os-doc-hygiene
plugin/os-sdlc
plugin/os-vault
project/cc-os
ready-for-agent
ready-for-human
recurring
review
rule-proposal
update
waiting
wayfinder:grilling
wayfinder:map
wayfinder:map
wayfinder:research
wayfinder:task
wayfinder:task
wontfix
worklist/deviations
worklist/lint-rule
worklist/new-implement-build
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
jared/cc-os#1292
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem
The predict-mode answer format is prose in
~/dev/os-me-store/ENTRY.mdsection 5: aPREDICTIONheader, thenAction:,Toward:,He would type:, aWhy:list, andConfidence:lines. Step 1 item 5 ofplugins/os-me/skills/predict/SKILL.mdaddsSource:,Cost:,Band:, andEvidence:lines. Code reads this text with regexes inplugins/os-me/lib/os_me/prediction_parser.rb(RULING,CONFIDENCE,WHY,TRIGGER,LEVER_LINES). A small change in format, such as bold markup, a moved line, or a percent sign that is not there, gives a wrong or empty row and no error.Jared ruled on 2026-10-23 that agent output that code reads must be machine-readable. Thus the predictor must return YAML. Old transcripts keep the line format and stay read-only, and the Stop-hook recorder and the extractor still read them. Thus the parser must read both formats. Past data is not converted: the Store
prediction-log.jsonlis already JSON.Readers of the format (grep for
PredictionParser,prediction_candidates,Confidence:,PREDICTION,Toward):plugins/os-me/lib/os_me/prediction_parser.rb: the only parser (last_prediction,prediction?,parse, andtowardfrom the blocking ticket).plugins/os-me/hooks/cli_appender.rb(prediction_shown?, the Stop-hook ready check) callslast_prediction.plugins/os-me/hooks/record_prediction.rbandplugins/os-me/hooks/prediction_recorder.rb: the Stop hook. It callsCliAppender, which runsOsMe::CLIappend.plugins/os-me/lib/os_me/cli/append_request.rb(append) callslast_predictionandparse, then writes the rows throughPredictionLog.plugins/os-me/lib/os_me/transcript_reader.rb(prediction_candidates) supplies the texts. It does not parse the format.plugins/os-me/lib/os_me/prediction_log.rbreads and writes JSONL rows only. It does not read the transcript format. No change.plugins/os-me/lib/os_me/confidence_extractor.rb, called frommoment_extractor.rb(stated_percent), reads a free-text percent from any AI message. A YAMLconfidence: 80has no%, so without a change every new prediction message givesstated_percentnil. S10 calibration needs that number, so this ticket makesConfidenceExtractortake the confidence from a YAML prediction block first.plugins/os-me/lib/os_me/predictor.rbandscorer.rbgive the prediction text tojudge.rb, which sends it verbatim to a model througheval/judge-prompt.md. No regex. No change.Approach
Decisions:
YAML shape. The predictor returns one fenced block with the info string
yaml. Its only top key isprediction::actionis one of accept, correct, or redirect.whyis a list of one to three quoted strings.confidenceis an integer from 0 to 100, with no%.source,cost,band, andevidenceare present only when the brief has row labels or the target-confidence step ran. One block holds one ruling, with the idsession:1. Lettered multi-ruling predictions stay readable only in the old format.Display. Skill step 3 shows the agent's result as is under the heading "Prediction": the YAML block, not changed. The blind reminder line from step 2 goes after the closing fence. Thus the transcript holds the YAML block, and the recorder parses it there. No markdown rendering.
Format detection. A text uses the YAML format when it has a line that is exactly
```yaml(or```yml) and the next line starts withprediction:. The block body runs to the next```line, or to the end of the text. If a text has more than one such block, the last block is used. All other text goes to the existing line path, which does not change.prediction?uses only this detection and the existingCONFIDENCEline, so it never raises.Malformed block. When a detected block does not load with
YAML.safe_load, whenprediction:is not a mapping, or whenconfidenceis not an integer from 0 to 100,parseandtowardraiseOsMe::PredictionParser::MalformedPrediction(anOsMe::Error). The message names the fault. The error surfaces inOsMe::CLI.dispatch, which changes it to aThor::Error.appendthen exits 1 and writes the message to stderr. In the Stop hook this is a hook error that the session shows.mark_recordeddoes not run, so no row is written and the pending invocation stays unrecorded.Orchestrator work: in the same change, the orchestrator makes these text edits, because the pipeline tests cover only the parser. (a) In
~/dev/os-me-store/ENTRY.mdsection 5 (the Store repo, not cc-os), replace thePREDICTIONline block with the YAML block of decision 1, inside a four-backtick outer fence. State "return exactly this fenced yaml block, nothing before it; quote every string that holds a colon". (b) In~/dev/os-me-store/brief.mdline 18, replace "under a "## Prediction" heading, with the action, reasons, and a confidence number" with "as the fenced yaml block of ENTRY.md section 5". (c) Inplugins/os-me/skills/predict/SKILL.md: in step 1, change theEvidence: <before>% -> <after>%line instruction to theevidence:mapping, and change item 5 to "Add the keyssource,cost,band, andevidenceto theprediction:mapping". In step 2, exit 1, put the printed line after the closing fence. In step 3, show the result as is under the heading "Prediction". Then runbin/refresh-plugins, and commit the Store edit in~/dev/os-me-store.Required work:
PredictionParserdetects the fencedprediction:YAML block, andOsMe::PredictionParser::YamlPredictionloads it withYAML.safe_load. check: presentYAML.safe_loadinplugins/os-me/lib/os_me/prediction_parser/yaml_prediction.rbOsMe::PredictionParser::MalformedPrediction < OsMe::Errorin its own file, likeprediction_log/unknown_id.rb. check: presentclass MalformedPrediction < OsMe::Errorinplugins/os-me/lib/os_me/prediction_parser/malformed_prediction.rbPredictionParser#towardreturns thetowardkey of a YAML block throughYamlPrediction#towardand keeps the line path for old text. check: presentdef towardinplugins/os-me/lib/os_me/prediction_parser/yaml_prediction.rbConfidenceExtractorreturns theconfidenceof a YAML prediction block when the text has one, else its existing percent path. A malformed block raisesMalformedPrediction. check: presentPredictionParserinplugins/os-me/lib/os_me/confidence_extractor.rbAcceptance criteria
action: correct,toward: "Keep it in Gate.", andconfidence: 80, and no lever keys, whenPredictionParser#parsereads it, then it returns one row withid: 'session:1',text: 'correct: Keep it in Gate.', andconfidence: 80, and the row has no lever keys.whylist is["T2: widens Runner", "framing 6", "A5 and T2 apply"], whenparsereads it, thentriggersis["T2", "A5"].source: precedent,cost: medium,band: 80, andevidence: {before: 55, after: 82, moved_by: "R12"}, whenparsereads it, then the row hassource: 'precedent',cost: 'medium',band: 80,evidence_before: 55, andevidence_after: 82.PREDICTION,Action: accept,Toward: ...,He would type: ...,Why:with a- T2 ...bullet,Confidence: 70%), whenparsereads it, then it returns one row with idsession:1and confidence 70, as before this change. The existingprediction_parser_test.rb,lever_fields_test.rb, andprediction_fields_test.rbcases pass unchanged.["no prediction here", <YAML block text with no "Confidence:" line>, "Done."], whenlast_predictionreads them, then it returns the YAML block text, andprediction?is true for that text.```yamlblock whose first key isplan:, whenprediction?reads it, then it returns false.why: [unclosed), whenparsereads it, then it raisesOsMe::PredictionParser::MalformedPrediction, andprediction?on the same text returns true and does not raise. WhenOsMe::CLIappendreads a transcript whose last assistant message is that text, it exits 1 with the message on stderr, and the log has no row.confidence: "80%", orconfidence: 150, or noconfidencekey, whenparsereads it, then it raisesMalformedPredictionwith a message that containsconfidence.OsMe::CLIappendruns with--origin agent --autopilot false, then it exits 0, andPredictionLog#recordshas one row with idsession:1, confidence 80, band 80, evidence_before 55, and evidence_after 82.toward: "Keep it in Gate.", whenPredictionParser#towardreads it, then it returns"Keep it in Gate.". Given the old text with the lineToward: Keep it in Gate.,towardreturns the same value.confidence: 80and no%anywhere, whenConfidenceExtractorreads it, then the stated percent is 80. Given a message with no YAML block and the lineI recommend B, 70% confident., then the stated percent is 70, as before this change.Owner sketch
Scope and blocking
PredictionParser#toward(text)for theToward:value. This ticket extendstowardto the YAML block.plugins/os-me/lib/os_me/confidence_extractor.rb,plugins/os-me/lib/os_me/prediction_parser.rb,plugins/os-me/lib/os_me/prediction_parser/yaml_prediction.rb, andplugins/os-me/lib/os_me/prediction_parser/malformed_prediction.rb. Tests go only underplugins/os-me/test.~/dev/os-me-store/ENTRY.md,~/dev/os-me-store/brief.md, andplugins/os-me/skills/predict/SKILL.mdedits listed in Approach.prediction-log.jsonl; a YAML form for lettered multi-ruling predictions; the wording of theappenderror "no Prediction with a Confidence line".Origin
DESIGN(~/dev/os-me-store/ENTRY.mdsection 5,plugins/os-me/lib/os_me/prediction_parser.rb).VERDICT: CREATE (user-approved; backlog search only)
Work started on branch ticket-1292 (os-sdlc implement). #1289 is merged, so the toward method exists.
Resolution
Done: Predictor returns a fenced YAML prediction block; PredictionParser reads it and the old line format; ConfidenceExtractor reads YAML confidence; predict skill shows the block verbatim; Store ENTRY.md section 5 and brief.md updated
Evidence: Implementation 155 complete in 6 min 45 s, 10 dispatches, all pass; suite gate green
Follow-ups: #1299 uses PredictionParser#toward on the predictor agent result; no YAML form for lettered multi-ruling predictions (deferred, not filed: no current use); append error text names a Confidence line (deferred, cosmetic)