jared/cc-os

Watch

Fork 0

os-me: Stop hook appends prediction records; invocation hook records event and arguments #1075

New issue

Closed

opened 2026-10-17 11:54:35 +00:00 by jared · 3 comments

jared commented

2026-10-17 11:54:35 +00:00

Owner

Problem

hooks/invocation_log.rb drops the hook event name and the skill arguments. Nothing can tell a typed prediction from an agent-called one. Nothing can tell an autopilot prediction either. No hook writes the record after the response.

Design

/home/jared/dev/cc-os/docs/takedown/1075-20261017-092011/final-design.json

Approach

InvocationLog keeps event and args in each entry, and a new PredictionRecorder (the single owner of origin/autopilot/idempotency) runs on Stop and calls OsMe::CLI.append in-process, then writes a Stop marker line into the same log.

Required work:

Write plugins/os-me/hooks/invocation_log.rb
Write plugins/os-me/hooks/prediction_recorder.rb
Write plugins/os-me/hooks/hooks.json
Write docs/implementation-status/os-me.md
Write plugins/os-me/hooks/record_prediction.rb
Write test plugins/os-me/hooks/test/invocation_log_test.rb
Write test plugins/os-me/hooks/test/prediction_recorder_test.rb

Acceptance criteria

Stop after a typed predict entry: CLI append origin typed, then Stop marker
Stop after a PreToolUse predict entry: origin agent
Stop with no entry for the session: nothing appended, no queue file created
Second Stop: last relevant line is the marker, nothing appended
Args contain the word autopilot: autopilot true
Append fails (Thor exits nonzero): SystemExit propagates, no marker, next Stop retries, error is shown
Entry with unknown event: KeyError, loud
Last entry is a respond skill: ignored

Owner sketch

Responsibility	Owner
Read and write queue/invocations.jsonl: event, args, entries, Stop marker	OsMe::InvocationLog
Decide origin (PreToolUse→agent, UserPromptExpansion→typed), autopilot (word in args), and whether an entry is unrecorded	OsMe::PredictionRecorder
Parse the transcript and write prediction records	existing `OsMe::CLI#append` (unchanged)
Call CLI#append in-process; a Thor failure exits nonzero	InProcessAppender in record_prediction.rb
Register the Stop event	hooks.json
Document the hook contract	docs/implementation-status/os-me.md

plugins/os-me/hooks/test/invocation_log_test.rb

# frozen_string_literal: true

require 'minitest/autorun'
require 'tmpdir'
require 'fileutils'
require 'json'
require_relative '../invocation_log'

class InvocationLogTest < Minitest::Test
  def setup
    @tmp = Dir.mktmpdir
    @store = File.join(@tmp, 'store')
    @queue = File.join(@store, 'queue', 'invocations.jsonl')
    @now = Time.utc(2026, 10, 4, 12, 0, 0)
  end

  def teardown
    FileUtils.remove_entry(@tmp)
  end

  def log
    OsMe::InvocationLog.new(store: @store, clock: -> { @now })
  end

  def lines
    File.readlines(@queue).map { |line| JSON.parse(line) }
  end

  def skill_call(name, args = nil)
    { 'hook_event_name' => 'PreToolUse', 'session_id' => 's1', 'transcript_path' => '/t/s1.jsonl',
      'tool_name' => 'Skill', 'tool_input' => { 'skill' => name, 'args' => args }.compact }
  end

  def typed_command(name, rest = nil)
    { 'hook_event_name' => 'UserPromptExpansion', 'session_id' => 's2', 'transcript_path' => '/t/s2.jsonl',
      'expansion_type' => 'slash_command', 'command_name' => name, 'prompt' => "/#{name} #{rest}".strip }
  end

  def test_logs_a_model_invoked_skill
    FileUtils.mkdir_p(@store)
    log.record(skill_call('os-me:predict'))

    assert_equal [{ 'session_id' => 's1', 'transcript_path' => '/t/s1.jsonl',
                    'skill' => 'os-me:predict', 'event' => 'PreToolUse', 'args' => nil,
                    'at' => '2026-10-04T12:00:00Z' }], lines
  end

  def test_pre_tool_use_entry_holds_event_and_args
    FileUtils.mkdir_p(@store)
    log.record(skill_call('os-me:predict', 'autopilot focus'))

    assert_equal 'PreToolUse', lines.first['event']
    assert_equal 'autopilot focus', lines.first['args']
  end

  def test_user_prompt_expansion_entry_holds_event_and_remainder
    FileUtils.mkdir_p(@store)
    log.record(typed_command('os-me:predict', 'focus now'))

    assert_equal 'UserPromptExpansion', lines.first['event']
    assert_equal 'focus now', lines.first['args']
  end

  def test_logs_a_typed_slash_command
    FileUtils.mkdir_p(@store)
    log.record(typed_command('os-me:respond'))

    assert_equal 'os-me:respond', lines.first['skill']
    assert_equal 's2', lines.first['session_id']
  end

  def test_appends_lines
    FileUtils.mkdir_p(@store)
    log.record(skill_call('os-me:predict'))
    log.record(skill_call('os-me:respond'))

    assert_equal(%w[os-me:predict os-me:respond], lines.map { |l| l['skill'] })
  end

  def test_ignores_other_skills
    FileUtils.mkdir_p(@store)
    log.record(skill_call('os-adr:find'))
    log.record(typed_command('os-adr:find'))

    refute File.exist?(@queue)
  end

  def test_does_nothing_when_store_is_absent
    log.record(skill_call('os-me:predict'))

    refute File.exist?(@store)
  end
end

plugins/os-me/hooks/test/prediction_recorder_test.rb

# frozen_string_literal: true

require 'minitest/autorun'
require 'tmpdir'
require 'fileutils'
require 'json'
require_relative '../invocation_log'

class PredictionRecorderTest < Minitest::Test
  def setup
    @tmp = Dir.mktmpdir
    @store = File.join(@tmp, 'store')
    FileUtils.mkdir_p(@store)
    @queue = File.join(@store, 'queue', 'invocations.jsonl')
    @calls = []
    @log = OsMe::InvocationLog.new(store: @store, clock: -> { Time.utc(2026, 10, 17, 9, 0, 0) })
  end

  def teardown
    FileUtils.remove_entry(@tmp)
  end

  def recorder
    require_relative '../prediction_recorder'
    OsMe::PredictionRecorder.new(log: @log, appender: ->(**call) { @calls << call })
  end

  def agent_call(args)
    { 'hook_event_name' => 'PreToolUse', 'session_id' => 's1', 'transcript_path' => '/t/s1.jsonl',
      'tool_name' => 'Skill', 'tool_input' => { 'skill' => 'os-me:predict', 'args' => args } }
  end

  def typed_call(rest)
    { 'hook_event_name' => 'UserPromptExpansion', 'session_id' => 's1', 'transcript_path' => '/t/s1.jsonl',
      'command_name' => 'os-me:predict', 'prompt' => "/os-me:predict #{rest}" }
  end

  def stop(session = 's1')
    { 'hook_event_name' => 'Stop', 'session_id' => session, 'transcript_path' => "/t/#{session}.jsonl" }
  end

  def test_stop_after_typed_entry_records_typed
    @log.record(typed_call(''))
    recorder.stop(stop)

    assert_equal [{ transcript_path: '/t/s1.jsonl', origin: 'typed', autopilot: false }], @calls
  end

  def test_stop_after_agent_entry_records_agent
    @log.record(agent_call(nil))
    recorder.stop(stop)

    assert_equal 'agent', @calls.first[:origin]
  end

  def test_stop_without_predict_entry_appends_nothing
    rec = recorder
    rec.stop(stop)
    rec.stop(stop('other'))

    assert_empty @calls
    refute File.exist?(@queue)
  end

  def test_second_stop_appends_nothing
    @log.record(agent_call(nil))
    rec = recorder
    rec.stop(stop)
    rec.stop(stop)

    assert_equal 1, @calls.size
  end

  def test_autopilot_word_sets_autopilot_true
    @log.record(typed_call('autopilot'))
    recorder.stop(stop)

    assert @calls.first[:autopilot]
  end

  def test_missing_autopilot_word_sets_false
    @log.record(agent_call('focus'))
    recorder.stop(stop)

    refute @calls.first[:autopilot]
  end

  def test_unknown_event_raises
    FileUtils.mkdir_p(File.dirname(@queue))
    File.write(@queue, "#{JSON.generate(session_id: 's1', skill: 'os-me:predict', event: 'Other')}\n")

    assert_raises(KeyError) { recorder.stop(stop) }
  end
end

## Problem hooks/invocation_log.rb drops the hook event name and the skill arguments. Nothing can tell a typed prediction from an agent-called one. Nothing can tell an autopilot prediction either. No hook writes the record after the response. ## Design /home/jared/dev/cc-os/docs/takedown/1075-20261017-092011/final-design.json ## Approach InvocationLog keeps event and args in each entry, and a new PredictionRecorder (the single owner of origin/autopilot/idempotency) runs on Stop and calls OsMe::CLI.append in-process, then writes a Stop marker line into the same log. Required work: 1. Write `plugins/os-me/hooks/invocation_log.rb` 2. Write `plugins/os-me/hooks/prediction_recorder.rb` 3. Write `plugins/os-me/hooks/hooks.json` 4. Write `docs/implementation-status/os-me.md` 5. Write `plugins/os-me/hooks/record_prediction.rb` 6. Write test `plugins/os-me/hooks/test/invocation_log_test.rb` 7. Write test `plugins/os-me/hooks/test/prediction_recorder_test.rb` ## Acceptance criteria - Stop after a typed predict entry: CLI append origin typed, then Stop marker - Stop after a PreToolUse predict entry: origin agent - Stop with no entry for the session: nothing appended, no queue file created - Second Stop: last relevant line is the marker, nothing appended - Args contain the word autopilot: autopilot true - Append fails (Thor exits nonzero): SystemExit propagates, no marker, next Stop retries, error is shown - Entry with unknown event: KeyError, loud - Last entry is a respond skill: ignored ## Owner sketch | Responsibility | Owner | |---|---| | Read and write queue/invocations.jsonl: event, args, entries, Stop marker | OsMe::InvocationLog | | Decide origin (PreToolUse→agent, UserPromptExpansion→typed), autopilot (word in args), and whether an entry is unrecorded | OsMe::PredictionRecorder | | Parse the transcript and write prediction records | existing `OsMe::CLI#append` (unchanged) | | Call CLI#append in-process; a Thor failure exits nonzero | InProcessAppender in record_prediction.rb | | Register the Stop event | hooks.json | | Document the hook contract | docs/implementation-status/os-me.md | `plugins/os-me/hooks/test/invocation_log_test.rb` ```ruby # frozen_string_literal: true require 'minitest/autorun' require 'tmpdir' require 'fileutils' require 'json' require_relative '../invocation_log' class InvocationLogTest < Minitest::Test def setup @tmp = Dir.mktmpdir @store = File.join(@tmp, 'store') @queue = File.join(@store, 'queue', 'invocations.jsonl') @now = Time.utc(2026, 10, 4, 12, 0, 0) end def teardown FileUtils.remove_entry(@tmp) end def log OsMe::InvocationLog.new(store: @store, clock: -> { @now }) end def lines File.readlines(@queue).map { |line| JSON.parse(line) } end def skill_call(name, args = nil) { 'hook_event_name' => 'PreToolUse', 'session_id' => 's1', 'transcript_path' => '/t/s1.jsonl', 'tool_name' => 'Skill', 'tool_input' => { 'skill' => name, 'args' => args }.compact } end def typed_command(name, rest = nil) { 'hook_event_name' => 'UserPromptExpansion', 'session_id' => 's2', 'transcript_path' => '/t/s2.jsonl', 'expansion_type' => 'slash_command', 'command_name' => name, 'prompt' => "/#{name} #{rest}".strip } end def test_logs_a_model_invoked_skill FileUtils.mkdir_p(@store) log.record(skill_call('os-me:predict')) assert_equal [{ 'session_id' => 's1', 'transcript_path' => '/t/s1.jsonl', 'skill' => 'os-me:predict', 'event' => 'PreToolUse', 'args' => nil, 'at' => '2026-10-04T12:00:00Z' }], lines end def test_pre_tool_use_entry_holds_event_and_args FileUtils.mkdir_p(@store) log.record(skill_call('os-me:predict', 'autopilot focus')) assert_equal 'PreToolUse', lines.first['event'] assert_equal 'autopilot focus', lines.first['args'] end def test_user_prompt_expansion_entry_holds_event_and_remainder FileUtils.mkdir_p(@store) log.record(typed_command('os-me:predict', 'focus now')) assert_equal 'UserPromptExpansion', lines.first['event'] assert_equal 'focus now', lines.first['args'] end def test_logs_a_typed_slash_command FileUtils.mkdir_p(@store) log.record(typed_command('os-me:respond')) assert_equal 'os-me:respond', lines.first['skill'] assert_equal 's2', lines.first['session_id'] end def test_appends_lines FileUtils.mkdir_p(@store) log.record(skill_call('os-me:predict')) log.record(skill_call('os-me:respond')) assert_equal(%w[os-me:predict os-me:respond], lines.map { |l| l['skill'] }) end def test_ignores_other_skills FileUtils.mkdir_p(@store) log.record(skill_call('os-adr:find')) log.record(typed_command('os-adr:find')) refute File.exist?(@queue) end def test_does_nothing_when_store_is_absent log.record(skill_call('os-me:predict')) refute File.exist?(@store) end end ``` `plugins/os-me/hooks/test/prediction_recorder_test.rb` ```ruby # frozen_string_literal: true require 'minitest/autorun' require 'tmpdir' require 'fileutils' require 'json' require_relative '../invocation_log' class PredictionRecorderTest < Minitest::Test def setup @tmp = Dir.mktmpdir @store = File.join(@tmp, 'store') FileUtils.mkdir_p(@store) @queue = File.join(@store, 'queue', 'invocations.jsonl') @calls = [] @log = OsMe::InvocationLog.new(store: @store, clock: -> { Time.utc(2026, 10, 17, 9, 0, 0) }) end def teardown FileUtils.remove_entry(@tmp) end def recorder require_relative '../prediction_recorder' OsMe::PredictionRecorder.new(log: @log, appender: ->(**call) { @calls << call }) end def agent_call(args) { 'hook_event_name' => 'PreToolUse', 'session_id' => 's1', 'transcript_path' => '/t/s1.jsonl', 'tool_name' => 'Skill', 'tool_input' => { 'skill' => 'os-me:predict', 'args' => args } } end def typed_call(rest) { 'hook_event_name' => 'UserPromptExpansion', 'session_id' => 's1', 'transcript_path' => '/t/s1.jsonl', 'command_name' => 'os-me:predict', 'prompt' => "/os-me:predict #{rest}" } end def stop(session = 's1') { 'hook_event_name' => 'Stop', 'session_id' => session, 'transcript_path' => "/t/#{session}.jsonl" } end def test_stop_after_typed_entry_records_typed @log.record(typed_call('')) recorder.stop(stop) assert_equal [{ transcript_path: '/t/s1.jsonl', origin: 'typed', autopilot: false }], @calls end def test_stop_after_agent_entry_records_agent @log.record(agent_call(nil)) recorder.stop(stop) assert_equal 'agent', @calls.first[:origin] end def test_stop_without_predict_entry_appends_nothing rec = recorder rec.stop(stop) rec.stop(stop('other')) assert_empty @calls refute File.exist?(@queue) end def test_second_stop_appends_nothing @log.record(agent_call(nil)) rec = recorder rec.stop(stop) rec.stop(stop) assert_equal 1, @calls.size end def test_autopilot_word_sets_autopilot_true @log.record(typed_call('autopilot')) recorder.stop(stop) assert @calls.first[:autopilot] end def test_missing_autopilot_word_sets_false @log.record(agent_call('focus')) recorder.stop(stop) refute @calls.first[:autopilot] end def test_unknown_event_raises FileUtils.mkdir_p(File.dirname(@queue)) File.write(@queue, "#{JSON.generate(session_id: 's1', skill: 'os-me:predict', event: 'Other')}\n") assert_raises(KeyError) { recorder.stop(stop) } end end ```

jared added the

project/cc-os

filed-by/user

category/code-change

labels

2026-10-17 11:54:35 +00:00

jared referenced this issue

2026-10-17 11:54:38 +00:00

os-me: leaf History row for the record log and a pointer to bin/os-me-report #1080

~~jared referenced this issue 2026-10-17 12:31:36 +00:00~~

os-me: prediction records carry triggers, verdict, origin, and autopilot #1081

jared commented

2026-10-17 12:31:37 +00:00

Author

Owner

Also blocked by #1081: records must carry origin/autopilot (1075) and triggers/verdict (1077) first.

jared referenced this issue

2026-10-17 12:34:49 +00:00

os-me: bin/os-me-record appends and scores prediction records #1074

jared commented

2026-10-17 13:27:42 +00:00

Author

Owner

Work started on branch ticket-1075 (os-sdlc takedown-implement).

jared commented

2026-10-17 13:35:06 +00:00

Author

Owner

Resolution

Done: InvocationLog keeps event and args per entry; PredictionRecorder on Stop records each pending os-me:predict with origin (typed/agent) and autopilot through OsMe::CliAppender, then writes a Stop marker; a failed append writes no marker so the next Stop retries.

Evidence: Hand-finished after escalation 28 (implementation 55): b8460ae9 on ticket-1075. os-me suite 76 runs green (now includes hooks/test), rubocop clean, Stop-hook smoke test (loads, fails loudly with no marker; no-pending Stop exits 0 without loading the CLI). Opus design audit: 2 defects + 3 design findings + 2 nits, all corrected. Two pre-change Store sessions closed with Stop markers (os-me-store commit). Related placer defect fixed in 12feb876.

Follow-ups: none

## Resolution **Done:** InvocationLog keeps event and args per entry; PredictionRecorder on Stop records each pending os-me:predict with origin (typed/agent) and autopilot through OsMe::CliAppender, then writes a Stop marker; a failed append writes no marker so the next Stop retries. **Evidence:** Hand-finished after escalation 28 (implementation 55): b8460ae9 on ticket-1075. os-me suite 76 runs green (now includes hooks/test), rubocop clean, Stop-hook smoke test (loads, fails loudly with no marker; no-pending Stop exits 0 without loading the CLI). Opus design audit: 2 defects + 3 design findings + 2 nits, all corrected. Two pre-change Store sessions closed with Stop markers (os-me-store commit). Related placer defect fixed in 12feb876. **Follow-ups:** none

jared closed this issue

2026-10-17 13:35:06 +00:00

jared referenced this issue

2026-10-17 14:14:15 +00:00

os-me: leaf History row for the record log and a pointer to bin/os-me-report #1080

jared referenced this issue

2026-10-17 14:16:47 +00:00

os-me Store: learning loop scores through bin/os-me-record and learns only from discussed sessions #1078

jared referenced this issue from a commit

2026-10-17 14:47:02 +00:00

docs(takedown): record run 1075-20261017-092011 (final design for #1075)

jared referenced this issue from a commit