jared/cc-os

Watch

Fork 0

decision-system S6: SessionStart extraction hook #1248

New issue

Closed

opened 2026-10-23 15:17:27 +00:00 by jared · 3 comments

jared commented

2026-10-23 15:17:27 +00:00

Owner

Problem

Extraction runs only when Jared types os-me-extract. Moments from new sessions do not reach the Store until he does. MomentExtractor#run reads every unread session in one run and has no limit on model calls, so a hook cannot call it as it is. Each new memo costs one claude subprocess (ReplyShape#classify through OsMe::LabelerClient.for(OsMe::HeadlessClaude)), and the first run sees about 650 unread sessions.

Approach

A SessionStart hook starts a detached os-me-extract --budget 20 child and exits 0 at once. It prints nothing. The hook follows the two existing background patterns in this repository: OsSdlc::Db::MirrorTickle.spawn_sync (Process.detach(Process.spawn(...))) and the os-vault SessionStart rebuild (start_new_session=True). No hooks.json in this repository uses an async field, so the hook script itself does the detach.

Budget unit: model calls per run. One new memo moment is one ReplyShape#classify call, which is one claude subprocess, so the extractor counts the moments it writes. The extractor checks the budget between sessions. It starts no new session after it has made budget calls. A session that has started always finishes, because MomentLog#append writes the moments of one session together and ExtractCursor#advance moves after that write. A run can thus go past the budget by the memos of its last session. The next run continues at the first unread session.

Two sessions can start at almost the same time. A second run then exits 0 at once when another run holds the lock file queue/extract.lock in the Store. This stops double model calls on the same unread sessions.

Held-out routing does not change: Split#held_out? sends held-out moments to eval/moments-held-out.jsonl and never to moments.jsonl. os-me-extract with no --budget keeps its current behavior.

Required work:

MomentExtractor#run(budget: nil) counts model calls and starts no new session after budget calls — check: present budget in plugins/os-me/lib/os_me/moment_extractor.rb
MomentExtractor#run holds a non-blocking flock on queue/extract.lock in the Store and returns without work when another run holds it — check: present extract.lock in plugins/os-me/lib/os_me/moment_extractor.rb
CLI extract takes --budget N (numeric, optional) and passes it to run — check: present option :budget in plugins/os-me/lib/os_me/cli.rb
OsMe::ExtractLauncher (hook object, same place as OsMe::InvocationLog) spawns the detached extract child only when the Store directory exists — check: present class ExtractLauncher in plugins/os-me/hooks/extract_launcher.rb
Entry script extract_moments.rb builds the launcher with the Store ~/dev/os-me-store, rescues StandardError, and exits 0, like log_invocation.rb — check: present exit 0 in plugins/os-me/hooks/extract_moments.rb
hooks.json gains a SessionStart entry with matcher startup, command ruby ${CLAUDE_PLUGIN_ROOT}/hooks/extract_moments.rb, and timeout 5 — check: present hooks/extract_moments.rb in plugins/os-me/hooks/hooks.json
Budget, lock, and held-out tests through the CLI with a fake claude on PATH, as extract_cli_test.rb does — check: present --budget in plugins/os-me/test/extract_budget_test.rb
Launcher and entry-script tests — check: present ExtractLauncher in plugins/os-me/test/extract_launcher_test.rb

Orchestrator work: no ADR. O-10 was challenged and rejected on 2026-10-23 (docs/decision-system/afk-adr-log.md). Run bin/refresh-plugins after the hook lands. Gate: start two real Claude Code sessions with startup; moments.jsonl or eval/moments-held-out.jsonl in the Store gains rows, and session start time grows by no more than 1 s. If the start grows by more than 1 s, make BUDGET smaller; do not change the hook contract. Tests never call the real claude.

Acceptance criteria

Given a Store whose queue/extract-cursor.json holds the current mtime of every transcript under --transcript-root, when os-me-extract --budget 20 runs, then the fake claude gets 0 calls, moments.jsonl does not change, stdout is empty, and the exit code is 0.
Given an empty cursor and 3 train sessions that each hold one memo and one typed reply, when os-me-extract --budget 2 runs, then the fake claude gets 2 calls, moments.jsonl holds 2 rows, and queue/extract-cursor.json holds 2 session ids.
Given the Store that criterion 2 left, when os-me-extract --budget 2 runs again, then the fake claude gets 1 more call (3 in total), moments.jsonl holds 3 rows, and the cursor holds 3 session ids.
Given an empty cursor and 2 train sessions that each hold two memos with replies, when os-me-extract --budget 1 runs, then the fake claude gets 2 calls, moments.jsonl holds 2 rows from one session, and the cursor holds 1 session id.
Given an empty cursor, 1 held-out session, and 1 train session that each hold one memo and reply, when os-me-extract --budget 20 runs, then eval/moments-held-out.jsonl holds 1 row with the held-out session id and moments.jsonl holds 1 row with "split":"train" and no held-out session id.
Given 1 unread train session with a memo and another process that holds flock(LOCK_EX) on <store>/queue/extract.lock, when os-me-extract --budget 20 runs, then the fake claude gets 0 calls, stdout is empty, the exit code is 0, and the cursor does not change.
Given a Store path that does not exist, when OsMe::ExtractLauncher.new(store: path, spawner: recorder).launch runs, then the recorder gets 0 calls; and when ruby plugins/os-me/hooks/extract_moments.rb runs with HOME set to an empty directory and stdin {"hook_event_name":"SessionStart","source":"startup"}, then stdout and stderr are empty and the exit code is 0.
Given HOME with dev/os-me-store and one unread train session with a memo under .claude/projects, and a fake claude that sleeps 2 s before it answers, when ruby plugins/os-me/hooks/extract_moments.rb runs with the stdin of criterion 7, then it exits 0 in less than 1 s with empty stdout, and dev/os-me-store/moments.jsonl holds 1 row within 15 s.
Given plugins/os-me/hooks/hooks.json, when it is parsed, then hooks.SessionStart holds one entry with "matcher":"startup" whose one hook has "type":"command", "command":"ruby ${CLAUDE_PLUGIN_ROOT}/hooks/extract_moments.rb", and "timeout":5.

Owner sketch

module OsMe
  class MomentExtractor
    LOCK = File.join('queue', 'extract.lock')

    def run(budget: nil)
      File.open(@lock_path, File::CREAT | File::RDWR) do |lock|
        return unless lock.flock(File::LOCK_EX | File::LOCK_NB)

        calls = 0
        @cursor.unread(paths).each do |path|
          break if budget && calls >= budget

          calls += extract_and_advance(path)
        end
      end
    end
  end

  class ExtractLauncher
    BUDGET = 20
    EXTRACT_BIN = File.expand_path('../bin/os-me-extract', __dir__)
    DETACHED = lambda do |argv|
      Process.detach(Process.spawn(*argv, in: File::NULL, out: File::NULL, err: File::NULL, pgroup: true))
    end

    def initialize(store:, spawner: DETACHED)
      @store = store
      @spawner = spawner
    end

    def launch
      return unless File.directory?(@store)

      @spawner.call([RbConfig.ruby, EXTRACT_BIN, '--store', @store, '--budget', BUDGET.to_s])
    end
  end
end

extract_and_advance returns the number of moments that it wrote (0 when the transcript vanished), and extract returns batch.size.

Non-goals

New moment kinds (S7).
An async field in hooks.json.
A daemon or a run on resume, clear, or compact.
A change to Split, ExtractCursor, or the record format of DecisionMoment.

Origin

Trigger: User asked for an implementation-ready rewrite of the S6 ticket, because loose acceptance criteria made an earlier stage build an off-design shape
Improvised this session: none
Chain: #1248 acceptance criteria named no exact inputs or outputs ← the S6 row of the build plan names the hook but not the budget unit or the detach mechanism. Stop at DESIGN (docs/decision-system/build-plan.md S6).
Root candidate: none found; searched extract_moments, SessionStart hook, async, Process.detach, extract cursor, budget
Where: docs/decision-system/build-plan.md S6; plugins/os-me/lib/os_me/moment_extractor.rb; plugins/os-me/lib/os_me/cli.rb; plugins/os-me/hooks/hooks.json; plugins/os-me/hooks/extract_moments.rb
Session: 4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-ticket-1250/4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5.jsonl

VERDICT: CREATE (user-approved; backlog search only)

## Problem Extraction runs only when Jared types `os-me-extract`. Moments from new sessions do not reach the Store until he does. `MomentExtractor#run` reads every unread session in one run and has no limit on model calls, so a hook cannot call it as it is. Each new memo costs one `claude` subprocess (`ReplyShape#classify` through `OsMe::LabelerClient.for(OsMe::HeadlessClaude)`), and the first run sees about 650 unread sessions. ## Approach A SessionStart hook starts a detached `os-me-extract --budget 20` child and exits 0 at once. It prints nothing. The hook follows the two existing background patterns in this repository: `OsSdlc::Db::MirrorTickle.spawn_sync` (`Process.detach(Process.spawn(...))`) and the os-vault SessionStart rebuild (`start_new_session=True`). No `hooks.json` in this repository uses an `async` field, so the hook script itself does the detach. Budget unit: model calls per run. One new memo moment is one `ReplyShape#classify` call, which is one `claude` subprocess, so the extractor counts the moments it writes. The extractor checks the budget between sessions. It starts no new session after it has made `budget` calls. A session that has started always finishes, because `MomentLog#append` writes the moments of one session together and `ExtractCursor#advance` moves after that write. A run can thus go past the budget by the memos of its last session. The next run continues at the first unread session. Two sessions can start at almost the same time. A second run then exits 0 at once when another run holds the lock file `queue/extract.lock` in the Store. This stops double model calls on the same unread sessions. Held-out routing does not change: `Split#held_out?` sends held-out moments to `eval/moments-held-out.jsonl` and never to `moments.jsonl`. `os-me-extract` with no `--budget` keeps its current behavior. Required work: 1. `MomentExtractor#run(budget: nil)` counts model calls and starts no new session after `budget` calls — check: present `budget` in `plugins/os-me/lib/os_me/moment_extractor.rb` 2. `MomentExtractor#run` holds a non-blocking `flock` on `queue/extract.lock` in the Store and returns without work when another run holds it — check: present `extract.lock` in `plugins/os-me/lib/os_me/moment_extractor.rb` 3. CLI `extract` takes `--budget N` (numeric, optional) and passes it to `run` — check: present `option :budget` in `plugins/os-me/lib/os_me/cli.rb` 4. `OsMe::ExtractLauncher` (hook object, same place as `OsMe::InvocationLog`) spawns the detached extract child only when the Store directory exists — check: present `class ExtractLauncher` in `plugins/os-me/hooks/extract_launcher.rb` 5. Entry script `extract_moments.rb` builds the launcher with the Store `~/dev/os-me-store`, rescues `StandardError`, and exits 0, like `log_invocation.rb` — check: present `exit 0` in `plugins/os-me/hooks/extract_moments.rb` 6. `hooks.json` gains a `SessionStart` entry with matcher `startup`, command `ruby ${CLAUDE_PLUGIN_ROOT}/hooks/extract_moments.rb`, and timeout 5 — check: present `hooks/extract_moments.rb` in `plugins/os-me/hooks/hooks.json` 7. Budget, lock, and held-out tests through the CLI with a fake `claude` on PATH, as `extract_cli_test.rb` does — check: present `--budget` in `plugins/os-me/test/extract_budget_test.rb` 8. Launcher and entry-script tests — check: present `ExtractLauncher` in `plugins/os-me/test/extract_launcher_test.rb` Orchestrator work: no ADR. O-10 was challenged and rejected on 2026-10-23 (`docs/decision-system/afk-adr-log.md`). Run `bin/refresh-plugins` after the hook lands. Gate: start two real Claude Code sessions with `startup`; `moments.jsonl` or `eval/moments-held-out.jsonl` in the Store gains rows, and session start time grows by no more than 1 s. If the start grows by more than 1 s, make `BUDGET` smaller; do not change the hook contract. Tests never call the real `claude`. ## Acceptance criteria 1. Given a Store whose `queue/extract-cursor.json` holds the current mtime of every transcript under `--transcript-root`, when `os-me-extract --budget 20` runs, then the fake `claude` gets 0 calls, `moments.jsonl` does not change, stdout is empty, and the exit code is 0. 2. Given an empty cursor and 3 train sessions that each hold one memo and one typed reply, when `os-me-extract --budget 2` runs, then the fake `claude` gets 2 calls, `moments.jsonl` holds 2 rows, and `queue/extract-cursor.json` holds 2 session ids. 3. Given the Store that criterion 2 left, when `os-me-extract --budget 2` runs again, then the fake `claude` gets 1 more call (3 in total), `moments.jsonl` holds 3 rows, and the cursor holds 3 session ids. 4. Given an empty cursor and 2 train sessions that each hold two memos with replies, when `os-me-extract --budget 1` runs, then the fake `claude` gets 2 calls, `moments.jsonl` holds 2 rows from one session, and the cursor holds 1 session id. 5. Given an empty cursor, 1 held-out session, and 1 train session that each hold one memo and reply, when `os-me-extract --budget 20` runs, then `eval/moments-held-out.jsonl` holds 1 row with the held-out session id and `moments.jsonl` holds 1 row with `"split":"train"` and no held-out session id. 6. Given 1 unread train session with a memo and another process that holds `flock(LOCK_EX)` on `<store>/queue/extract.lock`, when `os-me-extract --budget 20` runs, then the fake `claude` gets 0 calls, stdout is empty, the exit code is 0, and the cursor does not change. 7. Given a Store path that does not exist, when `OsMe::ExtractLauncher.new(store: path, spawner: recorder).launch` runs, then the recorder gets 0 calls; and when `ruby plugins/os-me/hooks/extract_moments.rb` runs with `HOME` set to an empty directory and stdin `{"hook_event_name":"SessionStart","source":"startup"}`, then stdout and stderr are empty and the exit code is 0. 8. Given `HOME` with `dev/os-me-store` and one unread train session with a memo under `.claude/projects`, and a fake `claude` that sleeps 2 s before it answers, when `ruby plugins/os-me/hooks/extract_moments.rb` runs with the stdin of criterion 7, then it exits 0 in less than 1 s with empty stdout, and `dev/os-me-store/moments.jsonl` holds 1 row within 15 s. 9. Given `plugins/os-me/hooks/hooks.json`, when it is parsed, then `hooks.SessionStart` holds one entry with `"matcher":"startup"` whose one hook has `"type":"command"`, `"command":"ruby ${CLAUDE_PLUGIN_ROOT}/hooks/extract_moments.rb"`, and `"timeout":5`. ## Owner sketch ```ruby module OsMe class MomentExtractor LOCK = File.join('queue', 'extract.lock') def run(budget: nil) File.open(@lock_path, File::CREAT | File::RDWR) do |lock| return unless lock.flock(File::LOCK_EX | File::LOCK_NB) calls = 0 @cursor.unread(paths).each do |path| break if budget && calls >= budget calls += extract_and_advance(path) end end end end class ExtractLauncher BUDGET = 20 EXTRACT_BIN = File.expand_path('../bin/os-me-extract', __dir__) DETACHED = lambda do |argv| Process.detach(Process.spawn(*argv, in: File::NULL, out: File::NULL, err: File::NULL, pgroup: true)) end def initialize(store:, spawner: DETACHED) @store = store @spawner = spawner end def launch return unless File.directory?(@store) @spawner.call([RbConfig.ruby, EXTRACT_BIN, '--store', @store, '--budget', BUDGET.to_s]) end end end ``` `extract_and_advance` returns the number of moments that it wrote (0 when the transcript vanished), and `extract` returns `batch.size`. ## Non-goals - New moment kinds (S7). - An `async` field in `hooks.json`. - A daemon or a run on `resume`, `clear`, or `compact`. - A change to `Split`, `ExtractCursor`, or the record format of `DecisionMoment`. ## Origin - Trigger: User asked for an implementation-ready rewrite of the S6 ticket, because loose acceptance criteria made an earlier stage build an off-design shape - Improvised this session: none - Chain: #1248 acceptance criteria named no exact inputs or outputs ← the S6 row of the build plan names the hook but not the budget unit or the detach mechanism. Stop at `DESIGN` (`docs/decision-system/build-plan.md` S6). - Root candidate: none found; searched extract_moments, SessionStart hook, async, Process.detach, extract cursor, budget - Where: docs/decision-system/build-plan.md S6; plugins/os-me/lib/os_me/moment_extractor.rb; plugins/os-me/lib/os_me/cli.rb; plugins/os-me/hooks/hooks.json; plugins/os-me/hooks/extract_moments.rb - Session: 4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-ticket-1250/4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5.jsonl VERDICT: CREATE (user-approved; backlog search only)

jared added the

project/cc-os

filed-by/user

category/code-change

labels

2026-10-23 15:17:27 +00:00

jared referenced this issue from a commit

2026-10-23 18:41:05 +00:00

docs(decision-system): O-10 ADR challenge rejected (#1248)

jared commented

2026-10-23 18:43:58 +00:00

Author

Owner

Work started on branch ticket-1248 (os-sdlc implement).

jared commented

2026-10-23 18:48:51 +00:00

Author

Owner

Run 135 failed: diff-test does not map plugins/os-me/hooks/test (the os-me target has one test_path), so the launcher test had no red run and the test-reviewer returned error. Moved the launcher test path to plugins/os-me/test/extract_launcher_test.rb and reopened on ticket-1248.

jared commented

2026-10-23 18:57:44 +00:00

Author

Owner

Resolution

Done: Budgeted, locked os-me-extract run (--budget N, flock on queue/extract.lock); OsMe::ExtractLauncher and SessionStart hook extract_moments.rb (matcher startup, detached child, budget 20, prints nothing, exits 0)

Evidence: implementation 136 merge-ready, 8 dispatches, 0 retries; suite and lint gates pass; criterion 8 test shows the hook exits under 1 s with a 2 s fake claude; commit on ticket-1248 (9 files, +299/-7)

Follow-ups: S6 gate (two real SessionStarts add moments, start time grows under 1 s) is measured live and recorded in build-plan.md; os-sdlc single test_path defect filed after merge

## Resolution **Done:** Budgeted, locked os-me-extract run (--budget N, flock on queue/extract.lock); OsMe::ExtractLauncher and SessionStart hook extract_moments.rb (matcher startup, detached child, budget 20, prints nothing, exits 0) **Evidence:** implementation 136 merge-ready, 8 dispatches, 0 retries; suite and lint gates pass; criterion 8 test shows the hook exits under 1 s with a 2 s fake claude; commit on ticket-1248 (9 files, +299/-7) **Follow-ups:** S6 gate (two real SessionStarts add moments, start time grows under 1 s) is measured live and recorded in build-plan.md; os-sdlc single test_path defect filed after merge

jared closed this issue

2026-10-23 18:57:45 +00:00

jared referenced this issue from a commit

2026-10-23 18:58:51 +00:00

feat(os-me): Audited POODR boundaries for the budgeted, locked extract run and the SessionStart launcher; design and contracts hold, all eight WORK items present. (#1248)

jared referenced this issue from a commit

2026-10-23 18:58:58 +00:00

docs(decision-system): S6 built, gate open (#1248)

jared referenced this issue

2026-10-23 18:59:38 +00:00

Map tests under a second test folder of an os-sdlc target (os-me hooks/test) #1276

jared referenced this issue