jared/cc-os

Watch

Fork 0

decision-system S9: demotion and the One Thing question #1251

New issue

Closed

opened 2026-10-23 15:17:29 +00:00 by jared · 2 comments

jared commented

2026-10-23 15:17:29 +00:00

Owner

Problem

Confirmed patterns must become lessons so agents need Jared less (D-6, #1240). Today nothing groups the train moments in the Store moments.jsonl into patterns, and nothing finds the patterns that are ready to become a One Thing question. S9b asks the question; it needs a list of candidates first.

Approach

Required work:

Add OsMe::OllamaEmbedder: #embed(texts) posts {"model" => MODEL, "input" => texts} to api/embed through a connection that answers post_json(path, body) and returns the embeddings array. MODEL is nomic-embed-text. .standard builds it on Decision::Model::Http::Connection at http://localhost:11434. No vector is written to disk; each run embeds again. check: present class OllamaEmbedder in plugins/os-me/lib/**/*.rb
Add OsMe::PatternClusterer: takes an embedder (any object with #embed(texts)), embeds the headline of each ruling moment in one call, and groups moments in one greedy pass by id order: a moment joins the first cluster whose seed vector has cosine similarity at or above SIMILARITY (0.85), else it starts a new cluster. A ruling moment is memo_reply with outcome accept, modify, or override, or kind question_answer, correction, or direction. check: present class PatternClusterer in plugins/os-me/lib/**/*.rb
Add OsMe::PatternCluster: holds the moment rows; #sessions (distinct session ids), #agreement (mean of CalibrationReport::HIT over the memo_reply outcomes; each question_answer, correction, or direction moment counts 1.0, because it is Jared's own ruling), #moment_ids, #headline (the headline of the seed moment). check: present class PatternCluster in plugins/os-me/lib/**/*.rb
Add OsMe::LessonCandidate: MIN_SESSIONS = 3, MIN_AGREEMENT = 0.8, #eligible?, #id (first 12 hex digits of the SHA-256 of the sorted moment ids joined by newlines), #question, and #to_h for the log row. check: present class LessonCandidate in plugins/os-me/lib/**/*.rb
Add OsMe::CandidateLog over the Store file queue/lesson-candidates.jsonl and OsMe::CandidateRefresh: #run reads only MomentExtractor::TRAIN_LOG, clusters, and appends one row {id, question, moment_ids, sessions, agreement, moments, status: "open", created_at} per eligible cluster that shares no moment id with a row already in the log. check: present class CandidateRefresh in plugins/os-me/lib/**/*.rb
Add the os-me-eval candidates command: it runs CandidateRefresh on --store with OllamaEmbedder.standard and prints the number of new rows. check: present desc 'candidates' in plugins/os-me/lib/os_me/cli.rb

Notes: The rule sentence is the seed headline, not a model draft: the decision-model gem has no free-text question type (types are choice, noul, score), and a deterministic sentence costs no model call. MomentLog stays unchanged (O-2 is not touched). Tests use a fake embedder that returns fixed vectors and a fake connection; no test calls Ollama. Fail loudly: an embedder error ends the run before any row is written. AFK defaults used: O-8 (no Ollama model pull without approval; the nomic-embed-text model is not on this machine, so the live command fails until Jared approves the pull).

Acceptance criteria

Given three train moments whose fake headline vectors are [1, 0], [0.99, 0.1], and [0, 1], when PatternClusterer#cluster runs, then one PatternCluster holds the first two moment ids and a second cluster holds the third.
Given moments of kind none and over_escalation and a memo_reply moment with outcome none, when PatternClusterer#cluster runs, then no cluster holds them and the fake embedder receives none of their headlines.
Given a cluster of five memo_reply moments from 3 sessions with outcomes accept, accept, modify, accept, accept, when a LessonCandidate is built from it, then agreement is 0.9 and eligible? is true.
Given a cluster of four accept moments from 2 sessions, when a LessonCandidate is built from it, then eligible? is false.
Given a cluster from 3 sessions with outcomes accept, accept, override, accept, when a LessonCandidate is built from it, then agreement is 0.75 and eligible? is false.
Given a cluster of one direction, one correction, and one question_answer moment from 3 sessions, when a LessonCandidate is built from it, then agreement is 1.0 and eligible? is true.
Given an eligible cluster from 4 sessions whose seed headline is Decision: cache the index, when LessonCandidate#question is read, then it returns In 4 sessions you ruled on "Decision: cache the index". Is this a rule? (y/n).
Given a Store whose moments.jsonl holds one eligible cluster and a fake embedder, when CandidateRefresh#run runs twice, then queue/lesson-candidates.jsonl holds exactly one row, with status open, the question, and the cluster's moment ids.
Given a fake embedder that raises, when CandidateRefresh#run runs, then the error reaches the caller and queue/lesson-candidates.jsonl does not exist.
Given a fake connection that answers {"embeddings" => [[0.1, 0.2], [0.3, 0.4]]}, when OllamaEmbedder#embed(["a", "b"]) runs, then the connection receives path api/embed and body {"model" => "nomic-embed-text", "input" => ["a", "b"]}, and the result is [[0.1, 0.2], [0.3, 0.4]].

Owner sketch

module OsMe
  class OllamaEmbedder
    MODEL = 'nomic-embed-text'

    def self.standard; end
    def initialize(connection:, model: MODEL); end
    def embed(texts); end
  end

  class PatternClusterer
    SIMILARITY = 0.85
    RULING_KINDS = %w[question_answer correction direction].freeze

    def initialize(embedder:); end
    def cluster(moments); end
  end

  class PatternCluster
    def initialize(moments); end
    def sessions; end
    def agreement; end
    def moment_ids; end
    def headline; end
  end

  class LessonCandidate
    MIN_SESSIONS = 3
    MIN_AGREEMENT = 0.8

    def initialize(cluster); end
    def eligible?; end
    def id; end
    def question; end
    def to_h; end
  end

  class CandidateLog
    FILE_NAME = 'queue/lesson-candidates.jsonl'

    def initialize(path); end
    def records; end
    def append(candidates); end
  end

  class CandidateRefresh
    def initialize(store_path:, embedder:); end
    def run; end
  end
end

Scope and blocking

Blocked by: #1245
Blocked by: #1249
Design is in docs/decision-system/learning-loops.md (loop 2: demotion). Related: #1240. S9b (the One Thing question, the session ledger, and the verdict readback) is a separate ticket blocked by this one.

Non-goals: the One Thing question and the SessionStart hook (S9b); a model-drafted rule sentence; the calibration factor in demotion_value (S9b orders by moment count); tuning SIMILARITY (S10); a stored vector index; writing a confirmed lesson to its destination.

Origin

Trigger: User asked for the decision-system build tickets this session
Improvised this session: none
Chain: no ticket for this stage ← the build plan names the stage and the user asked for the tickets. Stop at DESIGN (docs/decision-system/build-plan.md).
Root candidate: none found; related #1240 (design ticket, different mechanism)
Where: docs/decision-system/build-plan.md S9; plugins/os-me/lib/os_me/; plugins/os-me/hooks/
Session: 4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5.jsonl

VERDICT: CREATE (user-approved; backlog search only)

## Problem Confirmed patterns must become lessons so agents need Jared less (D-6, #1240). Today nothing groups the train moments in the Store `moments.jsonl` into patterns, and nothing finds the patterns that are ready to become a One Thing question. S9b asks the question; it needs a list of candidates first. ## Approach Required work: 1. Add `OsMe::OllamaEmbedder`: `#embed(texts)` posts `{"model" => MODEL, "input" => texts}` to `api/embed` through a connection that answers `post_json(path, body)` and returns the `embeddings` array. `MODEL` is `nomic-embed-text`. `.standard` builds it on `Decision::Model::Http::Connection` at `http://localhost:11434`. No vector is written to disk; each run embeds again. check: present `class OllamaEmbedder` in `plugins/os-me/lib/**/*.rb` 2. Add `OsMe::PatternClusterer`: takes an embedder (any object with `#embed(texts)`), embeds the `headline` of each ruling moment in one call, and groups moments in one greedy pass by id order: a moment joins the first cluster whose seed vector has cosine similarity at or above `SIMILARITY` (0.85), else it starts a new cluster. A ruling moment is `memo_reply` with outcome `accept`, `modify`, or `override`, or kind `question_answer`, `correction`, or `direction`. check: present `class PatternClusterer` in `plugins/os-me/lib/**/*.rb` 3. Add `OsMe::PatternCluster`: holds the moment rows; `#sessions` (distinct session ids), `#agreement` (mean of `CalibrationReport::HIT` over the `memo_reply` outcomes; each `question_answer`, `correction`, or `direction` moment counts 1.0, because it is Jared's own ruling), `#moment_ids`, `#headline` (the headline of the seed moment). check: present `class PatternCluster` in `plugins/os-me/lib/**/*.rb` 4. Add `OsMe::LessonCandidate`: `MIN_SESSIONS = 3`, `MIN_AGREEMENT = 0.8`, `#eligible?`, `#id` (first 12 hex digits of the SHA-256 of the sorted moment ids joined by newlines), `#question`, and `#to_h` for the log row. check: present `class LessonCandidate` in `plugins/os-me/lib/**/*.rb` 5. Add `OsMe::CandidateLog` over the Store file `queue/lesson-candidates.jsonl` and `OsMe::CandidateRefresh`: `#run` reads only `MomentExtractor::TRAIN_LOG`, clusters, and appends one row `{id, question, moment_ids, sessions, agreement, moments, status: "open", created_at}` per eligible cluster that shares no moment id with a row already in the log. check: present `class CandidateRefresh` in `plugins/os-me/lib/**/*.rb` 6. Add the `os-me-eval candidates` command: it runs `CandidateRefresh` on `--store` with `OllamaEmbedder.standard` and prints the number of new rows. check: present `desc 'candidates'` in `plugins/os-me/lib/os_me/cli.rb` Notes: The rule sentence is the seed headline, not a model draft: the `decision-model` gem has no free-text question type (types are `choice`, `noul`, `score`), and a deterministic sentence costs no model call. `MomentLog` stays unchanged (O-2 is not touched). Tests use a fake embedder that returns fixed vectors and a fake connection; no test calls Ollama. Fail loudly: an embedder error ends the run before any row is written. AFK defaults used: O-8 (no Ollama model pull without approval; the `nomic-embed-text` model is not on this machine, so the live command fails until Jared approves the pull). ## Acceptance criteria 1. Given three train moments whose fake headline vectors are `[1, 0]`, `[0.99, 0.1]`, and `[0, 1]`, when `PatternClusterer#cluster` runs, then one `PatternCluster` holds the first two moment ids and a second cluster holds the third. 2. Given moments of kind `none` and `over_escalation` and a `memo_reply` moment with outcome `none`, when `PatternClusterer#cluster` runs, then no cluster holds them and the fake embedder receives none of their headlines. 3. Given a cluster of five `memo_reply` moments from 3 sessions with outcomes `accept`, `accept`, `modify`, `accept`, `accept`, when a `LessonCandidate` is built from it, then `agreement` is 0.9 and `eligible?` is true. 4. Given a cluster of four `accept` moments from 2 sessions, when a `LessonCandidate` is built from it, then `eligible?` is false. 5. Given a cluster from 3 sessions with outcomes `accept`, `accept`, `override`, `accept`, when a `LessonCandidate` is built from it, then `agreement` is 0.75 and `eligible?` is false. 6. Given a cluster of one `direction`, one `correction`, and one `question_answer` moment from 3 sessions, when a `LessonCandidate` is built from it, then `agreement` is 1.0 and `eligible?` is true. 7. Given an eligible cluster from 4 sessions whose seed headline is `Decision: cache the index`, when `LessonCandidate#question` is read, then it returns `In 4 sessions you ruled on "Decision: cache the index". Is this a rule? (y/n)`. 8. Given a Store whose `moments.jsonl` holds one eligible cluster and a fake embedder, when `CandidateRefresh#run` runs twice, then `queue/lesson-candidates.jsonl` holds exactly one row, with status `open`, the question, and the cluster's moment ids. 9. Given a fake embedder that raises, when `CandidateRefresh#run` runs, then the error reaches the caller and `queue/lesson-candidates.jsonl` does not exist. 10. Given a fake connection that answers `{"embeddings" => [[0.1, 0.2], [0.3, 0.4]]}`, when `OllamaEmbedder#embed(["a", "b"])` runs, then the connection receives path `api/embed` and body `{"model" => "nomic-embed-text", "input" => ["a", "b"]}`, and the result is `[[0.1, 0.2], [0.3, 0.4]]`. ## Owner sketch ```ruby module OsMe class OllamaEmbedder MODEL = 'nomic-embed-text' def self.standard; end def initialize(connection:, model: MODEL); end def embed(texts); end end class PatternClusterer SIMILARITY = 0.85 RULING_KINDS = %w[question_answer correction direction].freeze def initialize(embedder:); end def cluster(moments); end end class PatternCluster def initialize(moments); end def sessions; end def agreement; end def moment_ids; end def headline; end end class LessonCandidate MIN_SESSIONS = 3 MIN_AGREEMENT = 0.8 def initialize(cluster); end def eligible?; end def id; end def question; end def to_h; end end class CandidateLog FILE_NAME = 'queue/lesson-candidates.jsonl' def initialize(path); end def records; end def append(candidates); end end class CandidateRefresh def initialize(store_path:, embedder:); end def run; end end end ``` ## Scope and blocking Blocked by: #1245 Blocked by: #1249 Design is in `docs/decision-system/learning-loops.md` (loop 2: demotion). Related: #1240. S9b (the One Thing question, the session ledger, and the verdict readback) is a separate ticket blocked by this one. Non-goals: the One Thing question and the SessionStart hook (S9b); a model-drafted rule sentence; the calibration factor in `demotion_value` (S9b orders by moment count); tuning `SIMILARITY` (S10); a stored vector index; writing a confirmed lesson to its destination. ## Origin - Trigger: User asked for the decision-system build tickets this session - Improvised this session: none - Chain: no ticket for this stage ← the build plan names the stage and the user asked for the tickets. Stop at `DESIGN` (`docs/decision-system/build-plan.md`). - Root candidate: none found; related #1240 (design ticket, different mechanism) - Where: docs/decision-system/build-plan.md S9; plugins/os-me/lib/os_me/; plugins/os-me/hooks/ - Session: 4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5.jsonl VERDICT: CREATE (user-approved; backlog search only)

jared added the

project/cc-os

filed-by/user

category/code-change

labels

2026-10-23 15:17:29 +00:00

jared referenced this issue

2026-10-23 19:37:30 +00:00

decision-system S9b: the One Thing question at SessionStart and its verdict readback #1278

jared commented

2026-10-23 19:43:34 +00:00

Author

Owner

Work started on branch ticket-1251 (os-sdlc implement, AFK decision-system build, S9a).

jared commented

2026-10-23 19:51:53 +00:00

Author

Owner

Resolution

Done: Train ruling moments are clustered by headline embedding; clusters with 3+ sessions and 80% agreement become open lesson candidates in queue/lesson-candidates.jsonl; os-me-eval candidates runs the refresh.

Evidence: Implementation 141: 8 dispatches, 0 retries, 7 min 58 s, implementation_complete. Suite gate green. Records commit on ticket-1251.

Follow-ups: S9b is #1278 (the One Thing question and verdict readback). The live candidates command needs an embedding model; pulling nomic-embed-text waits for Jared's approval (O-8), reported to Jared in session. SIMILARITY tuning belongs to S10.

## Resolution **Done:** Train ruling moments are clustered by headline embedding; clusters with 3+ sessions and 80% agreement become open lesson candidates in queue/lesson-candidates.jsonl; os-me-eval candidates runs the refresh. **Evidence:** Implementation 141: 8 dispatches, 0 retries, 7 min 58 s, implementation_complete. Suite gate green. Records commit on ticket-1251. **Follow-ups:** S9b is #1278 (the One Thing question and verdict readback). The live candidates command needs an embedding model; pulling nomic-embed-text waits for Jared's approval (O-8), reported to Jared in session. SIMILARITY tuning belongs to S10.

jared closed this issue

2026-10-23 19:51:53 +00:00

jared referenced this issue from a commit

2026-10-23 19:52:55 +00:00

feat(os-me): Audited POODR boundaries for OllamaEmbedder, PatternClusterer, PatternCluster, LessonCandidate, CandidateLog, CandidateRefresh and the candidates command. Design and contracts hold; the change-auditor work list has no WORK items. (#1251)

jared referenced this issue from a commit

2026-10-23 19:52:55 +00:00

docs(decision-system): S9a built (#1251)

jared referenced this issue

2026-10-23 21:43:10 +00:00

S9: cluster lesson candidates on the topic line, not the headline #1289