jared/cc-os

Watch

Fork 0

S9: cluster lesson candidates on the topic line, not the headline #1289

New issue

Closed

opened 2026-10-23 21:43:10 +00:00 by jared · 2 comments

jared commented

2026-10-23 21:43:10 +00:00

Owner

Problem

S9 lesson candidates cluster ruling moments on the stored headline, the first line of the AI message. On the real Store (355 memo_reply moments, nomic-embed-text, measured 2026-10-23) this gives 211 clusters: 205 singletons, one cluster of 60 moments over 43 sessions with agreement 0.70, and 0 candidates. The headline is usually a stock line, not the topic: 69 headlines start with ## Prediction because the main session shows predictions under that heading, so they cluster together whatever their topic. Other stock first lines: ## Short answer (8), You are right (9), **Short answer (4). Also, nomic-embed-text expects a task prefix and PatternClusterer sends none; the prefix clustering: alone gave 1 candidate (a 3-session cluster, agreement 1.0).

Jared ruled option B-prime: the clusterer embeds clustering: + the topic text of each moment.

Approach

Topic text of a moment, in order of preference:

The Toward: value when the AI message has a PREDICTION block (format: Store ENTRY.md section 5, lines PREDICTION, Action:, Toward:, ...).
Else the first non-blank line that is not a markdown heading (#...) and not a stock opener (You are right, Short answer with or without leading **).
Else the stored headline.

Moment rows store only the headline (DecisionMoment), so the topic text comes from the transcript at cluster time. MomentTopic reads each distinct transcript_path once through ExchangeFinder#pairs and finds the AI text by the moment's ai_index. No stored data changes; MomentLog and DecisionMoment do not change.

PredictionParser#toward(text) owns the Toward: format: the value of the first Toward: line after a PREDICTION line, or nil. The later YAML ticket for the predictor output then changes one method.

Missing transcript: when transcript_path no longer exists (Errno::ENOENT, the same case MomentExtractor#extract_and_advance rescues), the moment uses its stored headline. Reason: the moment is a real ruling with a stored outcome. Dropping it would silently shrink a cluster below 3 sessions, and raising would stop the refresh for every other moment because one transcript was cleaned up. The headline is a worse topic, but it is the only text the Store still holds.

CandidateRefresh takes a catalog: and builds MomentTopic from ExchangeFinder.new(catalog); os-me-eval candidates passes its catalog. LessonCandidate#question still shows the cluster headline (out of scope).

Required work:

Add PredictionParser#toward(text) that returns the Toward: value of a PREDICTION block or nil — check: present def toward(text) in plugins/os-me/lib/os_me/prediction_parser.rb
Add OsMe::MomentTopic that returns one topic text per moment, falls back to the stored headline, and reads each transcript once — check: present class MomentTopic in plugins/os-me/lib/os_me/moment_topic.rb
Make PatternClusterer embed clustering: + topic text instead of the headline — check: present 'clustering: ' in plugins/os-me/lib/os_me/pattern_clusterer.rb
Remove the headline-only embed input from PatternClusterer — check: absent rulings.map { |moment| moment['headline'] } in plugins/os-me/lib/os_me/pattern_clusterer.rb
Pass the transcript catalog from os-me-eval candidates into CandidateRefresh — check: present catalog: catalog).run in plugins/os-me/lib/os_me/cli.rb

Orchestrator work: after merge, add one History row to docs/implementation-status/os-me.md, then run a live os-me-eval candidates on the real Store and report the cluster and candidate counts.

Rejected: store a topic field on each moment row — changes stored data and DecisionMoment for a text the transcript already holds. Rejected: prefix only — measured 1 candidate, the ## Prediction cluster stays.

Acceptance criteria

Given a ruling moment whose AI message has a PREDICTION block with Toward: cache the index in memory, when CandidateRefresh#run runs with a fake embedder, then the embedder receives clustering: cache the index in memory.
Given a ruling moment whose AI message has no PREDICTION block and starts with ## Prediction, then **Short answer:** yes, then You are right about the cache., then Drop the second worker., when the refresh runs, then the embedder receives clustering: Drop the second worker.
Given a ruling moment whose AI message holds only headings and stock openers, when the refresh runs, then the embedder receives clustering: followed by the moment's stored headline.
Given a ruling moment whose transcript_path does not exist, when the refresh runs, then it completes without error and the embedder receives clustering: followed by the moment's stored headline.
Given three ruling moments in three sessions, all with the headline ## Prediction and PREDICTION blocks whose Toward: texts the fake embedder maps to dissimilar vectors, when the refresh runs, then no candidate row is written.
Given three accepted ruling moments in three sessions whose Toward: texts the fake embedder maps to the same vector, when the refresh runs, then one open candidate row holds the three moment ids.
Given a non-ruling moment (kind none) next to ruling moments, when the refresh runs, then the embedder receives no text for it.
Given a refresh run, when it finishes, then moments.jsonl is byte-for-byte unchanged.
Given a text with a PREDICTION line followed by Action: accept and Toward: use option B, when PredictionParser#toward reads it, then it returns use option B; given a text with a Toward: line but no PREDICTION line, then it returns nil.

All criteria run in Minitest under plugins/os-me/test, with no model call and no network: transcripts and the Store are fixtures in a temp dir, and the embedder is a fake that records the texts it receives.

Owner sketch

module OsMe
  class PredictionParser
    def toward(text)
    end
  end

  class MomentTopic
    STOCK_OPENERS = [/\A\**\s*Short answer/i, /\AYou are right/i].freeze

    def initialize(finder:, parser: PredictionParser.new)
    end

    def texts(moments)
    end
  end

  class PatternClusterer
    PREFIX = 'clustering: '

    def initialize(embedder:, topics:)
    end

    def cluster(moments)
    end
  end

  class CandidateRefresh
    def initialize(store_path:, embedder:, catalog:)
    end

    def run
    end
  end
end

Retired tests

plugins/os-me/test/pattern_clusterer_test.rb
plugins/os-me/test/candidate_refresh_test.rb

Scope and blocking

In scope: plugins/os-me/lib/os_me/ (prediction_parser.rb, new moment_topic.rb, pattern_clusterer.rb, candidate_refresh.rb, cli.rb) and tests under plugins/os-me/test.

Out of scope: MomentLog, DecisionMoment, MomentExtractor, stored Store data, OllamaEmbedder, the similarity threshold (0.85), LessonCandidate eligibility and question text, and the predictor output format change to YAML (separate ticket; it will change only PredictionParser#toward).

Blocking: none. Blocks the S9 live refresh and the S9b One Thing question, which have no candidates to ask until this ships.

Origin

Trigger: Live S9 refresh on 2026-10-23 found 0 candidates; Jared ruled B-prime
Improvised this session: none
Chain: 0 lesson candidates ← 69 "Prediction" heading headlines and other stock first lines cluster by layout, not topic ← PatternClusterer embeds moment['headline'] (plugins/os-me/lib/os_me/pattern_clusterer.rb) with no nomic task prefix ← Pair#headline is the first AI line (plugins/os-me/lib/os_me/pair.rb) ← DESIGN: S9a (#1251) chose the headline as the cluster text
Root candidate: none found
Where: plugins/os-me/lib/os_me/pattern_clusterer.rb; candidate_refresh.rb; prediction_parser.rb
Session: 4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5.jsonl

VERDICT: CREATE (user-approved; backlog search only)

## Problem S9 lesson candidates cluster ruling moments on the stored headline, the first line of the AI message. On the real Store (355 `memo_reply` moments, `nomic-embed-text`, measured 2026-10-23) this gives 211 clusters: 205 singletons, one cluster of 60 moments over 43 sessions with agreement 0.70, and 0 candidates. The headline is usually a stock line, not the topic: 69 headlines start with `## Prediction` because the main session shows predictions under that heading, so they cluster together whatever their topic. Other stock first lines: `## Short answer` (8), `You are right` (9), `**Short answer` (4). Also, `nomic-embed-text` expects a task prefix and `PatternClusterer` sends none; the prefix `clustering: ` alone gave 1 candidate (a 3-session cluster, agreement 1.0). Jared ruled option B-prime: the clusterer embeds `clustering: ` + the topic text of each moment. ## Approach Topic text of a moment, in order of preference: 1. The `Toward:` value when the AI message has a PREDICTION block (format: Store `ENTRY.md` section 5, lines `PREDICTION`, `Action:`, `Toward:`, ...). 2. Else the first non-blank line that is not a markdown heading (`#`...) and not a stock opener (`You are right`, `Short answer` with or without leading `**`). 3. Else the stored `headline`. Moment rows store only the headline (`DecisionMoment`), so the topic text comes from the transcript at cluster time. `MomentTopic` reads each distinct `transcript_path` once through `ExchangeFinder#pairs` and finds the AI text by the moment's `ai_index`. No stored data changes; `MomentLog` and `DecisionMoment` do not change. `PredictionParser#toward(text)` owns the `Toward:` format: the value of the first `Toward:` line after a `PREDICTION` line, or nil. The later YAML ticket for the predictor output then changes one method. Missing transcript: when `transcript_path` no longer exists (`Errno::ENOENT`, the same case `MomentExtractor#extract_and_advance` rescues), the moment uses its stored headline. Reason: the moment is a real ruling with a stored outcome. Dropping it would silently shrink a cluster below 3 sessions, and raising would stop the refresh for every other moment because one transcript was cleaned up. The headline is a worse topic, but it is the only text the Store still holds. `CandidateRefresh` takes a `catalog:` and builds `MomentTopic` from `ExchangeFinder.new(catalog)`; `os-me-eval candidates` passes its `catalog`. `LessonCandidate#question` still shows the cluster headline (out of scope). Required work: 1. Add `PredictionParser#toward(text)` that returns the `Toward:` value of a PREDICTION block or nil — check: present `def toward(text)` in `plugins/os-me/lib/os_me/prediction_parser.rb` 2. Add `OsMe::MomentTopic` that returns one topic text per moment, falls back to the stored headline, and reads each transcript once — check: present `class MomentTopic` in `plugins/os-me/lib/os_me/moment_topic.rb` 3. Make `PatternClusterer` embed `clustering: ` + topic text instead of the headline — check: present `'clustering: '` in `plugins/os-me/lib/os_me/pattern_clusterer.rb` 4. Remove the headline-only embed input from `PatternClusterer` — check: absent `rulings.map { |moment| moment['headline'] }` in `plugins/os-me/lib/os_me/pattern_clusterer.rb` 5. Pass the transcript catalog from `os-me-eval candidates` into `CandidateRefresh` — check: present `catalog: catalog).run` in `plugins/os-me/lib/os_me/cli.rb` Orchestrator work: after merge, add one History row to `docs/implementation-status/os-me.md`, then run a live `os-me-eval candidates` on the real Store and report the cluster and candidate counts. Rejected: store a topic field on each moment row — changes stored data and `DecisionMoment` for a text the transcript already holds. Rejected: prefix only — measured 1 candidate, the `## Prediction` cluster stays. ## Acceptance criteria 1. Given a ruling moment whose AI message has a PREDICTION block with `Toward: cache the index in memory`, when `CandidateRefresh#run` runs with a fake embedder, then the embedder receives `clustering: cache the index in memory`. 2. Given a ruling moment whose AI message has no PREDICTION block and starts with `## Prediction`, then `**Short answer:** yes`, then `You are right about the cache.`, then `Drop the second worker.`, when the refresh runs, then the embedder receives `clustering: Drop the second worker.` 3. Given a ruling moment whose AI message holds only headings and stock openers, when the refresh runs, then the embedder receives `clustering: ` followed by the moment's stored headline. 4. Given a ruling moment whose `transcript_path` does not exist, when the refresh runs, then it completes without error and the embedder receives `clustering: ` followed by the moment's stored headline. 5. Given three ruling moments in three sessions, all with the headline `## Prediction` and PREDICTION blocks whose `Toward:` texts the fake embedder maps to dissimilar vectors, when the refresh runs, then no candidate row is written. 6. Given three accepted ruling moments in three sessions whose `Toward:` texts the fake embedder maps to the same vector, when the refresh runs, then one open candidate row holds the three moment ids. 7. Given a non-ruling moment (kind `none`) next to ruling moments, when the refresh runs, then the embedder receives no text for it. 8. Given a refresh run, when it finishes, then `moments.jsonl` is byte-for-byte unchanged. 9. Given a text with a `PREDICTION` line followed by `Action: accept` and `Toward: use option B`, when `PredictionParser#toward` reads it, then it returns `use option B`; given a text with a `Toward:` line but no `PREDICTION` line, then it returns nil. All criteria run in Minitest under `plugins/os-me/test`, with no model call and no network: transcripts and the Store are fixtures in a temp dir, and the embedder is a fake that records the texts it receives. ## Owner sketch ```ruby module OsMe class PredictionParser def toward(text) end end class MomentTopic STOCK_OPENERS = [/\A\**\s*Short answer/i, /\AYou are right/i].freeze def initialize(finder:, parser: PredictionParser.new) end def texts(moments) end end class PatternClusterer PREFIX = 'clustering: ' def initialize(embedder:, topics:) end def cluster(moments) end end class CandidateRefresh def initialize(store_path:, embedder:, catalog:) end def run end end end ``` ## Retired tests plugins/os-me/test/pattern_clusterer_test.rb plugins/os-me/test/candidate_refresh_test.rb ## Scope and blocking In scope: `plugins/os-me/lib/os_me/` (`prediction_parser.rb`, new `moment_topic.rb`, `pattern_clusterer.rb`, `candidate_refresh.rb`, `cli.rb`) and tests under `plugins/os-me/test`. Out of scope: `MomentLog`, `DecisionMoment`, `MomentExtractor`, stored Store data, `OllamaEmbedder`, the similarity threshold (0.85), `LessonCandidate` eligibility and question text, and the predictor output format change to YAML (separate ticket; it will change only `PredictionParser#toward`). Blocking: none. Blocks the S9 live refresh and the S9b One Thing question, which have no candidates to ask until this ships. ## Origin - Trigger: Live S9 refresh on 2026-10-23 found 0 candidates; Jared ruled B-prime - Improvised this session: none - Chain: 0 lesson candidates ← 69 "Prediction" heading headlines and other stock first lines cluster by layout, not topic ← `PatternClusterer` embeds `moment['headline']` (`plugins/os-me/lib/os_me/pattern_clusterer.rb`) with no nomic task prefix ← `Pair#headline` is the first AI line (`plugins/os-me/lib/os_me/pair.rb`) ← DESIGN: S9a (#1251) chose the headline as the cluster text - Root candidate: none found - Where: plugins/os-me/lib/os_me/pattern_clusterer.rb; candidate_refresh.rb; prediction_parser.rb - Session: 4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/4f3f95ea-3c6f-4d23-a0ea-e349d6ab02a5.jsonl VERDICT: CREATE (user-approved; backlog search only)

jared added the

project/cc-os

filed-by/user

category/code-change

labels

2026-10-23 21:43:10 +00:00

jared commented

2026-10-23 21:43:21 +00:00

Author

Owner

Work started on branch ticket-1289 (os-sdlc implement).

jared referenced this issue

2026-10-23 21:44:57 +00:00

os-me: the predictor returns YAML; PredictionParser reads YAML and the old line format #1292

jared referenced this issue

2026-10-23 21:46:20 +00:00

issue-create ends the Origin section at an inline heading mark and refuses a valid body #965

jared commented

2026-10-23 21:50:09 +00:00

Author

Owner

Resolution

Done: PatternClusterer embeds clustering: plus the moment topic (Toward value, else first non-heading non-stock line, else headline) through MomentTopic; PredictionParser#toward added; CandidateRefresh takes catalog:

Evidence: Implementation 151 complete in 6 min 12 s, 8 dispatches, all pass; suite gate green; commits 292f53be and f7a5697c on ticket-1289

Follow-ups: #1292 extends toward to the YAML block; live candidates rerun reported in session

## Resolution **Done:** PatternClusterer embeds clustering: plus the moment topic (Toward value, else first non-heading non-stock line, else headline) through MomentTopic; PredictionParser#toward added; CandidateRefresh takes catalog: **Evidence:** Implementation 151 complete in 6 min 12 s, 8 dispatches, all pass; suite gate green; commits 292f53be and f7a5697c on ticket-1289 **Follow-ups:** #1292 extends toward to the YAML block; live candidates rerun reported in session

jared closed this issue

2026-10-23 21:50:09 +00:00

jared referenced this issue from a commit

2026-10-23 21:51:12 +00:00

feat(os-me): Audited POODR boundaries for #1289: MomentTopic owns topic selection, PredictionParser#toward owns the Toward format, PatternClusterer takes a topics role and prefixes clustering; contracts held; required work 1-5 present. (#1289)

jared referenced this issue from a commit

2026-10-23 21:51:12 +00:00

docs(os-me): Record #1289 cluster text; remove retired headline-keyed tests (#1289)

jared referenced this issue

2026-10-23 21:52:37 +00:00

S9: skip the prediction disclaimer line when choosing the moment topic #1293

jared referenced this issue