Extract exits, result_shape and value_semantics to measure three rules #1551

Closed
opened 2026-11-04 21:46:55 +00:00 by jared · 2 comments
Owner

Problem

system-model opportunities prints three rules as not measurable: P&F R2 needs fact exits, CMD R2 needs fact result_shape, SPEC R2 needs fact value_semantics. Each rule uses Rule::Unmeasured, so it can never flag a line. A reader cannot tell "no misfit" from "not measured" for these rules. Run 3 records finding #14 as a miss with the cause "missing fact (result_shape)".

Reproduce

n/a: this ticket adds facts and rules; no defect exists to reproduce.

Approach

Only value_semantics has a landscape entry (F16). exits and result_shape have no F number. This ticket defines both.

Fact to rule map.

  1. exits to P&F R2, "Flow runs one way, with no exit in the middle of the chain."
    1. Fact: ClassFacts#exits, a list of Exit(path, line), one entry for each process exit: a call of exit, exit! or abort with no receiver or with receiver Kernel or Process. io holds exit and abort as terminal entries, but an Io entry keeps no call name, so io cannot answer this.
    2. Rule: Rule::ForbiddenExit.new(roles: %i[Filter Pipe]) flags each exit of a class that fits the role when the terminal io entries on its exit lines are removed. ForbiddenIo uses the same removal. An exit in Sink is the end of the chain, so it is not a misfit.
    3. raise is not an exit. pattern-matching.md section 2 treats a guard that raises as a decision. 65 of 797 Filter classes in os-sdlc sit in a file with raise, so counting raise gives false positives on guards.
    4. Expected result today: 0 sites. No file under lib/ in os, os-backlog, os-me or os-sdlc calls exit or abort. The run 1 hand count ("exits in CliOptions, CliTrackerRequirement, Commands") describes code that no longer exists; Backlog::ExitStatus is now raised instead.
  2. result_shape to CMD R2, "Commands share one result shape."
    1. Fact: ClassFacts#result_shape, one ResultShape(shape, path, line) for a public call, or nil when the class has no call. shape comes from the last expression of call: :nil, :string, :integer, :boolean, :array, :hash, :self, the last name segment of Const.new, or :unknown. A self-send follows the sent method in the same class. :unknown covers every other expression and a conditional whose branches differ.
    2. Rule: Rule::DivergentResult.new(roles: [:Command]). In one component, take the Command classes with a known shape. When at least 3 exist and one shape holds a strict majority, flag the call line of each other class. :unknown is never a misfit. A component with no strict majority gives no site.
    3. A scratch run found 0 sites in os-backlog and os-me, and 3 in os-sdlc: Crc::Records and Crc::Sweep (Report where the majority is Result), and Evidence::DirectUnitEntries (:array where the majority is Entry). Run 3 finding #14 stays a miss, because half of the os-backlog commands have shape :unknown. The misses are acceptable.
  3. value_semantics to SPEC R2, "A specification is a value object that combines with and, or, not."
    1. Fact: ClassFacts#state_writes, a list of Write(path, line), one entry for each ivar write outside initialize that is not ||=. Derived reader: value_semantics = value? || state_writes.empty?.
    2. The fact leaves out ||= because mutates_state? counts memoization. Inspector::ReferenceMatch, ForwardingClass::RequiredLibraries and ForwardingClass::MethodMatch mutate only through @x ||=, and these writes change no visible value.
    3. The fact leaves out the F16 marks freeze and ==. 45 of 61 classes that fit Specification have none of these marks and no mutation. A rule on the marks would flag those 45 and rank them at threshold 1.
    4. Rule: Rule::BrokenValue.new(roles: [:Specification]) flags each state_writes entry of a class with value_semantics false. A scratch run found 1 site: RequireInsideCode::TopRun at top_run.rb:30.
    5. The rule sentence loses "that combines with and, or, not". Ruby code combines predicates with &&, so the combination half has no fact, and a rule on it would flag every specification. The sentence becomes "A specification is a value object."

Effect on calibration. P&F, CMD and SPEC are strict, so these rules now add flags to convergence and violations to Match#distance. This does not change a threshold. It moves the detector toward the calibration: run 2 counted R2 by hand, and run 3 could not count it. A scratch run with CMD R2 and SPEC R2 measured gave these changes over os-backlog, os-me and os-sdlc:

  1. Group A rows: no change.
  2. Group C rows: one added row in os-sdlc, top_run.rb:30 SPEC R2.
  3. present rows: one row leaves in os-sdlc, because a violation raises its distance.

This ticket does not change the 0.05 threshold or Match#present?. A later change can move a file from convergence 2 to convergence 3 through these rules. That is the intended behavior of a measured rule.

Required work:

  1. Add the exit value and record each process exit — check: present Exit = Data.define in plugins/os-sdlc/lib/os_sdlc/system_model/class_facts/exit.rb
  2. Add the result shape value — check: present ResultShape = Data.define in plugins/os-sdlc/lib/os_sdlc/system_model/class_facts/result_shape.rb
  3. Add the write value and record each non-memoized ivar write outside initialize — check: present Write = Data.define in plugins/os-sdlc/lib/os_sdlc/system_model/class_facts/write.rb
  4. Add the three fields and the derived reader to ClassFacts — check: present def value_semantics in plugins/os-sdlc/lib/os_sdlc/system_model/class_facts.rb
  5. Add the P&F R2 rule class — check: present ForbiddenExit = Data.define in plugins/os-sdlc/lib/os_sdlc/system_model/pattern/rule/forbidden_exit.rb
  6. Add the CMD R2 rule class — check: present DivergentResult = Data.define in plugins/os-sdlc/lib/os_sdlc/system_model/pattern/rule/divergent_result.rb
  7. Add the SPEC R2 rule class — check: present BrokenValue = Data.define in plugins/os-sdlc/lib/os_sdlc/system_model/pattern/rule/broken_value.rb
  8. Measure P&F R2 — check: absent Unmeasured in plugins/os-sdlc/lib/os_sdlc/system_model/catalog/pipes_and_filters.rb
  9. Measure CMD R2 — check: absent Unmeasured in plugins/os-sdlc/lib/os_sdlc/system_model/catalog/command_router.rb
  10. Measure SPEC R2 and shorten its sentence — check: absent Unmeasured in plugins/os-sdlc/lib/os_sdlc/system_model/catalog/specification.rb
  11. Define exits, result_shape and value_semantics in section 2 of the method document — check: present result_shape in docs/system-design-format/pattern-matching.md

Orchestrator work: add one History row to docs/implementation-status/os-sdlc-system-model.md.

Rejected: mark the three rules as measured with no fact — a zero would then mean nothing. Count raise as an exit — false positives on guards. Require == or freeze for value semantics — 45 false positives at threshold 1. Move the rules out of convergence — no per-rule switch exists, and the hand calibration counted them.

Acceptance criteria

  1. Given the catalog, when opportunities runs, then the not measurable section has no rows, and the JSON unmeasured list is empty.
  2. Given a class with exit in a method, when ClassFacts is built, then exits holds the path and line of that call.
  3. Given a class with Process.exit or exit!, when ClassFacts is built, then exits holds that call.
  4. Given a class with raise, when ClassFacts is built, then exits is empty.
  5. Given a class with an exit that fits Filter when the exit is removed, when P&F matches the component, then R2 flags the exit line.
  6. Given a non-entry class with an exit and other I/O, so that it stays Sink, when P&F matches the component, then R2 flags no line.
  7. Given a call whose last expression is Result.new(...), when ClassFacts is built, then result_shape holds the shape Result.
  8. Given a call whose last expression is a self-send to a method that returns a string, when ClassFacts is built, then result_shape holds the shape :string.
  9. Given a call whose last expression is a call on another object, when ClassFacts is built, then result_shape holds the shape :unknown.
  10. Given a component with three Command classes of shape Result and one of shape :array, when CMD matches the component, then R2 flags the call line of the :array class only.
  11. Given a component with three Command classes of three different shapes, when CMD matches the component, then R2 flags no line.
  12. Given a component with two Command classes of different shapes and one of shape :unknown, when CMD matches the component, then R2 flags no line.
  13. Given a class with @x ||= 1 in a method, when ClassFacts is built, then state_writes is empty and value_semantics is true.
  14. Given a class with @x = 1 in a method other than initialize, when ClassFacts is built, then state_writes holds that line and value_semantics is false.
  15. Given a Data.define value, when ClassFacts is built, then value_semantics is true.
  16. Given a class that fits Specification and writes an ivar outside initialize, when SPEC matches the component, then R2 flags the write line.
  17. Given the opportunities --json output for os-backlog, os-me and os-sdlc before this ticket, when the command runs after this ticket, then the group A rows are the same.

Owner sketch

class OsSdlc::SystemModel::ClassFacts
  Exit = Data.define(:path, :line)
  ResultShape = Data.define(:shape, :path, :line)
  Write = Data.define(:path, :line)

  def value_semantics = value? || state_writes.empty?
end

class OsSdlc::SystemModel::Extractor
  class ProcessExit
    def initialize(node)
    def entry(path)
  end

  class ResultExpression
    def initialize(def_node, defs)
    def shape
  end
end

class OsSdlc::SystemModel::Pattern::Rule
  ForbiddenExit = Data.define(:roles) do
    def sites(assignment)
  end

  DivergentResult = Data.define(:roles) do
    def sites(assignment)
  end

  BrokenValue = Data.define(:roles) do
    def sites(assignment)
  end
end

Non-goals

Do not change the 0.05 threshold, Match#present?, mutates_state?, or Catalog.thresholds. Do not delete Rule::Unmeasured; tests and later patterns use it. Do not add a memoization fact (F18). Do not widen routes_to.

Retired tests

plugins/os-sdlc/tests/system_model_class_scale_patterns_test.rb#test_specification_value_semantics_shows_as_not_measurable
plugins/os-sdlc/tests/system_model_pattern_match_hand_counts_test.rb#test_b7_cmd_r2_result_shape_shows_as_not_measurable

Origin

  • Trigger: plugins/os-sdlc/bin/system-model opportunities plugins/os-backlog prints not measurable for P&F R2, CMD R2 and SPEC R2.
  • Improvised this session: none
  • Chain: three rules print not measurable ← each rule uses Rule::Unmeasured ← ClassFacts has no field for the rule. DESIGN: docs/system-design-format/pattern-matching.md section 2, "The fact schema limits the catalog".
  • Root candidate: none found; searched exits, result_shape, value_semantics, Unmeasured, not measurable, ClassFacts
  • Where: plugins/os-sdlc/lib/os_sdlc/system_model/class_facts.rb, plugins/os-sdlc/lib/os_sdlc/system_model/extractor/fact_scan.rb
  • Session: 16528b06-720e-4ee8-b195-838770c1efe8
  • Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/16528b06-720e-4ee8-b195-838770c1efe8.jsonl

VERDICT: CREATE

## Problem `system-model opportunities` prints three rules as `not measurable`: P&F R2 needs fact `exits`, CMD R2 needs fact `result_shape`, SPEC R2 needs fact `value_semantics`. Each rule uses `Rule::Unmeasured`, so it can never flag a line. A reader cannot tell "no misfit" from "not measured" for these rules. Run 3 records finding #14 as a miss with the cause "missing fact (`result_shape`)". ## Reproduce n/a: this ticket adds facts and rules; no defect exists to reproduce. ## Approach Only `value_semantics` has a landscape entry (F16). `exits` and `result_shape` have no F number. This ticket defines both. **Fact to rule map.** 1. `exits` to P&F R2, "Flow runs one way, with no exit in the middle of the chain." 1. Fact: `ClassFacts#exits`, a list of `Exit(path, line)`, one entry for each process exit: a call of `exit`, `exit!` or `abort` with no receiver or with receiver `Kernel` or `Process`. `io` holds `exit` and `abort` as terminal entries, but an `Io` entry keeps no call name, so `io` cannot answer this. 2. Rule: `Rule::ForbiddenExit.new(roles: %i[Filter Pipe])` flags each exit of a class that fits the role when the terminal `io` entries on its exit lines are removed. `ForbiddenIo` uses the same removal. An exit in `Sink` is the end of the chain, so it is not a misfit. 3. `raise` is not an exit. pattern-matching.md section 2 treats a guard that raises as a decision. 65 of 797 `Filter` classes in os-sdlc sit in a file with `raise`, so counting `raise` gives false positives on guards. 4. Expected result today: 0 sites. No file under `lib/` in os, os-backlog, os-me or os-sdlc calls `exit` or `abort`. The run 1 hand count ("exits in CliOptions, CliTrackerRequirement, Commands") describes code that no longer exists; `Backlog::ExitStatus` is now raised instead. 2. `result_shape` to CMD R2, "Commands share one result shape." 1. Fact: `ClassFacts#result_shape`, one `ResultShape(shape, path, line)` for a public `call`, or nil when the class has no `call`. `shape` comes from the last expression of `call`: `:nil`, `:string`, `:integer`, `:boolean`, `:array`, `:hash`, `:self`, the last name segment of `Const.new`, or `:unknown`. A self-send follows the sent method in the same class. `:unknown` covers every other expression and a conditional whose branches differ. 2. Rule: `Rule::DivergentResult.new(roles: [:Command])`. In one component, take the `Command` classes with a known shape. When at least 3 exist and one shape holds a strict majority, flag the `call` line of each other class. `:unknown` is never a misfit. A component with no strict majority gives no site. 3. A scratch run found 0 sites in os-backlog and os-me, and 3 in os-sdlc: `Crc::Records` and `Crc::Sweep` (`Report` where the majority is `Result`), and `Evidence::DirectUnitEntries` (`:array` where the majority is `Entry`). Run 3 finding #14 stays a miss, because half of the os-backlog commands have shape `:unknown`. The misses are acceptable. 3. `value_semantics` to SPEC R2, "A specification is a value object that combines with and, or, not." 1. Fact: `ClassFacts#state_writes`, a list of `Write(path, line)`, one entry for each ivar write outside `initialize` that is not `||=`. Derived reader: `value_semantics = value? || state_writes.empty?`. 2. The fact leaves out `||=` because `mutates_state?` counts memoization. `Inspector::ReferenceMatch`, `ForwardingClass::RequiredLibraries` and `ForwardingClass::MethodMatch` mutate only through `@x ||=`, and these writes change no visible value. 3. The fact leaves out the F16 marks `freeze` and `==`. 45 of 61 classes that fit `Specification` have none of these marks and no mutation. A rule on the marks would flag those 45 and rank them at threshold 1. 4. Rule: `Rule::BrokenValue.new(roles: [:Specification])` flags each `state_writes` entry of a class with `value_semantics` false. A scratch run found 1 site: `RequireInsideCode::TopRun` at `top_run.rb:30`. 5. The rule sentence loses "that combines with and, or, not". Ruby code combines predicates with `&&`, so the combination half has no fact, and a rule on it would flag every specification. The sentence becomes "A specification is a value object." **Effect on calibration.** P&F, CMD and SPEC are strict, so these rules now add flags to convergence and violations to `Match#distance`. This does not change a threshold. It moves the detector toward the calibration: run 2 counted R2 by hand, and run 3 could not count it. A scratch run with CMD R2 and SPEC R2 measured gave these changes over os-backlog, os-me and os-sdlc: 1. Group A rows: no change. 2. Group C rows: one added row in os-sdlc, `top_run.rb:30` `SPEC R2`. 3. `present` rows: one row leaves in os-sdlc, because a violation raises its distance. This ticket does not change the 0.05 threshold or `Match#present?`. A later change can move a file from convergence 2 to convergence 3 through these rules. That is the intended behavior of a measured rule. Required work: 1. Add the exit value and record each process exit — check: present `Exit = Data.define` in `plugins/os-sdlc/lib/os_sdlc/system_model/class_facts/exit.rb` 2. Add the result shape value — check: present `ResultShape = Data.define` in `plugins/os-sdlc/lib/os_sdlc/system_model/class_facts/result_shape.rb` 3. Add the write value and record each non-memoized ivar write outside `initialize` — check: present `Write = Data.define` in `plugins/os-sdlc/lib/os_sdlc/system_model/class_facts/write.rb` 4. Add the three fields and the derived reader to `ClassFacts` — check: present `def value_semantics` in `plugins/os-sdlc/lib/os_sdlc/system_model/class_facts.rb` 5. Add the P&F R2 rule class — check: present `ForbiddenExit = Data.define` in `plugins/os-sdlc/lib/os_sdlc/system_model/pattern/rule/forbidden_exit.rb` 6. Add the CMD R2 rule class — check: present `DivergentResult = Data.define` in `plugins/os-sdlc/lib/os_sdlc/system_model/pattern/rule/divergent_result.rb` 7. Add the SPEC R2 rule class — check: present `BrokenValue = Data.define` in `plugins/os-sdlc/lib/os_sdlc/system_model/pattern/rule/broken_value.rb` 8. Measure P&F R2 — check: absent `Unmeasured` in `plugins/os-sdlc/lib/os_sdlc/system_model/catalog/pipes_and_filters.rb` 9. Measure CMD R2 — check: absent `Unmeasured` in `plugins/os-sdlc/lib/os_sdlc/system_model/catalog/command_router.rb` 10. Measure SPEC R2 and shorten its sentence — check: absent `Unmeasured` in `plugins/os-sdlc/lib/os_sdlc/system_model/catalog/specification.rb` 11. Define `exits`, `result_shape` and `value_semantics` in section 2 of the method document — check: present `result_shape` in `docs/system-design-format/pattern-matching.md` Orchestrator work: add one History row to `docs/implementation-status/os-sdlc-system-model.md`. Rejected: mark the three rules as measured with no fact — a zero would then mean nothing. Count `raise` as an exit — false positives on guards. Require `==` or `freeze` for value semantics — 45 false positives at threshold 1. Move the rules out of convergence — no per-rule switch exists, and the hand calibration counted them. ## Acceptance criteria 1. Given the catalog, when `opportunities` runs, then the `not measurable` section has no rows, and the JSON `unmeasured` list is empty. 2. Given a class with `exit` in a method, when `ClassFacts` is built, then `exits` holds the path and line of that call. 3. Given a class with `Process.exit` or `exit!`, when `ClassFacts` is built, then `exits` holds that call. 4. Given a class with `raise`, when `ClassFacts` is built, then `exits` is empty. 5. Given a class with an `exit` that fits `Filter` when the exit is removed, when `P&F` matches the component, then R2 flags the exit line. 6. Given a non-entry class with an exit and other I/O, so that it stays `Sink`, when `P&F` matches the component, then R2 flags no line. 7. Given a `call` whose last expression is `Result.new(...)`, when `ClassFacts` is built, then `result_shape` holds the shape `Result`. 8. Given a `call` whose last expression is a self-send to a method that returns a string, when `ClassFacts` is built, then `result_shape` holds the shape `:string`. 9. Given a `call` whose last expression is a call on another object, when `ClassFacts` is built, then `result_shape` holds the shape `:unknown`. 10. Given a component with three `Command` classes of shape `Result` and one of shape `:array`, when `CMD` matches the component, then R2 flags the `call` line of the `:array` class only. 11. Given a component with three `Command` classes of three different shapes, when `CMD` matches the component, then R2 flags no line. 12. Given a component with two `Command` classes of different shapes and one of shape `:unknown`, when `CMD` matches the component, then R2 flags no line. 13. Given a class with `@x ||= 1` in a method, when `ClassFacts` is built, then `state_writes` is empty and `value_semantics` is true. 14. Given a class with `@x = 1` in a method other than `initialize`, when `ClassFacts` is built, then `state_writes` holds that line and `value_semantics` is false. 15. Given a `Data.define` value, when `ClassFacts` is built, then `value_semantics` is true. 16. Given a class that fits `Specification` and writes an ivar outside `initialize`, when `SPEC` matches the component, then R2 flags the write line. 17. Given the `opportunities --json` output for os-backlog, os-me and os-sdlc before this ticket, when the command runs after this ticket, then the group A rows are the same. ## Owner sketch ```ruby class OsSdlc::SystemModel::ClassFacts Exit = Data.define(:path, :line) ResultShape = Data.define(:shape, :path, :line) Write = Data.define(:path, :line) def value_semantics = value? || state_writes.empty? end class OsSdlc::SystemModel::Extractor class ProcessExit def initialize(node) def entry(path) end class ResultExpression def initialize(def_node, defs) def shape end end class OsSdlc::SystemModel::Pattern::Rule ForbiddenExit = Data.define(:roles) do def sites(assignment) end DivergentResult = Data.define(:roles) do def sites(assignment) end BrokenValue = Data.define(:roles) do def sites(assignment) end end ``` ## Non-goals Do not change the 0.05 threshold, `Match#present?`, `mutates_state?`, or `Catalog.thresholds`. Do not delete `Rule::Unmeasured`; tests and later patterns use it. Do not add a memoization fact (F18). Do not widen `routes_to`. ## Retired tests plugins/os-sdlc/tests/system_model_class_scale_patterns_test.rb#test_specification_value_semantics_shows_as_not_measurable plugins/os-sdlc/tests/system_model_pattern_match_hand_counts_test.rb#test_b7_cmd_r2_result_shape_shows_as_not_measurable ## Origin - Trigger: `plugins/os-sdlc/bin/system-model opportunities plugins/os-backlog` prints `not measurable` for P&F R2, CMD R2 and SPEC R2. - Improvised this session: none - Chain: three rules print `not measurable` ← each rule uses `Rule::Unmeasured` ← ClassFacts has no field for the rule. DESIGN: `docs/system-design-format/pattern-matching.md` section 2, "The fact schema limits the catalog". - Root candidate: none found; searched exits, result_shape, value_semantics, Unmeasured, not measurable, ClassFacts - Where: `plugins/os-sdlc/lib/os_sdlc/system_model/class_facts.rb`, `plugins/os-sdlc/lib/os_sdlc/system_model/extractor/fact_scan.rb` - Session: 16528b06-720e-4ee8-b195-838770c1efe8 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os/16528b06-720e-4ee8-b195-838770c1efe8.jsonl VERDICT: CREATE
Author
Owner

Work started on branch ticket-1551.

Work started on branch ticket-1551.
Author
Owner

Resolution

Done: Facts exits, result_shape, state_writes and reader value_semantics added; rules ForbiddenExit, DivergentResult, BrokenValue replace Unmeasured for P&F R2, CMD R2, SPEC R2.

Evidence: main d9a42338; suite 3422 runs 0 failures; opportunities on os-backlog, os-me, os-sdlc: group A rows unchanged, group C +1 at top_run.rb:30, present -1, unmeasured empty. Hand-finished after escalation 105 (test pinned the field list); design audit 6 defects and 7 preferences fixed or answered.

Follow-ups: none

## Resolution **Done:** Facts exits, result_shape, state_writes and reader value_semantics added; rules ForbiddenExit, DivergentResult, BrokenValue replace Unmeasured for P&F R2, CMD R2, SPEC R2. **Evidence:** main d9a42338; suite 3422 runs 0 failures; opportunities on os-backlog, os-me, os-sdlc: group A rows unchanged, group C +1 at top_run.rb:30, present -1, unmeasured empty. Hand-finished after escalation 105 (test pinned the field list); design audit 6 defects and 7 preferences fixed or answered. **Follow-ups:** none
jared closed this issue 2026-11-05 01:15:16 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
jared/cc-os#1551
No description provided.