Make collaborator-pass delta-check fail when a clump moves to a new subject #1214

New issue

Open

opened 2026-10-20 14:05:48 +00:00 by jared · 1 comment

jared commented

2026-10-20 14:05:48 +00:00

Owner

Problem

collaborator-pass delta-check passes a ticket when the target finding goes down and each shape total stays at or below the baseline run. It compares the shape total against the baseline run, not against the state before the ticket. Earlier merged tickets lower the total and leave headroom. A clump that moves to a new subject then passes. The data_clump problem stays and the check says ok.

Observed

Ticket #1202 replaced root and tracker with a Project in Implementation::Open and Implementation::RerouteIntake. The check passed before and after the commit. The orchestrator reported:

data_clump `root+tracker`  12 -> 0
data_clump `project+tracker`  0 -> 12   (new subject)
data_clump total: 209 -> 209
name_collision `tracker`  0 -> 4        (new finding)

The data_clump total stayed at 209. The baseline run held 246, so 209 <= 246 passed. The code is DeltaCheck#shape_row: "ok" => after <= before, where before is the baseline run tally. DeltaCheck#finding_row checks only the target subject (after <= limit). The orchestrator wrote: "This is the second example after #1200, and the delta-check does not detect it."

Reproduce

BASE=docs/crc/os-sdlc/collaborator-runs/20261020T020325Z
plugins/os-sdlc/bin/collaborator-pass --target os-sdlc delta-check --baseline $BASE --ledger docs/crc/os-sdlc/ledger.tsv $BASE/recipes/project.yml

Run it on the tree at the #1202 commit. The check exits 0.

Approach

Compare against the tree before the change for each shape. Report a subject that is new in the after tally and shares a member name with the target subject as a moved clump. Fail the check when the target score leaves and an equal score appears on a subject with a shared name.

Required work:

DeltaCheck::Tally exposes the subjects of a shape with their scores — check: present def subjects in plugins/os-sdlc/lib/os_sdlc/collaborator_pass/delta_check/tally.rb
DeltaCheck#call adds a moved row for each new subject in the same shape that shares a name with the target subject. ok is false when one exists.
The delta-check command accepts the pre-change measure, or measures the tree before the change, as the second tally.
skills/collaborator-pass/SKILL.md step 7 names the moved row.

Rejected: A stricter shape-total limit alone — it does not see a clump that is replaced by a clump of the same score.

Acceptance criteria

Given a finding root+tracker at 12 before and 0 after, and a new project+tracker at 12 after, when delta-check runs, then it exits 1 and names project+tracker as the moved subject.
Given a finding that drops to 0 and no new subject shares a name with it, when delta-check runs, then it exits 0.
Given the #1200 case (implementation+project 20 to 0, implementation+root 6 to 44), when delta-check runs, then it still exits 1.

Owner sketch

class OsSdlc::CollaboratorPass::DeltaCheck
  def call
end

Origin

Trigger: collaborator-pass round 2, ticket #1202 (also #1200)
Improvised this session: the orchestrator ran delta-check before and after the commit, kept the change, and listed the moved clump as a follow-up
Chain: moved clump passes ← DeltaCheck#shape_row compares to the baseline run total, which has headroom from earlier tickets ← DESIGN (plugins/os-sdlc/lib/os_sdlc/collaborator_pass/delta_check.rb)
Root candidate: none found
Where: plugins/os-sdlc/lib/os_sdlc/collaborator_pass/delta_check.rb
Session: 2e167079-f404-4139-9cfa-6c0c937ce9f0
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-crc-os-sdlc-run-3/2e167079-f404-4139-9cfa-6c0c937ce9f0.jsonl

VERDICT: CREATE

## Problem `collaborator-pass delta-check` passes a ticket when the target finding goes down and each shape total stays at or below the baseline run. It compares the shape total against the baseline run, not against the state before the ticket. Earlier merged tickets lower the total and leave headroom. A clump that moves to a new subject then passes. The data_clump problem stays and the check says ok. ## Observed Ticket #1202 replaced `root` and `tracker` with a `Project` in `Implementation::Open` and `Implementation::RerouteIntake`. The check passed before and after the commit. The orchestrator reported: ``` data_clump `root+tracker` 12 -> 0 data_clump `project+tracker` 0 -> 12 (new subject) data_clump total: 209 -> 209 name_collision `tracker` 0 -> 4 (new finding) ``` The data_clump total stayed at 209. The baseline run held 246, so `209 <= 246` passed. The code is `DeltaCheck#shape_row`: `"ok" => after <= before`, where `before` is the baseline run tally. `DeltaCheck#finding_row` checks only the target subject (`after <= limit`). The orchestrator wrote: "This is the second example after #1200, and the delta-check does not detect it." ## Reproduce ``` BASE=docs/crc/os-sdlc/collaborator-runs/20261020T020325Z plugins/os-sdlc/bin/collaborator-pass --target os-sdlc delta-check --baseline $BASE --ledger docs/crc/os-sdlc/ledger.tsv $BASE/recipes/project.yml ``` Run it on the tree at the #1202 commit. The check exits 0. ## Approach Compare against the tree before the change for each shape. Report a subject that is new in the after tally and shares a member name with the target subject as a moved clump. Fail the check when the target score leaves and an equal score appears on a subject with a shared name. Required work: 1. `DeltaCheck::Tally` exposes the subjects of a shape with their scores — check: present `def subjects` in `plugins/os-sdlc/lib/os_sdlc/collaborator_pass/delta_check/tally.rb` 2. `DeltaCheck#call` adds a `moved` row for each new subject in the same shape that shares a name with the target subject. `ok` is false when one exists. 3. The `delta-check` command accepts the pre-change measure, or measures the tree before the change, as the second tally. 4. `skills/collaborator-pass/SKILL.md` step 7 names the `moved` row. Rejected: A stricter shape-total limit alone — it does not see a clump that is replaced by a clump of the same score. ## Acceptance criteria 1. Given a finding `root+tracker` at 12 before and 0 after, and a new `project+tracker` at 12 after, when delta-check runs, then it exits 1 and names `project+tracker` as the moved subject. 2. Given a finding that drops to 0 and no new subject shares a name with it, when delta-check runs, then it exits 0. 3. Given the #1200 case (implementation+project 20 to 0, implementation+root 6 to 44), when delta-check runs, then it still exits 1. ## Owner sketch ```ruby class OsSdlc::CollaboratorPass::DeltaCheck def call end ``` ## Origin - Trigger: collaborator-pass round 2, ticket #1202 (also #1200) - Improvised this session: the orchestrator ran delta-check before and after the commit, kept the change, and listed the moved clump as a follow-up - Chain: moved clump passes ← `DeltaCheck#shape_row` compares to the baseline run total, which has headroom from earlier tickets ← DESIGN (plugins/os-sdlc/lib/os_sdlc/collaborator_pass/delta_check.rb) - Root candidate: none found - Where: plugins/os-sdlc/lib/os_sdlc/collaborator_pass/delta_check.rb - Session: 2e167079-f404-4139-9cfa-6c0c937ce9f0 - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-crc-os-sdlc-run-3/2e167079-f404-4139-9cfa-6c0c937ce9f0.jsonl VERDICT: CREATE

jared added the

plugin/os-sdlc

filed-by/agent

category/code-change

labels

2026-10-20 14:05:48 +00:00

jared commented

2026-10-20 18:22:08 +00:00

Author

Owner

Occurrence: a moved clump passed delta-check in #1202 (root+tracker 12 -> project+tracker 12; data_clump total 209 -> 209).
Seen: plugins/os-sdlc/lib/os_sdlc/collaborator_pass/delta_check.rb matches the finding by subject string and accepts total <= baseline.
Run: session 86f833b9 (run 3 retrospective, 2026-10-20).
Adds: the rule that separates moved from removed. An after-finding is the same clump moved when it shares at least one name and at least two sites with the before-finding and its score is at or above the old score; #1202 matches (tracker; Open, RerouteIntake; 12 >= 12) and #1199, #1201, #1203 do not. Also require the shape total delta to be below zero, not at or below.

Occurrence: a moved clump passed delta-check in #1202 (root+tracker 12 -> project+tracker 12; data_clump total 209 -> 209). Seen: plugins/os-sdlc/lib/os_sdlc/collaborator_pass/delta_check.rb matches the finding by subject string and accepts total <= baseline. Run: session 86f833b9 (run 3 retrospective, 2026-10-20). Adds: the rule that separates moved from removed. An after-finding is the same clump moved when it shares at least one name and at least two sites with the before-finding and its score is at or above the old score; #1202 matches (tracker; Open, RerouteIntake; 12 >= 12) and #1199, #1201, #1203 do not. Also require the shape total delta to be below zero, not at or below.