CRC census --since: a run batches only changed and open files

jared commented

2026-10-23 19:43:07 +00:00

Owner

Problem

Each CRC run starts with an empty ledger. The census then puts every file into wave 1 and wave 2 batches. In CRC run 4, waves 1-2 cost about 2.0M subagent tokens over 905 files and produced 13 non-keep rows. Run 3 had already judged almost every one of those files, and most of them had not changed.

Reproduce

n/a: design gap. Census#census (plugins/os-sdlc/lib/os_sdlc/crc/census.rb) passes every ledger row to Batches.

Approach

Keep the ledger between runs. Add --since <sha> to bin/crc-census. With --since, a Census::RunScope selects the rows for this run, and only those rows go into the batch files. The scope is the union of three path sets: the files under the library root and the test support root that git diff --name-only <sha>..HEAD lists, the files of open detector rows (Crc::OpenRows#open_paths; blocked rows are not in it), and the files the ledger does not list yet. For each row in the scope, the census clears the cells job, concept, job_category, role, verdict, new_path, new_constant, new_test_path, and question, so waves 1 to 3 write them again. It keeps evidence, ruling, wave, and commit. A row out of the scope keeps every cell. The JSON report adds scope with the count of each path set. Without --since, the census works as it does today.

crc-refactor SKILL.md phase 1 changes: a run keeps the ledger of the run before it and passes --since with that run's merge commit. A full pass is a choice the run prompt names.

Depends on the "CRC open rows" ticket (Crc::OpenRows).

Required work:

Add the RunScope class — check: present class RunScope in plugins/os-sdlc/lib/os_sdlc/crc/census/run_scope.rb
bin/crc-census accepts --since — check: present --since in plugins/os-sdlc/lib/os_sdlc/crc/census/*.rb
crc-refactor SKILL.md phase 1 tells the run to keep the ledger and pass --since — check: present --since in plugins/os-sdlc/skills/crc-refactor/SKILL.md

Acceptance criteria

Given a ledger with rows for a.rb and b.rb and a commit after SHA that changes only a.rb, when crc-census --since SHA runs, then the batch files list a.rb and not b.rb.
Given the same ledger and an open holders row whose path is b.rb, when crc-census --since SHA runs, then the batch files list b.rb.
Given a new file c.rb that the ledger does not list, when crc-census --since SHA runs, then the ledger gets a row for c.rb and the batch files list it.
Given a.rb in the scope with filled job, verdict, ruling, and commit cells, when crc-census --since SHA runs, then its job and verdict cells are empty and its ruling and commit cells are unchanged.
Given b.rb out of the scope, when crc-census --since SHA runs, then every cell of its row is unchanged.
Given no --since, when crc-census runs on the same ledger, then the batch files list every ledger row, as today.
Given --since names a commit that does not exist, when crc-census runs, then it exits 1 with one stderr line that names the commit and changes no file.

Owner sketch

module OsSdlc
  module Crc
    class Census
      class RunScope
        def initialize(changed_paths:, open_paths:, new_paths:)
        def include?(path)
        def counts
      end
    end
  end
end

Census#census passes only views where scope.include?(view.path) to Batches; Settings carries the scope, or nil without --since.

Origin

Trigger: CRC run 4 close-out (docs/crc/os-sdlc/HANDOFF.md run stats: rename and move round 2,642k tokens, 13 non-keep rows)
Improvised this session: none
Chain: full re-census each run ← SKILL.md phase 1 archives the ledger and Census batches every row ← DESIGN (plugins/os-sdlc/lib/os_sdlc/crc/census.rb)
Root candidate: none found; searched crc-census, census, --since, RunScope, OpenRows, incremental, keep the ledger
Where: plugins/os-sdlc/lib/os_sdlc/crc/census.rb
Session: c1bedec6-7e01-45ab-b9db-22d63652c19f
Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-crc-os-sdlc-run-4/c1bedec6-7e01-45ab-b9db-22d63652c19f.jsonl

VERDICT: CREATE

## Problem Each CRC run starts with an empty ledger. The census then puts every file into wave 1 and wave 2 batches. In CRC run 4, waves 1-2 cost about 2.0M subagent tokens over 905 files and produced 13 non-keep rows. Run 3 had already judged almost every one of those files, and most of them had not changed. ## Reproduce n/a: design gap. `Census#census` (plugins/os-sdlc/lib/os_sdlc/crc/census.rb) passes every ledger row to `Batches`. ## Approach Keep the ledger between runs. Add `--since <sha>` to `bin/crc-census`. With `--since`, a `Census::RunScope` selects the rows for this run, and only those rows go into the batch files. The scope is the union of three path sets: the files under the library root and the test support root that `git diff --name-only <sha>..HEAD` lists, the files of open detector rows (`Crc::OpenRows#open_paths`; blocked rows are not in it), and the files the ledger does not list yet. For each row in the scope, the census clears the cells `job`, `concept`, `job_category`, `role`, `verdict`, `new_path`, `new_constant`, `new_test_path`, and `question`, so waves 1 to 3 write them again. It keeps `evidence`, `ruling`, `wave`, and `commit`. A row out of the scope keeps every cell. The JSON report adds `scope` with the count of each path set. Without `--since`, the census works as it does today. crc-refactor SKILL.md phase 1 changes: a run keeps the ledger of the run before it and passes `--since` with that run's merge commit. A full pass is a choice the run prompt names. Depends on the "CRC open rows" ticket (Crc::OpenRows). Required work: 1. Add the RunScope class — check: present `class RunScope` in `plugins/os-sdlc/lib/os_sdlc/crc/census/run_scope.rb` 2. bin/crc-census accepts `--since` — check: present `--since` in `plugins/os-sdlc/lib/os_sdlc/crc/census/*.rb` 3. crc-refactor SKILL.md phase 1 tells the run to keep the ledger and pass `--since` — check: present `--since` in `plugins/os-sdlc/skills/crc-refactor/SKILL.md` ## Acceptance criteria 1. Given a ledger with rows for `a.rb` and `b.rb` and a commit after SHA that changes only `a.rb`, when `crc-census --since SHA` runs, then the batch files list `a.rb` and not `b.rb`. 2. Given the same ledger and an open holders row whose `path` is `b.rb`, when `crc-census --since SHA` runs, then the batch files list `b.rb`. 3. Given a new file `c.rb` that the ledger does not list, when `crc-census --since SHA` runs, then the ledger gets a row for `c.rb` and the batch files list it. 4. Given `a.rb` in the scope with filled `job`, `verdict`, `ruling`, and `commit` cells, when `crc-census --since SHA` runs, then its `job` and `verdict` cells are empty and its `ruling` and `commit` cells are unchanged. 5. Given `b.rb` out of the scope, when `crc-census --since SHA` runs, then every cell of its row is unchanged. 6. Given no `--since`, when `crc-census` runs on the same ledger, then the batch files list every ledger row, as today. 7. Given `--since` names a commit that does not exist, when `crc-census` runs, then it exits 1 with one stderr line that names the commit and changes no file. ## Owner sketch ```ruby module OsSdlc module Crc class Census class RunScope def initialize(changed_paths:, open_paths:, new_paths:) def include?(path) def counts end end end end ``` `Census#census` passes only views where `scope.include?(view.path)` to `Batches`; `Settings` carries the scope, or nil without `--since`. ## Origin - Trigger: CRC run 4 close-out (docs/crc/os-sdlc/HANDOFF.md run stats: rename and move round 2,642k tokens, 13 non-keep rows) - Improvised this session: none - Chain: full re-census each run ← SKILL.md phase 1 archives the ledger and Census batches every row ← DESIGN (plugins/os-sdlc/lib/os_sdlc/crc/census.rb) - Root candidate: none found; searched crc-census, census, --since, RunScope, OpenRows, incremental, keep the ledger - Where: plugins/os-sdlc/lib/os_sdlc/crc/census.rb - Session: c1bedec6-7e01-45ab-b9db-22d63652c19f - Transcript: /home/jared/.claude/projects/-home-jared-dev-cc-os--claude-worktrees-crc-os-sdlc-run-4/c1bedec6-7e01-45ab-b9db-22d63652c19f.jsonl VERDICT: CREATE

jared added the

plugin/os-sdlc

filed-by/user

category/code-change

labels

2026-10-23 19:43:07 +00:00

jared referenced this issue

2026-10-23 19:43:26 +00:00

CRC open rows: the progress table counts open and blocked detector rows, not only raw rows #1279

jared commented

2026-10-23 19:43:26 +00:00

Author

Owner

Queued (session c1bedec6): starts after #1279 merges, on branch crc-os-sdlc-run-4.

jared commented

2026-10-23 20:06:50 +00:00

Author

Owner

Work started (session c1bedec6). Branch crc-os-sdlc-run-4 (persistent worktree). Dependency met: #1279 and #1281 merged to main; Crc::OpenRows#open_paths exists (52 files on docs/crc/os-sdlc).

jared commented

2026-10-23 20:14:23 +00:00

Author

Owner

Implementation 144 failed at the change-auditor (dispatch 1224, verdict error, 400 s). Cause: the evidence CLI crashed with ruby-lsp InvalidLocationError (Utf16Scanner#find_char_position via evidence/probe/lsp/lookup.rb:34) on a references lookup for OpenRows. The work list itself was complete. Deviation: not marked blocked; no code was written and the failure is a tool defect, so a fresh implementation-open follows now. Tool defect captured separately.

jared referenced this issue

2026-10-23 20:14:54 +00:00

Evidence CLI crashes with ruby-lsp InvalidLocationError on a references lookup #1284

jared commented

2026-10-23 20:18:39 +00:00

Author

Owner

Implementation 146 failed the same way (dispatch 1233, verdict error): the evidence CLI crashed twice with ruby-lsp InvalidLocationError. Blocked by #1284. #1280 resumes after #1284 merges.

jared commented

2026-10-23 20:26:05 +00:00

Author

Owner

Unblocked: the crash came from a stale ruby-lsp daemon (opened crc/open_rows.rb before #1281 changed it). The daemon was stopped; bin/evidence --root . OpenRows exits 0. The code fix is #1284. Re-opening the implementation now.

jared commented

2026-10-23 20:45:13 +00:00

Author

Owner

Resolution

Done: crc-census --since SHA batches only changed files, files of open detector rows (Crc::OpenRows), and unlisted files; it keeps the ledger and clears judgment cells of scoped rows only. Without --since, the full pass runs as before. crc-refactor SKILL.md phase 1 keeps the ledger between runs. Commit 5c285895, implementation 148.

Evidence: Pipeline implementation 148 complete: suite-check pass (suite 55 s), contract-auditor pass. Real-data check on a scratch copy of docs/crc/os-sdlc with --since ed03c3ea: exit 0, scope changed=11 open=52 new=4, 63 files in 2 batches (run 4 batched all 905).

Follow-ups: #1284 (evidence CLI LSP stale daemon) is next in this session; #1282 stays on the backlog.

## Resolution **Done:** crc-census --since SHA batches only changed files, files of open detector rows (Crc::OpenRows), and unlisted files; it keeps the ledger and clears judgment cells of scoped rows only. Without --since, the full pass runs as before. crc-refactor SKILL.md phase 1 keeps the ledger between runs. Commit 5c285895, implementation 148. **Evidence:** Pipeline implementation 148 complete: suite-check pass (suite 55 s), contract-auditor pass. Real-data check on a scratch copy of docs/crc/os-sdlc with --since ed03c3ea: exit 0, scope changed=11 open=52 new=4, 63 files in 2 batches (run 4 batched all 905). **Follow-ups:** #1284 (evidence CLI LSP stale daemon) is next in this session; #1282 stays on the backlog.

jared closed this issue

2026-10-23 20:45:13 +00:00

jared referenced this issue from a commit

2026-10-23 20:47:09 +00:00

feat(os-sdlc): Audited POODR boundaries for crc-census --since; RunScope, Ledger#restart, ScopedReport and Settings placement hold, R1-R3 present. (#1280)

Rows
Columns

CRC census --since: a run batches only changed and open files #1280

Problem

Reproduce

Approach

Acceptance criteria

Owner sketch

Origin

Resolution