Build bin/evidence: deterministic code-probe fingerprint CLI (ADR-0142) #483

New issue

Closed

opened 2026-08-22 18:09:57 +00:00 by jared · 2 comments

jared commented

2026-08-22 18:09:57 +00:00

Owner

Copy link

Context

User-approved 2026-08-22, designed in-session from the #482 defect. ADR-0142 records the decision: code-probe steps 2-3 (verify definitions, trace call sites) move from prose-prescribed ast-grep queries into a plugin-owned CLI so patterns are frozen in tested code and one tool call replaces N model-composed queries.

Tasks

TDD (red-green-refactor, minitest, Sandi Metz style) OsSdlc::Evidence::Resolver: mixed class/file varargs → deduplicated (class,path) units; multi-class files emit one unit per class; unresolvable tokens returned as phantoms, never raised.
TDD OsSdlc::Evidence::Probe (+ concrete probes): memoized command/result provenance; bare $RECV.NAME sender pattern primary; grep escalation fired only on the two triggers (0-hit known use; dynamic :NAME/send hunt), hits tagged unclassified.
TDD OsSdlc::Evidence::Fingerprint: def enumeration with public/private split, per-method sender rows.
TDD OsSdlc::Evidence::Report: to_brief renders EVIDENCE:/UNKNOWN:/phantom lines with path:line; explained renders same memoized data plus command+raw output (no re-run).
bin/evidence Thor CLI (ADR-0123): --root, --explain, varargs tokens.
Zeitwerk conformance for lib/os_sdlc/evidence/ (ADR-0090); rubocop clean.
Rewrite code-probe SKILL.md steps 2-3 to invoke the CLI; keep judgment steps (classify flagged rows, UNKNOWN, duck-type rule); keep the #482 trigger-scoped grep rule as the CLI's internal behavior description.
Widen agents/code-probe.md Bash allowance: ast-grep and bin/evidence only.
bin/refresh-plugins; one History row appended at the bottom of docs/implementation-status/os-sdlc.md.

Acceptance criteria

bin/evidence WriteGuard emits an EVIDENCE line naming plugins/os-sdlc/hooks/pre_tool_use.rb:24 as a deny? sender.
bin/evidence write_guard.rb WriteGuard produces one unit (dedupe on resolved pair).
bin/evidence NoSuchClass exits 0 with a phantom finding line.
--explain output includes the exact ast-grep command per line, from the same run.
Full os-sdlc test suite green; rubocop clean on new files.

Origin

Trigger: teaching session 2026-08-22 walking code-probe steps 2.2-2.3 by hand on #474; cost analysis showed the steps are deterministic given a symbol.
Improvised this session: none (design only; #482 fixed the prose patterns separately).
Chain: per-run model-composed queries ← evidence steps prescribed as prose ← DESIGN (code-probe SKILL.md under ADR-0118; superseded for steps 2-3 by ADR-0142).
Root candidate: this ticket.

## Context User-approved 2026-08-22, designed in-session from the #482 defect. ADR-0142 records the decision: code-probe steps 2-3 (verify definitions, trace call sites) move from prose-prescribed ast-grep queries into a plugin-owned CLI so patterns are frozen in tested code and one tool call replaces N model-composed queries. ## Tasks - [ ] TDD (red-green-refactor, minitest, Sandi Metz style) OsSdlc::Evidence::Resolver: mixed class/file varargs → deduplicated (class,path) units; multi-class files emit one unit per class; unresolvable tokens returned as phantoms, never raised. - [ ] TDD OsSdlc::Evidence::Probe (+ concrete probes): memoized command/result provenance; bare $RECV.NAME sender pattern primary; grep escalation fired only on the two triggers (0-hit known use; dynamic :NAME/send hunt), hits tagged unclassified. - [ ] TDD OsSdlc::Evidence::Fingerprint: def enumeration with public/private split, per-method sender rows. - [ ] TDD OsSdlc::Evidence::Report: to_brief renders EVIDENCE:/UNKNOWN:/phantom lines with path:line; explained renders same memoized data plus command+raw output (no re-run). - [ ] bin/evidence Thor CLI (ADR-0123): --root, --explain, varargs tokens. - [ ] Zeitwerk conformance for lib/os_sdlc/evidence/ (ADR-0090); rubocop clean. - [ ] Rewrite code-probe SKILL.md steps 2-3 to invoke the CLI; keep judgment steps (classify flagged rows, UNKNOWN, duck-type rule); keep the #482 trigger-scoped grep rule as the CLI's internal behavior description. - [ ] Widen agents/code-probe.md Bash allowance: ast-grep and bin/evidence only. - [ ] bin/refresh-plugins; one History row appended at the bottom of docs/implementation-status/os-sdlc.md. ## Acceptance criteria - [ ] bin/evidence WriteGuard emits an EVIDENCE line naming plugins/os-sdlc/hooks/pre_tool_use.rb:24 as a deny? sender. - [ ] bin/evidence write_guard.rb WriteGuard produces one unit (dedupe on resolved pair). - [ ] bin/evidence NoSuchClass exits 0 with a phantom finding line. - [ ] --explain output includes the exact ast-grep command per line, from the same run. - [ ] Full os-sdlc test suite green; rubocop clean on new files. ## Origin - Trigger: teaching session 2026-08-22 walking code-probe steps 2.2-2.3 by hand on #474; cost analysis showed the steps are deterministic given a symbol. - Improvised this session: none (design only; #482 fixed the prose patterns separately). - Chain: per-run model-composed queries ← evidence steps prescribed as prose ← DESIGN (code-probe SKILL.md under ADR-0118; superseded for steps 2-3 by ADR-0142). - Root candidate: this ticket.

jared added the

plugin/os-sdlc

label

2026-08-22 18:09:57 +00:00

jared commented

2026-08-22 18:10:10 +00:00

Author

Owner

Copy link

Work started in-session 2026-08-22: TDD build dispatched to a worker agent per ADR-0142 design (Resolver → Probes → Fingerprint → Report → Thor CLI → skill/agent doc edits). No branch; working tree of main.

jared commented

2026-08-22 18:52:36 +00:00

Author

Owner

Copy link

Resolution

Done: bin/evidence built per ADR-0142: Thor CLI + OsSdlc::Evidence::{Resolver,Probe,Fingerprint,Report}; mixed-token varargs with (class,path) dedupe and phantom-as-finding; bare $RECV.NAME sender probe primary; dynamic-send escalation gated to the zero-hit trigger with NEVER_SWEPT guard and grep -P constrained to :name/name:/send(:name) shapes; UNKNOWN lines capped at 3 sites; --explain re-renders memoized probe provenance; code-probe SKILL.md/cheatsheet/agent Bash allowance rewired; plugin cache refreshed; History row added

Evidence: Commit f1fb658 on main (19 files, +1006/-43). All four acceptance criteria verified unfiltered by the orchestrator: WriteGuard brief is 10 lines incl. EVIDENCE deny? sender pre_tool_use.rb:24 and 4 UNKNOWNs correctly flagging VIOLATION_CHECKS hash-shorthand dynamic dispatch at write_guard.rb:59-62; write_guard.rb+WriteGuard dedupe to 1 unit; NoSuchClass exits 0 with phantom line; --explain shows exact ast-grep commands from the same run. Suite 1052 runs/2694 assertions/0 failures; rubocop clean on all touched files. One red-gate round mid-build: initial escalation noise-bombed 78KB/class; fixed with regression test capping the real-repo brief at ≤30 lines.

Follow-ups: Three surfaced by the builder, all dropped as speculative (no observed failure): surface captured $RECV collaborators in the brief; normalize trailing-slash root in phantom lines; empty-.rb-file resolver test. One disclosed limitation, deliberate per design: implicit-self-only senders (deny?→write?) emit no line — classification stays agent judgment; revisit only if it bites on a real ticket.

## Resolution **Done:** bin/evidence built per ADR-0142: Thor CLI + OsSdlc::Evidence::{Resolver,Probe,Fingerprint,Report}; mixed-token varargs with (class,path) dedupe and phantom-as-finding; bare $RECV.NAME sender probe primary; dynamic-send escalation gated to the zero-hit trigger with NEVER_SWEPT guard and grep -P constrained to :name/name:/send(:name) shapes; UNKNOWN lines capped at 3 sites; --explain re-renders memoized probe provenance; code-probe SKILL.md/cheatsheet/agent Bash allowance rewired; plugin cache refreshed; History row added **Evidence:** Commit f1fb658 on main (19 files, +1006/-43). All four acceptance criteria verified unfiltered by the orchestrator: WriteGuard brief is 10 lines incl. EVIDENCE deny? sender pre_tool_use.rb:24 and 4 UNKNOWNs correctly flagging VIOLATION_CHECKS hash-shorthand dynamic dispatch at write_guard.rb:59-62; write_guard.rb+WriteGuard dedupe to 1 unit; NoSuchClass exits 0 with phantom line; --explain shows exact ast-grep commands from the same run. Suite 1052 runs/2694 assertions/0 failures; rubocop clean on all touched files. One red-gate round mid-build: initial escalation noise-bombed 78KB/class; fixed with regression test capping the real-repo brief at ≤30 lines. **Follow-ups:** Three surfaced by the builder, all dropped as speculative (no observed failure): surface captured $RECV collaborators in the brief; normalize trailing-slash root in phantom lines; empty-.rb-file resolver test. One disclosed limitation, deliberate per design: implicit-self-only senders (deny?→write?) emit no line — classification stays agent judgment; revisit only if it bites on a real ticket.

jared closed this issue

2026-08-22 18:52:36 +00:00

jared referenced this issue from a commit

2026-08-23 12:44:58 +00:00

feat(os-sdlc): deterministic evidence CLI for code-probe

No Branch/Tag specified

main

No results found.

Labels

Clear labels

Priority 0 — drop everything

Ticket creates something new

delete

Ticket removes something

enhancement

New feature or improvement

filed-by/agent

filed-by/user

frozen

Pre-reset backlog freeze 2026-08-16; do not work unless rediscovered by a live run

lint-rule

Ticket proposes/modifies a lint rule or cop

needs-info

Waiting on reporter for more information

needs-triage

Maintainer needs to evaluate this issue

Human-curated next up — AI never applies or removes

plugin/cc-architect

Scope: the cc-architect plugin

plugin/os

Scope: the os plugin

plugin/os-adr

Scope: the os-adr plugin

plugin/os-aidd-lint

Scope: the os-aidd-lint plugin

plugin/os-backlog

Scope: the os-backlog plugin

plugin/os-context

Scope: the os-context plugin

plugin/os-doc-hygiene

Scope: the os-doc-hygiene plugin

plugin/os-sdlc

Scope: the os-sdlc plugin

plugin/os-vault

Scope: the os-vault plugin

project/cc-os

Parent-scope: cc-os repo-wide, not specific to one plugin

ready-for-agent

Triage complete; an agent can pick this up

ready-for-human

Human-owned; never picked up autonomously

recurring

Recurring flavor: occurrence outcomes commented, never closed by AI

review

Shipped semi work awaiting human sign-off

update

Ticket updates something existing

waiting

Blocked; blocker named in a comment

wayfinder map child: task

wontfix

Will not be actioned

worklist/deviations

Deviation incidents from skill/pipeline runs — evidence pile for correcting the owning artifact

worklist/lint-rule

Proposed os-sdlc or os-aidd-lint rule

worklist/new-implement-build

Updated implementation harness (ast-grep + stepper)

No labels

plugin/os-doc-hygiene

worklist/new-implement-build

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

jared/cc-os#483

Reference in a new issue

Repository

jared/cc-os

Title

Body

No description provided.

Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?

Rows
Columns