os-backlog: ticket-skeptic gauntlet — verdict agent gating AI follow-up capture and ticket-decompose #407

Closed
opened 2026-08-15 17:22:50 +00:00 by jared · 2 comments
Owner

Approved by user 2026-08-15 (option B). Origin: #405 was AI-created from an implementer's limited view; half of it was a non-bug caused by an incomplete process run (#403 never went through the pipeline). Standing pain: AI-corrected tickets spawn 2-3 follow-ups each, so slices never complete.

Amended 2026-08-15: NEEDS-HUMAN verdict removed at user's direction — the goal is removing the human from initial implementation; ambiguous depth-limited follow-ups DROP rather than queue for review.

Design: a read-only verdict agent, plugins/os-backlog/agents/ticket-skeptic.md, modeled on the charter-judge / os-sdlc:reviewer pattern (reads, judges, emits structured verdict, never edits). Its gauntlet is derived from the ponytail skill's lazy-programmer ladder (github.com/DietrichGebert/ponytail):

  1. Complete-process test: did the symptom occur in a COMPLETE, in-policy process run? Partial/hand-orchestrated run -> default DROP, one-line note on the parent instead. (provenance: #405/#403)
  2. Need-to-exist test (ladder rung 1): what happens if nobody ever does this? Acceptable -> DROP. Speculative need -> DROP.
  3. Better-seam test: is there an existing mechanism/helper/convention that already covers it, or a one-line fix upstream instead of a new ticket?
  4. Speculative-hardening test: provenance = existing pain, observed failure, recorded decision, or stated requirement — else DROP.
  5. Hard floors (never DROPped): explicitly user-requested work; security/data-loss/validation issues. -> CREATE.

Verdicts: CREATE | DROP (one-line reason, recorded on the parent). No human-review verdict exists — the gauntlet decides autonomously.

Stopping rules (anti-loop, mandatory): the gauntlet runs at most once per ticket (verdict recorded in the ticket body is the marker; never re-gauntlet a survivor); the skeptic agent never creates tickets itself; a follow-up whose parent is itself an AI-created follow-up defaults to DROP unless it hits a hard floor (depth-1 limit); "lean already, ship" — if nothing to cut, approve and stop.

Tasks

  • Create plugins/os-backlog/agents/ticket-skeptic.md implementing the gauntlet + verdict format above (read-only tools; single job; never edits, never creates issues).

  • skills/capture/SKILL.md: insert a gate between scope-label selection and issue-create — fires ONLY for AI-initiated follow-up tickets not explicitly user-approved this session; dispatch ticket-skeptic; on DROP, post the one-line reason on the parent ticket instead of creating.

  • skills/ticket-decompose/SKILL.md: add a skeptic gate between Phase Decide and Phase Decompose — dispatch ticket-skeptic on the ticket itself (all tickets, including user-created); a DROP verdict on a user-created ticket is surfaced as a decision memo, never acted on silently.

  • Update the close-protocol follow-ups prose (the os-backlog session-start rules source — find it under plugins/os-backlog, e.g. hooks/ or prompts/) so "--follow-ups" states: AI follow-ups must carry a ticket-skeptic verdict or be explicitly "none".

  • Riders: record the convention as an ADR via /os-adr:create (gauntlet + depth-1 stop rule); one History row in docs/implementation-status/os-backlog.md; run bin/refresh-plugins.

Acceptance criteria

  • ticket-skeptic.md exists, follows the repo's verdict-agent pattern, and contains all five tests, two verdicts, and all stopping rules verbatim-equivalent.

  • capture's gate triggers only for unapproved AI follow-ups; user-named tickets flow through untouched.

  • ticket-decompose runs the gauntlet exactly once per ticket and records the verdict in the ticket body.

  • Depth-1 rule present in both skills.

  • ADR recorded; status leaf row appended; plugin cache refreshed.

Out of scope

Ruby changes to bin/os-backlog or lib/ (enforcement stays instruction-level, matching the existing close-protocol prose); ponytail's code-review ladder rungs (stdlib/native/dependency) — code-level laziness belongs to code review, not ticket triage; retrofitting verdicts onto existing open tickets.

Blocking edges

Agent definition (task 1) blocks the two skill edits (tasks 2, 3); tasks 2 and 3 are parallel after 1; close-protocol prose (task 4) and riders (task 5) last.

Approved by user 2026-08-15 (option B). Origin: #405 was AI-created from an implementer's limited view; half of it was a non-bug caused by an incomplete process run (#403 never went through the pipeline). Standing pain: AI-corrected tickets spawn 2-3 follow-ups each, so slices never complete. Amended 2026-08-15: NEEDS-HUMAN verdict removed at user's direction — the goal is removing the human from initial implementation; ambiguous depth-limited follow-ups DROP rather than queue for review. Design: a read-only verdict agent, plugins/os-backlog/agents/ticket-skeptic.md, modeled on the charter-judge / os-sdlc:reviewer pattern (reads, judges, emits structured verdict, never edits). Its gauntlet is derived from the ponytail skill's lazy-programmer ladder (github.com/DietrichGebert/ponytail): 1. Complete-process test: did the symptom occur in a COMPLETE, in-policy process run? Partial/hand-orchestrated run -> default DROP, one-line note on the parent instead. (provenance: #405/#403) 2. Need-to-exist test (ladder rung 1): what happens if nobody ever does this? Acceptable -> DROP. Speculative need -> DROP. 3. Better-seam test: is there an existing mechanism/helper/convention that already covers it, or a one-line fix upstream instead of a new ticket? 4. Speculative-hardening test: provenance = existing pain, observed failure, recorded decision, or stated requirement — else DROP. 5. Hard floors (never DROPped): explicitly user-requested work; security/data-loss/validation issues. -> CREATE. Verdicts: CREATE | DROP (one-line reason, recorded on the parent). No human-review verdict exists — the gauntlet decides autonomously. Stopping rules (anti-loop, mandatory): the gauntlet runs at most once per ticket (verdict recorded in the ticket body is the marker; never re-gauntlet a survivor); the skeptic agent never creates tickets itself; a follow-up whose parent is itself an AI-created follow-up defaults to DROP unless it hits a hard floor (depth-1 limit); "lean already, ship" — if nothing to cut, approve and stop. ## Tasks - [ ] Create plugins/os-backlog/agents/ticket-skeptic.md implementing the gauntlet + verdict format above (read-only tools; single job; never edits, never creates issues). - [ ] skills/capture/SKILL.md: insert a gate between scope-label selection and issue-create — fires ONLY for AI-initiated follow-up tickets not explicitly user-approved this session; dispatch ticket-skeptic; on DROP, post the one-line reason on the parent ticket instead of creating. - [ ] skills/ticket-decompose/SKILL.md: add a skeptic gate between Phase Decide and Phase Decompose — dispatch ticket-skeptic on the ticket itself (all tickets, including user-created); a DROP verdict on a user-created ticket is surfaced as a decision memo, never acted on silently. - [ ] Update the close-protocol follow-ups prose (the os-backlog session-start rules source — find it under plugins/os-backlog, e.g. hooks/ or prompts/) so "--follow-ups" states: AI follow-ups must carry a ticket-skeptic verdict or be explicitly "none". - [ ] Riders: record the convention as an ADR via /os-adr:create (gauntlet + depth-1 stop rule); one History row in docs/implementation-status/os-backlog.md; run bin/refresh-plugins. ## Acceptance criteria - [ ] ticket-skeptic.md exists, follows the repo's verdict-agent pattern, and contains all five tests, two verdicts, and all stopping rules verbatim-equivalent. - [ ] capture's gate triggers only for unapproved AI follow-ups; user-named tickets flow through untouched. - [ ] ticket-decompose runs the gauntlet exactly once per ticket and records the verdict in the ticket body. - [ ] Depth-1 rule present in both skills. - [ ] ADR recorded; status leaf row appended; plugin cache refreshed. ## Out of scope Ruby changes to bin/os-backlog or lib/ (enforcement stays instruction-level, matching the existing close-protocol prose); ponytail's code-review ladder rungs (stdlib/native/dependency) — code-level laziness belongs to code review, not ticket triage; retrofitting verdicts onto existing open tickets. ## Blocking edges Agent definition (task 1) blocks the two skill edits (tasks 2, 3); tasks 2 and 3 are parallel after 1; close-protocol prose (task 4) and riders (task 5) last.
Author
Owner

Work started in-session (main branch). Implementing ticket-skeptic agent + capture/ticket-decompose gates per spec.

Work started in-session (main branch). Implementing ticket-skeptic agent + capture/ticket-decompose gates per spec.
Author
Owner

Resolution

Done: Ticket-skeptic gauntlet shipped: read-only verdict agent plugins/os-backlog/agents/ticket-skeptic.md (five tests, CREATE|DROP, all stopping rules incl. depth-1); skeptic gate in capture (fires only for unapproved AI follow-ups, DROP posts reason on parent) and in ticket-decompose (all tickets, between Decide and Decompose, once per ticket via VERDICT: body marker, user-ticket DROP surfaced as decision memo); close-protocol follow-ups prose updated in hooks/session_start.py; ADR-0120 recorded; History row appended to docs/implementation-status/os-backlog.md; bin/refresh-plugins run.

Evidence: Commit 75d54e2 on main; os-backlog suite green (185 runs, 439 assertions, 0 failures); session_start.py ast-parses; /code-review low returned no findings on this diff.

Follow-ups: none — implementer-suggested items (capture allowed-tools/Agent dispatch, hardening-audit overlap with test 4, eval scenario for the gate) fail the gauntlet's need-to-exist/speculative-hardening tests and are dropped; two unrelated code-review findings on pre-existing commits surfaced to the user in-session for their decision.

## Resolution **Done:** Ticket-skeptic gauntlet shipped: read-only verdict agent plugins/os-backlog/agents/ticket-skeptic.md (five tests, CREATE|DROP, all stopping rules incl. depth-1); skeptic gate in capture (fires only for unapproved AI follow-ups, DROP posts reason on parent) and in ticket-decompose (all tickets, between Decide and Decompose, once per ticket via VERDICT: body marker, user-ticket DROP surfaced as decision memo); close-protocol follow-ups prose updated in hooks/session_start.py; ADR-0120 recorded; History row appended to docs/implementation-status/os-backlog.md; bin/refresh-plugins run. **Evidence:** Commit 75d54e2 on main; os-backlog suite green (185 runs, 439 assertions, 0 failures); session_start.py ast-parses; /code-review low returned no findings on this diff. **Follow-ups:** none — implementer-suggested items (capture allowed-tools/Agent dispatch, hardening-audit overlap with test 4, eval scenario for the gate) fail the gauntlet's need-to-exist/speculative-hardening tests and are dropped; two unrelated code-review findings on pre-existing commits surfaced to the user in-session for their decision.
jared closed this issue 2026-08-15 19:48:20 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
jared/cc-os#407
No description provided.