SecondBrain/reference/ai-workflow-audit-findings-...

3.8 KiB
Raw Blame History

summary tags scope type source date last_updated
Cross-project audit findings on how Jared works with Claude Code — working-style profile, repeated instructions worth persisting, and deterministic-tool opportunities (audited 2026-07-09, 45 days of transcripts).
type/reference
domain/ai-workflow
tool/claude-code
global reference transcript audit 2026-07-09 2026-07-09 2026-07-09

AI Workflow Audit Findings (Jared × Claude Code)

Full report: ~/Desktop/ai-workflow-audit-2026-07-09.md. Method: audit-ai-session-transcripts. Corpus: 3,058 extracted messages / 45 days / 15+ projects; ~600-700 genuinely human-authored across 157 interactive sessions after filtering Claude-authored subagent-dispatch prompts (50-70% of raw rows).

Working-style profile

Orchestrator-architect: makes every architectural/business call personally, delegates nearly all execution to tightly scoped subagents. Median interactive message 413 chars; 20-27% of direct messages are terse approvals ("proceed", "commit this") — the propose→approve loop is the default and most effective mode. High quality bar on maintainability and client-facing correctness; low trust in single-agent claims — personal verification built into nearly every mutating task, and it repeatedly caught real agent errors. Risk-escalates agent scope (read-only recon → guarded writes → verify). Plans extensively for architectural work, dives in tactically. Ritualizes session close. Designs around ADHD: pull-based dashboards over push digests. Expects and accepts pushback. Prefers Ruby > Python > bash for scripts; Sandi Metz OOP; TDD; lowercase-kebab filenames.

Where effort is wasted (coach findings)

  1. Kickoff re-briefing: house conventions (TDD, Sandi Metz, vertical slices, delegation defaults, "audit defaults before customizing") restated at nearly every project/gem kickoff — should be one vault pointer.
  2. Safety preambles retyped per dispatch: "read-only, do NOT change anything" appeared near-verbatim 15+ times; "never echo the secret", "release.sh needs explicit authorization" similar — template candidates.
  3. Session-close ritual typed 20+ times (commit + docs + todos + handoff/vault) — top candidate for a /wrap skill or SessionEnd hook.
  4. Gotchas re-derived on second encounter (wp-cli quoting, SSH-key recovery, Docker PATH) — rule: any environment surprise costing >3 turns gets a runbook/vault note before session end.
  5. Deterministic tools built at step six, not step one (release.sh after ~6h of manual loops; verify script only after an incident).
  6. Client-side double-submit bug: ~8% duplicate messages in one batch.

Repeated-instruction persistence candidates (by frequency)

  • Session-close ritual (20+) → /wrap skill / SessionEnd hook
  • Read-only/guarded-write safety preamble (15+) → dispatch template or CLAUDE.md rule
  • "Dispatch subagents to..." (12+) → global orchestration default (partly exists)
  • "Commit this / commit and push" (9+) → wrap into session-close
  • TDD/Sandi Metz/gem-stack conventions (per kickoff) → conventions note + gem scaffold generator
  • Plan-before-implement gate; staging-first deploys; destructive-op pre-flight → hooks/rules, not prose

Deterministic-tool opportunities

Task-checkbox flipping (~10+ LLM dispatches for pure mechanics); Proxmox VM lifecycle script (5× re-dispatched); SSH-hardening drop-in playbook; Docker image lift-and-shift (3×); dns-preflight; ACF write wrapper; doc-citation drift check in CI; use installed os-doc-hygiene instead of ad-hoc staleness sweeps (re-invented 5+ times).

Keep doing

Propose→approve with numbered options; pointing at files/URLs instead of pasting; phased risk escalation with personal verification; empirical dispute resolution (frozen-grid evals); scope self-correction before code exists.