85 lines
2.6 KiB
Markdown
85 lines
2.6 KiB
Markdown
|
# Audit Pattern
|
||
|
|
|
||
|
|
Smart implicit audits that run automatically when modifying tools or handling vague requests.
|
||
|
|
|
||
|
|
## Purpose
|
||
|
|
|
||
|
|
Prevent drift and degradation by auditing before changes, not just when explicitly requested.
|
||
|
|
|
||
|
|
## When to Use
|
||
|
|
|
||
|
|
**Implicit audit triggers:**
|
||
|
|
- User asks to "modify" or "update" an existing tool
|
||
|
|
- User gives vague request like "improve this skill"
|
||
|
|
- User asks to "add" something to an existing tool
|
||
|
|
- Any change to a tool that has golden examples
|
||
|
|
|
||
|
|
**Explicit audit triggers:**
|
||
|
|
- User asks to "audit" or "review" a tool
|
||
|
|
- User pulls a tool from marketplace and wants evaluation
|
||
|
|
|
||
|
|
**Skip audit when:**
|
||
|
|
- Creating a brand new tool (nothing to audit yet)
|
||
|
|
- User gives specific, unambiguous instructions
|
||
|
|
- Quick fixes to obvious bugs (typos, syntax errors)
|
||
|
|
|
||
|
|
## Implementation Checklist
|
||
|
|
|
||
|
|
Tools implementing this pattern must:
|
||
|
|
|
||
|
|
- [ ] Define audit scope in workflow (what gets checked)
|
||
|
|
- [ ] Check for prior decisions in `.decisions/` before auditing
|
||
|
|
- [ ] Run audit as subagent(s) to keep main thread light
|
||
|
|
- [ ] Write findings to scratch workspace (gitignored)
|
||
|
|
- [ ] Classify findings by severity
|
||
|
|
- [ ] Check golden examples for behavioral impact
|
||
|
|
- [ ] Record new decisions made during audit
|
||
|
|
- [ ] Clean up scratch files after user acts on findings
|
||
|
|
|
||
|
|
## Workflow Structure
|
||
|
|
|
||
|
|
```
|
||
|
|
1. Detect implicit trigger
|
||
|
|
└─→ "modify" / "update" / "add to" / vague request
|
||
|
|
|
||
|
|
2. Load prior context
|
||
|
|
└─→ Read .decisions/ for this tool
|
||
|
|
└─→ Read invariants.md if exists
|
||
|
|
└─→ Read golden examples
|
||
|
|
|
||
|
|
3. Run audit subtasks (parallel where possible)
|
||
|
|
└─→ Structure audit (haiku)
|
||
|
|
└─→ Content audit (opus)
|
||
|
|
└─→ Quality audit (opus)
|
||
|
|
└─→ Change impact analysis
|
||
|
|
|
||
|
|
4. Assemble findings
|
||
|
|
└─→ Classify severity
|
||
|
|
└─→ Flag golden example impacts
|
||
|
|
|
||
|
|
5. Present to user
|
||
|
|
└─→ Summary with verdict
|
||
|
|
└─→ What will change and why
|
||
|
|
|
||
|
|
6. Record decisions
|
||
|
|
└─→ Write to .decisions/
|
||
|
|
```
|
||
|
|
|
||
|
|
## Example Reference
|
||
|
|
|
||
|
|
See `${CLAUDE_PLUGIN_ROOT}/workflows/audit-skill.md` for a complete audit workflow implementation.
|
||
|
|
|
||
|
|
## Anti-patterns
|
||
|
|
|
||
|
|
**Audit theater:** Running audits but ignoring findings. Every significant finding must be addressed or explicitly deferred.
|
||
|
|
|
||
|
|
**Audit fatigue:** Auditing every tiny change. Reserve for modifications that could affect behavior.
|
||
|
|
|
||
|
|
**Silent audits:** Running audits without telling the user. Always surface findings.
|
||
|
|
|
||
|
|
## Cross-references
|
||
|
|
|
||
|
|
- [Verification Pattern](verification-pattern.md) - Audits feed into verification
|
||
|
|
- [Decisions Record](decisions-record.md) - Audits must record decisions
|
||
|
|
- [Reversion Protection](reversion-protection.md) - Audits check golden examples
|