153 lines
3.8 KiB
Markdown
153 lines
3.8 KiB
Markdown
|
|
# Reversion Protection Pattern
|
||
|
|
|
||
|
|
Four-layer system to prevent behavioral regression in tools.
|
||
|
|
|
||
|
|
## Purpose
|
||
|
|
|
||
|
|
Protect critical behaviors from accidental changes. Make regressions structurally difficult.
|
||
|
|
|
||
|
|
## The Four Layers
|
||
|
|
|
||
|
|
### Layer 1: Invariants Declaration
|
||
|
|
|
||
|
|
Tools declare their behavioral invariants in `invariants.md`.
|
||
|
|
|
||
|
|
```markdown
|
||
|
|
# Invariants
|
||
|
|
|
||
|
|
Behaviors that must not change without explicit human approval.
|
||
|
|
|
||
|
|
## Critical Invariants
|
||
|
|
|
||
|
|
- Description always follows "When/Why, Not How" formula
|
||
|
|
- Trigger must be unambiguous (no escape hatches)
|
||
|
|
- SKILL.md must be routing-only (no prose)
|
||
|
|
|
||
|
|
## Structural Invariants
|
||
|
|
|
||
|
|
- Maximum file depth: 3 levels
|
||
|
|
- Maximum routing file length: 50 lines
|
||
|
|
- Required directories: workflows/, references/
|
||
|
|
```
|
||
|
|
|
||
|
|
**Implementation:**
|
||
|
|
- Create `invariants.md` in tool root
|
||
|
|
- List behaviors that define the tool's identity
|
||
|
|
- Distinguish critical (never change) from structural (rarely change)
|
||
|
|
|
||
|
|
### Layer 2: Golden Examples
|
||
|
|
|
||
|
|
3-5 canonical examples showing correct behavior.
|
||
|
|
|
||
|
|
```
|
||
|
|
examples/
|
||
|
|
golden/
|
||
|
|
example-1.md # Input + expected output
|
||
|
|
example-2.md
|
||
|
|
example-3.md
|
||
|
|
```
|
||
|
|
|
||
|
|
**Golden example format:**
|
||
|
|
```markdown
|
||
|
|
# Golden Example: [Name]
|
||
|
|
|
||
|
|
## Input
|
||
|
|
[The input or request]
|
||
|
|
|
||
|
|
## Expected Output
|
||
|
|
[What the tool should produce]
|
||
|
|
|
||
|
|
## Why This Matters
|
||
|
|
[What invariant this tests]
|
||
|
|
```
|
||
|
|
|
||
|
|
**Implementation:**
|
||
|
|
- Create `examples/golden/` directory
|
||
|
|
- Add 3-5 examples covering key invariants
|
||
|
|
- Each example tests a specific behavior
|
||
|
|
|
||
|
|
### Layer 3: Change Impact Analysis
|
||
|
|
|
||
|
|
Every audit must analyze impact on invariants and golden examples.
|
||
|
|
|
||
|
|
```markdown
|
||
|
|
## Change Impact Analysis
|
||
|
|
|
||
|
|
### Invariants Affected
|
||
|
|
- [ ] None
|
||
|
|
- [x] Critical: description formula (explain why)
|
||
|
|
- [ ] Structural: ...
|
||
|
|
|
||
|
|
### Golden Examples Affected
|
||
|
|
- [ ] None
|
||
|
|
- [x] example-2.md: output will change because...
|
||
|
|
|
||
|
|
### Risk Assessment
|
||
|
|
[LOW/MEDIUM/HIGH] - [explanation]
|
||
|
|
```
|
||
|
|
|
||
|
|
**Implementation:**
|
||
|
|
- Include change impact section in every audit
|
||
|
|
- Flag any changes affecting invariants or examples
|
||
|
|
- Require explicit acknowledgment for critical changes
|
||
|
|
|
||
|
|
### Layer 4: Human Review Gate
|
||
|
|
|
||
|
|
Changes affecting golden examples require human approval.
|
||
|
|
|
||
|
|
```
|
||
|
|
Golden example affected?
|
||
|
|
│
|
||
|
|
↓
|
||
|
|
Yes ──→ Present change to human
|
||
|
|
│ │
|
||
|
|
│ ↓
|
||
|
|
│ Approved? ──→ No ──→ Do not proceed
|
||
|
|
│ │
|
||
|
|
│ ↓
|
||
|
|
│ Yes
|
||
|
|
│ │
|
||
|
|
↓ ↓
|
||
|
|
No ──────→ Proceed with change
|
||
|
|
```
|
||
|
|
|
||
|
|
**Implementation:**
|
||
|
|
- Detect golden example changes during audit
|
||
|
|
- Present specific changes to user before applying
|
||
|
|
- Get explicit approval: "This will change how X works. Proceed?"
|
||
|
|
- Log approval in decisions record
|
||
|
|
|
||
|
|
## Implementation Checklist
|
||
|
|
|
||
|
|
- [ ] Create `invariants.md` listing behavioral invariants
|
||
|
|
- [ ] Create `examples/golden/` with 3-5 canonical examples
|
||
|
|
- [ ] Audit workflow includes change impact analysis
|
||
|
|
- [ ] Changes affecting golden examples flagged for review
|
||
|
|
- [ ] Human approval required for critical invariant changes
|
||
|
|
- [ ] Approvals logged in `.decisions/`
|
||
|
|
|
||
|
|
## Verification
|
||
|
|
|
||
|
|
Before completing any modification:
|
||
|
|
|
||
|
|
1. Run tool against all golden examples
|
||
|
|
2. Compare output to expected
|
||
|
|
3. If any mismatch: flag for review
|
||
|
|
4. If intentional change: update golden example + record decision
|
||
|
|
|
||
|
|
## Anti-patterns
|
||
|
|
|
||
|
|
**Invariants creep:** Adding too many invariants makes everything "critical." Keep to 3-7 true invariants.
|
||
|
|
|
||
|
|
**Stale examples:** Golden examples that no longer represent real usage. Review annually.
|
||
|
|
|
||
|
|
**Rubber-stamp reviews:** Auto-approving golden example changes. Each should be a real decision.
|
||
|
|
|
||
|
|
**Missing layer:** Implementing some layers but not all. The system works together.
|
||
|
|
|
||
|
|
## Cross-references
|
||
|
|
|
||
|
|
- [Audit Pattern](audit-pattern.md) - Audits must check reversion protection
|
||
|
|
- [Verification Pattern](verification-pattern.md) - Golden examples are verification
|
||
|
|
- [Decisions Record](decisions-record.md) - Approvals are recorded as decisions
|