257 lines
9.1 KiB
Markdown
257 lines
9.1 KiB
Markdown
|
|
# Deterministic Scripting Pattern
|
||
|
|
|
||
|
|
## Purpose
|
||
|
|
|
||
|
|
Shift mechanical, repeatable work from AI context to deterministic scripts. Scripts improve consistency, reduce context bloat, and let AI focus on judgment-based work.
|
||
|
|
|
||
|
|
This pattern applies during skill design and planning phases. The AI should recognize scripting opportunities and recommend them, rather than asking the user unprompted.
|
||
|
|
|
||
|
|
## Core Principle
|
||
|
|
|
||
|
|
When the AI encounters a task component during planning, it should ask:
|
||
|
|
|
||
|
|
> "Is this mechanical, repeatable, and unambiguous? If yes, a script will do this more reliably than I can."
|
||
|
|
|
||
|
|
Scripts are not replacements for AI judgment—they're complements. Scripts handle the deterministic foundation; AI handles interpretation and decisions built on that foundation.
|
||
|
|
|
||
|
|
## Quick Heuristics
|
||
|
|
|
||
|
|
Self-check during planning:
|
||
|
|
|
||
|
|
1. **Binary correctness?** - Is the output valid/invalid rather than good/better/best?
|
||
|
|
2. **Identical execution?** - Would I do this the same way every time?
|
||
|
|
3. **Structured data?** - Does it involve parsing, transforming, or validating structured formats?
|
||
|
|
4. **No interpretation?** - Can correctness be determined without judgment?
|
||
|
|
|
||
|
|
If 3+ answers are "yes" → strong script candidate.
|
||
|
|
|
||
|
|
## Examples
|
||
|
|
|
||
|
|
| Task | Script? | Reasoning |
|
||
|
|
|------|---------|-----------|
|
||
|
|
| Validate YAML frontmatter | Yes | Binary correctness, structured data |
|
||
|
|
| Scaffold directory structure | Yes | Deterministic file operations |
|
||
|
|
| Parse and normalize field values | Yes | Mechanical transformation |
|
||
|
|
| Check description length limits | Yes | Quantitative constraint |
|
||
|
|
| Verify naming conventions | Yes | Pattern matching, no judgment |
|
||
|
|
| Generate boilerplate from template | Yes | Deterministic substitution |
|
||
|
|
| Evaluate description quality | No | Qualitative judgment required |
|
||
|
|
| Choose between design approaches | No | Trade-off analysis |
|
||
|
|
| Assess code readability | No | Subjective evaluation |
|
||
|
|
| Decide what to include in a skill | No | Requires understanding intent |
|
||
|
|
|
||
|
|
### Hybrid Tasks
|
||
|
|
|
||
|
|
Some tasks appear qualitative but have quantitative components scripts can handle:
|
||
|
|
|
||
|
|
| Task | Script Portion | AI Portion |
|
||
|
|
|------|----------------|------------|
|
||
|
|
| Review description | Length limits, forbidden characters, required fields | Clarity, specificity, tone |
|
||
|
|
| Audit skill structure | File existence, naming conventions, size limits | Content quality, completeness |
|
||
|
|
| Validate workflow | Required sections present, link validity | Logical flow, clarity |
|
||
|
|
|
||
|
|
**Principle:** Extract quantitative guardrails into scripts. Let AI focus on the qualitative judgment that remains.
|
||
|
|
|
||
|
|
## Language Selection
|
||
|
|
|
||
|
|
Choose language based on project context and tool fit, not personal preference.
|
||
|
|
|
||
|
|
### Principles
|
||
|
|
|
||
|
|
1. **Match project context** - A Ruby script in a Rails project integrates naturally. A Python script in a Rails project adds cognitive overhead.
|
||
|
|
|
||
|
|
2. **Use best-in-class for domains** - Some tools have clear language winners. Playwright is best supported in Python. Data science tasks favor Python. Shell automation favors Bash.
|
||
|
|
|
||
|
|
3. **Prefer OOP-capable languages** - Ruby, Python, JavaScript/TypeScript support clean object-oriented design. Bash does not—use it only for simple orchestration or when it's genuinely the best fit.
|
||
|
|
|
||
|
|
4. **Consider maintenance** - Who will maintain this script? If the project team knows Ruby, write Ruby. If you're building a general-purpose skill, Python has broader reach.
|
||
|
|
|
||
|
|
### Examples
|
||
|
|
|
||
|
|
| Project Context | Recommended | Reasoning |
|
||
|
|
|-----------------|-------------|-----------|
|
||
|
|
| Rails application | Ruby | Matches project, team knows it |
|
||
|
|
| Ruby gem | Ruby | Same ecosystem, natural fit |
|
||
|
|
| General-purpose skill | Python | Broad reach, well-supported |
|
||
|
|
| Browser automation | Python | Playwright's best support |
|
||
|
|
| Data transformation | Python | Rich ecosystem (pandas, etc.) |
|
||
|
|
| Simple file operations | Bash | Lightweight, universal |
|
||
|
|
| Node.js project | JavaScript/TypeScript | Matches project context |
|
||
|
|
| Cross-platform CLI tool | Python or Go | Portability matters |
|
||
|
|
|
||
|
|
### Anti-patterns
|
||
|
|
|
||
|
|
- Writing Bash for complex logic (use a real language)
|
||
|
|
- Choosing Python for a Ruby project because "Python is more popular"
|
||
|
|
- Using JavaScript for non-JS projects just because you know it
|
||
|
|
- Mixing languages within a single skill's scripts without good reason
|
||
|
|
|
||
|
|
## OOP Principles
|
||
|
|
|
||
|
|
Scripts should follow object-oriented principles for maintainability and evolution. These principles, drawn from Sandi Metz's teachings, apply to Ruby, Python, and JavaScript alike.
|
||
|
|
|
||
|
|
### Single Responsibility
|
||
|
|
|
||
|
|
Each class/module does one thing. Each method does one thing.
|
||
|
|
|
||
|
|
```ruby
|
||
|
|
# Good: Single responsibility
|
||
|
|
class FrontmatterValidator
|
||
|
|
def validate(content)
|
||
|
|
# Only validates frontmatter
|
||
|
|
end
|
||
|
|
end
|
||
|
|
|
||
|
|
class StructureValidator
|
||
|
|
def validate(path)
|
||
|
|
# Only validates directory structure
|
||
|
|
end
|
||
|
|
end
|
||
|
|
|
||
|
|
# Bad: Multiple responsibilities
|
||
|
|
class SkillValidator
|
||
|
|
def validate_everything(path)
|
||
|
|
# Validates frontmatter AND structure AND content AND...
|
||
|
|
end
|
||
|
|
end
|
||
|
|
```
|
||
|
|
|
||
|
|
### Dependency Injection
|
||
|
|
|
||
|
|
Objects receive their dependencies; they don't create them.
|
||
|
|
|
||
|
|
```python
|
||
|
|
# Good: Dependencies injected
|
||
|
|
class SkillScaffolder:
|
||
|
|
def __init__(self, file_system, template_loader):
|
||
|
|
self.fs = file_system
|
||
|
|
self.templates = template_loader
|
||
|
|
|
||
|
|
# Bad: Dependencies created internally
|
||
|
|
class SkillScaffolder:
|
||
|
|
def __init__(self):
|
||
|
|
self.fs = RealFileSystem() # Hard to test
|
||
|
|
self.templates = TemplateLoader() # Tightly coupled
|
||
|
|
```
|
||
|
|
|
||
|
|
### Small, Composable Objects
|
||
|
|
|
||
|
|
Prefer many small objects over few large ones. Compose behavior through collaboration.
|
||
|
|
|
||
|
|
```ruby
|
||
|
|
# Good: Small, composable
|
||
|
|
validator = CompositeValidator.new([
|
||
|
|
NameValidator.new,
|
||
|
|
FrontmatterValidator.new,
|
||
|
|
StructureValidator.new
|
||
|
|
])
|
||
|
|
|
||
|
|
# Bad: Monolithic
|
||
|
|
validator = MegaValidator.new # 500 lines, does everything
|
||
|
|
```
|
||
|
|
|
||
|
|
### Immutable Data Where Possible
|
||
|
|
|
||
|
|
Prefer transformations that return new objects over mutations.
|
||
|
|
|
||
|
|
```python
|
||
|
|
# Good: Returns new object
|
||
|
|
def with_updated_name(skill, new_name):
|
||
|
|
return Skill(name=new_name, **skill.other_attrs)
|
||
|
|
|
||
|
|
# Bad: Mutates in place
|
||
|
|
def update_name(skill, new_name):
|
||
|
|
skill.name = new_name # Side effect
|
||
|
|
```
|
||
|
|
|
||
|
|
### Tell, Don't Ask
|
||
|
|
|
||
|
|
Tell objects what to do; don't ask for their data and make decisions externally.
|
||
|
|
|
||
|
|
```ruby
|
||
|
|
# Good: Tell the object
|
||
|
|
validator.validate_and_report(skill_path)
|
||
|
|
|
||
|
|
# Bad: Ask and decide externally
|
||
|
|
if validator.has_frontmatter?(skill_path) && validator.frontmatter_valid?(skill_path)
|
||
|
|
# External decision-making
|
||
|
|
end
|
||
|
|
```
|
||
|
|
|
||
|
|
## Integration with Workflows
|
||
|
|
|
||
|
|
This pattern integrates at two points:
|
||
|
|
|
||
|
|
### During Brainstorming
|
||
|
|
|
||
|
|
When refining a skill concept, the AI should identify script candidates:
|
||
|
|
|
||
|
|
> "This skill involves validating workflow YAML and scaffolding directories. Both are mechanical tasks—I recommend scripts for consistency. The qualitative review of workflow clarity stays with the AI."
|
||
|
|
|
||
|
|
### During Planning (new-skill workflow)
|
||
|
|
|
||
|
|
Step 2 (Analyze and Plan) should explicitly consider:
|
||
|
|
- Which components are script candidates?
|
||
|
|
- What language fits this project?
|
||
|
|
- What quantitative guardrails can be extracted?
|
||
|
|
|
||
|
|
The planning output should list identified scripts before implementation begins.
|
||
|
|
|
||
|
|
## Model Guidance
|
||
|
|
|
||
|
|
| Task | Recommended Model |
|
||
|
|
|------|-------------------|
|
||
|
|
| Writing scripts | Haiku (mechanical, clear requirements) |
|
||
|
|
| Designing script interfaces | Opus (API design is judgment) |
|
||
|
|
| Reviewing script correctness | Haiku (mechanical verification) |
|
||
|
|
| Deciding what to script | Opus (requires understanding intent) |
|
||
|
|
|
||
|
|
## Verification
|
||
|
|
|
||
|
|
Two distinct concepts:
|
||
|
|
|
||
|
|
**Scripts must be verifiable** - Scripts themselves should be reliable and testable:
|
||
|
|
1. **Exit codes** - 0 for success, non-zero for failure
|
||
|
|
2. **Structured output** - JSON or clear text for parsing
|
||
|
|
3. **Idempotent** - Running twice produces same result
|
||
|
|
4. **Testable** - Can be run in isolation with test inputs
|
||
|
|
|
||
|
|
**Scripts as verification tools** - Scripts can serve as verification mechanisms for skills:
|
||
|
|
```bash
|
||
|
|
python scripts/validate_skill.py path/to/skill
|
||
|
|
# Exit code 0 = valid, 1 = invalid
|
||
|
|
# Output describes any issues found
|
||
|
|
```
|
||
|
|
|
||
|
|
This complements the Verification Pattern—scripts provide deterministic evidence that work is complete and correct.
|
||
|
|
|
||
|
|
## Anti-patterns
|
||
|
|
|
||
|
|
### Over-scripting
|
||
|
|
|
||
|
|
Not everything needs a script. If a task requires judgment, context, or interpretation, keep it in AI domain.
|
||
|
|
|
||
|
|
**Signs of over-scripting:**
|
||
|
|
- Script has many special cases and edge case handling
|
||
|
|
- Script needs to "understand" content, not just parse it
|
||
|
|
- Script requires frequent updates as requirements evolve
|
||
|
|
- Script is longer than the AI instructions it replaced
|
||
|
|
|
||
|
|
### Under-scripting
|
||
|
|
|
||
|
|
Repeated mechanical work that stays in AI context wastes tokens and introduces inconsistency.
|
||
|
|
|
||
|
|
**Signs of under-scripting:**
|
||
|
|
- Same validation logic described in multiple places
|
||
|
|
- AI makes occasional errors on mechanical tasks
|
||
|
|
- Structured data processed differently each time
|
||
|
|
- No verification possible because there's no script to run
|
||
|
|
|
||
|
|
### Wrong Abstraction Level
|
||
|
|
|
||
|
|
Scripts should operate at the right level—not too granular, not too broad.
|
||
|
|
|
||
|
|
**Too granular:** Separate scripts for checking each frontmatter field
|
||
|
|
**Too broad:** One script that validates, scaffolds, and generates content
|
||
|
|
**Right level:** One script for frontmatter validation, one for scaffolding, one for structure checks
|