diff --git a/agentic-sdlc-ai-developer-workflow-taxonomy.md b/agentic-sdlc-ai-developer-workflow-taxonomy.md new file mode 100644 index 0000000..934c221 --- /dev/null +++ b/agentic-sdlc-ai-developer-workflow-taxonomy.md @@ -0,0 +1,364 @@ +--- +type: reference +subtype: pattern/framework +summary: Taxonomy of AI Developer Workflow (ADW) structures — from a single agent loop up through kanban-driven software factories — sourced from IndyDevDan's "Forget Loop Engineering" video, with mermaid diagrams and a marked line between his claims and my own os-sdlc extrapolations. +tags: + - type/reference + - domain/agentic-engineering + - tool/claude-code + - convention/agentic-sdlc +scope: global +date: 2026-07-14 +--- + +# Agentic SDLC / AI Developer Workflow (ADW) taxonomy + +Source: IndyDevDan, ["Forget Loop Engineering"](https://youtu.be/VQy50fuxI34), 2026. All +diagrams and claims tagged `[dan]` are his, reconstructed from the video transcript +(`indydevdan-agentic-engineering-overview-video.txt` in cc-os repo root) and 8 paused frames. +Anything tagged `[jrs]` is my own synthesis/extrapolation for `os-sdlc` — do not attribute it +to Dan. Timestamps are exact where the video scrubber was visible in the frame, and marked +`≈` where inferred from chapter placement + on-screen content. + +## Core thesis `[dan]` + +"Loop engineering" is a bad rebrand of the software development life cycle. The real unit of +work is not "the loop" — it's an **AI Developer Workflow (ADW)**: a composition of three +actors of value creation, combined deliberately per stage: + +| Actor | Cost | Reliability | Role | +|---|---|---|---| +| **Code** | Free (no tokens), deterministic, fastest | Highest | Linting, formatting, type-checking, tests, CI/CD, ticket-state transitions — anything that doesn't need judgment | +| **Engineer** | Human time, most expensive per-hour | High (but slow, doesn't scale) | Prompting/planning (start) and reviewing/validating (end) — the two fixed constraints of agentic engineering | +| **Agent** | Token cost, variable reliability | Lowest of the three, improves with scale-out | Judgment-requiring work: planning, building, scouting, testing-with-interpretation | + +"Loops" (fail → retry) are just one control-flow primitive inside an ADW — condition +branches, retries, and routing all show up too. Naming each one "X engineering" doesn't scale; +the workflow is the unit, not the primitive. + +## The escalation ladder `[dan]` + +Dan builds up ADW complexity in one continuous demonstration. Each rung adds either an actor +or a scaling axis. This is the single most useful map for "what does the next level of +sophistication look like" — treat it as a menu, not a mandate to reach the top. + +1. **Prompt-and-review** (4:44) — engineer prompts an agent, engineer reviews the result. No + code, no loop. The floor every ADW is built on. +2. **First loop** (4:58–5:50) — add one piece of deterministic code (a linter) with a + pass/fail condition that routes failures back to the build agent. This condition + routing + is literally what "loop engineering" is pointing at — Dan's argument is that it's too + narrow a name for the whole pattern. +3. **Multiple validation gates** (6:03–6:55) — stack lint → format → type-check, each with its + own pass/fail routing back to the build agent. Still one agent, more code. +4. **Add testing** (6:55–7:12) — the build↔test loop runs until everything passes, then + engineer review, then ship. This is the smallest complete ADW: `prompt → build ⇄ test → + review → ship`. +5. **Scale compute, not scope** (7:40–8:08) — collapse all validation (lint/format/type-check/ + test) into a single **test agent** with its own internal toolset, rather than the engineer + hand-wiring each check. "Add compute to add confidence," not more engineering effort. +6. **Add planning** (8:26–9:07) — a planner agent precedes build. The ADW now mirrors the + classic manual SDLC (plan → build → test → review → ship) with agents inserted at each + step Dan and the engineer used to do by hand. +7. **Worktree parallelism** (9:12–10:07, `[dan]`, diagram below) — a deterministic + "build worktree code" step fans out N parallel `planner → build ⇄ test → review` pipelines, + each in its own git worktree (isolation, no stepping on each other), converging on a single + merge → ship. Explicitly framed as "a great place to start, not a great place to end." +8. **Agent sandboxes** (10:34–11:06, `[dan]`, diagram below) — same fan-out shape as worktrees, + but each pipeline gets a full sandboxed machine instead of a worktree. Buys full isolation + (you can jump in and inspect the running app/web page/tests directly), at real setup cost. +9. **Kanban-driven intake** (11:56–15:21, `[dan]`, diagram below) — a ticket system (support/ + product/engineer intake) becomes the trigger for the pipeline instead of a raw engineer + prompt. Adds a scout agent (searches code/tickets/docs/prior specs) ahead of the planner. + Advanced teams skip the "engineer translates ticket to prompt" step once ticket quality is + good enough. +10. **Production incident ADW** (15:27–17:46, `[dan]`) — a specialized branch: support files a + ticket → engineer triggers a scout → **hotfix agent** (a narrow, "get it out ASAP, not the + fancy way" specialist) → human approve/reject gate (a hotfix needs sign-off before it + burns compute) → N parallel sandboxes race the same fix → first pass wins → engineer + validates → ship. The interesting structural idea: a *specialized* agent + an *extra* + human gate inserted specifically because the blast radius is higher. +11. **Software factory** (17:48–26:39, `[dan]`, diagram below) — the kanban ticket now routes + through a **factory router agent** that (a) sets up a sandbox and (b) picks which + specialized sandboxed ADW to run — chore / bug / feature / hotfix / a custom ADW you add — + each sized to the job (a chore gets a workhorse/lightweight model and skips human review + once the system is trusted; a feature gets full planner → build → test → review). This is + the "meta-layer" Dan argues is where senior engineering effort should go: building the + system that runs the ADWs, not touching the app layer directly. + +## Diagrams + +### 1. Worktree-parallel pipeline — 9:12 `[dan]` + +The pattern Jared is targeting as the near-term "scaled" flow for os-sdlc: N parallel +worktrees, each running a full mini-ADW, converging on one merge/ship gate. + +```mermaid +flowchart LR + EP["👤 Engineer Prompt"] --> BWC{{"🔧 Build Worktree Code"}} + + subgraph WT1["🌿 Worktree 1"] + direction LR + P1["🤖 Planner Agent"] --> B1["🤖 Build Agent"] + B1 -->|fail| P1 + B1 --> T1["🤖 Test Agent"] + T1 -->|fail| B1 + T1 -->|pass| R1["👤 Engineer Review"] + R1 -->|fail| P1 + end + + subgraph WT2["🌿 Worktree 2"] + direction LR + P2["🤖 Planner Agent"] --> B2["🤖 Build Agent"] + B2 -->|fail| P2 + B2 --> T2["🤖 Test Agent"] + T2 -->|fail| B2 + T2 -->|pass| R2["👤 Engineer Review"] + R2 -->|fail| P2 + end + + subgraph WT3["🌿 Worktree 3"] + direction LR + P3["🤖 Planner Agent"] --> B3["🤖 Build Agent"] + B3 -->|fail| P3 + B3 --> T3["🤖 Test Agent"] + T3 -->|fail| B3 + T3 -->|pass| R3["👤 Engineer Review"] + R3 -->|fail| P3 + end + + BWC --> WT1 + BWC --> WT2 + BWC --> WT3 + + R1 -->|pass| M["👤 Merge"] + R2 -->|pass| M + R3 -->|pass| M + M --> SH["👤 Ship"] +``` + +Source frame: `assets/agentic-sdlc-adw/01-worktree-pipeline-0912.png` + +### 2. Individual cycle (zoomed in) — ≈7:00–7:12 `[dan]` + +The single-worktree unit that's tiled N times above. Note the two distinct fail edges: a +**test failure loops back to build** ("loop back"), while an **engineer-review failure loops +back to build** too, but is a distinct, human-gated edge — Dan draws them as separate arrows +because a review rejection can carry different feedback than a test failure. + +```mermaid +flowchart LR + EP["👤 Engineer Prompt"] --> B["🤖 Build Agent"] + B --> T["🤖 Test Agent"] + T -->|"fail: loop back"| B + T -->|pass| R["👤 Engineer Review"] + R -->|fail| B + R -->|pass| SH["👤 Ship"] +``` + +Source frame: `assets/agentic-sdlc-adw/02-individual-cycle-approx-0700.png` + +### 3. Individual cycle with lint + format — ≈6:03–6:25 `[dan]` + +Same shape, with deterministic code gates (lint, format) ahead of test/review instead of a +test agent — this is the "adding code to your ADW" stage, and the shape `os-sdlc`'s Build +Agent stage should actually implement (see "My extrapolation" below). + +```mermaid +flowchart LR + EP["👤 Engineer Prompt"] --> B["🤖 Build Agent"] + B --> L{{"⚙️ Lint Code"}} + L -->|fail| B + L -->|pass| F{{"⚙️ Format Code"}} + F -->|fail| B + F -->|pass| R["👤 Engineer Review"] +``` + +Source frame: `assets/agentic-sdlc-adw/03-lint-format-cycle-approx-0603.png` + +### 4. Agent sandboxes (parallel, full isolation) — ≈10:34–11:06 `[dan]` + +Structurally identical to the worktree diagram — the only change is the isolation unit +(sandbox/VM instead of git worktree). Reuse diagram 1's shape; the distinguishing feature is +each unit is now `🖥️ Agent Sandbox N` instead of `🌿 Worktree N`, and the fan-out node is +"Build Agent Sandbox Code" instead of "Build Worktree Code." + +Source frame: `assets/agentic-sdlc-adw/04-agent-sandboxes-approx-1040.png` + +### 5. Kanban-queue-driven ADW — 12:08 `[dan]` + +Ticket intake replaces the raw engineer prompt as the trigger. A scout agent now precedes +planning (searches code/tickets/docs/prior specs); ticket status transitions (Planning → +Building → Testing) are themselves deterministic code, not agent judgment. + +```mermaid +flowchart LR + SUP["👤 Support"] --> KT{{"🎫 Kanban Ticket"}} + PROD["👤 Product"] --> KT + ENG["👤 Engineer"] --> KT + + KT -->|advanced teams| SBX + KT --> EPr["👤 Engineer Prompt"] --> SBX + + subgraph SBX["🖥️ Sandbox"] + direction LR + SP{{"Status: Planning"}} --> SC["🤖 Scout Agent"] --> PL["🤖 Plan Agent"] + PL --> SB2{{"Status: Building"}} --> BD["🤖 Build Agent"] + BD --> ST{{"Status: Testing"}} + ST --> TS["🤖 Test Agent"] + TS -->|fail| BD + TS -->|pass| CI{{"⚙️ CI/CD"}} + CI -->|fail| BD + end +``` + +Source frame: `assets/agentic-sdlc-adw/05-kanban-queue-1208.png` + +### 6. Software factory — 17:59 `[dan]` + +The kanban ticket now triggers a **Factory Router Agent** that sets up a sandbox and picks +which specialized sandboxed ADW to run: feature, bug, chore, or hotfix — plus a note that any +custom ADW you build slots into the same router. + +```mermaid +flowchart LR + SUP["👤 Support"] --> KT{{"🎫 Kanban Ticket"}} + PROD["👤 Product"] --> KT + ENG["👤 Engineer"] --> KT + + KT --> SF{{"⚙️ Start Factory"}} --> IP{{"Status: In Progress"}} --> FR["🤖 Factory Router Agent"] + FR --> SS{{"🔧 Setup Sandbox"}} + + SS -->|hotfix| HFS + SS -->|feature| FES + SS -->|bug| BGS + SS -->|chore| CHS + SS -->|"any specialized ADW you need"| YOUR["🤖 Your ADW"] + + subgraph HFS["🖥️ Hotfix Sandbox"] + direction LR + HSC["🤖 Scout Agent"] --> HFA["🤖 Hot Fix Agent"] --> APR{{"👤 Approve/Reject"}} + APR -->|reject| HFA + APR -->|approve| HB["🤖 Build Agent"] --> HT["🤖 Test Agent"] + HT -->|fail| HB + HT -->|pass| HR["👤 Engineer Review"] + HR -->|fail| HB + end + + subgraph FES["🖥️ Feature Sandbox"] + direction LR + FPL["🤖 Planner Agent"] --> FB["🤖 Build Agent"] --> FT["🤖 Test Agent"] + FT -->|fail| FB + FT -->|pass| FCI{{"⚙️ CI/CD"}} + FCI -->|fail| FB + FCI -->|pass| FR2["👤 Engineer Review"] + FR2 -->|fail| FB + end + + subgraph BGS["🖥️ Bug Sandbox"] + direction LR + BPL["🤖 Plan Agent"] --> BB["🤖 Build Agent"] --> BT["🤖 Test Agent"] + BT -->|fail| BB + BT -->|pass| BCI{{"⚙️ CI/CD"}} + BCI -->|fail| BB + BCI -->|pass| BR["👤 Engineer Review"] + BR -->|fail| BB + end + + subgraph CHS["🖥️ Chore Sandbox"] + direction LR + CB["🤖 Build Agent"] --> CL{{"⚙️ Lint"}} + CL -->|fail| CB + CL -->|pass| CCI{{"⚙️ CI/CD"}} + CCI -->|fail| CB + CCI -->|pass| CR["👤 Engineer Review"] + CR -->|fail| CB + end + + HR -->|pass| MG["👤 Merge"] + FR2 -->|pass| MG + BR -->|pass| MG + CR -->|pass| MG + MG --> SH["👤 Ship"] +``` + +Source frame: `assets/agentic-sdlc-adw/06-software-factory-1759.png` + +## Dan's build-loop principle, and my extrapolation for os-sdlc + +**`[dan]` (27:13–27:36, "How to Build Great AI Developer Workflows")**: separate code from +agents structurally, not just conceptually. Don't write a skill where the agent both builds +*and* runs the linter internally — that's still "an agent calling code," not separation of +concerns. Instead: run a build agent via the SDK/harness, do work, exit; run the linter as +a **separate deterministic step**; on failure, feed the failure back into the *same session* +of the build agent. His three tips, in order: (1) keep it simple, add nodes only as you hit +real problems; (2) design the ADW by walking it yourself end-to-end first (he recommends +mermaid — the exact tool used for the diagrams above); (3) use agents *and* code, don't let +everything live inside a skill once you're past the prototype stage — code is free, instant, +and deterministic; agents are neither. + +**`[jrs]` — my design decision for os-sdlc's Build Agent, going beyond what Dan says**: take +his separation-of-concerns principle further than "run lint as a separate step" and apply it +to the *build agent's own permissions*. The Build Agent should get the minimum tool +surface needed to write code — plausibly just `Read` + `Write` (maybe `Edit`) — and should +**not** have `Bash` access to run tests, linters, or formatters at all. Enforcement mechanism: +`os-sdlc` hooks trigger lint/format/test after the build agent's turn ends (delta-refinery +style Pre/Post/Handoff), and failures get piped back into the build agent's next turn with +the failing output as context. This is a deliberately stronger claim than Dan's video makes — +he argues for separating *invocation*, not for stripping the build agent's tool grants. Revisit +if this turns out to over-constrain (e.g. a build agent that legitimately needs to run a quick +sanity script mid-task) — not locked, no ADR yet, since the plugin has no code to reverse. + +## Structures/approaches this is NOT the near-term focus for `[dan]` + +Documented for completeness per the taxonomy, explicitly out of scope for the first os-sdlc +iteration (see `plugins/os-sdlc/OVERVIEW.md` in cc-os for the actual build plan): + +- **Production-incident ADW** (racing sandboxes + human approve/reject gate) — a big-team, + high-blast-radius pattern; revisit once os-sdlc has a real production surface to protect. +- **Full software factory + factory router agent** — the "operate the org's entire ADW + portfolio" endgame. `[dan]`'s own framing: start simple, this is where you end up after + scaling, not where you start. +- **Agent sandboxes as the default isolation unit** — plausible but heavier to stand up than + worktrees; `[jrs]`: worth reconsidering once/if worktree isolation proves insufficient + (e.g. an ADW needs a running browser/server the build agent must interact with, not just a + clean filesystem). See `[jrs]` note below on the worktree-vs-sandbox call. + +## `[jrs]` — worktree vs. sandbox, held loosely + +Jared's read (2026-07-14), not Dan's: full agent sandboxes are probably overkill for most +os-sdlc work right now — a plugin skill or small script doesn't need a worktree, let alone a +sandboxed VM. Likely shape: **tiered isolation**, chosen per task — +1. No isolation — trivial single-file edits, tiny plugin scripts. +2. Single worktree — anything touching multiple files/tests in one repo, no parallelism needed. +3. N parallel worktrees — the diagram-1 shape, once a task is decomposable into independent + slices worth racing or reviewing separately. +4. Full sandbox — reserved for cases needing a live running app/server the agent must poke at, + not just source control isolation. + +If more than one tier is actually used in practice, a **router** (probably a cheap +deterministic check, not an agent call, per Dan's "code is free" argument) will be needed to +pick the tier per task — but per both Dan's and Jared's "start simple" instinct, defer building +the router until tier 3 is proven useful and tier 4 has a concrete forcing use case. At a +minimum, os-sdlc needs to scale using worktrees — whether that's 1 or 20 — before anything +else on this list. + +## Are "issues" and "tickets" the same thing? + +Related but deliberately two layers in cc-os's existing model, not synonymous: +- A **Planka card** (`os-backlog`) is lightweight kanban/queue state — the thing that moves + through columns (Backlog → Doing → Review/Done), analogous to Dan's "Kanban Ticket" node. +- A **git issue** (Forgejo, created via `/to-tickets`) is the durable spec — the actual + breakdown of work, meant to survive and be referenced. +- `os-backlog`'s own PROMOTION rule already encodes this: when a Planka card accretes real + spec content, that content moves to a git issue and the card becomes a pointer (title + + link), never a duplicate of the spec text. + +So: Dan's "Kanban Ticket" node in the diagrams above maps to the *combination* of a Planka +card (queue position, trigger) and, once work is non-trivial, a linked git issue (the actual +spec the Scout/Plan agent reads). + +## Related + +- `[[agentic-sdlc-os-sdlc-build-components]]` — not yet written; the repo-specific build plan + lives in `plugins/os-sdlc/OVERVIEW.md` in cc-os instead of the vault (repo-specific, not + cross-project knowledge). Link added here as a pointer once that note/section exists. diff --git a/assets/agentic-sdlc-adw/00-chapters.png b/assets/agentic-sdlc-adw/00-chapters.png new file mode 100644 index 0000000..a901d60 Binary files /dev/null and b/assets/agentic-sdlc-adw/00-chapters.png differ diff --git a/assets/agentic-sdlc-adw/01-worktree-pipeline-0912.png b/assets/agentic-sdlc-adw/01-worktree-pipeline-0912.png new file mode 100644 index 0000000..d2b9ed4 Binary files /dev/null and b/assets/agentic-sdlc-adw/01-worktree-pipeline-0912.png differ diff --git a/assets/agentic-sdlc-adw/02-individual-cycle-approx-0700.png b/assets/agentic-sdlc-adw/02-individual-cycle-approx-0700.png new file mode 100644 index 0000000..7386ff3 Binary files /dev/null and b/assets/agentic-sdlc-adw/02-individual-cycle-approx-0700.png differ diff --git a/assets/agentic-sdlc-adw/03-lint-format-cycle-approx-0603.png b/assets/agentic-sdlc-adw/03-lint-format-cycle-approx-0603.png new file mode 100644 index 0000000..1d1f586 Binary files /dev/null and b/assets/agentic-sdlc-adw/03-lint-format-cycle-approx-0603.png differ diff --git a/assets/agentic-sdlc-adw/04-agent-sandboxes-approx-1040.png b/assets/agentic-sdlc-adw/04-agent-sandboxes-approx-1040.png new file mode 100644 index 0000000..18ff675 Binary files /dev/null and b/assets/agentic-sdlc-adw/04-agent-sandboxes-approx-1040.png differ diff --git a/assets/agentic-sdlc-adw/05-kanban-queue-1208.png b/assets/agentic-sdlc-adw/05-kanban-queue-1208.png new file mode 100644 index 0000000..61e0826 Binary files /dev/null and b/assets/agentic-sdlc-adw/05-kanban-queue-1208.png differ diff --git a/assets/agentic-sdlc-adw/06-software-factory-1759.png b/assets/agentic-sdlc-adw/06-software-factory-1759.png new file mode 100644 index 0000000..2d27ae5 Binary files /dev/null and b/assets/agentic-sdlc-adw/06-software-factory-1759.png differ diff --git a/assets/agentic-sdlc-adw/07-build-lint-loop-2746.png b/assets/agentic-sdlc-adw/07-build-lint-loop-2746.png new file mode 100644 index 0000000..2b8f3d2 Binary files /dev/null and b/assets/agentic-sdlc-adw/07-build-lint-loop-2746.png differ diff --git a/journal/2026-07-14.md b/journal/2026-07-14.md index e0e5f2c..12b9c76 100644 --- a/journal/2026-07-14.md +++ b/journal/2026-07-14.md @@ -37,3 +37,17 @@ tags: [scope/global, type/log] **Reason:** prompt_input_exit **Vault notes touched:** (none) + +## Session — 2026-07-14T14:22:48Z + +**Project:** /home/jared/dev/cc-os +**Reason:** prompt_input_exit +**Vault notes touched:** +/home/jared/Documents/SecondBrain/matt-pocock-skills-v1-1-changes.md + +## Session — 2026-07-14T15:25:37Z + +**Project:** /home/jared/dev/cc-os +**Reason:** prompt_input_exit +**Vault notes touched:** +/home/jared/Documents/SecondBrain/skills-sh-security-scans-are-heuristic.md diff --git a/skills-sh-security-scans-are-heuristic.md b/skills-sh-security-scans-are-heuristic.md new file mode 100644 index 0000000..81803ea --- /dev/null +++ b/skills-sh-security-scans-are-heuristic.md @@ -0,0 +1,40 @@ +--- +type: reference +title: skills.sh security risk assessments are heuristic capability flags, not malware verdicts +summary: "How to interpret skills.sh's Gen/Socket/Snyk security table: High/Med ratings fire on benign capability patterns, so read the per-skill detail page and the actual files before trusting a flag." +tags: + - type/reference + - tool/skills-sh + - domain/agent-skills + - domain/security +scope: global +last_updated: 2026-07-14 +date: 2026-07-14 +source: cc-os +--- + +# skills.sh security scans are heuristic + +The "Security Risk Assessments" table shown by the `npx skills` installer (Gen, Socket, +Snyk columns) reports heuristic capability flags, not malware verdicts. High/Med ratings +routinely fire on benign patterns: + +- **Snyk W007 (credential handling)** — skill instructs pasting CLI output or bundles an + executable `.sh` file, even an inert interactive one. +- **Snyk W011 (indirect prompt injection)** — skill reads external issue/ticket/PR text + into context (any tracker-reading skill triggers this). +- **Socket anomaly alerts** — trust-chain patterns like "posts autonomously to an external + tracker"; explicitly not malware findings. + +Verified 2026-07-14 on Matt Pocock's skill pack: all five flags (one High, three Med, one +Socket alert) traced to legitimate documented features; full local file reads found no +injection, obfuscation, network calls, or confirmation bypasses. Third-party testing +(dev.to bolhasec obfuscation study; caveman#28) shows the scanners are inconsistent — +Socket missed a known-malicious baseline while Gen over-flagged benign scripts. + +**Procedure when a flag appears:** open +`skills.sh////security/` for the specific finding, then read +the installed skill files (typically `~/.agents/skills//`) checking for network +calls, obfuscation, credential access, and confirmation bypasses. Trust the file contents +and publisher reputation over the badge — in both directions: a "Safe" row is equally +weak evidence.