681 lines
38 KiB
Plaintext
681 lines
38 KiB
Plaintext
|
|
source: https://youtu.be/VQy50fuxI34?si=fENPEi7_hFppIftM
|
||
|
|
Forget Loop Engineering
|
||
|
|
0:00
|
||
|
|
It's time for us to talk about loop engineering. I was hoping this would just blow over after a couple weeks. I
|
||
|
|
0:05
|
||
|
|
was hoping a cracked engineer would call this stupid phrase what it really is, but no one has. So, I'll do it myself.
|
||
|
|
0:13
|
||
|
|
Forget about loop engineering. It's the wrong way to think about building valuable software with agents at scale
|
||
|
|
0:20
|
||
|
|
consistently. Loop engineering is a terrible rebrand of the software development life cycle. It's as unclear
|
||
|
|
0:28
|
||
|
|
as it is hype-filled. In this video, we simplify loop engineering and call it
|
||
|
|
0:33
|
||
|
|
what it really is. If you understand this concept properly, we're going to break down in this video, [music]
|
||
|
|
0:39
|
||
|
|
you'll accelerate far ahead of the AI industry. Clarity and simplicity of
|
||
|
|
0:45
|
||
|
|
information gives you [music] speed and performance in your work. It's much more
|
||
|
|
0:50
|
||
|
|
valuable and helpful to think about building with [music] agents as if you're building developer workflows
|
||
|
|
0:57
|
||
|
|
inside your software factory. Your props [music] go into your software factory. A
|
||
|
|
1:03
|
||
|
|
specific workflow runs. Each workflow [music] is a combination of code plus agents and then your results [music]
|
||
|
|
1:10
|
||
|
|
come out. Forget about loop engineering. Focus your valuable engineering time and
|
||
|
|
1:16
|
||
|
|
tokens on building AI developer workflows.
|
||
|
|
Who Is IndyDevDan?
|
||
|
|
1:25
|
||
|
|
So, first off, who am I to go up against big ideas from AI engineers like Boris
|
||
|
|
1:31
|
||
|
|
Churnney from Anthropic and Peter Steinberg, now from OpenAI? If you already know who I am and the work I've
|
||
|
|
1:37
|
||
|
|
done, or you don't care, skip to this timestamp. My name is Dan Eisler aka Indie Dev Dan. I'm a software engineer
|
||
|
|
1:43
|
||
|
|
with over 15 years of EXP. I started out building Adobe Flash games with Action Script 2 and three with my brothers. I
|
||
|
|
1:50
|
||
|
|
interned at Blizzard and then I quickly moved into the finance and accounting space programming in C, TypeScript,
|
||
|
|
1:57
|
||
|
|
Python, God awful React and God bless Evanu for creating Vue. I was AI coding
|
||
|
|
2:03
|
||
|
|
before anyone had a name for it with tools like Ader and classic models like GPT3.5 Turbo, GPT4, and Sonnet 3. Tools
|
||
|
|
2:12
|
||
|
|
and models you've probably never heard of or used. I own the domain name agenticengineer.com
|
||
|
|
2:18
|
||
|
|
where thousands of engineers you've heard of from companies you know have improved their agentic engineering
|
||
|
|
2:24
|
||
|
|
thanks to information products I've built by hand. But anyone can pick up a domain name and create a course though,
|
||
|
|
2:31
|
||
|
|
right? Sure. I also have an irrefutable trail of code and content for engineers
|
||
|
|
2:37
|
||
|
|
on GitHub and this YouTube channel every single week for years now. You can see
|
||
|
|
2:42
|
||
|
|
myself and engineers that follow this channel consistently ahead of the curve of the AI industry. I don't just farm
|
||
|
|
2:49
|
||
|
|
news for views like every other tech content creator. On this channel, we think, plan, and build. Every few
|
||
|
|
2:56
|
||
|
|
months, it's important for me to sit down and say this. I'm not a content creator. I'm a software engineer that
|
||
|
|
3:03
|
||
|
|
does content creation on the side. Why? Because this technology is too valuable
|
||
|
|
3:09
|
||
|
|
to fall into the hands of a lucky few. And yes, I sell courses. And yes, I
|
||
|
|
3:14
|
||
|
|
benefit from it. Big whoop. Don't do anything you're great at for free. Welcome to capitalism. Enough about me.
|
||
|
|
3:19
|
||
|
|
I don't show up here every single Monday to gloat and talk about myself. I show up here to give engineers like you an
|
||
|
|
3:25
|
||
|
|
advantage you can use to accelerate your career, your work, your business, your engineering in the age of AI, no hype.
|
||
|
|
3:33
|
||
|
|
So, forget about loop engineering and focus on this instead. There are now
|
||
|
|
Your 3 Actors of Value Creation
|
||
|
|
3:39
|
||
|
|
three actors of value creation for engineering work. The engineers like you and I, there are the agents and there's
|
||
|
|
3:46
|
||
|
|
the code. Knowing when and where to place each of these is the name of the
|
||
|
|
3:52
|
||
|
|
game of agentic engineering. And you might be thinking, where is he going with this? How does this relate to loop engineering and developer workflows?
|
||
|
|
3:59
|
||
|
|
Stick with me here. We're going to work up to it. If you master the fundamentals, you'll master the compositions. Everyone in their AI
|
||
|
|
4:05
|
||
|
|
psychosis seems to forget code is fast, always runs the same way unless you tell
|
||
|
|
4:11
|
||
|
|
it not to. And guess what? It costs nothing. There are no token costs associated with code. the thing that
|
||
|
|
4:17
|
||
|
|
truly moves at light speed. There's a hidden cost to implementing every single one of these actors of value creation.
|
||
|
|
4:25
|
||
|
|
Everyone's talking about agents. We're all well aware of the cost of engineers, but code is the unsung hero of all of
|
||
|
|
4:32
|
||
|
|
this. Consistent value creation creates consistent business value. And out of these three, code is the most reliable
|
||
|
|
4:39
|
||
|
|
by miles followed by engineers [music] and then agents. So let's start with the
|
||
|
|
Your First Ever AI Developer Workflow
|
||
|
|
4:44
|
||
|
|
most basic developer workflow.
|
||
|
|
4:50
|
||
|
|
An engineer prompts an LLM and the engineer reviews the result. This is the simple foundation that makes up every
|
||
|
|
4:58
|
||
|
|
single loop, every single workflow, every single piece of work moving forward. Now, of course, we're not just
|
||
|
|
5:05
|
||
|
|
using the LLM anymore. In this central node, we have an agent. Insert your favorite agent. Insert your favorite
|
||
|
|
5:11
|
||
|
|
model. It doesn't matter anymore. It's about the workflow that you and I execute every single day. Great. Let's
|
||
|
|
5:19
|
||
|
|
scale it up. Now, we have code, agents, and engineers all involved in the
|
||
|
|
5:24
|
||
|
|
process. We're building up to more and more advanced developer workflows. You'll notice something really important
|
||
|
|
5:31
|
||
|
|
here. We now have code. In this case, we're just running a llinter. We have a condition. If the llinter fails, the
|
||
|
|
5:37
|
||
|
|
results go back into our build agent. If they're successful, it passes. This condition and this routing back to our
|
||
|
|
5:45
|
||
|
|
build agent creates our first loop. Hence the term loop engineering. But
|
||
|
|
5:50
|
||
|
|
loop engineering is too simple. It's too inaccurate. And there's a lot more to this story. So what comes next? How can
|
||
|
|
5:56
|
||
|
|
we continue to enhance our developer workflow to get better results? We can of course add more deterministic code.
|
||
|
|
Adding Code to Your ADW
|
||
|
|
6:03
|
||
|
|
Now we have multiple pass fill statements routing back into our build agent. Your codeex, your cloud code,
|
||
|
|
6:08
|
||
|
|
your pi coding agent, whatever your agent harness is. This is the foundation of what it means to build with agents.
|
||
|
|
6:14
|
||
|
|
Now you have three actors in this. Engineers, agents, and raw code. You
|
||
|
|
6:20
|
||
|
|
have to leverage them all in the right location at the right time. Here we're adding a code formatter. It doesn't
|
||
|
|
6:25
|
||
|
|
matter what language you're in. linting your code, formatting your code, type check your code, and then keep scaling
|
||
|
|
6:31
|
||
|
|
up the validation loops to run back into your agents. Once again, you'll see here
|
||
|
|
6:37
|
||
|
|
these conditions is what makes up what is called the loop. But there's a lot more going on here. This is really a
|
||
|
|
6:44
|
||
|
|
workflow of how information travels within a system. Okay, keep scaling it
|
||
|
|
6:50
|
||
|
|
up. What comes next? We can add more code. A very valuable piece of code here
|
||
|
|
6:55
|
||
|
|
is testing your code. So test. Now we take all these results, feed it back
|
||
|
|
7:00
|
||
|
|
into our build agent over and over and over until the results all pass. And
|
||
|
|
7:05
|
||
|
|
that gives us our final engineering review. You'll notice a pattern here. You and I always show up at the ends.
|
||
|
|
7:12
|
||
|
|
These are the two constraints of a gentic engineering. Prompting, also known as planning, and reviewing, also
|
||
|
|
7:18
|
||
|
|
known as validation. If you're a gent engineering at scale properly, you're showing up at the beginning and the end
|
||
|
|
7:25
|
||
|
|
with a few exceptions. Your AI developer workflows start simple like this, but they should continue to grow. Real
|
||
|
|
7:31
|
||
|
|
engineering work looks a lot more complex than this, [music] right? What do we do next?
|
||
|
|
Scale Your Compute to Scale Your Impact
|
||
|
|
7:40
|
||
|
|
We can scale all of our testing, all of our validation, all of our linting, all of our type checking into a single test
|
||
|
|
7:47
|
||
|
|
agent. So now we're scaling our compute to scale our impact. We're adding compute to add confidence. Now you can
|
||
|
|
7:54
|
||
|
|
imagine we've handed this test agent all the things we want to do to test. And if something goes wrong, we send the
|
||
|
|
8:01
|
||
|
|
context back to the build agent. If it passes, the engineer reviews and then we can ship the deliverable. We can ship
|
||
|
|
8:08
|
||
|
|
the code. All right. So notice a couple themes working here. As you scale up your developer workflows, you add agents
|
||
|
|
8:14
|
||
|
|
and you add code. But what you don't want to add is more engineering effort outside of building the system that
|
||
|
|
8:21
|
||
|
|
builds the system. Let's keep scaling this up. Let's add planning to the workflow. You're very familiar with
|
||
|
|
8:26
|
||
|
|
these ideas, right? We're building workflows. These are all steps that you and I, the engineer, used to take and
|
||
|
|
8:33
|
||
|
|
used to execute ourself, right? We would plan work, we would build the work, we would test the work, we would then have
|
||
|
|
8:40
|
||
|
|
another engineer review the work, and then we would finally ship it into production. All right? It's a developer
|
||
|
|
8:45
|
||
|
|
workflow and all we've done here is added AI to it. The loops is just one
|
||
|
|
8:50
|
||
|
|
piece of it. You can call it loop engineering but it's inaccurate and it's not encapsulating the whole picture. If
|
||
|
|
8:55
|
||
|
|
we have loop engineering, we need to have condition engineering and then we need to have function engineering and then we need to have a word plus
|
||
|
|
9:02
|
||
|
|
engineering for every type of control flow inside of the software development life cycle which is going to go on
|
||
|
|
9:07
|
||
|
|
forever. Okay. So a very popular pattern is to push each one of your agents into
|
||
|
|
9:13
|
||
|
|
their own work tree. This creates isolation. This creates parallelism. This lets you do more work in parallel.
|
||
|
|
9:19
|
||
|
|
And there's a nice side effect here where the agents don't trip over each other. So, guess what we're going to do?
|
||
|
|
9:24
|
||
|
|
We're going to write another prompt. And this time, we're going to write a prompt into a piece of code that's going to
|
||
|
|
9:31
|
||
|
|
build our work trees. All right. So, here we have a deterministic piece of code that's going to kick off multiple
|
||
|
|
9:37
|
||
|
|
work trees based on the prompt. And then we're going to execute several different agents running in line. So we have once
|
||
|
|
9:44
|
||
|
|
again scaled our compute to scale our impact. We have multiple work trees
|
||
|
|
9:49
|
||
|
|
running our plan, build, test, review, merge, ship pipeline, our developer
|
||
|
|
9:56
|
||
|
|
workflow. The workflow you and I used to go through as engineers building by
|
||
|
|
10:02
|
||
|
|
hand. Now we have AI, hence AI developer workflows. And this is really important
|
||
|
|
10:07
|
||
|
|
to think through. This is where you should focus in your engineering time. How can I combine the three actors of
|
||
|
|
10:14
|
||
|
|
value creation, engineers, agents, and code to create workflows that execute
|
||
|
|
10:20
|
||
|
|
large amounts of work on my behalf, on behalf of my company, on behalf of your users and products? That's the name of
|
||
|
|
10:27
|
||
|
|
the game. Okay, so great, we have work trees. Work trees are um like I like to say, a great place to start, not a great
|
||
|
|
10:34
|
||
|
|
place to end. There are a lot of problems with work trees. We can do one better by giving our agents each their
|
||
|
|
10:40
|
||
|
|
own sandbox. Okay? So instead of spinning up work trees, we're now giving every single agent their own computer,
|
||
|
|
10:47
|
||
|
|
right? Their own agent sandbox to operate. Because once you do this, you have full isolation. You yourself can
|
||
|
|
10:54
|
||
|
|
jump into the sandbox to look at the work, look at the result, look at the web page, look at the tests, look at the
|
||
|
|
11:00
|
||
|
|
application, whatever you need to do, do your review. And then of course once all the work comes back in you merge and you
|
||
|
|
11:06
|
||
|
|
ship. So once again notice the three actors of value creation working together. And notice how your ability to
|
||
|
|
11:14
|
||
|
|
create these AI developer workflows is your ability to scale your impact with
|
||
|
|
11:19
|
||
|
|
agents. It's designing these AI developer workflows that is the most value accreative thing an engineer can
|
||
|
|
11:26
|
||
|
|
do. Hence the term I like to say on the channel all the time. You want to be building the system that builds the
|
||
|
|
11:33
|
||
|
|
system. Okay? As you can see here, you'll start to see similarities in these workflows with a lot of work
|
||
|
|
11:39
|
||
|
|
you've seen, a lot of work that you've done, and hopefully work that you're building into your teams, your
|
||
|
|
11:44
|
||
|
|
co-workers, your business, your tools themselves. Okay? We're just getting started, [laughter] right? Uh real work
|
||
|
|
11:51
|
||
|
|
gets more and more complex. And this is the art and science of agentic engineering. So, let's keep scaling it.
|
||
|
|
The Kanban Queue
|
||
|
|
12:02
|
||
|
|
A very common thing to do as you continue to progress is to set up a conbon board, some type of ticket
|
||
|
|
12:09
|
||
|
|
system. Input comes from all over your organization, right? It comes from support, it comes from product, and it
|
||
|
|
12:14
|
||
|
|
comes from engineers, right? So now things get interesting because now we have a new unit, a new wrapper around
|
||
|
|
12:20
|
||
|
|
our code, right? We have some type of ticketing system. And so once again, like there's this really important delineation that I make inside of
|
||
|
|
12:28
|
||
|
|
tactical agentic coding. And it is this idea of the agentic layer. The agents,
|
||
|
|
12:33
|
||
|
|
the prompts, the skills, the system prompts that wrap your application are
|
||
|
|
12:39
|
||
|
|
the thing to be focused on right now. Because when you put those together with your code, with your system, with your
|
||
|
|
12:45
|
||
|
|
entire team and your organization, you are agentic engineering. Agentic engineering is not just about the
|
||
|
|
12:51
|
||
|
|
agents. Spoiler alert, it's about your team and most importantly your users and putting together the most valuable stack
|
||
|
|
12:58
|
||
|
|
of the actors of value creation. Okay, so conbon board, right? Let's let's jump into this. What does this look like?
|
||
|
|
13:04
|
||
|
|
Okay, so for most teams, your tickets are then analyzed by your engineer who actually knows what's going on and
|
||
|
|
13:10
|
||
|
|
they'll translate that into a mid to low-level prompt that you'll then pass into another full workflow, another
|
||
|
|
13:17
|
||
|
|
agent sandbox. But some advanced teams if you're teaching your organization how to write prompts well enough and as
|
||
|
|
13:23
|
||
|
|
models become more capable uh you can skip your engineer input prompt here right because your engineer's job should
|
||
|
|
13:29
|
||
|
|
be building the system we act on the meta layer we act on the layer that can compound across our organization okay so
|
||
|
|
13:35
|
||
|
|
advanced teams can skip the engineering prompt if this step is done properly all right but so you know this is code right
|
||
|
|
13:42
|
||
|
|
conbon boards it's just code there are no agents there then we enter the meat of our workflow where we run code to
|
||
|
|
13:48
|
||
|
|
move that ticket into planning. Guess what happens next? Our agents take over the pipeline. We'll have a scout agent
|
||
|
|
13:54
|
||
|
|
look for all the code, look for all the tickets, look for all the documentation, look for previous spec files, and then
|
||
|
|
13:59
|
||
|
|
it'll hand that to a plan agent. Right? So, we're splitting up our searching and our planning between two agents here.
|
||
|
|
14:05
|
||
|
|
Once again, scaling compute, scaling impact. And after that happens, you know, plan phase is complete. So, we run
|
||
|
|
14:11
|
||
|
|
code to update our ticket to move context to do some specific work inside of our sandbox. And then of course our
|
||
|
|
14:17
|
||
|
|
build agent kicks off. And you know what happens from here. You've done it a million times yourself. And now with
|
||
|
|
14:23
|
||
|
|
agents, your build agent moves it into testing after it's done. And then your test agent tests, right? This individual
|
||
|
|
14:30
|
||
|
|
loop, which is just one part of the developer workflow, executes until the result passes. And then we're going to
|
||
|
|
14:37
|
||
|
|
run our CI/CD. And guess what can happen here? This can pass or it can fail and
|
||
|
|
14:42
|
||
|
|
go right back to the build agent to resolve the issues. All right. And then we get outside the sandbox. Engineer
|
||
|
|
14:47
|
||
|
|
reviews the code. You know what this looks like now? Fail, pass, ship. Right? So you can see this is much more than
|
||
|
|
14:54
|
||
|
|
just prompt engineering. It's much more than context engineering. It's much more than harness engineering. It's much more than loop engineering. This is about how
|
||
|
|
15:00
|
||
|
|
teams move as an organism with all the actors inside. Okay? And if you're a
|
||
|
|
15:07
|
||
|
|
small solo dev shop, same deal, right? It's about how you and your agents work together with code to generate valuable
|
||
|
|
15:15
|
||
|
|
results. But this isn't the end. The future of engineering is vast. Okay, let's keep pushing this. What happens
|
||
|
|
15:21
|
||
|
|
next? Let's let's imagine another scenario here. Imagine you have a support crisis. Production is down.
|
||
|
|
Production Goes Down
|
||
|
|
15:28
|
||
|
|
Okay, production has crashed. So, what AI developer workflow do you have planned in your organization right now
|
||
|
|
15:35
|
||
|
|
when production goes down? How are you leveraging engineers, agents, and code
|
||
|
|
15:40
|
||
|
|
to resolve this issue and to stop your business from leaking cash as your production system is down? Let's walk
|
||
|
|
15:47
|
||
|
|
through it, okay? Because we've thought through this, right? We've designed, we've architected the AI developer
|
||
|
|
15:53
|
||
|
|
workflow to account for this situation. Support files a ticket. In our case here, this goes right to Slack. It goes
|
||
|
|
15:58
|
||
|
|
right to Teams, goes right to your communication channel, and one of your cracked engineers picks this up immediately. What do they do? They
|
||
|
|
16:04
|
||
|
|
prompt a scout agent that routes right into a hot fix agent. Your hot fix agent
|
||
|
|
16:10
|
||
|
|
has a specialized set of mental memory. It's an agent expert that knows and is
|
||
|
|
16:16
|
||
|
|
prioritized to get things fixed. It's not doing things the right way. It's not doing things the fancy way. It's not
|
||
|
|
16:21
|
||
|
|
optimizing anything. It's getting the fix out ASAP and nothing else. This is a surgical hotfix agent, a custom agent
|
||
|
|
16:28
|
||
|
|
that you have specialized, that you've templated your engineering into. Now, what happens here? Human in the loop.
|
||
|
|
16:34
|
||
|
|
This is a hot fix. We need to know the solution is going to work. So you put in human effort, right? You use engineering
|
||
|
|
16:39
|
||
|
|
effort to approve or reject. This creates a single loop. All right? If we approve, guess what happens? We're going
|
||
|
|
16:45
|
||
|
|
to build up a bunch of sandboxes to run the solution in parallel. And guess what? We're using multiple sandboxes. I
|
||
|
|
16:52
|
||
|
|
want the first fastest agent that has the solution to win. Okay? Whatever your compute budget is, you'll scale this up.
|
||
|
|
16:57
|
||
|
|
You'll scale this down. If you're in a production system that's complex, you might want three, five, 10 agents
|
||
|
|
17:02
|
||
|
|
running and racing toward a solution in their own agent sandbox, you don't care. You have the compute, you've done the agentic engineering to scale your
|
||
|
|
17:09
|
||
|
|
impact. And guess what happens here? You already know what happens. It runs its individual loop in their sandbox. And
|
||
|
|
17:15
|
||
|
|
then it passes or fails. If it fails, it goes right back to your hot fix agent and to you to resolve. And of course, if
|
||
|
|
17:21
|
||
|
|
it's all successful, you the engineer validate it and you get the hot fix shipped ASAP. Okay. A question for you
|
||
|
|
17:28
|
||
|
|
and your organization. Do you have an agentic workflow for production crashes? Can you get that resolved in record time
|
||
|
|
17:35
|
||
|
|
using the three actors of value creation in the age of agents? Engineers, agents,
|
||
|
|
17:41
|
||
|
|
and code. All three. Okay, this continues to scale and scale and scale.
|
||
|
|
17:46
|
||
|
|
Let's push these workflows further. After some point, what you get is a structure like this.
|
||
|
|
The Software Factory
|
||
|
|
17:57
|
||
|
|
And this is what really starts turning into a software factory. Okay, you'll see here we have many different types of
|
||
|
|
18:04
|
||
|
|
specialized agent sandbox workflows. Some are for chores, one is for a bug,
|
||
|
|
18:09
|
||
|
|
one is for a feature, one is for this hot fix that we just walked through. And you get the idea here, right? Any
|
||
|
|
18:15
|
||
|
|
specialized AI developer workflow you need can be built and routed to thanks
|
||
|
|
18:21
|
||
|
|
to your routing system. Okay, this is the art and science of agentic
|
||
|
|
18:26
|
||
|
|
engineering, right? This is all of it put together. The loops are just one small piece of this picture. I hope you
|
||
|
|
18:32
|
||
|
|
can see that. Now, in all this is a great level of prompt context harness
|
||
|
|
18:38
|
||
|
|
engineering. There are a million ways to do this. There are a million different multi- aent orchestration patterns to
|
||
|
|
18:45
|
||
|
|
build into this. The key here is this. is that you have the right combination
|
||
|
|
18:51
|
||
|
|
at the right time to push engineering work through end to end with agents with
|
||
|
|
18:57
|
||
|
|
code with engineers. Okay, I know I'm repeating myself. I'm doing it on purpose. Most success in any domain is
|
||
|
|
19:04
|
||
|
|
about doing a few things and saying a few things and focusing on a few things over and over and over. Let's walk
|
||
|
|
19:11
|
||
|
|
through a full software factory. You can imagine how this looks, right? Let's keep with a conbon ticket example. Now,
|
||
|
|
19:18
|
||
|
|
anyone can file a ticket. This is a feature. This is a bug. This is a chore. Advanced teams are going to skip wasting
|
||
|
|
19:24
|
||
|
|
engineering time transferring your conbon ticket into a low-level or mid-level engineering breakdown of what
|
||
|
|
19:31
|
||
|
|
needs to happen. Advanced teams are going to go right to kicking off a software factory. The moment your conbon ticket lands, once the factory starts,
|
||
|
|
19:37
|
||
|
|
it's going to mark that ticket in progress and move it. And now we have a factory router agent. This could just be
|
||
|
|
19:43
|
||
|
|
a simple LLM call. This could be some deterministic code. The exact nodes are
|
||
|
|
19:48
|
||
|
|
up to you to decide, but you get the idea here. I'm going to throw a factory agent here to intake the results. Do a
|
||
|
|
19:55
|
||
|
|
quick look at the codebase, understand what AI developer workflow we need to execute for the system. First though,
|
||
|
|
20:01
|
||
|
|
we're going to set up a sandbox. We're not limiting our agents anymore. We know that agents are going to continue to
|
||
|
|
20:07
|
||
|
|
expand. This is what the CPU crunch is all about. CPUs are getting wiped off the board outside of scaling RL and
|
||
|
|
20:14
|
||
|
|
other ML engineering related work. Agent sandboxes are going to, I can guarantee you this, be the majority of computers
|
||
|
|
20:20
|
||
|
|
out there in the world. You and I will be using fewer and fewer devices while our agents continue to scale up and use
|
||
|
|
20:25
|
||
|
|
more sandboxes. Okay, but set up the sandbox. After that, our agent has already decided what type of workflow we
|
||
|
|
20:31
|
||
|
|
need to get the job done at the best price, at the best performance, and at the right speed. Because as you likely
|
||
|
|
20:37
|
||
|
|
know, you're not going to run your hot fix AI developer workflow or your feature AI developer workflow where
|
||
|
|
20:43
|
||
|
|
you're scaling out your very best agents. Maybe your build agent is a workhorse model, but your planner and
|
||
|
|
20:48
|
||
|
|
your scouters are going to be state-of-the-art model so nothing gets missed. Of course, there's a whole slew of multi- aent orchestration work that
|
||
|
|
20:55
|
||
|
|
can happen here. But the whole point is you're not going to deploy your heavy AI developer workflows for a chore, right?
|
||
|
|
21:00
|
||
|
|
for your chore. Throw a single agent at this with a workhorse model, maybe even a lightweight model. Build it, run the
|
||
|
|
21:06
|
||
|
|
lint, run the CI/CD, engineer reviews it, and ship it out. We'll talk about ZTE in a second, but the best teams are
|
||
|
|
21:12
|
||
|
|
going to start dropping off engineering review because they've built the best system possible that they know is going
|
||
|
|
21:18
|
||
|
|
to execute for them. But every single unique workflow is unique for a reason,
|
||
|
|
21:24
|
||
|
|
right? There are multiple workflows you want to build out here, multiple AI developer workflows you should be building out, not just one. I'll give my
|
||
|
|
21:30
|
||
|
|
recommendation some really, really great practices you can use when building these out in a moment here. But the
|
||
|
|
21:36
|
||
|
|
general rule of thumb is just to start simple. Once you start scaling this up, what you're going to end up with is a
|
||
|
|
21:41
|
||
|
|
software factory. A software factory that can operate your application as
|
||
|
|
21:46
|
||
|
|
well. And if you're doing it right, better than you and your engineering team. This is why all your effort, all
|
||
|
|
21:52
|
||
|
|
of your effort goes into this. Now, the agentic layer, right? This is the al the
|
||
|
|
21:57
|
||
|
|
agentic layer, not the app layer. The app layer is for your agents. The the
|
||
|
|
22:03
|
||
|
|
best engineering teams never touch the product themselves. Okay, I know this might be like controversial. Some
|
||
|
|
22:09
|
||
|
|
engineers are going to hate hearing this, but the best teams are doing meta work on the agentic layer. They're
|
||
|
|
22:16
|
||
|
|
building the system that builds the system. That is the central thesis inside of tactical agentic coding.
|
||
|
|
22:22
|
||
|
|
Thousands of engineers know that and you're going to figure it out too sooner or later. Okay, that doesn't mean you can't jump into the app to do work. But
|
||
|
|
22:29
|
||
|
|
when you have a successful product scaled with users, the name of the game is this. Building a software factory
|
||
|
|
22:37
|
||
|
|
that operates everything better than you alone could, better than code alone could, and better than agents alone
|
||
|
|
22:44
|
||
|
|
could. Right? Three actors of value creation, agents, engineers, code. Where does that all lead us? It leads us to
|
||
|
|
22:50
|
||
|
|
the simple conclusion that it's not a loop you're after. It's an AI developer workflow. Okay, you might be, you know,
|
||
|
|
22:56
|
||
|
|
listening to this and thinking, but you you drew a million loops. Like, isn't this a loop? Fine. If you want to call
|
||
|
|
23:02
|
||
|
|
it a loop, I don't really care. I think a loop is too constrained. If you're going to call it a loop, we're going to need if engineer, we're going to need
|
||
|
|
23:09
|
||
|
|
throw engineering. We're going to need exception engineering, right? We're going to need to name all these things engineering. This is a developer
|
||
|
|
23:14
|
||
|
|
workflow. This used to be what engineers did. Engineers used to decide if something was a chore, if something was
|
||
|
|
23:20
|
||
|
|
a bug. We used to write the plan, we used to execute it, so on and so forth. But now we have a new tool and that's
|
||
|
|
23:26
|
||
|
|
all it is. It's a new tool. We have agents to work with that gives us AI
|
||
|
|
23:32
|
||
|
|
developer workflows. Okay. So at the highest levels of agentic engineering, you're building software factories that
|
||
|
|
23:39
|
||
|
|
execute the right work and the right combination of engineers, agents, and code across your organization. Once you
|
||
|
|
23:45
|
||
|
|
really start to scale it up, you're going to add your other teammates, right? Your other team members from other cross cutting concerns inside of
|
||
|
|
23:52
|
||
|
|
your business. But at the core of it, the engineers are responsible for the code. Okay? I think a lot of orgs are
|
||
|
|
23:58
|
||
|
|
going to have a a problem with this once they start scaling in and adding other team members, right? Especially ones
|
||
|
|
24:04
|
||
|
|
that can't write clear tickets for the life of them. You've seen this a million times, right? It's the most painful
|
||
|
|
24:09
|
||
|
|
thing when your product manager, your your CTO, your your tech lead [laughter]
|
||
|
|
24:14
|
||
|
|
just writes a ticket and you have to translate it, right? So there's there's a lot of like, you know, people
|
||
|
|
24:19
|
||
|
|
organizational level work to be done here. But you at the end of the day, you know, you the engineer plus the agents
|
||
|
|
24:25
|
||
|
|
plus the code making up the AI developer workflow. This is what it's all about. This is where value is going to be
|
||
|
|
24:31
|
||
|
|
created at absurd levels, at absurd scales. Because once you get this right,
|
||
|
|
24:37
|
||
|
|
here's the dirty secret of all software, right? You already know it. Once you get this right, you set up the right
|
||
|
|
24:43
|
||
|
|
guardrails, the right harness, right? Again, prompt, context, harness engineering, all of it. Once you do this
|
||
|
|
24:48
|
||
|
|
right, you have a repeatable workflow that you can run tens, hundreds, and thousands of times, delivering
|
||
|
|
24:54
|
||
|
|
consistent results to you over and over and over again if you template your
|
||
|
|
25:00
|
||
|
|
engineering into the fabric of your AI developer workflows. Okay? And so I've
|
||
|
|
25:05
|
||
|
|
been pushing against out of the box agents for a long time. Um, you know, specialization is the name of the game.
|
||
|
|
25:11
|
||
|
|
What is a product? What is a company? Right? Unless you're a big tech giant, like a product in a company is a a set
|
||
|
|
25:18
|
||
|
|
of people in technology that solve a specific problem for a specific avatar for a specific user for a specific
|
||
|
|
25:24
|
||
|
|
customer. By very definition is specialization, right? Your expertise is
|
||
|
|
25:30
|
||
|
|
the most valuable thing you have now. And you can template that into your engineering. You can template that into
|
||
|
|
25:36
|
||
|
|
your AI developer workflows. All right, this is the greatest leverage point of agentic coding. It's building out these
|
||
|
|
25:43
|
||
|
|
full AI developer workflows that puts it all together. Okay, and so you know once again we are pushing away from vibe
|
||
|
|
25:51
|
||
|
|
coding. This is not vibe coding. Vibe coding is not knowing how the system works and it's not looking at how the
|
||
|
|
25:56
|
||
|
|
system works. Okay, agentic engineering is knowing your system works so well you don't have to look. And that is because
|
||
|
|
26:03
|
||
|
|
you the engineer have moved up a layer. You're meta-engineering. You're compounding an advantage by optimizing
|
||
|
|
26:10
|
||
|
|
the three actors of agentic engineering engineer code agents into the right
|
||
|
|
26:16
|
||
|
|
developer workflow at the right time at the right performance at the right price with the right speed. And after time
|
||
|
|
26:22
|
||
|
|
you'll realize something really important is that you'll be building AI developer workflows into your products
|
||
|
|
26:28
|
||
|
|
for your customers right with your customers as nodes and then as mentioned with your companies every user that can
|
||
|
|
26:35
|
||
|
|
prompt into the system and receive results out the system right you have to design this system it's just another
|
||
|
|
How to Build Great AI Developer Workflows
|
||
|
|
26:40
|
||
|
|
system
|
||
|
|
26:45
|
||
|
|
so I've written hundreds and probably thousands of these AI developer workflows by now. So, let me give you
|
||
|
|
26:50
|
||
|
|
the oil of everything I've learned so far with what I've seen and what I've recommended to engineers as they're
|
||
|
|
26:56
|
||
|
|
building out their ADWs. Um, first off, keep it simple. When you start building these out, start with the simplest
|
||
|
|
27:02
|
||
|
|
workflow you can think of, right? And typically that looks something like this, right? After you get an agent
|
||
|
|
27:07
|
||
|
|
running, you prompt back and forth and you're babysitting your agent. Everyone knows what this looks like. Just let it lent your code. You know, to be clear,
|
||
|
|
27:13
|
||
|
|
so that you can really feel this. Separate this out. I'm not saying write a skill, have your agent build, and then
|
||
|
|
27:19
|
||
|
|
at the bottom of the skill, run lint. Separate this out. Use an agent SDK, run
|
||
|
|
27:24
|
||
|
|
a build agent, do work, and then run a llinter. And when the llinter fails, pass that back into the build agent with
|
||
|
|
27:30
|
||
|
|
the same session ID. You have to separate your code and your agents. Otherwise, you just have an agent
|
||
|
|
27:36
|
||
|
|
calling code. That's not what we want. We want separation of concerns all the way through. If that doesn't make sense
|
||
|
|
27:42
|
||
|
|
to you right now, don't worry. It'll make sense once you start building it. This is not a big skill where you run a
|
||
|
|
27:48
|
||
|
|
hundred different nodes of workflows. There are massive testing, massive validation problems with doing that. And
|
||
|
|
27:54
|
||
|
|
then what do you do after that? You add a couple nodes, right? Start solving real problems. Run your type checker,
|
||
|
|
28:00
|
||
|
|
run your llinter. If things go wrong, funnel it back into your build agent. What you'll notice here is that you're
|
||
|
|
28:05
|
||
|
|
starting to build a larger unit, a larger system that operates without you.
|
||
|
|
28:10
|
||
|
|
You show the beginning and the end. the two constraints of agent coding, planning and reviewing, and your system
|
||
|
|
28:16
|
||
|
|
does everything else. Okay? And then once you get to a certain point, you'll start separating out your agents. You'll
|
||
|
|
28:21
|
||
|
|
start specializing your agents. Maybe you want to separate your front end and your back end. Maybe you want building
|
||
|
|
28:26
|
||
|
|
and testing. Again, the key here is just that you separate the context out so that your context can move between
|
||
|
|
28:33
|
||
|
|
individual agents and code. When you're starting, remember KISS, keep it simple, stupid. You can absolutely start with
|
||
|
|
28:40
|
||
|
|
pure skill-based workflows where it's all one skill outside of the prompt and the review. But as soon as you start
|
||
|
|
28:47
|
||
|
|
productionizing, as soon as you run to get serious about your AWS, you must separate code out of the skills because
|
||
|
|
28:53
|
||
|
|
that's still your agent running it, right? You have to be super super clear about those steps so that you can set up
|
||
|
|
28:58
|
||
|
|
the proper guard rails and information flows in your AI developer workflows. My next really big piece of advice here is
|
||
|
|
Do It by Hand First
|
||
|
|
29:04
|
||
|
|
design your ADWs by doing the work yourself first. For a lot of engineers, this will sound insanely painful, but
|
||
|
|
29:11
|
||
|
|
you can like, you know, use your agent in the terminal. Run the build workflow. Do the testing, right? You can still use
|
||
|
|
29:17
|
||
|
|
your agent for that. I'm not saying do it by hand. That would be a waste of time now. But what I am saying is whatever workflow you're setting up, run
|
||
|
|
29:23
|
||
|
|
it end to end. Step into each node yourself. run the pass, run the condition, watch the functions get
|
||
|
|
29:30
|
||
|
|
executed, do the review, and then do the ship to production and then start writing this all as a combination of
|
||
|
|
29:37
|
||
|
|
agents, engineers, and code. And I recommend you take something like mermaid. And you know, by the way, this is like a adaptation of mermaid diagram.
|
||
|
|
29:45
|
||
|
|
I had an agent create using a plan build test AI developer workflow in one shot. I created a animated application which
|
||
|
|
29:52
|
||
|
|
is of course based on a mermaid diagram. Okay, so shout out to mermaid. Shout out to mermaid.live. But that's my second
|
||
|
|
29:58
|
||
|
|
piece of advice, right? Walk through it all yourself first. Sit down pencil and a piece of paper or use mermaid or use
|
||
|
|
30:04
|
||
|
|
whatever. Really sit down and like write out your workflow. And then lastly, make sure you're not just using agents,
|
||
|
|
Make Sure You're Not Just Using Agents
|
||
|
|
30:11
|
||
|
|
right? Use agents and code. As I mentioned, you can always start with agents and skills, but as soon as you
|
||
|
|
30:18
|
||
|
|
start hitting production, as soon as you want to get serious, move some of that skill work into code. This is not just
|
||
|
|
30:23
|
||
|
|
about token cost. This is about performance, reliability, and speed. Again, everyone in their AI psychosis
|
||
|
|
30:28
|
||
|
|
has like forgotten that speed costs zero tokens. There's no hallucination. It
|
||
|
|
30:34
|
||
|
|
does the exact same thing every time. And it literally runs at the speed of light. So, don't overleverage on agents.
|
||
|
|
30:41
|
||
|
|
Okay? Balance it out with actual code, right? Code execution. And yes, there's
|
||
|
|
30:46
|
||
|
|
information orchestration. There is this is what context engineering is. You're going to need a place for all the
|
||
|
|
30:52
|
||
|
|
results in between each step. Yes, it's going to take some time. Yes, it's going to be a little knowing. Yes, during the
|
||
|
|
30:57
|
||
|
|
process, you'll wonder, I should just throw this all in the skill. You'll be wrong down the road. I can guarantee you that. I've been there. Don't waste your
|
||
|
|
31:04
|
||
|
|
time doing other engineers have already done wrong. Separate it out as you scale this. Okay, so that's the big
|
||
|
|
31:09
|
||
|
|
third tip. Use agents and code. Okay, because agents plus code beats either
|
||
|
|
31:15
|
||
|
|
alone, especially when you start really scaling these into legitimately large AI
|
||
|
|
31:20
|
||
|
|
developer workflows that do serious work for you and your organization. Why? Because you're going to need to test
|
||
|
|
31:26
|
||
|
|
this node. You're going to need to test plan into build. You're going to need to test plan into build and to update the
|
||
|
|
31:32
|
||
|
|
status and to testing and to fail. This is all still a system you the engineer are responsible for. So keep using great
|
||
|
|
31:39
|
||
|
|
classic engineering patterns, isolatable, decoupled, single interface, right? All that stuff matters probably
|
||
|
|
31:47
|
||
|
|
even more, right? It matters even more now because once you do it right and you set up your AI developer workflow, it
|
||
|
|
31:52
|
||
|
|
gets multiplied hundreds and thousands of times and your agents plus your code can drive the outcomes for you. Okay?
|
||
|
|
31:59
|
||
|
|
So, you know, let's step away from the vibes a little bit and let's step out away from the AI psychosis a little bit
|
||
|
|
32:04
|
||
|
|
because if we're going to do serious agentic engineering, you need to know what's going to happen in your system.
|
||
|
|
Tactical Agentic Coding Pitch
|
||
|
|
32:10
|
||
|
|
So, if you made it to the end here, I want to just say thank you for trusting me. For everyone that's been with the channel for a while, for years now, you
|
||
|
|
32:16
|
||
|
|
know, big shout out to you. I appreciate you trusting me and following along this massive journey of Agentic Engineering
|
||
|
|
32:21
|
||
|
|
that we're on. Um, if you want more, I recommend you check out agenticengineer.com, specifically
|
||
|
|
32:27
|
||
|
|
tactical agentic coding. As I mentioned, I've been pretty early to a lot of these ideas. In tactical agent coding, you're
|
||
|
|
32:34
|
||
|
|
going to hear a lot of what I just said really broken down step by step across eight lessons and then six additional
|
||
|
|
32:41
|
||
|
|
upgradable lessons if you're interested. Okay, so the big idea here is AI developer workflows. It's building
|
||
|
|
32:47
|
||
|
|
systems that build systems. We're not touching the application layer anymore. We're touching the agentic system, the
|
||
|
|
32:53
|
||
|
|
agentic layer that builds it on our behalf. Okay? So, if you want to pay to play, you want to get a big advantage
|
||
|
|
32:59
|
||
|
|
that again thousands of engineers that you know of and have heard of at companies you know the names of, they
|
||
|
|
33:05
|
||
|
|
are in here and they have gotten the advantage and they are getting the advantage, right? It's tactical agent coding is the first eight lessons you
|
||
|
|
33:11
|
||
|
|
can upgrade to agentic horizon to get some upgradeable ideas. really big idea there is of course multi- aent
|
||
|
|
33:16
|
||
|
|
orchestration but agent experts is turning out to be a massively banger idea a massively important idea for
|
||
|
|
33:23
|
||
|
|
engineering in the age of agents if you want to build true specialists that outperform out of the box agent anyway
|
||
|
|
33:29
|
||
|
|
lots more in there's a very clear 30-day refund before you start lesson 4. So if
|
||
|
|
33:34
|
||
|
|
you don't like my style or you're not getting the core of it it's fine I don't want you in here if you don't want to be
|
||
|
|
33:39
|
||
|
|
30-day refund before you start lesson 4. This is going to be linked in the description for you. Also, I recommend if you vibe with the ideas here, if you
|
||
|
|
33:46
|
||
|
|
understand the ideas and you don't want to jump in the tactical agent coding right away, check out this blog. I'll
|
||
|
|
33:51
|
||
|
|
link in the description as well, thinking in threads. It covers a lot of the same ideas we've been discussing 99%
|
||
|
|
33:58
|
||
|
|
of everything I do here on this channel. It's all free. It's out there for you to understand and master agentic
|
||
|
|
34:04
|
||
|
|
engineering. If you made it to the end, do me a favor, like this video, leave a comment, and share it with your coworker
|
||
|
|
34:09
|
||
|
|
before your competition sends it to theirs. You know where to find me every single Monday. Stay focused and keep
|
||
|
|
34:16
|
||
|
|
building.
|