The word "agent" is hiding a decision

Ask five people what an "agent" is and you will get five answers, and they will all be right, which is exactly the problem. The word has become a bucket for everything from a glorified if statement to a fully autonomous system that books your flights. When a word can mean anything, using it in an architecture decision is a form of cheating.

I have been on the receiving end of this. Someone comes to me with a plan to "build an agent" and the plan turns out to be a script with a loop. Or they have spent two months evaluating agent frameworks for a task that needed one prompt. The research team at Anthropic said it plainly in their essay on building effective agents, and it matches what I have seen in practice: the successful systems use simple, composable patterns, and the difference between a workflow and an agent is not marketing, it is architecture.

Workflows and agents are different things

A workflow is a system where the code decides the path. The model is called at fixed points to do a fixed job. Maybe step one classifies the request, step two writes a draft, step three critiques it. The order is decided by a developer and encoded in code.

An agent is a system where the model decides the path. You give it tools, you give it a goal, and it chooses which tools to call, in what order, and when to stop. The code does not know the route in advance, because the route depends on what the model discovers along the way.

This distinction matters because they have completely different risk profiles. A workflow is predictable. You can test it, you know how long it takes, you know what it costs. An agent is a negotiation with a stochastic system. It can take ten steps or a hundred. It can decide to do something you never anticipated. The latency and cost are unbounded unless you bound them.

Here is the rule I actually use: if you can predict the steps, it is a workflow, and it should be a workflow. Reach for an agent only when the task is open ended enough that a fixed path would be a fiction.

Start with less

The most common mistake is starting at the top of the complexity ladder. Before any agent framework, before any orchestration library, ask whether a single model call does the job. For a shocking number of features, the answer is yes. One call, with good retrieval and a few examples in the prompt, beats a three agent dance every time, because there is nothing to go wrong.

If a single call is not enough, add structure a little at a time. The useful middle ground is a collection of small, named patterns. Prompt chaining, where each call passes its output to the next, with a programmatic check between steps. Routing, where a cheap model classifies the request and sends it to the right specialist prompt, and where easy questions can go to a small model and hard ones to a big one. Parallelization, where independent subtasks run at the same time and the results are merged. An evaluator loop, where one call produces and another critiques, iterating until the critique passes.

These are boring names for boring patterns, and that is the point. Each one is testable. Each one has known cost. You can reason about where it fails. The framework will sell you all of them at once with a fancy vocabulary, but you get the same result from a few functions, and you understand the result.

The tools are the real interface

Here is what changed my mind about where the engineering effort goes. When I built my first tool using system, I spent ages on the prompt and almost no time on the tool definitions, and the results were bad. The model kept calling the tools wrong. Then I flipped the effort. The prompt got simpler, the tool definitions got careful, and everything got better.

The Anthropic team calls this the agent computer interface, and it is the right framing. A tool definition is documentation for a reader that has never seen your API. Write it like you are documenting for a junior developer. Describe what the tool does, what the arguments mean, when to use it and when not to, what the edge cases are. Give examples. Make the mistakes impossible: if a tool fails on relative paths, require absolute paths. If two tools are easy to confuse, give them obviously different names and descriptions.

The model is doing its part of the reasoning with the words you give it. If your tool names are ambiguous, the model is guessing. I have stopped thinking of tool definitions as boilerplate and started thinking of them as the actual product surface of the system.

Bounded autonomy is not cowardice

People hear "agent" and imagine an unbounded loop that runs until success. In production, unbounded is not a feature, it is a liability. Every agent loop needs stopping conditions: a maximum number of iterations, a budget, a timeout, a point where it must come back to a human. The first version of a system I worked on could, in theory, retry forever on a tool that would never succeed. In theory is where these systems live before they meet real traffic.

The other guardrail that earns its keep is a checkpoint where the agent pauses for a human when it is about to do something consequential. Not for every step, but for the ones where being wrong is expensive. A draft email can loop freely. Sending the email should be a human decision. The systems that survive are the ones where the autonomy is sized to the blast radius of a mistake.

Measure, then trust

An agent is a system with many moving parts, which means it needs the same discipline as any other system: a test set and a way to measure regressions. The tricky part is that agent behavior is not deterministic, so your tests cannot be exact snapshots. They need to assert on outcomes. Did the tool get called with the right arguments for this input? Did the loop terminate in the budget? Did the final output contain the required fields? Did it not call the destructive tool?

You also need logging that lets you replay what happened. When an agent misbehaves in production, you need the full trace: every model call, every tool call, every decision in between. A system where you cannot reconstruct the sequence of decisions is a system you cannot debug, and debugging agents without traces is like debugging multithreaded code without a log. The trace is not a nice to have. It is the minimum.

The pattern is the point

Strip away the hype and an agent is a loop: model picks a tool, tool runs, result goes back to the model, repeat, until done. The entire craft is in the details around that loop. The quality of the tool definitions. The stopping conditions. The evaluation set. The traces. The decision about whether you needed a loop at all.

The next time someone proposes an agent for a feature, ask them what problem the loop solves. If the answer is "it seemed more agentic that way", you have saved everyone a month. If the answer is a concrete failure mode that a fixed path cannot handle, build the loop. Start small, make the tools legible, bound the autonomy, and measure everything. The word will stay muddy, but your architecture will not.