What is Agent Where a Chain Would Do?
Agent Where a Chain Would Do is an AI code smell, a sign of trouble in LLM systems, in the Control-flow abusers group. The path never changes, yet an autonomous agent decides the order on every run.
Trail content
Chapters
Transcript
Opening
A marketplace moderates its listings with an agent. It gets a listing, four tools, check text, check image, check price, and publish the decision. And a prompt that says: review the listing and decide.
Here are two traces from the same day. The first one looks perfect. Text, image, price, decision. Four turns, about eleven seconds.
The second one is faster. Text, and then the model writes: the image looks consistent with the text. And it approves. The image check never ran.
Look at that sentence again. The model gave an opinion on an image it never analyzed. And listing eighty-eight two fourteen had a banned image.
Nobody on the team wanted the image check to be optional. The rule was clear: every listing goes through all three checks. But that rule lived in a prompt. And a prompt is a strong suggestion, not a guarantee. In this lesson, you'll learn to spot this smell in your traces, and the refactoring that removes it for good.
The diagnosis
Here's what's going on. Today, on every turn, the model picks which check to run next. Or whether to stop.
So nothing guarantees that every check runs.
But the task has a fixed path. In a chain, code always runs all three checks. And the model only steps in where judgment is actually needed: weighing the results and deciding.
How do you recognize it? You can draw the task's flowchart, and it's always the same.
The agent's prompt lists the steps in order. First do this, then that. If you're writing the order in the prompt, you already know the order.
The traces show different orders, skipped steps, or repeated steps, for similar inputs.
The number of turns varies a lot, with no business reason.
And when it goes wrong, the explanation is: the agent thought it wasn't needed.
Why it happens
Agents became the default. Frameworks and tutorials start with agents, and they look more modern than a sequence of functions.
The demo convinces. In ten tests, the agent did everything in the right order. The variation shows up in the thousands of runs.
Flexibility for the future. What if someday we want to skip the price check for some categories? Flexibility that might be useful someday costs reliability every day.
And bundling everything looks simpler. One prompt and four tools look like less code than four chained functions. The code disappears. The complexity stays, hidden in the model's behavior.
What it costs
The cost starts with reliability. Steps that were supposed to be required quietly become optional. In moderation, compliance or finance, a skipped step is an incident.
And small rates become real volume. Say the marketplace moderates fifty thousand listings a day. If the agent skips the image check on just zero point one percent of runs, that's fifty listings a day published with nobody looking at the photo.
In a month, one thousand five hundred. A rate that looks great in the eval becomes real volume in production.
Then money. Every agent turn resends the whole context, and pays for one more call to decide something the code already knew. In that perfect trace, three of the four model calls were only there to pick the next tool.
Then latency. The checks run one after another, waiting for the model to choose. They could run at the same time.
And testing. A fixed flow is easy to test. An agent needs a statistical eval just to prove it always does the same thing.
A challenge
Here's one for you. To guarantee the image check, the team wrote, in capital letters: always run check image. The skip rate fell from two percent to zero point three. Is the problem solved?
No. The guarantee is still statistical. As long as the model decides, the step can be skipped. At fifty thousand listings a day, zero point three percent is one hundred fifty listings with no image check.
And a better model won't get you to zero either. It lowers the rate again, but the problem is the design, not the model's ability. A required step has to be code. Then the rate is zero, by construction. Louder prompts also push you toward another smell, rule stacking, where all capital rules pile up and fight each other.
The treatment
The treatment is a refactoring called replace agent with chain. And the chain version of the moderator fits in a few lines.
The three checks don't depend on each other, so code runs all three at the same time.
Then the model is called once, with the three results, to weigh them and decide.
Now, skipping the image takes deleting a line of code, with review and tests. The model stays where it adds value. And latency becomes the slowest check, plus one model call, instead of four calls in a row.
Then the team learns that vehicle listings need a fourth check, on the car's registration document. The other categories keep three. Should you go back to the agent, since the path now varies?
No. The path varies, but in a way you know before the run. That's a simple if on the category, or a separate route for vehicles. Known branches are routing. And each branch is still a chain, with every step guaranteed. Writing it into the prompt would just bring the same smell back, now on one branch.
When to ignore
This smell isn't a rule against agents. When the path really depends on what you discover, an agent is the right tool. A support agent gets: the system is slow. Based on what it finds, it decides whether to look at logs, metrics, recent deploys or the database. That's not this smell. That's open investigation, the classic agent case. What it needs is a step budget and stop conditions.
An agent can also be a good prototype, to discover which steps a task needs. When the traces start repeating, turn the path into code.
And if skipping a step is truly acceptable, and cheap, the flexibility may be worth it.
One last case. A report agent always calls the same three tools, in the same order, and never skipped one in six months of traces. Is it still worth refactoring? Yes, but with no rush. Its behavior is already a chain. Turning it into code cuts cost and latency, and trades a statistical guarantee for a real one. Do it with the next change to that flow.
Wrap-up
So here's the idea to take with you. If you can write the steps in the prompt, you can write them in code. And in code, a required step runs every single time. Keep the model for judgment, and keep the order for your code.
In the next lesson, we close this trail with a real system that puts everything together: a support bot with human escalation. See you there.
Agent Where a Chain Would Do
Agente Onde Bastava uma Chain
Agent Where a Chain Would Do is an AI code smell, a sign of trouble in LLM systems, in the Control-flow abusers group. The path never changes, yet an autonomous agent decides the order on every run.
- Element
- 10/37
- Level
- Intermediate
- Read
- 4 min
- Updated
Signs and symptoms
A marketplace’s listing moderation pipeline runs as an agent: it receives a listing, has four tools (check_text, check_image, check_price, publish_decision), and a prompt that says “review the listing and decide”. Two traces from the same day, side by side:
trace a91f listing 88213
llm -> check_text ok
llm -> check_image ok
llm -> check_price ok
llm -> publish_decision approved
4 turns, 11.2 s
trace b07c listing 88214
llm -> check_text ok
llm "The image looks consistent with the text."
llm -> publish_decision approved
2 turns, 5.8 s <- check_image never ranListing 88214 had a banned image. On that run, the agent decided that looking at the image was unnecessary. Notice the sentence in the second trace: the model gave an opinion on an image it never analyzed.
Nobody on the team wanted the image check to be optional. The business rule was “every listing goes through all three checks”. But that rule lived in a prompt, and a prompt is a strong suggestion, not a guarantee.
Signs of the smell:
- You can draw the task’s flowchart, and it is always the same.
- The agent’s prompt lists the steps in order (“first do X, then Y”).
- Traces show different orders, skipped steps, or repeated steps for similar inputs.
- The number of turns varies a lot with no business reason.
- When it goes wrong, the explanation is “the agent thought it was not needed”.
flowchart LR
subgraph AG["Today: the agent decides"]
A(["1. Model picks the next step"]):::llm --> T["2. Some tool"]:::tool
T --> A
end
subgraph CH["Enough: a chain"]
C1["3. Text"]:::tool --> C2["Image"]:::tool --> C3["Price"]:::tool --> C4(["Decision"]):::llm
end
- Today, on every turn, the model picks which check to run, or whether to stop.
- Nothing guarantees every check runs.
- The task has a fixed path. In a chain (a fixed sequence of calls), code always runs all three checks, and the model only steps in where judgment is needed.
The distinction comes from Anthropic’s “Building effective agents” guide (Dec. 2024). Workflows follow paths defined in code. Agents let the model direct its own process. The guide recommends starting with the simplest solution and using an agent only when the path cannot be predicted.
Reasons
- Agents became the default. Frameworks and tutorials start with agents, and they look more modern than a sequence of functions.
- The demo convinces. In ten tests, the agent did everything in the right order. Variation shows up in the thousands of runs.
- Flexibility “for the future”. “What if someday we want to skip the price check for some categories?” Flexibility that might be useful someday costs reliability every day.
- Bundling everything looks simpler. One prompt and four tools look like less code than four chained functions. The code disappears, but the complexity stays, hidden in the model’s behavior.
Cost of ignoring
- Reliability. Required steps become optional. In moderation, compliance, or finance, a skipped step is an incident.
- Cost. Every agent turn resends the whole context and pays for an extra call to decide something the code already knew.
- Latency. Checks run one after another, waiting for the model to choose the next, when they could run in parallel.
- Testability. A fixed flow is easy to test. An agent needs a statistical eval to prove it always does the same thing.
The math of a small rate
Suppose the marketplace moderates 50 thousand listings a day. If the agent skips the image check on 0.1% of runs, that is 50 listings a day published with nobody looking at the photo. In a month, 1,500. A rate that looks great in the eval becomes real volume in production.
In trace a91f, the full path cost four model calls in series, three of them only to pick the next tool. In the chain, the three checks run in parallel in code and the model is called once, for the decision. Latency becomes that of the slowest check plus that one call.
The common mistake: louder prompts
The first reaction is usually to write “ALWAYS run check_image” in capitals. The skip rate drops, and the team calls it solved. It never reaches zero, because the model is still deciding. And every instruction like that pushes the prompt toward the Rule Stacking smell.
Treatment
| Refactoring | Choose it when |
|---|---|
| Replace Agent with Chain | The path is fixed. Steps become code, and the model stays only in the steps that need judgment, as in Prompt Chaining. |
The chain version of the moderator fits in a few lines. The checks are independent, so they run together:
async def moderate(listing):
text, image, price = await asyncio.gather(
check_text(listing), check_image(listing), check_price(listing),
)
decision = await llm(DECISION_PROMPT, listing=listing, checks=[text, image, price])
return publish_decision(listing.id, decision)Now skipping the image takes deleting a line of code, with review and tests. The model stays where it adds value: weighing the three results and deciding.
If the path has a few known branches (“vehicle listings get an extra check”), the final shape is a workflow with Routing, not an agent. If the checks do not depend on each other, as here, run them in parallel with Sectioning.
Payoff
- Every required step runs, every time.
- Fewer model calls per run.
- Independent checks can run in parallel.
- Similar traces for similar inputs, easy to compare.
- Regular tests covering the flow, and evals only on the judgment steps.
When to ignore
- The path really depends on what you discover. Investigating a bug, researching an open topic, exploring an unknown site: there an agent is the right tool. See Chains, agents, and workflows.
- A prototype to discover the path. Sometimes an agent is useful to explore which steps a task needs. When the traces start repeating, turn the path into code.
- Truly optional, low-risk steps. If skipping a step is acceptable and cheap, the agent’s flexibility may be worth it.
Is it this smell?
Lesson check
Answer the 6 questions to complete this lesson. Watching alone does not complete it: the check is what counts toward the trail and the certificate.
Quick answers
How do you fix Agent Where a Chain Would Do?
With Replace Agent with Chain. Each refactoring has safe step-by-step mechanics with an eval before and after.
Where does Agent Where a Chain Would Do show up in practice?
In the real-world scenarios Content moderation.
Go further
Relations
- Refactorings that treat it 1
- Shows up in scenarios 1
- Part of the tracks 1