Prompt Chaining vs Agent Loops: Choosing the Right Control Pattern

Teams reach for “an agent” when they really need a sequence of LLM calls, and they force a rigid chain when the next step truly depends on what a tool just returned. Prompt chaining and agent loops are both valid control patterns. They answer different questions about predictability and governance. Choosing wrong is expensive: chains that cannot adapt stall; loops that need not exist burn tokens and create audit fog.

Two control patterns, one decision#

Prompt chaining is a fixed (or lightly branched) sequence of model stages. Stage N’s output becomes stage N+1’s input. The graph is mostly known at design time. Humans can draw it on a whiteboard and point to where policy, retrieval, or validation sits.

Agent loops let the model (inside a harness) decide the next action from observations: call a tool, ask a clarifying question, finish, or escalate. The path is discovered at runtime within budgets you define.

Neither is “more AI.” One is more scheduled. The other is more adaptive.

What prompt chaining is good for#

Use a chain when the business process is already a pipeline:

• Classify intent → retrieve policy → draft response → validate schema → optionally human-approve
• Extract fields → normalize → enrich → write to system of record
• Translate → tone-check → compliance scan → publish

Why it works: each stage has a clear contract. You can swap models per stage, cache early stages, and put deterministic validators between probabilistic ones. Latency is roughly the sum of stages — boring, which is a feature. Auditors like chains because the path is explainable without replaying a model’s private reasoning.

Where chains break: the next stage depends on open-ended discovery (“search until you find the right document family”), or recovery requires inventing new steps the designer did not anticipate. Bolting a mini-loop onto every stage is how chains quietly become unmaintainable agents.

What agent loops are good for#

Use a loop when observation changes the plan:

• Investigative support: search ticket history, then CRM, then knowledge base, stop when evidence is enough
• Ops triage: try a diagnostic tool, interpret, choose the next diagnostic
• Research assistants: gather sources, notice gaps, fetch more, then draft

Why it works: you refuse to pretend the world fits a five-box flowchart. The harness still enforces allowlists, schemas, and budgets — the loop is not “the model does whatever.”

Where loops break: the task was actually a known pipeline, but someone wanted the word “agentic” on a slide. Unbounded or weakly bounded loops thrash tools, invent duplicate refunds, and produce trajectories nobody can review in an hour.

Decision criteria at a glance#

If three or more rows point to chaining, do not start with a loop “for flexibility.” Flexibility you do not need is liability you will debug.

A concrete contrast#

Same product goal: answer a billing dispute email.

Chain version

1. Classify dispute type (billing error, duplicate charge, service credit)
2. Retrieve the matching policy snippet and account facts
3. Draft a reply with structured fields (amount, reason code)
4. Validate schema and tone rules
5. If money movement is proposed → human queue; else send

Loop version

Observe case → decide among {search_ledger, search_tickets, ask_user, draft_reply, escalate} → act → verify → repeat until an exit fires.

Both can be correct. The chain wins when dispute types and evidence sources are well mapped. The loop wins when evidence location is genuinely unknown and agents must hunt. Many enterprises discover, after a painful loop pilot, that 80% of volume was chainable and only the ugly 20% needed a bounded loop behind a router.

Governance differences that matter#

Chaining makes stage ownership natural: security owns the compliance scan stage; ops owns the human approval stage. You can disable a stage without rewriting the world.

Loops make tool policy the centre of gravity. The audit artifact is the trajectory: which tools, which arguments, which observations. Your observability stack must treat trajectories as first-class — not optional debug logs.

If legal asks “show me every path this system can take,” a chain has a finite answer. A loop’s honest answer is “any path inside the allowlist and budget.” That may be acceptable. Pretending a loop is as enumerable as a chain is how reviews fail late.

Latency, cost, and ops#

Chains are easier to capacity-plan: p95 ≈ sum of stage p95s plus validators. Loops need envelopes: max steps, max tool failures, max spend. Without those numbers, finance and SRE will invent them for you during an incident.

from dataclasses import dataclass

@dataclass(frozen=True)
class ControlChoice:
    pattern: str  # "chain" | "loop"
    reason: str

def choose(predictable: bool, needs_discovery: bool, audit_fixed_path: bool) -> ControlChoice:
    if audit_fixed_path and predictable:
        return ControlChoice("chain", "Explainable stages beat open trajectories")
    if needs_discovery and not audit_fixed_path:
        return ControlChoice("loop", "Next step depends on observations")
    if predictable and not needs_discovery:
        return ControlChoice("chain", "Do not pay loop tax for a pipeline")
    # Hybrid: router → chain for common; loop for tail
    return ControlChoice("loop", "Tail cases need discovery; isolate behind a router")

That sketch is intentionally small. The judgment lives in the boolean inputs — product and risk must set them, not the framework default.

When a hybrid is allowed#

The useful hybrid is router → chain for the common path, loop for the long tail, not “chain stages that each contain unbounded agents.” Keep the hybrid visible in architecture diagrams. If reviewers cannot see where the loop starts and stops, you will not be able to eval it cleanly either.

Related reading in your own stack: loop engineering (budgets and exits) and core AI design patterns (Router, ReAct, Planner–Executor). This article’s job is narrower: pick the control family before you pick the brand of graph library.

Summary#

Prompt chaining schedules work; agent loops discover work. Prefer chains when the pipeline is known and governance wants a fixed path. Prefer loops when observations must change the next action — and only with explicit exits and budgets. Default to the simpler pattern. Earn the loop.