# AI Agent Control Flow Patterns [2026]: Retries, HITL, Checkpoints

> A practical catalog of ai agent control flow patterns that actually ship: bounded retries, idempotent tools, durable checkpoints, human approvals, and debuggable replays.

- Canonical: https://www.kunalganglani.com/blog/ai-agent-control-flow-patterns
- Author: Kunal Ganglani
- Published: 2026-08-02 · Updated: 2026-08-02
- Category: AI and Machine Learning · Tags: ai-agents, agent-architecture, langgraph, reliability, production-ai

## TL;DR

AI agents break in predictable ways: they retry the wrong thing, double-execute side effects, or lose progress when a run crashes. The fix isn’t “better prompts”. It’s control flow. Treat the agent run like a workflow with explicit states, bounded retries, durable checkpoints, and human approvals where risk is real. Make every side-effecting tool idempotent so retries don’t create duplicate refunds, tickets, or emails. Then add budgets and circuit breakers to prevent runaway costs, plus correlation IDs so you can replay and debug failures. That’s how agent demos become production systems.

AI agent control flow patterns are the reliability primitives that turn a flashy agent demo into something you can put behind a pager. If your agent can’t retry safely, pause for a human approval, or resume after a crash, you don’t have an “agent”. You have an expensive while-loop.

**Key takeaways**

- Reliable agents borrow from workflow engines: explicit states, bounded retries, and durable checkpoints.
- Retrying is only safe when your tools are idempotent and you store tool receipts in your checkpoint.
- Human-in-the-loop steps should be modeled as first-class states with timeouts, SLAs, and an audit trail.
- You prevent runaway cost with max steps, budgets, and circuit breakers, not with “better prompts”.
- Debuggability comes from correlation IDs, structured error names, and replayable run logs.
> If your agent can’t resume from a checkpoint, it’s not autonomous. It’s fragile.

I’m writing this in 2026 because the bar quietly moved. We’re not doing “one chat turn, one response” anymore. We’re doing long-lived runs that touch real systems. Payments. Tickets. Customer data. Anything that can wake you up at 2 a.m.

Frameworks like LangGraph and CrewAI have been inching toward durable execution and interrupt/resume primitives. Good. That’s not a nice-to-have. It’s the only way this stuff survives contact with production.

Based on the benchmark data I maintain at [kunalganglani.com/llm-benchmarks](/llm-benchmarks), the best teams already treat agent runs as workflows, not chats. The performance arguments are real, but the bigger win is operational. When something inevitably fails, you don’t throw away the entire run and start over like it’s 2023.

At a meta level, running this blog’s multi-agent publishing pipeline (7 agents, deterministic SEO quality gate, idempotent publishing steps) taught me a boring truth: deterministic gates and resumable steps catch more failures than “just use a bigger model”. Same story with production agents. Intelligence helps. Control flow is what keeps you out of trouble.

## What I mean by “ai agent control flow patterns”

When I say **ai agent control flow patterns**, I mean the reusable ways you structure an agent run like a state machine.

![a computer screen with a bunch of data on it](https://cdn.sanity.io/images/vzekdneq/production/2b38d84b2153e8a9c17b65fad1a001b5fcce4cf4-1200x675.webp)

What happens on success. What happens on failure. Which failures get retries. Where you branch. What you persist so the run can pause, crash, resume, and still do the right thing.

If you’ve used AWS Step Functions, Temporal, or any workflow engine, you already know the shape of this. The same primitives show up everywhere:

- **Retry** with backoff and a max attempt count
- **Catch** and route to a fallback state
- **Timeouts** and explicit cancellation
- **Durable checkpoints** so execution can resume after crashes
- **Human approval** as a first-class state
Most agent tutorials still start with prompts and tool calling. That’s backwards. Here’s the catalog view I wish they started with.

| Pattern | When you use it | The failure it prevents |
| --- | --- | --- |
| Bounded retries (with backoff) | LLM calls, flaky APIs, transient errors | Infinite loops, runaway costs, thundering herds |
| Catch + fallback state | When “best effort” is acceptable | Whole-run failure for a recoverable step |
| Durable checkpoints | Long runs, multi-step tool use | Losing progress on crashes/timeouts |
| Idempotent tools | Any side-effecting call | Double charges, duplicate tickets/emails |
| Human-in-the-loop (HITL) | Risky actions, approvals, exceptions | Silent bad automation, compliance issues |
| Compensation (Saga-style) | Multi-step side effects | Partial completion leaving systems inconsistent |
| Circuit breaker + budget | Cost control, high error rates | “Agent went brrrr” bills |
| Replay + correlation IDs | Debugging, postmortems | “Works on my machine” agent runs |

Now let’s map these patterns to how you actually build LangGraph/CrewAI-style agents.

## Agents, Flows, Tasks & Processes: stop thinking “chatbot”

I keep seeing teams build “agents” the way they built chatbots. A blob of instructions, a couple tools, and vibes.

![a computer screen with a bunch of data on it](https://cdn.sanity.io/images/vzekdneq/production/2b38d84b2153e8a9c17b65fad1a001b5fcce4cf4-1200x675.webp)

That works until the first time you need to pause for approval, recover from a crash, or explain why the agent emailed a customer the wrong thing. Then you realize you didn’t build an agent. You built a conversational script with side effects.

CrewAI is explicit about the shift. In their docs they describe **Flows** as orchestrations with start/listen/router steps that **manage state, persist execution, and resume long-running workflows** ([CrewAI Documentation](https://docs.crewai.com/concepts/process/)). That’s not fluff. That’s the contract.

[Watch: LangGraph interrupt: Making it easier to build human-in-the-loop agents](https://www.youtube.com/watch?v=6t7YJcEFUIY)

Here’s the vocabulary that matters:

- **Agents**: the reasoning units (LLM + tools + memory) that decide what to do.
- **Tasks**: discrete units of work you can run, retry, and observe.
- **Processes**: how tasks are sequenced (sequential vs hierarchical vs hybrid).
- **Flows**: the control plane. Routing, persistence, resumability, triggers, HITL.
If you’re building serious [AI agents](/pillars/ai-agents), flows are where the reliability story lives. The agent can be “smart”. The flow needs to be *safe*.

A concrete example.

“Draft a customer refund email” is an agent task.

“Send a refund email and issue a refund” is a flow, because it contains irreversible side effects.

## Error names: you need a taxonomy, not a blob of text

One of the best ideas in AWS Step Functions is also one of the most boring: **error names**.

![Red and blue buttons on a control panel.](https://cdn.sanity.io/images/vzekdneq/production/32b71ee1aa8aec9f3a78be8fe8ed9b2daf67cbb5-1200x675.webp)

Step Functions identifies errors using **case-sensitive strings**, with built-in ones that start with `States.`. It also allows custom error names, but they **cannot** begin with `States.` ([AWS Step Functions Developer Guide](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html)).

That tiny rule forces discipline. You can’t just throw a stack trace at the wall and hope your retry logic figures it out.

In agent systems, I like a three-layer taxonomy:

1. **LLM errors**: model timeouts, overloaded, safety refusals.
1. **Tool errors**: validation errors, auth errors, rate limits, upstream 500s.
1. **Control-flow errors**: budget exceeded, max steps exceeded, checkpoint missing.
Give them names that are stable, searchable, and actionable:

- `llm.timeout`
- `llm.refusal`
- `tool.validation_failed`
- `tool.rate_limited`
- `tool.upstream_5xx`
- `flow.budget_exceeded`
- `flow.max_steps_exceeded`
This isn’t bureaucracy. This is the switchboard for your system. It’s how you decide whether to retry, fallback, page a human, or dead-letter.

## Retrying after an error (without turning your agent into a fraud engine)

AWS Step Functions defaults to failing the entire state machine when a state reports an error, unless you configure advanced error handling like Retry/Catch ([AWS Step Functions Developer Guide](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html)). That’s the correct default.

Retries are not “free reliability”. Retries are *duplicated work*. If you haven’t designed for them, they’re how you double-charge a customer and then write a very sincere postmortem.

### Rule #1: retries are for transient failures

Retry on:

- timeouts
- rate limits
- network failures
- upstream 5xx
Do not retry on:

- validation errors (bad args)
- auth errors
- “human said no”
- tool precondition failures
If you want a heuristic that holds up in practice: if the exact same request would succeed 30 seconds later, it’s retryable.

### Rule #2: separate LLM retries from tool retries

LLM calls and tool calls fail differently, and pretending they’re the same is how you get weird behavior.

- **LLM retries** often succeed on the second attempt, but the output can drift. Keep retry counts low (2–3), and prefer structured outputs so the arguments don’t mutate while you’re “just retrying”.
- **Tool retries** are where the real danger lives, because tools have side effects. You can do more attempts (3–5) only if the tool is idempotent.
### A safe retry recipe (5 steps)

1. **Classify the error** into a stable error name.
1. **Check the retry policy** for that error name (max attempts, backoff).
1. **Rehydrate from a checkpoint** so you retry from a known state.
1. **Call the tool with an idempotency key** (details below).
1. **Persist the tool receipt** (request + response + side-effect identifiers) to the checkpoint.
This is the difference between “retry” and “re-execute and hope”.

## Fallback states: graceful degradation beats perfect answers

Step Functions supports Catch blocks that route you to fallback states for **Task, Parallel, and Map** states ([AWS Step Functions Developer Guide](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html)). In practice, it’s just another edge on the graph.

In agent land, fallback states are where you stop pretending you can always be perfect and start being a product.

Examples that actually ship:

- If the web search tool fails, fall back to “answer from cached knowledge base” and label the response.
- If the CRM write fails, fall back to “create a human ticket” and stop.
- If the model refuses, fall back to a smaller / more permissive policy model for harmless content.
The key: fallback states should produce *observable outcomes*. Not “agent apologizes”. Apologies don’t fix incidents.

I like to treat each fallback as a different “exit type”:

- `completed`
- `completed_with_degradation`
- `needs_human`
- `failed_permanently`
Those exit types become metrics. And metrics become decisions.

## State machine examples using Retry and Catch (agent-graph edition)

Here’s a production-ish state machine for “issue refund”:

1. **Validate request** (no retry)
1. **Fetch order** (retry on rate limits)
1. **Decide refund amount** (LLM; retry 1–2 times)
1. **Human approval** if amount > $200
1. **Issue refund** (tool; idempotent + retry)
1. **Send email** (tool; idempotent + retry)
1. **Write audit log** (tool; retry)
Catch routes:

- On `tool.validation_failed` → `failed_permanently`
- On `tool.rate_limited` after max attempts → `needs_human` (or dead-letter)
- On `flow.budget_exceeded` → `needs_human`
Conceptually, this is Step Functions. Your “Task” might be an LLM call, a tool call, or an entire subgraph. The control flow doesn’t care.

If you want the mental model closest to real production reliability, Temporal’s positioning is dead-on: workflows should resume “exactly where they left off after crashes, network failures, or infrastructure outages” ([Temporal Platform Documentation](https://docs.temporal.io/)). That’s the bar. Everything else is a demo.

## Durable checkpoints: what to persist so you can resume (and trust the resume)

Durable checkpoints are the dividing line between “agent demo” and “agent system”. Without them, every transient failure becomes a full restart.

That’s wasted cost. It’s also more chances for the agent to take a different path the next time and create new problems.

### What to checkpoint (practical list)

Checkpoint at *state boundaries*, not after every token.

Persist:

- **Run identifiers**: `run_id`, `correlation_id`, `user_id` (if applicable)
- **State machine position**: current state name, step index
- **Inputs**: validated input payload (post-normalization)
- **Tool receipts**: request params, response payload, side-effect IDs (ticket ID, payment ID)
- **Decisions**: the agent’s chosen plan or next action in a structured form
- **Budgets**: remaining token budget, remaining tool budget, max steps remaining
Avoid persisting:

- raw chain-of-thought (for both safety and determinism reasons)
### Checkpoint frequency

A good default is **one checkpoint per tool call** plus major state transitions.

If you do 20 tool calls in a run, expect ~20 durable writes. That’s fine. Storage is cheaper than reruns. And if you’re worried about the storage bill, wait until you see the “agent restarted itself 14 times” bill.

### Schema evolution (the part everyone ignores)

If your checkpoint schema is JSON, it will change. Plan for it.

- Version your checkpoint schema (`schema_version: 3`).
- Write migrations for “resume” paths.
- Treat unknown fields as forward-compatible.
I learned the hard way that identity is a one-way door. In one incident on this blog, rewriting slugs on live URLs burned 907K impressions of link equity. Checkpoint IDs are the same kind of identity. Don’t casually rewrite them.

## Idempotent tool design for agents (at-least-once is reality)

Agents run in distributed systems. The only honest execution guarantee you get is **at-least-once**.

If your tool call times out, you don’t actually know whether the side effect happened. Retrying without idempotency is how you double-refund a customer and then spend the next day doing cleanup.

### The idempotency contract

Every side-effecting tool should accept:

- `idempotency_key`: stable per logical operation
- optionally `request_hash`: hash of normalized request to detect key reuse with different payloads
And every tool should return a **receipt**:

- `status`: created/already_exists
- `external_id`: payment ID, ticket ID, message ID
- `raw_response`: what the upstream returned
Then store that receipt in your checkpoint.

### Choosing the idempotency key

Two practical schemes that cover most cases:

- `run_id + state_name` for “exactly one call per state per run”
- `business_id + action + version` for “dedupe across runs” (e.g., order_id + refund)
The second one matters when users can re-trigger flows. Which they will.

### Exactly-once is a lie (so design for dedupe)

You can approximate exactly-once with careful dedupe storage. Operationally, assume:

- retries happen
- tasks get re-delivered
- workers crash mid-flight
Temporal leans into this: it gives you resumption and deterministic replay, but activities are still something you design to be idempotent. That’s not a weakness. That’s reality.

## Human-in-the-loop agent workflow (pause, resume, audit)

Human-in-the-loop (HITL) is not a hack and it’s not a “UI feature”. It’s a state.

LangGraph has an explicit interrupt mechanism designed for exactly this. Here’s the official explanation video from LangChain: LangChain (LangGraph interrupt).

### Model HITL as a branch, not a modal

You want two explicit transitions:

- `awaiting_approval`
- `approved` / `rejected`
And you want a timeout:

- if no response in 24 hours → `expired` → fallback (ticket, cancel, etc.)
Treat “human didn’t respond” as a first-class failure mode, not an awkward edge case.

### Store an audit trail

Persist:

- who approved
- when
- what they saw (rendered summary)
- what changed (diff vs original proposal)
This matters for compliance. It also matters when your agent does something dumb and you need to explain it without hand-waving.

### HITL isn’t just for approvals

HITL works best for:

- ambiguous inputs (“which customer did you mean?”)
- exception handling (“refund tool is rate-limited, do you want to retry later?”)
- policy boundaries (“this email needs legal review”)
You’ll notice the common thread. Humans aren’t there to rubber-stamp. They’re there to resolve ambiguity and risk.

## Partial failures and compensating actions (Saga-style)

The most dangerous failures are partial.

Refund succeeded, email failed. Ticket created, database write failed. The run “kinda worked”, which is how you end up with customers angry and your internal systems lying to each other.

You need a compensation strategy. This is basically the Saga pattern:

- define forward actions
- define compensating actions
- checkpoint each step
Practical compensations in agent systems:

- If you created a ticket, close it if later steps fail.
- If you sent an email, send a follow-up correction if you later discover it was wrong.
- If you updated a record, write a compensating update.
Not every side effect is reversible. That’s why you push high-risk steps behind HITL and keep irreversible actions as late as possible in the flow.

## Preventing infinite loops and runaway costs (budgets + circuit breakers)

Runaway agents are not an “oops”. They’re a product bug.

And no, you don’t fix it with “better prompts”. You fix it with limits.

Controls that work:

- **Max steps**: hard stop at, say, 25 tool calls.
- **Token budget**: hard cap per run.
- **Wall clock timeout**: stop after 5 minutes.
- **Circuit breaker**: if error rate > X% in last N minutes, trip and route to fallback.
If you want more concrete cost math, see my write-up on [LLM cost](/blog/agent-per-task-cost-calculation) and [AI in production](/pillars/ai-engineering-production). Budgets only work when you measure.

## Debugging and replay: correlation IDs or it didn’t happen

If you can’t replay a run, you can’t debug it.

If you can’t debug it, you can’t improve it.

Minimum viable observability:

- `correlation_id` propagated across every tool call
- a run timeline with state transitions and timestamps
- per-step latency, retry count, and error name
- links between traces and tool logs
If you’re instrumenting agents seriously, I’d start with [OpenTelemetry instrumentation for AI agents](/blog/opentelemetry-ai-agents-instrumentation). You want spans for:

- LLM call
- tool call
- checkpoint write
- HITL wait
And you want to be able to answer: “what happened between 13:02:11 and 13:03:40?” without reconstructing it from Slack messages and vibes.

## A pragmatic failure-mode checklist (mapped to patterns)

Here are the failures I see over and over, and the pattern that fixes them:

1. **Rate limits** → bounded retries + backoff, circuit breaker
1. **Tool timeouts** → idempotent tools + retry, checkpoint before/after
1. **Hallucinated tool args** → validation state + structured outputs
1. **Non-determinism on rerun** → checkpoint decisions, reduce re-planning
1. **Partial failures** → compensation + explicit saga steps
1. **Human never responds** → HITL timeout + fallback state
1. **Infinite loops** → max steps + budget + guardrails
For security-specific failure modes (prompt injection, tool abuse), pair this with my [AI security](/blog/ai-security-complete-guide) and [prompt injection](/blog/prompt-injection-2026-owasp-llm-vulnerability) guides.

## Putting it together: the “production agent run” template I’d ship

If you’re building in LangGraph, CrewAI, or rolling your own [agent orchestration](/pillars/ai-agents), here’s the template I’d start with:

- **State machine skeleton** with explicit state names
- **Error taxonomy** with retry policies per error
- **Checkpoint store** with schema versioning
- **Idempotent tool interfaces** and dedupe storage
- **HITL states** with audit trail and timeouts
- **Budgets and circuit breakers**
- **OpenTelemetry traces + correlation IDs**
It’s boring. That’s the point. Boring is what you want when real money and real customers are on the line.

If you want a deeper architectural framing, I wrote a companion post on [AI agent control flow](/blog/ai-agent-control-flow-architecture) and another on [agentic AI](/blog/rise-of-agentic-ai).

## Conclusion: agent reliability is workflow engineering in a trench coat

The industry keeps trying to buy reliability with “better prompts” and “more tools”. That’s backwards.

Reliability comes from control flow. From treating an agent run like a workflow you can reason about, pause, resume, and audit.

My prediction for the next 12 months: the teams that win will look less like prompt engineers and more like workflow engineers. They’ll have state machines, idempotency keys, checkpoints, and run replays. Their agents will feel boring.

And boring is exactly what you want.

Photo by Beatriz Cattel on Unsplash.