AI Agent Control Flow Patterns [2026]: Retries, HITL, Checkpoints

AI agent control flow patterns are the reliability primitives that turn a flashy agent demo into something you can put behind a pager. If your agent can’t retry safely, pause for a human approval, or resume after a crash, you don’t have an “agent”. You have an expensive while-loop.

Key takeaways

• Reliable agents borrow from workflow engines: explicit states, bounded retries, and durable checkpoints.
• Retrying is only safe when your tools are idempotent and you store tool receipts in your checkpoint.
• Human-in-the-loop steps should be modeled as first-class states with timeouts, SLAs, and an audit trail.
• You prevent runaway cost with max steps, budgets, and circuit breakers, not with “better prompts”.
• Debuggability comes from correlation IDs, structured error names, and replayable run logs.

If your agent can’t resume from a checkpoint, it’s not autonomous. It’s fragile.

I’m writing this in 2026 because the bar quietly moved. We’re not doing “one chat turn, one response” anymore. We’re doing long-lived runs that touch real systems. Payments. Tickets. Customer data. Anything that can wake you up at 2 a.m.

Frameworks like LangGraph and CrewAI have been inching toward durable execution and interrupt/resume primitives. Good. That’s not a nice-to-have. It’s the only way this stuff survives contact with production.

Based on the benchmark data I maintain at kunalganglani.com/llm-benchmarks, the best teams already treat agent runs as workflows, not chats. The performance arguments are real, but the bigger win is operational. When something inevitably fails, you don’t throw away the entire run and start over like it’s 2023.

At a meta level, running this blog’s multi-agent publishing pipeline (7 agents, deterministic SEO quality gate, idempotent publishing steps) taught me a boring truth: deterministic gates and resumable steps catch more failures than “just use a bigger model”. Same story with production agents. Intelligence helps. Control flow is what keeps you out of trouble.

What I mean by “ai agent control flow patterns”

When I say ai agent control flow patterns, I mean the reusable ways you structure an agent run like a state machine.

What happens on success. What happens on failure. Which failures get retries. Where you branch. What you persist so the run can pause, crash, resume, and still do the right thing.

If you’ve used AWS Step Functions, Temporal, or any workflow engine, you already know the shape of this. The same primitives show up everywhere:

• Retry with backoff and a max attempt count
• Catch and route to a fallback state
• Timeouts and explicit cancellation
• Durable checkpoints so execution can resume after crashes
• Human approval as a first-class state

Most agent tutorials still start with prompts and tool calling. That’s backwards. Here’s the catalog view I wish they started with.

Now let’s map these patterns to how you actually build LangGraph/CrewAI-style agents.

→ Related: AI Agent Memory State Management Guide [2026]

Agents, Flows, Tasks & Processes: stop thinking “chatbot”

I keep seeing teams build “agents” the way they built chatbots. A blob of instructions, a couple tools, and vibes.

That works until the first time you need to pause for approval, recover from a crash, or explain why the agent emailed a customer the wrong thing. Then you realize you didn’t build an agent. You built a conversational script with side effects.

CrewAI is explicit about the shift. In their docs they describe Flows as orchestrations with start/listen/router steps that manage state, persist execution, and resume long-running workflows (CrewAI Documentation). That’s not fluff. That’s the contract.

Here’s the vocabulary that matters:

• Agents: the reasoning units (LLM + tools + memory) that decide what to do.
• Tasks: discrete units of work you can run, retry, and observe.
• Processes: how tasks are sequenced (sequential vs hierarchical vs hybrid).
• Flows: the control plane. Routing, persistence, resumability, triggers, HITL.

If you’re building serious AI agents, flows are where the reliability story lives. The agent can be “smart”. The flow needs to be safe.

A concrete example.

“Draft a customer refund email” is an agent task.

“Send a refund email and issue a refund” is a flow, because it contains irreversible side effects.

Error names: you need a taxonomy, not a blob of text

One of the best ideas in AWS Step Functions is also one of the most boring: error names.

Step Functions identifies errors using case-sensitive strings, with built-in ones that start with States.. It also allows custom error names, but they cannot begin with States. (AWS Step Functions Developer Guide).

That tiny rule forces discipline. You can’t just throw a stack trace at the wall and hope your retry logic figures it out.

In agent systems, I like a three-layer taxonomy:

1. LLM errors: model timeouts, overloaded, safety refusals.
2. Tool errors: validation errors, auth errors, rate limits, upstream 500s.
3. Control-flow errors: budget exceeded, max steps exceeded, checkpoint missing.

Give them names that are stable, searchable, and actionable:

• llm.timeout
• llm.refusal
• tool.validation_failed
• tool.rate_limited
• tool.upstream_5xx
• flow.budget_exceeded
• flow.max_steps_exceeded

This isn’t bureaucracy. This is the switchboard for your system. It’s how you decide whether to retry, fallback, page a human, or dead-letter.

Retrying after an error (without turning your agent into a fraud engine)

AWS Step Functions defaults to failing the entire state machine when a state reports an error, unless you configure advanced error handling like Retry/Catch (AWS Step Functions Developer Guide). That’s the correct default.

Retries are not “free reliability”. Retries are duplicated work. If you haven’t designed for them, they’re how you double-charge a customer and then write a very sincere postmortem.

Rule #1: retries are for transient failures

Retry on:

• timeouts
• rate limits
• network failures
• upstream 5xx

Do not retry on:

• validation errors (bad args)
• auth errors
• “human said no”
• tool precondition failures

If you want a heuristic that holds up in practice: if the exact same request would succeed 30 seconds later, it’s retryable.

Rule #2: separate LLM retries from tool retries

LLM calls and tool calls fail differently, and pretending they’re the same is how you get weird behavior.

• LLM retries often succeed on the second attempt, but the output can drift. Keep retry counts low (2–3), and prefer structured outputs so the arguments don’t mutate while you’re “just retrying”.
• Tool retries are where the real danger lives, because tools have side effects. You can do more attempts (3–5) only if the tool is idempotent.

A safe retry recipe (5 steps)

1. Classify the error into a stable error name.
2. Check the retry policy for that error name (max attempts, backoff).
3. Rehydrate from a checkpoint so you retry from a known state.
4. Call the tool with an idempotency key (details below).
5. Persist the tool receipt (request + response + side-effect identifiers) to the checkpoint.

This is the difference between “retry” and “re-execute and hope”.

Fallback states: graceful degradation beats perfect answers

Step Functions supports Catch blocks that route you to fallback states for Task, Parallel, and Map states (AWS Step Functions Developer Guide). In practice, it’s just another edge on the graph.

In agent land, fallback states are where you stop pretending you can always be perfect and start being a product.

Examples that actually ship:

• If the web search tool fails, fall back to “answer from cached knowledge base” and label the response.
• If the CRM write fails, fall back to “create a human ticket” and stop.
• If the model refuses, fall back to a smaller / more permissive policy model for harmless content.

The key: fallback states should produce observable outcomes. Not “agent apologizes”. Apologies don’t fix incidents.

I like to treat each fallback as a different “exit type”:

• completed
• completed_with_degradation
• needs_human
• failed_permanently

Those exit types become metrics. And metrics become decisions.

State machine examples using Retry and Catch (agent-graph edition)

Here’s a production-ish state machine for “issue refund”:

1. Validate request (no retry)
2. Fetch order (retry on rate limits)
3. Decide refund amount (LLM; retry 1–2 times)
4. Human approval if amount > $200
5. Issue refund (tool; idempotent + retry)
6. Send email (tool; idempotent + retry)
7. Write audit log (tool; retry)

Catch routes:

• On tool.validation_failed → failed_permanently
• On tool.rate_limited after max attempts → needs_human (or dead-letter)
• On flow.budget_exceeded → needs_human

Conceptually, this is Step Functions. Your “Task” might be an LLM call, a tool call, or an entire subgraph. The control flow doesn’t care.

If you want the mental model closest to real production reliability, Temporal’s positioning is dead-on: workflows should resume “exactly where they left off after crashes, network failures, or infrastructure outages” (Temporal Platform Documentation). That’s the bar. Everything else is a demo.

Durable checkpoints: what to persist so you can resume (and trust the resume)

Durable checkpoints are the dividing line between “agent demo” and “agent system”. Without them, every transient failure becomes a full restart.

That’s wasted cost. It’s also more chances for the agent to take a different path the next time and create new problems.

What to checkpoint (practical list)

Checkpoint at state boundaries, not after every token.

Persist:

• Run identifiers: run_id, correlation_id, user_id (if applicable)
• State machine position: current state name, step index
• Inputs: validated input payload (post-normalization)
• Tool receipts: request params, response payload, side-effect IDs (ticket ID, payment ID)
• Decisions: the agent’s chosen plan or next action in a structured form
• Budgets: remaining token budget, remaining tool budget, max steps remaining

Avoid persisting:

• raw chain-of-thought (for both safety and determinism reasons)

Checkpoint frequency

A good default is one checkpoint per tool call plus major state transitions.

If you do 20 tool calls in a run, expect ~20 durable writes. That’s fine. Storage is cheaper than reruns. And if you’re worried about the storage bill, wait until you see the “agent restarted itself 14 times” bill.

Schema evolution (the part everyone ignores)

If your checkpoint schema is JSON, it will change. Plan for it.

• Version your checkpoint schema (schema_version: 3).
• Write migrations for “resume” paths.
• Treat unknown fields as forward-compatible.

I learned the hard way that identity is a one-way door. In one incident on this blog, rewriting slugs on live URLs burned 907K impressions of link equity. Checkpoint IDs are the same kind of identity. Don’t casually rewrite them.

Idempotent tool design for agents (at-least-once is reality)

Agents run in distributed systems. The only honest execution guarantee you get is at-least-once.

If your tool call times out, you don’t actually know whether the side effect happened. Retrying without idempotency is how you double-refund a customer and then spend the next day doing cleanup.

The idempotency contract

Every side-effecting tool should accept:

• idempotency_key: stable per logical operation
• optionally request_hash: hash of normalized request to detect key reuse with different payloads

And every tool should return a receipt:

• status: created/already_exists
• external_id: payment ID, ticket ID, message ID
• raw_response: what the upstream returned

Then store that receipt in your checkpoint.

Choosing the idempotency key

Two practical schemes that cover most cases:

• run_id + state_name for “exactly one call per state per run”
• business_id + action + version for “dedupe across runs” (e.g., order_id + refund)

The second one matters when users can re-trigger flows. Which they will.

Exactly-once is a lie (so design for dedupe)

You can approximate exactly-once with careful dedupe storage. Operationally, assume:

• retries happen
• tasks get re-delivered
• workers crash mid-flight

Temporal leans into this: it gives you resumption and deterministic replay, but activities are still something you design to be idempotent. That’s not a weakness. That’s reality.

Human-in-the-loop agent workflow (pause, resume, audit)

Human-in-the-loop (HITL) is not a hack and it’s not a “UI feature”. It’s a state.

LangGraph has an explicit interrupt mechanism designed for exactly this. Here’s the official explanation video from LangChain: LangChain (LangGraph interrupt).

Model HITL as a branch, not a modal

You want two explicit transitions:

• awaiting_approval
• approved / rejected

And you want a timeout:

• if no response in 24 hours → expired → fallback (ticket, cancel, etc.)

Treat “human didn’t respond” as a first-class failure mode, not an awkward edge case.

Store an audit trail

Persist:

• who approved
• when
• what they saw (rendered summary)
• what changed (diff vs original proposal)

This matters for compliance. It also matters when your agent does something dumb and you need to explain it without hand-waving.

HITL isn’t just for approvals

HITL works best for:

• ambiguous inputs (“which customer did you mean?”)
• exception handling (“refund tool is rate-limited, do you want to retry later?”)
• policy boundaries (“this email needs legal review”)

You’ll notice the common thread. Humans aren’t there to rubber-stamp. They’re there to resolve ambiguity and risk.

Partial failures and compensating actions (Saga-style)

The most dangerous failures are partial.

Refund succeeded, email failed. Ticket created, database write failed. The run “kinda worked”, which is how you end up with customers angry and your internal systems lying to each other.

You need a compensation strategy. This is basically the Saga pattern:

• define forward actions
• define compensating actions
• checkpoint each step

Practical compensations in agent systems:

• If you created a ticket, close it if later steps fail.
• If you sent an email, send a follow-up correction if you later discover it was wrong.
• If you updated a record, write a compensating update.

Not every side effect is reversible. That’s why you push high-risk steps behind HITL and keep irreversible actions as late as possible in the flow.

Preventing infinite loops and runaway costs (budgets + circuit breakers)

Runaway agents are not an “oops”. They’re a product bug.

And no, you don’t fix it with “better prompts”. You fix it with limits.

Controls that work:

• Max steps: hard stop at, say, 25 tool calls.
• Token budget: hard cap per run.
• Wall clock timeout: stop after 5 minutes.
• Circuit breaker: if error rate > X% in last N minutes, trip and route to fallback.

If you want more concrete cost math, see my write-up on LLM cost and AI in production. Budgets only work when you measure.

Debugging and replay: correlation IDs or it didn’t happen

If you can’t replay a run, you can’t debug it.

If you can’t debug it, you can’t improve it.

Minimum viable observability:

• correlation_id propagated across every tool call
• a run timeline with state transitions and timestamps
• per-step latency, retry count, and error name
• links between traces and tool logs

If you’re instrumenting agents seriously, I’d start with OpenTelemetry instrumentation for AI agents. You want spans for:

• LLM call
• tool call
• checkpoint write
• HITL wait

And you want to be able to answer: “what happened between 13:02:11 and 13:03:40?” without reconstructing it from Slack messages and vibes.

A pragmatic failure-mode checklist (mapped to patterns)

Here are the failures I see over and over, and the pattern that fixes them:

1. Rate limits → bounded retries + backoff, circuit breaker
2. Tool timeouts → idempotent tools + retry, checkpoint before/after
3. Hallucinated tool args → validation state + structured outputs
4. Non-determinism on rerun → checkpoint decisions, reduce re-planning
5. Partial failures → compensation + explicit saga steps
6. Human never responds → HITL timeout + fallback state
7. Infinite loops → max steps + budget + guardrails

For security-specific failure modes (prompt injection, tool abuse), pair this with my AI security and prompt injection guides.

Putting it together: the “production agent run” template I’d ship

If you’re building in LangGraph, CrewAI, or rolling your own agent orchestration, here’s the template I’d start with:

• State machine skeleton with explicit state names
• Error taxonomy with retry policies per error
• Checkpoint store with schema versioning
• Idempotent tool interfaces and dedupe storage
• HITL states with audit trail and timeouts
• Budgets and circuit breakers
• OpenTelemetry traces + correlation IDs

It’s boring. That’s the point. Boring is what you want when real money and real customers are on the line.

If you want a deeper architectural framing, I wrote a companion post on AI agent control flow and another on agentic AI.

Conclusion: agent reliability is workflow engineering in a trench coat

The industry keeps trying to buy reliability with “better prompts” and “more tools”. That’s backwards.

Reliability comes from control flow. From treating an agent run like a workflow you can reason about, pause, resume, and audit.

My prediction for the next 12 months: the teams that win will look less like prompt engineers and more like workflow engineers. They’ll have state machines, idempotency keys, checkpoints, and run replays. Their agents will feel boring.

And boring is exactly what you want.

Photo by Beatriz Cattel on Unsplash.