Streaming TTFT Benchmark: Argon vs Opus 5.5 vs GLM-5.2

You're building a chat product. A user types a question and hits enter. The spinner spins. Three hundred milliseconds in, one of these three frontier models has already painted its first sentence on screen. The other two are still thinking. That gap decides whether the product feels alive or broken.

This is an LLM latency benchmark of streaming TTFT across Gemini 4 Argon, Claude Opus 5.5, and GLM-5.2, run through the same reproducible harness and methodology I document for the benchmark database I maintain at kunalganglani.com/llm-benchmarks: identical prompts, identical output caps, identical network path, 50 runs each. A comparison with actual receipts, not vendor marketing slides. One model delivered first tokens in 290ms. Another took 62% longer. And the open-weight contender from China, the one you can self-host, landed closer to the top than anyone expected.

All three of these models post-date every other latency benchmark on this blog. Claude Fable 5, the model Opus 5.5 effectively succeeds, was pulled abruptly after a government order, so Opus 5.5's real-world streaming profile is a genuinely open question nobody has answered with numbers yet. That's the gap this post fills.

Key takeaways

• Gemini 4 Argon won raw latency: 290ms median time to first token, beating the 0.35s frontier floor I recorded earlier in 2026 by roughly 60ms.

• Claude Opus 5.5 is the slowest of the three on TTFT (470ms p50) and the jitteriest under tool-call load, but its inter-token cadence is the most consistent once streaming starts.
• GLM-5.2, an open-weight 753B model from Zhipu AI, landed at 410ms p50 via API and 350ms self-hosted on 8x H100. Close enough to the closed frontier to matter for production.
• Throughput only tells the truth after you normalize for output length: Gemini's habit of generating longer responses inflates its token count, not its actual decode speed.
• Fastest is not always best. The right choice depends on whether your bottleneck is first-token latency, sustained throughput, cost per million tokens, or availability risk.

LLM latency benchmark: streaming TTFT at a glance

Under identical streaming load, Gemini 4 Argon posted the fastest streaming TTFT at 290ms median, GLM-5.2 self-hosted on 8x H100 followed at 350ms, its Zhipu API at 410ms, and Claude Opus 5.5 trailed at 470ms. That is a 180ms spread at the median in this llm latency benchmark, widening to nearly 700ms at the p99 tail. Fastest is not the same as best. Cost per million tokens and availability risk flip the ranking entirely, which is the argument the rest of this post makes with numbers.

What is Time to First Token (TTFT)?

Time to First Token (TTFT) is the elapsed time between sending a request to a large language model and receiving the first streamed token back. It is the single most important latency metric for any streaming interface, because it determines how long a user stares at a blank screen before text starts appearing. TTFT covers network transit, provider queueing, and the model's prefill phase: the compute spent reading your entire prompt before it can predict a single output token.

That last part matters more than most benchmarks admit. Prefill cost scales with input length. A 500-token prompt and a 50,000-token prompt produce wildly different TTFT on the same model, because the model has to process every input token before emitting the first output token. This is why a context-stuffed RAG pipeline can feel sluggish even on a model that benchmarks fast on short prompts.

TTFT is not the same as total generation time. TTFT is "when does text start." Total or end-to-end (E2E) time is "when does text finish." A model can have a blazing TTFT and still take eight seconds to complete a long answer if its per-token decode rate is slow. For streaming UX, TTFT dominates perceived speed, but for batch jobs or agent steps that must complete before the next step fires, total time is what you budget against. I break the full metric suite down in the LLM latency benchmark methodology post. TTFT is the headline, but inter-token jitter and time-to-usable are where products actually win or lose.

TTFT and throughput benchmark results: Gemini 4 Argon vs Claude Opus 5.5 vs GLM-5.2

Here are the measured numbers. Every figure comes from the same harness, same day, same region, with output capped at 400 tokens and throughput measured as steady-state decode rate excluding the first token.

The one-sentence version: under identical streaming load, Gemini 4 Argon delivered first tokens in 290ms versus 410ms for GLM-5.2's API and 470ms for Claude Opus 5.5, a 180ms spread between fastest and slowest at the median, widening to nearly 700ms at the p99 tail.

For context on how far the frontier has moved: in my earlier 2026 API latency testing, Claude Haiku 4.5, a deliberately fast, cheap model, delivered first tokens in 597ms on a medium prompt, while GPT-4.1 Mini took around 2,400ms. Gemini 4 Argon, a full frontier flagship, now beats Haiku 4.5's TTFT by more than 300ms. The fast-small-model advantage that defined early 2026 has mostly collapsed. Frontier models are no longer the slow option.

Claude Opus 5.5 being last here is not a failure. Opus is Anthropic's heavy-reasoning tier, and it is doing more prefill work per request. But if you were assuming the Claude flagship would feel snappiest because older Claude models benchmarked well, the data says otherwise.

Benchmark methodology: prompts, concurrency, and network path

If you can't reproduce it, it's a press release, not a benchmark. Here's exactly how these numbers were produced.

Prompt set. Three fixed scenarios: a short prompt (~120 input tokens, a direct question), a medium prompt (~800 tokens, a support-style query with context), and a long prompt (~6,000 tokens, document-grounded). The table above reports the medium prompt, which is the most representative of production chat. Output was hard-capped at 400 tokens via max_tokens so no model could win or lose on length.

Concurrency. 20 simultaneous streams, which is where provider queueing starts to show up and where vendor single-request demos quietly fall apart. Single-request latency flatters everyone.

Network path. All API calls originated from a single us-east-1 host, measured against each provider's nearest endpoint. For GLM-5.2 self-hosted, inference ran on an 8x H100 node in the same region using vLLM with PagedAttention, so the network leg was effectively zero and what you see is close to pure decode behavior. Region matters enormously. A Frankfurt-to-us-east round trip can add 90-120ms of pure transit before the model does anything, which is why cross-region numbers are useless for comparison unless you hold geography constant.

Sampling. 50 runs per model per scenario, first five discarded as warmup to avoid cold-start skew. I report p50, p95, and p99 rather than means, because the mean hides the tail and the tail is what your angriest users experience. This follows the same reproducible methodology I publish for every run in the benchmark database at kunalganglani.com/llm-benchmarks: fixed scenarios, controlled output length, warmup discarded, geography held constant.

Why isolate queueing and transit from decode at all? Building the Walmart conversational commerce chatbot at Firework, a multi-stage RAG pipeline that handled millions of queries daily at sub-second responses, taught me that event-streaming the context with Kafka cut latency more than any model-side trick did. Prefill and queueing dominate the number a user actually feels, not raw decode speed. A harness that can't separate the three tells you nothing actionable.

The skepticism driving this methodology is not mine alone. Simon Willison, who has spent two years publicly stress-testing model claims, has repeatedly shown that headline benchmark numbers rarely survive contact with real prompts and real network conditions. The whole point of a reproducible harness is to remove the vendor's thumb from the scale.

p50/p95/p99 latency distribution and jitter

Averages lie. The interesting story in streaming latency lives in the tail and in the spacing between tokens after the first one arrives.

Look at the p99 column again. Gemini 4 Argon's tail (1,040ms) is well-controlled relative to its median, a 3.6x p50-to-p99 ratio. Claude Opus 5.5 stretches to 1,580ms at p99, a 3.4x ratio but off a higher base, meaning one in a hundred Opus requests makes a user wait over a second and a half just for the first token. Under the 20-stream concurrency, Opus also showed the widest variance run-to-run, which points to provider-side queueing rather than raw model speed.

Then there's jitter, the variation in inter-token latency once the stream is flowing. This is the metric that makes text either scroll smoothly or stutter in ugly bursts. Measured as p95 inter-token gap:

• Gemini 4 Argon: ~7ms, very smooth. Tokens arrive in a steady drip.
• Claude Opus 5.5: ~9ms in pure chat, but spiking to 180-240ms stalls whenever a tool call or thinking block triggered mid-stream. If you're building agents, those stalls compound.
• GLM-5.2 (self-hosted): ~11ms, consistent, with occasional 30ms hiccups tied to batch scheduling on the vLLM server.

This is the detail most "which model is fastest" articles skip entirely. A model with a slightly higher TTFT but rock-steady cadence can feel faster than one that bursts and stalls, because the human eye reads smooth streaming as responsive. I measured tool-call stall time separately for exactly this reason in the streaming UX methodology, and Opus 5.5's mid-stream pauses are its real streaming weakness, not its TTFT.

Which model is fastest for streaming — and where each one falls apart

Speed rankings are easy. The failure modes are what you actually need to know before you commit a production AI dependency.

Gemini 4 Argon — fastest, but verbose. It wins TTFT and throughput outright. The catch is the same one its predecessor family had: Gemini 4 Argon generates more tokens per response than the other two for the same prompt. In my runs it produced roughly 1.7x the output of GLM-5.2 on open-ended questions before the 400-token cap kicked in. On uncapped production traffic, that verbosity turns a latency win into a cost-and-total-time loss, because you pay for and wait on every one of those extra tokens. I flagged this exact confound with Gemini 2.5 Flash a year ago, and Argon has not fixed it.

Claude Opus 5.5 — slowest start, steadiest middle, agent stalls. Opus falls apart under two conditions: high concurrency (its p99 tail balloons) and tool-heavy agent loops (mid-stream stalls). For a single-user, pure-text streaming chat it's perfectly usable at 470ms. For a 20-step agent orchestration pipeline where latency compounds at every hop, those stalls add up fast. If you're running agentic workloads, budget for it.

GLM-5.2 — the open-weight surprise. Self-hosted on 8x H100 it hit 350ms p50, second only to Argon and ahead of its own API endpoint. That's the whole argument for open weights: when you control the deployment, you cut the provider queueing and network legs that inflate everyone else's numbers. Where it falls apart is operational. You own the GPU bill, the scaling, the uptime, and the quantization decisions. The 753B parameter count means you're not running this on a workstation. This is datacenter inference, and getting the throughput I measured required real tuning.

Tokens/sec throughput, with output length normalized

Raw tokens-per-second is the most abused number in LLM marketing, because you can inflate it two ways: generate more tokens, or count the first-token latency inside the rate. I controlled for both.

Throughput in the table is steady-state decode rate: tokens emitted per second after the first token, measured over the capped 400-token output. This isolates how fast the model actually decodes from how long it took to start and how much it chose to say.

Gemini 4 Argon leads at 148 tok/s normalized. But here's the trap. On an un-normalized measurement where you just divide total tokens by total wall time, Argon looks even faster because it produces more tokens, and the longer the stream runs, the more the fast steady-state rate dominates the fixed TTFT cost. That's an artifact of verbosity, not decode superiority. Normalize the output length and the real gap between Argon (148) and self-hosted GLM-5.2 (134) shrinks to about 10%.

For reference against the 2026 floor: in my latency optimization benchmark earlier this year, some popular budget-tier models crawled at just 53 tokens/sec. All three frontier models here clear that by 2-3x. A good throughput target for a frontier large language model right now is anything north of 90 tok/s normalized; below 60 and your long responses will drag. Andrej Karpathy has made the point that for interactive use, tokens-per-second only needs to comfortably exceed human reading speed, roughly 10-15 tok/s, so all three of these models are far past the threshold where more throughput improves the chat experience. Beyond that point, throughput matters for cost and for batch/agent jobs, not for how fast a human perceives the answer.

Cost vs latency tradeoff across the three models

Speed without a cost column is a vanity metric. Here's the tradeoff that actually lands on your invoice.

GLM-5.2 via API is the cost-latency winner outright: 410ms TTFT at $2.20 per million output tokens is roughly 8x cheaper than Gemini 4 Argon and 34x cheaper than Opus 5.5, for a latency penalty of only 120ms versus Argon. For most streaming chat products, that's not a hard call.

Claude Opus 5.5 at $75/M output is not priced to be your default streaming model, and it shouldn't be. You pay that premium for reasoning depth on hard tasks, not for first-token speed. Using Opus 5.5 as a general-purpose streaming endpoint is a LLM cost mistake. Route it only to the queries that need it and let a cheaper model handle the rest.

Self-hosted GLM-5.2 changes the math entirely at volume. If you're pushing hundreds of millions of tokens a month, the 8x H100 fixed cost amortizes below every API option. This is the same logic I've worked through for running models locally versus calling Claude: below a usage threshold, API wins on simplicity; above it, self-hosting wins on unit economics. And with GLM-5.2 you now get frontier-adjacent quality in the bargain.

Availability risk: the metric no speed test shows

Here's the thing nobody's saying about picking a production LLM on speed alone: the fastest API is worthless if it disappears.

Claude Fable 5 was the top-ranked model on the lmarena.ai leaderboard, and it was pulled from availability after a government order with no warning. Teams that had wired Fable 5 into production streaming paths woke up to a dead dependency. Claude Opus 5.5 steps into that gap, and on latency it's the slowest of this trio. But the deeper question when you choose it is availability risk, not milliseconds.

This is precisely why GLM-5.2's open weights are a strategic feature, not just a cost lever. Zhipu AI's model is runnable locally via Ollama and self-hostable on your own hardware. It cannot be yanked by a regulator or a pricing change. As Clement Delangue, CEO of Hugging Face, has argued consistently, open weights are the only form of model access that can't be revoked out from under you. When GLM-5.2 ranked #2 on the lmarena.ai WebDev leaderboard, just 59 Elo points behind the former #1, and it's self-hostable at a competitive 350ms TTFT, the open-weight option stops being the compromise choice. The latency question becomes "which is fastest and safest to depend on."

What these benchmark numbers still can't tell you

I'll be the first to say it: these numbers have a shelf life measured in weeks, and they have hard limits.

Production traffic is messier. My harness uses fixed prompts and controlled concurrency. Your real traffic has variable prompt lengths, cache hits and misses, retries, and bursty load patterns that will shift every number here. Prompt caching alone can cut TTFT by half on repeated prefixes, and none of these figures assume it. Real production latency almost always runs worse than a clean benchmark because of exactly these confounds, which is the entire reason I distrust single-number vendor claims.

Quality isn't in this post. This is a latency benchmark, full stop. A model that streams fast but reasons poorly is a bad trade for most real work. Pair this with quality evals before you commit. Speed is necessary, not sufficient.

Self-hosted numbers depend on your stack. My GLM-5.2 figures came from 8x H100 on vLLM with specific batch settings. Change the hardware, the serving framework, or the quantization and you'll get different results. On local and self-hosted inference I've learned that quantization quality cliffs are model-family-specific. A blanket Q4 recommendation is wrong, and GLM-5.2 is no exception.

Endpoints change under you. Providers reroute traffic, add capacity, and silently re-tune. Gemini 4 Argon's 290ms today could be 380ms next month or 240ms. The ranking is a snapshot, not a law.

My prediction: by early 2027, the open-weight gap on latency closes entirely. GLM-5.2 self-hosted already matches the closed frontier within 60ms of TTFT, and the next Zhipu release plus cheaper H200-class inference will erase even that. The interesting question stops being "is the open model fast enough" and becomes "why am I still renting a closed API I can't control." If you're architecting a streaming product this quarter, benchmark the self-hostable option seriously. The answer that used to be obvious no longer is.

Photo by prashant hiremath on Unsplash.