Skip to content
KG.
  • Blog
  • Hire me, or see what I ship.

    • ProjectsCase studies & shipped work
    • ServicesWork with me
    • SponsorSponsor the blog
  • Essays and references on AI engineering.

    • BlogEssays on AI engineering
    • Learning PathsGuided curricula
    • Topic PillarsDeep-dive hubs
    • GlossaryAI & dev terms, defined
    • CheatsheetsQuick references
    • ComparisonsX vs Y, decided
  • Things to click, play, and take apart.

    • Tools29 dev & AI utilities
    • Games16 browser games
    • DemosInteractive explainers
    • ChallengesDaily coding puzzles
  • Me

    • AboutWho I am
    • UsesMy gear & setup
    • ResumeCV (PDF)

    Shelf

    • BookshelfBooks I recommend
    • Reading ListWhat I'm reading
  • Subscribe
  • Let's talk
  1. Home
  2. ›
  3. Blog
  4. ›
  5. Time to First Token

Time to First Token

2 posts tagged #time-to-first-token

a close up of a stopwatch on a black background AI and Machine Learning

AI Agent Latency Budgets: Performance Guide

Single-model TTFT benchmarks lie to agent builders. Here's the 6-tier latency budget framework for production AI agents in 2026, with real math for multi-hop tool calls.

July 6, 2026 16 min read
Read more
llm-api, ai-benchmarks, latency, gpt-4-1, claude-haiku, gemini-flash, time-to-first-token, ai-performance, production-ai, developer-tools Technology

5 LLM APIs Tested for Latency: TTFT, Throughput, Cost

I benchmarked Claude Haiku 4.5, Claude Sonnet 4, GPT-4.1, GPT-4.1 Mini, and Gemini 2.5 Flash for TTFT, throughput, and end-to-end latency — with a cost-latency decision matrix for production builders.

March 7, 2026 13 min read
Read more

Keep exploring

Close to this topic, not on this page.

  • Agent Evaluation Roadmap for Small Teams: 30 Min a Week A pragmatic map of offline vs online vs HITL evals, what to measure beyond “response quality,” and the smallest program that actually… 8 min read
  • OpenTelemetry for AI Agents: Tracing Schema and Spans A vendor-neutral tracing schema for AI agents: model LLM calls, retrieval, tool runs, retries, and token cost as spans. 11 min read
  • Netflix Headroom: Cut AI Agent Costs 10x in Production Netflix engineer Tejas Chopra open-sourced Headroom, a context optimization layer that cuts LLM inference costs by up to 10x. 12 min read

Content

  • Blog
  • Topic Pillars
  • Learning Paths
  • Games
  • Demos

Resources

  • Tools
  • Glossary
  • Cheatsheets
  • Comparisons
  • LLM Prices
  • Embedding Prices

About Me

  • About
  • Uses
  • Bookshelf
  • Reading List

Meta

  • Subscribe
  • Changelog
  • Sitemap
  • Privacy
  • Terms
  • RSS
KG

Building intelligent systems. Still chasing those sour icecreams.

Made with coffee and curiosity in Toronto. 2026.