How to Build a Gemini 3.8 Live Voice Agent [2026 Tutorial]
A practical Gemini 3.8 Live real-time voice agent tutorial: full-duplex streaming, barge-in interruptions, async tool calls, ephemeral tokens, and a latency budget you can actually hit.
You can build a Gemini 3.8 Live voice agent that feels “alive” (streaming audio in and out, full-duplex, barge-in, and tool calls) in a day.
The prerequisite that trips people up: you need to treat audio as a real-time stream with backpressure, not as “upload a file, wait, download a file.” If you don’t design a tiny state machine for turn-taking and cancellation, your demo will work once and your production app will be a glitchy mess.
This post is my opinionated, copy/paste-first Gemini 3.8 Live real-time voice agent tutorial. I’ll show the raw WebSocket shape, the barge-in recipe, and the async tool-calling patterns that keep users from hearing dead air.
I’m also going to be blunt: voice UX is mostly latency engineering. If you don’t instrument the pipeline, you’re guessing.
What is Gemini 3.8 Live?
Gemini 3.8 Live is a native speech-to-speech model in the Gemini Live API that can process continuous audio streams and return immediate streaming audio responses for low-latency, real-time conversations.

Google positions it (and Gemini 3.8 Live Extended Thinking) specifically for “voice-first product experiences,” with a key capability that matters in production: asynchronous function calling so tools can run in the background while the model continues speaking.
If you’ve built AI agents before, you already know the concept. It’s tool calling. The difference is the UX penalty.
In text chat, a 2-second stall is mildly annoying. In voice, it turns into “hello? … are you there?” and the user bails.
Here’s the core promise of Live, straight from Google’s docs: the Live API processes continuous streams of audio, images, and text to deliver immediate spoken responses over a long-lived session (Gemini Live API overview).
Gemini 3.8 Live in 9 steps (the recipe I actually use)
This is the build order that keeps you out of the usual traps.

- Pick an implementation approach: GenAI SDK for speed, raw WebSockets for control.
- Decide your audio format (PCM16 vs Opus) and a chunk size target.
- Create a Live session and start streaming mic audio immediately.
- Play assistant audio with a jitter buffer (don’t play packets the instant they arrive).
- Add VAD (voice activity detection) on the mic stream.
- Implement barge-in: on VAD start while TTS is playing, stop or duck playback and notify the session.
- Add tool calling with explicit timeouts and idempotency keys.
- Narrate tool progress (“Let me check that”) while tools run.
- Instrument the latency budget end-to-end and add reconnection logic.
If you only do steps 1–4, you’ll have a demo.
Steps 5–9 is where it becomes a product.
I’ve shipped conversational systems that handle millions of queries daily at sub-second response times (Walmart conversational commerce chatbot). The lesson that keeps repeating is boring and consistent: the pipeline wins or loses, not the model. Voice agents just make the failure mode louder.
Gemini 3.8 Live vs Gemini 3.8 Live Extended Thinking (when to use each)
Treat these as two different modes of voice UX.

- Gemini 3.8 Live: high-volume, cost-sensitive, near-real-time voice where the model’s job is to keep the conversation moving.
- Gemini 3.8 Live Extended Thinking: when the user’s request implies actual work. Multi-step planning, tool orchestration, deeper reasoning.
Extended Thinking exists because streaming voice wants fast first-audio, but real tasks take time.
Reid Marlow describes the production failure mode well: you either get a lightweight model that answers in ~300ms, or a heavier reasoning path that takes ~6 seconds before the first audio frame. That dead air kills voice UX (Reid Marlow).
Gemini 3.8 Live Extended Thinking’s key claim is that it decouples the interactive speech stream from the heavier reasoning and tool loop, so the model can keep talking while background work runs.
The heuristic I use:
- If the user is asking for facts or actions (“book,” “check,” “cancel,” “compare,” “pull up”) and you’ll hit external systems, lean Extended Thinking.
- If the user is doing light conversational navigation (“what can you do,” “repeat that,” “help”), stick to Live.
It’s the same tradeoff you make in AI in production, except voice punishes you immediately.
Live API basics: session, streaming, and the WebSocket shape
Google documents three primary ways to integrate: the GenAI SDK, raw WebSockets, or Google’s Agent Development Kit (ADK) (Live API docs). For this tutorial, I’m using raw WebSockets because it forces you to deal with the stuff that breaks in real apps.
Reference architecture (minimal, production-shaped)
You don’t need a giant system to start. You do need clean boundaries.
- Browser/mobile client
- mic capture
- VAD
- playback jitter buffer
- barge-in controller
- Your gateway (WebSocket)
- mints ephemeral tokens
- rate limits by user/session
- optional recording/redaction
- Tool server
- idempotent tool endpoints
- timeouts, retries
- audit logging
This maps cleanly onto “voice agent as a real-time stream,” not “LLM as a request/response API.”
If you’ve built tool-heavy agents, connect this to your agent orchestration patterns. Voice is the same playbook, just less forgiving.
Raw WebSocket message loop (TypeScript)
Below is a runnable skeleton for a web client. It focuses on the shape: open a socket, stream audio frames up, consume audio frames down, handle backpressure.
Assumptions: you already have a backend endpoint that returns an ephemeral token (covered later). Audio capture uses WebAudio. The exact Live API message types can change, so treat this as a structural template.
// client/live.ts
type EphemeralTokenResponse = { token: string; expiresAtMs: number };
type LiveState =
| { kind: 'idle' }
| { kind: 'connecting' }
| { kind: 'connected' }
| { kind: 'reconnecting'; attempt: number }
| { kind: 'closed' };
const AUDIO_SAMPLE_RATE = 16000; // Hz (common for speech)
const CHUNK_MS = 20; // target 20ms frames
const SAMPLES_PER_CHUNK = (AUDIO_SAMPLE_RATE * CHUNK_MS) / 1000; // 320
export async function startLiveSession() {
let state: LiveState = { kind: 'connecting' };
const tokenRes = await fetch('/api/live/ephemeral-token');
if (!tokenRes.ok) throw new Error('Failed to mint token');
const { token }: EphemeralTokenResponse = await tokenRes.json();
// NOTE: Use the correct wss endpoint from the Live API docs for your project.
const ws = new WebSocket(
`wss://YOUR_LIVE_ENDPOINT?access_token=${encodeURIComponent(token)}`
);
// Backpressure: keep outbound queue small.
const outboundQueue: Array<ArrayBuffer> = [];
const MAX_OUTBOUND_FRAMES = 50; // 50 * 20ms = 1s of audio. Past this, you're already broken.
// Jitter buffer for inbound audio.
const inboundAudioQueue: Array<Float32Array> = [];
const MAX_INBOUND_FRAMES = 100;
ws.binaryType = 'arraybuffer';
ws.onopen = () => {
state = { kind: 'connected' };
// Send session config / system prompt once.
ws.send(
JSON.stringify({
type: 'session.configure',
inputAudio: { format: 'pcm16', sampleRateHz: AUDIO_SAMPLE_RATE },
outputAudio: { format: 'pcm16', sampleRateHz: AUDIO_SAMPLE_RATE },
// Voice UX prompt: short, confirm critical alphanumerics.
systemInstruction:
'You are a voice assistant. Be brief (1-2 sentences). ' +
'If user says letters/numbers, repeat them back for confirmation. ' +
'If calling tools, narrate progress in short phrases.'
})
);
};
ws.onmessage = (evt) => {
if (typeof evt.data === 'string') {
const msg = JSON.parse(evt.data);
if (msg.type === 'output.audio.delta') {
// Some APIs ship audio deltas as base64 in JSON.
const pcm16 = base64ToArrayBuffer(msg.data);
inboundAudioQueue.push(pcm16ToFloat32(new Int16Array(pcm16)));
if (inboundAudioQueue.length > MAX_INBOUND_FRAMES) inboundAudioQueue.shift();
}
if (msg.type === 'tool.call') {
// Delegate tool execution to your backend. See tool patterns section.
}
return;
}
// Some APIs send raw audio as binary frames.
const ab = evt.data as ArrayBuffer;
inboundAudioQueue.push(pcm16ToFloat32(new Int16Array(ab)));
if (inboundAudioQueue.length > MAX_INBOUND_FRAMES) inboundAudioQueue.shift();
};
ws.onerror = () => {
// Surface this in UI.
};
ws.onclose = () => {
state = { kind: 'closed' };
};
// Mic capture loop: produce 20ms frames, enqueue, flush.
const mic = await startMicCapture(
AUDIO_SAMPLE_RATE,
SAMPLES_PER_CHUNK,
(pcm16Frame) => {
if (state.kind !== 'connected') return;
if (outboundQueue.length >= MAX_OUTBOUND_FRAMES) {
// Drop oldest audio to keep “now” relevant.
outboundQueue.shift();
}
outboundQueue.push(pcm16Frame);
while (outboundQueue.length) {
// WebSocket bufferedAmount is a rough backpressure signal.
if (ws.bufferedAmount > 2_000_000) break; // ~2MB
ws.send(outboundQueue.shift()!);
}
}
);
// Playback loop with jitter buffer.
startPlayback(AUDIO_SAMPLE_RATE, () => inboundAudioQueue.shift() ?? null);
return { ws, mic };
}
function base64ToArrayBuffer(b64: string): ArrayBuffer {
const binary = atob(b64);
const len = binary.length;
const bytes = new Uint8Array(len);
for (let i = 0; i < len; i++) bytes[i] = binary.charCodeAt(i);
return bytes.buffer;
}
function pcm16ToFloat32(pcm16: Int16Array): Float32Array {
const out = new Float32Array(pcm16.length);
for (let i = 0; i < pcm16.length; i++) out[i] = pcm16[i] / 32768;
return out;
}
async function startMicCapture(
sampleRate: number,
samplesPerChunk: number,
onChunk: (pcm16: ArrayBuffer) => void
) {
// Real code would use AudioWorklet for low latency.
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
const ctx = new AudioContext({ sampleRate });
const source = ctx.createMediaStreamSource(stream);
const processor = ctx.createScriptProcessor(2048, 1, 1);
const buffer: number[] = [];
processor.onaudioprocess = (e) => {
const input = e.inputBuffer.getChannelData(0);
for (let i = 0; i < input.length; i++) buffer.push(input[i]);
while (buffer.length >= samplesPerChunk) {
const chunk = buffer.splice(0, samplesPerChunk);
const pcm16 = new Int16Array(samplesPerChunk);
for (let i = 0; i < samplesPerChunk; i++) {
const s = Math.max(-1, Math.min(1, chunk[i]));
pcm16[i] = s < 0 ? s * 32768 : s * 32767;
}
onChunk(pcm16.buffer);
}
};
source.connect(processor);
processor.connect(ctx.destination);
return { stream, ctx, stop: () => stream.getTracks().forEach((t) => t.stop()) };
}
function startPlayback(sampleRate: number, pull: () => Float32Array | null) {
const ctx = new AudioContext({ sampleRate });
const script = ctx.createScriptProcessor(2048, 1, 1);
script.onaudioprocess = (e) => {
const out = e.outputBuffer.getChannelData(0);
out.fill(0);
// Simple jitter buffer: keep ~60-120ms queued.
// Real production code should use an AudioWorklet ring buffer.
const next = pull();
if (!next) return;
out.set(next.subarray(0, Math.min(out.length, next.length)));
};
script.connect(ctx.destination);
}This skeleton bakes in two opinions that will save you later:
- I cap the outbound queue at 1 second (
MAX_OUTBOUND_FRAMES = 50at 20ms). If your uplink stalls more than that, you’d rather drop old audio than make the agent respond to something the user said a second ago. - I treat
ws.bufferedAmountas a “you’re about to glitch” signal. It’s not perfect, but it catches the failure before users do.
If you want a deeper streaming and TTFT mental model, see my post on LLM latency.
Streaming audio without dead air: format, chunk size, buffering, latency budgets
When a voice agent doesn’t feel real-time, it’s almost always one of these:
- you stream chunks that are too big
- you buffer too much on playback
- your network RTT is high and you didn’t plan for it
- your tool calls block the speech stream
What audio format and chunk size should you stream?
Most “it works everywhere” stacks start with PCM16 @ 16 kHz, mono.
- PCM16 is boring. That’s why it’s good. You can generate it in the browser, and most APIs accept it.
- Opus is more bandwidth-efficient, but you just moved complexity into encoding, jitter behavior, and cross-platform quirks.
Chunk size is where people shoot themselves.
- 20ms frames (common for real-time comms) is a good default.
- 40ms is often fine.
- 100ms+ feels sluggish, and barge-in starts to get weird.
Concrete numbers:
- At 16 kHz, 20ms is 320 samples.
- PCM16 means 2 bytes/sample, so 20ms is 640 bytes of audio payload before framing overhead.
A realistic latency budget (and what to log)
You don’t need perfection. You need a budget and visibility.
Here’s what I aim for in a “feels snappy” assistant:
| Stage | Target (P50) | Notes |
|---|---|---|
| Mic capture frame | 20ms | 10–20ms is ideal. Bigger chunks add feelable lag. |
| Client processing (VAD + encode) | 5–15ms | Use AudioWorklet if possible. |
| Network uplink + RTT component | 50–150ms | Toronto-to-us-east is often in this range on good networks. |
| Model first-audio latency | 150–350ms | This is what users perceive as “alive.” |
| Network downlink | 50–150ms | Symmetric-ish with uplink. |
| Playback jitter buffer | 60–120ms | Enough to avoid glitches, not enough to feel delayed. |
If you’re above ~700ms P50 from end-of-utterance to first assistant audio, people start talking over it. That’s not a user problem. That’s your pipeline telling you it needs barge-in.
Instrumentation checklist (minimum):
t_mic_frame_captured(client)t_frame_sent(client)t_frame_received(server, if you proxy) or first message receiptt_first_audio_received(client)t_first_audio_played(client)- playback queue depth (frames)
- WebSocket
bufferedAmount
I like building observability early because you can’t debug “it feels laggy” from vibes. If you want a vendor-neutral approach, I wrote up LLM observability metrics and an OpenTelemetry instrumentation setup that maps cleanly to voice pipelines.
How to implement barge-in (interruptions) in Gemini Live
Barge-in is the feature users assume exists. If they can’t interrupt, your agent feels like an IVR with a nicer voice.
Two definitions to keep straight:
- Full-duplex: your system is always listening and can speak at the same time.
- Barge-in: if the user starts speaking while the assistant is talking, you interrupt the assistant and prioritize the user.
The barge-in state machine (copy/paste logic)
You need a tiny state machine. Don’t sprinkle if statements across random callbacks and hope it behaves.
States:
LISTENING(mic frames go up)SPEAKING(assistant audio is playing)BARGE_IN(user speech detected during speaking)RECOVERING(clearing buffers, syncing with model)
Transitions:
- Start in
LISTENING. - When you receive assistant audio deltas, enter
SPEAKING. - If VAD fires
speech_startwhileSPEAKING:- stop playback immediately (or duck to -18dB)
- clear playback jitter buffer
- send a control message upstream: “user barge-in start”
- enter
BARGE_IN
- Continue sending mic audio. When VAD fires
speech_end, enterRECOVERING:- send “user barge-in end”
- wait for model to acknowledge and resume
Concrete timing targets:
- detect VAD start within <50ms of speech onset
- stop or duck playback within <30ms after VAD start
If you can’t hit those, barge-in feels like the assistant is fighting the user for the floor.
VAD detection and TTS stop/duck
In the browser, you can start with a lightweight energy-based VAD, then upgrade.
- energy VAD: cheap, okay for a close-talk mic
- WebRTC VAD: better in noisy environments
On interruption, I prefer a hard stop over ducking for assistants. Ducking is great for “navigation voice” overlays. Assistants should yield.
Also: when you stop playback, flush the queued assistant audio. If you keep it and resume later, you’ll restart mid-sentence after the user finishes. It’s uncanny.
Keeping the Live session coherent
The model is receiving mic audio continuously. Your app also needs to convey “that last assistant utterance is cancelled.”
Some APIs support an explicit “cancel response” message. If not, the fallback is:
- stop playback locally
- send a new user turn (“Actually, I’m interrupting: …”) as the authoritative next turn
This is one of those things where the boring answer is actually the right one. A voice agent is still a turn-taking system. Full-duplex is a UX layer, not a logical free-for-all.
For more on building interruption-safe systems, my webhook idempotency post is surprisingly relevant: retries, ordering, idempotency.
Tool calls in Live sessions: async UX, cancellation, and idempotency
Google explicitly calls out asynchronous function calling as a key capability: “execute API and tool calls in the background while continuing to stream audio responses” (Thor 雷神 Schaeff, Google AI).
That’s the whole ballgame for voice.
Because without async narration, tool calling creates dead air.
Pattern: narrate progress while the tool runs
I bake this into the system instruction:
- immediate acknowledgement (under 300ms): “One sec.” / “Let me check.”
- progress if >1s: “Still loading that.”
- completion: “Got it.” then answer
Reid Marlow’s post nails why. You can’t make deep reasoning and streaming run on the same serial thread and expect good UX.
Pattern: tool calls must be cancelable
The ugly production scenario:
- User: “What’s my next meeting?”
- Agent starts a calendar tool call.
- While it’s fetching, user barges in: “Actually cancel it.”
If you don’t handle this, you either:
- answer the original question (wrong)
- or you cancel everything and lose session context
What I do instead:
- every tool call gets an
idempotencyKeyand aturnId - if barge-in starts, mark all in-flight tool calls as “stale unless they match current turnId”
- do not try to “kill” HTTP requests unless your infra actually supports it. Just ignore late results.
This is the same control-flow thinking as agent tool failure testing and non-deterministic AI system testing.
Tool server contract (Node/Express)
This is a minimal tool executor endpoint that supports:
- timeouts
- idempotency keys
- stale result suppression
// server/tools.ts
import express from 'express';
import crypto from 'crypto';
const app = express();
app.use(express.json());
type ToolRequest = {
toolName: string;
args: any;
idempotencyKey: string;
turnId: string;
};
const seen = new Map<string, { createdAt: number; result: any }>();
const TTL_MS = 10 * 60 * 1000;
function now() {
return Date.now();
}
function gc() {
const t = now();
for (const [k, v] of seen.entries()) {
if (t - v.createdAt > TTL_MS) seen.delete(k);
}
}
app.post('/tools/execute', async (req, res) => {
gc();
const body = req.body as ToolRequest;
if (!body?.toolName || !body?.idempotencyKey || !body?.turnId) {
return res.status(400).json({ error: 'missing fields' });
}
// Idempotency
const cacheKey = `${body.toolName}:${body.idempotencyKey}`;
if (seen.has(cacheKey)) {
return res.json({ ok: true, cached: true, result: seen.get(cacheKey)!.result });
}
// Timeout wrapper
const timeoutMs = 2500;
const timeout = new Promise((_, reject) =>
setTimeout(() => reject(new Error('tool timeout')), timeoutMs)
);
try {
const result = await Promise.race([runTool(body.toolName, body.args), timeout]);
seen.set(cacheKey, { createdAt: now(), result });
return res.json({ ok: true, cached: false, result });
} catch (e: any) {
return res.status(502).json({ ok: false, error: e?.message ?? 'tool failed' });
}
});
async function runTool(toolName: string, args: any) {
// Replace with your real tools.
if (toolName === 'random_joke') {
return { joke: 'I tried to catch fog yesterday. Mist.' };
}
if (toolName === 'lookup_order') {
// Simulate I/O
await new Promise((r) => setTimeout(r, 600));
return { orderId: args.orderId, status: 'shipped', etaDays: 2 };
}
throw new Error(`unknown tool: ${toolName}`);
}
app.listen(3001, () => console.log('tool server on :3001'));Notice what’s missing: “just retry forever.” Voice assistants can’t do that.
If a tool hasn’t returned in 2.5 seconds, you should either:
- fall back to a simpler answer
- ask a clarifying question
- or explicitly offer to continue in the background
That’s a product decision, not a model setting.
Prompting/system instructions for voice: brevity and alphanumeric precision
Google calls out alphanumeric precision and “incremental content updates” as Live capabilities.
But you still have to make the model behave.
My default voice system instruction rules:
- 1–2 sentences max per response unless the user asks for detail
- confirm critical numbers and letters (confirmation codes, addresses, order IDs)
- when tool calling, narrate progress in short phrases
- never read large JSON blobs aloud
If you want a broader prompting framework for agentic systems, I’ve written about context engineering and why it beats raw prompt engineering in production.
Gemini 3.5 Transcribe alongside Live (when you still want STT)
Gemini 3.8 Live is speech-to-speech, so yes, you can build without a dedicated STT model.
But there are legit reasons to add STT anyway:
- captions and accessibility
- searchable logs
- analytics (intent classification, funnel drop-off)
- compliance workflows (where you don’t want to store raw audio)
Google’s announcement post says Gemini 3.5 Transcribe supports 85+ languages and reports 4.0% WER streaming and 2.6% WER non-streaming (Thor 雷神 Schaeff, Google AI) (Thor 雷神 Schaeff).
Those are solid numbers for production transcription.
Pattern: parallel STT for captions + logging
Run Transcribe in parallel with Live:
- Live drives the conversational loop
- Transcribe generates a “best effort” text transcript stream
If they diverge, that’s fine. The transcript is for users and logs, not for controlling the agent.
This also makes reconnection smoother. If Live drops, you still have a recent transcript to show the user and to prime a new session.
If you’re serious about production logging policies, connect this to your data retention and redaction strategy. I go deep on that in LLM data leakage.
Session management + ephemeral tokens (browser/mobile security model)
Do not ship your long-lived API key to the browser. Seriously.
The Live API docs explicitly discuss ephemeral tokens and session management. The model is a long-lived socket. Your auth should be short-lived too.
How ephemeral tokens should work
The clean pattern:
- user signs into your app
- your backend validates the user session
- backend mints an ephemeral Live token with a short TTL (think minutes)
- client opens WebSocket with the ephemeral token
- client refreshes before expiry or reconnects with a new one
This gives you:
- revocation by cutting off token minting
- rate limiting at your gateway
- audit logging
Minimal token mint endpoint (Node)
This is a skeleton. You’ll need to wire it to Google’s token minting mechanism as described in the docs.
// server/ephemeral-token.ts
import express from 'express';
const app = express();
app.get('/api/live/ephemeral-token', async (req, res) => {
// 1) Authenticate user session (cookie/JWT/etc)
const userId = req.header('x-demo-user') ?? null;
if (!userId) return res.status(401).json({ error: 'unauthenticated' });
// 2) Rate limit by userId/sessionId (not shown)
// 3) Mint ephemeral token via Google API (pseudo-code)
// const token = await mintGoogleLiveEphemeralToken({ userId, ttlSeconds: 300 });
const token = 'EPHEMERAL_TOKEN_FROM_GOOGLE';
const expiresAtMs = Date.now() + 5 * 60 * 1000;
res.json({ token, expiresAtMs });
});
app.listen(3000, () => console.log('app server on :3000'));Production notes I’ve learned the hard way:
- make token minting its own service if you expect load spikes
- log token issuance (userId, sessionId, TTL) but never log raw tokens
- fail closed. If you can’t mint a token, don’t “fallback to using the main key”
If you’re building this as part of a broader agent platform, read my AI security and prompt injection work. Voice doesn’t change the threat model. It just adds more sensitive data.
Reconnection strategy for flaky networks (and what to do with partial audio)
Long-lived sockets drop. Mobile networks are rude.
A reconnection strategy that works:
- heartbeat/ping every 5–10 seconds
- if no pong within 2 seconds, mark degraded
- if socket closes:
- stop playback
- keep capturing mic but buffer at most 500ms (25 frames)
- reconnect with exponential backoff (250ms, 500ms, 1s, 2s) up to a ceiling
- on reconnect, send a compact “session resume” message including:
- last transcript text (last 1–2 user turns)
- any tool calls in flight (by idempotencyKey)
Do not try to “replay” several seconds of mic audio. You’ll create hallucinated turn boundaries and the model will respond to stale speech.
Also watch for VAD false positives. If your VAD triggers on keyboard clicks, you’ll barge-in constantly. A practical mitigation is to require N consecutive frames over threshold (e.g., 3 frames = 60ms) before declaring speech.
A few production concerns competitors skip (but you can’t)
Most competitor posts are announcement-level. Fine. But if you’re building this for real users, you need to care about the unsexy stuff:
- Abuse prevention: rate limit by user/session and by audio minutes.
- PII: treat raw audio as sensitive by default. Decide retention up front.
- Logging: store timestamps and metrics, not audio blobs, unless you have a reason.
- Tool safety: tools should have allowlists and scoped permissions.
This is where I lean on the same controls I use for LLM security and AI security. Voice agents increase the blast radius because they’re always-on and they feel human.
One internal data anchor from my own work: I maintain live model pricing comparisons at kunalganglani.com/llm-prices. Voice agents are usage-heavy. Before you ship, model your spend as “audio minutes per day,” not “tokens per request.” Per-token math hides the real bill.
What to build next
If you implement full-duplex, barge-in, and async tool calls, you’re already ahead of 90% of the “talking agent” demos.
The next edge is boring and non-negotiable. Measure everything and ship guardrails.
In 2026, the voice agents people keep are the ones that never make them say “hello?” twice. Your job is to make silence impossible.
Photo by MARCO on Unsplash.
Kunal Ganglani (2026, September 18). How to Build a Gemini 3.8 Live Voice Agent [2026 Tutorial]. Kunal Ganglani. Retrieved September 18, 2026, from https://www.kunalganglani.com/blog/gemini-3-8-live-voice-agent-tutorial
Frequently Asked Questions
What is Gemini Live API and how does it work for real-time voice?
Gemini Live API is Google’s interface for low-latency, real-time interactions over a long-lived session. Instead of sending one request and waiting, you stream audio (and optionally text or images) continuously and receive streaming responses back, including spoken audio. That session model is what enables natural “back and forth” voice experiences.
How do you implement barge-in (interruptions) in a streaming TTS voice assistant?
You need two pieces: voice activity detection (VAD) on the microphone stream, and a playback controller that can stop or duck the assistant audio instantly. When VAD detects the user speaking while the assistant is talking, you stop playback, flush queued audio, and treat the user’s speech as the next turn. Without that explicit state machine, interruptions will feel laggy or confused.
What’s the difference between Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking?
Gemini 3.8 Live is optimized for low-latency, speech-to-speech conversations that need to stay responsive. Gemini 3.8 Live Extended Thinking is intended for more complex requests where deeper reasoning or tool orchestration takes time. The practical difference is that Extended Thinking is designed to keep the conversation moving while heavier work runs in the background.



