Google AI Releases Gemini 3.5 Transcribe: A Speech-to-Text Model Reporting 2.6% Average WER Across 85+ Languages 谷歌AI发布Gemini 3.5 Transcribe：跨85+语言的语音转文本模型，平均词错误率仅2.6%

TL;DR

• Google released Gemini 3.5 Transcribe as two distinct API endpoints: gemini-3.5-transcribe (Interactions API for pre-recorded audio) and gemini-3.5-transcribe-live (Live API for bidirectional streaming), each with different feature sets, limits, and pricing.
• Word error rates of 4.0% (streaming) and 2.6% (non-streaming) per Artificial Analysis, with a 70% improvement in time-to-final-transcription over the previous Chirp 3 model.
• Smart mode (disfluency removal, self-correction resolution) cannot be combined with word-level timestamps or speaker diarization, forcing a design tradeoff between readable summaries and auditable transcripts.
• Hard constraints include 10-minute live session limits, no speaker diarization or word timestamps on the Live API, and API-only deployment with no open weights or self-hosting option.
• Coverage spans 85+ languages with automatic detection and mid-sentence code-switching support; ecosystem integrations include LiveKit, Pipecat, Agora, Vercel, and Google's own consumer products.

Why It Matters

This release establishes a new benchmark for production-grade speech-to-text with industry-leading latency improvements, making it immediately relevant for anyone building real-time voice agents, live captioning systems, or multilingual transcription pipelines. The deliberate split between streaming and batch endpoints—and the mutual exclusivity of smart mode with diarization/timestamps—forces architectural decisions that will shape how developers design voice-driven applications going forward.

Technical Details

• Dual API architecture: The Live API delivers sub-second continuous transcription via 100ms chunks of raw 16-bit PCM at 16kHz mono, emitting interim_input_transcription for partials and input_transcription on turn finalization, with support for automatic, hybrid, and manual voice-activity detection. The Interactions API handles pre-recorded files up to 1 hour (30 minutes with diarization or word timestamps enabled) and provides speaker diarization, word-level start/end offsets, and custom vocabulary biasing (up to 1,000 terms, optimal below 100).
• Verbatim vs. Smart modes: Verbatim returns raw transcription including fillers and false starts; Smart removes disfluencies and resolves spoken self-corrections inline (e.g., "Um, so for the meeting, I think we should, uh, invite Alice and, wait no, Bob and Carol" becomes "For the meeting, I think we should invite Bob and Carol"), but Smart mode is mutually exclusive with word timestamps and diarization.
• Performance benchmarks: Artificial Analysis reports 4.0% WER for streaming and 2.6% for non-streaming; multilingual FLEURS benchmark shows 5.50% streaming and 5.04% non-streaming across top languages and locales. Language coverage exceeds 85 locales with automatic detection and code-switching handled without configuration.
• Deployment model: API-only managed service with no open weights. Pricing approximately $0.005/min for batch and $0.009/min for live. Developer track available via Gemini API free tier on Google AI Studio; enterprise track through Gemini Enterprise Agent Platform with provisioned throughput, compliance controls, and volume discounts. Both tracks currently in public preview.
• Ecosystem integrations: Live API already integrated with LiveKit, Pipecat, Agora, Fishjam, Vercel, and Vision Agents. Powers Rambler on Android, Gemini app on macOS, Google Antigravity, with Chrome integration forthcoming.

Industry Insight

• The forced separation between streaming and batch capabilities—particularly the inability to combine Smart mode with diarization or word timestamps—means production systems will likely require dual API calls for applications needing both readable summaries and auditable records, increasing latency and cost; architects should plan for this tradeoff early rather than discovering it post-deployment.
• The 10-minute hard limit on Live API sessions and the absence of speaker diarization in streaming mode signal that Google is targeting real-time voice agent use cases over long-form meeting transcription; teams building enterprise meeting tools should rely on the Interactions API for post-call processing rather than attempting to stretch the Live API beyond its intended scope.
• The API-only, no-open-weights strategy reinforces the trend of major providers locking transcription capabilities behind managed services, which benefits teams prioritizing speed-to-market but creates vendor lock-in risk for regulated or cost-sensitive organizations; the enterprise track with content-non-improvement guarantees and compliance controls is a meaningful differentiator for healthcare, legal, and financial verticals.

TL;DR

• Google发布Gemini 3.5 Transcribe语音转文本模型，分为两个独立端点：Interactions API（预录制文件）和Live API（实时双向流式）
• 词错误率（WER）达4.0%（流式）和2.6%（非流式），最终转录时间较Chirp 3提升70%，支持85+语言及句中代码切换
• 提供verbatim和smart两种输出模式，smart模式可自动清理口语赘词但无法与词级时间戳或说话人分离功能同时使用
• 严格API限制：Live API单次流式上限10分钟，Interactions API标准支持1小时音频，启用说话人分离后降至30分钟
• 仅通过API提供，无开源权重，定价约$0.005/分钟（批量）和$0.009/分钟（实时），已集成LiveKit、Pipecat、Agora等生态

为什么值得看

本文详细拆解了Google最新语音转文本模型的技术架构与API设计取舍，对需要构建实时语音交互系统的开发者和企业具有直接参考价值。两个端点的功能差异和限制条件揭示了生产级语音AI系统的工程权衡，帮助从业者避免部署陷阱。

技术解析

双端点架构设计：Gemini 3.5 Transcribe拆分为gemini-3.5-transcribe（通过Interactions API处理预录制文件）和gemini-3.5-transcribe-live（通过Live API处理双向流式）。两者功能集、限制和定价均不相同，需根据场景选择。Live API支持100ms音频块、自动/混合/手动语音活动检测，以及临时令牌（Ephemeral tokens）供移动端无需持有API密钥即可流式传输。

输出模式与功能取舍：verbatim模式保留完整原始转录（含填充词、重复和口误），smart模式自动清理赘词、内联修正并应用结构化格式。关键限制是smart模式无法与词级时间戳或说话人分离（diarization）同时使用，意味着可读摘要和可审计转录需分别调用API。

性能指标与基准：经Artificial Analysis测量，流式WER为4.0%，非流式为2.6%；在多语言FLEURS基准上，流式5.50%，非流式5.04%。相比前代Chirp 3，最终转录时间缩短70%，支持85+语言自动检测及无配置代码切换。

部署与成本：纯API服务，无开源权重或自托管选项。开发者可通过Google AI Studio免费层起步，付费层提供更高速率限制和内容不用于改进Google产品的保证；受监管企业可通过Gemini Enterprise Agent Platform获取预留吞吐量、合规控制和批量折扣。当前处于公开预览阶段。

行业启示

语音AI产品架构需重新审视API设计：Google将流式与非流式拆分为两个独立端点且功能不对等，反映出实时语音交互与批量转录在工程实现上的根本差异。开发者在选型时应明确场景需求（延迟敏感vs.精度优先），避免因功能限制导致返工。

"智能清理"与"可审计性"的权衡将成为行业标配：smart模式与时间戳/说话人分离的互斥设计，揭示了语音AI在用户体验和数据合规之间的深层矛盾。企业级应用（如医疗记录、法律取证）必须建立双轨API调用策略，同时满足可读性和审计需求。

语音转文本正从工具层向平台层演进：已集成LiveKit、Pipecat、Agora等实时通信框架，并 powering Rambler、Gemini app等消费级产品，表明Google正将语音能力嵌入更广泛的AI Agent生态。开发者应关注API与现有语音基础设施的兼容性，抢占实时语音Agent赛道的集成先机。