ElevenLabs Releases Eleven Realtime Voice Models: Max Speaks in 0.57 Seconds with 98.8% Speech Reasoning Accuracy

ElevenLabs has released the Eleven Realtime family of conversational voice models (launching with two configurations: v1 Mini and v1 Max), balancing rapid speech response with spoken reasoning capabilities.

• Division of Labor Across Two Configurations: v1 Mini specializes in natural conversation and low operating costs; v1 Max specializes in multi-step logical reasoning and tool calling.
• Sub-Second First Audio Response: In benchmark testing, the average time to first audio was 570 milliseconds for Max and 620 milliseconds for Mini.
• Joint Optimization of a Cascaded System: Speech recognition, turn-taking, language modeling, and speech synthesis are co-optimized as a unified system to reduce module-to-module friction.
• Dual Integration Paths: Supported natively within the ElevenAgents platform for deployment across phone, web, and WhatsApp channels, alongside an OpenAI-compatible Realtime API for code-level migration (the API operates independently and does not include platform-level knowledge bases or guardrails).
• Voice Support and Compliance Environments: Supports over 17,500 voices and more than 90 languages; in US and EU data environments, v1 Max supports Zero Retention Mode and HIPAA compliance.

Mini Keeps Conversation Flowing, Max Looks Up the Answer

Voice Agents often face a dilemma: pursuing near-instant replies frequently sacrifices depth of reasoning, while complex operational logic brings noticeable latency. Eleven Realtime addresses this through two models with a clear division of labor:

• Eleven Realtime v1 Mini: Optimized for natural conversation and lower operating costs. Its strength lies in capturing the subtle nuances that make voice interactions feel human: accurately sensing when the speaker has genuinely finished, gracefully cutting off and resetting conversational cadence when interrupted, and not halting abruptly when the user merely utters backchannel cues like "mm-hmm" or "yeah." These characteristics make it ideal for personal AI assistants, language practice tutors, and interactive virtual characters.
• Eleven Realtime v1 Max: Built for complex operational workflows. It features stronger spoken logical reasoning, understanding multi-step requests packed with constraints and topic pivots. During conversation, it autonomously invokes backend tool interfaces to query data, update records, and take real-world action. Typical use cases include enterprise inbound call support, inbound lead qualification, and appointment scheduling.

The official demonstration video illustrates how both models handle distinct tasks:

Download Bilingual Subtitled Video (launch-bilingual.mp4) Download Bilingual Subtitle File (launch-bilingual.srt) Download Original Official Video (launch-original.mp4)

The first part of the video showcases Mini's responsiveness during a user interview: when user Sarah shares that her first month using an app was challenging and almost led her to give up, Mini responds quickly with an empathetic acknowledgement and smoothly follows up by asking what specific parts felt confusing.

The second part demonstrates Max handling an outage report for Eleven Mobile: Max acknowledges the user first, then triggers a tool query with a postal code parameter: check_network_status(postcode="10002"). Once the API returns that an outage exists in the area, engineers are on-site, and service is expected to restore within an hour, Max conveys these findings back to the customer in natural spoken language.

Co-Optimized as a Unified System, Built from Cascaded Models

Eleven Realtime employs a cascaded architecture: multiple independent models work in concert rather than functioning as a single fused model.

The role of each component across the pipeline:

1. Automatic Speech Recognition (ASR): Transcribes the user's incoming microphone audio stream into text.
2. Turn-Taking Management: Determines whether the user has finished speaking and whether incoming speech represents an interruption or a backchannel cue.
3. Large Language Model (LLM): Comprehends context, conducts logical reasoning, generates response text, and triggers tool calls.
4. Text-to-Speech (TTS): Synthesizes streaming response text into streaming audio in the selected voice.

ElevenLabs noted that underlying model specifics remain undisclosed. What the team terms "end-to-end co-optimization" refers to refining speech recognition, turn-taking, reasoning, and speech synthesis components jointly as an integrated system so they coordinate seamlessly.

The Distinction Between Time to First Audio and Task Completion

The published figures of 570 milliseconds (Max) and 620 milliseconds (Mini) specifically represent the average Time to First Audio (TTFA) measured across the Big Bench Audio benchmark. As an average from this specific test suite, this metric cannot be taken directly as real-world call latency across arbitrary networks and client devices.

Crucially, beginning to speak does not equate to completing the task. In the official outage troubleshooting demo, the Agent first informs the user that it will look up the status, subsequently triggers the network status API in the background, waits for the response indicating the disruption and estimated restoration window, and only then reports the findings. Initial speech response reflects how quickly the conversational exchange begins, while actual task completion remains dependent on tool round-trips and subsequent processing.

Speed and Accuracy Benchmark Performance

Independent evaluation platform Artificial Analysis evaluated both models across multiple dimensions, with ElevenLabs disclosing a scatter plot alongside five detailed benchmark scores.

Examining the chart's axes and notations:

• Axis Definitions: The horizontal axis measures Time to First Audio (lower values indicate faster response); the vertical axis measures Big Bench Audio spoken reasoning accuracy (higher values indicate superior performance).
• Frontier Positioning: Eleven Realtime Max sits on the dashed Pareto frontier curve along the top-left of the distribution, posting the highest reasoning accuracy within the sub-600ms latency bracket.
• Omitted Models and Parameters: The chart only displays models with time to first audio under 3 seconds, omitting 5 slower models (for instance, StepAudio3 Realtime achieved 99.7% accuracy with an 8,830-millisecond first audio latency); the original chart did not specify sample sizes. Max represents a frontier model balancing speed against accuracy, rather than holding the absolute highest accuracy or absolute lowest latency across the entire leaderboard.

Beyond the speed and accuracy scatter plot, ElevenLabs released detailed five-dimension benchmark scorecards for Mini and Max. Importantly, these five metrics evaluate different objectives across distinct benchmark suites and cannot be generalized into a single percentage accuracy.

1. Speech Reasoning (Big Bench Audio): 98.8% for Max versus 75.9% for Mini, giving Max a 22.9 percentage point advantage. This metric reflects how accurately the model parses layered logical relations and common-sense spoken reasoning.
2. Time to First Audio: 570 milliseconds for Max versus 620 milliseconds for Mini, with Max 50 milliseconds faster.
3. Task Success Rate (Speech Agent Arena): In simulated Agent workflow benchmarks, Max reached 87.1% compared to 60.3% for Mini, a 26.8 percentage point lead for Max. This metric strictly reflects results from this specific benchmark.
4. Human Preference Elo (Speech Agent Arena): 998 for Max versus 988 for Mini.
5. Conversational Dynamics (Full-Duplex-Bench): Mini achieved 95.8%, surpassing Max's 94.1% by 1.7 percentage points. This benchmark specifically measures turn-taking smoothness and handling interruptions during full-duplex conversations.

Business Scenarios and Compliance Selection

When choosing a model, evaluate the complexity of operational tasks against compliance and data residency requirements:

Regarding compliance and data residency: if an Agent requires Zero Retention Mode or operates in a HIPAA-compliant environment within US or EU data regions, developers can select v1 Max or remain in Custom mode; v1 Mini cannot be selected under either compliance setting. If the Agent is configured for data residency in India or Singapore, v1 Max is currently not supported, meaning developers can select v1 Mini or stay with Custom mode. Furthermore, when selecting v1 Max, the token ceiling must be configured to at least 1,024 tokens (inclusive) or left uncapped.

ElevenAgents Platform Configuration and Realtime API Integration

Eleven Realtime offers two parallel integration routes: ElevenAgents for low-code and enterprise workflows, and the Realtime API for code-level development.

1. Selecting the Model in ElevenAgents

Within the ElevenAgents console, the realtime models appear as a selectable Voice mode. A 7.9-second console walkthrough demonstrates the mode-switching workflow:

Download Bilingual Subtitled Video (setup-bilingual.mp4) Download Bilingual Subtitle File (setup-bilingual.srt) Download Original Official Video (setup-original.mp4)

The full configuration flow to use realtime models in ElevenAgents involves: navigating to the Agent's Agent tab, switching Voice mode to Realtime, selecting Eleven Realtime v1 Mini or Eleven Realtime v1 Max from the model dropdown, and clicking Publish to apply changes. Notably, the official demo only shows switching and selection without confirming a final successful publish; furthermore, availability is rolling out gradually—if the Voice mode option does not appear in an Agent tab, the feature is not yet active for that workspace.

Switching to Realtime mode fully manages the underlying LLM, thinking budget, reasoning effort, TTS expressiveness mode, and ASR provider. The Agent's existing system prompt, first message greeting, voice, language, tools, knowledge bases, and channel deployments across telephony, web, and WhatsApp remain fully usable.

Configuration and deployment require keeping these key migration constraints in mind:

• Custom and Third-Party LLMs: Agents utilizing custom LLMs, Amazon Bedrock, or Speech Engine cannot switch to Realtime mode and must remain in Custom mode.
• Workflow Overrides: If workflow nodes override managed settings such as LLM, TTS, or token limits, overrides must be removed or the Agent must stay in Custom mode.
• Cross-Mode Transfer Restrictions: Realtime mode Agents can only transfer calls to other Agents also configured in Realtime mode (transfers between Mini and Max are supported); cross-mode transfers to Custom mode Agents are not supported.
• Max Token Floor: v1 Max requires token ceilings to be configured at 1,024 tokens or higher (inclusive), or left uncapped.

2. Realtime API: Reusing Existing WebSocket Integrations

For developers who have already built systems on the OpenAI Realtime API, ElevenLabs provides a Realtime API that preserves wire-protocol compatibility.

Integration follows a decoupled two-step client-server model:

Step 1: Backend Mints Temporary Credential (Mint Client Secret)
Using a private master key, the backend service calls the ElevenLabs SDK to issue a short-lived clientSecret (600 seconds in documentation examples), binding initial session configurations—such as selecting eleven_realtime_v1_mini, system prompt instructions, and voice brian—directly to this credential.

Step 2: Frontend Establishes WebSocket Connection Directly
Client-side browsers can reuse OpenAI's official @openai/agents-realtime client SDK, redirecting the transport endpoint to the ElevenLabs node:

import { OpenAIRealtimeWebSocket, RealtimeAgent, RealtimeSession } from "@openai/agents-realtime";

// 1. Fetch the temporary session credential minted by your backend
const { value: clientSecret } = await fetch("/api/token").then((res) => res.json());

const agent = new RealtimeAgent({
  name: "ElevenLabs agent",
  instructions: "You are a helpful, friendly voice assistant. Keep replies concise.",
});

// 2. Direct the WebSocket transport URL to the ElevenLabs realtime endpoint
const transport = new OpenAIRealtimeWebSocket({
  url: "wss://api.elevenlabs.io/v1/convai/realtime",
});

// 3. Specify the Realtime model identifier and voice ID
const session = new RealtimeSession(agent, {
  transport,
  model: "eleven_realtime_v1_mini",
  config: {
    outputModalities: ["audio"],
    audio: {
      output: { voice: "brian" }
    },
  },
});

// 4. Connect the persistent session using the ephemeral secret
await session.connect({ apiKey: clientSecret });

Key Engineering Considerations for the Realtime API:

• WebSocket Only at Launch, No WebRTC: The Eleven Realtime API currently supports audio streaming exclusively over WebSocket. Because the official OpenAI SDK only manages browser microphone and speaker access automatically in WebRTC mode, WebSocket deployments require developers to implement client-side 24kHz PCM16 audio capture and playback streaming manually (for example, using open-source libraries like wavtools).
• Separate Capability Stacks: The Realtime API and ElevenAgents run on completely independent architectural stacks. Models connected directly via the Realtime API cannot access knowledge bases, guardrails, or telephony integrations configured inside the ElevenAgents console.

Billing Structure and Usage Costs

Eleven Realtime employs a hybrid billing model combining per-minute charges with token consumption:

• ElevenAgents Channel: Billed at standard ElevenAgents per-minute rates plus charges for actual input tokens, cached input tokens, and output tokens consumed by the selected model. The console model selector displays estimated per-minute costs.
• Realtime API Channel: Billed on connected audio duration per minute plus actual token usage, with token unit pricing held consistent across subscription tiers.

Across both models, v1 Mini incurs lower overall costs, whereas v1 Max carries a higher per-token rate than v1 Mini. Developers can choose the appropriate model tier and integration path according to operational task complexity and regulatory requirements.