26 AI Voice Models Tested: Why Do Speed, Accuracy and Naturalness Have Different Leaders?

The Cekura team evaluated 26 text-to-speech (TTS) models by invoking them from the same US East cloud region to read the exact same set of 104 English phone call sentences three times each, testing different scenarios including full-sentence input, word-by-word streaming input, and user interruptions. The team reported that the overall benchmark comprised 30,564 requests across multiple test conditions, rather than thirty thousand unique sentences.

Each model was tested using a designated stock US English female voice. This comparison reflects a specific set of APIs, voices, and network locations; its findings cannot be directly extrapolated to Chinese speech or custom voice cloning.

Across the exact same set of models, changing the evaluation metric completely shifts the leader:

• Full-sentence input latency: Inworld TTS-2 Flash led with a median time-to-first-audible-sound of 46 milliseconds;
• Word-by-word streaming latency: Gradium registered a median of −64 milliseconds, demonstrating "negative latency" by speaking before the client had even finished sending the final word of the sentence;
• Numbers and technical terminology accuracy: Inworld TTS-2 reached 93.4%, forming a top tier alongside Inworld TTS-2 Flash (93.2%) and Fish Audio S2.1-Pro (92.0%);
• Predicted voice naturalness: Fish Audio Drama 3 led with an estimated score of 4.54 out of 5.

No single model ranked first across all metrics. Speaking quickly, pronouncing telephone numbers accurately, sounding natural, and properly clearing previous audio when a user interrupts each challenge distinct technical capabilities. Choosing a voice for a Voice Agent requires first identifying which failure mode your product can least afford.

Why Does the Leader Change for the Exact Same Sentence Under a Different Input Method?

When submitting written text to a batch speech tool, the entire passage can be dispatched at once. Real-time conversational voice agents, however, frequently speak while still generating their replies. As large language models generate text incrementally, an Agent can forward finished phrases to the TTS engine, allowing speech synthesis to overlap with ongoing text generation.

This benchmark evaluated two distinct text input modes:

1. Full-sentence input mode (Non-streaming): The client transmits the complete sentence in a single payload, measuring latency from the moment of dispatch to the arrival of the first audible sound. Inworld TTS-2 Flash recorded a median of 46 milliseconds, Gradium clocked 76 milliseconds, and Inworld TTS-2 took 126 milliseconds. In production, systems can also dispatch natural phrasal chunks rather than rigidly waiting for a terminal period.
2. Word-by-word streaming mode (Streaming): The client streams text word by word, giving the TTS engine an opportunity to start speaking before the entire sentence has finished transmitting. The benchmark simulated an input cadence of 30 words per second and measured latency from the dispatch of the final word to the first audible sound: Gradium recorded −64 milliseconds, Inworld TTS-2 Flash recorded 45 milliseconds, and ElevenLabs v4 Turbo recorded 69 milliseconds. The 30 words-per-second pace is a benchmark testing parameter, not a fixed generation rate across all large language models.

The 8 models omitted from the scatter plot—including GPT-4o mini TTS, Gemini 3.8 Flash, ElevenLabs v3, and Qwen3-TTS—did not accept word-by-word streaming input under the tested interfaces and configurations. Even if these models can stream their synthesized audio chunks back to the client, that remains fundamentally different from beginning synthesis before incoming text transmission finishes. This distinction shapes how pipelines must be architected, though it should not be taken as proof that these models cannot achieve low latency under alternative deployment architectures.

Median numbers also fail to reveal how long outlier delays can stretch. P90 is the wait time to the first audible sound that roughly 90% of tests do not exceed; the slowest 10% wait longer. Under the identical full-sentence first-sound measurement:

• Inworld TTS-2 Flash: median of approximately 46 milliseconds, with a P90 around 73 milliseconds;
• Smallest Lightning v3.1 Pro: median of approximately 357 milliseconds, with a P90 around 679 milliseconds.

For the latter, the P90 latency is nearly double its median. Selecting a model requires evaluating both the typical interaction rhythm and how severely slow requests degrade; P90 itself is still not the absolute maximum wait time.

Deconstructing "-64 ms": Is Negative Latency Time Travel or a Different Starting Point?

"Negative latency" is an artifact of where the timing clock begins; it does not indicate that a model anticipates text it has not yet received.

The reference zero point for streaming latency is defined as the exact moment the client pushes the final word of the sentence into the TTS API.

Consider this measurement convention using a 10-word sentence: the timestamp when the final word is transmitted is set to 0. If earlier words have already triggered synthesis and the first audible sound emerges 64 milliseconds before that zero mark, the result is logged as −64 milliseconds. This is merely an illustrative timing example; the −64 milliseconds on the leaderboard represents a median across multiple runs, meaning it does not guarantee every sentence starts that early, nor does it reveal how many words the model needed to receive before computation began.

Connection overhead and end-to-end conversational pipeline delays also reshape the practical significance of these numbers.

• Pre-warmed connections exclude handshakes, but network round-trips remain included: Protocol handshakes, authentication, and context initialization are not counted in the timing; however, the interval from the client sending text to receiving audio and detecting sound is included. Consequently, testing from a different network location may yield different real-world latencies.
• TTS time-to-first-sound is only one segment of an end-to-end conversation: A full conversational exchange also entails determining when the speaker has finished, speech recognition, reply generation, and client audio playback—stages that can also partially overlap. The 46-millisecond figure measures solely the window from transmitting complete text to the first audible sound over an established connection; it cannot be treated as the total round-trip time between a user finishing speaking and hearing the AI's reply.

Packet Arrival Is Not Speaking: The Leading Silence Trap

In voice interfaces, the arrival of the first data packet (TTFB) does not guarantee that human ears can hear any sound.

The beginning of a returned audio stream may contain a stretch of sound below the audible detection threshold. If an evaluation only logs when the first packet lands, the system will appear to have responded while the user may still hear no speech.

This benchmark compared models using time to first audible sound (TTFA): it first records packet arrival and then factors in any low-volume silence preceding audible speech. The detection pipeline applies a 10-millisecond DC-removed audio window sliding in 1-millisecond steps, identifying the point where the root-mean-square (RMS) amplitude first surpasses 1% of full scale. This represents a standardized automated detection rule rather than a subjective hearing assessment across individual listeners.

Under full-sentence testing, detected median leading silence durations included:

• Deepgram Aura-2: 223 milliseconds;
• Cartesia Sonic 3.5 and Sonic 3.6: 132 and 142 milliseconds, respectively;
• Smallest Lightning v3.1 Pro: 141 milliseconds.

These figures illustrate that receiving audio packets and hearing spoken words are two distinct events. One cannot take the median silence from a full-sentence test and append it directly to streaming test results as if they formed consecutive phases of a single recording.

What Does the Automated Accuracy Score Actually Measure?

When a voice assistant recites a telephone number, sounding smooth is insufficient if the digits are wrong; when reading street addresses, the exact same abbreviation must be pronounced differently depending on context.

The benchmark annotated acceptable pronunciations for challenging spans across conversational sentences, categorizing them into four groups:

1. Numbers and abbreviated shorthand: such as 7 a.m., 20%, 36°F, and approx.;
2. Codes and addresses: such as the coupon code SAVE20, the email help@north-clinic.org, and the toll-free number 1-800-555-0199;
3. Context-dependent abbreviations: such as St. Mary's on Main St. (where the first St. is Saint and the second is Street), and expanding Dr. to Doctor in Dr. Okafor;
4. Uncommon medical and pharmaceutical terminology: such as the medication levothyroxine and the specialty gastroenterology.

The automated pass rates across these flagged hard spans reached 93.4% for Inworld TTS-2, 93.2% for Inworld TTS-2 Flash, 92.0% for Fish Audio S2.1-Pro, and 91.3% for Fish Audio Drama 3. This metric represents neither the proportion of entirely error-free sentences nor overall call completion rates.

The scoring methodology dictates how these results should be interpreted:

• Transcribe first, then verify hard spans: Synthesized audio was submitted to two independent automated speech recognition systems—Deepgram Nova-3 and OpenAI Whisper—to check whether transcriptions contained any of the annotated acceptable pronunciations. Phone numbers and alphanumeric codes were further checked character by character. Because speech recognizers themselves make errors, these scores remain estimates.
• Two automated "listeners" can disagree: Documentation notes that the estimated pronunciation accuracy from these two recognizers could diverge by up to 10 percentage points for a given model.
• Small differences at the top should be treated as ties: The evaluation recommends treating top scores within two percentage points as ties. For instance, the difference between 93.4% and 93.2% is insufficient to claim a definitive real-world quality advantage.
• Early speech onset cannot substitute for pronunciation verification: Gradium achieved the lowest streaming latency but posted a hard-span pass rate of 79.5%; Inworld Flash recorded 45 milliseconds alongside 93.2%. This demonstrates that these two dimensions must be assessed independently, rather than assuming that fast models mispronounce words simply because they are fast.

Certain pronunciation issues cannot be resolved from transcribed text alone. For example, the present and past tenses of the word "read" differ in spoken pronunciation, yet both transcribe identically as "read". The underlying benchmark tagged such cases as requiring manual listening and excluded them from the automated accuracy metric. Consequently, achieving a "100% contextual score" does not mean that every homograph or syllable stress pattern was read correctly.

Composite scores also obscure category-specific trade-offs. Among items that automated transcription could evaluate, Gradium scored approximately 67% on contextual abbreviations but reached roughly 99% on medical terms; Inworld TTS-2 reached approximately 100% on contextual abbreviations but around 86% on codes and addresses. These figures remain categorized estimates from transcription engines. Assistants frequently reciting verification codes must explicitly validate codes and digits, rather than presuming all hard spans are reliable based on high medical terminology scores.

Interruptions and Playout Stalls: Engineering Experience Beyond Voice Timbre

When a user interrupts by saying "wait a second," continuing to stream stale audio disrupts the conversation, no matter how natural the voice sounds. Managing interruptions requires orchestrating server-side synthesis tasks and client-side playback simultaneously.

1. Protocol Reality of Interruption Mechanisms

In this benchmark, "Stopping" measured how much playable audio continued to arrive after dispatching a cancellation request, quantified in audio duration. It does not measure wall-clock response time from sending a cancellation request to the arrival of the final data packet.

• Results from available cancellation interfaces: Grok TTS and Soniox recorded a median of 0 milliseconds of extra audio following cancellation, Deepgram Aura-2 recorded 40 milliseconds, and Cartesia Sonic 3.6 recorded 764 milliseconds. The 0 figure here is a median; it does not mean zero extra audio on every single run, nor does it mean an entire phone call goes silent in zero elapsed time.
• Different reasons behind n/a designations: Gradium, Inworld, and the tested ElevenLabs v3 lacked explicit cancellation messages in the evaluated interfaces. Conversely, the close-control mechanisms tested on ElevenLabs Flash v2.5, v4 Turbo, and v4 finish generating audio for already submitted text before closing the context. Neither case should be treated as 0, nor does an n/a mean users cannot interrupt the voice in a finished production application.
• Audio players must still discard stale audio immediately: The Agent must instantly pause local playback, discard buffered audio, and cancel, close, or abandon the previous stream according to API capabilities. Otherwise, sound already buffered locally may continue playing even if the backend stops transmitting.

2. Running Out of Audio Mid-Playout (Playout Stall / Underrun)

Audio players can buffer initial frames before beginning playback while subsequent chunks continue streaming in. This operates much like a water tank filling and draining simultaneously: with an established reserve, brief supply fluctuations may not interrupt the outgoing stream.

However, if subsequent audio arrives too slowly and exhausts the buffer, playback stalls. Consequently, producing the first sound rapidly and maintaining smooth playback across an entire sentence represent two entirely different requirements.

In benchmark full-sentence playout testing, ElevenLabs v4 experienced audio supply lagging behind real-time playback on roughly 7% of sentences, while audio supply for v4 Turbo fell behind real-time playback on roughly 5% of sentences. These percentages reflect the entire test corpus across varied lengths, not solely long paragraphs.

3. Can Requests Complete Normally?

Beyond mid-sentence playout stalls, evaluations must track whether requests complete successfully under testing constraints. In the published snapshot, GPT-4o mini TTS recorded 52 timeouts across 936 tracked synthesis calls. This reflects performance under the benchmark's specific connection setup and timeout thresholds; it cannot be interpreted directly as a long-term production failure rate.

The evaluation separately cataloged account-level rejections stemming from API keys, billing balances, quotas, or rate limits, and these must not be conflated with service timeouts. Similarly, a multi-step test failing to meet cancellation criteria should not be logged as an API service timeout.

Naturalness Estimates and Practical Listening Advice

Voice naturalness was evaluated using UTMOS, an open-source model that predicts mean opinion scores on a 1-to-5 scale for full-sentence audio recordings. This metric estimates how natural human listeners might perceive the speech; the benchmark did not employ human listening panels to produce these rankings.

Fish Audio Drama 3 recorded a predicted score of 4.54, Inworld TTS-2 scored 4.52, and Gradium reached 4.51. With the top three scores clustered so closely, a 0.02-point margin does not indicate a clear real-world user preference for one over another.

Start by comparing conversational greetings to evaluate intonation and pausing, then listen to sentences containing both tenses of "read" to verify whether differing pronunciations were preserved. The players display the identical input sentence alongside reference text, leaving the final pronunciation check to human ears.

Additional voice timbres and test sentences can be compared directly on the official audio listening page.

Practical Reference for Production Selection

This leaderboard is best utilized to narrow down candidate models before validating them against your specific production sentences:

1. Evaluate conversational responsiveness first: Gradium and Inworld TTS-2 Flash can enter a low-latency shortlist. Keep the hard-span accuracy column in view to check whether each model’s pronunciation meets your application’s needs alongside its rapid speech onset.
2. Review numbers, addresses, and specialized vocabulary: Shortlist Inworld TTS-2 (93.4%), Flash (93.2%), Fish Audio S2.1-Pro (92.0%), and Drama 3 (91.3%) for deeper testing. Embed actual business telephone numbers and terminology into full production sentences and verify with human reviewers where necessary; automated English accuracy scores cannot prove reliability for Chinese-language applications.
3. Compare speech synthesis pricing models: Reference list pricing tracked in this benchmark includes Soniox TTS RT v2 at $14.81 per million characters, and Inworld Flash alongside Grok TTS at $15 per million characters; no price was provided for Drama 3 in this leaderboard. These reflect vendor quotes or standardized conversions cataloged in the benchmark rather than total telephony call costs spanning ASR, LLMs, and carrier connections, and lower list prices do not demonstrate high-concurrency throughput.

Once a short list is established, run verification tests under your target language and target network deployment: check how long users wait for initial speech, whether telephone numbers are read correctly, whether previous audio cuts off cleanly upon interruption, whether full paragraphs play without buffering stalls, and whether requests complete reliably. These operational factors determine how benchmark rankings translate into real-world product experiences.