The three previous approaches to video chat, and where each fell short

To see why this move matters, you need to know the three paths to "see-and-talk" that came before—each one got stuck somewhere.

Path one: the relay race. Speech-to-text, a vision model to analyze frames, an LLM to think, text-to-speech to talk—four modules in a row, each waiting for the one before it. Latency stacks up with every hop. Worse, your tone, pace, and hesitations get flattened at the "convert to text" step; all the next model sees is a bare string of words. This is technically called a cascade system. Think of it as a relay: every handoff loses a little something.

Path two: one model takes audio in and produces audio out directly, with no intermediate text. That's the end-to-end approach. It's faster and smoother, but it has ears and no eyes. OpenAI's GPT-Live from July and ByteDance's own Seeduplex from April both sit here.

The third path is the one that looks solved but isn't

Path three: it can see your camera, but the part that decides if you've finished talking is still a little module that only hears audio. That module is called VAD (voice activity detection). Its job is almost sad in its simplicity: it just detects "is someone making noise right now?" It guesses whether you're done based on silence. Guess too early—it cuts you off mid-sentence. Guess too late—it stares blankly after you've finished.

Take Google's Gemini Live as the example. It can indeed take camera and screen input. But LiveKit's official integration docs spell it out: Gemini Live API's turn detection is "built-in VAD-based," enabled by default. The same docs give the frame extraction cadence: 1 frame per second while you speak, dropping to 1 frame every 3 seconds when you're quiet.

The frame rates are LiveKit's defaults, not a hard limit of Gemini itself—you can tune them. But the VAD-based turn detection is baked into the Gemini Live API.

The change this time: video now has a say in when to speak

SeedRealtime does one thing: it unifies audio, video, timing, and expression into a single end-to-end model. No more "listen first, then look, then answer." On a continuous audio-video stream, it watches, listens, and computes at every moment—constantly deciding: speak now, keep looking, or stay quiet.

The bolted-on VAD no longer has a reason to exist. Turn detection goes from an external rule to an internal decision the model makes every instant—and that decision can see the video.

On the engineering side, the official page (and only there, not in the WeChat post) mentions two things: shorter response times via continuous audio-video chunked input and streaming output, plus better serving efficiency through quantization and inference optimizations. No specific latency numbers.

Two hard parts: video has no pauses to lean on, and "this" can mean anything

Handing the decision to the model sounds simple; in practice, it gets stuck in two places. The team flagged both themselves.

Challenge one: video has no pauses to borrow

Speech naturally has pauses, so you can use a silence as a "done talking" signal. Video is different—it's always on, always changing, and has no natural breakpoints. The model has to keep understanding what's happening in the frame, avoid interrupting just because someone in the background is talking, and continuously decide which object to track, whose voice to follow, and whether to respond at all right now.

Challenge two: when you say "this," which thing do you mean?

When you ask "how does this work," what is "this"? The model has to combine the current frame, your gesture, your gaze, and what you did a few seconds ago to line it up. Homophones or mumbled words also need the scene to disambiguate—if you're at a Sichuan restaurant, "yuxiang" won't get misheard as something else.

Only by modeling audio, video, and timing together can the model truly connect what it sees, hears, and says. That's also why "getting video into the decision" can't be done by bolting a vision module onto an old system.

The seven official demo videos

The team released 7 full-length demos, each 1–3 minutes, grouped into three capability sets. First, a look at what it's like in the app: the call screen always shows whether it's speaking or listening:

Top-left corner reads "Speaking…". Source: ByteDance Seed

This moment reads "Listening…". Source: ByteDance Seed

Group one: recognizes who's in the frame, and who's talking

Four friends at dinner, everyone talking at once. The user introduces each person; the model matches names to faces using visual features—Qiqi with light hair, Julia with glasses—and even greets Lele unprompted. Then the conversation turns to travel: one wants beach photos and an aquarium, another hates heat and fatigue, someone else has a seafood allergy. The model tracks who said what and ends up proposing a plan that fits everyone. Recognizing people, voices, and individual needs—one model, in real time, in the same conversation.

At a Sichuan restaurant, a foreign diner stares at the Chinese menu. The model identifies dishes from the video, recommends in English, and goes deeper: why "fish-flavored shredded pork" has no fish, how century eggs are made. When the server casually remarks "this goes great with rice," the model understands from the food in the frame and translates it for the guest. Visual info doesn't need to be turned into text first—whatever's in front of you, it can answer about.

Group two: it takes initiative without being asked

At the Hebei Museum, the user says, "Remind me when I see the gold-and-silver copper tiger-deer screen base," then wanders off. As the camera moves and the exhibit comes into view, the model speaks up. It then notices details and explains the gold-and-silver four-dragon four-phoenix square base and the Changxin Palace Lamp, plus techniques like casting, gold-silver inlay, and soldering. This one shows the model holding a task in context and speaking up when the target appears.

The user pours whole coffee beans directly into the portafilter. The model sees it and immediately corrects: "You can't pour beans in like that—they need to be ground first." After extraction, it looks at the crema's color and volume and proactively suggests, "Try 2–3 seconds less next time." Throughout, the user never asks—the model just watches and acts.

While reading the ResNet paper, the model walks through skip connections and how they ease vanishing gradients, using the network diagram. Then the user says, "Keep an eye out—stop when you get to the training parameters." As pages fly by, the model keeps watching the frames, catches "3.4 Implementation," stops, and reads out the learning rate, momentum, and weight decay settings.

Group three: in the chaos, it knows whose voice to follow

Daxing Airport is packed and loud. A companion casually mentions "Old Li's flight"—the model doesn't react; that's not directed at it. When the user formally asks, even though the flight info has already scrolled off the screen, it answers the arrival time from what it saw earlier and adds the baggage carousel number via a live lookup. As the two walk and chat, it catches a ride-hailing sign and naturally chimes in with directions to the pickup point.

Mom is busy nearby and asks the model to help her daughter practice animal words in English. Dad is on a call in the background, voices everywhere. The model doesn't get pulled away—it stays on the girl's pointing finger, correcting pronunciation and making example sentences in real time.

The official stat: half as many conversational stumbles as the old way

The demos look great, but there's exactly one quantitative claim: in end-to-end human evaluation, audio-video conversations have half as many pacing issues as the cascade model. "Pacing issues" covers three kinds of stumbles: getting cut off mid-sentence, waiting forever for a response, and false triggers from background noise or side chatter.

There's also a line about "a marked improvement in the probability of a complete, smooth single conversation," without a number.

The absences deserve their own bullet list: end-to-end latency in milliseconds (the hardest metric in real-time interaction), model size, any public benchmark, or third-party replication—none are provided. The roadmap even says future work includes "further reducing heard-understand-respond end-to-end latency," which is an admission that there's room to compress, but the current baseline goes unmentioned.

For context, the previous voice-only Seeduplex, launched in April, gave more granular numbers: misreply and misinterruption rates cut in half, speaking-over proportion down 40% relative, pause-detection up 8%, and overall call satisfaction up 8.34 points absolute in a large A/B test. Same team, four months ago—the disclosure was considerably more detailed.

How ByteDance's line got here, and where OpenAI and Google stand now

On the timeline, this move's position is clear. ByteDance's line has taken four steps:

Where the other two sit: OpenAI's GPT-Live, out July 8, is full-duplex, but it doesn't yet support camera or screen sharing in ChatGPT—"later," per the official line. Google's Gemini Live has done camera and screen for a while, with turn detection handled by built-in VAD as noted.

So ByteDance's claim of being "first in the industry to scale audio-video full-duplex" checks out against these two: the one that can see still delegates turns to VAD; the one that's most fully full-duplex hasn't hooked up a camera. This check is bounded—it only covers the public status of OpenAI and Google, not the rest of the domestic field.

How to use it: update Doubao and tap "Call"

The entry point is short: update the Doubao app, tap "Call" in the chat box, and you're in video-call mode. Free, no application needed. That's the only announced entry—no API or developer access. For reference, the previous Seeduplex launched in Doubao on April 9 and only got its Volcano Engine API on June 18.

There were three ways to get an AI to see and talk. The first two have obvious flaws. The third is the most dangerous because it looks solved: the camera is connected, sure, but the "are you done talking" call still sits with VAD (voice activity detection—a tiny module that only checks whether someone is making noise). It guesses from silence: too early, it cuts you off; too late, it stares at you after you've finished.

ByteDance's claim of being "first in the industry to scale audio-video full-duplex" checks out against these two: Gemini Live can see, but turns still go to VAD; GPT-Live is the most fully full-duplex, but it hasn't hooked up a camera in ChatGPT. This check only covers the public status of OpenAI and Google.

SeedRealtime unifies audio, video, timing, and expression into one end-to-end model (one model, direct in and out, no intermediate text). The bolted-on VAD no longer has a reason to exist: turn detection goes from an external rule to an internal decision the model makes every moment—and that decision can see.

And here's the hard part. Speech has natural pauses to lean on; video is always on, with no natural breakpoints. And when you say "how does this work," the model has to combine the current frame, your gesture, your gaze, and your last few seconds of actions to figure out what "this" is.

ByteDance Seed released 7 full-length demos (1–3 min each), in three groups:

Three groups, one thread: because video is in the decision, it can speak up when the target appears, can't be dragged off by side chatter, and can resolve what "this" means.

There's exactly one quantitative claim: compared with the relay cascade model, pacing issues in audio-video conversation are cut in half—that's three kinds of stumbles: being cut off mid-sentence, waiting forever for a reply, and false triggers from background noise or side chatter.