Four Character Demos + a Real-Time Screen Recording

The four demos below all come from the same model — only the character, voice, and setting change. Let's look at the results first, then get into how it works.

The clip below is different: it's a screen recording of an actual live networked conversation, showing the full real-time response chain.

Why the Old Approach Is Slow: A Relay Race Where Every Step Waits

The old approach is slow because it's a pipeline stitched together from a chain of independent models: speech is first converted to text (ASR), the text is fed to a language model to work out an answer (LLM), the answer is then synthesized into speech (TTS), and finally a face is animated to move (animation rendering). Each stage has to wait for the previous one to hand off its output, so wait times stack up step by step, and recognition and lip-sync errors accumulate along the way.

The original writeup makes a judgment call here: real-time audio-video interaction isn't simply "multimodal understanding" plus "multimodal generation" added together — it's inherently full-duplex, so streamability is a modeling constraint, not just a post-launch engineering optimization. Systems built on offline encoders, bidirectional decoders, or turn-based dialogue can't engineer their way to true low-latency full duplex no matter how much tuning they do. That's exactly what the next section breaks down.

Its Approach: One Model Covers Everything from Listening to Speaking

Wan Streamer's core idea fits in one sentence: interleave the input and output tokens for vision, audio, and text into a single sequence, and hand it to one Transformer; block-causal attention coordinates the process so it can compute and emit output as data arrives.

The model treats interaction as one continuous causal stream: your observations and its responses jointly update the current context. In each streaming unit, it encodes whatever user observations are currently available, then predicts the next response segment based on the full causal history from both sides. The language response is a sequence of discrete tokens trained with next-token prediction; the audio and video responses live in a continuous latent space, jointly generated with conditional flow matching — conditioned on the same clean context, so speech, motion, appearance, and scene evolution are denoised together as one coupled whole, rather than generated separately and stitched together.

Wan Streamer overall framework: language, audio, and video are modeled simultaneously as input and output within a single Transformer, coordinated by block-causal attention, generating in a streaming fashion as data arrives. Source: Wan Streamer official

To sustain this stream, the entire stack is causal by design from the ground up: a strictly causal audio-video VAE for streaming latent encoding, causal audio-video encoders, causal audio-video decoders, and a temporally causal Transformer coordinated by block-causal attention. After denoising, the estimated clean latents are appended directly to the history, becoming context for subsequent units; causal decoders then render them into the final audio-video output. The external modules this design erases are:

How It Listens While Speaking, and Can Be Interrupted Anytime

Human interaction with the world is inherently streaming and full-duplex: we don't listen fully, then think separately, then finally answer — we watch, listen, and speak all at once, pausing and interrupting at will, with perception and expression overlapping on the timescale of audio and video. A real-time interaction model has to work the same way.

This mechanism works thanks to block-causal attention. It treats a small chunk — say, a 160ms audio-video segment — as one processing unit: tokens within a block can attend to each other bidirectionally, but a block can only see past blocks, never future ones. That preserves within-block context while still allowing computation as data arrives, without waiting for the whole utterance to finish.

Tokens within a block attend to each other; blocks only look leftward, into the past: block 3 can start computing as soon as it arrives, because it only depends on blocks 1 and 2, without waiting for future block 4. That's streaming generation.

Click to see deployment details: how the thinker–performer split gets latency down to 200ms

Wan Streamer is trained as a single end-to-end model; for real-time deployment, that same model is split into a thinker–performer pipeline spanning two GPUs, designed to overlap computation as much as possible. Once the system finishes prefill, the thinker broadcasts the initial KV cache to the performer, and both sides share the same full-history state — the unified model's behavior is preserved completely.

The thinker handles the causal audio-video encoder, a short computation for language prediction and state updates, KV-cache construction, and decoding the previous unit's latent into audio-video for immediate output. The performer handles only latent generation — running the flow-matching solver for the next audio-video unit based on the shared full-history KV context. Because the performer never runs the decoder and the thinker never runs the costly solver, decoding and generation never block each other.

As long as the performer's compute time plus communication time fits within one 160ms unit, real-time throughput holds. The signal-to-signal path — "encode → update state → generate latent → decode" — is exactly the roughly 200ms of model-side latency, kept within budget through CUDA graph capture, compilation, and optimized kernels.

thinker–performer streaming inference overlap: for unit k, the thinker encodes the current observation, updates the KV cache, and decodes the previous unit's latent for immediate output; the performer only runs the flow-matching solver to generate the next latent, which returns for the next unit. Perception, decoding, communication, and denoising overlap across adjacent units. Source: Wan Streamer official

How It Compares to Other Systems: What's Faster, What It Can Do

The two groups of latency numbers below measure different things and need to be read separately. The top group covers the complete end-to-end interaction loop (perceiving the user and producing a response), and Wan Streamer is the only one among them that also outputs video. The bottom group covers digital-human / audio-video renderers, counted only through the rendering stage — excluding the external language model, ASR, and TTS they depend on — so the latency users actually experience is higher than what's shown.

Capability coverage is shown below; Wan Streamer is the only row with every box checked:

✓ = yes　~ = partial support / undisclosed　✗ = no. Full duplex means the system keeps perceiving continuously while generating — understanding and responding at the same time. Cells marked "~" are either partially supported or the metric isn't publicly disclosed.

Its approach is straightforward: fold that entire relay pipeline into a single model, going from watching and listening to you straight through to responding, in one continuous motion (this is called "end-to-end" — one model handles everything from start to finish, with no hand-offs to other modules in between).

But merging several modules into one is only the first step. The real challenge is getting AI to listen and speak at once like a person does — so that when you interject, it can stop immediately — and that requires rebuilding how it processes information from the ground up.

It relies on a mechanism called block-causal attention: your video and audio are sliced into small chunks (each about 160 milliseconds), and the model computes chunk by chunk as it receives them, without waiting for you to finish your whole sentence.

Most people don't have a feel for "how many times faster" — what registers is "how long do I wait." For the same moment — you finish a sentence and wait for the AI to respond — different systems vary a lot, and Wan Streamer is the only one that responds with a moving face too.