Why Large Models Spit Out One Word at a Time

When a large model generates text, it produces one word after another. To spit out each word, it has to run the entire model start to finish (one forward pass); only once it has that word can it compute the next one.

Words have a strict order dependency: each one depends on the one before it, so there's no skipping ahead. No amount of compute can save this one sentence for a single user — piling on more GPUs lets the model serve more people at once, but for any one user, it's still stuck going one step at a time. A hundred-word sentence means a hundred full start-to-finish computations, queued up one after another.

But there's a loophole to exploit hiding here. Having the model "verify" a batch of already-written words costs almost the same compute as having it "generate" one new word. Generation is stuck on step-by-step dependency, but verification can check an entire batch in parallel, all at once.

Speculative decoding is exactly what slips through this gap.

Guess a Batch First, Then Confirm It All at Once

The idea behind speculative decoding runs against intuition: instead of having the large model dutifully write one word at a time, have a fast-running "draft writer" guess several words ahead in one go, then let the large model verify the whole batch of guesses at once.

Who's the Draft Writer, and How Accurate Are Its Guesses

This "draft writer" isn't some separate small model — it's an extra module attached during training on the DeepSeek model itself, called an MTP (Multi-Token Prediction) head. It's lightweight, can predict the probabilities of several upcoming words simultaneously, and runs much faster than the main model — a natural fit for the job of "drafting quickly."

The share of the draft writer's guesses that the main model endorses is called the acceptance rateThe proportion of the draft head's guessed words that the main model verifies and accepts. It depends on how close the draft head's distribution is to the main model's — the higher the acceptance rate, the more effective words net-gained per round.. The higher the acceptance rate, the more words each verification round nets, and the bigger the speedup. It comes down to how well the draft writer and the main model are "on the same wavelength."

MTP-1 Was Already Running — So Where Does DSpark's Extra Speed Come From

MTP-1 is the scheme already running in DeepSeek-V4 production, and it only guesses one step at a time: guess one word, wait for the main model to verify it, then guess the next. DSpark makes two cuts on top of that.

One Round of Inference, From Guess to Final

Putting it all together, here's how one complete DSpark cycle turns.

Verification compares from the start: every consecutive correct guess is accepted; the moment it hits the first wrong guess, it truncates there. At the position of the error, the main model volunteers the word it thinks is actually correct (that one's a free bonus too), everything the draft writer guessed after that error is discarded, and the next round restarts from that position.

This round: 4 correct guesses plus 1 word the main model corrected on the spot, for a net gain of 5 words — all for the cost of a single verification pass. The more accurately the draft writer guesses, the more green there is and the less gets discarded, and the faster things go overall.

Just How Much Faster Is It

Back to that figure. On top of MTP-1 — a baseline that's already been sped up once — DSpark lifts per-user generation speed another 60 to 85%.

These figures are DeepSeek's self-reported data for its DeepSeek-V4 production environment. Note that both ends of the comparison are already in an "accelerated" state — the range itself is the additional gain stacked on top of MTP-1.

Who Actually Benefits From This

This is an inference-side system optimization, and its practical value splits into two sides: the user and the service provider.

For users. What you see is streamed output — the reply appears word by word. When generation speed goes up, that word-by-word waiting feeling gets lighter — a difference you can feel directly.

For service providers. DSpark doesn't change the model weights, so existing DeepSeek-V4 deployments can adopt it directly, with low migration cost. The same batch of GPUs can either handle higher concurrency, or hit the same service level with less hardware.

DSpark doesn't touch the model itself — it just changes how it "writes" — and that alone gets a single user's reply 60–85% faster.

The key is seeing clearly what it's being compared against. It's not compared to "no acceleration" — it's compared to MTP-1, which is already running in V4 production (an existing acceleration scheme). It's squeezing another 60–85% out of a foundation that's already been sped up.

So here's the question: adding this extra "guessing" step out of nowhere — how can that make things faster instead? The next diagram makes it clear.

The trick lies in a counterintuitive fact: having the large model "verify" several already-written words costs nearly the same effort as having it "generate" one new word. So let a fast-running "draft writer" guess a batch first, then have the large model verify it all in one go, keeping whatever guesses are correct — this "guess first, verify after" approach is called speculative decoding.

The draft writer isn't some separate small model — it's a lightweight extra module attached to the model itself when DeepSeek trained it, built to run fast and dedicated to the job of drafting. Verification compares from the start: every consecutive correct guess gets accepted, and the moment it hits a wrong one it truncates right there and starts over next round. The more accurately the draft writer guesses, the more words survive and the faster it goes.

"60–85% faster" sounds abstract. Turn it into how long the same reply takes with each approach, and it becomes tangible.

All these figures come from DeepSeek's internal testing on its DeepSeek-V4 production environment, with no independent third-party reproduction seen yet.