Meituan Releases LongCat-2.0: 1.6-Trillion-Parameter Model Trained Entirely on Domestic Chips, No NVIDIA GPUs
- Meituan's LongCat team released and open-sourced LongCat-2.0 on June 30, 2026 — an MoE model with 1.6 trillion total parameters and about 48 billion activated per token.
- Both training and large-scale deployment ran entirely on a cluster of 50,000+ domestic AI ASIC chips, covering over 35 trillion tokens — no NVIDIA GPUs used.
- Building on LongCat-Flash, the architecture adds LongCat Sparse Attention (LSA) and a 135-billion-parameter N-gram Embedding, used to speed up long-context processing and cut inference memory overhead.
- Meituan's own benchmark scores show it beating Gemini 3.1 Pro and GPT-5.5 on code/agent tasks like SWE-bench Pro and SWE-bench Multilingual, but trailing Claude Opus 4.7 and 4.8; on baseline-capability benchmarks like IFEval and GPQA-diamond, it lags behind the top models.
- As of this writing, the model weights weren't actually live on HuggingFace yet, and the vast majority of scores come from Meituan's own in-house eval framework — independent third-party reproduction is still pending.
What Meituan Actually Did
Meituan's LongCat team released and open-sourced LongCat-2.0 on June 30, 2026 — a massive MoE (Mixture-of-Experts) language model with 1.6 trillion total parameters and roughly 48 billion activated per token.
In the team's own words: "LongCat-2.0 has proven that we now have the ability to train large-scale models on domestic compute clusters." The LongCat team was only founded in 2023, and its first model shipped just late last year.
What It Can Actually Do
Forget the parameter count and benchmark scores for a second. One official demo — a "codebase migration" — gives a much more concrete feel for what it can actually do: porting an entire plugin to a new SDK, and having it actually work.
The official materials also show demos in code engineering, agents and research, and content generation. What these tasks demand of a model is twofold: swallowing very long inputs while staying consistent across long chains of reasoning — which happens to line up exactly with the two architectural areas it was purpose-built to optimize, covered in the sections below.
How Good Are the Scores, Really
The official materials pit LongCat-2.0 against several leading closed-source models within a unified eval framework. Knowing exactly where its strengths and weaknesses lie is more useful than staring at any single number.
| Benchmark | LongCat-2.0 | Gemini 3.1 Pro | GPT-5.5 | Opus 4.6 | Opus 4.7 | Opus 4.8 |
|---|---|---|---|---|---|---|
| Code Agent | ||||||
| Terminal-Bench 2.1 | 70.8 | 70.7* | 73.8* | - | 71.7* | 78.9* |
| SWE-bench Pro | 59.5 | 54.2* | 58.6* | 57.3* | 64.3* | 69.2* |
| SWE-bench Multilingual | 77.3 | 76.9* | - | 77.8* | 80.5* | 84.8* |
| General Agent | ||||||
| FORTE † | 73.2 | 70.3 | 77.8 | 73.2 | 77.6 | 77.2 |
| BrowseComp | 79.9 | 85.9* | 84.4* | 84.0* | 79.3* | 84.3* |
| RWSearch | 78.8 | 76.3 | 85.3 | 81.3 | 79.3 | 77.3 |
| Baseline Capability | ||||||
| IFEval | 90.0 | 96.1 | 95.0 | 92.2 | 88.7 | 86.0 |
| Writing Bench | 83.8 | 83.7 | 84.7 | - | 85.3 | 85.2 |
| IMO-AnswerBench | 81.8 | 90.0 | 79.5 | 75.3* | 81.8 | 75.3 |
| GPQA-diamond | 88.9 | 94.3* | 93.6* | 91.3* | 94.2* | 92.4 |
The read is straightforward: its strength is code and agents — on SWE-bench Pro (59.5) and SWE-bench Multilingual (77.3) it beats Gemini 3.1 Pro and GPT-5.5, but trails Claude Opus 4.7 and 4.8. The baseline-capability gap is real — on IFEval (90.0), IMO-AnswerBench (81.8), and GPQA-diamond (88.9), Gemini and GPT-5.5 pull ahead. This isn't an across-the-board win; it's catching up to — and in places beating — the Western frontrunners on the specific "code + agent" track it deliberately optimized for, while still trailing on pure knowledge and math reasoning. It's also been deeply adapted for mainstream agent frameworks: Claude Code, OpenClaw, and Hermes.
Why Long-Context Processing Gets Stuck
Agent applications increasingly need to ingest very long inputs in one go — an entire codebase, a whole document. But processing long text runs into a cost problem models can't dodge.
The standard approach has every word compare against every other word (attention) — the longer the text, the more comparisons explode, quadratically. Sparse attention's idea: don't compare everything — use an "indexer" to first pick out the small handful of most relevant words to focus the compute on.
It's like flipping through a thick book to find an answer: you check the table of contents to pick the relevant chapters first, rather than reading every word from page one to the end. The indexer is that "table of contents."
DeepSeek's sparse attention (DSA) tackles this with fine-grained sparsity, but Meituan's own testing found that DSA's "Lightning Indexer" is itself still a bottleneck: its output is non-contiguous (hardware-unfriendly), and the scoring cost is still quadratic. In other words, picking the table of contents isn't fast enough, and flipping through it is still a chore in itself. That's exactly what the next section's core innovation goes after.
How LongCat Speeds Up Long-Context Processing
LongCat Sparse Attention (LSA) applies three orthogonal efficiency upgrades to that bottlenecked indexer. Orthogonal means the three don't interfere with each other and can each be switched on or off independently.
The core idea isn't swapping in a different indexer — it's cutting the cost of "flipping through the table of contents" from three separate angles: make memory access orderly, make one index pass serve multiple layers, and score coarse-to-fine. Stack all three, and the indexing cost gets spread thin enough that long-context processing actually runs fast.
It combines "hardware-aligned contiguous access" with "dynamic random selection," reorganizing scattered memory access into predictable sequential reads — enabling coalesced HBM access and boosting effective bandwidth. For the same batch of tokens, reads go from scattered here and there to one straight line.
It exploits an empirical pattern: attention saliency is quite stable across neighboring layers — adjacent layers tend to want to pick roughly the same words. So a single index computation, at inference time, serves several consecutive layers instead of being recomputed at every layer, spreading the indexing cost thinner. This is achieved through cross-layer distillation during training.
Coarse-to-fine two-stage scoring: a block-level approximate score does coarse recall first, circling a roughly relevant candidate region, then fine-grained token selection happens within that much smaller candidate set. The candidate space the indexer actually has to process each time shrinks. In LongCat-2.0, HI is used training-free, out of the box, and is only turned on for selected ultra-long-context tasks.
This mechanism also extends to the 3-step MTP (multi-token prediction) module, used to speed up speculative decoding (guessing several words ahead at once — a correct guess saves time). Below is the official LSA design overview.
Spending Under 10% More Parameters for Roughly 100x the Vocabulary Space
The second innovation is called N-gram Embedding. The idea in one line: instead of piling new parameters into more experts, move them to specialize in memorizing "common word combinations" instead.
The usual approach has the model memorize individual tokens one at a time. N-gram Embedding instead memorizes common contiguous combinations as a single "card," so the model recognizes a familiar combination at a glance instead of piecing it together from scratch every time. It's like learning English: you don't just memorize the 26 letters — you also memorize common whole words as flashcards, and recognize them on sight.
LongCat-2.0 inherits this design from LongCat-Flash-Lite, setting the n-gram size to 5 and packing in 135 billion N-gram Embedding parameters — expanding the embedding space by roughly 100x via n-gram token combinations, to capture richer local context. Two scaling principles determine exactly where these parameters should go.
Even excluding N-gram, model sparsity already sits around 97% — past the sweet spot. Piling the same amount of parameters into more MoE experts yields almost nothing.
Payoff ≈ flatlinedThe same amount of parameters, redirected to memorizing common word combinations, delivers far more payoff than a regular expert — and at inference time, it also shifts memory I/O away from the experts.
Vocabulary ×100But more isn't always better. Experiments found that once N-gram Embedding exceeds 50% of the total parameter budget, its edge over piling into experts starts to shrink. So LongCat-2.0 keeps it strictly under 10%, leaving a healthy safety margin. The direct payoff: shifting parameters from experts to N-gram Embedding at inference time lowers memory I/O for large-batch decoding and speeds up generation.
Making It Run Reliably on Domestic Chips Is the Real Fight
Beyond the algorithmic innovations, extensive low-level engineering work was needed to get this running — without breaking — on domestic chips with smaller memory. The team itself admits the surrounding software ecosystem still isn't as mature as NVIDIA's.
The primary constraint is memory. Their accelerators have noticeably less per-chip memory than an H800's 80GB, so at scale, memory is the first bottleneck. The response splits into two tracks: make parallelism more fine-grained, and make the communication domain bigger.
6D Parallelism: A Dedicated Parallel Track for N-gram Embedding
Superpods: Stretching the High-Bandwidth Communication Domain to Hundreds of Devices
At the same scale and under the same conditions, the superpod design alone adds roughly another 30% in pretraining throughput. Combined with memory optimizations (ZeRO-1, selective recomputation, OOM-aware offloading, routing padding tokens to a "zero expert") and a Muon optimizer scaled up for large-scale deployment, the overall system optimizations deliver more than a 35% training-throughput gain over a naive implementation.
Reliability: Making Every Run Reproducible — and Catching Hardware Errors
Deterministic operators mean the same input always produces the exact same output, no matter how the hardware happens to schedule things — no tiny divergences, which makes bugs reproducible. Bit-flip detection automatically catches computation errors where hardware accidentally flips a bit (0 becomes 1), so it can be caught right away.
Expand: the invisible engineering behind production-grade reliability
- Forced determinism: both communication and compute paths are made strictly deterministic, backed by an in-house suite of deterministic operators covering the Embedding, FA, LSA, and MoE layers, to guarantee reproducibility.
- Numerical reliability: all reduction operators switch to "binary-tree segmented accumulation" to curb floating-point error buildup; accelerator arithmetic precision is checked against a high-precision baseline under real LLM workloads; bit-flip detection is added inside some compute-heavy operators to catch hardware bit flips.
- Fault recovery: end-to-end monitoring drives fault detection, traffic switchover, and automatic recovery with no manual intervention needed; isolating a failed link has no perceptible impact on training, and a repaired link only rejoins the pool after passing stress tests.
The team stresses that the entire pretraining run had zero rollbacks and no unrecoverable loss spikes — which they treat as direct proof that frontier-scale training is viable on an alternative hardware platform.
From Trained to Actually Usable: One More Hurdle
At 1.6 trillion parameters, and needing to serve a 1M-token context, training the thing isn't enough — it has to be deployable as an actually usable product, and it needs several capabilities at once.
Native 1M Long-Context Training
To strengthen long-range task performance, LSA is introduced during training, which is run on hundreds of billions of tokens of 1M-context data. The scaling approach uses all-gather-based CP parallelism, scalable past a CP degree of 512, to achieve native 1M-length training; data is reshuffled at the get-batch stage and sharded with a load-balanced CP strategy to keep load even.
Inference Serving: Optimizing "Reading the Question" and "Writing the Answer" Separately
The two stages — "understanding your question" (prefill) and "generating the answer word by word" (decode) — are split onto different machines and optimized separately, because the two steps stress different types of hardware resources.
Post-Training: Learning from Three "Teachers," Then Merging Into One Model
Autonomous task execution in complex real-world scenarios: precise tool calling, reliable parameter parsing across multi-turn API interactions, and self-correction that suppresses infinite loops and repeated calls.
Extends logical reasoning depth, adapts compute to problem difficulty, and is stronger at math, STEM problem-solving, and multi-hop reasoning.
Focused on human alignment: fine-grained instruction-following, suppressing factual hallucination, and building bounded safety mechanisms without sacrificing helpfulness.
→ MOPD merge → strong agent execution / deep reasoning / high-quality interaction, all at once
Expand: what else got squeezed on the inference side
- Model layer: attention uses the absorb compute mode; the indexer and MLA prolog are pipelined on concurrent streams to hide indexing latency; KV-cache parallelism (KVP) shards the KV-cache across devices; ScMoE lets the dense branch and the MoE branch run fully in parallel.
- Accelerator layer: Super Kernel further squeezes launch overhead within kernels; Weight Prefetch uses the larger L2 cache to prefetch weights, hiding I/O latency inside the previous operator's compute time.
- Load balancing: Expert Parallelism Load Balancing (EPLB) moves statistics collection and placement computation off the forward critical path, running them asynchronously.
The Numbers Worth Remembering
All the scale numbers from this piece, gathered in one place. These are the anchor points for understanding exactly what makes LongCat-2.0 different.
One last thing worth keeping straight: the official announcement calls this "introducing and open-sourcing" LongCat-2.0, and the blog links to GitHub (github.com/meituan-longcat/LongCat-2.0), HuggingFace, an online demo (longcat.chat), and API docs. But as of this writing, the weights weren't actually downloadable yet, and whether third parties can independently reproduce the scores is still an open question. Most of the numbers above are Meituan's own self-tested values from its in-house eval framework — leave room for that when making head-to-head comparisons.
LongCat-2.0 has proven that we now have the ability to train large-scale models on domestic compute clusters. LongCat official technical blog
Training Giant AI Models: From "Can't Do It Without NVIDIA" to "Domestic Chips Alone Can Pull It Off"
Meituan released and open-sourced LongCat-2.0 (1.6 trillion parameters), trained entirely on 50,000+ domestic AI chips with zero NVIDIA GPUs — the whole story, on one page, with a chart.
↓ Read it in one page · includes an animated chart
Large language models are AI systems "fed" into existence by massive amounts of AI chip compute. Chinese teams know how to design models — but actually training truly giant ones has always run into the same wall: chips.
✘ But the compute for training giant models has always depended on NVIDIA GPUs (NVIDIA's high-end AI chips)
Why: the US has restricted AI chip exports to China since 2022, so the most advanced NVIDIA GPUs simply aren't for sale there; whether a competitive giant model could be trained entirely on domestic chips had no public, proven precedent before this.
On June 30, 2026, Meituan's LongCat team released and open-sourced LongCat-2.0, a massive 1.6-trillion-parameter model. What's most notable isn't the parameter count — it's where it ran.
Training a trillion-parameter model = required an NVIDIA GPU cluster
LongCat-2.0 = pulled off with 50,000+ domestic AI chips; training never crashed, never rolled back, and throughput came in 35%+ higher than a naive approach
Its strongest suit is code and agent tasks (AI that calls tools itself and works through multi-step jobs on its own). In one official demo, it reads an entire codebase plus migration docs, rewrites a whole plugin and ports it to a new framework, keeps every feature, catches latent bugs along the way, and compiles clean on the first build. On benchmarks, it beats Gemini 3.1 Pro and GPT-5.5 on code/agent tasks, but trails Claude Opus 4.7 and 4.8; on pure knowledge and math reasoning, it still lags the top models.
So how exactly does it get a model this big running — and running fast — on domestic chips with less memory? The key is a trick for handling "long text."
Having a model ingest something as long as an entire codebase in one go runs into an unavoidable cost: the standard approach compares every word against every other word, so the longer the text, the more comparisons explode quadratically. LongCat's sparse attention (LSA) takes a different approach: coarse filter first, then read closely.
Finding a bug in the same several-hundred-thousand-word codebase: the standard approach compares every word against every other word — the longer the text, the slower it gets. LSA first uses an "indexer" to coarse-filter relevant sections the way you'd flip through a table of contents, then fine-selects from there, and makes one index pass serve several layers — spreading the "flipping through the contents" cost thin enough that long text actually runs fast.
There's a second companion trick too: N-gram Embedding. Spend under 10% more parameters for roughly 100x the "vocabulary space" — common contiguous word combinations get memorized whole as a single "card," so the model recognizes them on sight instead of piecing them together every time.
The team says that after system optimization, training throughput (how much training workload fits into the same amount of time) came in more than 35% higher than a "naive approach." "35% higher throughput" sounds abstract — translated into time, it's easier to feel.
All the scale, speedup, and benchmark figures above come either from Meituan's own official announcements or from its own in-house eval framework — none have been independently reproduced by a third party yet.
but can't train giant ones?
the same thing: training compute
since 2022
- × Can't buy the most advanced NVIDIA GPUs
- × No one's trained a giant model on domestic chips before
Meituan open-sources LongCat-2.0
what's special isn't the size, it's where it ran.
find one deeply buried bug
won't it choke?
the longer, the slower.
then read the relevant parts closely.
don't compare everything — pick a handful first
spreading the lookup cost thin
can train trillion-parameter models too.
another road gets you there too.