Have Chinese AI Models Caught Up to the US Frontier? The Gap, Explained.
- By Epoch AI's ECI capability index, Chinese models haven't caught up: Kimi K3 trails by 4.37 to 5.29 months, with no crossover point on the trend line.
- But the "caught up" narrative has real evidence: Moonshot AI claims Kimi K3 beats Claude Opus 4.8 on 30 of its 35 self-reported benchmarks, tops the Frontend Code Arena, and matches Claude Fable 5 on GPU kernel optimization.
- Those results, though, hinge entirely on benchmark selection. Of the 35 self-reported tests, 24 are vision or agentic tasks. There are no long-context tests, no real math, no safety evals, and not a single one measuring the token efficiency of reasoning.
- The Artificial Analysis Index has the same bias: Agents at 34% plus Coding at 24% puts 58% of the weight on agentic and short coding tasks — and it completely ignores each test's difficulty.
- Epoch AI's ECI takes a different approach. Borrowing from exam theory, it estimates the difficulty and discriminative power of each benchmark, then infers model capability onto a shared scale. Anthropic uses the same method in its own system cards.
- The two rulers give opposite answers: the AAI shows China trailing by 1.22 to 1.61 months, overtaking in November 2027; the ECI shows a 4.37 to 5.29-month lag with no crossover on the trend line. Even the ranking of the seven labs' speeds is nearly inverted.
Not yet — but the leaderboards say otherwise
The answer is no.
So the real question this analysis tackles is: why are there two answers, and which one should you trust? To understand that, you first need to see just how solid the "caught up" case actually is. It's much stronger than the usual position-taking.
First, the scorecard
Kimi K3 is currently the largest open-weight model, with nearly twice DeepSeek V4's parameters and more than the original GPT-4 (1.8 trillion) that kicked off this AI boom. On the 35 benchmarks Moonshot AI published, it beats Claude Opus 4.8 on 30, GPT-5.6 Sol on 19, and Claude Fable 5 on 12.
Third-party benchmarks also give it a seat at the table. On the Artificial Analysis leaderboard, Kimi K3 scores 57.1, placing third behind Claude Fable 5 and GPT-5.6 Sol — a clear jump over previous Chinese flagships like GLM-5.2, Kimi K2.6, Qwen3.7-Max, and DeepSeek-V4-Pro.
On Frontend Code Arena, which specializes in web front-end quality, Kimi K3 took first place with 1,679 points — 48 points ahead of second-place Claude Fable 5, the largest gap on the entire leaderboard.
Then there's a field that rarely makes headlines but matters deeply to anyone working on AI: GPU kernel optimization. How fast a model runs depends on how thoroughly the hardware is squeezed. Every wasted clock cycle means less throughput and higher latency — and ultimately, more money. This matters in production, but it also matters in research. Every new architecture idea needs a batch of experiments to validate, and running PyTorch directly doesn't saturate the hardware. You need hand-written kernels for speed.
Put it all together, and "China has caught up" looks settled. So where does that opening "not yet" come from? The first layer of divergence hides inside the word "capability."
Split capability up, and only one category is truly level
"Caught up" can be both true and false at once because capability is domain-specific. A model can match its rival at writing code while lagging by generations elsewhere. Run through each domain, and the gaps snap into focus.
Coding is the most visible domain: it has the most public leaderboards and is the capability everyone uses daily. The other rows either require specialized evaluation labs or never made the self-report list. The public "caught up" narrative essentially grows from that first row.
But coding is precisely the domain that matters most
The most solidly measured trend in software engineering: the length of tasks an AI can complete independently roughly doubles every 4 months. The author notes this trend likely holds in math and other fields — software engineering is just where measurement is best. The term for this "task length" is time horizon.
If a task takes a human a certain amount of time and a model has a 50% chance of completing it, that duration is the model's time horizon. As a difficulty yardstick, it's far more reliable than counting lines of code, and it translates directly into economic value: replacing a day of someone's work and replacing three months of it are completely different things.
Project this trend a couple of years forward, and you get "models taking on tasks that take a person months to years." Even if this only holds for coding, whoever holds a superhuman-level programmer model first holds a strategic position that's hard to catch up to. That's why those few months of gap are worth nailing down.
The comparison is only between China and the US because the best models, most AI research, and most compute are concentrated there. Europe has the money and people in theory, but has shown no serious intention to compete.
So if the domain-by-domain gaps are this obvious, why do leaderboards say "caught up overall"? Digging two layers down reveals the answer. The first layer is the word "gap" itself — it describes two different things, and most people only use one.
The word "gap" describes two different things at once
The definition of a gap is a single sentence: how long it takes one lab to build a capability another lab already has. But inside that sentence are two questions pointing in entirely different directions.
- Asks: how far in the past did the leader reach today's level of the laggard?
- Computable from history alone — the data has already happened.
- This is what people usually mean by "a few months behind."
- Asks: how long until the laggard reaches the leader's current level?
- Requires predicting the future, based on assumptions about compute, talent, and AI-accelerated AI research.
- Far more uncertain.
There's another layer that's easy to skip: capability is domain-specific. A model can match another on coding while still being worse in other domains, or overall. So "caught up" demands the follow-up question: caught up on which one?
Models ship in batches — there's no continuous curve
When you actually try to compute this, you run into a practical problem: progress is discrete. Models launch every few weeks to months, and scores are never exactly equal, so you can't directly measure the time between them. Three methods each have trade-offs.
The author chose the third option. It's symmetric, better reflects the underlying trend, and shows whether the gap is widening or narrowing. The cost is also stated clearly: the result depends on the regression model chosen, and it smooths over any trend breaks. This works against Kimi K3 — if it truly represents a breakpoint, the regression line will pull it back down.
Over the past five years or so, only one or two moments qualify as trend breaks: the reasoning models of Q3 2024, and possibly the surge of agentic models following Claude Opus 4.5's release in November 2025. Neither was tied to model size, so a model less than twice the size of DeepSeek-V4-Pro interrupting the trend on size alone doesn't add up.
That's why every conclusion below gives two sets of numbers: the adjacent-model range (the current instantaneous gap, with an interpolated value in the figure), and the trend-line estimate (the long-run level). With the method settled, we hit the second and most critical layer of disagreement: which questions you use to measure.
The choice of benchmark questions directly determines the conclusion
"Won 30 out of 35" carries weight only if you know how those 35 were picked. Break down the composition, and the absent categories say more than the ones included.
- 12 vision
- 12 agentic
- 8 coding
- 3 "reasoning" and knowledge
- Long context: zero
- Safety-related (cyber, bio, chem): zero
- Serious math: only MathVision, GPQA-D, and HLE
- Pure reasoning: the 3 "reasoning" entries are really knowledge questions with a bit of math
- Token efficiency of reasoning: zero
Vision and agentic together make up 24 of the 35 — nearly 70% of the list. A few coding benchmarks, though, are ones the author considers high-signal, like ProgramBench and SWE-Marathon, which have broader task scopes and longer time horizons.
Only one claim holds up: Kimi K3's coding ability is comparable to current US frontier models.
You can't conclude "Kimi K3 is overall stronger than Claude Opus 4.8 and GPT-5.6 Sol." And against Claude Fable 5, definitely not — it lost most of the 35 benchmarks to it.
Third-party leaderboards have the same bias
So what about the third-party Artificial Analysis Index? It's one of the most popular indices today, but the author argues it shouldn't be used to judge overall model strength — for two reasons.
The first is weighting. The index composition is clearly tilted toward agentic and short-time-horizon coding tasks.
The second is difficulty. How hard each eval is, and how much information a score provides — the index ignores both entirely. All evals are averaged with hand-assigned weights. A trivial eval that anyone can pass and a near-impossible one that almost no model solves carry the same weight in the index.
That doesn't mean it's useless. Its rankings match real-world experience because it measures economically useful work. The author's solution is to reposition it: treat it as a measure of daily usefulness, not of general intelligence. That's why in his article, it's called the "Artificial Analysis Index" — with the word "Intelligence" from the official name deliberately dropped. So is there a ruler that avoids hand-picked weights and factors in question difficulty? Yes — and Anthropic uses it too.
How ECI factors question difficulty into its scores
Epoch AI's Capability Index (ECI) targets exactly those two flaws. It aims to measure the underlying general capability of a model and place all models on a single shared scale. The method is called Item Response Theory.
Two people both score 60 — one on a grade-school test, one on an Olympiad-level test. The implied ability is worlds apart. To compare them, you first need to know how hard each test is. That's what Item Response Theory does: the probability of answering a question correctly depends on both the student's ability and the question's difficulty, so you estimate both together.
Applied to evaluations, each benchmark gets two parameters estimated:
- How much ability is needed to score 50 on this benchmark.
- How sensitive scores are to changes in ability — i.e., how well it distinguishes between models of similar strength.
After estimating these, a unified statistical model is fitted to find the single ability value that best explains all observed scores. That value is the model's ECI.
First, it reduces reliance on hand-picked weights. Second, it doesn't require every model to have run every benchmark — missing entries are fine. Third, it accounts for the difficulty and discriminative power of each benchmark.
Anthropic has recently adopted the same approach to measure model strength, calling it AECI in its own system cards. Their chart also conveniently maps the difficulty of different benchmarks.
Running one row through the model shows what this method does. GPQA Diamond's 50-score mark lands near 129, meaning a model with an ability score of about 129 should score roughly 50 on it. Claude 3 Opus has an AECI of about 126 — slightly below 129 — so the model predicts it should score just under 50. The actual result: 50.4, slightly above expectation.
Capability values on a shared scale, less biased than hand-weighted indices — that's ECI's value. With both rulers in hand, let's measure the same batch of models on each.
Two rulers, same models — the gap changes by a factor of three
Same models, same time period, different ruler. Both the size and direction of the China-US gap change.
First, the AAI ruler
Kimi K3 scores 57.1 on this index. Because the index is bounded between 0 and 100, the author fits a sigmoid curve to the frontier trends of both countries.
By this ruler, the Chinese frontier trails the US by a little over a month, and extrapolating the trend line gives a crossover on November 9, 2027.
Switch to ECI, and the conclusion inverts
Once Epoch released its preliminary scores for Kimi K3, you can take those plus Moonshot AI's published results — 9 benchmarks in total — and fit its ECI directly: 155.53, with a 90% interval of 153.87 to 158.21. This number is the author's own calculation, not an official Epoch ECI. It lands between GPT-5.3 Codex and GPT-5.4 Pro, closer to the former.
AAI: trails by 1.22–1.61 months, 2.89 months on the trend line, crossover on 2027-11-09.
ECI: trails by 4.37–5.29 months, 6.08 months on the trend line, crossover: none.
The backward gap differs by a factor of three. On the first ruler, China is accelerating faster. On the second, the US is.
At the lab level, the ranking is nearly inverted
Country-level frontiers hide the pace of individual labs, so both charts have per-lab versions. Putting the annual progress rates side by side shows just how much the methodology matters.
Moonshot drops from the top of the left chart to behind OpenAI on the right. Anthropic rises from fifth on the left to a clear first on the right. The only lab whose rank stays the same across both is Google: 21.4 and 6.3, last on both.
On the AAI per-lab trend, Moonshot has the steepest slope and crosses the US frontier line on February 8, 2027 — the original text says "surpasses Anthropic." On the ECI per-lab trend, Moonshot's crossover column also says "none." Even excluding Mythos Preview from Anthropic's data, its progress rate remains 18.1 points per year — the fastest.
The two per-lab trend charts, and how Kimi K3's ECI was estimated
Before Epoch's scores came out, the author made an initial estimate. Community polling put Kimi K3's ECI at 158–159; he guessed 156–157; analyst Teortaxes guessed 157–158.
To avoid pure intuition, he used the high correlation between AAI and ECI for conversion. A straight line doesn't work: AAI is bounded between 0 and 100, so scores bunch up near both ends, while ECI has no upper bound and is linear. The result: a straight line overestimates at the weak Claude 2 and Claude 3 end, underestimates in the middle, and overestimates again with strong models.
The solution is a logit transform of AAI — the inverse of the sigmoid — which re-spreads the compressed ends before fitting.
The conversion gives an estimate of 158.33, with a bootstrap 80% interval of 154.49 to 162.02. When Epoch's scores were released, directly fitting the 9 benchmarks yielded 155.53 — inside that interval, but lower than the converted value. The analysis uses 155.53 going forward, keeping the conversion formula for later.
The Mythos Preview side also needs estimation. Anthropic only provides its own AECI scale, which requires conversion: the Substack column Point Estimate calculates 161.5 using the ECI method; Ramez Naam gets 161 with least squares; the author gets 161, with a 90% interval of 158–166. This is close to Claude Fable 5's actual ECI of 160 (interval 158–165), on the reasoning that Fable 5 is likely a smaller model distilled from Mythos Preview with additional safety training — both of which would push its score down slightly.
Forward gap: how long to catch today's strongest model
Reword the question as the forward gap, and here's the answer: using country-level trend lines, Chinese models won't reach Mythos Preview's score of 161 until March 7, 2027 — roughly 11 months. The 90% confidence interval spans December 28, 2026 to June 18, 2027, which translates to a lag of 8.72 to 14.38 months.
On the hardest questions, the gap is even clearer
One item in Epoch's preliminary results is especially telling. On FrontierMath Tier 4 — research-grade math problems — Kimi K3 scores about 39, still behind GPT-5.2 Pro (about 45) from seven months ago. On the same chart, Claude Fable 5 scores 88.
Five things this method still misses
The author's own assessment is that both gap numbers above are too small, because several factors with major impact never make it into the formula — usually because the numbers aren't public.
First, model size. Kimi K3 has 2.8 trillion parameters. Cursor's Michael Truell has said the Opus and GPT series are roughly the size of Cursor's newly released 1.5-trillion model — likely referring to Claude Opus 4.8 and GPT-5.4. GPT-5.5 and GPT-5.6 are estimated closer to 3 trillion. In other words, a 2.8-trillion model beating a 1.5-trillion-scale Claude Opus 4.8 on several tasks doesn't say much from an architecture standpoint.
Second, inference efficiency. None of those 35 benchmarks measure it, and in the author's judgment, Kimi K3's inference efficiency still trails US models. Spending more tokens to get the same score at a similar parameter count is a cost that appears on no leaderboard.
Third, real service cost and total compute. Whether you hold a million latest-generation Blackwells or a fleet of older, lower-throughput Hoppers determines how many instances you can run simultaneously and how fast you can serve users. In a scenario of all-out competition, this matters as much as model scores.
Fourth, safety testing time. US labs' safety testing is far more thorough than Chinese labs'. This work could add another month to the gap.
Fifth, benchmark representativeness. Most benchmarks don't reflect real-world tasks, and due to cost and time constraints, the scores don't push models to their absolute maximum performance either.
But an underestimated gap doesn't mean Chinese labs aren't genuinely accelerating. They are — unusually fast.
One more layer: US labs are holding back unreleased models
The Mythos rumor is a 10-trillion-parameter model. SpaceX AI, OpenAI, and Meta are all building their own 10-trillion-scale models. The author's view: Kimi K3's apparent catch-up is likely a Chinese model catching the previous generation, while US labs have already moved to the next — holding back their strongest models due to legal uncertainty around cyber and bio capabilities.
- Kimi K3 is 2.8T parameters, open weights
- Michael Truell's claim that Cursor's 1.5T model is close in size to Opus and GPT
- Epoch has published Kimi K3's FrontierMath and other benchmark scores (ECI 155.53 is the author's fit from these, not an official Epoch value)
- Anthropic uses AECI in its system cards
- Mythos is 10T parameters — rumor
- GPT-5.5 and GPT-5.6 near 3T — estimate
- Claude Fable 5.1 and GPT-6 trained but unreleased
- Safety testing could add another month — author's magnitude judgment
- All 2027 crossover dates are trend-line extrapolations
If Anthropic and OpenAI actually release Claude Fable 5.1 and GPT-6, there won't be any conversation about catching up — these 10-trillion-parameter models are an entirely different thing.Lisan al Gaib, scaling01
Why Chinese models suddenly got faster last August
Chinese labs' progress rate has genuinely increased, and the starting point is specific. The author uses the derivative of an exponential over time to locate the inflection point, while acknowledging it's not a particularly precise method.
This is itself unusual: US labs have more compute, data, talent, and stronger models to accelerate their own research — yet their progress is slower. The author first checked a technical explanation: could it just be that the index has a ceiling, and US scores are already high enough that climbing is harder, making them look slower? He applied a logit transform to remove that compression. Moonshot and Z AI's peak speeds were still higher.
Distillation is one factor, but not the whole story
Strict distillation requires access to the model itself, but using a stronger model to score a weaker one's outputs already counts. Anthropic has repeatedly reported tracing distillation attacks back to Chinese labs. So it's real — we just don't know the scale or how much it helps. The author's phrasing: a portion, large or small, of Chinese models' capability comes from distillation.
Beyond that, several explanations exist, and the author believes the reality is a combination:
- Catching up is easier than blazing the trail; directions for improvement are given by the frontier.
- AAI-measured abilities are publicly visible and amenable to RL-driven incremental gains — natural targets for optimization.
- Only the two fastest-growing Chinese labs were selected — the ones that took the risk and won. Selection bias.
- The number of models included is small; this could just be a lucky observation window.
One important caveat: this catch-up narrative holds on the AAI, but not on the ECI.
A counter-evidence: the harder and longer the task, the more the gap shows
The UK AI Safety Institute's cyber range is currently one of the hardest eval categories. The tasks are genuinely difficult, and models must burn 100M tokens to be pushed to their limits. Epoch's MirrorCode goes further, using up to 1B tokens per model.
In this 32-step enterprise cyberattack simulation, GLM-5.2 averages about 11 steps — the same level as Claude Opus 4.5. GPT-5.6 Sol reaches about 29 steps, and Anthropic's newer Claude Mythos 5 about 27. Yet on self-reported coding benchmarks, GLM-5.2 scores close to Claude Opus 4.8.
The same model — near the latest Claude Opus 4.8 on publicly visible, easily optimized coding boards, yet back at the level of Claude Opus 4.5 from seven or eight months ago on hard long-horizon tasks. That gap itself is a signal.
The article closes with an observation: if Chinese models were truly as strong as the US frontier, they should by now have solved math and physics problems that were previously unsolved. So far, none have.
Same models, different ruler → China-US AI gap goes from ~1 month to ~5
Independent analyst Lisan al Gaib turns "has China caught up?" into a computable question, and walks through why two rulers give opposite answers — with visuals.
↓ One page · one animated figure
On July 16, Moonshot AI released Kimi K3, a 2.8-trillion-parameter model, and "China has caught up" was everywhere. Its scorecard is genuinely strong: it beats Claude Opus 4.8 on 30 of its 35 self-reported benchmarks and tops Frontend Code Arena at 1,679 points. But how much that claim can support depends on how those 35 were chosen.
✘ What they can't prove: how far behind it is overall.
Because the choice of benchmarks determines the conclusion. Of the 35, 24 are vision and agentic tasks — nearly 70%. Long context: zero. Cyber and bio safety: zero. Serious math: almost none. And nothing measures how many tokens it costs to answer a question (token efficiency — i.e., how much compute is burned). What's absent says more than what's present.
Run the same models over the same period through two different evaluation systems, and you don't just get different numbers — you get opposite directions.
Agents 34% + Coding 24% = 58%
Eval difficulty → not counted
Result: trails by 1.22–1.61 months
Crossover: 2027-11-09
Estimates "difficulty + discrimination"
then infers model capability
Result: trails by 4.37–5.29 months
Crossover: none
Even who's accelerating flips: on the AAI, all four Chinese labs post faster annual progress than all three US labs, with Moonshot first at 39.4. Switch to ECI, and Anthropic leads decisively at 25.0, while Moonshot drops to 13.8 — behind OpenAI's 14.3.
The method is borrowed from exam theory: Item Response Theory. In one line: two people both score 60 — one on a grade-school test, one on an Olympiad test. The implied ability is worlds apart. To compare, you first need to know how hard each test is.
"Three times bigger" sounds abstract. In days, it clicks. Same question — how many days ago did the US frontier reach today's Chinese frontier level — the two rulers answer like this:
vs Claude Opus 4.8
- × Long context
- × Real math
- × Safety
- × Tokens per answer
needs ability 129
hardest tier
where "3× bigger" comes from