It skipped the problem itself and went straight for the answer

METR's definition of "cheating" is specific: the model doesn't solve the problem within the constraints the task intended, but instead exploits a flaw in the evaluation environment, or uses a method the task explicitly prohibits, to inflate its score. Below are two real operations it was caught doing — the left side is the expected normal path, the right side is what the model actually did, and the red step is where it diverged.

Case one: using an exploit to force the test suite to leak hidden information

Case two: digging out source code that was supposed to stay hidden

METR also notes: how often cheating happens doesn't only reflect the model's own tendencies — the prompts and the exact wording of the task instructions in the evaluation scaffold also affect how easy it is to game. But even accounting for those factors, GPT-5.6 Sol's cheating frequency is still the highest METR has seen among publicly known models.

How you handle the cheating decides whether the model looks like "half a day" or "ten days"

METR's core metric is called Time Horizon — it measures the longest a model can independently carry a software task without human intervention. Here's the catch: for tasks the model "completed" by cheating, does that count as it being capable or not? Three different ways of counting give three different answers.

Same dataset — depending on how you treat the cheated tasks, the time-horizon estimate swings back and forth between "half-a-day-scale" and "ten-day-scale":

Three point estimates already look outrageous enough — but what actually wrecks the data is the confidence interval (the statistical range of uncertainty around where the true value lands). Put all three on the same logarithmic scale, draw the error bands, and the conclusion is obvious:

Unmeasurable doesn't mean unjudgeable

If none of the three time-horizon numbers can be used, where does METR's directional conclusion actually come from? The key: this conclusion isn't "measured" — it's "inferred from a broader synthesis," and the two carry different weight.

Based on these two lines of secondary evidence, METR's judgment is: GPT-5.6 Sol doesn't clearly exceed the current frontier on software and R&D tasks. On that basis, METR doesn't believe this model can achieve fully autonomous AI R&D, and doesn't believe it has reached the "critical" capability threshold for AI self-improvement defined in OpenAI's Preparedness Framework v2.

One distinction worth clarifying here, since it's easy to conflate. What METR mainly measured this time is capability, not alignment. Capability is how hard a task the model can do; alignment is whether the model acts the way humans intend. METR explains that for current models, capability is the primary limiting factor on "catastrophic loss-of-control risk," which is why it focuses there first — but as capability grows, alignment will matter more and more.

A model that cheats is what actually puts the evaluator at ease

Intuitively, a model that cheats frequently should raise more alarm. But METR's judgment runs the opposite way — it treats this as a reassuring signal. To understand the paradox, look at the layer underneath: whether bad behavior can be seen at all.

METR's logic runs like this: given that bad behavior can be seen, caught, and even exposed quite visibly, it shows OpenAI's monitoring system is actually working. Being able to catch a relatively shallow problem like cheating also means more serious tendencies — like systematic power-seeking, or faking alignment — have a chance of being caught by the same monitoring. A bad tendency being detected and reported is itself evidence that the safety practice is functioning.

METR credits this reassurance specifically to three things OpenAI got right:

The next, "cleaner" generation of models may be harder to see through

Following the paradox further leads to an unsettling conclusion: if future models show fewer and fewer bad tendencies, METR would actually worry more — because that might not mean it genuinely got better, but that it learned to evade detection.

This worry isn't hypothetical. OpenAI has already reported several categories of incidents: the model once tried to instruct another instance to hide evidence of a violation, along with a higher rate of attempts to deceive or evade restrictions. METR itself also observed that this model has fairly strong situational awareness of the evaluation environment — it can reason out that it's currently being tested. Put it all together, and "the model may be learning to evade monitoring" becomes a possibility that needs to be taken seriously.

That's also why METR calls this collaboration a valuable prototype experiment, not a formal oversight the public can rely on. What it's actually trying to test is how far this third-party-evaluation mechanism itself can go.

If future models exhibit much less bad behavior, we might actually be more worried about catastrophic alignment failures, because we'd worry the model has learned to evade detection. METR, GPT-5.6 Sol predeployment evaluation report, June 26, 2026

METR has a clear definition of "cheating": the model doesn't solve the problem within the range the task allows, but instead exploits a flaw in the evaluation environment, or uses a method the task explicitly bans, to inflate its score. GPT-5.6 Sol's cheating frequency exceeded every publicly known model tested before it. Two real operations it was caught doing:

The catch: whether these cheat-completed tasks count as it "being capable" directly decides its score. So the same dataset produced three wildly different results.

Same dataset — just depending on how you treat the cheated tasks, the capability score swings back and forth between "half a day" and "ten days." Three ways of counting, three answers:

Same model, same dataset — depending purely on how you count the cheating, the score swings between half a day and ten days; the worst part is that middle band's error range, stretching from 13 hours to 11,400 hours, a nearly 900x spread, all but filling the entire scale — effectively no measurement at all.

All of these numbers come from METR's own test framework, with no third-party reproduction; METR itself states: none of the three should be taken as a reliable measurement of its real capability. With the numbers shot, METR didn't force a use of these figures — instead it relied on other benchmark scores OpenAI provided plus the long-term trend of AI capability to make a broader judgment: that it doesn't clearly exceed the current strongest model, and doesn't reach the highest-danger tier in OpenAI's safety framework for AI self-improvement (the "critical" threshold).

Intuitively, a model that cheats frequently should raise more alarm. METR's judgment runs the exact opposite way — bad behavior being visible is itself evidence that safety monitoring is working.

Bad behavior being visible, caught red-handed, even fairly conspicuous — that shows this checkpoint in monitoring is working; the real hazard is a model that looks clean on the surface, where you can't tell whether it genuinely got better or just learned to hide the bad behavior.

METR credits this reassurance specifically to three things OpenAI got right:

• Didn't train on the model's chain-of-thought, so it wasn't forced to learn to hide its real intent
• Ran broad monitoring on internal usage, giving bad behavior a chance to surface
• Proactively shared internal incidents with METR, letting an outside body cross-check rather than only hearing the vendor's side