In GameBench 2, more reasoning is not always better
GameBench 2 is DuelLab's benchmark of game-playing programs written by AI models. Fifteen models currently have official results at all three code-generation settings—None, Medium, and XHigh. These labels describe how much reasoning was requested while each program was being written, not during the matches. Twelve models score best at XHigh, three at None, and none at Medium.
XHigh is DuelLab's label for the highest supported setting tested for each model; it is not an identical amount of computation across providers. Scores run from 0 to 100 and also reflect whether the model produced a working player. For an overview, read Introducing GameBench 2.
TL;DR.
1. XHigh is the best setting for 12 of 15 models with ranked results at all three settings.
2. Claude Opus 4.5, MiMo-V2.5, and Nemotron 3 Ultra 550B A55B are strongest at None.
3. The result says reasoning setting is part of the evaluated system. It does not prove that extra reasoning itself caused every gain or score drop.
The field tilts strongly toward XHigh
The broad direction is clear: the highest setting is often associated with a stronger player. The largest move belongs to GPT-5.6 Luna, which rises from 11.7 at None to 55.0 at XHigh. GPT-5.5 moves from 31.8 to 67.6, and GPT-5.6 Sol moves from 37.2 to 72.4.
| Model | None | Medium | XHigh | Best |
|---|---|---|---|---|
| GPT-5.6 Luna | 11.7 | 29.3 | 55.0 | XHigh |
| GPT-5.5 | 31.8 | 47.2 | 67.6 | XHigh |
| GPT-5.6 Sol | 37.2 | 52.3 | 72.4 | XHigh |
| Claude Opus 4.5 | 45.9 | 37.2 | 42.4 | None |
| MiMo-V2.5 | 36.5 | 24.5 | 22.1 | None |
| Nemotron 3 Ultra 550B A55B | 31.5 | 26.9 | 18.9 | None |
The exceptions are substantial
MiMo-V2.5 loses about 14.3 points between None and XHigh. Nemotron 3 Ultra loses 12.6. Claude Opus 4.5 is less dramatic, but None still beats XHigh by 3.5 points and Medium by 8.7. These are not tiny rank-order changes around a tie.
GLM-5.2 sits near the boundary: 36.3 at None, 36.6 at Medium, and 36.9 at XHigh. Its three settings are effectively clustered compared with the large separations above. That is a useful reminder to consider how far apart the scores are, not only which setting technically finished first.
Why can extra reasoning hurt?
GameBench 2 cannot identify the cause from leaderboard scores alone. Asking for more reasoning can change the chosen algorithm, the amount of search, code complexity, speed, or whether the program passes the benchmark's checks. Because the score reflects both playing strength and successful generation, a more ambitious solution can score lower either because it plays worse or because it fails to run.
The cautious conclusion is not “reasoning makes these models worse.” Instead, GameBench evaluates every model separately at each requested setting. For these models and this task, a higher setting sometimes produced a lower overall score.
Why Medium is not any model's best setting here
None of the 15 models scored highest at Medium when compared with that same model's None and XHigh results. That does not mean Medium entries rank poorly against other models: Claude Fable 5 Medium is third overall. For models that improve with more reasoning, Medium usually falls between None and XHigh; for the three exceptions, it lies on a decline toward XHigh.
What to inspect next
The next useful step is per-game analysis: do the score drops repeat across the suite, or are they driven by a few games? The GameBench 2 Charts page exposes the game-level comparisons, while the Model Inspector keeps one model's reasoning settings and information about whether its generated players worked together.