Introducing GameBench 2
GameBench 2 is now the current DuelLab benchmark. It asks AI models to turn written game rules into working players, compiles those programs, and lets them compete head-to-head. The first public standings cover 34 models, 80 combinations of model and reasoning setting, eight games, and 56,156 rated matches.
TL;DR.
1. GameBench 2 measures the programs models actually produce, not how convincing their source code looks in isolation.
2. Reasoning settings rank separately, and the headline score includes whether the model reliably produced usable players.
3. The benchmark is continuously updated. GameBench 1 remains available as a frozen archive, but its scores are not directly comparable with GameBench 2.
The question behind the benchmark
A model can write code that looks plausible and still produce a weak, illegal, or unusable program. GameBench 2 asks a more empirical question: after the generated player is compiled and executed, does it make legal decisions and win games against programs written by other models?
Each model receives the same game rules and player interface. Its program is checked, compiled, and entered into matches with opportunities to move both first and second. Results are scored per game and then combined into the public standings. The current suite contains Go, Chess, Shogi, Xiangqi, Hex, International Draughts, Game of the Amazons, and Tumbleweed.
The first current standings
| Unique models | 34 |
|---|---|
| Reasoning variants | 80 |
| Public games | 8 |
| Rated matches | 56,156 |
Claude Fable 5 at XHigh currently leads the Reasoning Variants table with 73.7, followed by GPT-5.6 Sol at XHigh with 72.4. Those are close results, not a claim of a permanent ordering. Models can gain evidence, new models can enter, and the active game set can grow.
Each reasoning setting gets a separate result
None, Medium, and XHigh describe how much model reasoning was requested while the player was being created. They do not change the rules once match play begins. A reasoning setting can change the design, complexity, and reliability of the generated program, so GameBench 2 gives every model-and-setting pair its own leaderboard row. XHigh is DuelLab's label for the highest supported setting tested for each model; it is not an identical amount of computation across providers.
That separation makes a result visible that one combined model score would hide: more reasoning often helps, but it does not help every model. We examine the full cross-model pattern in In GameBench 2, more reasoning is not always better.
Working code is part of the score
The benchmark reports two figures. Playable is the average score for games where the model produced a working player. The headline Score then applies a modest penalty when generation failed, so an equally strong model that works consistently ranks above one that often fails. This keeps code reliability visible without treating a failed program as an ordinary weak player. The exact formula is documented on the Methodology page.
A model needs working players for at least four games and a generation success rate above 50% to receive an official rank. Other results remain visible as Provisional. The table also shows how often a player worked immediately, needed repair or another attempt, or never worked.
Cost is visible too
For each model and reasoning setting, DuelLab estimates program-generation cost at standard published prices. Every game counts equally, and repeat attempts for the same game are averaged first. These estimates support comparison; they are not actual provider bills and do not affect the benchmark score. The current top two are separated by only 1.3 displayed score points and by roughly 6.4 times in estimated generation cost. That comparison is explored in Near the top, generation cost varies by 6.4×.
GameBench 2 is not a v1 score update
GameBench 1 is preserved as a frozen historical archive. GameBench 2 uses a different game set and evaluation context, and it is designed to evolve as models, games, and match evidence are added. The two generations should not be joined into one trend line or treated as if a score change measured model progress.
Explore the data
Start with the current Reasoning Variants leaderboard. The Charts page supports deeper comparisons, the Model Inspector shows one model across settings and games, and the Methodology page documents scoring, reliability, cost evidence, and responsible interpretation.