Working code vs strong code: the redesigned Charts page
In GameBench 2, AI models write programs for eight strategy games, and the programs then play each other. In the 24 October release, 66 models have an Overall rank (based on three reasoning settings per model), and 32 of them turned every attempt we counted into a working program. Those 32 are far apart in how well their programs play. Their working-program strength (the 0 to 100 score of the programs that worked) runs from 41.6 for GPT-5.6 Luna to 89.8 for GPT-6 Astra. The redesigned Charts page on our benchmark site has a new chart, Working code vs strong code, that plots how often a model's code works against how well it plays.
Reading the chart
Each point is one model. Its horizontal position is the program success rate (the share of counted attempts that produced a working program). An attempt is one request for a program for one game at one reasoning setting, and the model may repair its code once after a compile error and once after a failed check. A program rejected only for being too slow counts neither as a success nor as a failure. Neither does an attempt lost to a fault outside the model, such as a provider, API or test-machine error.
The vertical position is working-program strength, with failed attempts left out. A strength of 60 means that the model's programs would be expected to take 60% of the points against a fixed set of reference programs, averaged over games and settings. The larger the marker, the more counted attempts stand behind the rate. Most models have 24, one per game at each setting.
The leaderboard itself ranks models by the headline score, which scores every failed attempt as 0 and averages it in with the working programs of its game. For the 32 models at 100% the two scores are identical, and their order on the leaderboard follows working-program strength alone. For each of the other 34 the headline score is lower.
What it shows in this release
The models at 100% have a median strength of 59.0, against 45.9 for the 34 that failed at least once. Grok 4.7 goes against that pattern. It produced working programs in 21 of its 24 counted attempts (87.5%). Those programs have a strength of 74.3, the ninth highest of all 66 models and higher than 24 of the 32 at 100%. Counting its 3 failed attempts as 0 brings its headline score down to 65.0 and its Overall rank to 13.
GLM-5.3 failed once in 24 counted attempts. Its strength of 69.0 becomes a headline score of 65.5, enough for an Overall rank of 12, one place above Grok 4.7. After these two, the strongest model below 100%, GPT-5.4, reaches 58.3, just under the 59.0 median of the models at 100%.
At the far left, Granite 4.2 8B turned one of its 24 counted attempts into a working program. That single program scores 38.8, only a little below the weakest model in the 100% column, but the 23 failures each count as 0 and leave the model a headline score of 1.6. It has no working program at all at its lowest and highest settings (Baseline and Intensive on the page), and in those two views the chart lists it below the plot instead of drawing it.
The rest of the Charts page
Every chart with a score now says which score it shows, and each chart has a "Show the numbers" table. Several let you switch between the headline score and working-program strength. Program reliability, beside the new chart, splits each model's attempts into those that worked first time, worked after a repair, were too slow, or failed. In Reasoning effect, a model's Baseline, Balanced and Intensive settings share one row, with a Gain column for Intensive minus Baseline.
Economics now draws every model with an Overall rank against a logarithmic cost axis. A best-value line joins the models for which no other model costs as little and scores as high. Costs are provider-reported or estimated from list prices, and models with incomplete cost evidence are drawn hollow and kept off the line. The line replaces the score-per-dollar view.
The per-game heatmap prints every value, on a colour scale centred on 50 that we checked for colour-blind readers. Head-to-head shows each pairing as the row model's share of the match points, with a draw counting as half a win. Game spread, under More charts, shows which games separate the selected models most.