Gemini 4 Carbon debuts in second place
In GameBench 2, AI models write programs for eight strategy games, and the programs then play each other. Our results of 14 November add two Google models, Gemini 4 Carbon and Gemini 4 Hydrogen, beside the Gemini 4 Argon we already rank. Carbon is second in the Overall ranking, which averages a model's three reasoning settings, with 90.1. Only Claude Fable 5.2 is higher, at 95.0. GPT-6 Astra is third with 88.8, and the 1.3 points between them are too close to call in this release. Hydrogen is 11th Overall, but its XHigh programs score 97.0, second only to Claude Fable 5.2's XHigh programs among 273 ranked model-and-setting rows. A program from Carbon at High costs about $0.80 to generate and one from Hydrogen at XHigh about $0.83, against $4.58 for one from Claude Fable 5.2 at XHigh.
Setting by setting
A program's 0–100 score is its expected share of the points against the reference field (a fixed set of programs that every program is scored against), and a row's score is the mean over the eight games. We score up to three reasoning settings for each model: Low, Medium and High for Carbon and Argon, and Low, Medium and XHigh for Hydrogen and Claude Fable 5.2. Each cell below gives the score, the rank among all 273 ranked rows (for the Overall line, among the 77 models with an Overall rank) and the cost per program.
Scroll sideways to see every column.
| Setting | Gemini 4 Carbon | Gemini 4 Hydrogen | Gemini 4 Argon | Claude Fable 5.2 |
|---|---|---|---|---|
| Low | 85.7 (19), $0.16 | 58.0 (89), $0.06 | 79.2 (33), $0.08 | 91.7 (9), $0.47 |
| Medium | 89.8 (13), $0.43 | 79.8 (29), $0.13 | 76.7 (38), $0.10 | 93.9 (7), $0.65 |
| High or XHigh | 94.9 (3), $0.80 | 97.0 (2), $0.83 | 89.0 (14), $0.44 | 99.2 (1), $4.58 |
| Overall | 90.1 (2), $0.47 | 78.3 (11), $0.34 | 81.6 (8), $0.21 | 95.0 (1), $1.90 |
Carbon's programs score higher than Argon's at all three settings, by 5.9 to 13.1 points, and in 20 of the 24 pairs of game and setting. At Medium, Carbon scores close to Argon at High (89.8 against 89.0) at nearly the same cost per program. Claude Fable 5.2 is above Carbon at every setting, by 4.1 to 6.0 points, and in 19 of the 24 pairs. We read these two gaps as orders, Carbon above Argon and Claude Fable 5.2 above Carbon, because they hold at every setting and in most games.
From its lowest setting to its highest, Carbon's score rises 9.2 points and Hydrogen's 39.0. Of the 73 models that have an Overall rank and all three settings ranked, only Claude Sonnet 5.5 spreads wider (51.7). Hydrogen's Overall average of 78.3 puts it level with Kimi K3.1 Pro, also at 78.3.
Hydrogen from Low to XHigh
Hydrogen's XHigh programs won 4,386 of their 4,580 matches. In every game they rank between second and eighth among all scored rows, with scores from 93.5 in Go to 99.1 in Tumbleweed. Claude Fable 5.2's XHigh row is 2.2 points higher and has the top score in all 8 games. The next four rows (Carbon at High and the XHigh rows of Claude Opus 5.5, GPT-6 Astra and Claude Fable 5.1) are 2.1 to 3.0 points lower. Game by game, Hydrogen's XHigh programs are ahead of Claude Fable 5.1's in all 8 games, and ahead of each of the other three in 4 or 5 of them.
At Low, Hydrogen's programs score 58.0 and rank 89th. The Low Chess program won 16 and drew 405 of its 562 matches, for a game score of 41.1, and the Hex program scored 38.8. Going from Low to XHigh raises the cost per program about 15 times, from $0.06 to $0.83, and the score by 39.0 points.
Carbon, game by game
At High, Carbon's programs are in the top four of 6 of the 8 games: second in Go and in Game of the Amazons, third in Chess, Shogi and International Draughts, fourth in Hex. They are ninth in Tumbleweed, and outside Xiangqi their game scores run from 94.8 to 99.1. In Xiangqi the High program scored 77.8, 38th of 267 scored rows, and won 334 of 578 matches. Xiangqi is Carbon's lowest game at every setting, with scores of 62.7 to 79.6. Argon's Xiangqi programs score 67.5 to 74.3, while Hydrogen's XHigh program scores 95.6 there.
Cost per program
A cost per program is the mean cost of generating one program, with repairs and failed attempts included. For these models it is an estimate from Google's list prices and the token usage we recorded, not a provider bill. For Carbon at Medium and Hydrogen at XHigh, one interrupted request each left no usage record, and its charge is missing from those rows' costs, from both models' Overall costs and from the totals below. Google lists Carbon, like Argon, at $2 per million input tokens and $10 per million output tokens. These introductory rates end on 1 December, when they double. Hydrogen's list prices are $1.50 and $7.50.
Among the top five models in the Overall ranking, Carbon has the lowest cost per program, $0.47. The other four cost $0.65 to $2.32, and sixth-ranked GPT-6.1 Sol (84.9) costs $0.13. At the same list prices as Argon, a Carbon program costs 2.2 times as much, because it uses more tokens. Near the top of the model-and-setting ranking, Carbon's High programs cost $0.80 and Hydrogen's XHigh programs $0.83. The XHigh rows of Claude Fable 5.2, Claude Opus 5.5, GPT-6 Astra and Claude Fable 5.1 cost $1.30 to $5.15, and Claude Fable 5.2 at Medium scores 93.9 for $0.65.
Both models have a working program for all 24 game-and-setting pairs. For Carbon, 20 came from the first request and 4 needed a repair request; for Hydrogen the split is 22 and 2. Generating all of them cost about $11.18 for Carbon and $8.10 for Hydrogen.
What we don't know yet
Each setting's score rests on eight programs, one per game. The release gives uncertainty intervals for individual programs but not yet for model-and-setting rows or Overall scores, and we cannot say how far a fresh set of programs would move them. Carbon is above GPT-6 Astra at each setting, but only by 0.6 to 2.2 points, and in 15 of the 24 pairs of game and setting (8 go the other way and 1 is tied). The gaps between Hydrogen's XHigh row and the four rows below it are also small, and only against Claude Fable 5.1 do the games all go one way. All six Xiangqi programs from Carbon and Argon, one per setting, score below 80, and we cannot yet tell whether that is a weakness the two models share or chance.
Also in this release
Besides Carbon and Hydrogen, 6 models are new in this release. The two with an Overall rank are Nex N2.6, at 19th, and Muse Ember, at 36th. The release covers 113 models, 2,365 programs and 699,004 rated matches. The leaderboard always shows the newest release. The model pages of Carbon and Hydrogen have their game-by-game results, and the methodology explains the scoring in full.