Opus 5.5 picked Squirtle because of Brock. Fable 5.1 picked Charmander because it was the closest ball. One cost $8, the other $100.
retained text
I run a benchmark that drops a model into Pokemon Red on a fresh save and scores it on milestones read out of the game's RAM. Every model gets the same prompt, one screenshot per turn, the game state and the words on screen. No pathfinding, no map hints. 1,000 turns to beat Brock. Opus 5.5 beat Brock on turn 271. $8.44, 1 hour 6 minutes, never blacked out. Fable 5.1 beat him on turn 554. $99.76, 3 hours 17 minutes, blacked out three times. The run was decided on turn 13. Opus walked past the first ball on purpose. Its reasoning on turn 14: "press A to pick Squirtle, since it's strong against Brock." Fable walked up to the first ball it reached and said "Charmander is a fine starter." Brock never came up. Charmander is the one starter that is bad against his Rock types. Fable was still on pace after that. It walked into Brock's gym on turn 230, two turns ahead of Opus. Then the gym's Jr. Trainer and his Sandshrew wiped the team. Its reasoning right before the whiteout on turn 246: "Likely losing, but nothing to do but attack with Tackle; if I whiteout I go to the Pokecenter and grind as planned." It lost to the same Sandshrew again on turn 362 and finally changed plans. Turn 363: "Brute-forcing isn't working and I'm down to $169." It said it was going to Route 22 to catch a Mankey. It never went. It ground in Viridian Forest instead, got Charmeleon on turn 511, and went back and beat Brock on 554. Every turn log is published, including the exact prompt the model saw and its reasoning: [https://pokebench.tv](https://pokebench.tv) Edit: to clear up the top comment, the prompt only says "Overall goal: beat the game." The models aren't told about the turn budget or that the run ends at Brock. That's on purpose. I want to see who plans ahead without being told to. If you want a different prompt tested, the harness is open source. Bring your own API key.
document fetched 2026-09-27 00:22 · canonical https://reddit.com/r/ClaudeAI/comments/1wqzubb/opus_55_picked_squirtle_because_of_brock_fable_51