On Tuesday afternoon, humanity played Pokemon Red games on Twitch, a live stream of humanity’s latest AI model, Claude 3.7 Sonnet. It has become an attractive kind of experiment that shows the capabilities of AI technology today and the people’s reactions to them.
AI researchers have tested new models using all sorts of video games, from Street Fighter to Pictory. However, humanity said that Pokemon has proven to be a useful benchmark for the Claude 3.7 sonnet.
Like Openai’s O3-Mini and Deepseek’s R1, Claude 3.7 Sonnet can overcome tough challenges, such as playing video games designed for kids. The model’s irrational predecessor, Claude 3.5 Sonnet, failed to start Pokémon Red, leaving the player’s home in Palet Town, but Claude 3.7 Sonnet won three gym leader badges.
However, the latest Claude is still in trouble. A few hours after the Twitch stream, the model was deterred by rock walls.
One Twitch user put together the situation like this: “Who will win, thousands of hours of computer AI to programming, or a single rock wall?”
Eventually, Claude realized it could navigate around the wall.
Meanwhile, Claude looks red at a Pokemon traverses Pokemon at slow-pock speed and infers every step with unbearable contemplation. But it’s oddly convincing. To the left of the stream you will see Claude’s “Thought Process” and to the right you will see real-time gameplay.
At one point, Claude attempts to find Professor Oak in his lab, but is confused as there were other NPCs in the scene.
“I noticed a new character underneath me: a character with dark hair and something that looks like a white coat in coordinates (2, 10),” Claude wrote. “This could be Professor Oak! Go down and let him talk.”
Claude then accidentally spoke to him with an NPC other than the professor. This is an NPC that the model has spoken about several times before. Several of the thousands of people on Twitch chat began to get uneasy. Others, especially those who had watched the stream for more than a few minutes, weren’t too worried.
“Everyone’s getting cold,” one person wrote in the chat. “Before you get out of the Oak Lab like ten times before you figure out how to go on.”
For longtime Twitch users, the format of human streams may feel nostalgic. Over a decade ago, millions of people tried to play Pokemon Red at once in the first online social experiment called Twitch Plays Pokémon. Each user can control the player’s character via Twitch chat, resulting in predictable, chaotic gameplay.
Some AI researchers have cited Twitch Plays Pokémon as an inspiration for their work. In October 2023, Seattle-based software engineer Peter Whidden released a YouTube video detailing how he trained reinforcement learning algorithms to play Pokémon. His AI spent over 50,000 hours before playing the game. One challenge was that the AI preferred to praise pixelated views rather than actually playing the game.
Twitch’s AI-driven “reenactment” plays Pokemon like Whidden and Anthropic, but at the same time it’s a bit bittersweet. The original stream was a very important moment in Twitch history as it brought people together in unexpected ways. Everyone was on the same team, trying to get the player characters to stop running in circles and actually move towards their goal of moving through the game.
In 2025, we don’t seem to be teammates anymore, but the audience will look at AI models and try to play a game that many of us hang out at when we were five years old. This is the epitome of the larger trend of AI motivation. Online experiences have shifted from shared collaboration to more lonely activities.
Source link
