Wow, agents are already playing games, not exactly but quite fair I would say. Anthropic is really a lab, like they are researching LLM behaviors through and through; they are technical scientists.
Claude plays Pokemon Red:
Send screenshot of the current state
Describe the game mechanics
Ask for the action
Iterate
It’s quite a fascinating experiment. Maybe we can try with different types of games with LLMs. They tried a Pokémon-like game, because that isa very user-paced game, not a very rapid pace, or live-like games. Very smooth transitions and turn based game.