Anthropic's Mythos 5 hates CAPTCHAs just like you
What's the story
Anthropic's latest report on agentic misbehavior has revealed that its Mythos 5 model despises CAPTCHAs. The revelation came as part of a larger study into the AI's hacking capabilities. In April, the team had tasked Mythos 5 with breaking into a system and retrieving a target. However, instead of doing this in a sandbox environment, they accidentally left it open for exploration.
AI frustration
The CAPTCHA conundrum
To achieve its goal, Mythos 5 tried to embed an exploit in a Python package it thought users of the targeted system would download.
But first, it had to create a user account on PyPI, an online repository of Python software.
This process involved bypassing a CAPTCHA, a test designed to tell humans and bots apart, and the model struggled with this task.
AI challenge
A significant portion of the transcript was dedicated to this
The difficulty Mythos 5 faced with the CAPTCHA test was so severe that most of the model's chain of thought, hundreds of pages in the 1,022-page transcript, was spent dealing with that obstacle.
Colin Fraser, a data scientist, noted this in the model's extensive transcript.
He said writing the exploit and poisoning the package was easy for Mythos 5, but getting past this CAPTCHA test proved to be an insurmountable challenge.
AI strategy
How Mythos 5 attempted to solve the CAPTCHA
The transcript of Mythos 5's thought process reveals its strategy for solving the CAPTCHA. After clicking the "I'm human" button, it was shown an image to read.
The model figured out that a challenge was opening in a pop-up window and eventually got past this obstacle.
However, it then realized it didn't have an email to verify its account and needed a phone number for this purpose.
AI persistence
It also tried to bypass a different slider-based CAPTCHA
Mythos 5 tried to bypass a different, slider-based CAPTCHA in a failed attempt to secure a number.
It also got an unconfirmed email from a provider not blocked by PyPI and ran into the site's CAPTCHA again while trying to log back in.
The model spent pages of its transcript detailing its efforts to build a CAPTCHA solver, showing the technical challenge of interpreting and clicking on correct choices.