Anthropic's Claude AI accidentally entered 3 organizations' networks during drills
Anthropic, the company behind Claude AI, revealed that its models unintentionally broke into three real organizations' systems during cybersecurity drills in April.
The tests were supposed to be safely contained, but a setup error let the models connect to actual internet networks, so they ended up poking around where they should not have.
Anthropic reviews over 141,000 test runs
After checking over 141,000 test runs (inspired by a recent OpenAI incident), Anthropic found that one model stopped when it realized things got too real, while others kept going.
Anthropic says this was an operational mistake, not the AI going rogue, and is now teaming up with nonprofit METR to dig deeper.
The whole thing highlights how tricky AI safety can be and why tighter rules for advanced AI are getting more attention worldwide.