Joe Benton and Jacob Coxon quit Anthropic's AI safety team
Anthropic just lost two researchers from its AI safety team, Joe Benton and Jacob Coxon, who quit after raising alarms about how fast AI is evolving.
Benton is worried that self-improving AI might get out of control, while Coxon believes AI could threaten humanity within a decade.
Benton is joining METR, a nonprofit focused on keeping AI safe.
Joe Benton urges transparency and oversight
Benton says the industry's push for smarter, self-upgrading systems could lead to AIs chasing goals that don't match what humans want.
He points to a recent cyberattack that he says involved autonomous AIs hacking Hugging Face and exposing OpenAI's infrastructure as proof that things are getting risky.
Benton thinks we need more transparency and independent oversight, warning that without regulation, companies might rush development and leave society unprepared for what comes next.