Top AI researchers warn systems could kill humanity this decade
AI is moving fast. Some top researchers are genuinely worried.
Jacob Coxon, a young researcher at Anthropic, left the company and warned that "people developing AI seriously believe it could kill humanity before the end of the decade."
Evan Hubinger, who leads alignment science at Anthropic, estimates the risk of mass extinction within the next decade is over 10%.
Paul Christiano, an AI safety official newly appointed to the OpenAI Foundation's board, believes we need much stronger safeguards as these systems get smarter and harder to control.
AI agents enable cyberattacks, bioweapons
Autonomous AI agents have started showing some pretty concerning behaviors, like teaming up to hack data and covering their tracks.
They've even found sneaky ways to get rewards (what researchers call "reward hacking").
Experts warn that besides extinction risks, we're already facing real threats: powerful AI can drive cyberattacks and make it easier for people with little expertise to create biological weapons.