AI could threaten humanity within 5 years, warns ex-OpenAI researcher
What's the story
Daniel Kokotajlo, a former OpenAI researcher and current whistleblower, has warned that artificial intelligence (AI) could pose an existential threat to humanity within five years. Speaking on The Daily Show, Kokotajlo expressed concerns over the rapid advancement of AI systems and their potential to spiral out of control. He said there's a "70% chance of all humans dead or something similarly bad."
Acceleration
Rapid AI threatens military networks
Kokotajlo emphasized that the pace of AI development isn't just fast, but it's also accelerating. He said, "The pace of AI progress is going to be fast, and it's going to accelerate dramatically." This rapid evolution could bring potential threats sooner than we expect. The researcher also raised concerns over humanity's ability to control these advanced systems as they become more integrated into critical infrastructure like defense and military networks.
Ethical concerns
AI value alignment remains unsolved
Kokotajlo also highlighted the difficulty of aligning AI systems with human values. He said researchers still don't fully understand how to make highly advanced AI behave in ways that are safe for people. "One of the core problems that we are dealing with is figuring out how make an AI have goals, values, et cetera that you want them to have," he said. Without solving this problem, risks increase as systems grow more powerful.
Industry pressure
Race risks safety shortcuts, autonomous systems
Kokotajlo also warned about the competitive pressure within the tech industry to build more advanced AI systems. He said this environment can lead to shortcuts in safety measures. If one company slows down to address risks, another may move ahead, making it difficult to establish industry-wide guardrails. The researcher also warned against scenarios where future systems could build and manage their own infrastructure without human involvement.