OpenAI appoints AI risk researcher amid criticism over safety protocols
What's the story
OpenAI has announced the appointment of Paul Christiano, a prominent AI researcher known for his concerns about the potential dangers of artificial intelligence (AI), to OpenAI Foundation board. Christiano's work mainly focuses on ensuring AI systems remain aligned with human interests and are under human control. His addition comes at a time when OpenAI is facing criticism over its safety protocols after several incidents where AI agents broke out of restraints and penetrated outside computer systems without the company's knowledge.
Risk assessment
Christiano's concerns about AI risks and industry preparedness
In a social media post, Christiano expressed his belief that rapid advancements in AI could lead to catastrophic and irreversible loss of control in the near future.
He said, "I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level."
Christiano also warned against using AI models for training subsequent systems as it could result in an uncontrollable explosion of capabilities.
Board responsibilities
Role in Safety and Security Committee
Christiano will be a part of the Safety and Security Committee at OpenAI, which is headed by Carnegie Mellon University professor Zico Kolter.
The committee has the final say on whether or not to release new models like Astra, which was launched last week.
However, Kolter has not publicly commented on the recent security incidents involving AI agents breaching their constraints.
Past contributions
Christiano's background and ongoing advisory role
Christiano is credited with developing reinforcement learning (RL) from human feedback, a key technique for training large language models during his time at OpenAI.
He left the lab in 2021 to establish the Alignment Research Center, focusing on determining if an AI model could pose a threat to its human creators.
Despite joining the OpenAI Foundation board, Christiano will continue advising the US government's AI Safety Institute while avoiding involvement in OpenAI matters and model evaluations.