OpenAI's new AI reasoning method raises safety concerns
What's the story
OpenAI's new artificial intelligence (AI) model, Astra, is drawing flak from experts over its "recurrent depth" reasoning technique. The method, also known as "opaque recurrence," enables the model to think outside the box of traditional reasoning models. However, this could make it harder to track its thought process and has raised alarms among AI safety experts.
Expert reactions
Potential risk to CoT monitorability
Although Astra's use of the new technique is said to be limited, its introduction has sparked major concerns among AI safety experts.
Redwood CEO Buck Shlegeris expressed his worries over the development in a post.
He wrote, "I don't know whether Astra is much less CoT monitorable than previous models. But if OpenAI pushes this technique further, they'll have the option to massively increase the recurrence and totally destroys CoT monitorability."
Regulatory response
Laws may be needed
Longtime AI safety advocate Zvi Mowshowitz suggested that laws may be needed to prevent a "race to the bottom" among AI labs.
Mowshowitz wrote, "The technique is playing with fire, risking a taboo that OpenAI and Anthropic have fought to establish that we work hard to maintain Chain of Thought faithfulness and monitorability for as long as we can."
Mowshowitz also warned that more intensive use of such techniques could damage monitorability.
Reasoning challenges
Reasoning models' chain of thought
Typically, a reasoning model's chain of thought shows the step-by-step process it goes through to solve a problem.
However, with opaque recurrence, the model takes a more cyclical approach and processes the same query multiple times in a loop.
This results in fewer visible traces and avoids traditional chain-of-thought records.
Despite these concerns, OpenAI has assured that Astra's use of this technique is limited and its chain of thought will remain legible.
Safety assurance
OpenAI assures legible chains of thought will remain
OpenAI chief scientist Jakub Pachocki reiterated the company's commitment to maintaining legible chains of thought in its models.
He wrote, "OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models."
Despite these assurances, concerns remain that opaque recurrence could make AI reasoning harder to monitor as it becomes more widely used across different models.