Are AI companies taking 'existential' safety threats seriously?
What's the story
A recent report by the Future of Life Institute, a US-based AI safety think tank, has raised concerns over the global artificial intelligence (AI) industry's ability to combat "existential" threats. The study examined nine leading AI companies and found that none of them are doing enough to tackle these potential dangers. These threats include the development of models that could achieve human-level intelligence or artificial general intelligence (AGI).
Ranking details
Anthropic tops safety ranking with 'C+' score
US AI lab Anthropic topped the semiannual safety ranking with a score of "C+." However, the report stressed that this is not enough to address the "existential" threats posed by AGI. The rankings were determined by seven researchers and governance experts who assessed each company's efforts in six categories: risk assessment, current harms, safety frameworks, existential safety, governance and accountability, and information sharing.
Response
Mistral's last place in ranking disappointing, says expert
Mistral, a French company that makes open models for AI development, was included in the report for the first time. The company said that the report's framework doesn't fit its model development approach. Max Tegmark, an MIT professor and Future of Life president, expressed disappointment at Mistral's last place in the ranking considering Europe's leadership role in AI safety.
Military engagement
Shift in military use policy by some AI companies
The report also flagged companies that had previously barred their tech from military use but have now "gradually reversed course." This includes Anthropic, which has been criticized for its "questionable military engagements." The US government reportedly used Anthropic's tech in military operations in Venezuela and Iran last year. However, the company was recently banned by the Pentagon over AI safety disagreements.
AI risks
Report highlights potential cyberattack risks
The report warned about the potential misuse of AI models for cyberattacks or harmful tasks. It also noted that while there are "constructive attempts," the overall efforts to address these risks are "entirely inadequate." This underscores the need for more robust safety measures in the development and deployment of artificial intelligence technologies.