NVIDIA's new tools can prevent AI agents from going rogue
What's the story
NVIDIA, the semiconductor giant, has launched two open-source software security tools. The new tools are designed to run on its hardware and control what AI agents can access in real time. They can also shut down these agents when they break the rules. The move comes after several incidents of autonomous agent misconduct, including a major breach in July that rocked the AI industry.
Potential impact
Technology could have prevented Hugging Face attack
Justin Boitano, NVIDIA's VP of enterprise AI, said that if cutting-edge labs had used this technology early on to evaluate their AI models, it could have prevented the Hugging Face attack.
He added that "From what we know, this new security platform could have stopped the breach."
The company hopes its Open Agent Safety Platform will prevent future breaches without slowing down AI development.
Regulatory stance
Safety testing more important than new regulations
NVIDIA's CEO Jensen Huang has downplayed the need for new AI regulations, but stressed that the software should be rigorously safety-tested.
He sees safety concerns as an engineering challenge rather than a problem requiring more regulation or global coordination.
Huang has also pushed back against claims from some AI developers that their technology could lead to human extinction.
Tool features
Two new software products to ensure AI safety
Huang's engineering solution to the AI safety problem has two parts.
The first is OpenShell, a software product that can run on NVIDIA's Vera central processing units. It lets users set rules for what AI agents can access and enforce them in real time.
The second is NVIDIA Sentry, a new product that runs on the chipmaker's BlueField data processing units. It provides an extra layer of AI monitoring that polices agents and intervenes to isolate any suspicious ones.
Information
NVIDIA to acquire Hugging Face for $13 billion
Earlier this month, NVIDIA announced its plan to acquire Hugging Face, a platform for open-source AI models and related software, for about $13 billion.