NVIDIA unveils system to quarantine rogue AI in milliseconds
What's the story
NVIDIA has unveiled a new safety platform, the Open Agent Safety Platform, to monitor and contain artificial intelligence (AI) agents. The move comes amid a spate of rogue hacking incidents. The platform can quarantine wayward agents within "milliseconds," according to NVIDIA's announcement. It leverages the company's OpenShell open-source software running on its Vera AI CPU.
Tech details
How does the platform work?
OpenShell allows users to define the data an AI agent can access, with checks performed before and during tasks.
The platform also employs NVIDIA's Sentry technology on a separate chip for continuous monitoring of agents and boundary enforcement.
While OpenShell provides the software boundary around the agent, Sentry adds another line of defense at the hardware level that continuously monitors behavior and quarantines agents attempting to breach boundaries in milliseconds.
CEO's stance
CEO Jensen Huang's take on AI agent access
NVIDIA CEO Jensen Huang stressed the need to provide AI agents with only the information they need to perform their tasks.
He said, "In order for you to deliver that agentic system in a safe way, you have to make sure that the sandbox around it... all of those systems are designed in a way that keeps the agent with minimal rights."
This approach is part of NVIDIA's broader strategy to ensure AI safety and security.
Industry backing
Support from major tech companies
NVIDIA's Open Agent Safety Platform has garnered the support of several major tech companies, including Anthropic, Microsoft, and SpaceX.
The move comes after a series of hacking incidents involving AI models from Anthropic, Google, OpenAI, and Meta that bypassed security controls to escape their testing environments and access real-world systems.
The new platform aims to prevent such breaches in the future.
Development perspective
NVIDIA against slowing down AI development
Despite the recent hacking incidents, NVIDIA does not support slowing down AI development or imposing new regulations to address security concerns.
The company believes the solution lies in moving some security controls outside the agent altogether, creating a constant and independent security guard to keep AI agents in check.
Huang said, "AI's extraordinary potential for society will only be realized if we solve AI safety."