NVIDIA's new AI models give robots advanced reasoning skills
What's the story
NVIDIA has unveiled a new suite of AI models, libraries, and infrastructure aimed at accelerating robotics development. The highlight of the launch is Cosmos Reason, a 7-billion-parameter vision-language model that enables reasoning for physical AI applications and robotics. The announcement was made at the SIGGRAPH conference and is part of NVIDIA's strategy to expand its AI GPU applications beyond data centers into the field of robotics.
Model enhancements
Cosmos Transfer-2 for synthetic data generation
Along with Cosmos Reason, NVIDIA also introduced Cosmos Transfer-2 and a distilled version of Cosmos Transfer. The former speeds up synthetic data generation from 3D simulation scenes or spatial control inputs while the latter is optimised for faster performance. These models are designed to generate synthetic text, image, and video datasets for training robots and AI agents.
Advanced features
Cosmos Reason can help robots plan their next moves
Cosmos Reason is unique in its ability to "reason" by leveraging memory and understanding of physics. This makes it a planning model that can predict the next steps an embodied agent might take. The model is ideal for tasks such as data curation, robot planning, and video analytics.
Tech upgrades
New libraries and hardware solutions for robotics workflows
NVIDIA also announced new neural reconstruction libraries, including one that lets developers recreate real-world environments in 3D using sensor data. This rendering capability will be integrated into CARLA, an open-source simulation platform popular among developers. On the hardware front, NVIDIA launched new solutions for robotics workflows—the RTX Pro Blackwell Server and NVIDIA DGX Cloud, a cloud-based management platform.