All news

NVIDIA opens four Cosmos 3 models for interactive robot and driving simulation

Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .

Researchers have an open foundation for simulators where a robot's or vehicle's action changes the next observation, rather than simply generating prerecorded video.

All four Cosmos 3 Nano checkpoints contain 16 billion trainable parameters and are distributed under the OpenMDW 1.1 license. Transfer-Auto generates synchronized data from up to 11 cameras and LiDAR with controls for weather, lighting and road scenes; Sim-Auto turns these into streaming observations for interactive driving scenarios. Sim-Transfer creates photorealistic video from a physics simulator's spatial controls, while Sim-Bimanual predicts the next frames from the actions of a two-armed robot or camera. NVIDIA is also publishing recipes for fine-tuning on users' own sensors and robots. The company specifically notes that the ability of such simulations to reliably train and evaluate policies still needs to be measured.

History

NVIDIA expands the Cosmos platform

2 events
  1. NVIDIA opens four Cosmos 3 models for interactive robot and driving simulationOpen now
  2. Nvidia expands Cosmos: five ready-made data pipelines for physical SI and tools for fine-tuning open models, including GR00T and Alpamayo