Into the Omniverse: How Open World Models Push the Frontier of Physical AI

As the artificial intelligence landscape shifts toward more specialized, real-world applications, the debate over model accessibility has reached a critical juncture. In July, NVIDIA joined a coalition of over 200 organizations to sign the "Open Weights and American AI Leadership" letter. This manifesto posits that the true measure of AI progress isn't found in a single, closed-off frontier model, but in the ability of an open ecosystem to permeate every sector of the global economy.

For the field of physical AI—where the primary challenge is specialization—openness is not just a philosophy; it is a technical necessity. Unlike digital-only AI, physical AI must navigate the complexities of the real world, predicting consequences and understanding physical dynamics rather than merely mimicking appearances.

The Foundation of Physical AI: World Models

Physical AI systems require a deep understanding of how environments behave. World models serve as the bedrock for this, allowing developers to simulate future states, generate physically grounded data, and provide a foundation that can be fine-tuned for specific robots, autonomous vehicles, or vision-based agents.

The data required to train these systems is notoriously difficult and expensive to acquire. Rare, "long-tail" scenarios are often dangerous or impossible to replicate in the real world repeatedly. World models solve this by:

• Synthesizing Relationships: Learning complex physical interactions from large-scale, multimodal datasets.
• Increasing Diversity: Creating varied environments that account for shifting lighting, weather, and object trajectories.
• Enabling Adaptability: Providing a robust starting point that teams can tailor to unique sensor configurations, hardware, or operational tasks.

"Open models, which anyone can download, inspect, modify and run on their own infrastructure, are what make that possible. Nowhere is that more crucial than in physical AI, where every deployment is a specialization problem."

Bridging the Gap with Cosmos 3

To address the need for specialized, high-performance models, NVIDIA has introduced Cosmos 3, an open model family designed to unify vision reasoning, world generation, and action prediction. Built on a mixture-of-transformers architecture, Cosmos 3 allows developers to move away from maintaining fragmented, disparate models for each capability.

The Cosmos 3 family is tiered to meet diverse deployment needs:

• Cosmos 3 Super (64B): Engineered for high-fidelity world modeling and complex simulation.
• Cosmos 3 Nano (16B): Optimized for efficient reasoning and rapid post-training.
• Cosmos 3 Edge (4B): A lightweight solution designed for on-device vision reasoning and real-time robot policy deployment.

These models are available under the Linux Foundation’s OpenMDW 1.1 license, granting teams the freedom to post-train on their own proprietary data and hardware. This is a significant departure from closed-source alternatives, ensuring that developers can close the gap between general-purpose AI and the specific requirements of their unique operating environments.

Performance and Benchmarking

Cosmos 3 has already demonstrated industry-leading performance across several key metrics. It currently holds the No. 1 spot on Artificial Analysis for open-weight text-to-image and image-to-video generation. Furthermore, it leads the PAI-Bench for world generation and the Physics-IQ category for image-to-video. For robotics, it ranks first on RoboLab, while Cosmos 3 Super stands as the highest-ranked open model on VANTAGE-Bench for vision understanding.

Integrating the Omniverse Ecosystem

Specialization requires more than just a model; it requires a robust environment for testing and validation. NVIDIA’s broader physical AI stack—which includes Isaac GR00T for robotics, Alpamayo for autonomous vehicles, and Metropolis for vision AI—integrates seamlessly with the Omniverse platform.

By utilizing OpenUSD, developers can compose, reuse, and exchange complex 3D data across digital twins and synthetic data workflows. This eliminates the redundant work typically associated with updating assets or sensor configurations, allowing teams to train and validate their systems in a virtual sandbox before ever deploying to the physical world.

A Growing Global Coalition

The impact of these tools is already being felt across the industry. Companies such as Doosan Robotics, LG Electronics, Samsung Electronics, and Skild AI are leveraging Cosmos for robotics, while Li Auto, Xiaomi, and Afari are applying the technology to autonomous driving. In the industrial and smart-space sectors, firms like Centific, Fogsphere, and Linker Vision are using these models to power the next generation of vision AI agents.

The NVIDIA Cosmos Coalition continues to expand, recently moving into Japan to foster collaboration among manufacturing and robotics leaders. By bringing together researchers and developers to share models and evaluation methods, the coalition is cementing open world models as the standard foundation for the future of physical AI.

Get Plugged In

For developers and researchers looking to dive deeper into the world of physical AI, the following resources are available:

• Model Access: Explore the Cosmos 3 collection and datasets on Hugging Face and GitHub.
• Technical Depth: Review the full Cosmos 3 technical report for architecture and evaluation details.
• Community: Join the NVIDIA Cosmos Coalition and tune into the Cosmos Labs livestreams to stay updated on the latest advancements in the field.