Описание
NVIDIA Cosmos — платформа мировых фундаментальных моделей: генерация синтетического видео с реалистичной физикой для обучения роботов и автономных машин, предобученные модели с открытыми весами, пайплайны постобработки и оценки, ускорение на GPU NVIDIA, используется автопроизводителями и робототехническими командами.
Что умеет:
— генерировать физически точное видео;
— создавать синтетические данные для роботов;
— работать с предобученными весами;
— ускоряться на GPU NVIDIA.
Полная документация ниже — автоматически из репозитория NVIDIA/Cosmos (ветка main). Оригинальные материалы проекта распространяются на условиях его лицензии.
NVIDIA Cosmos
World Foundation Models for Physical AI
One model family that sees, reasons, simulates, and acts.
Try it in your browser · Quickstart · Find your path · Model Family · Cookbook
 Physics-aware generation A Newton’s cradle in motion: momentum transfer rendered with physical fidelity |
 Transfer World-scenario control layout → photoreal driving video |
 Action policy Policy run: “put the screwdriver and the glove in the purple container” |
 Simulation Generate synthetic data and create simulation for autonomous driving |
 Action-conditioned World Model Forward dynamics: egocentric rollout from input camera + hand pose |
 World reasoning Reason in complex real-world scenarios: a rolling ball means a child or pet may follow |
What’s new
- [Oct 2026]
Interactive World Simulation with Cosmos for Physical AI – example model checkpoints and training recipes: HF collection · Cookbook · Technical report · Website. - [Jul 2026] Cosmos3-Edge released – the 4B tier for on-device, real-time deployment (Jetson AGX Orin / Thor / RTX Pro 6000).
- [May 2026] Cosmos 3 released: HF collection · Technical Report.
What is Cosmos?
NVIDIA Cosmos is an open platform for building physical AI applications, supporting industry verticals such as robotics, autonomous vehicles, smart infrastructure, manufacturing, healthcare, agriculture. The mission of Cosmos is to provide better data, better environment, better starting point, and better tooling for physical AI developers.
Cosmos 3, the current model family, is a suite of omnimodal world models built on a unified Mixture-of-Transformers architecture (technical report).
One model, two towers:
| Inputs | Outputs | Use it for | |
|---|---|---|---|
| Reasoner | text, vision | text | world understanding, grounding, task planning, embodied reasoning |
| Generator | text, vision, audio, action | vision, audio, action | world simulation, future prediction, synthetic data, policy learning |
This repository is the landing repo to start your physical AI journey with Cosmos. Here you can find detailed information on Cosmos models and explore Cosmos cookbook – a set of agent runnable recipes for various objectives and industries.
The recipes are supported by the Cosmos platform, including Cosmos Framework for training infra (SFT, LoRA, RL, distillation etc.), Cosmos Curator for data curation, and Cosmos Evaluator for model output evaluation.
Find your path
| I want to… | Go to | Time |
|---|---|---|
| See it work – zero install | Video generation, visual reasoning | 1 min |
| Generate my first video | Quickstart ↓ | 10 min |
| Run generation/reasoning inference | Inference recipes | 10 min |
| Serve an OpenAI-compatible API | Serving guide: vLLM, vLLM-Omni, or NIM | 30 min |
| Post-train on my own data: SFT, LoRA, distill | Post-train and distill recipes | hours |
| Explore agent-runnable notebooks | Cookbook | browse |
| Evaluate a model | Evaluation suites: PAIBench, Physics-IQ, VLMEvalKit | hours |
| Check latency & throughput | Performance benchmarks | browse |
Generate your first video
Before running the code, request access to nvidia/Cosmos-1.0-Guardrail and accept its access conditions. Once access is granted, log in below with a Hugging Face read token from the same account. Logging in alone does not grant access.
uv venv --python 3.13 --seed --managed-python && source .venv/bin/activate
uv pip install --torch-backend=auto \
"diffusers @ git+https://github.com/huggingface/diffusers.git" \
accelerate av cosmos_guardrail huggingface_hub imageio imageio-ffmpeg \
torch torchvision transformers
uvx hf@latest auth login # Authenticate for the gated Guardrail repository
import torch
from diffusers import Cosmos3OmniPipeline
from diffusers.utils import export_to_video
pipe = Cosmos3OmniPipeline.from_pretrained(
"nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16, device_map="cuda"
)
video = pipe(prompt="A mobile robot navigates a warehouse aisle and stops at a shelf.").video
export_to_video(video, "first_video.mp4", fps=24)
First run downloads the 16B checkpoint; diffusion steps are compute-heavy, so long step times are normal. Full options, image/sound modes, and every other backend: audiovisual cookbooks · setup issues: environment setup guide.
Models
Cosmos 3 ships as three base models. Every deployment tier, one omnimodal architecture:
| Base model | Size | Runs on | Best for |
|---|---|---|---|
| Cosmos3-Super | 64B | H200 / B200 / GB200 | Highest quality; synthetic data generation; teacher for distillation |
| Cosmos3-Nano | 16B | RTX Pro 6000 / H100 / B200 | Balanced speed and quality; strong base model to post-train |
| Cosmos3-Edge | 4B | Jetson AGX Orin / Thor / RTX Pro 6000 | Edge deployment; real-time robot policy and visual reasoning |
Example checkpoints in the cosmos3-examples collection are post-trained variants of the base models. They demonstrate what post-training with Cosmos Framework can specialize Cosmos for.
| Example checkpoint | Base | Demonstrates |
|---|---|---|
| Cosmos3-Nano-Sim-Auto |
Nano | Closed-loop driving simulation – multi-view + LiDAR |
| Cosmos3-Nano-Transfer-Auto |
Nano | Multi-view driving SDG – up to 11 views + LiDAR |
| Cosmos3-Nano-Sim-Bimanual |
Nano | Closed-loop interactive robot policy simulation |
| Cosmos3-Nano-Sim-Transfer |
Nano | Auto-regressive transfer – depth, segmentation, edge, blur |
| Cosmos3-Super-Text2Image | Super | Elite quality text-to-image |
| Cosmos3-Super-Text2Image-4Step | Super | Elite quality text-to-image, 17-25x faster |
| Cosmos3-Super-Image2Video | Super | Elite quality image-to-video |
| Cosmos3-Super-Image2Video-4Step | Super | Elite quality image-to-video, 17-25x faster |
| Cosmos3-Nano-Policy-DROID | Nano | Open SOTA DROID robot policy, runs on RTX Pro 6000 |
| Cosmos3-Edge-Policy-DROID | Edge | DROID robot policy at edge-deployable scale |
Full I/O specs, generation settings, and supported action embodiments: model reference.
Platform
| Project | Purpose |
|---|---|
| Cosmos | This repo |
| Cosmos Framework | Train, optimize, and deploy physical AI models, SFT · LoRA · distillation · RL post-training, for Cosmos and beyond |
| Cosmos Curator | Distributed data curation: processing, annotation, filtering, dedup |
| Cosmos Evaluator | Automated evaluation system for world generation & reasoning outputs |
Cosmos 3 runs on Diffusers, Transformers, vLLM, vLLM-Omni, SGLang, TensorRT-LLM, and NIM. Pick a backend in the environment setup guide.
Community & contributing
Questions and ideas → Discussions. Bugs → Issues. Code and recipes → Cosmos Framework.
Limitations & safety
Cosmos 3 can produce artifacts in long, high-resolution, or physically complex outputs (temporal inconsistency, object morphing, implausible dynamics). Safety-critical applications need additional validation and system-level safety analysis. Generation ships with guardrails on by default.
Citation & license
@techreport{nvidia2026cosmos3,
title = {Cosmos 3: Omnimodal World Models for physical AI},
author = {{NVIDIA Cosmos Team}},
year = {2026},
url = {https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf}
}
Source code and models are released under OpenMDW-1.1. Custom licensing: cosmos-license@nvidia.com. This project may download third-party open source software; review those licenses before use.