source: hugging face blog: introducing cosmos 3 edge
level: technical
nvidia has released cosmos 3 edge, a 4-billion-parameter open world model designed for physical ai systems operating on edge devices. the model helps robots and vision ai agents understand their surroundings, reason in real time, and generate robot actions. it runs efficiently on nvidia edge computers including jetson, rtx pro gpus, and dgx, delivering memory-efficient, high-throughput inference. as a small vision language model, it offers best-in-class throughput and accuracy with real-time inference. as a post-trained world action model, it operates at robot-control resolution, generating 32 actions per inference on jetson thor while achieving real-time control at 15 hz.
cosmos 3 edge combines two transformer towers sharing multimodal attention layers. an autoregressive tower processes vision and text tokens for understanding and reasoning, while a diffusion tower handles vision, audio, and action tokens for prediction and generation. this design allows the model to reason about a scene before generating outputs. it maps different physical actions into a common geometric representation covering translation, rotation, and manipulation state. this connects control directly to visual structure, enabling the model to associate pixel changes with physical motion and control inputs. the model can predict the visual consequence of an action or infer an action from its effects, supporting policy learning and simulation.
developers can fine-tune cosmos 3 edge for custom robot policies using a small cluster of h100 gpus or nvidia dgx station before deploying to edge platforms. nvidia also released reference post-trained checkpoints and training recipes, including a policy checkpoint post-trained on the droid dataset for pick-and-place tasks. additionally, a cosmos 3 super 4-step distillation checkpoint reduces diffusion steps from 35–50 to just 4, delivering up to 25× faster inference for text-to-image and image-to-video generation. these open resources enable domain adaptation and faster deployment of world models for robotics and smart infrastructure.
why it matters: it brings data center-level world modeling and real-time action generation to memory-constrained edge devices, enabling more capable and responsive physical ai systems in factories, warehouses, and hospitals.