source: PyTorch Blog: PyTorch 2.15 Release Blog

level: technical

pytorch 2.15 is now available, bringing major updates to distributed training and hardware support. the rewritten nccl backend becomes the default for all nccl process groups, with a flight recorder that dumps on hangs and rollback paths for compatibility. one-sided transports are introduced with a nixl backend, allowing data movement without process groups and with cuda graph capture. fsdp2 and pipeline parallelism gain finer memory and communication control, including fp32 for precision-sensitive parameters and overlapped gradient reduction.

the release includes 3,117 commits from 583 contributors. on apple silicon, fp8 support and a new grouped_mm with a decode fast path deliver 1.32 to 1.34 times higher tokens per second on mixture-of-experts decode. nvidia blackwell gpus see faster 4-bit nvfp4 decoding through fewer kernel launches in nvgemm. amd gfx950 gets flydsl kernels, and intel gpus gain simpler sycl kernel authoring and multi-gpu profiling. cuda 13.2 is the default wheel, with cuda 13.4 wheels adding experimental support for nvidia rubin gpus.

pytorch 2.15 continues the framework's evolution from research-first to a unified, hardware-agnostic platform for production training and inference. the flight recorder in the nccl backend helps debug cluster-scale hangs by capturing what each rank was doing at the moment of failure. python 3.10 support is dropped, requiring python 3.11 or newer. the release also expands the stable c++ abi and header-only surface, and adds cuda graph kernel attribution for profiling. these changes aim to improve reliability and performance for large-scale ai workloads.

why it matters: pytorch 2.15 improves distributed training reliability and inference speed across major gpu vendors, directly benefiting ai teams running large models.


source: PyTorch Blog: PyTorch 2.15 Release Blog