PyTorch 2.15: Default NCCL backend and hardware optimizations
In brief
- The rewritten NCCL backend is now the default for multi-GPU communications in PyTorch 2.15.
- New optimized kernels for Apple Silicon (FP8), AMD (DSLs) and Intel (simplified model).
- Extended compatibility with CUDA 13.2/13.4, ROCm 10.0 and Python 3.11 to 3.15.
PyTorch has released version 2.15, featuring 3,117 commits from 583 contributors since 2.14. The update strengthens performance and reliability, notably by making the rewritten NCCL backend the default choice for multi-GPU communications.
The rewritten NCCL backend improves GPU-to-GPU communication handling, with a fallback mode for legacy systems. A flight recorder captures hangs in production to ease debugging. PyTorch 2.15 also optimizes performance on Apple Silicon (FP8, dedicated kernels), AMD (new compilation DSLs) and Intel (simplified programming model). Enhancements include pipeline parallelism, gradient precision management and broader hardware support (CUDA 13.2/13.4, ROCm 10.0, Python 3.11 to 3.15).
This release solidifies PyTorch as a unified platform for large-scale training and inference. Professionals should monitor performance gains on target architectures and the stability of new features, especially for cluster deployments.
What it changes for you
- CybersecurityEnhanced stability and debugging (flight recorder) lower risks of service disruptions during distributed training in critical environments.
This summary gives you the essentials. The full article, its figures and quotes are on PyTorch Blog.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error