pytorch/pytorch v2.12.0: PyTorch 2.12.0 Release
extract
# PyTorch 2.12.0 Release Notes - [Highlights](#highlights) - [Backwards Incompatible Changes](#backwards-incompatible-changes) - [Deprecations](#deprecations) - [New Features](#new-features) - [Improvements](#improvements) - [Bug fixes](#bug-fixes) - [Performance](#performance) - [Documentation](#documentation) - [Developers](#developers) - [Security](#security) # Highlights <table> <tr><td><strong>Batched linalg.eigh on CUDA</strong> is up to 100x faster due to updated cuSolver backend selection.</td></tr> <tr><td>New <strong>torch.accelerator.Graph</strong> API unifies graph capture and replay across CUDA, XPU, and out-of-tree backends.</td></tr> <tr><td><strong>torch.export.save</strong> now supports Microscaling (MX) quantization formats, enabling full export of aggressively compressed models.</td></tr> <tr><td><strong>Adagrad</strong> now supports <code>fused=True</code>, joining Adam, AdamW, and SGD with a single-kernel optimizer implementation.</td></tr> <tr><td><strong>torch.cond</strong> control flow can now be captured and replayed inside CUDA Graphs.</td></tr> <tr><td><strong>ROCm</strong> users gain expandable memory segments, rocSHMEM symmetric memory collectives, and FlexAttention pipelining.</td></tr> </table> For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release. # Backwards Incompatible Changes ## Build Frontend - Strengthened SVE compile checks in `FindARM.cmake`, which may reject previously accepted but incorrect SVE configurations ([#176646](https://github.com/pytorch/pytorch/pull/176646)) Source builds that enable SVE now validate the compiler configuration more strictly. If a build previously passed with an incomplete or mismatched SVE setup, it may now fail during CMake configuration instead of later in compilation. Update the compiler/toolchain flags so they accurately describe the target SVE support, or disable SVE for that build. - Updated the minimum CUDA version required to build PyTorch from source to CUDA 12.6 ([#178925](https://github.com/pytorch/pytorch/pull/178925)) Building PyTorch from source with CUDA versions older than 12.6 is no longer supported. Users building custom binaries should install CUDA 12.6 or newer and make sure `CUDA_HOME` points to that installation. Version 2.11: ```bash CUDA_HOME=/usr/local/cuda-12.4 python setup.py develop ``` Version 2.12: ```bash CUDA_HOME=/usr/local/cuda-12.6 python setup.py develop ``` - Enforced a C++20 minimum in CMake build files ([#178662](https://github.com/pytorch/pytorch/pull/178662)) Source builds now require a compiler and build configuration that support C++20. If you maintain custom build scripts or downstream extensions that build PyTorch from source, update the compiler and remove assumptions that PyTorch can be built as C++17. ## Distributed - `torch.distributed.nn.functional` ops now raise `RuntimeError` under `torch.compile` ([#177342](https://github.com/pytorch/pytorch/pull/177342)) All ops in `torch.distributed.nn.functional` (e.g., `broadcast`, `all_reduce`, `all_gather`, `reduce_scatter`, `all_to_all_single`) now raise `RuntimeError` when called inside `torch.compile`. Users should migrate to the functional collectives API in `torch.distributed._functional_collectives`. Version 2.11: ```python @torch.compile def my_func(x): return torch.distributed.nn.functional.all_reduce(x, op=ReduceOp.SUM) ``` Version 2.12: ```python @torch.compile def my_func(x): return torch.distributed._functional_collectives.all_reduce(x, reduceOp="sum", group=group) ``` ## TorchElastic - `torchrun` now defaults to an OS-assigned free port for single-node training instead of port 29500 ([#175699](https://github.com/pytorch/pytorch/pull/175699)) When running `torchrun --nproc-per-node=N script.py` without specifying `--master-port` or `--standalone`, the default behavior now automatically uses an OS-assigned free port via the `c10d` rendezvous backend. This eliminates "Address already in use" errors when running multiple training jobs concurrently. Multi-node training, explicit `--master-port`, `PET_MASTER_PORT` env var, and `--standalone` are unchanged. Version 2.11: ```bash # Used static rendezvous on port 29500 by default torchrun --nproc-per-node=4 train.py ``` Version 2.12: ```bash # Uses OS-assigned free port by default torchrun --nproc-per-node=4 train.py # To explicitly use a fixed port: torchrun --nproc-per-node=4 --master-port=29500 train.py ``` ## MPS - All MPS tensors are now allocated in unified memory ([#175818](https://github.com/pytorch/pytorch/pull/175818)) Previously, MPS tensors could be allocated in either device-only or unified memory. Now all MPS tensors use unified memory unconditionally. This simplifies memory management and enables CPU access to MPS tensor data without explicit copies. Code that relied on device-only memory placement may observe different performance characteristics. ## Inductor - The `max_autotune` layout-constraint deferral introduced in 2.11 is now opt-in ([#175330](https://github.com/pytorch/pytorch/pull/175330)) In 2.11, Inductor deferred layout freezing for `max_autotune` templates to expose more fusion opportunities. This caused a regional-inductor failure mode, so the default in 2.12 reverts to immediate layout freezing. Users who relied on the deferred behavior for fusion opportunities should opt in explicitly via `torch._inductor.config.max_autotune_defer_layout_freezing` or `TORCHINDUCTOR_MAX_AUTOTUNE_DEFER_LAYOUT_FREEZING=1`. Version 2.11: ```python # Deferred layout freezing was the default torch.compile(model, mode="max-autotune") ``` Version 2.12: ```python import torch._inductor.config as cfg cfg.max_autotune_defer_layout_freezing = True # or set TORCHINDUCTOR_MAX_AUTOTUNE_DEFER_LAYOUT_FREEZING=1 torch.compile(model, mode="max-autotune") ``` # Deprecations ## Release Engineering - Deprecate CUDA 12.8 builds in favor of CUDA 13.0 ([#179072](https://github.com/pytorch/pytorch/pull/179072)) CUDA 12.8 binaries have been removed from the PyTorch binary build matrix. CUDA 13.0 is now the stable default and CUDA 12.6 remains available for users on older drivers. Users explicitly pinning the `cu128` index URL will need to switch to `cu130` (recommended) or `cu126`. Version 2.11: ```bash pip install torch --index-url https://download.pytorch.org/whl/cu128 ``` Version 2.12: ```bash # Use CUDA 13.0 (default on PyPI): pip install torch # Or explicitly: pip install torch --index-url https://download.pytorch.org/whl/cu130 # Older driver fallback: pip install torch --index-url https://download.pytorch.org/whl/cu126 ``` - Compatibility with CMake < 3.10 will be removed in a future release ([#166259](https://github.com/pytorch/pytorch/pull/166259)) Source builds against CMake versions older than 3.10 now emit a deprecation warning. A future release will require CMake 3.10 or newer; please upgrade CMake before then. ## Linear Algebra - Several CUDA linear algebra operators no longer use the MAGMA backend and now dispatch to cuSolver or cuBLAS unconditionally: - `torch.linalg.eigh` now dispatches to cuSolver ([#174619](https://github.com/pytorch/pytorch/pull/174619)) - `torch.linalg.lu_solve` now dispatches to cuSolver/cuBLAS ([#174248](https://github.com/pytorch/pytorch/pull/174248)) - `torch.linalg.cholesky_inverse` now dispatches to cuSolver ([#174681](https://github.com/pytorch/pytorch/pull/174681)) - `torch.linalg.cholesky_solve` now dispatches to cuSolver ([#174769](https://github.com/pytorch/pytorch/pull/174769)) User code calling these APIs does not need to change. The practical impact is for users who depended on MAGMA-specific numerical behavior, performance characteristics, or debugging. Those calls now use the cuSolver/cuBLAS implementations on CUDA. ## FullyShardedDataParallel2 (FSDP2) - Compiling through FSDP2 hooks without graph breaks is no longer supported ([#174863](https://github.com/pytorch/pytorch/pull/174863), [#174906](https://github.com/pytorch/pytorch/pull/174906)). If you use compiled autograd with FSDP2, update your code to allow graph breaks around FSDP2 hooks or disable compiled autograd for the FSDP2 training step. Version 2.11: ```python with torch._dynamo.config.patch(compiled_autograd=True): compiled_model = torch.compile(fsdp_model, fullgraph=True) loss = compiled_model(input).sum() loss.backward() ``` Version 2.12: ```python # Either run FSDP2 backward without fullgraph. compiled_model = torch.compile(fsdp_model, fullgraph=False) loss = compiled_model(input).sum() loss.backward() # Or apply compile before applying FSDP. compiled_model_pre_fsdp = torch.compile(model, fullgraph=True) compiled_model = fully_shard(compiled_model_pre_fsdp, ...) loss = compiled_model(input).sum() loss.backward() ``` ## Profiler - Profiler's `metadata_json` field is now deprecated; use `event_metadata` instead ([#179417](https://github.com/pytorch/pytorch/pull/179417)) Version 2.11: ```python metadata = event.metadata_json ``` Version 2.12: ```python metadata = event.event_metadata ``` ## Dynamo - `torch.compile(fullgraph=True)` now warns when a call runs no compiled code; will error in 2.13 ([#181940](https://github.com/pytorch/pytorch/pull/181940)) Previously `fullgraph=True` was only validated once Dynamo actually compiled and ran the function. If Dynamo was bypassed at call time (e.g. under a user-defined `TorchDispatchMode`), the annotation silently had no effect. 2.12 emits a warning; **2.13 will raise.** For graph-break errors without `fullgraph`'s stronger guarantees, use `torch._dynamo.error_on_graph_break`. Version 2.12: ```python from torch.utils._python_dispatch import TorchDispatchMode class LoggingMode(TorchDispatchMode): def __torch_dispatch__(self, func, types, args=(), kwargs=None): return func(*args, **(kwargs or {})) @torch.compile(fullgraph=True) def model(x): return x.sin() + 1 # A user-defined TorchDispatchMode is active, so Dynamo skips the frame # and no compiled code runs — emits a warning in 2.12, will raise in 2.13. with LoggingMode(): # Remove this to fix warning model(torch.randn(3, 4)) ``` - The `inline_inbuilt_nn_modules` Dynamo config is deprecated ([#177489](https://github.com/pytorch/pytorch/pull/177489), [#178205](https://github.com/pytorch/pytorch/pull/178205)) Inlining of in-built `nn.Module` instances is now the default; setting the flag emits a deprecation warning and it will be removed in a future release. Version 2.11: ```python import torch._dynamo.config as cfg cfg.inline_inbuilt_nn_modules = True # was a tunable knob ``` Version 2.12: ```python # No action needed — inlining is on by default. # Remove any explicit references to torch._dynamo.config.inline_inbuilt_nn_modules. ``` - Added a deprecation framework to the `torch.compile` config module so individual options can be marked deprecated ([#169837](https://github.com/pytorch/pytorch/pull/169837)) # New Features ## Release Engineering - Add Claude-powered autorevert AI advisor workflow ([#177404](https://github.com/pytorch/pytorch/pull/177404), [#178810](https://github.com/pytorch/pytorch/pull/178810)) - Add torchtitan tests to PyTorch CI ([#175901](https://github.com/pytorch/pytorch/pull/175901), [#176774](https://github.com/pytorch/pytorch/pull/176774), [#179749](https://github.com/pytorch/pytorch/pull/179749), [#177572](https://github.com/pytorch/pytorch/pull/177572)) - Add Pallas TPU CI configuration ([#173870](https://github.com/pytorch/pytorch/pull/173870), [#174532](https://github.com/pytorch/pytorch/pull/174532), [#175650](https://github.com/pytorch/pytorch/pull/175650), [#175590](https://github.com/pytorch/pytorch/pull/175590)) - Add downloadable profiler traces and TLParse output from CI runs ([#178488](https://github.com/pytorch/pytorch/pull/178488)) - Enable full AArch64 unit testing for pull requests, with periodic m7g coverage maintained on trunk ([#178270](https://github.com/pytorch/pytorch/pull/178270)) ## Python Frontend - Introduced `torch.accelerator.Graph` as a unified frontend Graph interface ([#171285](https://github.com/pytorch/pytorch/pull/171285)) ## Foreach - Add `_foreach_clone` operator, with a fast path for CUDA utilizing `_foreach_copy_` ([#177421](https://github.com/pytorch/pytorch/pull/177421)) ## Distributed - Add `Store::barrier` API and TCPStore client `BARRIER` support, reducing synchronization round trips compared to the existing `ADD`+`WAIT` pattern ([#174920](https://github.com/pytorch/pytorch/pull/174920)) - Add NCCL communicator `suspend()`, `resume()`, and `memory_stats()` APIs for managing communicator memory lifecycle ([#176300](https://github.com/pytorch/pytorch/pull/176300)) - Add `all_to_all` support in the Gloo backend ([#165435](https://github.com/pytorch/pytorch/pull/165435)) - Add `reduce_scatter_offset` to symmetric memory, supporting variable-sized block reductions with NVLink multicast or LSA fallback ([#177791](https://github.com/pytorch/pytorch/pull/177791)) - Enable `batch_isend_irecv` to work under `torch.compile` ([#161213](https://github.com/pytorch/pytorch/pull/161213)) - Add `torch.distributed.symmetric_memory.is_symm_mem_tensor()` API to check if a tensor is a symmetric memory tensor ([#178947](https://github.com/pytorch/pytorch/pull/178947)) - Convert `NanCheck` to a standalone op (`torch.ops.c10d.check_for_nan`) usable outside of `ProcessGroupNCCL` ([#174990](https://github.com/pytorch/pytorch/pull/174990)) ## DTensor - Add support for twice-differentiable DTensor redistribution ([#160509](https://github.com/pytorch/pytorch/pull/160509)) - DeviceMesh is now traceable by torch.compile. Make DeviceMesh opaque ([#176661](https://github.com/pytorch/pytorch/pull/176661)), Make placements opaque ([#171482](https://github.com/pytorch/pytorch/pull/171482)). - Add `grad_placements` parameter to `DTensor.from_local()`, allowing explicit control over gradient placements in the backward pass ([#175867](https://github.com/pytorch/pytorch/pull/175867)) ## FullyShardedDataParallel2 (FSDP2) - Support per-parameter meshes in FSDP2, enabling different parameter groups to shard over different meshes ([#173509](https://github.com/pytorch/pytorch/pull/173509)) - Support `fully_shard` with DTensors on a full SPMD mesh via `DataParallelMeshDims` ([#176334](https://github.com/pytorch/pytorch/pull/176334)) - Add FSDP2 support for non-floating-point parameters by excluding non-float parameters from reduce-scatter while still sharding and all-gathering them as needed ([#177948](https://github.com/pytorch/pytorch/pull/177948)) ## TorchElastic - Add configurable `--shutdown-timeout` to `torchrun` for controlling the SIGTERM-to-SIGKILL timeout during worker shutdown ([#172596](https://github.com/pytorch/pytorch/pull/172596)) ## CPU x86 - Expose a `CPUBlas` brgemm API for fp8 (e4m3 & e5m2) GEMM, backed by oneDNN ([#172548](https://github.com/pytorch/pytorch/pull/172548)) ## CUDA - Added support for `torch.cond` with CUDA graphs, using conditional graph nodes (CUDA 12.4+) so data-dependent control flow can be captured entirely inside a single CUDA graph. Works with the `eager` and `cudagraphs` `torch.compile` backends (no Inductor support yet). ([#168912](https://github.com/pytorch/pytorch/pull/168912)) ## MPS - Implemented `linalg_qr` for MPS ([#172536](https://github.com/pytorch/pytorch/pull/172536)) - Added `cholesky_solve` support on MPS ([#176703](https://github.com/pytorch/pytorch/pull/176703)) - Added `index_reduce` on MPS ([#174936](https://github.com/pytorch/pytorch/pull/174936)) - Implemented `torch.distributions.Gamma` (forward + backward) on MPS ([#179228](https://github.com/pytorch/pytorch/pull/179228)) - Enabled `mvlgamma` on MPS ([#178914](https://github.com/pytorch/pytorch/pull/178914)) - Added `nonzero_static` implementation on MPS ([#179589](https://github.com/pytorch/pytorch/pull/179589)) _(from miscategorized)_ ## ROCm - Enable expandable segments ([#173330](https://github.com/pytorch/pytorch/pull/173330), [#177974](https://github.com/pytorch/pytorch/pull/177974), [#179930](https://github.com/pytorch/pytorch/pull/179930), [#179781](https://github.com/pytorch/pytorch/pull/179781)) - hipSPARSELt - Enable for ROCm >= 7.12 ([#170852](https://github.com/pytorch/pytorch/pull/170852), [#178285](https://github.com/pytorch/pytorch/pull/178285)) - Enable FP8 semi-structured sparsity ([#179310](https://github.com/pytorch/pytorch/pull/179310)) - amdgcnspirv is now a supported offload target, not enabled by default ([#172559](https://github.com/pytorch/pytorch/pull/172559)) ## XPU - Support `torch.accelerator.Graph` on XPU ([#176421](https://github.com/pytorch/pytorch/pull/176421)) - Added `memory_clock_rate` and `memory_bus_width` to XPU device properties ([#171967](https://github.com/pytorch/pytorch/pull/171967)) - Enable `split_group` API when TorchComms is used as a backend for TorchTitan on XPU ([#178236](https://github.com/pytorch/pytorch/pull/178236)) ## Profiler - Profiler's Activity selection allows for fine-grained activity type selection. ([#176351](https://github.com/pytorch/pytorch/pull/176351)) - Memory visualize has a new tab to show private pool memory view ([#177289](https://github.com/pytorch/pytorch/pull/177289)) ## Dynamo - Made `torch._dynamo.aot_compile` public, with `aot_eager` and `inductor` backend support and docs ([#179917](https://github.com/pytorch/pytorch/pull/179917), [#180008](https://github.com/pytorch/pytorch/pull/180008)) - Added a `recompile_limit` keyword argument to `torch.compile` to override the per-function recompile cap without touching global config ([#177936](https://github.com/pytorch/pytorch/pull/177936)) - Added min/max bounds to `torch._dynamo.mark_unbacked` for communicating value ranges to the symbolic shape system ([#176313](https://github.com/pytorch/pytorch/pull/176313)) - Added `bdb`, a `pdb`-style debugger for stepping through nested frames during Dynamo tracing (`n`, `u`, `d`, `r`, `bt`), plus a user-callable `breakpoint()` that auto-starts it ([#174626](https://github.com/pytorch/pytorch/pull/174626), [#174746](https://github.com/pytorch/pytorch/pull/174746), [#175200](https://github.com/pytorch/pytorch/pull/175200)) ## Inductor - Added user-defined stream support to `torch.compile`. Inductor now codegens stream context managers (enter/exit) and `record_stream` calls in the wrapper, enabling user streams to flow through compiled regions with proper synchronization, scheduler integration, and cross-stream dependency tracking ([#165390](https://github.com/pytorch/pytorch/pull/165390), [#165391](https://github.com/pytorch/pytorch/pull/165391), [#165504](https://github.com/pytorch/pytorch/pull/165504), [#165505](https://github.com/pytorch/pytorch/pull/165505), [#174223](https://github.com/pytorch/pytorch/pull/174223), [#176700](https://github.com/pytorch/pytorch/pull/176700), [#177694](https://github.com/pytorch/pytorch/pull/177694)) - Added `ao::offload`, `ao::reload`, and `ao::wait` ops for asynchronous activation offloading. These ops encapsulate async CPU offloading stream management following the same async 2-op pattern as c10d functional collectives, reducing IR size from 7 nodes (offload) and 5 nodes (reload) down to 2 nodes each ([#177621](https://github.com/pytorch/pytorch/pull/177621)) - Added user-defin
retained text
No document body was retained for this record — only its metadata was stored.