NVIDIA/cutlass v4.8.0: CUTLASS 4.8.0
retained text
### CuTe DSL
* New features
- Initial Rubin support to accelerate dense GEMMs. The following features are available:
- CuTe DSL and CuTe extensions
- Support for higher-throughput FP8 (MMA_K=64) and FP4 (MMA_K=128) Tensor Core MMA instructions
- B collector reuse
- Extended TMEM size from 512 COL to 576 COL
- Larger shared memory allocations (328KB)
- Enhanced mixed precision throughput (FP8/FP4)
- Softmax acceleration related features
- Primitives
- Support for higher-throughput FP8 (MMA_K=64) and FP4 (MMA_K=128) Tensor Core MMA instructions
- B collector reuse
- Extended TMEM size from 512 COL to 576 COL
- Larger shared memory allocations (328KB)
- Enhanced mixed precision throughput (FP8/FP4)
- Softmax acceleration related feature
- 2:4 sparsity support for FP4
- CuTe DSL extensions has several new features:
- CTA-V maps are now inferred automatically for `cute_ext` TMA load, store, multicast, and reduce-store operations. Explicit CTA-V maps remain supported as overrides.
- Added asynchronous atomic TMA reduce-store and sparse MMA operations.
- Added reusable `cute_ext` GEMM mainloop and TMA epilogue helpers.
- Added opt-in TMEM accumulator-buffer planning, including overlapping ping-pong storage for capacity-constrained kernels.
- Improved device-side TMA descriptor updates and grouped GEMM performance through SMEM-staged updates, workspace reuse, and reduced prologue and synchronization overhead.
- This release includes an opt-in preview of the CuTe DSL extensions (`cute_ext`) compiler pipeline for ordinary Cute DSL kernels. This pipeline lets user mix `cute_ext` APIs directly into `@cute.jit` and `@cute.kernel` code and is required for kernels that mix the two API surfaces. You may test this feature with the following:
`CUTE_DSL_USE_EXTENSION_COMPILER=1 python your_program.py`
The pipeline is expected to preserve program behavior and performance, but generated PTX/SASS may differ. Note that this pipeline will become the default in the future, no earlier than 4.10.
- Added examples for better control over Primitives' compiler warnings/errors introduced in 4.7.0. See the `CuTeDSL/experimental/compiler_diagnostic/` directory.
- IKET Profiler Tool
- Rubin kernels (sm107) can now be profiled.
- It is now possible to only dump timing data for a specific cluster to reduce profiling overhead. Previously all clusters were profiled.
- Task Scheduling can instrument the schedule with IKET ranges when constructing TaskManager objects (`iket_enable_profiling=True`). Task execution will generate an IKET range and individual pipeline stages in a schedule may generate separate ranges (`iket_profiling_stages`).
- A number of new examples were added in this release:
- Rubin (CuTe):
- Dense GEMM for legacy data type with B collector reuse as applicable
- Grouped GEMM with B collector reuse
- Dense blockscaled GEMM with FP4/FP6/FP8 mixed precision and UE5M3 / block-32 scale-factor support
- Grouped blockscaled GEMM with B collector reuse as applicable
- Blockwise GEMM
- Rubin (CuTe extension):
- Support for higher-throughput FP8 (MMA_K=64) and FP4 (MMA_K=128) blockscaled GEMM with UE5M3 scale-factor
- Grouped GEMM with B collector reuse
- Blackwell (CuTe extension):
- Dense GEMMs
- Back-to-back GEMM
- Blockscaled GEMM
- Persistent GEMM with alpha/beta scaling
- CLC scheduler/dynamic persistent GEMM
- GLU GEMM
- Mixed input GEMM
- Planar complex GEMM
- Input transform GEMM
- GeForce pingpong dense GEMM
- Blackwell Ultra blockscaled GEMM
- Dense Convolutions
- Implicit-Gemm Fprop Conv
- Blocksclaed Implicit-Gemm Fprop Conv
- GeForce Implicit-Gemm Fprop Conv
- GeForce Blockscaled Implicit-Gemm Fporp Conv
- Attention
- GQA Decode
- Grouped GEMM
- Unscaled and blockscaled grouped GEMM
- Top-K
- Ampere (CuTe extension):
- SIMT GEMM
- CuTe DSL now supports x86_64 Windows
- CuTe DSL AoT now supports new host target: QNX8.0
- Notebooks are restructured under examples/python/CuTeDSL/cute/notebooks and new notebooks for primitives will be added under examples/python/CuTeDSL/notebooks
- Numpy is now not a default dependency
* Bug fixes and improvements:
- `nvidia-cuda-nvdisasm` is now an optional dependency of `nvidia-cutlass-dsl` via the optional `[sass]` extra. SASS dumping (`CUTE_DSL_KEEP=sass` / KeepSASS) now resolves `nvdisasm` from the bundled wheel (recommended since its version matches the DSL toolchain) or from a local CUDA Toolkit (`CUDA_HOME`/`CUDA_PATH`). A locally-provided `nvdisasm` must come from a CUDA Toolkit at least as new as the toolchain that produced the CUBIN. Installations that never dump SASS are unaffected.
- Reduced the protobuf version requirement of IKET profiler from 6.30 to 4.21. This should make protobuf an easier requirement to satisfy in preparation for transioning IKET to an optional extra.
- Improved JAX PyTree input/output aliasing for `cutlass.jax.cutlass_call`
- Fixed a regression from 4.6.0 where `cute.autovec_copy` emitted per-element instead of
vectorized instructions for tensors with a dynamic stride ([!3463](https://github.com/NVIDIA/cutlass/issues/3463))
- Fixed TVM-FFI env stream detection for GPU tensors in tuple
([!3444](https://github.com/NVIDIA/cutlass/issues/3444))
- Fixed GPU `link-libraries` compile-option order so it is stable across processes
([!3564](https://github.com/NVIDIA/cutlass/issues/3564))
- Fixed preprocessor `IndexError` on staged `bool()` with no arguments
([!3506](https://github.com/NVIDIA/cutlass/issues/3506))
- Rejected `cute.compile` on `@cute.kernel` with a user error instead of an ICE
([!3429](https://github.com/NVIDIA/cutlass/issues/3429))
- Fixed CuTe DSL crashing the Python interpreter when used in a REPL
([!3413](https://github.com/NVIDIA/cutlass/issues/3413))
- Fixed a cuDNN Frontend FROST SDPA backward compilation failure issue ([!3594](https://github.com/NVIDIA/cutlass/issues/3594))
- Fixed SIGABRTs in TVM-FFI launch for cuDNN Frontend SM100 ragged SDPA kernel ([!3595](https://github.com/NVIDIA/cutlass/issues/3595))
This release has been tested against the following packages:
- FlashAttention: [main (8d3a3b8)](https://github.com/Dao-AILab/flash-attention/commit/8d3a3b80d4758ebde5a867c50d24d4351443cf2b)
- Quack: [main (35266c3)](https://github.com/Dao-AILab/quack/commit/35266c3298f0e9bf6d5f46c30aace2eaeae517e3)
- FlashInfer: [main (5d0c89e)](https://github.com/flashinfer-ai/flashinfer/commit/5d0c89eacae6ca08f2a1ce92eba557bbad7a1bfc)
- cuDNN-Frontend: [deveop (e0317d1)](https://github.com/NVIDIA/cudnn-frontend/commit/e0317d1f6cbc1bb8e50abce5870b6bf59b9d4a74)
- Pytorch: [main (7d5f021)](https://github.com/pytorch/pytorch/commit/7d5f0216450b8253e62d88284d49de725d88bd42)
- TensorRT-LLM: [main (c295dd9)](https://github.com/NVIDIA/TensorRT-LLM/commit/c295dd9fca143a3fdd2769617699fdd4d4a93eb5)
### CUTLASS Operator API
* Dense and blockscaled GEMMs in Operator API have preliminary Rubin support. These are provided as a preview and may need additional performance tuning.
- Updated GEMMs include:
- Dense GEMMs: FP8xFP8
- Blockscaled GEMM: {MXFP8}x{MXFP4, MXFP8} and {MXFP4, NVFP4}x{MXFP4, NVFP4} (including support for the new UE5M3 scale factor dtype for NVFP4).
- These kernels utilize the below new features in Rubin:
- Higher SMEM (328KB) and TMEM capacity (288KB)
- B-buffer reuse
- Enhanced mixed precision throughput
* Operators can now be ranked by their estimated performance when nvMatmulHeuristics is available. See tutorial here. NOTE: This currently only supports Blackwell kernels as nvMatmulHeuristics does not yet support Rubin.
* Standalone kernel implementations are now exposed through `cutlass.kernels`. These kernels can be used directly, in addition to being discoverable and usable via the Operator interface in `cutlass.operators`.
* Custom Epilogue fusions now support per-row or per-column reductions.
* `IndexPtrGroupedGemmArguments` is now used to represent Grouped GEMM with contiguous-offset/index-pointers. Existing `GroupedGemmArguments` is deprecated and will be removed in a future release.
### C++
* Added initial Rubin support (SM107) with CuTe C++ building blocks:
- [Rubin Tensor Core MMA instructions](https://github.com/NVIDIA/cutlass/blob/main/include/cute/arch/mma_sm107_umma.hpp) and corresponding [CuTe MMA traits](https://github.com/NVIDIA/cutlass/blob/main/include/cute/atom/mma_traits_sm107.hpp).
* CuTe examples that demonstrate the use of Rubin SM107 Tensor Core instructions:
- [Dense FP8 GEMM](https://github.com/NVIDIA/cutlass/blob/main/examples/cute/rubin/rubin_fp8.cu).
- [Block-scaled FP8 GEMM](https://github.com/NVIDIA/cutlass/blob/main/examples/cute/rubin/rubin_fp8_blockscaled.cu).
- [Mixed-precision block-scaled FP8/FP4 GEMM](https://github.com/NVIDIA/cutlass/blob/main/examples/cute/rubin/rubin_fp8_blockscaled.cu).
- [Block-scaled FP4 GEMM](https://github.com/NVIDIA/cutlass/blob/main/examples/cute/rubin/rubin_fp4_blockscaled.cu).
* Adjusted shared-memory and tensor-memory capacity handling for Rubin SM107:
- Set the [SM107 shared-memory capacity](https://github.com/NVIDIA/cutlass/blob/main/include/cutlass/arch/arch.h) to 327 KiB and added launch support for oversized shared-memory configurations.
- Set the [SM107 TMEM capacity](https://github.com/NVIDIA/cutlass/blob/main/include/cute/arch/tmem_capacity_sm100.hpp) to 576 columns per SM, updated the [CuTe 1SM and 2SM TMEM allocators](https://github.com/NVIDIA/cutlass/blob/main/include/cute/arch/tmem_allocator_sm100.hpp) for Rubin's exclusive allocation path.
* Enabled the existing SM100-compatible [GEMM](https://github.com/NVIDIA/cutlass/tree/main/include/cutlass/gemm/collective/builders) and [convolution](https://github.com/NVIDIA/cutlass/blob/main/include/cutlass/conv/collective/builders/sm100_umma_builder.inl) for the new SM107 [`sm_107a` and `sm_107f` targets](https://github.com/NVIDIA/cutlass/blob/main/CMakeLists.txt):
- Set of unit tests for Rubin SM107 [SIMT GEMM](https://github.com/NVIDIA/cutlass/blob/main/test/unit/gemm/device/sm107_gemm_f32_f32_f32_simt_align1_multi_cluster_shape.cu), [dense FP8 GEMM](https://github.com/NVIDIA/cutlass/blob/main/test/unit/gemm/device/sm107_gemm_f8_f8_f8_tensor_op_f32_alignx.cu), [block-scaled FP8 GEMM](https://github.com/NVIDIA/cutlass/blob/main/test/unit/gemm/device/sm107_gemm_f8_f8_f8_tensor_op_f32_blockwise.cu), [block-scaled FP4 GEMM](https://github.com/NVIDIA/cutlass/blob/main/test/unit/gemm/device/sm107_gemm_f4_f4_f32_tensor_op_f32_2sm_256x192.cu), and [mixed-precision, complex, and 9xBF16 GEMM](https://github.com/NVIDIA/cutlass/blob/main/test/unit/gemm/device/sm107_gemm_umma.cu)
* Various improvements and fixes from the community and CUTLASS team. Thanks to everyone who submitted PRs!
* Optimal code generation with CUDA toolkit versions 13.4.
NOTE: Executing Rubin kernels (SM107) requires the R615 driver which will be released
with CUDA Toolkit 13.4 GA. R610 from CUDA Toolkit 13.4 Developer Preview is not
sufficient.document fetched 2026-09-29 02:33