← sources

NVIDIA/Megatron-LM core_v0.20.0: NVIDIA Megatron Core 0.20.0

source: github (github:NVIDIA/Megatron-LM)published 2026-10-29 21:06ingested 2026-10-30 02:33

extract

#### MoE * **DeepEP v2 dispatcher:** adds an experimental DeepEP v2 backend for the flex token dispatcher ([\#5153](https://github.com/NVIDIA/Megatron-LM/pull/5153)). * **NCCL EP MXFP8 communication (experimental):** adds MXFP8 dispatch-forward and combine-backward communication to the NCCL EP backend ([\#6524](https://github.com/NVIDIA/Megatron-LM/pull/6524)). * **NCCL EP eager mode**: now NCCL EP supports both eager mode and full-iteration graph mode ([\#6229](https://github.com/NVIDIA/Megatron-LM/pull/6229)). * [**ShortcutMoE**](https://arxiv.org/pdf/2404.05019)**:** adds shortcut-connected routing in HybridStack and overlaps expert all-to-all communication with paired attention or Mamba computation; CUDA graphs are not supported ([\#6959](https://github.com/NVIDIA/Megatron-LM/pull/6959)). #### Model Architecture and Support * **DeepSeek V4:** fully supported by the next-gen `HybridModel` class ([\#6403](https://github.com/NVIDIA/Megatron-LM/pull/6403), [\#6404](https://github.com/NVIDIA/Megatron-LM/pull/6404), [\#6405](https://github.com/NVIDIA/Megatron-LM/pull/6405), [\#7469](https://github.com/NVIDIA/Megatron-LM/pull/7469)). * **Manifold-constrained hyper-connections (mHC):** adds fused mHC connections to HybridModel, including pipeline and virtual-pipeline parallel support ([\#6401](https://github.com/NVIDIA/Megatron-LM/pull/6401), [\#7336](https://github.com/NVIDIA/Megatron-LM/pull/7336)). * **Gated Delta Product (GDP):** adds GDP layers and model-building support for hybrid architectures ([\#6074](https://github.com/NVIDIA/Megatron-LM/pull/6074)). * **Wide residual streams:** adds streamwise residual connections, selective residual-stream recomputation, and integration with MTP and MIMO; currently requires PP=1 and CUDA graphs disabled ([\#6716](https://github.com/NVIDIA/Megatron-LM/pull/6716), [\#6805](https://github.com/NVIDIA/Megatron-LM/pull/6805), [\#6855](https://github.com/NVIDIA/Megatron-LM/pull/6855), [\#7440](https://github.com/NVIDIA/Megatron-LM/pull/7440)). * **MTP-only training:** adds a mode that freezes the base model and trains only Multi-Token Prediction heads ([\#7021](https://github.com/NVIDIA/Megatron-LM/pull/7021)). #### Parallelism * **Megatron-FSDP v2 integration:** adds the experimental Megatron Core adapter and Megatron Bridge, with context, expert, pipeline, and hybrid data-parallel sharding support; the adapter requires TP=1 and expert TP=1 ([\#5865](https://github.com/NVIDIA/Megatron-LM/pull/5865), [\#6450](https://github.com/NVIDIA/Megatron-LM/pull/6450), [\#6486](https://github.com/NVIDIA/Megatron-LM/pull/6486), [\#7220](https://github.com/NVIDIA/Megatron-LM/pull/7220), [\#6873](https://github.com/NVIDIA/Megatron-LM/pull/6873)). * **Megatron-FSDP v2 checkpointing:** adds PyTorch distributed checkpoint save/load for model and optimizer state; optimizer checkpointing currently excludes expert-parallel layouts ([\#6024](https://github.com/NVIDIA/Megatron-LM/pull/6024), [\#6215](https://github.com/NVIDIA/Megatron-LM/pull/6215)). * Megatron-FSDP v2 supports hybrid FSDP. * The user can specify different ZeRO levels for inter-nvlink domains and intra-nvlink domains. * The user can specify different configs for the dense section and the MoE section. * **Generalized Tensor Parallelism (GTP):** adds configurable batch/sequence partitioning, opt-in latent-projection sharding, and MTP support ([\#7748](https://github.com/NVIDIA/Megatron-LM/pull/7748), [\#6383](https://github.com/NVIDIA/Megatron-LM/pull/6383), [\#6242](https://github.com/NVIDIA/Megatron-LM/pull/6242)). #### Performance and Memory * **Determinism:** multi-release effort, please read [our blog post](https://developer.nvidia.com/blog/scale-bitwise-deterministic-pretraining-with-nvidia-megatron-core/) to learn more * **Megatron-FSDP v2 CUDA graphs:** adds full-iteration graph capture and replay for supported dense-model configurations ([\#7075](https://github.com/NVIDIA/Megatron-LM/pull/7075)). * **Fused GDN execution:** adds Transformer Engine fused GDN and opt-in preprocessing fusion for convolution, layout transforms, normalization, and gates ([\#7583](https://github.com/NVIDIA/Megatron-LM/pull/7583), [\#5532](https://github.com/NVIDIA/Megatron-LM/pull/5532)). * **MLA and MoE kernel optimizations:** adds fused MLA Q up-projection, RoPE, and MXFP8 quantization for supported SM100 configurations, vectorized MLA projection copies, and fused shared-expert/residual additions after MoE combine ([\#6213](https://github.com/NVIDIA/Megatron-LM/pull/6213), [\#7957](https://github.com/NVIDIA/Megatron-LM/pull/7957), [\#7943](https://github.com/NVIDIA/Megatron-LM/pull/7943)). * **Weighted checkpoint merging:** adds CPU-only weighted merging of compatible, same-layout torch\_dist checkpoints ([\#5114](https://github.com/NVIDIA/Megatron-LM/pull/5114)). * **Training observability:** adds optional OpenTelemetry traces and metrics for training, pipeline communication, gradient synchronization, and checkpointing ([\#6415](https://github.com/NVIDIA/Megatron-LM/pull/6415), [\#6438](https://github.com/NVIDIA/Megatron-LM/pull/6438)). #### Precision and Optimizers * **Megatron-FSDP v2 mixed precision:** enables MXFP8 compute and quantized parameter gathers through the adapter, alongside mixed-precision optimizer support ([\#7588](https://github.com/NVIDIA/Megatron-LM/pull/7588), [\#7114](https://github.com/NVIDIA/Megatron-LM/pull/7114), [\#6506](https://github.com/NVIDIA/Megatron-LM/pull/6506)). * **Selective FP32 execution:** adds FP32 handling for selected tensors and their optimizer/checkpoint lifecycle, plus an option to compute language-model output logits in FP32 ([\#5929](https://github.com/NVIDIA/Megatron-LM/pull/5929), [\#6252](https://github.com/NVIDIA/Megatron-LM/pull/6252)). * **Distributed Muon improvements:** adds layer-sharded Muon computation across TP and GTP-rematerialization ranks and an optional Triton SYRK path for Newton-Schulz iterations ([\#6683](https://github.com/NVIDIA/Megatron-LM/pull/6683), [\#6381](https://github.com/NVIDIA/Megatron-LM/pull/6381)). * **DeepSeek-V3 NVFP4 training:** adds NVFP4 support including FP4 quantization in active MTP layers ([\#7328](https://github.com/NVIDIA/Megatron-LM/pull/7328)). #### Inference * **GDN and GDP dynamic inference:** adds GDN serving and extends GDP inference with CUDA graphs, chunked prefill, prefix caching, and speculative decoding ([\#6499](https://github.com/NVIDIA/Megatron-LM/pull/6499), [\#6595](https://github.com/NVIDIA/Megatron-LM/pull/6595), [\#6711](https://github.com/NVIDIA/Megatron-LM/pull/6711), [\#6780](https://github.com/NVIDIA/Megatron-LM/pull/6780), [\#7245](https://github.com/NVIDIA/Megatron-LM/pull/7245)). * **Prefill-Decode Disaggregation:** adds NCCL and NIXL KV-cache transfer paths and SSM-state handoff primitives for prefill/decode disaggregation ([\#5861](https://github.com/NVIDIA/Megatron-LM/pull/5861), [\#6222](https://github.com/NVIDIA/Megatron-LM/pull/6222), [\#6584](https://github.com/NVIDIA/Megatron-LM/pull/6584)). * **Batch-Invariant MoE inference:** adds batch-invariant BF16/MXFP8 execution paths, including vLLM fused MoE and MXFP8 expert GEMMs ([\#6521](https://github.com/NVIDIA/Megatron-LM/pull/6521), [\#7302](https://github.com/NVIDIA/Megatron-LM/pull/7302), [\#7562](https://github.com/NVIDIA/Megatron-LM/pull/7562)). * **Token checkpointing and staging**: adds engine hooks to preprocess requests and stage completed token payloads ([\#7015](https://github.com/NVIDIA/Megatron-LM/pull/7015)), extends staged records with compact prompt IDs and media tensors for multimodal NRL training ([\#7557](https://github.com/NVIDIA/Megatron-LM/pull/7557)), and moves staging off the inference loop so slow checkpoint writes do not stall decoding ([\#8169](https://github.com/NVIDIA/Megatron-LM/pull/8169)). * **Expert Tensor Parallel**: [\#4743](https://github.com/NVIDIA/Megatron-LM/pull/4743) supports ETP in the inference engine. * **ShortcutMoE Inference CG**: [\#7772](https://github.com/NVIDIA/Megatron-LM/pull/7772) adds CG support for shortcut MoE, GDP, and wide residuals. * **Inference Service Features and Resiliency**: [\#7828](https://github.com/NVIDIA/Megatron-LM/pull/7828) makes seeded sampling independent of batching, request movement, and other requests’ RNG consumption, [\#6916](https://github.com/NVIDIA/Megatron-LM/pull/6916) aborts generation when HTTP clients disconnect, preventing orphaned requests from consuming all serving capacity, and large-scale / concurrency optimizations with frontend-coordinator-engine data transfer and replicas in[\#6497](https://github.com/NVIDIA/Megatron-LM/pull/6497) and [\#6223](https://github.com/NVIDIA/Megatron-LM/pull/6223). #### Datasets * **DP-balanced sequence packing:** adds variable-length dataset scheduling, THD-format batch fetching with padding masks, and training-loop integration to balance packed batches across data-parallel ranks ([\#5903](https://github.com/NVIDIA/Megatron-LM/pull/5903), [\#6625](https://github.com/NVIDIA/Megatron-LM/pull/6625), [\#6684](https://github.com/NVIDIA/Megatron-LM/pull/6684), [\#6742](https://github.com/NVIDIA/Megatron-LM/pull/6742)). #### Multimodal * **Multimodal (Vision) Inference**: [\#6260](https://github.com/NVIDIA/Megatron-LM/pull/6260) adds image modality support to the dynamic inference engine, [\#6809](https://github.com/NVIDIA/Megatron-LM/pull/6809) adds video inference support, vision-embedding caching, media-affinity routing, and media-safe prefix hashes; [\#7598](https://github.com/NVIDIA/Megatron-LM/pull/7598) adds multi-turn expanded-prefix stitching; [\#7839](https://github.com/NVIDIA/Megatron-LM/pull/7839) serves MIMO-trained VLM checkpoints. * **Heterogeneous MIMO extensions:** adds GTP, non-colocated context parallelism, configurable input-projection placement, and model-declared weight refit ([\#6570](https://github.com/NVIDIA/Megatron-LM/pull/6570), [\#7435](https://github.com/NVIDIA/Megatron-LM/pull/7435), [\#6843](https://github.com/NVIDIA/Megatron-LM/pull/6843), [\#7367](https://github.com/NVIDIA/Megatron-LM/pull/7367)). * **RoPE audio encoder:** adds dense and packed execution modes with NeMo checkpoint compatibility ([\#7066](https://github.com/NVIDIA/Megatron-LM/pull/7066)). <details><summary>Changelog Details</summary> - fix(ci): AUT-1080 retry live NCCL watchdog timeouts by @svcnemo-autobot :: PR: #6054 - Prevent coordinator crash if an engine disconnects by @tdene :: PR: #6025 - Make nightly sync checks advisory by @Phlip79 :: PR: #5713 - test(optimizer): MCORE-560 cover MoE gradient zero counts by @svcnemo-autobot :: PR: #6050 - RL: fix logging issues from failed rollouts by @tdene :: PR: #6061 - chore(ci): AUT-1135 pin GitHub Actions to commit SHAs by @svcnemo-autobot :: PR: #6100 - chore(deps): AUT-1080 pin merged NeMo Run cancellation fix by @svcnemo-autobot :: PR: #6110 - chore: AUT-1142 bump package versions by @svcnemo-autobot :: PR: #6120 - test(inference): mark prefix caching CUDA graph test flaky by @ko3n1g :: PR: #6131 - Make BertModel lm_head optional. by @bbuschkaemper :: PR: #5690 - ci: AUT-1141 skip MBridge dispatch for docs-only changes by @svcnemo-autobot :: PR: #6116 - Fix gradient counting for muon+expert biases by @philipcmonk :: PR: #6099 - test(inference): MCORE-561 trust FP8 metadata in DeepSeek checkpoints by @svcnemo-autobot :: PR: #6049 - Allow the MInf sync API to run serve() by @tdene :: PR: #6123 - ci: fix test notifications for weekly/release tests by @balasaajay :: PR: #6055 - Add disaggregated KV transfer backends by @nvcsathe :: PR: #5861 - Reorganize a few docs by @Phlip79 :: PR: #6109 - Perf: skip per-param copy_ dispatch in the MXFP8 param copy-back by @fanshiqing :: PR: #6094 - test: add async scheduling inference coverage by @lmcafee-nvidia :: PR: #6104 - test(ci): AUT-1173 set functional tests to ft_launcher by @svcnemo-autobot :: PR: #6128 - Fix --freeze-all-layers for new model builder path + unit tests by @AAnoosheh :: PR: #5926 - fix(tests): prefer staged release assets by @ko3n1g :: PR: #6149 - Fix inference overflow into dummy blocks by @tdene :: PR: #5950 - Rebind MFSDP v2 sharded grads once per gradient reduction by @wujingyue :: PR: #6041 - Fix Megatron FSDP optimizer under-update by @wujingyue :: PR: #5976 - Normalize MFSDP data-parallel axes at fully_shard by @wujingyue :: PR: #6067 - Fix MLA dynamic inference decode flag by @cuichenx :: PR: #4902 - Add Agent Compose package and runtime interface skeleton by @ISEEKYAN :: PR: #5937 - [fix] Fix GTP full-iteration CUDA-graph capture regression by @prajwal1210 :: PR: #6077 - Harden dynamic prefix cache allocation - mamba mostly by @wdykas :: PR: #6091 - ci(cache): AUT-1188 harden container cache selection by @svcnemo-autobot :: PR: #6160 - ci(nvskills): AUT-1188 fix reusable workflow startup failure by @svcnemo-autobot :: PR: #6162 - Add MFSDP runtime schedule design by @wujingyue :: PR: #6031 - Feature/enable inference optimized qwen moe by @utkarsh530 :: PR: #5700 - Bs invar moe by @wdykas :: PR: #4871 - chore: update pytest version to 9.1.1 in pyproject.toml and uv.lock by @balasaajay :: PR: #6170 - Fix LRU block accounting for resumed requests by @lmcafee-nvidia :: PR: #5995 - [Main] update moe single weight unit test by @zhongbozhu :: PR: #6035 - fix ckpt conversion issue by @dimapihtar :: PR: #6151 - Add determinism developer documentation by @Connor-XY :: PR: #5718 - Add MFSDP v2 MCore integration with FullyShardedOptimizer by @wujingyue :: PR: #5865 - Add streaming replies for dynamic engine by @nvcsathe :: PR: #5727 - deps: Update urllib3 to version 2.7.0 by @balasaajay :: PR: #6182 - deps: Update black dependency to version 26.3.0 by @balasaajay :: PR: #6180 - Add meta-parameter support by @wujingyue :: PR: #5369 - Fix tied parameters by @wujingyue :: PR: #6164 - chore(ci): AUT-1231 bump preflight workflow template by @svcnemo-autobot :: PR: #6198 - Clarify single communication stream benefits by @wujingyue :: PR: #6177 - Bump black version in pre-commit config to match pyproject.toml by @xuantengh :: PR: #6201 - set weights_only=True by @dimapihtar :: PR: #6150 - [fix] Give same-key GTP chain neighbours distinct gather buffers by @fanshiqing :: PR: #6207 - Fix gradient reduction issue when EP=1, EP=TP, and EGTP != GTP by @philipcmonk :: PR: #6080 - Apply black 26.3.0 formatting to four files it now reformats by @deepakn94 :: PR: #6210 - Add context-parallel causal conv1d support by @Mellonta :: PR: #5917 - Refactor TE fused ops integration into mixin by @CarlosGomes98 :: PR: #4630 - [feat] Add reduce-scatter-with-fp32-accumulation support for GTP by @fanshiqing :: PR: #6200 - chore(ci): AUT-1298 bump community workflow to v1.8.7 by @svcnemo-autobot :: PR: #6219 - fix: pretrain_hybrid.py --help by @paulwambergue :: PR: #6211 - refactor(ci): migrate triage consumer to Cerno by @thomasdhc :: PR: #6250 - Validate NIXL CUDA transports and share transfer agents by @wdykas :: PR: #6221 - Add prefix-cache lifecycle hooks by @nvcsathe :: PR: #5860 - Make Bridge Communicator aware of GTP by @yashaswikarnati :: PR: #6263 - Fix GTP reduce-scatter overlap across local CUDA graphs by @JF-D :: PR: #6060 - revert(dist_checkpointing): AUT-1325 remove streaming checkpoint dequantization by @svcnemo-autobot :: PR: #6275 - avoid circular dependency usage between mcore and modelopt- #5644 by @dimapihtar :: PR: #6247 - Update DEEPEP_COMMIT in Dockerfile.ci.dev to commit de0dd1… by @balasaajay :: PR: #6261 - revert: revert community-bot bump to v1.8.7 (5adbee2) by @kajalj22 :: PR: #6288 - [fix] Cleanup usage of pg_collection in gtp by @fanshiqing :: PR: #6234 - [Copy to main] Update Nemotron 3 Super GB200 release config by @svcnvidia-nemo-ci :: PR: #6282 - Revert "Add Agent Compose package and runtime interface skeleton" by @Phlip79 :: PR: #6292 - Release MFSDP storage after model deletion by @wujingyue :: PR: #6230 - Document MFSDP optimizer design by @wujingyue :: PR: #6186 - Handle checkpoint and text-only batches in MIMO encoder prefetch by @yashaswikarnati :: PR: #6039 - Revert "avoid circular dependency usage between mcore and modelopt- #5644" by @Phlip79 :: PR: #6302 - Fix multimodal MTP loss masking for modality tokens by @kevjshih :: PR: #6010 - Replace lazy MFSDP context initialization with an explicit scope by @wujingyue :: PR: #6190 - chore(ci): bump community workflow to v1.8.8 by @ko3n1g :: PR: #6322 - fix(moe): free activations immediately in full recompute mode by @liuyun7345 :: PR: #5197 - Allocate main_grad on the reduce-scatter stream by @wujingyue :: PR: #6187 - Encapsulate symmetric-memory AVG in DBuffer by @wujingyue :: PR: #6169 - fix(ci): remove community workflow PAT by @ko3n1g :: PR: #6350 - Add multimodal argument and config plumbing by @matthieule :: PR: #6332 - Fix FSDP activation recompute prefetch by @shjwudp :: PR: #6153 - Add Nemotron 3.5 Lightning GB200 nightly test by @Phlip79 :: PR: #6312 - Exit non-zero when --exit-on-missing-checkpoint triggers by @shanmugamr1992 :: PR: #6321 - Divide expert gradients by the expert-parallel size by @wujingyue :: PR: #5956 - [feat] GTP+MTP by @fanshiqing :: PR: #6242 - fix(mtp): scatter the MTP embedding when the model's embedding does not by @DanialTaheri :: PR: #6256 - Apply the layerwise param layout on the ModelBuilder DDP path by @deepakn94 :: PR: #6286 - test: AUT-1363 prevent recipe fetch-token exposure by @svcnemo-autobot :: PR: #6364 - fix(ci): AUT-1396 use one global external queue worker by @svcnemo-autobot :: PR: #6366 - Add generic interface for SSM inference by @santhnm2 :: PR: #5382 - Add PyTorch DCP checkpoint save/load for MFSDP v2 by @ahmadki :: PR: #6024 - Skip save-side sharding validation via --no-ckpt-load-validate-sharding-integrity by @asolergi-nv :: PR: #5554 - [feat] opt-in GTP_remat sharding support for 'moe-latent-proj' by @fanshiqing :: PR: #6383 - Support RL eval-only runs by @tdene :: PR: #6043 - fix(ci): redact sensitive environment values from test output by @ko3n1g :: PR: #6406 - Add --muon-use-syrk option for Triton SYRK kernel in Newton-Schulz by @deepakn94 :: PR: #6381 - Add 1-node GB200 functional test for checkpoint load with load_optim=False by @asolergi-nv :: PR: #6347 - Selective-FP32 runtime and native-FP32 optimizer checkpoint lifecycle by @FDecaYed :: PR: #5929 - Update docstring for moe_router_topk_scaling_factor by @JavaZeroo :: PR: #2178 - Add --ckpt-drop-redundant-extra-state to skip persisting redundant TE _extra_state by @asolergi-nv :: PR: #5552 - Bump FLA to 0.5.1 and integrate GDN2 kernels by @xuantengh :: PR: #5765 - Add construction-scoped MFSDP v2 fully shard options by @wujingyue :: PR: #6127 - Support explicit Hugging Face finetune data files in finetune.sh by @jenchen13 :: PR: #6413 - Add megatron.core.telemetry base layer and observability guide by @deepakn94 :: PR: #6415 - [NCCL EP] Support eager mode and overflow handling by @YangFei1990 :: PR: #6229 - Gate expensive work behind log levels by @shurkat-nvidia :: PR: #5805 - Fixes for SP + MTP all-gathers by @santhnm2 :: PR: #5748 - TE Layernorm dtype guard with fp32 residuals by @mkhona-nvidia :: PR: #6272 - Treat LayerWise bucket_size as a soft minimum and fill would-be padding with real parameters by @deepakn94 :: PR: #5415 - Gated Delta Product (GDP) implementation by @devnkong :: PR: #6074 - fix(mfsdp): refresh V2 compute weights after optimizer step by @wujingyue :: PR: #6336 - [core] Clarify GPT output_processor callback contract by @chengcuiping :: PR: #5739 - Validate MFSDP module phase transitions by @wujingyue :: PR: #6359 - Refine MFSDP v2 configuration validation by @wujingyue :: PR: #6333 - Add hybrid layer config definitions by @Phlip79 :: PR: #6410 - Propagate RL runtime config updates to module configs by @

retained text (truncated)

#### MoE

* **DeepEP v2 dispatcher:** adds an experimental DeepEP v2 backend for the flex token dispatcher ([\#5153](https://github.com/NVIDIA/Megatron-LM/pull/5153)).  
* **NCCL EP MXFP8 communication (experimental):** adds MXFP8 dispatch-forward and combine-backward communication to the NCCL EP backend ([\#6524](https://github.com/NVIDIA/Megatron-LM/pull/6524)).  
* **NCCL EP eager mode**: now NCCL EP supports both eager mode and full-iteration graph mode ([\#6229](https://github.com/NVIDIA/Megatron-LM/pull/6229)).  
* [**ShortcutMoE**](https://arxiv.org/pdf/2404.05019)**:** adds shortcut-connected routing in HybridStack and overlaps expert all-to-all communication with paired attention or Mamba computation; CUDA graphs are not supported ([\#6959](https://github.com/NVIDIA/Megatron-LM/pull/6959)).

#### Model Architecture and Support

* **DeepSeek V4:** fully supported by the next-gen `HybridModel` class ([\#6403](https://github.com/NVIDIA/Megatron-LM/pull/6403), [\#6404](https://github.com/NVIDIA/Megatron-LM/pull/6404), [\#6405](https://github.com/NVIDIA/Megatron-LM/pull/6405), [\#7469](https://github.com/NVIDIA/Megatron-LM/pull/7469)).  
* **Manifold-constrained hyper-connections (mHC):** adds fused mHC connections to HybridModel, including pipeline and virtual-pipeline parallel support ([\#6401](https://github.com/NVIDIA/Megatron-LM/pull/6401), [\#7336](https://github.com/NVIDIA/Megatron-LM/pull/7336)).  
* **Gated Delta Product (GDP):** adds GDP layers and model-building support for hybrid architectures ([\#6074](https://github.com/NVIDIA/Megatron-LM/pull/6074)).  
* **Wide residual streams:** adds streamwise residual connections, selective residual-stream recomputation, and integration with MTP and MIMO; currently requires PP=1 and CUDA graphs disabled ([\#6716](https://github.com/NVIDIA/Megatron-LM/pull/6716), [\#6805](https://github.com/NVIDIA/Megatron-LM/pull/6805), [\#6855](https://github.com/NVIDIA/Megatron-LM/pull/6855), [\#7440](https://github.com/NVIDIA/Megatron-LM/pull/7440)).  
* **MTP-only training:** adds a mode that freezes the base model and trains only Multi-Token Prediction heads ([\#7021](https://github.com/NVIDIA/Megatron-LM/pull/7021)).

#### Parallelism

* **Megatron-FSDP v2 integration:** adds the experimental Megatron Core adapter and Megatron Bridge, with context, expert, pipeline, and hybrid data-parallel sharding support; the adapter requires TP=1 and expert TP=1 ([\#5865](https://github.com/NVIDIA/Megatron-LM/pull/5865), [\#6450](https://github.com/NVIDIA/Megatron-LM/pull/6450), [\#6486](https://github.com/NVIDIA/Megatron-LM/pull/6486), [\#7220](https://github.com/NVIDIA/Megatron-LM/pull/7220), [\#6873](https://github.com/NVIDIA/Megatron-LM/pull/6873)).  
* **Megatron-FSDP v2 checkpointing:** adds PyTorch distributed checkpoint save/load for model and optimizer state; optimizer checkpointing currently excludes expert-parallel layouts ([\#6024](https://github.com/NVIDIA/Megatron-LM/pull/6024), [\#6215](https://github.com/NVIDIA/Megatron-LM/pull/6215)).  
* Megatron-FSDP v2 supports hybrid FSDP.   
  * The user can specify different ZeRO levels for inter-nvlink domains and intra-nvlink domains.   
  * The user can specify different configs for the dense section and the MoE section.   
* **Generalized Tensor Parallelism (GTP):** adds configurable batch/sequence partitioning, opt-in latent-projection sharding, and MTP support ([\#7748](https://github.com/NVIDIA/Megatron-LM/pull/7748), [\#6383](https://github.com/NVIDIA/Megatron-LM/pull/6383), [\#6242](https://github.com/NVIDIA/Megatron-LM/pull/6242)).

#### Performance and Memory

* **Determinism:** multi-release effort, please read [our blog post](https://developer.nvidia.com/blog/scale-bitwise-deterministic-pretraining-with-nvidia-megatron-core/) to learn more  
* **Megatron-FSDP v2 CUDA graphs:** adds full-iteration graph capture and replay for supported dense-model configurations ([\#7075](https://github.com/NVIDIA/Megatron-LM/pull/7075)).  
* **Fused GDN execution:** adds Transformer Engine fused GDN and opt-in preprocessing fusion for convolution, layout transforms, normalization, and gates ([\#7583](https://github.com/NVIDIA/Megatron-LM/pull/7583), [\#5532](https://github.com/NVIDIA/Megatron-LM/pull/5532)).  
* **MLA and MoE kernel optimizations:** adds fused MLA Q up-projection, RoPE, and MXFP8 quantization for supported SM100 configurations, vectorized MLA projection copies, and fused shared-expert/residual additions after MoE combine ([\#6213](https://github.com/NVIDIA/Megatron-LM/pull/6213), [\#7957](https://github.com/NVIDIA/Megatron-LM/pull/7957), [\#7943](https://github.com/NVIDIA/Megatron-LM/pull/7943)).  
* **Weighted checkpoint merging:** adds CPU-only weighted merging of compatible, same-layout torch\_dist checkpoints ([\#5114](https://github.com/NVIDIA/Megatron-LM/pull/5114)).  
* **Training observability:** adds optional OpenTelemetry traces and metrics for training, pipeline communication, gradient synchronization, and checkpointing ([\#6415](https://github.com/NVIDIA/Megatron-LM/pull/6415), [\#6438](https://github.com/NVIDIA/Megatron-LM/pull/6438)).

#### Precision and Optimizers

* **Megatron-FSDP v2 mixed precision:** enables MXFP8 compute and quantized parameter gathers through the adapter, alongside mixed-precision optimizer support ([\#7588](https://github.com/NVIDIA/Megatron-LM/pull/7588), [\#7114](https://github.com/NVIDIA/Megatron-LM/pull/7114), [\#6506](https://github.com/NVIDIA/Megatron-LM/pull/6506)).  
* **Selective FP32 execution:** adds FP32 handling for selected tensors and their optimizer/checkpoint lifecycle, plus an option to compute language-model output logits in FP32 ([\#5929](https://github.com/NVIDIA/Megatron-LM/pull/5929), [\#6252](https://github.com/NVIDIA/Megatron-LM/pull/6252)).  
* **Distributed Muon improvements:** adds layer-sharded Muon computation across TP and GTP-rematerialization ranks and an optional Triton SYRK path for Newton-Schulz iterations ([\#6683](https://github.com/NVIDIA/Megatron-LM/pull/6683), [\#6381](https://github.com/NVIDIA/Megatron-LM/pull/6381)).  
* **DeepSeek-V3 NVFP4 training:** adds NVFP4 support including FP4 quantization in active MTP layers ([\#7328](https://github.com/NVIDIA/Megatron-LM/pull/7328)).

#### Inference

* **GDN and GDP dynamic inference:** adds GDN serving and extends GDP inference with CUDA graphs, chunked prefill, prefix caching, and speculative decoding ([\#6499](https://github.com/NVIDIA/Megatron-LM/pull/6499), [\#6595](https://github.com/NVIDIA/Megatron-LM/pull/6595), [\#6711](https://github.com/NVIDIA/Megatron-LM/pull/6711), [\#6780](https://github.com/NVIDIA/Megatron-LM/pull/6780), [\#7245](https://github.com/NVIDIA/Megatron-LM/pull/7245)).  
* **Prefill-Decode Disaggregation:** adds NCCL and NIXL KV-cache transfer paths and SSM-state handoff primitives for prefill/decode disaggregation ([\#5861](https://github.com/NVIDIA/Megatron-LM/pull/5861), [\#6222](https://github.com/NVIDIA/Megatron-LM/pull/6222), [\#6584](https://github.com/NVIDIA/Megatron-LM/pull/6584)).  
* **Batch-Invariant MoE inference:** adds batch-invariant BF16/MXFP8 execution paths, including vLLM fused MoE and MXFP8 expert GEMMs ([\#6521](https://github.com/NVIDIA/Megatron-LM/pull/6521), [\#7302](https://github.com/NVIDIA/Megatron-LM/pull/7302), [\#7562](https://github.com/NVIDIA/Megatron-LM/pull/7562)).  
* **Token checkpointing and staging**: adds engine hooks to preprocess requests and stage completed token payloads ([\#7015](https://github.com/NVIDIA/Megatron-LM/pull/7015)), extends staged records with compact prompt IDs and media tensors for multimodal NRL training ([\#7557](https://github.com/NVIDIA/Megatron-LM/pull/7557)), and moves staging off the inference loop so slow checkpoint writes do not stall decoding ([\#8169](https://github.com/NVIDIA/Megatron-LM/pull/8169)).  
* **Expert Tensor Parallel**: [\#4743](https://github.com/NVIDIA/Megatron-LM/pull/4743) supports ETP in the inference engine.  
* **ShortcutMoE Inference CG**: [\#7772](https://github.com/NVIDIA/Megatron-LM/pull/7772) adds CG support for shortcut MoE, GDP, and wide residuals.  
* **Inference Service Features and Resiliency**: [\#7828](https://github.com/NVIDIA/Megatron-LM/pull/7828) makes seeded sampling independent of batching, request movement, and other requests’ RNG consumption, [\#6916](https://github.com/NVIDIA/Megatron-LM/pull/6916) aborts generation when HTTP clients disconnect, preventing orphaned requests from consuming all serving capacity, and large-scale / concurrency optimizations with frontend-coordinator-engine data transfer and replicas in[\#6497](https://github.com/NVIDIA/Megatron-LM/pull/6497) and [\#6223](https://github.com/NVIDIA/Megatron-LM/pull/6223).

#### Datasets

* **DP-balanced sequence packing:** adds variable-length dataset scheduling, THD-format batch fetching with padding masks, and training-loop integration to balance packed batches across data-parallel ranks ([\#5903](https://github.com/NVIDIA/Megatron-LM/pull/5903), [\#6625](https://github.com/NVIDIA/Megatron-LM/pull/6625), [\#6684](https://github.com/NVIDIA/Megatron-LM/pull/6684), [\#6742](https://github.com/NVIDIA/Megatron-LM/pull/6742)).

#### Multimodal

* **Multimodal (Vision) Inference**: [\#6260](https://github.com/NVIDIA/Megatron-LM/pull/6260) adds image modality support to the dynamic inference engine, [\#6809](https://github.com/NVIDIA/Megatron-LM/pull/6809) adds video inference support, vision-embedding caching, media-affinity routing, and media-safe prefix hashes; [\#7598](https://github.com/NVIDIA/Megatron-LM/pull/7598) adds multi-turn expanded-prefix stitching; [\#7839](https://github.com/NVIDIA/Megatron-LM/pull/7839) serves MIMO-trained VLM checkpoints.  
* **Heterogeneous MIMO extensions:** adds GTP, non-colocated context parallelism, configurable input-projection placement, and model-declared weight refit ([\#6570](https://github.com/NVIDIA/Megatron-LM/pull/6570), [\#7435](https://github.com/NVIDIA/Megatron-LM/pull/7435), [\#6843](https://github.com/NVIDIA/Megatron-LM/pull/6843), [\#7367](https://github.com/NVIDIA/Megatron-LM/pull/7367)).  
* **RoPE audio encoder:** adds dense and packed execution modes with NeMo checkpoint compatibility ([\#7066](https://github.com/NVIDIA/Megatron-LM/pull/7066)).

<details><summary>Changelog Details</summary>

- fix(ci): AUT-1080 retry live NCCL watchdog timeouts by @svcnemo-autobot :: PR: #6054
- Prevent coordinator crash if an engine disconnects by @tdene :: PR: #6025
- Make nightly sync checks advisory by @Phlip79 :: PR: #5713
- test(optimizer): MCORE-560 cover MoE gradient zero counts by @svcnemo-autobot :: PR: #6050
- RL: fix logging issues from failed rollouts by @tdene :: PR: #6061
- chore(ci): AUT-1135 pin GitHub Actions to commit SHAs by @svcnemo-autobot :: PR: #6100
- chore(deps): AUT-1080 pin merged NeMo Run cancellation fix by @svcnemo-autobot :: PR: #6110
- chore: AUT-1142 bump package versions by @svcnemo-autobot :: PR: #6120
- test(inference): mark prefix caching CUDA graph test flaky by @ko3n1g :: PR: #6131
- Make BertModel lm_head optional. by @bbuschkaemper :: PR: #5690
- ci: AUT-1141 skip MBridge dispatch for docs-only changes by @svcnemo-autobot :: PR: #6116
- Fix gradient counting for muon+expert biases by @philipcmonk :: PR: #6099
- test(inference): MCORE-561 trust FP8 metadata in DeepSeek checkpoints by @svcnemo-autobot :: PR: #6049
- Allow the MInf sync API to run serve() by @tdene :: PR: #6123
- ci: fix test notifications for weekly/release tests by @balasaajay :: PR: #6055
- Add disaggregated KV transfer backends by @nvcsathe :: PR: #5861
- Reorganize a few docs by @Phlip79 :: PR: #6109
- Perf: skip per-param copy_ dispatch in the MXFP8 param copy-back by @fanshiqing :: PR: #6094
- test: add async scheduling inference coverage by @lmcafee-nvidia :: PR: #6104
- test(ci): AUT-1173 set functional tests to ft_lau

document fetched 2026-10-30 02:33