# NVIDIA/Megatron-LM core_v0.18.0: NVIDIA Megatron Core 0.18.0 — retained record

> Retained github record from github:NVIDIA/Megatron-LM.

Canonical: https://highsignal.app/data/records/49d2de9b8f944935

[← sources](https://highsignal.app/data)


source: [github](https://highsignal.app/data/github) (github:NVIDIA/Megatron-LM)  published 2026-06-23 00:16 ingested 2026-06-28 12:34

[NVIDIA Corporation](https://highsignal.app/entities/NVDA)[original source ↗](https://github.com/NVIDIA/Megatron-LM/releases/tag/core_v0.18.0)

## extract

<details><summary>Changelog Details</summary>

- fix(ci): replace actions/setup-python with apt-get to avoid 429 rate limits by @ko3n1g :: PR: #4072
- ci: Fix package name for code-freeze workflow by @ko3n1g :: PR: #4077
- chore: bump `_code_freeze` workflow to `v0.86.0` by @ko3n1g :: PR: #4078
- Fix checkpoint inspector by @janEbert :: PR: #4079
- Update docs to conform to NVIDIA style guides by @megnvidia :: PR: #4068
- Miscellaneous inference fixes by @santhnm2 :: PR: #4030
- fix fine_grained_callables with fused rmsnorm residual by @CarlosGomes98 :: PR: #4026
- [Main][feat] Support overlapping A2A Combine backprop with wgrad GEMM by @Wohox :: PR: #3795
- Modify mfsdp default data-parallel-sharding-strategy by @wplf :: PR: #3691
- Fix fsdp_dtensor conversion for pretrained-only checkpoints by @DAISY-gh :: PR: #3912
- Guard NVshmem issues by @wdykas :: PR: #4093
- m-fsdp: wire use_precision_aware_optimizer from ddp_config to ParamAn… by @rapatel :: PR: #4024
- Megatron-FSDP: Add MXFP8 transpose helper buffer for Hybrid FSDP by @shjwudp :: PR: #3918
- feat(fsdp): use TE general_gemm for mixed-precision wgrad in FSDP path by @Victarry :: PR: #3822
- Megatron-FSDP: Fix insufficient double buffers during gradient reduce by @shjwudp :: PR: #4054
- Fix M-FSDP MXFP8 related BUGs by @shjwudp :: PR: #3991
- Megatron-FSDP: Make _pre_forward_param_unshard and _register_post_backward_hook formal by @shjwudp :: PR: #4029
- FIX: Use decoupled gradients for precision-aware M-FSDP grad norm by @XueSongTap :: PR: #3746
- Align chat completions endpoint with vLLM by @santhnm2 :: PR: #4063
- [Megatron-FSDP] Fix compatibility with frozen parameters and add unit tests by @shjwudp :: PR: #3287
- [M-FSDP] Refactor uneven dtensor to full tensor and add UT by @shjwudp :: PR: #3190
- Add agent instruction files by @Phlip79 :: PR: #4102
- Bump eopt version by @skyw :: PR: #4100
- Refactor emerging optimizer integration by @skyw :: PR: #4113
- Fix over provisioning of Mamba state memory when max_requests is set by @santhnm2 :: PR: #4114
- base strategy simplification by @dimapihtar :: PR: #4001
- add support for DCP and FSDP async save by @dimapihtar :: PR: #4027
- Add more emerging optimizers (#3907) by @skyw :: PR: #4119
- Fix FSDP checkpoint conversion and loading for Qwen3.5-VL by @DAISY-gh :: PR: #3936
- docs: update mcore optimizer docstrings to google style by @Akshat8510 :: PR: #2799
- Set tensor-parallel attributes irrespective of perform_initialization by @ilml :: PR: #4084
- docs: add developer-guide skill with CI/CD and failure navigation guidance by @ko3n1g :: PR: #4035
- chore: Move skills by @ko3n1g :: PR: #4136
- ci: Let Claude react to comment by @ko3n1g :: PR: #4135
- Nemotron3 Super GB200 release config by @maanug-nv :: PR: #4118
- Enable CUDA graph for ADAM optimizer by @vasunvidia :: PR: #3429
- Claude review should recommend testing by @Phlip79 :: PR: #4137
- cleanup: remove unused `scatter_gather_tensors_in_pipeline` argument by @Phlip79 :: PR: #4140
- fix: Remove fail-fast (-x) and guard distributed teardown against deadlock by @ko3n1g :: PR: #4139
- Claude: add respond-to-issue skill by @Phlip79 :: PR: #4141
- Fix muon getter backward compatability by @skyw :: PR: #4157
- Audit of user guide by @megnvidia :: PR: #4098
- Fix `RerunStateMachine` crash (`TypeError: 'NoneType' object is not subscriptable`) by not saving a checkpoint after a transient NaN / Inf by @yezhengmao1 :: PR: #3981
- Preserve type of decorated methods/classes by @nschank :: PR: #4062
- update muon test case to use new interface by @skyw :: PR: #4163
- [M-FSDP] Fix Tensor Parallel mode detection by @shjwudp :: PR: #3191
- fix: remove weights_only=False for multimodal example by @faradawn :: PR: #4104
- Cudagraphs: Fix sequence packing segfault more generally by @mathemakitten :: PR: #4162
- Make MTP work with materialize_only_last_token_logits by @santhnm2 :: PR: #4166
- Add unit test for Mamba EP inference (eager fallback with mixed CUDA graphs) by @santhnm2 :: PR: #4085
- update docs in respect to async changes by @dimapihtar :: PR: #4177
- update checkpointing docs in respect to async changes by @dimapihtar :: PR: #4208
- chore: improve build-and-test skill with trigger rules and dependency workflow by @ko3n1g :: PR: #4199
- Fix layerwise optimizer with `expt_dp_size=1` and contention with element-wise distributed optimizer by @skyw :: PR: #4138
- ci: add --cluster-a100/h100/gb200 args to trigger_internal_ci.py by @ko3n1g :: PR: #4195
- ci: Update golden values for nightly tests by @chtruong814 :: PR: #4215
- rename async_allgather to overlap_param_gather by @skyw :: PR: #4217
- Fix Slack sync for users with GitHub email privacy enabled by @Phlip79 :: PR: #4220
- Miscellaneous MTP inference fixes by @santhnm2 :: PR: #4191
- Move inference guards out of arguments.py by @mathemakitten :: PR: #4210
- Fix: enable fine-grained activation offloading for Mamba model. by @fanshiqing :: PR: #4173
- bump NVRx by @dimapihtar :: PR: #4178
- Update tokenizer args for Nemotron3 release config by @maanug-nv :: PR: #4239
- build: add dynamic git-versioning and drop rc0 pre-release tag by @ko3n1g :: PR: #4212
- Fix unnecessary permute padding for non-quantized MoE dispatch by @xiaoxi-wangfj :: PR: #4038
- Fix split state dict main by @kunlunl :: PR: #3676
- Add /split-pr Claude Code command for splitting PRs by CODEOWNERS by @Phlip79 :: PR: #4160
- Enable FP8 DPA for MXFP8 recipe by @vasunvidia :: PR: #4066
- Enable AG/RS overlap with explicit process group passing by @jeffnvidia :: PR: #3249
- Enable cpu_offloading with Full iteration CUDA graph by @vasunvidia :: PR: #3969
- Fix TransformerConfig validation for mixed dense/MoE upcycling by @rkteddy :: PR: #3647
- Remove cross-rank synchronization during checkpoint load & deprecate torch.distributed.checkpoint.state_dict_loader.load_state_dict by @asolergi-nv :: PR: #2864
- Fix incorrectly set decoupled_grad and DistOpt mechanics for MFSDP. by @cspades :: PR: #4133
- Refit Miscelaneous by @wdykas :: PR: #3973
- Add conditions_embeddings argument to TransformerBlock, TransformerLayer for DiT (diffusion transformer) by @huvunvidia :: PR: #4134
- Fix build_sequences_per_dataset output path arg usage by @DhineshPonnarasan :: PR: #4144
- ci: Flush pending CUDA work before the barrier in destroy_model_parallel by @chtruong814 :: PR: #4259
- Update oncall schedule by @Phlip79 :: PR: #4257
- docs(moe): Update MoE README by @sbhavani :: PR: #3664
- Revert "Add conditions_embeddings argument to TransformerBlock, Trans… by @ko3n1g :: PR: #4270
- reduce the number of shared expert streams by @yangbofun :: PR: #3752
- remove legacy Bert code by @dimapihtar :: PR: #4204
- [Main] Feat(moe): Gated delta net context parallel (CP) by @yuzhongw-nvidia :: PR: #2642
- remove t5 legacy code by @dimapihtar :: PR: #4203
- fix: handle list-typed process groups in ProcessGroupCollection.__repr__ by @cluster2600 :: PR: #3753
- Fix Context Parallelism documentation link by @liangxs :: PR: #4149
- [MLA] fix: Pad V when Q/V head dims differ for THD by @HollowMan6 :: PR: #3003
- fix(megatron-fsdp): build expt_device_mesh only for MoE models by @xuwchen :: PR: #3831
- Allow the evaluation batch size to differ from the training batch size by @michal2409 :: PR: #4014
- Add @NVIDIA/transformer review group to megatron/core/transformer/ by @Phlip79 :: PR: #4281
- Reset AG_pipeline bucket status after validation step. by @vasunvidia :: PR: #3155
- Enhance and fix NVTX for training by @yaox12 :: PR: #3642
- NVFP4 native weights for DDP by @WanZzzzzz :: PR: #4005
- Remove unnecessary arguments for layerwise distributed optimizer by @FDecaYed :: PR: #4272
- reuse grad buffer for layer-wise param allgather by @FDecaYed :: PR: #3751
- feat(ci): add strict review mode to Claude review workflow by @Victarry :: PR: #4197
- Fix stale approvals by @Phlip79 :: PR: #4280
- [MoE] Add a new score function to the router by @yaox12 :: PR: #3673
- [MoE] Improvement of shared expert overlap, support shared expert overlap for FlexDispatcher by @Victarry :: PR: #2207
- build: bump DeepEP to 34152ae by @ko3n1g :: PR: #4228
- ci: mark test_fused_indexer_loss_gradient_tp_consistency as flaky_in_dev by @ko3n1g :: PR: #4299
- Fix typo in PR4133. by @cspades :: PR: #4277
- ci: add retry loop to apt-get update to handle transient mirror sync failures by @ko3n1g :: PR: #4209
- fix: enforce correct pass thresholds for deterministic and approximate tests by @ko3n1g :: PR: #4238
- remove legacy biencoder and realm models by @dimapihtar :: PR: #4205
- ci: add configurable launcher support for functional tests (ft_launcher / torchrun) by @ko3n1g :: PR: #4298
- chore: document --target main for local Docker builds by @ko3n1g :: PR: #4307
- Extract args init to launch scripts by @maanug-nv :: PR: #4225
- [Main] Fix TE version check for retain_pinned_cpu_buffers in cpu offload by @BestJuly :: PR: #4267
- Fix documented shape by @janEbert :: PR: #3486
- ci: add sync-skills workflow, rename CLAUDE.md → AGENTS.md, move .claude/skills → skills/ by @ko3n1g :: PR: #4303
- Get `device` correctly when module returns a dict instead of individual tensor by @shifangx :: PR: #4265
- remove vision legacy code by @dimapihtar :: PR: #4202
- feat: long convergence resiliency for release tests by @ko3n1g :: PR: #4335
- ci(action): improve GitHub Actions output UX by @ko3n1g :: PR: #4337
- build: bump TransformerEngine to release_v2.14 by @ko3n1g :: PR: #4331
- feat: add create-issue skill by @ko3n1g :: PR: #4338
- M4 leftover for TE cuda graph by @shifangx :: PR: #3137
- fix: wait for async P2P send before deallocating output tensor by @ZhiyuLi-Nvidia :: PR: #4047
- ci(gb200): add 1-node mr-github functional test variants by @ko3n1g :: PR: #4334
- Fix potential coredump issue that occurs when saving a checkpoint by @ezioliao :: PR: #1871
- docs: bump versions1.json to 0.17.0 (latest) by @ko3n1g :: PR: #4360
- Port DeepSeek Sparse Attention to `MambaModel` by @janEbert :: PR: #3553
- Add tables and histogram for RL staleness by @tdene :: PR: #4097
- [docs] ci: use parent-relative json_url for version picker by @ko3n1g :: PR: #4367
- Fix bug with non-partial rollouts by @tdene :: PR: #3964
- Add QK layernorm support for dot-product attention in MambaModel by @Phlip79 :: PR: #4067
- Docs: improve docstrings and comments in example training loop by @DhineshPonnarasan :: PR: #4041
- feat(ckpt): add --async-ckpt-use-cpu-shm argument by @sbak5 :: PR: #4355
- cp: Fix UT timeout (#4310) by @chtruong814 :: PR: #4373
- Fix RL reward due to stop token by @tdene :: PR: #4096
- FA4 Inference by @wdykas :: PR: #4186
- Make param_index_map always use unpacked (full numel) offsets by @deepakn94 :: PR: #4328
- Add activation logging and tokens per expert logging by @Mellonta :: PR: #3842
- Fix RL to once again work with --skip-train by @tdene :: PR: #4249
- Fix Megatron initialization with extra_args_provider by @santhnm2 :: PR: #4327
- Rename MambaModel/MambaStack to HybridModel/HybridStack by @Phlip79 :: PR: #4099
- fix(ci): wrap uv install in retry block by @ko3n1g :: PR: #4387
- Call save_checkpoint_and_time() when saving checkpoint and compute elapsed duration for saving checkpoint before logging timer by @awsankur :: PR: #4263
- refactor(tests): move NCCL env vars from docker launcher to shell training script by @ko3n1g :: PR: #4390
- Remove packed_attention_mask unused parameter by @tdene :: PR: #3859
- Second batch of audit edits by @megnvidia :: PR: #4115
- revert: replace rampup batch size scheduler with custom step batch size schedules (#3779) by @ko3n1g :: PR: #4404
- Replace rampup batch size scheduler with custom step batch size schedules by @deepakn94 :: PR: #4411
- Megatron-FSDP: log mcore detection only after imports succeed by @wujingyue :: PR: #4400
- ci(gb200): re-enable tunable_overlap 1-node mr-github test by @ko3n1g :: PR: #4405
- Fix local docs building by @Phlip79 :: PR: #4416
- RL: Onload optimizer after logprobs computation by @tdene :: PR: #4235
- Add RL token throughput and packing metrics by @tdene :: PR: #3877
- ci: remove publish:merge_into_dev job by @ko3n1g :: PR: #4421
- docs: add data loading best practices for large-scale training by @sbhavani :: PR: #4236
- Fix: Auto enable manual registration and enhance the docummentation by @youngeunkwon0405 :: PR: #3295
- Fix nvtx_decorator to check _nvtx_enabled at call time by @minitu :: PR: #4184
- fix merges_file typo in megatron_hf_tokenizer by @chelseajohn :: PR: #4392
- Enable NullTokenizer for pretraining to reduce I/O access  by @asolergi-nv :: PR: #4057
- docs: Add SECURITY.md by @chtruong814 :: PR: #4431
- Mamba inference opt by @wdykas :: PR: #4414
- DDP refactoring: Extract parameter layout computation into optimizer classmethod by @deepakn94 :: PR: #3812
- Update PR template with explicit request for issue by @Phlip79 :: PR: #4409
- Misc inference fixes by @sidsingh-nvidia :: PR: #4397
- Rename Mamba to Hybrid outside megatron/core by @Phlip79 :: PR: #4159
- Include mtp layers in token per expert logging by @Mellonta :: PR: #4412
- fix: NVRx async compatibility and defer resiliency import by @sbak5 :: PR: #4420
- fix(checkpoint_inspector): allow empty --param-to-param-group-map-json by @DAISY-gh :: PR: #4403
- Add the YARN support for hybrid_model by @guihong-nv :: PR: #4244
- [training migration] Add container class for config dataclasses by @maanug-nv :: PR: #4227
- Inference: Fix broken functional tests on gitlab by @sidsingh-nvidia :: PR: #4454
- SafeUnpickler class for safe pickle usage by @dimapihtar :: PR: #4319
- get rid of weights_only=False by @dimapihtar :: PR: #4434
- Inference | Per-block MoE routing storage for prefix caching by @lmcafee-nvidia :: PR: #4301
- Add troubleshooting tip for 'access forbidden'  by @balasaajay :: PR: #4449
- Fix checkpoint loading with rerun state machine  by @YangFei1990 :: PR: #4448
- Add misc CUDA graph sugar to CudaGraphManager by @tdene :: PR: #4425
- Inference: Add the embedding and output layer in the full_iteration_inference cuda graph scope for hybrid models by @sidsingh-nvidia :: PR: #4440
- Important bugfixes in local CG implementation that were leading to loss curve gaps for latent MoE models by @jiemingz :: PR: #4433
- fix: Replace polynomial rolling hash with SHA-256 for prefix caching by @lmcafee-nvidia :: PR: #4158
- feat(ckpt): expose validate_access_integrity knob on dist-ckpt load by @asolergi-nv :: PR: #4422
- Fix multivalidation by @RPrenger :: PR: #3388
- Add missing knob for reduce_scatter_with_fp32_accumulation by @WanZzzzzz :: PR: #4410
- Enable CUDA graphs for MTP inference by @santhnm2 :: PR: #4260
- checkpoint integrity verification by @dimapihtar :: PR: #4305
- Fix cache gating by @wdykas :: PR: #4455
- [Main] Fix FusedAdam.use_decoupled_grad mis-set for Megatron-FSDP. by @cspades :: PR: #4427
- add permute fusion into hybrid ep by @Autumn1998 :: PR: #4089
- Add ColocatedBridgeCommunicator for heterogeneous TP/DP MIMO training (NMFW-17) by @yashaswikarnati :: PR: #4368
- Fix incorrect bias display in extra_repr of Column/RowParallelLinear by @HelloWorldBeginner :: PR: #4330
- Fix assertion logic in combined_1f1b_schedule_for_interleaved_pipelining by @joapolarbear :: PR: #4276
- ci: Fix event name reference in CI workflow condition for merge group by @balasaajay :: PR: #4462
- Add nightly sync workflow from main to dev by @Phlip79 :: PR: #4165
- fix: handle list-format quant_cfg from ModelOpt PR #1094 by @ChenhanYu :: PR: #4187
- ci: also add Run MBridge tests label in nightly sync workflow by @Phlip79 :: PR: #4499
- [training migration] Add serialization features to config container by @maanug-nv :: PR: #4309
- Fix conflict with inference graphs by @tdene :: PR: #4504
- Add tools/prepare_cache.py for offline GPT dataset cache preparation by @asolergi-nv :: PR: #4080
- [build] fix: move mamba-ssm and causal-conv1d to optional [ssm] extra by @ko3n1g :: PR: #4517
- mamba: avoid redundant HBM reloads in causal_conv1d_update shift loop by @wdykas :: PR: #4460
- Standardize misc graph interface by @tdene :: PR: #4485
- Fix inference graph override in RL flow by @tdene :: PR: #4323
- Unify and refactor Megatron-FSDP documentation. by @cspades :: PR: #4418
- Revert "ci: add base_sha to codecov/codecov-action upload step (#4445)" by @chtruong814 :: PR: #4526
- Skills for running unit tests and working with slurm by @yashaswikarnati :: PR: #4502
- Reorganize order of operations in inference context and text generation controller by @tdene :: PR: #2929
- ci: Update CI workflow conditions to include merge group handling by @balasaajay :: PR: #4532
- ci: add base_sha to codecov/codecov-action upload step by @chtruong814 :: PR: #4540
- Fix release tests: remove --global-batch-size conflicting with --step-batch-size-schedule by @deepakn94 :: PR: #4545
- docs: use @file-path notation for file references in skills by @ko3n1g :: PR: #4542
- Support YAML quant recipe in PTQ and remove first/last layer modifier code by @jenchen13 :: PR: #4503
- Avoid nsys profile crash with CUDA graphs by @tdene :: PR: #4541
- fix(ci): add retry with backoff to approve-test-queue bot by @ko3n1g :: PR: #4559
- New allgathervdispatcher for inference and simplify old dispatcher.  by @sidsingh-nvidia :: PR: #4258
- Fixes for modelopt examples and SFTTokenizer for transformers v5 by @jenchen13 :: PR: #4450
- Adding code for Flextron by @sheliang-nv :: PR: #4429
- Fix partial cudagraphs + HybridEP not properly triggering DDP hook by @jiemingz :: PR: #4500
- Ignore pytorch link anchors by @maanug-nv :: PR: #4582
- MoE dispatcher fixes: size NVLS dispatcher buffers from actual tensor sizes by @mathemakitten :: PR: #4576
- Finalize all builders in preprocess_data, not just the last key by @sayalinvidia :: PR: #4573
- refactor(skills): add when_to_use frontmatter, split ci-test-system, enforce skill workflow by @ko3n1g :: PR: #4574
- Make last_token_logits graphable by @tdene :: PR: #4552
- fix(ci): correct off-by-one in total_steps_evaluated formula by @ko3n1g :: PR: #4591
- Add fault injection support via nvidia_resiliency_ext. by @hexinw-nvidia :: PR: #4370
- Guard vocab reduce_scatter on TP > 1  by @mathemakitten :: PR: #4565
- Move inference context bookkeeping to CPU with ContextGPUView by @lmcafee-nvidia :: PR: #4306
- Enable InJob restart on failures. by @hexinw-nvidia :: PR: #4594
- Enable shared expert overlap with allgatherv in inference by @sidsingh-nvidia :: PR: #4570
- Add vLLM grouped gemm backend for MoE inference by @santhnm2 :: PR: #4566
- Move KD teacher loading to after Float16Module by @AAnoosheh :: PR: #4394
- ci: update gpt3_7b_tp4_pp1_memory_speed gb200 golden values by @ko3n1g :: PR: #4601
- Fix inference unit test by @maanug-nv :: PR: #4589
- Checkpoint conversion between GPT_model and Hybrid_model by @guihong-nv :: PR: #4482
- ci: add cadence input for test filtering in CI workflows by @balasaajay :: PR: #4561
- Fix `mtp_use_repeated_layer` behavior for GPT models by @rkarimimahab :: PR: #3965
- Handle SSM sharded tensor merge OOM with CPU fallback by @returnL :: PR: #4442
- FlashInfer sampling by @tdene :: PR: #2456
- Fix main2dev workflow by @Phlip79 :: PR: #4610
- Add logic to enable chunked MLP during training by @pengdurice :: PR: #3656
- Inference bug-fixes: Re-enable EP syncs for the legacy A2A dispatcher and re-simplify ep_sync accidentally reverted by #4306 by @sidsingh-nvidia :: PR: #4587
- Remove invalid `timeout` argument for dist.barrier by @zhaoyinglia :: PR: #4512
- Fix buffers in refit by @wdykas :: PR: #4580
- Named validation sets by @RPrenger :: PR: #4578
- Fix Hang in tests by @wdykas :: PR: #4575
- Single commit for main2dev nightly by @Phlip79 :: PR: #4614
- convert tokenizer args to config by @dimapihtar :: PR: #4406
- Siddharth/fix ep sync by @wdykas :: PR: #4607
- mmiranda working on another set of broken links by @megnvidia :: PR: #4534
- Fix gradient corruption with layerwise param all-gather overlap by @deepakn94 :: PR: #4609
- test: mark TestFu

## retained text (truncated)



```



<details><summary>Changelog Details</summary>

- fix(ci): replace actions/setup-python with apt-get to avoid 429 rate limits by @ko3n1g :: PR: #4072
- ci: Fix package name for code-freeze workflow by @ko3n1g :: PR: #4077
- chore: bump `_code_freeze` workflow to `v0.86.0` by @ko3n1g :: PR: #4078
- Fix checkpoint inspector by @janEbert :: PR: #4079
- Update docs to conform to NVIDIA style guides by @megnvidia :: PR: #4068
- Miscellaneous inference fixes by @santhnm2 :: PR: #4030
- fix fine_grained_callables with fused rmsnorm residual by @CarlosGomes98 :: PR: #4026
- [Main][feat] Support overlapping A2A Combine backprop with wgrad GEMM by @Wohox :: PR: #3795
- Modify mfsdp default data-parallel-sharding-strategy by @wplf :: PR: #3691
- Fix fsdp_dtensor conversion for pretrained-only checkpoints by @DAISY-gh :: PR: #3912
- Guard NVshmem issues by @wdykas :: PR: #4093
- m-fsdp: wire use_precision_aware_optimizer from ddp_config to ParamAn… by @rapatel :: PR: #4024
- Megatron-FSDP: Add MXFP8 transpose helper buffer for Hybrid FSDP by @shjwudp :: PR: #3918
- feat(fsdp): use TE general_gemm for mixed-precision wgrad in FSDP path by @Victarry :: PR: #3822
- Megatron-FSDP: Fix insufficient double buffers during gradient reduce by @shjwudp :: PR: #4054
- Fix M-FSDP MXFP8 related BUGs by @shjwudp :: PR: #3991
- Megatron-FSDP: Make _pre_forward_param_unshard and _register_post_backward_hook formal by @shjwudp :: PR: #4029
- FIX: Use decoupled gradients for precision-aware M-FSDP grad norm by @XueSongTap :: PR: #3746
- Align chat completions endpoint with vLLM by @santhnm2 :: PR: #4063
- [Megatron-FSDP] Fix compatibility with frozen parameters and add unit tests by @shjwudp :: PR: #3287
- [M-FSDP] Refactor uneven dtensor to full tensor and add UT by @shjwudp :: PR: #3190
- Add agent instruction files by @Phlip79 :: PR: #4102
- Bump eopt version by @skyw :: PR: #4100
- Refactor emerging optimizer integration by @skyw :: PR: #4113
- Fix over provisioning of Mamba state memory when max_requests is set by @santhnm2 :: PR: #4114
- base strategy simplification by @dimapihtar :: PR: #4001
- add support for DCP and FSDP async save by @dimapihtar :: PR: #4027
- Add more emerging optimizers (#3907) by @skyw :: PR: #4119
- Fix FSDP checkpoint conversion and loading for Qwen3.5-VL by @DAISY-gh :: PR: #3936
- docs: update mcore optimizer docstrings to google style by @Akshat8510 :: PR: #2799
- Set tensor-parallel attributes irrespective of perform_initialization by @ilml :: PR: #4084
- docs: add developer-guide skill with CI/CD and failure navigation guidance by @ko3n1g :: PR: #4035
- chore: Move skills by @ko3n1g :: PR: #4136
- ci: Let Claude react to comment by @ko3n1g :: PR: #4135
- Nemotron3 Super GB200 release config by @maanug-nv :: PR: #4118
- Enable CUDA graph for ADAM optimizer by @vasunvidia :: PR: #3429
- Claude review should recommend testing by @Phlip79 :: PR: #4137
- cleanup: remove unused `scatter_gather_tensors_in_pipeline` argument by @Phlip79 :: PR: #4140
- fix: Remove fail-fast (-x) and guard distributed teardown against deadlock by @ko3n1g :: PR: #4139
- Claude: add respond-to-issue skill by @Phlip79 :: PR: #4141
- Fix muon getter backward compatability by @skyw :: PR: #4157
- Audit of user guide by @megnvidia :: PR: #4098
- Fix `RerunStateMachine` crash (`TypeError: 'NoneType' object is not subscriptable`) by not saving a checkpoint after a transient NaN / Inf by @yezhengmao1 :: PR: #3981
- Preserve type of decorated methods/classes by @nschank :: PR: #4062
- update muon test case to use new interface by @skyw :: PR: #4163
- [M-FSDP] Fix Tensor Parallel mode detection by @shjwudp :: PR: #3191
- fix: remove weights_only=False for multimodal example by @faradawn :: PR: #4104
- Cudagraphs: Fix sequence packing segfault more generally by @mathemakitten :: PR: #4162
- Make MTP work with materialize_only_last_token_logits by @santhnm2 :: PR: #4166
- Add unit test for Mamba EP inference (eager fallback with mixed CUDA graphs) by @santhnm2 :: PR: #4085
- update docs in respect to async changes by @dimapihtar :: PR: #4177
- update checkpointing docs in respect to async changes by @dimapihtar :: PR: #4208
- chore: improve build-and-test skill with trigger rules and dependency workflow by @ko3n1g :: PR: #4199
- Fix layerwise optimizer with `expt_dp_size=1` and contention with element-wise distributed optimizer by @skyw :: PR: #4138
- ci: add --cluster-a100/h100/gb200 args to trigger_internal_ci.py by @ko3n1g :: PR: #4195
- ci: Update golden values for nightly tests by @chtruong814 :: PR: #4215
- rename async_allgather to overlap_param_gather by @skyw :: PR: #4217
- Fix Slack sync for users with GitHub email privacy enabled by @Phlip79 :: PR: #4220
- Miscellaneous MTP inference fixes by @santhnm2 :: PR: #4191
- Move inference guards out of arguments.py by @mathemakitten :: PR: #4210
- Fix: enable fine-grained activation offloading for Mamba model. by @fanshiqing :: PR: #4173
- bump NVRx by @dimapihtar :: PR: #4178
- Update tokenizer args for Nemotron3 release config by @maanug-nv :: PR: #4239
- build: add dynamic git-versioning and drop rc0 pre-release tag by @ko3n1g :: PR: #4212
- Fix unnecessary permute padding for non-quantized MoE dispatch by @xiaoxi-wangfj :: PR: #4038
- Fix split state dict main by @kunlunl :: PR: #3676
- Add /split-pr Claude Code command for splitting PRs by CODEOWNERS by @Phlip79 :: PR: #4160
- Enable FP8 DPA for MXFP8 recipe by @vasunvidia :: PR: #4066
- Enable AG/RS overlap with explicit process group passing by @jeffnvidia :: PR: #3249
- Enable cpu_offloading with Full iteration CUDA graph by @vasunvidia :: PR: #3969
- Fix TransformerConfig validation for mixed dense/MoE upcycling by @rkteddy :: PR: #3647
- Remove cross-rank synchronization during checkpoint load & deprecate torch.distributed.checkpoint.state_dict_loader.load_state_dict by @asolergi-nv :: PR: #2864
- Fix incorrectly set decoupled_grad and DistOpt mechanics for MFSDP. by @cspades :: PR: #4133
- Refit Miscelaneous by @wdykas :: PR: #3973
- Add conditions_embeddings argument to TransformerBlock, TransformerLayer for DiT (diffusion transformer) by @huvunvidia :: PR: #4134
- Fix build_sequences_per_dataset output path arg usage by @DhineshPonnarasan :: PR: #4144
- ci: Flush pending CUDA work before the barrier in destroy_model_parallel by @chtruong814 :: PR: #4259
- Update oncall schedule by @Phlip79 :: PR: #4257
- docs(moe): Update MoE README by @sbhavani :: PR: #3664
- Revert "Add conditions_embeddings argument to TransformerBlock, Trans… by @ko3n1g :: PR: #4270
- reduce the number of shared expert streams by @yangbofun :: PR: #3752
- remove legacy Bert code by @dimapihtar :: PR: #4204
- [Main] Feat(moe): Gated delta net context parallel (CP) by @yuzhongw-nvidia :: PR: #2642
- remove t5 legacy code by @dimapihtar :: PR: #4203
- fix: handle list-typed process groups in ProcessGroupCollection.__repr__ by @cluster2600 :: PR: #3753
- Fix Context Parallelism documentation link by @liangxs :: PR: #4149
- [MLA] fix: Pad V when Q/V head dims differ for THD by @HollowMan6 :: PR: #3003
- fix(megatron-fsdp): build expt_device_mesh only for MoE models by @xuwchen :: PR: #3831
- Allow the evaluation batch size to differ from the training batch size by @michal2409 :: PR: #4014
- Add @NVIDIA/transformer review group to megatron/core/transformer/ by @Phlip79 :: PR: #4281
- Reset AG_pipeline bucket status after validation step. by @vasunvidia :: PR: #3155
- Enhance and fix NVTX for training by @yaox12 :: PR: #3642
- NVFP4 native weights for DDP by @WanZzzzzz :: PR: #4005
- Remove unnecessary arguments for layerwise distributed optimizer by @FDecaYed :: PR: #4272
- reuse grad buffer for layer-wise param allgather by @FDecaYed :: PR: #3751
- feat(ci): add strict review mode to Claude review workflow by @Victarry :: PR: #4197
- Fix stale approvals by @Phlip79 :: PR: #4280
- [MoE] Add a new score function to the router by @yaox12 :: PR: #3673
- [MoE] Improvement of shared expert overlap, support shared expert overlap for FlexDispatcher by @Victarry :: PR: #2207
- build: bump DeepEP to 34152ae by @ko3n1g :: PR: #4228
- ci: mark test_fused_indexer_loss_gradient_tp_consistency as flaky_in_dev by @ko3n1g :: PR: #4299
- Fix typo in PR4133. by @cspades :: PR: #4277
- ci: add retry loop to apt-get update to handle transient mirror sync failures by @ko3n1g :: PR: #4209
- fix: enforce correct pass thresholds for deterministic and approximate tests by @ko3n1g :: PR: #4238
- remove legacy biencoder and realm models by @dimapihtar :: PR: #4205
- ci: add configurable launcher support for functional tests (ft_launcher / torchrun) by @ko3n1g :: PR: #4298
- chore: document --target main for local Docker builds by @ko3n1g :: PR: #4307
- Extract args init to launch scripts by @maanug-nv :: PR: #4225
- [Main] Fix TE version check for retain_pinned_cpu_buffers in cpu offload by @BestJuly :: PR: #4267
- Fix documented shape by @janEbert :: PR: #3486
- ci: add sync-skills workflow, rename CLAUDE.md → AGENTS.md, move .claude/skills → skills/ by @ko3n1g :: PR: #4303
- Get `device` correctly when module returns a dict instead of individual tensor by @shifangx :: PR: #4265
- remove vision legacy code by @dimapihtar :: PR: #4202
- feat: long convergence resiliency for release tests by @ko3n1g :: PR: #4335
- ci(action): improve GitHub Actions output UX by @ko3n1g :: PR: #4337
- build: bump TransformerEngine to release_v2.14 by @ko3n1g :: PR: #4331
- feat: add create-issue skill by @ko3n1g :: PR: #4338
- M4 leftover for TE cuda graph by @shifangx :: PR: #3137
- fix: wait for async P2P send before deallocating output tensor by @ZhiyuLi-Nvidia :: PR: #4047
- ci(gb200): add 1-node mr-github functional test variants by @ko3n1g :: PR: #4334
- Fix potential coredump issue that occurs when saving a checkpoint by @ezioliao :: PR: #1871
- docs: bump versions1.json to 0.17.0 (latest) by @ko3n1g :: PR: #4360
- Port DeepSeek Sparse Attention to `MambaModel` by @janEbert :: PR: #3553
- Add tables and histogram for RL staleness by @tdene :: PR: #4097
- [docs] ci: use parent-relative json_url for version picker by @ko3n1g :: PR: #4367
- Fix bug with non-partial rollouts by @tdene :: PR: #3964
- Add QK layernorm support for dot-product attention in MambaModel by @Phlip79 :: PR: #4067
- Docs: improve docstrings and comments in example training loop by @DhineshPonnarasan :: PR: #4041
- feat(ckpt): add --async-ckpt-use-cpu-shm argument by @sbak5 :: PR: #4355
- cp: Fix UT timeout (#4310) by @chtruong814 :: PR: #4373
- Fix RL reward due to stop token by @tdene :: PR: #4096
- FA4 Inference by @wdykas :: PR: #4186
- Make param_index_map always use unpacked (full numel) offsets by @deepakn94 :: PR: #4328
- Add activation logging and tokens per expert logging by @Mellonta :: PR: #3842
- Fix RL to once again work with --skip-train by @tdene :: PR: #4249
- Fix Megatron initialization with extra_args_provider by @santhnm2 :: PR: #4327
- Rename MambaModel/MambaStack to HybridModel/HybridStack by @Phlip79 :: PR: #4099
- fix(ci): wrap uv install in retry block by @ko3n1g :: PR: #4387
- Call save_checkpoint_and_time() when saving checkpoint and compute elapsed duration for saving checkpoint before logging timer by @awsankur :: PR: #4263
- refactor(tests): move NCCL env vars from docker launcher to shell training script by @ko3n1g :: PR: #4390
- Remove packed_attention_mask unused parameter by @tdene :: PR: #3859
- Second batch of audit edits by @megnvidia :: PR: #4115
- revert: replace rampup batch size scheduler with custom step batch size schedules (#3779) by @ko3n1g :: PR: #4404
- Replace rampup batch size scheduler with custom step batch size schedules by @deepakn94 :: PR: #4411
- Megatron-FSDP: log mcore detection only after imports succeed by @wujingyue :: PR: #4400
- ci(gb200): re-enable tunable_overlap 1-node mr-github test by @ko3n1g :: PR: #4405
- Fix local docs building by @Phlip79 :: PR: #4416
- RL: Onload optimizer after logprobs computation by @tdene :: PR: #
```



document fetched 2026-06-29 10:23
