# huggingface/transformers v5.23.0: Release v5.23.0 — retained record

> Retained github record from github:huggingface/transformers.

Canonical: https://highsignal.app/data/records/be901d46ef245249

[← sources](https://highsignal.app/data)


source: [github](https://highsignal.app/data/github) (github:huggingface/transformers)  published 2026-11-13 14:10 ingested 2026-11-14 02:34

[original source ↗](https://github.com/huggingface/transformers/releases/tag/v5.23.0)

## extract

# Release v5.23.0


## New Model additions

### Inkling2

<img width="1200" height="675" alt="image" src="https://github.com/user-attachments/assets/9115e142-1be3-4f79-a51d-ea0e5c25b152" />

Inkling 2 Small is a multimodal model from Thinking Machines Lab that accepts text and image inputs and generates text outputs, and it is intended for use in English and other languages and across multiple coding languages. Its language backbone has 48 mixture-of-experts decoder layers, each routing tokens to 8 of 384 experts, for 594B total and 17B active parameters. It keeps the Inkling decoder (learned relative attention, short causal convolutions, mixture-of-experts layers) and adds block attention residuals, gated attention with a learned sink, and dynamic key/value convolutions. Images are encoded by a Perception Encoder-based vision transformer and projected into the language model embeddings, one embedding per 40x40 pixel patch.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/inkling2) | [Blog Post](https://thinkingmachines.ai/news/inkling-2-small/)
* Add inkling2 (#50682) by @guarin in [#50682](https://github.com/huggingface/transformers/pull/50682)

### JEV

JEV is a decision head that scores single-token labels with a linear layer at the last token of the prompt, soft-capped like the logits of a language model head. `JevForDecision` is the model of the checkpoint and its head, and it reads the head files shipped next to the weights (`head.safetensors` and `judge_config.json` for JEV). The same head is also used by GEV and reads the 255 answer codes of pplx-decider and d3, and it can be run through the experimental `decision` pipeline or directly with `decide`.

**Note:** JEV and the decision pipeline are currently experimental. Their APIs, configuration, and behavior may change as development continues. They should be used with caution in production environments.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/jev)
* Add Jev + decision core (#50054) by @ArthurZucker in [#50054](https://github.com/huggingface/transformers/pull/50054)

### LFM3

<img width="732" height="815" alt="image" src="https://github.com/user-attachments/assets/5a67fe26-2cee-4bbf-bc51-65e8f843b756" />

LFM3 is the third generation of Liquid Foundation Models from Liquid AI. Like LFM2, it is a hybrid architecture that combines gated short convolutions with grouped-query attention, with the attention layers split between full-attention and sliding-window layers. The attention layers apply RMSNorm to the query and key heads and gate their output with a per-head sigmoid. Every layer applies a sandwich norm to the operator and feed-forward outputs before the residual addition.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/lfm3)
* [Model] LFM3 Family: LFM3-Text (#50322) by @paulpak58 in [#50322](https://github.com/huggingface/transformers/pull/50322)

### Dots 3 Note Preview

Dots 3 Note Preview is a mixture-of-experts causal language model that accepts native text, image, video, and audio inputs. It uses a shared vision encoder for images and videos and a Whisper-style audio encoder, and both encoders project their outputs into the language model's hidden space before autoregressive text generation. BF16 and FP8 inference are supported.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/dots3_note)
* Add dots3-note Preview model support (#47844) by @miraclezqc in [#47844](https://github.com/huggingface/transformers/pull/47844)

### LFM3-VL

LFM3-VL is a vision-language model family from Liquid AI, built on the LFM3 language model. It takes images and videos at their native resolution together with text. It combines a hybrid vision encoder (a linear patchifier, gated depthwise 2D convolution layers, and bidirectional grouped-query attention layers with 2D rotary embeddings) with a projector and an LFM3 language model. The projector merges each 2x2 block of patch features into one language token through a layer norm and a two-layer MLP.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/lfm3_vl)
* [Model] LFM3 Family: LFM3-VL (#50392) by @paulpak58 in [#50392](https://github.com/huggingface/transformers/pull/50392)

### IQuestCoder

IQuest-Coder-V1 is a family of code language models developed by IQuest Lab, available in 7B, 14B and 40B sizes with base, instruct and thinking variants. Its architecture is the same as Llama, so the Transformers auto classes load it with LlamaModel and LlamaForCausalLM, and only IQuestCoderConfig is specific to IQuestCoder. The looped `IQuest-Coder-V1-*-Loop-*` checkpoints use a different architecture and aren't supported.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/iquestcoder)
* Add IQuestCoder (#50579) by @hmellor in [#50579](https://github.com/huggingface/transformers/pull/50579)

### InternLM3

InternLM3 is an 8B instruction-tuned language model developed by Shanghai AI Laboratory (checkpoint: internlm/internlm3-8b-instruct). Its architecture is the same as Llama, so the auto classes load it with the Llama implementation (LlamaModel and LlamaForCausalLM), and only InternLM3Config is specific to InternLM3. The config names the attention biases `qkv_bias` and the output projection and MLP biases `bias`. InternLM3Config loads these as `attention_bias` and `mlp_bias` and requires them to be equal, as they are in the released checkpoints.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/internlm3)
* Add InternLM3 (#50578) by @hmellor in [#50578](https://github.com/huggingface/transformers/pull/50578)



## Breaking changes

* 🚨 Remove the `ep_router` expert-parallel style (#49615) by @IlyasMoutawwakil
* 🚨 Run MXFP4 GPT-OSS experts through `ep_dispatch_experts` (#49613) by @IlyasMoutawwakil
* 🚨 `deepgemm_megamoe` EP implementation for DeepGEMM Mega MoE (#49606) by @IlyasMoutawwakil
* 🚨 Slim down `[audio]` to torchaudio + soundfile + soxr (drop librosa and phonemizer) (#50610) by @eustlb
* 🚨 Deprecate Wav2Vec2ProcessorWithLM and pyctcdecode/kenlm LM decoding (#50408) by @eustlb
* 🚨 Exporters: `ExportedModel`, Hub save/load, one backend registry (#50029) by @IlyasMoutawwakil
* 🚨 Move processing defaults to class (#49089) by @guarin
* :rotating_light: [`kernels`] Bump version (#50307) by @vasqu
* :rotating_light: Finally drop rope deltas from ModelOutput! (#50153) by @zucchini-nlp
* 🚨 Safe-ize pi0 graphs (#50160) by @molbap
* 🚨 [serve][breaking] update attn for mac users  (#49730) by @SunMarc
* 🚨 build and expand once the tp/ep_plan/fsdp_plan (#49527) by @3outeille


## Exporters: export, share and run any model on its backend (#50029, #50032, #50309, #50311)

https://github.com/user-attachments/assets/2b17b2ab-b2b8-4e34-a983-4a7177330e4e

Every export now returns an `ExportedModel`. You call it like the original model (same kwargs, same output class), push it to the Hub, and load it back on the backend's own runtime. The same API covers Dynamo, ONNX Runtime, OpenVINO and ExecuTorch.

```python
from transformers import AutoModel, AutoTokenizer
from transformers.exporters import ExportedModel, OnnxConfig, OnnxExporter

model = AutoModel.from_pretrained("google/embeddinggemma-2")
tokenizer = AutoTokenizer.from_pretrained("google/embeddinggemma-2")
inputs = tokenizer(["Exporters are here!"], return_tensors="pt")

exported = OnnxExporter().export(model, inputs, OnnxConfig(dynamic=True, ort_fusions="cuda"))
exported.push_to_hub("my-org/embeddinggemma-2-onnx")

exported_model = ExportedModel.from_pretrained("my-org/embeddinggemma-2-onnx", device="cuda")
outputs = exported_model(**inputs)  # same output class as model(**inputs)
```

- **Fused for the runtime**: `OnnxConfig(ort_fusions="cpu" | "cuda" | "webgpu" | "dml")` rewrites the graph into ONNX Runtime's fused transformer ops at export time: attention, `GroupQueryAttention` over the KV cache, rotary, norms and `MoE`. OpenVINO exports now hit OpenVINO's fused CPU kernels for attention, RoPE, MoE experts and compressed int8/int4 weights, which makes decode 2.4–3.9× faster than before.
- **Faster than `torch.compile`**: on EmbeddingGemma2 (8 × 128 tokens), ONNX Runtime with the TensorRT EP is 7.6× faster than eager PyTorch and 1.4× faster than `torch.compile` on an A100. OpenVINO is 4.1× faster than eager and 2.2× faster than `torch.compile` on a Xeon Platinum 8570 (4.9× and 2.6× with int8 weights).
- **Shareable**: an export saves and loads like a model, optionally in a `subfolder` of the model repo. Its `export.json` records the config, input shapes, package versions and I/O names. Quantized exports (PT2E, NNCF, ONNX Runtime) save and run the same way.

🚨 `export()` returns an `ExportedModel`, and an ONNX program is an `onnx_ir.Model`. `OnnxConfig`'s and `OpenVINOConfig`'s `output_path` options are gone in favour of `save_pretrained`. Custom backends register with `register_backend`.

📚 Docs: https://huggingface.co/docs/transformers/main/en/exporters


## Vision

This release fixes a range of vision-model bugs: classification pipelines now respect `problem_type`, and several models get corrected behavior (Step3.7 RoPE axis order, MiniMax-M3-VL packed attention isolation, Qwen VL positions with reused caches, PaddleOCR-VL interpolation alignment, Kosmos2 SDPA masking, and Emu3 float image handling). Chunked prefill now keeps image inputs on the


* fix(pipelines): respect problem_type in image and video classification pipelines (#50582) by @surendra902 in [#50582]
* Fix Step3.7 vision RoPE spatial W/H axis order (#50547) by @caohch-1 in [#50547]
* Raise ValueError for out-of-range float images in convert_to_rgb (#50487) by @n24q02m in [#50487]
* CI: install OpenVINO and NNCF in the daily GPU image (#50422) by @IlyasMoutawwakil in [#50422]
* Fix MiniMax-M3-VL packed vision attention isolation (#50346) by @taking-lying-flat in [#50346]
* Fix Qwen VL positions when a text cache is reused with a new image (#50427) by @Arnav66692 in [#50427]
* [Docker] Update Nvidia CI and Flash Attn CI images to torch 2.15 / cu132 (#50467) by @ydshieh in [#50467]
* Fix PaddleOCR-VL vision position interpolation alignment (#50384) by @scyyh11 in [#50384]
* Fix Emu3 image processor crash on float numpy images (#50226) by @simpleqt in [#50226]
* Build Idefics-style vision position ids with torch.where (#49917) by @shoumikhin in [#49917]
* Keep image inputs on the first chunk of a chunked prefill (#49753) by @digit50 in [#49753]
* Docs: note in the v5 migration guide that hidden_states[-1] of vision models is pre-norm (#50303) by @ravediamond in [#50303]
* Remove the GitHub API cache-bust from the AMD docker images (#49351) by @tarekziade in [#49351]
* [docs] Vision backend differences (#50131) by @stevhliu in [#50131]
* [Kosmos2] Fix SDPA applying a causal mask in the image-to-text projection (#50118) by @tarekziade in [#50118]


## Parallelization

This release improves distributed training and inference. Expert parallelism gains a selectable `ep_implementation` (all-reduce for CUDA-graph-friendly inference, all-to-all for training). The Trainer now reduces FSDP/HSDP gradients only on the last micro-batch, and the token-count normalization is fixed to use true division. Tensor-parallel fixes cover attention sinks (including Beam attention) and tied-weight comparisons on TPU


* Expert-parallel implementations for `ep_dispatch_experts`: all-reduce at inference, all-to-all in training (#49429) by @IlyasMoutawwakil in [#49429]
* [FSDP tests] Fork CPU ranks from a forkserver instead of spawning them (#50444) by @ydshieh in [#50444]
* [distributed - TPU] Fix tied-weight comparisons on TPUs when using TP (#50406) by @tengomucho in [#50406]
* [`distributed`] Trainer: reduce FSDP/HSDP gradients on the last micro-batch only (#50286) by @3outeille in [#50286]
* Update FSDP non-rank-0 missing keys test after #45649 (#50260) by @SunMarc in [#50260]
* [`Beam`] Fix TP on attn (#50302) by @vasqu in [#50302]
* Fix/trainer parallel token count 50212 (#50214) by @BlazeSR10hawk in [#50214]
* Reuse the TP/EP test ranks across tests instead of spawning them per test (#49986) by @3outeille in [#49986]
* handle local sink in TP (#49567) by @3outeille in [#49567]


## Cache

This release fixes several cache correctness issues: static-cache prefill attention in Gemma4's KV-shared layers, Qwen VL positions when a cache is reused with a new image or video, and DeepSeek-V4 compression-state reordering during beam search. It also improves cache sharing for Mistral Large 4 and adds OpenVINO export fusion for the KV cache and attention. Smaller changes cover CI and test fixes, including vLLM CI memory


* Gemma4: fix static cache attending to empty cache slots in KV-shared layers at prefill (#50528) by @ydshieh in [#50528]
* vLLM CI: fix install and KV-cache memory, add failure report job (#50436) by @ydshieh in [#50436]
* Fix DeepSeek-V4 compression cache reordering for beam search (#50348) by @taking-lying-flat in [#50348]
* OpenVINO exporter: let OpenVINO fuse attention, RoPE, the KV cache and compressed matmuls (#50311) by @IlyasMoutawwakil in [#50311]
* [tests] Pin the chat-template date in Llama's FA static-cache test (#50296) by @tarekziade in [#50296]
* Fix a cache test (#50320) by @Cyrilvallez in [#50320]
* Improve Mistral Large 4 cache sharing (#50305) by @Cyrilvallez in [#50305]


## Kernels

This release fixes several hub kernel issues: the MPS RMSNorm kernel repo id was corrected, the MegaBlocksMoeMLP mapping was restored for MXFP4 GPT-OSS, and a failed flash-attention kernel load is now retried instead of leaving models unregistered. It also raises the minimum `kernels` version (fixing ROCm issues and a multi-GPU bug), bumps mamba-ssm kernels to v4


* Fix the MPS RMSNorm hub kernel repo id (#50714) by @tarekziade in [#50714]
* [`GptOss`] Fix the kernels autodispatch test (#50630) by @vasqu in [#50630]
* [`kernels`] Bump min version (#50509) by @vasqu in [#50509]
* Retry the attention kernel load after a failed `lazy_import_flash_attention` (#50391) by @albertvillanova in [#50391]
* Test the dispatched FA3 kernel per device (#50304) by @Abdennacer-Badaoui in [#50304]
* upgrade version for mamba-ssm and fla kernel (#50294) by @kaixuanliu in [#50294]
* Restore the `MegaBlocksMoeMLP` hub kernel mapping for MXFP4 GPT-OSS (#50150) by @jiqing-feng in [#50150]


## Attention

This release fixes a range of attention bugs: vision-attention isolation in MiniMax-M3-VL, SDPA decode under `torch.compile`, eager NaNs in Qwen4-Exp with left padding, an unintended causal mask in Kosmos-2's SDPA projection, and default full-attention mask selection that wrongly picked linear layers. It also retries failed attention kernel loads, excludes padding from the repetition penalty, and lets the O


* [`SDPA`] Fix decode under compile (#50388) by @vasqu in [#50388]
* Exclude attention-masked padding from repetition penalty (#50149) by @BlazeSR10hawk in [#50149]
* Fix Qwen4-Exp eager attention NaNs with left padding (#49923) by @taking-lying-flat in [#49923]
* [Masking] Skip linear layers for the default full-attention mask (#50059) by @paulpak58 in [#50059]


## Generation

This release fixes several generation bugs: padded generation in RecurrentGemma, batched decoding in Falcon-H1, beam search cache reordering in DeepSeek-V4, and batched metadata and beam scores in `TextGenerationPipeline`. It also lets an unchanged `GenerationConfig` be saved without strict-validation errors, avoids copying the KV cache when decoding with grouped-query attention (GQA), and fixes an `Attribut


* Fix RecurrentGemma padded generation by declaring its `layer_types` (#50577) by @ydshieh in [#50577]
* Fix Falcon-H1 gated RMSNorm shape during batched decoding (#50507) by @taking-lying-flat in [#50507]
* Allow saving an unchanged generation config (#49914) by @benthecarman in [#49914]
* Fold GQA query heads onto their KV head when decoding instead of repeating K/V (#49743) by @Yooniel in [#49743]
* fix(pipelines): preserve batched generation metadata (#50210) by @BlazeSR10hawk in [#50210]
* Fix AttributeError on LinearAttentionLayer in export generate test (#50140) by @ydshieh in [#50140]


## Quantization

This release fixes several quantization bugs: FP8 `modules_to_not_convert` entries are now renamed like weight names, so modules such as `f_a_proj` and `f_b_proj` are no longer wrongly quantized. MXFP4 experts now dequantize to the requested dtype instead of always using bfloat16. The `MegaBlocksMoeMLP` hub kernel mapping is restored for MXFP4 GPT-OSS, which fix


* fix some error in quantization xpu ci test (#49910) by @sywangyi in [#49910]
* Rename FP8 `modules_to_not_convert` entries like weight names (#50354) by @albertvillanova in [#50354]
* [docs] Common quantization tasks (#50095) by @stevhliu in [#50095]
* Dequantize MXFP4 experts to the requested dtype (#50256) by @qgallouedec in [#50256]
* [docs] Quantization (#50093) by @stevhliu in [#50093]


## Audio

This release makes audio handling more correct and robust. `AudioClassificationPipeline` now respects the model's problem type (raw outputs for regression, sigmoid for multi-label, softmax otherwise), and VibeVoice and VibeVoice-ASR avoid GPU syncs when removing audio padding and gain a `sample` argument to disable VAE noise for deterministic outputs. Chat templates can also take audio decoded from video via `{"type": "aud


* Tests: skip audio model tests when torchaudio or librosa is missing (#50423) by @IlyasMoutawwakil in [#50423]
* fix(pipelines): respect audio classification problem type (#50329) by @BlazeSR10hawk in [#50329]
* [VibeVoice] Avoid GPU syncs when removing audio padding (#50417) by @hmellor in [#50417]
* [VibeVoice] Add `sample` argument to disable VAE noise in audio features (#50179) by @hmellor in [#50179]
* Audio from video in jinja  (#49382) by @zucchini-nlp in [#49382]


## Bugfixes and improvements

* CI pass, CI no pass? (#50690) by @zucchini-nlp in [#50690]
* [`Gemma4`] Fix FA slicing (#50681) by @vasqu in [#50681]
* Restore the root-level PEFT adapter when resuming from a checkpoint w… (#50368) by @ziywang50 in [#50368]
* Granite for Docling (#49012) by @nassarofficial in [#49012]
* Move the remaining router-masked MoE plans to `ep_dispatch_experts` (#49607) by @IlyasMoutawwakil in [#49607]
* fix(tokenizers): stop treating a missing transformers_version as Mistral (#49733) by @cdcailf in [#49733]
* Load the released A.X-K1 checkpoints without unexpected or missing keys (#50644) by @hmellor in [#50644]
* Fix get_json_schema truncating multi-line argument descriptions (#50606) by @YusefSyed in [#50606]
* Make distributed Trainer tests actually resume from checkpoints (#49723) by @Titus-von-Koeller in [#49723]
* Fix Mask2Former _init_weights to match Meta's original initialization (#50596) by @KMaruthi2002 in [#50596]
* Fix Inkling router precision in BF16 and autocast (#50621) by @taking-lying-flat in [#50621]
* QA:  Make `make run-mic` easier to read and to target (#50652) by @tarekziade in [#50652]
* Fix `HfArgumentParser` on Python 3.15 (#50651) by @qgallouedec in [#50651]
* Avoid mutating input examples in DataCollatorForMultipleChoice (#50611) by @LKL-ZREO in [#50611]
* Fix XVector outputs of the wav2vec2 family when the TDNN layers are offloaded (#50646) by @ydshieh in [#50646]
* [`Offloading`] Fix tests for some models (#50631) by @vasqu in [#50631]
* [`FA`] Fix FA tests for unique models (#50632) by @vasqu in [#50632]
* Exporters: run exported models on their backend's runtime (`ModelRunner`) (#50032) by @IlyasMoutawwakil in [#50032]
* Exporters: ONNX Runtime fusio

## retained text (truncated)



```
# Release v5.23.0


## New Model additions

### Inkling2

<img width="1200" height="675" alt="image" src="https://github.com/user-attachments/assets/9115e142-1be3-4f79-a51d-ea0e5c25b152" />

Inkling 2 Small is a multimodal model from Thinking Machines Lab that accepts text and image inputs and generates text outputs, and it is intended for use in English and other languages and across multiple coding languages. Its language backbone has 48 mixture-of-experts decoder layers, each routing tokens to 8 of 384 experts, for 594B total and 17B active parameters. It keeps the Inkling decoder (learned relative attention, short causal convolutions, mixture-of-experts layers) and adds block attention residuals, gated attention with a learned sink, and dynamic key/value convolutions. Images are encoded by a Perception Encoder-based vision transformer and projected into the language model embeddings, one embedding per 40x40 pixel patch.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/inkling2) | [Blog Post](https://thinkingmachines.ai/news/inkling-2-small/)
* Add inkling2 (#50682) by @guarin in [#50682](https://github.com/huggingface/transformers/pull/50682)

### JEV

JEV is a decision head that scores single-token labels with a linear layer at the last token of the prompt, soft-capped like the logits of a language model head. `JevForDecision` is the model of the checkpoint and its head, and it reads the head files shipped next to the weights (`head.safetensors` and `judge_config.json` for JEV). The same head is also used by GEV and reads the 255 answer codes of pplx-decider and d3, and it can be run through the experimental `decision` pipeline or directly with `decide`.

**Note:** JEV and the decision pipeline are currently experimental. Their APIs, configuration, and behavior may change as development continues. They should be used with caution in production environments.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/jev)
* Add Jev + decision core (#50054) by @ArthurZucker in [#50054](https://github.com/huggingface/transformers/pull/50054)

### LFM3

<img width="732" height="815" alt="image" src="https://github.com/user-attachments/assets/5a67fe26-2cee-4bbf-bc51-65e8f843b756" />

LFM3 is the third generation of Liquid Foundation Models from Liquid AI. Like LFM2, it is a hybrid architecture that combines gated short convolutions with grouped-query attention, with the attention layers split between full-attention and sliding-window layers. The attention layers apply RMSNorm to the query and key heads and gate their output with a per-head sigmoid. Every layer applies a sandwich norm to the operator and feed-forward outputs before the residual addition.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/lfm3)
* [Model] LFM3 Family: LFM3-Text (#50322) by @paulpak58 in [#50322](https://github.com/huggingface/transformers/pull/50322)

### Dots 3 Note Preview

Dots 3 Note Preview is a mixture-of-experts causal language model that accepts native text, image, video, and audio inputs. It uses a shared vision encoder for images and videos and a Whisper-style audio encoder, and both encoders project their outputs into the language model's hidden space before autoregressive text generation. BF16 and FP8 inference are supported.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/dots3_note)
* Add dots3-note Preview model support (#47844) by @miraclezqc in [#47844](https://github.com/huggingface/transformers/pull/47844)

### LFM3-VL

LFM3-VL is a vision-language model family from Liquid AI, built on the LFM3 language model. It takes images and videos at their native resolution together with text. It combines a hybrid vision encoder (a linear patchifier, gated depthwise 2D convolution layers, and bidirectional grouped-query attention layers with 2D rotary embeddings) with a projector and an LFM3 language model. The projector merges each 2x2 block of patch features into one language token through a layer norm and a two-layer MLP.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/lfm3_vl)
* [Model] LFM3 Family: LFM3-VL (#50392) by @paulpak58 in [#50392](https://github.com/huggingface/transformers/pull/50392)

### IQuestCoder

IQuest-Coder-V1 is a family of code language models developed by IQuest Lab, available in 7B, 14B and 40B sizes with base, instruct and thinking variants. Its architecture is the same as Llama, so the Transformers auto classes load it with LlamaModel and LlamaForCausalLM, and only IQuestCoderConfig is specific to IQuestCoder. The looped `IQuest-Coder-V1-*-Loop-*` checkpoints use a different architecture and aren't supported.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/iquestcoder)
* Add IQuestCoder (#50579) by @hmellor in [#50579](https://github.com/huggingface/transformers/pull/50579)

### InternLM3

InternLM3 is an 8B instruction-tuned language model developed by Shanghai AI Laboratory (checkpoint: internlm/internlm3-8b-instruct). Its architecture is the same as Llama, so the auto classes load it with the Llama implementation (LlamaModel and LlamaForCausalLM), and only InternLM3Config is specific to InternLM3. The config names the attention biases `qkv_bias` and the output projection and MLP biases `bias`. InternLM3Config loads these as `attention_bias` and `mlp_bias` and requires them to be equal, as they are in the released checkpoints.

**Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/internlm3)
* Add InternLM3 (#50578) by @hmellor in [#50578](https://github.com/huggingface/transformers/pull/50578)



## Breaking changes

* 🚨 Remove the `ep_router` expert-parallel style (#49615) by @IlyasMoutawwakil
* 🚨 Run MXFP4 GPT-OSS experts through `ep_dispatch_experts` (#49613) by @IlyasMoutawwakil
* 🚨 `deepgemm_megamoe` EP implementation for DeepGEMM Mega MoE (#49606) by @IlyasMoutawwakil
* 🚨 Slim down `[audio]` to torchaudio + soundfile + soxr (drop librosa and phonemizer) (#50610) by @eustlb
* 🚨 Deprecate Wav2Vec2ProcessorWithLM and pyctcdecode/kenlm LM decoding (#50408) by @eustlb
* 🚨 Exporters: `ExportedModel`, Hub save/load, one backend registry (#50029) by @IlyasMoutawwakil
* 🚨 Move processing defaults to class (#49089) by @guarin
* :rotating_light: [`kernels`] Bump version (#50307) by @vasqu
* :rotating_light: Finally drop rope deltas from ModelOutput! (#50153) by @zucchini-nlp
* 🚨 Safe-ize pi0 graphs (#50160) by @molbap
* 🚨 [serve][breaking] update attn for mac users  (#49730) by @SunMarc
* 🚨 build and expand once the tp/ep_plan/fsdp_plan (#49527) by @3outeille


## Exporters: export, share and run any model on its backend (#50029, #50032, #50309, #50311)

https://github.com/user-attachments/assets/2b17b2ab-b2b8-4e34-a983-4a7177330e4e

Every export now returns an `ExportedModel`. You call it like the original model (same kwargs, same output class), push it to the Hub, and load it back on the backend's own runtime. The same API covers Dynamo, ONNX Runtime, OpenVINO and ExecuTorch.

```python
from transformers import AutoModel, AutoTokenizer
from transformers.exporters import ExportedModel, OnnxConfig, OnnxExporter

model = AutoModel.from_pretrained("google/embeddinggemma-2")
tokenizer = AutoTokenizer.from_pretrained("google/embeddinggemma-2")
inputs = tokenizer(["Exporters are here!"], return_tensors="pt")

exported = OnnxExporter().export(model, inputs, OnnxConfig(dynamic=True, ort_fusions="cuda"))
exported.push_to_hub("my-org/embeddinggemma-2-onnx")

exported_model = ExportedModel.from_pretrained("my-org/embeddinggemma-2-onnx", device="cuda")
outputs = exported_model(**inputs)  # same output class as model(**inputs)
```

- **Fused for the runtime**: `OnnxConfig(ort_fusions="cpu" | "cuda" | "webgpu" | "dml")` rewrites the graph into ONNX Runtime's fused transformer ops at export time: attention, `GroupQueryAttention` over the KV cache, rotary, norms and `MoE`. OpenVINO exports now hit OpenVINO's fused CPU kernels for attention, RoPE, MoE experts and compressed int8/int4 weights, which makes decode 2.4–3.9× faster than before.
- **Faster than `torch.compile`**: on EmbeddingGemma2 (8 × 128 tokens), ONNX Runtime with the TensorRT EP is 7.6× faster than eager PyTorch and 1.4× faster than `torch.compile` on an A100. OpenVINO is 4.1× faster than eager and 2.2× faster than `torch.compile` on a Xeon Platinum 8570 (4.9× and 2.6× with int8 weights).
- **Shareable**: an export saves and loads like a model, optionally in a `subfolder` of the model repo. Its `export.json` records the config, input shapes, package versions and I/O names. Quantized exports (PT2E, NNCF, ONNX Runtime) save and run the same way.

🚨 `export()` returns an `ExportedModel`, and an ONNX program is an `onnx_ir.Model`. `OnnxConfig`'s and `OpenVINOConfig`'s `output_path` options are gone in favour of `save_pretrained`. Custom backends register with `register_backend`.

📚 Docs: https://huggingface.co/docs/transformers/main/en/exporters


## Vision

This release fixes a range of vision-model bugs: classification pipelines now respect `problem_type`, and several models get corrected behavior (Step3.7 RoPE axis order, MiniMax-M3-VL packed attention isolation, Qwen VL positions with reused caches, PaddleOCR-VL interpolation alignment, Kosmos2 SDPA masking, and Emu3 float image handling). Chunked prefill now keeps image inputs on the


* fix(pipelines): respect problem_type in image and video classification pipelines (#50582) by @surendra902 in [#50582]
* Fix Step3.7 vision RoPE spatial W/H axis order (#50547) by @caohch-1 in [#50547]
* Raise ValueError for out-of-range float images in convert_to_rgb (#50487) by @n24q02m in [#50487]
* CI: install OpenVINO and NNCF in the daily GPU image (#50422) by @IlyasMoutawwakil in [#50422]
* Fix MiniMax-M3-VL packed vision attention isolation (#50346) by @taking-lying-flat in [#50346]
* Fix Qwen VL positions when a text cache is reused with a new image (#50427) by @Arnav66692 in [#50427]
* [Docker] Update Nvidia CI and Flash Attn CI images to torch 2.15 / cu132 (#50467) by @ydshieh in [#50467]
* Fix PaddleOCR-VL vision position interpolation alignment (#50384) by @scyyh11 in [#50384]
* Fix Emu3 image processor crash on float numpy images (#50226) by @simpleqt in [#50226]
* Build Idefics-style vision position ids with torch.where (#49917) by @shoumikhin in [#49917]
* Keep image inputs on the first chunk of a chunked prefill (#49753) by @digit50 in [#49753]
* Docs: note in the v5 migration guide that hidden_states[-1] of vision models is pre-norm (#50303) by @ravediamond in [#50303]
* Remove the GitHub API cache-bust from the AMD docker images (#49351) by @tarekziade in [#49351]
* [docs] Vision backend differences (#50131) by @stevhliu in [#50131]
* [Kosmos2] Fix SDPA applying a causal mask in the image-to-text projection (#50118) by @tarekziade in [#50118]


## Parallelization

This release improves distributed training and inference. Expert parallelism gains a selectable `ep_implementation` (all-reduce for CUDA-graph-friendly inference, all-to-all for training). The Trainer now reduces FSDP/HSDP gradients only on the last micro-batch, and the token-count normalization is fixed to use true division. Tensor-parallel fixes cover attention sinks (including Beam attention) and tied-weight comparisons on TPU


* Expert-parallel implementations for `ep_dispatch_experts`: all-reduce at inference, all-to-all in training (#49429) by @IlyasMoutawwakil in [#49429]
* [FSDP tests] Fork CPU ranks from a forkserver instead of spawning them (#50444) by @ydshieh in [#50444]
* [distributed - TPU] Fix tied-weight comparisons on TPUs when using TP (#50406) by @tengomucho in [#50406]
* [`distrib
```



document fetched 2026-11-14 02:34
