← sources

JetBrains/Mellum2.1-12B-A2.5B-Thinking-MTP · Hugging Face

source: reddit (reddit:LocalLLaMA)published 2026-11-02 03:58ingested 2026-11-03 02:42

retained text

# Mellum2.1 Thinking with the MTP head

# Multi-token prediction head

The multi-token prediction (MTP) head is one extra transformer block, trained together with the model, that predicts the token after the one just sampled. Inference engines use it as a speculative draft: the model verifies every drafted token, so the output is unchanged while decoding runs 1.4–1.8x faster, with about 90% of the drafted tokens accepted. The head lives in `model-mtp.safetensors` under the `mtp.` prefix, announced by `num_nextn_predict_layers: 1` in `config.json`; everything else is identical to [Mellum2.1 Thinking](https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking).

* **vLLM** 0.33 or newer: `--speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'`, as in the serving commands above.
* **transformers** 5.20 or newer: pass `use_mtp=True` to `generate`.
* **llama.cpp**: use the GGUF repository with `--spec-type draft-mtp --spec-draft-n-max 1`.

document fetched 2026-11-03 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1xln8f2/jetbrainsmellum2112ba25bthinkingmtp_hugging_face