← sources

Why is MTP slower than standard decode?

source: reddit (reddit:LocalLLaMA)published 2026-10-12 13:59ingested 2026-10-13 02:42

retained text

I'm trying to get Ornith 1.5 9B running (I can also barely run 35B A3B but I want something that I can run relatively easily) with the latest llama.cpp with this command:

`llama serve --flash-attn on --cache_type-k q8_0 --cache-type-v q8_0 --hf-repo ornith-ai/Ornith-1.5-9B-GGUF --spec-draft-p-min 0.9 --spec-type draft-mtp`

And this gives me \~15 tok/s TG and \~200 tok/s PP. However, without MTP, I am getting double these numbers. I have a very high draft acceptance rate of 0.95 with a mean length of 2.27.

I have an 8 GB RTX 4060 by the way, but I feel this is irrelevant. I have noticed with MTP my memory bandwidth and GPU utilisation is about half of what it is without MTP.

document fetched 2026-10-13 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1x42lp3/why_is_mtp_slower_than_standard_decode