← sources

Should I not use MTP draft for agentic work?

source: reddit (reddit:LocalLLaMA)published 2026-10-03 02:58ingested 2026-10-04 02:39

retained text

I run qwen3.8-27b iq3\_s with llama.cpp to serve local hermes agent, in 16gb vram.

I noticed that enable MTP draft make prefill slower and the model seems less smart.

and vram is very tight I need to set the context length to 96k. decode speed can go around 40 to 60 tps.

if I disable MTP I can use 128k context but decode speed drop to like 35 tps.

What will you choose?

document fetched 2026-10-04 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1wwcgg3/should_i_not_use_mtp_draft_for_agentic_work