# Two days tuning a single Tesla P40: 35B MoE, full 262k context, ~60 t/s short and ~30 t/s at 100k deep. No sub-Q4 quants. — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/8187e32280543edd

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-10-28 10:58 ingested 2026-10-29 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xhkwlm/two_days_tuning_a_single_tesla_p40_35b_moe_full/)

## retained text



```
i Got a bit obsessed with my P40 this week trying to max speed on Ornith 1.5 35B-A3B (qwen3.5 MoE arch) without going below Q4 weights or q8 KV. Shrinking quants to win a benchmark has not really been useful to me so ai wanted a logical middle ground that was still “fast”

Setup: one Tesla P40 24GB on PCIe x1 (yes, x1), Windows, my own llama.cpp fork. Q4\_0 built from the official BF16, MTP draft (n=2), full 262k context, q8\_0 KV.

What actually moved it:

\- **ECC off +**1.5 GB VRAM and \~9% decode.  
\- *q8\_0 KV, not q4\_0.* On Pascal q4 KV falls apart with depth (30.6 vs 42.2 t/s at 32k).   
\- **Check your MTP head.** The first upload of this model, and quants built from it, have an untrained MTP head. \`blk.40.nextn.shared\_head\_norm\` mean \~1.02 = untrained, \~2.93 = trained. Draft acceptance went from \~45% to 60-85%.  
\- **int8 dp4a for the KQ half of the flash-attention tile kernel.** Pascal has no tensor cores and crippled fp16, but dp4a is fast. Prefill +26% at 32k, +41% at 100k.  
\- **MTP prefill window.** The MTP head is a full-attention layer, so on long prompts it costs \~13% prefill. Letting it read only the last 8k of the prompt gets that back (121k prompt: 327 -> 373 t/s). Output doesn't change, the main model still verifies every draft.

Didn't work: f16 KV (faster in llama-bench, but slower and less context once MTP is on in a real server), dequantizing V inside the kernel (-5%), the official Q4\_K\_M (same quality score as my Q4\_0, slower, less context).

llama-bench, no MTP, 2 runs each:

| depth | prefill t/s | decode t/s |  
|---|---|---|  
| 0 | 1104 | 60.8 |  
| 8k | 880 | 57.0 |  
| 32k | 552 | 46.0 |  
| 64k | 366 | 35.8 |  
| 100k | 265 | 29.1 |  
| 190k | - | 18.8 |

Real server with MTP: 65-74 t/s on short prompts. A 101k-token prompt reads at \~415 t/s and generates at 35-39 t/s.

I couldn't find anyone else posting P40 numbers at depth, so hopefully the curve helps someone. Happy to share the patches if you want to try them on your own Pascal card.
```



document fetched 2026-10-29 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xhkwlm/two_days_tuning_a_single_tesla_p40_35b_moe_full
