Some people are skeptical of my (almost) 400 tokens per second claim from yesterday
retained text
I woke up to see people accusing me of (and in some cases banning me for) lying about the top speed number, but that's just how MTP works. Different prompts are going to get different speeds. This one gets damn close at 376, the prompt is "edit this python file" and then gives the file, the result is an edited file. This is a realistic scenario that anyone who codes has come across, not a unicorn best case (it \*is\* a best case prompt type, just not a rare one). If I run enough of these style prompts some of them go even faster, the fastest I've seen was 404 but I don't usually feel the need to video record them. No, I'm not going to put 140 in the headline as the number it gets on prose reasoning where MTP is not as successful at predicting tokens. Nobody puts their worst case numbers in the headline. But I'm not hiding that number either, I'm quite proud of it since it's double what any other engine can do with that prompt on this hardware. This is [https://github.com/johnfink8/sfnx](https://github.com/johnfink8/sfnx) (my own engine, strata/llama based but all the engine code is new) running flash-next Q4\_XL on 3 V100 32GB in an old Dell R740. The strata heritage gives it the flexibility to run larger quants that are partially offloaded to memory, but the engine rebuild focuses on VRAM and GPU efficiency so you want most of the experts in VRAM to use sFNX. I paid about $600 each for the cards (but they've apparently gone up this week), and the server itself is almost 10 years old and you can grab a barebones with CPUs for like $500. The hardware is finicky to get into an ideal config but once you do these old workhorses can do some **work**, son.
document fetched 2026-10-28 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1xgqkt7/some_people_are_skeptical_of_my_almost_400_tokens