← sources

Benchmarking: Qwen 27B, 35B, QFN180B; Nemotron Bolt 74B, Flint40B & Clover 35B (Q4 + KFlash) on MacBook M5 Max 2026 with Kapitsa (Apple Silicon Inference Engine)

source: reddit (reddit:LocalLLaMA)published 2026-11-01 22:47ingested 2026-11-02 02:41

retained text

hey folks, I'm Ye, one of the contributors to open-source project called Kapitsa, Apple Silicon Inference engine optimized specifically for Apple Silicon to run Qwen, Nemotron, Flint and Clover models locally. Our custom-written Metal kernels let us tune for each chip and each model. [https://github.com/nace-ai/kapitsa/](https://github.com/nace-ai/kapitsa/)

The latest update, 0.2.26, is out. Since our last M5 Max numbers, Flash-Next 180B reads prompts 46% faster and the 27B writes 8% faster, and DiffuSpace's Clover 35B joins the catalog. 



Kapitsa also runs **Qwen Flash-Next 180B with SSD offloading**: on a Mac that can't hold it, the experts that don't fit in RAM stream from the SSD, so it runs on a 48 GB MacBook Pro at 61 tok/s.

Here is the showcase on a 2026 MacBook Pro 16 with an M5 Max and 128 GB. We ran six models from 1K context all the way up to 32K, against the next-fastest engine for each model on the same Mac.

Try it (any Apple Silicon Mac; models in this post: `35b`, `27b`, `flash-next`, `bolt`, `flint`, `clover`):

    brew install nace-ai/tap/kapitsa
    kapitsa serve --model 35b       # OpenAI and Anthropic APIs on localhost
    kapitsa opencode --model 35b    # or claude, codex, pi
    kapitsa bench                   # benchmark your own Mac



# Results

https://preview.redd.it/0zmlaxm2x2zh1.png?width=2400&format=png&auto=webp&s=5f56c7445044fb825fa218b280e85d0307bc2d0b

Decode is a 512-token coding answer, prefill a 4K prompt, median of 3 runs:

|Model|Decode, tok/s|Prefill, tok/s|
|:-|:-|:-|
|Qwen3.6 35B-A3B|346|6,036|
|Clover 35B-A3B|304|5,981|
|Flint 40B|265|5,770|
|Nemotron Bolt 74B|154|1,075|
|Qwen3.8 27B|143|904|
|Qwen Flash-Next 180B|108|1,701 |

From 1K context all the way up to 32K. At 32K the 35B still writes 175 tok/s, and Flash-Next barely drops (90 to 78):

[Prefill from 1K to 32K context](https://preview.redd.it/z0h67f95x2zh1.png?width=2400&format=png&auto=webp&s=24fa262800cd76763a08a96fd056a840ceb60b99)



[Decode from 1K to 32K context](https://preview.redd.it/tepn1hw5x2zh1.png?width=2400&format=png&auto=webp&s=c14021efad32b136578239de9fee54f396e59f56)

# Against the other friends

Against the next-fastest engine for each model on the same Mac: 1.23x Splash on the 35B, 2.2x oMLX on Bolt, 1.18x rec on Flint and 1.12x mlx-serve on the 27B. Flash-Next is a tie with mlx-serve, Splash still reads prompts faster on the two Qwens, and nothing else runs Clover on a Mac.

[Decode against the other Mac engines](https://preview.redd.it/f0vfzki8x2zh1.png?width=2400&format=png&auto=webp&s=6c0207ed2433a509e94a51f3d0222df495027b71)

Every cell with versions and flags: [benchmarks](https://github.com/nace-ai/kapitsa/blob/main/docs/benchmarks.md#m5-max). Raw runs: [results](https://github.com/nace-ai/kapitsa/tree/main/bench/results/2026-11-01-m5-max-kapitsa).

  
**Why it's fast: KFlash**

Other engines check the drafter's guesses as one chain and stop at the first wrong one. KFlash checks a tree of the drafter's best guesses in the same pass, so more tokens survive each pass.

[How KFlash works: the drafter's chain against KFlash's tree](https://preview.redd.it/tawkriyax2zh1.png?width=2400&format=png&auto=webp&s=08eb104b27bca8df012fe91a64003511424a98e6)

On Apple GPUs that's nearly free: the GPU works in 8-row tiles, and a one-token step already pays for 8. Every token it keeps is one the big model would have written itself, so the output doesn't change, only the speed. [How KFlash works](https://github.com/nace-ai/kapitsa/blob/main/docs/kflash.md)



shoutout to splash, ollama, omlx, llama cpp and the folks who   


# Send us your benchmarks

Run the same test on your Mac and add it to the [community results](https://github.com/nace-ai/kapitsa/blob/main/bench/community/RESULTS.md). It takes 3-15 minutes:

    brew install nace-ai/tap/kapitsa
    brew install gh && gh auth login    # results are sent as a GitHub pull request
    kapitsa bench --submit              # pick a model; it asks before sending anything

Plug in and close heavy apps while it runs. At the end it asks once: Enter sends the result as a pull request from your GitHub fork, and `n` keeps it on your Mac. What it sends is about 2 KB: your Mac's model, chip, RAM, macOS version and the numbers, with no serial number, user name or paths (add `--dry-run` to see exactly what it would send). No GitHub? Paste the numbers it prints in a comment here.

  


**BY THE WAY**

This is a new codebase so, sometimes there could be some weird prefill/decode bugs.

document fetched 2026-11-02 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xlgu8m/benchmarking_qwen_27b_35b_qfn180b_nemotron_bolt