← sources

Nanbeige4.3-33B Q4 on a MacBook Pro M5 Max, mlx-lm vs llama.cpp vs Kapitsa, Prefill/Decode 512 to 32K range.

source: reddit (reddit:LocalLLaMA)published 2026-10-26 22:10ingested 2026-10-27 02:39

retained text

.  
  
hey there, I measured Nanbeige4.3-33B (dense, 4-bit) on three Apple silicon engines: mlx-lm, llama.cpp and Kapitsa. The comparison covers prefill, decode, time to first token, follow-up latency and memory, at prompt lengths from 512 to 32K tokens.

its a new model, folks from nanbeige claim these numbers.

https://preview.redd.it/328gzbcvvvxh1.png?width=2800&format=png&auto=webp&s=d296f9e03f66c98ab4f5da70e31d5a5be8c3c336

things to mention: I work on Apple Silicon Inference engine: [Kapitsa](https://github.com/nace-ai/kapitsa), its open source apache2, so treat the numbers with that in mind. The raw outputs and scripts are linked at the bottom so anyone can check or rerun them.

**Setup**

* **Machine:** MacBook Pro M5 Max, 128 GB, macOS 26.4, AC power, High Power mode.
* **Weights:**
   * mlx-lm 0.32.2 and Kapitsa 0.2.19 read the same 4-bit MLX AWQ pack (group 64). The current release, 0.2.20, adds two KFlash fixes for long contexts and changes none of the numbers here.
   * llama.cpp b11871 reads Q4\_K\_M.
* **Method:** the engines ran one at a time, in turn, under the same conditions.

# TL;DR

This model is SLOW and very inefficent KV-cache, every layer having full attention isnt optimal i guess, muse glimmer is 1.6gb for 128k tokens but nanbeige is 26gb lol. EDIT: coding is quite good but, I would say i would still stick to 35B/27B Qwen models and look for the qwen 4 release  


# 1. Prefill speed

prefill

|Context|mlx-lm|llama.cpp|Kapitsa|
|:-|:-|:-|:-|
|512|**757**|691|693|
|1K|**829**|669|721|
|2K|**840**|639|731|
|4K|**773**|556|692|
|8K|**663**|496|616|
|16K|**618**|437|540|
|32K|**546**|349|520|

Prompt tokens per second, from each engine's own benchmark tool.

# 2. Decode speed vs context length

|Context|mlx-lm|llama.cpp|Kapitsa with KFlash|
|:-|:-|:-|:-|
|512|28.8|27.3|**34.9**|
|1K|28.4|26.9|**32.6**|
|2K|27.9|26.4|**31.0**|
|4K|27.5|24.6|**30.3**|
|8K|25.8|23.2|**26.1**|
|16K|24.3|20.4|**25.6**|
|32K|20.8|16.7|**22.8**|

https://preview.redd.it/783i1abbxvxh1.png?width=1760&format=png&auto=webp&s=d60f9f66adc92a0e0f04b4394845ac617f5033c8

# 3. Time to first token, cold prompt

ttft cold

|Prompt tokens|mlx-lm|llama.cpp|Kapitsa|
|:-|:-|:-|:-|
|532|**0.90 s**|1.20 s|1.18 s|
|995|**1.75 s**|1.94 s|1.94 s|
|2K|**3.36 s**|4.28 s|4.07 s|
|4.1K|**6.44 s**|8.71 s|8.08 s|
|8.1K|**12.9 s**|17.9 s|15.1 s|
|16.1K|**28.0 s**|40.7 s|33.2 s|
|29.5K|**59.4 s**|89.7 s|69.4 s|

These are real chat requests to each engine's OpenAI-compatible server, with thinking off, so the times include the chat template and the server's own overhead. The order matches the prefill results.

https://preview.redd.it/053ey6ihxvxh1.png?width=1760&format=png&auto=webp&s=8003339f13619c909071dc6ac9efe1205098f6a0

# 4. Time to first token on a follow-up turn

ttft follow-up

|Conversation so far|mlx-lm|llama.cpp|Kapitsa|
|:-|:-|:-|:-|
|532|290 ms|220 ms|**140 ms**|
|995|310 ms|220 ms|**150 ms**|
|2K|340 ms|220 ms|**160 ms**|
|4.1K|370 ms|240 ms|**150 ms**|
|8.1K|460 ms|270 ms|**190 ms**|
|16.1K|580 ms|320 ms|**230 ms**|
|29.5K|880 ms|410 ms|**300 ms**|

The conversation so far is already cached, and a new user turn is added. This is the latency you feel in multi-turn chat and agent loops. All three engines reuse their cache here; they differ in how much each turn costs on top of it.

https://preview.redd.it/04ilir5fxvxh1.png?width=1760&format=png&auto=webp&s=1002135edfc3817bbb4f70cb4b38a0c2f8cc3d48

# 5. Peak memory as the context grows

memory

|Context reached|mlx-lm|llama.cpp|Kapitsa|
|:-|:-|:-|:-|
|532|**21.9 GB**|28.0 GB|22.6 GB|
|2K|**22.3 GB**|28.6 GB|22.6 GB|
|4.1K|24.9 GB|29.1 GB|**22.8 GB**|
|8.1K|31.6 GB|30.2 GB|**24.4 GB**|
|16.1K|36.7 GB|32.0 GB|**25.4 GB**|
|29.5K|52.1 GB|35.5 GB|**28.1 GB**|

https://preview.redd.it/4eozm8faxvxh1.png?width=1760&format=png&auto=webp&s=421e6612178ee93e86e75db87b5279f383422c29

notice the growth of memory with the context growth

# How Kflash works

Kflash - speculative decoding, but as a tree. a tiny drafter model guesses the next 7 tokens and the 27b checks all of them in one pass. most engines check those guesses as a chain, so the first wrong guess throws away everything after it. kflash (our version) checks a tree instead: where the drafter isnt sure, its second guess comes along in the same pass, and we keep the longest branch the 27b agrees with. heres one pass both ways:

https://preview.redd.it/7ajhl1d2vvxh1.png?width=880&format=png&auto=webp&s=9305f1bb5e32e226272fa11d33073206300fc2a3

One cycle, two ways to spend the same 8 rows (an illustration). Top: the drafter's chain. The target accepts `key`, then writes `not`, and the rest of the chain is discarded. Bottom: a best-first tree of the same candidates. Rows are numbered in the order the builder takes them, with each node's path probability. The target accepts `key`, `not` and `in`, then writes `self`. Positions count from P, the last token written.

To reproduce the Kapitsa side:

    brew install nace-ai/tap/kapitsa
    kapitsa pull nanbeige
    kapitsa serve --model nanbeige

document fetched 2026-10-27 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1xgaupg/nanbeige4333b_q4_on_a_macbook_pro_m5_max_mlxlm_vs