← sources

Kapitsa: open-source Apple Silicon optimized Metal engine built for Qwen and Nemotron. See benchmarks of Qwen 27B, Nemotron + 8 other models on an M1 Pro (16GB), M4 Pro (48GB) and M5 Max (128GB)

source: reddit (reddit:LocalLLaMA)published 2026-10-19 07:04ingested 2026-10-20 02:40

retained text

# 

hey, my name is Ye, I vibecode at [Nace.ai](http://Nace.ai) and we do a lot of work with enterprises, and a lot of the time we're inside their networks with no internet access and outdated cloud hosted models. considering we work on macbooks, we have built Kapitsa, an open-source (Apache 2.0) inference engine for Apple Silicon, to run coding agents on local models that would actually fit on your Macs.

Kapitsa runs only two model families: **Qwen** (3.5 **4B**, 3.8 **27B**, 3.6 **35B**\-A3B, and 3.8 **Flash-Next** on 128 GB Macs) and NVIDIA **Nemotron** (3.5 **Lightning**, **Bolt** and **Super-VL,** plus **Dory 4B** (this is new, we didnt test Dory too much yet). focusing on only these models allows us customize the Metal kernels for each one on Apple Silicon

GitHub: [https://github.com/nace-ai/kapitsa](https://github.com/nace-ai/kapitsa) \- It's open source, and Apache 2.0.

# Quickstart

`brew install nace-ai/tap/kapitsa`

`# or: curl -fsSL` [`https://raw.githubusercontent.com/nace-ai/kapitsa/main/install.sh`](https://raw.githubusercontent.com/nace-ai/kapitsa/main/install.sh) `| bash`

`kapitsa opencode --model 35b    # or kapitsa claude, kapitsa codex, etc`

# Decode Benchmarks 

(we at nace dont like benchmarks, **DO NOT** trust them)  
  
instead we have built a quite simple cli for you to be able to bench it yourself. please submit your results, we are looking for contributors for this project \^\^

`kapitsa bench`

**MacBook Pro M4 Pro (2024), 48 GB: Qwen Models**

|tok/s|**Kapitsa**|mlx-serve|oMLX|MTPLX|Ollama|MLX|llama.cpp|LM Studio|Rapid-MLX|
|:-|:-|:-|:-|:-|:-|:-|:-|:-|:-|
|Qwen3.8 27B|**63**|54|45|36|32|28|18|15|—|
|Qwen3.6 35B-A3B|**154**|124|127|80|89|86|69|86|—|
|Qwen3.5 4B|**179**|137|102|104|109|88|70|81|98|

**MacBook Pro M4 Pro (2024), 48 GB: Nemotron**

|tok/s|**Kapitsa**|oMLX|MLX|llama.cpp|
|:-|:-|:-|:-|:-|
|Nemotron 3.5 Bolt 74B-A9B|**64**|35|32|✗|
|Nemotron 3.5 Lightning 30B-A3B|**106**|—|65|—|
|Nemotron Dory1.0 4B|**64**|✗|48|6 (LOL)|

**MacBook Pro M5 Max (2025), 128 GB: Qwen**

|tok/s|**Kapitsa**|oMLX|Ollama|LM Studio|MTPLX|MLX|mlx-serve|llama.cpp|
|:-|:-|:-|:-|:-|:-|:-|:-|:-|
|Qwen3.6 35B-A3B|**272**|232|166|127|123|112|—|—|
|Qwen3.8 27B|**115**|—|68|32|69|30|—|—|
|Qwen3.5 4B|**294**|—|213|154|175|135|—|—|
|Qwen3.8 Flash-Next 180B-A7B|105|107|60|✗|88|77|**108**|96|

Flash-Next is the one model where Kapitsa isn't fastest: mlx-serve and oMLX are ahead by 2-3%.  (our leadership team told us that they will buy us 128gb to optimize flash next, if this thing gets 1k stars xd)

**MacBook Pro M5 Max(2025), 128 GB: Nemotron**

|tok/s|**Kapitsa**|oMLX|mlx-serve|llama.cpp|Ollama|MLX|MTPLX|LM Studio|
|:-|:-|:-|:-|:-|:-|:-|:-|:-|
|Lightning 30B-A3B|**239**|190|190|159|153|139|131|110|
|Super-VL 120B-A12B|**85**|52|—|66|64|46|—|61|
|Bolt 74B-A9B|**127**|—|✗|✗|✗|—|✗|✗|

**MacBook Pro M1 Pro, 16 GB (2021)**

|tok/s|**Kapitsa**|mlx-serve|oMLX|Ollama|LM Studio|MLX|MTPLX|llama.cpp|
|:-|:-|:-|:-|:-|:-|:-|:-|:-|
|Qwen3.5 4B|**85**|62|55|55|47|46|42|38|
|Nemotron Dory1.0 4B|**40**|—|✗|—|—|30|—|4|



# Where the speed comes from

1) custom Metal kernels per architecture (Gated DeltaNet + attention for Qwen3.5-family, Mamba-2 + attention + LatentMoE for Nemotron-H) and per weight format (MLX affine, NVFP4, NVIDIA's 2-bit experts).  
2) speculative decoding: a draft model proposes a block and the model checks it in one pass (DFlash2 for the Qwen 27B and 35B, z-lab's DFlash for the 4B, DSpark for Lightning, NVIDIA's LiLiCorr for Bolt and Super-VL).  
3) caching that works with hybrid models. Their recurrent layers break a plain KV prefix cache, so each conversation keeps KV plus recurrent-state checkpoints at turn boundaries, and the engine reads your last answer into the cache while you read it.  
4) on the 27B and Bolt, part of each layer's prefill runs on the Neural Engine alongside the GPU. (biggest boost)  
5) requests in flight at once (an agent's subagents, opencode's title request) share GPU passes instead of queueing.  
  
we also made sure that speculation doesn't change what the model writes: greedy keeps the longest drafted prefix the model itself would pick,  sampling is basically like standard speculative sampling, so tokens come from the models own distribution.   
  
quality drops a bit since its the quantized 4-bit weights, the as the other MLX engines. although you can pick it with --model 27b:6bit if need be, we reccomend using 4bit, it does the job



# CLI

    brew install nace-ai/tap/kapitsa
    kapitsa doctor                 # which models and context sizes fit this Mac
    kapitsa opencode --model 27b         # or: kapitsa pi / claude / codex

`kapitsa serve` spins up a server that works with your claude,codex etc, or any custom requests.

`kapitsa bench` runs the same benchmark on your Mac, and `kapitsa bench --submit` adds it to the community results (it shows the JSON and asks first). M2 and M3 Macs, and 16-36 GB ones, are the results we want most. (We need a wide variety of devices)

# why Qwen and Nemotron

We picked the models people here run for local coding and wrote kernels for exactly those architectures. Other Qwen3.5-family checkpoints load from Hugging Face, a folder or a GGUF; other architectures don't, and adding one is real work. \[What's next, if you want to say: Qwen 4 on release, Flash Next\]

# Things to mention

* Apple silicon only: M1 or newer, macOS 14+
* Lightning and Dory are text only.
* New code, so expect bugs. Issues welcome, we have quite a bit of fable credits dedicated to this project \^\^.

# Credits

* z-lab (DFlash) -awesome ideas, DeepSeek (DSpark), Inco(DFlash2) - cool, Qwen (MTP heads), NVIDIA (LiLiCorr and the Nemotron checkpoints), MLX(quantization), mlx-serve(amazing)
* Fable 5.1, Elorian Glimpse, Opus 5.5, [Nace.AI](http://Nace.AI) coding models

# Asks

try it yourself and upload your benches please. its a new codebase, bug reports/ suggestions are welcome.

document fetched 2026-10-20 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1x9pzt0/kapitsa_opensource_apple_silicon_optimized_metal