Splash fork optimised for M5 Max: ~1.5× faster (1.25× single request)
retained text
I’ve spent the last couple of days with Opus 5.5 working on a fork of Inco’s excellent and already blazingly fast Splash engine to optimise it for M5 Max chips. Taking liberties and referring to it as Splish. Roughly the opposite direction to u/Erp4759’s great M1 port ([Splash on M1, part 2](https://www.reddit.com/)). **Charts (stock Splash vs Splish):** [https://raw.githubusercontent.com/publicExcess/splish/m5/docs/m5/charts/png/single-request.png](https://raw.githubusercontent.com/publicExcess/splish/m5/docs/m5/charts/png/single-request.png) [https://raw.githubusercontent.com/publicExcess/splish/m5/docs/m5/charts/png/concurrency.png](https://raw.githubusercontent.com/publicExcess/splish/m5/docs/m5/charts/png/concurrency.png) **Results** Against Splash 1.1.0 as shipped, on the same Mac with the same models, Splish is: * **\~1.25× faster** at a single request (+11% to +35%) * **Up to 1.5× faster** at 2–4 requests * **Quality is unchanged** on everything I measured * In **real world use**, going from about 45-51 tok/s to 56 - 64 tok/s in short story prompts in Deepseek Harness. All figures are for 4-bit models on a 40-core M5 Max unless stated otherwise. **What worked** **1. Kernel choices measured specifically for the 40-core M5 Max**, using Splash’s own tuner. The tuner is in Splash’s source code but isn’t included in the packaged app. This was the biggest single-request win: Swift-1.5 went from **74.7 → 89.8 tok/s (+20%)**. **2. Loading those choices from a file (SPLASH\_KERNEL\_CHOICES).** No speedup by itself, but it means anyone can retune without rebuilding. **3. New verify kernels for the M5’s tensor units.** These use lighter barriers and compute row sums once per projection: * **+5%** at 1 request * **+10–19%** at 2–4 requests **4. Extending the same kernels to more projections.** A further **1–3%** at 3–4 requests. Together, #3 and #4 make a decode step at 2 / 3 / 4 requests: **1.32× / 1.49× / 1.36× faster** than tuned Splash. **5. Tuned Qwen3.6-35B-A3B with the new kernels.** Speedups at 1–4 requests: **+5% / +18% / +22% / +18%** **6. An attention tweak for the 27B shape.** Attention is 2–3% faster and prompt processing 3–4% faster. Too small to show up in the overall numbers. **7. GGUF (Q8\_0, Q4\_K\_M, Q6\_K): faster input loads in the decode kernel.** **+2–14% per kernel** and about **+2% per step**. Output is bit-identical. **8. A copy rule for coding agents**, borrowed from [TensorFold](https://github.com/ashhart/TensorFold). When the model is rewriting text it has already seen, the drafts copy it verbatim. Whole-file edits get **+24% to +42%**, while everything else stays within **±3%**, and output is exact. I’m exploring a complementary approach for a future version. The README also lists everything that *didn’t* work for me, which is probably useful if anyone wants to avoid going down the same rabbit holes. There’s lots more I’d like to test, but thought this was a nice start. The tuned settings are for a **40-core M5 Max**. Other M5 chips fall back to Splash’s defaults unless overridden. An auto-tuner is coming. If you run it, python3 dev/m5/report.py prints a performance report. **Results from other machines are very welcome, especially if you find cases where it’s slower.**
document fetched 2026-09-28 00:39 · canonical https://reddit.com/r/LocalLLaMA/comments/1wrd1p1/splash_fork_optimised_for_m5_max_15_faster_125