# I built BitRun: A standalone runner for 1-bit LLMs (100–127 t/s on Apple Silicon & 40–45 t/s on PC CPUs with zero compile) — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/ece74e1a5068937e

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-10-14 11:58 ingested 2026-10-15 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1x5pao2/i_built_bitrun_a_standalone_runner_for_1bit_llms/)

## retained text



```
I built BitRun because I wanted to run 1-bit ternary LLMs on ordinary laptops without melting my battery, spinning up a dedicated GPU, or wrestling with complex build environments. Right now, BitRun gives you 98 to 127 tokens/sec on an Apple M4 Pro CPU and 40 to 45 tokens/sec on an Intel Core i5 PC CPU, pulling minimal power with 0% GPU usage.

To make this actually work on modern hardware, I had to fix several low-level C++ kernel math bugs in Microsoft's research codebase (bitnet.cpp):

**Memory stride clobbering:** In ggml-bitnet-mad.cpp, batch stride was ignored in the write-back stage, causing evaluated batch rows to silently overwrite each other in memory.

**The infinite @ @ @ loop on ARM NEON:** Block size was hardcoded to 64 instead of the GGUF standard 128 elements in ggml-cpu-i2s.c. That 50% pointer arithmetic shift fed scrambled bytes into the NEON DotProd adders.

**Activation mismatch:** The model graph initialized the FFN gate with SiLU, but the 2.4B model was trained with squared ReLU. Using SiLU distorted the activation range and produced gibberish.

**Multi-turn BLAS crashes:** BLAS was claiming matrix multiplications past 32 context tokens and jumping into unmapped memory with unaligned byte offsets.

**Windows MSVC lockout:** Removed artificial CMake checks that blocked native Visual Studio compilation.

I submitted the fixes upstream to Microsoft:

* **PR #642:** [https://github.com/microsoft/BitNet/pull/642](https://github.com/microsoft/BitNet/pull/642)
* **PR #644:** [https://github.com/microsoft/BitNet/pull/644](https://github.com/microsoft/BitNet/pull/644)

Then I packaged the patched engine into BitRun so anyone can test ternary models in 60 seconds with an Ollama-style workflow. No CMake, no Clang toolchains, no Python virtualenvs, and no heavy Docker images.

**HOW TO RUN IT (ZERO COMPILE)**

`Mac / Linux: curl -fsSL` [`https://raw.githubusercontent.com/IshekKhal/bitrun/main/install.sh`](https://raw.githubusercontent.com/IshekKhal/bitrun/main/install.sh) `| sh`

`Windows (PowerShell): irm` [`https://raw.githubusercontent.com/IshekKhal/bitrun/main/install.ps1`](https://raw.githubusercontent.com/IshekKhal/bitrun/main/install.ps1) `| iex`

**Pull and chat on pure CPU:**

`bitrun pull falcon3-1b`

`bitrun run falcon3-1b`

You can also run "bitrun serve falcon3-1b" to get an OpenAI-compatible REST server on port 11435 (point Cursor, [Continue.dev](http://Continue.dev), or Open WebUI to it).

Custom personas can be created using standard Modelfiles (bitrun create engineer -f Modelfile).

**VERIFIED CPU SPEEDS (PURE CPU SIMD, 0% GPU)**

Apple M4 Pro CPU (12 Cores):

* Falcon3 1B Instruct (\~1.30 GB): 98 to 127 tokens/sec (\~1.9 GB RAM)
* BitNet b1.58 2.4B (\~1.13 GB): \~40 tokens/sec (\~1.8 GB RAM)
* BitNet 70M (\~79 MB): 1,350+ tokens/sec (< 150 MB RAM)

Windows x86\_64 (Intel Core i5-12600K, AVX2):

* Falcon3 1B Instruct: 39 to 45 tokens/sec (209.6 t/s prompt processing)
* BitNet 70M: 492 tokens/sec

Linux (WSL2 / 4 cores, AVX2):

* Falcon3 1B Instruct: 40 to 45 tokens/sec



**REALISTIC SCOPE**

Ternary weights require Quantization-Aware Training (QAT) from scratch. You cannot just take an existing 70B FP16 model and post-quantize it down to 1.58-bit without destroying it. Because of that, the public model catalog right now is limited to models under 3B parameters (falcon3-1b, bitnet-2b, falcon3-3b, bitnet-70m). They will not replace Claude or GPT-4 for heavy coding.

However, for low-power edge devices, battery-conscious laptop use, and background automations where you want 100+ tokens/sec on pure CPU, it is ready to run today.

**GitHub Repo:** [https://github.com/IshekKhal/bitrun](https://github.com/IshekKhal/bitrun)

**Upstream PRs:** [https://github.com/microsoft/BitNet/pull/642](https://github.com/microsoft/BitNet/pull/642) and [https://github.com/microsoft/BitNet/pull/644](https://github.com/microsoft/BitNet/pull/644)

**Pre-compiled Releases:** [https://github.com/IshekKhal/bitrun/releases/tag/v0.1.0-alpha](https://github.com/IshekKhal/bitrun/releases/tag/v0.1.0-alpha)

Drop your CPU benchmark numbers if you try it!
```



document fetched 2026-10-15 00:46 · canonical https://reddit.com/r/LocalLLaMA/comments/1x5pao2/i_built_bitrun_a_standalone_runner_for_1bit_llms
