# [Boole-32B-A2.5B-preview] 9GB 1.8-bit MoE, 565 token/s on 1 RTX 5090 — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/60c1d9c9c60b9c73

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-13 09:29 ingested 2026-11-14 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xv3h93/boole32ba25bpreview_9gb_18bit_moe_565_tokens_on_1/)

## retained text



```
https://preview.redd.it/prwlaz7tce1i1.png?width=768&format=png&auto=webp&s=f33077813ca59ef6de82b8592d1d6047be7fa24e

https://preview.redd.it/drmh7g6uce1i1.png?width=768&format=png&auto=webp&s=580e0ae3491be3350b52e4d25da3401a03b4c0f2

https://preview.redd.it/36j4ttmuce1i1.png?width=768&format=png&auto=webp&s=1bffcf3d7f2a3e8c76f4ed3616521e621d790a3d

Hi r/LocalLLaMA, we're releasing Boole-32B-A2.5B-Preview, an early 1.8-bit version of Recursive's [flint](https://huggingface.co/RecursiveSI/flint). With mixed-precision quantization, expert pruning, and custom kernels (kernel changes in attached figure), we made the model 9x smaller than BF16, and 2x faster than INT4.

After releasing our 1-bit LLM (Boole-1.7B), we heard a lot of requests for both faster and larger MoE models - so this is our first step in that direction.

Try it out, and let us know what you'd like to see next!

* Weights: [https://huggingface.co/precision-numerics/Boole-32B-A2.5B-Preview-GGUF](https://huggingface.co/precision-numerics/Boole-32B-A2.5B-Preview-GGUF)
* Getting started: [https://github.com/PrecisionNumerics/boole-demo](https://github.com/PrecisionNumerics/boole-demo)
* Our llama.cpp fork: [https://github.com/PrecisionNumerics/llama.cpp](https://github.com/PrecisionNumerics/llama.cpp)
```



document fetched 2026-11-14 00:46 · canonical https://reddit.com/r/LocalLLaMA/comments/1xv3h93/boole32ba25bpreview_9gb_18bit_moe_565_tokens_on_1
