← sources

Now you can grow Bonsai on your potato

source: reddit (reddit:LocalLLaMA)published 2026-10-11 12:58ingested 2026-10-12 02:39

retained text

No more excuses for GPU-poor folks not to start LLMing!

# 

Full 27B-class reasoning in ternary transformer weights, for llama.cpp (CUDA, Metal, CPU)

>**\~9.3x** smaller than FP16 (ideal) | **98.2%** of FP16 intelligence retained | **\~47 tok/s** on an Apple M5 Max laptop

# Highlights

* **\~5.9 GB** language model (down from \~54 GB FP16) — full 27B-class reasoning on a standard laptop or a single GPU
* **98.2% of FP16 intelligence retained**: 84.78 average across 14 thinking-mode benchmarks — far above the conventional IQ2\_XXS build (72.59) at about 82% of its footprint, and within 0.4 points of UD-Q4\_K\_XL at three times the footprint
* **Retains thinking, reasoning, and agentic behavior** deep in the sub-4-bit regime, where conventional low-bit representations collapse: math within half a point of full precision (96.57), coding level with the baseline (89.42), agentic tool calling at 74.92
* **End-to-end ternary language weights** across embeddings, attention projections, MLP projections, and LM head, at a *true* 1.72 bits per weight — no high-precision escape hatches behind a low-bit label; the vision tower ships as a separate Q8\_0 mmproj pack
* **262K-token context** on-device, kept practical by the Qwen3.8-27B hybrid-attention backbone (\~75% linear attention)
* **Two GGUF packings** with custom ternary hybrid-attention kernels for llama.cpp (CUDA, Metal) — **PTQ1\_0** packs trits densely (1.75 bits/weight, 5.95 GB), **PQ2\_0** stores each trit in a 2-bit slot (2.13 bits/weight, 7.21 GB); packed weights are consumed directly, never expanded back to FP16

document fetched 2026-10-12 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1x37kxn/now_you_can_grow_bonsai_on_your_potato