← sources

boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF · Hugging Face

source: reddit (reddit:LocalLLaMA)published 2026-10-26 07:49ingested 2026-10-27 02:40

retained text

>Ternary Nemotron 30B-A3B — 368K Context on 12 GB GPU (GGUF)

>A ternary (1.58-bit) 30B-A3B MoE model that runs on a 12 GB GPU with 368K context, 172 tok/s decode and 7000+ tok/s prefill — in both Stock **llama.cpp** and the PrismML fork. One file, both engines.

|Engines|✅ Stock **llama.cpp** (validated on b11555+, b11796) · ✅ **PrismML fork / Bonsai** (validated on b10793)|
|:-|:-|


**EDIT**: This is the 2nd Ternary MOE after [maple-preview](https://huggingface.co/deepgrove/maple-preview-GGUF) which works with stock llama.cpp. Somebody please share CPU-only numbers. Also share numbers of tiny VRAMs(\~10GB)

document fetched 2026-10-27 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1xfqi2d/boro64ternarynemotron35lightning30ba3bgguf