boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF · Hugging Face
retained text
>Ternary Nemotron 30B-A3B — 368K Context on 12 GB GPU (GGUF) >A ternary (1.58-bit) 30B-A3B MoE model that runs on a 12 GB GPU with 368K context, 172 tok/s decode and 7000+ tok/s prefill — in both Stock **llama.cpp** and the PrismML fork. One file, both engines. |Engines|✅ Stock **llama.cpp** (validated on b11555+, b11796) · ✅ **PrismML fork / Bonsai** (validated on b10793)| |:-|:-| **EDIT**: This is the 2nd Ternary MOE after [maple-preview](https://huggingface.co/deepgrove/maple-preview-GGUF) which works with stock llama.cpp. Somebody please share CPU-only numbers. Also share numbers of tiny VRAMs(\~10GB)
document fetched 2026-10-27 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1xfqi2d/boro64ternarynemotron35lightning30ba3bgguf