← sources

Ternary-Bonsai-2-27B TQ1_0 or Gemma 4 12B it QAT Q4_K_XL?

source: reddit (reddit:LocalLLaMA)published 2026-10-30 07:09ingested 2026-10-31 02:40

retained text

One of my setups (a laptop) includes an **Nvidia 4060 Mobile with 8 GB VRAM**.

I am using [Unsloth's Gemma 4 12B it QAT Q4\_K\_XL](https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF) on it.

I ran a quick test with the TQ1\_0 version of [Ternary-Bonsai-2-27B](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf) with [their llama.cpp port](https://github.com/PrismML-Eng/llama.cpp), IMHO it seemed promising to use with my [custom voice assistant](https://github.com/RhinoDevel/poldi).

Unfortunately it is [not yet supported](https://github.com/ggml-org/llama.cpp/issues/29058) by the official llama.cpp, if I understand correctly.

Is anybody using that TQ1\_0 quant of Ternary-Bonsai-2-27B or did some more extensive testing?

Is it not worth it, anyway? What's your opinion / are your test results?

document fetched 2026-10-31 00:45 · canonical https://reddit.com/r/LocalLLaMA/comments/1xj8kuj/ternarybonsai227b_tq1_0_or_gemma_4_12b_it_qat_q4