# An updated 2.3bpw model for qwen 27b + 12 GB cards: ties Qwen3.6 BF16 on LiveCode Bench and reduces error by 37-57% vs a previous checkpoint — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/dd2fe754ee877799

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-01 21:19 ingested 2026-11-02 02:41

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xlerw1/an_updated_23bpw_model_for_qwen_27b_12_gb_cards/)

## retained text



```
Hello everyone! A few weeks ago, I released a 2.3bpw model intended for the 30xx 12 GB Ampere cards. Using a custom version of sjkvarn and several runtime changes, this made it possible to hit 200K ctx on a single 12GB card. Benchmarking placed it somewhere around qwen3.5 27b, but below 3.6.

I've continued to work on quants for this size card and am happy to share that an updated checkpoint is now performing at Qwen3.6 BF16 levels on Livecode bench, **with minimal post training (<20M tokens) to reduce the risk of overfitting**. This was possible with the new quant format I recently added for [LlamAmpere](https://github.com/JakeATX/llamAmpere) (AmperInteger+), which uses fraction 2-3bpw (e.g., 2.5) formats + trellis-based encoding to more efficiently distribute mass across the weights (this was then paired with an RCO-like optimization).

**On LCB, across three runs, it scored 84.3% with a strict grader** (88% with a more lenient fixed variant), vs the previous checkpoint's 75% and 3.8 BF16's score of 90.3%. For reference, **the BF16 version of qwen3.6 27b scored 83.9%**. It represents nearly a 60% reduction in LCB degradation from the last release. All versions were evaluated under xhigh reasoning.

**The original checkpoint scored a KLD of 0.096 nats, which was usable for agentic and lightweight coding tasks, but clearly degraded. By comparison, this version was able to cut KLD down to 0.059**, an error reduction of nearly 40% (across a mix of agentic, RAG, and coding measurements). As expected, **this reduction in error also significantly reduced answer verbosity** relative to the previous checkpoint (median answer of 4.5K tokens vs 6K before). Prior to this checkpoint, the Mirai 2.5bpw model was the best small quant of Qwen I had found; at nearly 10% smaller, this variant has 20% lower KLD, despite the more compressed KV cache. **This model uses an upgraded embedding table which should be stored in standard RAM; the on-card size is 7.4GiB, including the MTP head.** This continues to make 200k+ ctx possible using SJKVaRN in the 3/2 configuration (check the small card recommended configs for more details).

https://preview.redd.it/fsoek3wai2zh1.png?width=416&format=png&auto=webp&s=59a72d6f1a136ca5577c606e5500831119d45285

[](https://preview.redd.it/an-updated-2-3bpw-model-for-qwen-27b-12-gb-cards-ties-qwen3-v0-4v33kbbpf2zh1.png?width=416&format=png&auto=webp&s=737a7ac59c454f2cd237a5aa7992523a7bac1248)

Like the last checkpoint, the base was the Swift-1.5 fine tune, which reduced thinking tokens 50% or more in some settings vs the original version.

Testing this in hermes personally the past couple of days, it is very pleasant to use and very fast for its size (on a 3090/80ti, well over 90 tps for agentic work). I do recommend taking the latency penalty of this at xhigh to get the maximum performance, especially since it is very fast on TPS. This will get you near the 3.8 BF16 performance on medium (86.5% LCB), though it is definitely still usable on medium.

https://preview.redd.it/zoqis2eci2zh1.png?width=1080&format=png&auto=webp&s=58312b2fbafa2e7a83a63e50bfa4f0d28048081a

[](https://preview.redd.it/an-updated-2-3bpw-model-for-qwen-27b-12-gb-cards-ties-qwen3-v0-iwb3285wb2zh1.png?width=1686&format=png&auto=webp&s=af32b1fa6080da3fc424e640c46136b2dfe4b58a)

As before, it is MIT license. You'll need to update to the most recent version of llamAmpere to run, given the new quant formats from the 0.6 version.

repo: [https://github.com/JakeATX/llamAmpere](https://github.com/JakeATX/llamAmpere)

model: [https://huggingface.co/jakeatx/qwen3.8-27b-Swift1.5-Uncensored-ATX-AI2](https://huggingface.co/jakeatx/qwen3.8-27b-Swift1.5-Uncensored-ATX-AI2)

Please keep the user feedback + results coming.

Enjoy!
```



document fetched 2026-11-02 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xlerw1/an_updated_23bpw_model_for_qwen_27b_12_gb_cards
