Comparing 4bit quants for MLX
retained text
<!-- SC_OFF --><div class="md"><p>Curious what people think are the ideal 4-bit quantization types on MLX</p> <p>These quants seem to be the most popular, at least for Gemma4 and Qwen3.6:<br /> - OptiQ 4bit (<a href="https://huggingface.co/mlx-community/Qwen3.6-27B-OptiQ-4bit">mlx-community/Qwen3.6-27B-OptiQ-4bit</a>)<br /> - Unsloth dynamic 2.0 MLX (<a href="https://huggingface.co/unsloth/Qwen3.6-27B-UD-MLX-4bit">unsloth/Qwen3.6-27B-UD-MLX-4bit</a>)<br /> - oQ (<a href="https://huggingface.co/Jundot/Qwen3.6-27B-oQ4e-mtp">Jundot/Qwen3.6-27B-oQ4e-mtp</a>)*<br /> - DWQ (can't find an example fo this one)<br /> - native (<a href="https://huggingface.co/mlx-community/Qwen3.6-27B-4bit">mlx-community/Qwen3.6-27B-4bit</a>)</p> <p>Does anyone have any insight here?</p> <p>*edited to adjust to newer version</p> </div><!-- SC_ON -->   submitted by   <a href="https://www.reddit.com/user/True_Tangerine_4706"> /u/True_Tangerine_4706 </a> <br /> <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1vj7i17/comparing_4bit_quants_for_mlx/">[link]</a></span>   <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1vj7i17/comparing_4bit_quants_for_mlx/">[comments]</a></span>
document fetched 2026-08-09 07:00 · canonical https://reddit.com/r/LocalLLaMA/comments/1vj7i17/comparing_4bit_quants_for_mlx