For long context does high quant matter?
retained text
Question: if I'm regularly at higher context - say 300-400k - do higher quant versions of the same model perform better? I'm thinking something like Qwen Flash Next q6 and q8. I recently swapped from q6 to q8 and performed a/b testing. I originally expected q6 to be lossless but there were areas where q6 got the wrong answer while q8 was fine. Was wondering whether this is because of my context. Maybe more context = more tokens for divergence = the differences get amplified more than benchmarks suggest
document fetched 2026-11-15 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xvud3r/for_long_context_does_high_quant_matter