← sources

For long context does high quant matter?

source: reddit (reddit:LocalLLaMA)published 2026-11-14 05:32ingested 2026-11-15 02:36

retained text

Question: if I'm regularly at higher context - say 300-400k - do higher quant versions of the same model perform better? I'm thinking something like Qwen Flash Next q6 and q8. I recently swapped from q6 to q8 and performed a/b testing. I originally expected q6 to be lossless but there were areas where q6 got the wrong answer while q8 was fine. Was wondering whether this is because of my context. Maybe more context = more tokens for divergence = the differences get amplified more than benchmarks suggest

document fetched 2026-11-15 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xvud3r/for_long_context_does_high_quant_matter