No wonder Qwen and Gemma are so different
retained text
<!-- SC_OFF --><div class="md"><p>Pasted the same HTML/JS code (330 lines) into Qwen 35B A3B and Gemma 26B A4B.</p> <p>Qwen: tokenized the input to 1609 tokens</p> <p>Gemma: tokenized the input to 4258 tokens.</p> <p>Damn. I've never noticed this before and I haven't seen people mention it. That alone helps explain why Qwen is regarded as better at coding and Gemma at language tasks.</p> <p>Qwen can literally see the code as some specific form of input/output, while Gemma is breaking it down into pieces of words like regular language. Qwen also gets a totally different reasoning personality when given coding tasks.</p> <p>Btw with the instruction document (55 lines), the tokenization breakdown is almost the same: 1025 vs. 1039 tokens.</p> <p>I've seen some project, by LiquidAI I think? To retrain existing models with a more efficient tokenizer. I wonder what that would do for a model like Gemma, whether it would help it catch up.</p> </div><!-- SC_ON -->   submitted by   <a href="https://www.reddit.com/user/WhoRoger"> /u/WhoRoger </a> <br /> <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1vjb15v/no_wonder_qwen_and_gemma_are_so_different/">[link]</a></span>   <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1vjb15v/no_wonder_qwen_and_gemma_are_so_different/">[comments]</a></span>
document fetched 2026-08-09 07:00 · canonical https://reddit.com/r/LocalLLaMA/comments/1vjb15v/no_wonder_qwen_and_gemma_are_so_different