any idea on how to deal with language confusion in gemma 4 ?
retained text
i'm using gemma 4 31b to do some long-text translation from chinese to vietnamese (hobby - not serious work). the biggest issue i'm having is language confusion (language contamination), i.e. output sometimes has non-latin characters not present in original text: korean, arabic, cyrillic, chinese, etc. - even q8 weight and fp16 kv cache won't help, same for sampling parameters (temp, top k, top p, etc.) the rate is like <0.5% of output text but it disrupts reading flow, so for each contamination (a simple spellcheck can identify all), i have to look up the original text and use google translate to replace the contamination. it seems to me the contaminations often occur at some very specific chinese characters. a simple mapping may work but i need to update it every time i detect one, and then when gemma 5 come out (hopefully) i need to make new one. i don't setup a pipeline or hardness or anything, because 31b is painfully slow, running it again as proofreader takes too much time, a simple run already produces translated text good enough for me **i'm looking for a more robust way to deal with this issue** at first i tried LLguidance with a huge regex to restrict the model ouput to a specific unicode range, but the model get stuck in like forever i read about vocabulary trimming but it seems to be nore suitable for small models, not for a big 31b, also the repo seem abandoned i found another method called token masking: identify unwanted tokens and make their logits extremely low by editing \`lm\_head\` in the weight, u can see here: https://huggingface.co/ThakiCloud/Qwen3.8-27B-vi-cjk-suppressed the above method is for qwen 3.8, unfortunately gemma 4 has \`lm\_head\` tied to \`embed\_tokens\`, so i tried to adapt to gemma 4 \`embed\_tokens\` but it doesn't have any effect do u guys ever get this problem and know how to deal with this ?
document fetched 2026-11-15 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xw31vx/any_idea_on_how_to_deal_with_language_confusion