ISTA Qwen3.8 27b on 3060 12gb
retained text
Hey guys I am a seasoned local llm user and I ain't quite ram rich like only 16gb ddr4 🥀 So I can't run qwen3.8 flash next on my 3060 This guy felt the best for me yet Have you guys tried it I have ran iq3\_xss quant non mtp with turboquant at 65k context completely on my 3060 though I did offload mmproj to cpu ram Have guys tried this it really felt like a blessing to my life using the peak of local ai on my 3060 It seriously packs the punch 👊 I was able to perform various of my agent task complete on device not like simple but the ones which require actual reasonings Here is my cmd if you want to see llama-tq serve -m "E:\\MODELS\\ISTA-DASLab\\Qwen3.8-27B-GSQ-RCO-GGUF\\Qwen3.8-27B-GSQ-RCO-IQ3\_XXS.gguf" --mmproj "E:\\MODELS\\ISTA-DASLab\\Qwen3.8-27B-GSQ-RCO-GGUF\\mmproj-Qwen3.8-27B-BF16.gguf" --no-mmproj-offload -ngl 99 -c 65536 -ctk turbo4 -ctv turbo3 -ub 256 -b 2048 -t 3 -tb 3 --reasoning-budget 256 --reasoning-preserve --alias Qwen --port 8080 -lm none Sources: for my recipe ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF/discussions Turboquant https://github.com/TheTom/llama-cpp-turboquant Pi coding agent https://pi.dev/ I would like know the communities views and any suggestions would be welcome 🤗 Thankyou for your time
document fetched 2026-11-16 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1xwx7uq/ista_qwen38_27b_on_3060_12gb