← sources

Finally got qwen 3.8 q4 k_M 27b runnin slick 40tkps load at 240k inf 4qnl for stable agentic coding on a 7900xtx. I have been finally able to give hermes a task and come back to results. Built a panel to manage my inference servers from hermes.

source: reddit (reddit:LocalLLaMA)published 2026-09-19 19:44ingested 2026-09-20 02:38

retained text

[https://github.com/W61k3r/LexiPanel](https://github.com/W61k3r/LexiPanel) for those interested in the panel  
  
Screenshot of heavy load. I did offload vision to a spare 2060 instead of local ram. The control panel was generated with qwen3.8-27b davidau q4k\_m max mtp. Made the panel from scratch over a week, basically just building for speed. Results int he picture are from a live deep coding job.

old pc intel 9700, 32gb ddr2400, 7900xtx, 2060, 850 watt psu, ubuntu, llama.cpp, hermes on a rasp pi. Speed is more or less the same with a 2060 running the vision mmproj instead of local cpu offload so I set up a 2nd instance to run from that card.

Prefill 900ish hard decode 40ish

    /home/admin/llama/b11011-vulkan/llama-b11011/llama-server -m /home/admin/models/Qwen3.8-27B-TurboFCFusion-735-882-Here-Uncen-NEO-CODER-MAX-MTP-Q4_K_M.gguf --device Vulkan0,Vulkan1 --split-mode layer --tensor-split 1,0 --mmproj /home/admin/models/mmproj-F16.gguf --image-min-tokens 1024 --mmproj-device Vulkan1 -c 240128 -np 1 --n-gpu-layers 99 --flash-attn on --cache-type-k iq4_nl --cache-type-v iq4_nl --cache-ram 5631 --cache-reuse 256 --jinja --chat-template-file /dev/shm/llama_qwen38/qwen3.8-safe-v2.jinja --chat-template-kwargs {"reasoning_effort":"xhigh","preserve_thinking":true,"max_tool_response_chars":9000} --reasoning-preserve --reasoning on --reasoning-format deepseek --reasoning-budget -1 --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0 --temp .77 --top-p 0.95 --top-k 20 --min-p 0.0 --metrics --no-warmup --host 0.0.0.0 --port 8081 --slot-save-path /dev/shm/llama_qwen38/slots --log-file /dev/shm/llama_qwen38/telemetry/engine_debug.log --batch-size 2048 --ubatch-size 512 --threads 6 --threads-batch 6 --n-predict 32000 --cont-batching /home/admin/llama/b11011-vulkan/llama-b11011/llama-server -m /home/admin/models/Qwen3.8-27B-TurboFCFusion-735-882-Here-Uncen-NEO-CODER-MAX-MTP-Q4_K_M.gguf --device Vulkan0,Vulkan1 --split-mode layer --tensor-split 1,0 --mmproj /home/admin/models/mmproj-F16.gguf --image-min-tokens 1024 --mmproj-device Vulkan1 -c 240128 -np 1 --n-gpu-layers 99 --flash-attn on --cache-type-k iq4_nl --cache-type-v iq4_nl --cache-ram 5631 --cache-reuse 256 --jinja --chat-template-file /dev/shm/llama_qwen38/qwen3.8-safe-v2.jinja --chat-template-kwargs {"reasoning_effort":"xhigh","preserve_thinking":true,"max_tool_response_chars":9000} --reasoning-preserve --reasoning on --reasoning-format deepseek --reasoning-budget -1 --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0 --temp .77 --top-p 0.95 --top-k 20 --min-p 0.0 --metrics --no-warmup --host 0.0.0.0 --port 8081 --slot-save-path /dev/shm/llama_qwen38/slots --log-file /dev/shm/llama_qwen38/telemetry/engine_debug.log --batch-size 2048 --ubatch-size 512 --threads 6 --threads-batch 6 --n-predict 32000 --cont-batching 

document fetched 2026-09-20 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1wkvl7h/finally_got_qwen_38_q4_k_m_27b_runnin_slick