← sources

Need help fixing Gemma 4 26B A4B toolcalls in Pi

source: reddit (reddit:LocalLLaMA)published 2026-09-20 10:58ingested 2026-09-21 02:38

retained text

Hello! So, yesterday i tried out Gemma 4 26B A4B QAT Q4\_K\_XL [from here](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF) with Q4\_0 MTP model in Pi agent. Suddenly, it became the first model to actually pass my simple test with HTTP server in C++, while being small enough to fit 128K ctx at FP16. It easily oneshotted the task, tested, and it worked, all of that in just 5 minutes! It was hella impressive, and i decided to push further.

It started experiencing many issues with toolcalling. It fails to submit the plan to Pi. It fails to edit or write the file because it mixes up the arguments or forgets some. It became really annoying when it got to \~30K of context, and i decided to stop.

What can I do to improve toolcalling success rate?

CLI flags for llama-server:

    build/bin/llama-server \
        -m ~/d/models/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf \
        -c 131072 \
        -ngl all \
        --cpu-moe \
        --no-host \
        -np 1 \
        --cache-ram 0 \
        -fit off \
        -lv 4 \
        --no-webui \
        --no-mmproj \
        --spec-type draft-mtp \
        -md ~/d/models/mtp-gemma-4-26B-A4B-it-Q4_0.gguf \
        --spec-draft-n-max 3 \
        --n-gpu-layers-draft 99 \
        --load-mode none \
        -ub 4096 \
        --temp 1.0 \
        --top-k 64 \
        --top-p 0.95 \
        --min-p 0.05 \
        --repeat-penalty 1.1 \
        --repeat-last-n 256 \
        --reasoning-budget 12288

It also repeated a lot, so i had to apply recommended settings. I check chat template, and looks like Unsloth applied fixed template that was released on July.

document fetched 2026-09-21 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1wle1f4/need_help_fixing_gemma_4_26b_a4b_toolcalls_in_pi