← sources

Is it possible to configure TabbyAPI to be like Strata for Qwen 3.8 Flash Next exl3?

source: reddit (reddit:LocalLLaMA)published 2026-10-28 14:22ingested 2026-10-29 02:42

retained text

I've read that exl3 quants are even more accurate than Unsloth's (UD). There are exl3 versions of Q3.8FN here: [https://huggingface.co/turboderp/Qwen3.8-Flash-Next-exl3](https://huggingface.co/turboderp/Qwen3.8-Flash-Next-exl3) . TabbyAPI and Exllamav3 support moe cpu expert offloading already. I don't know if there are gguf optimizations that Strata uses that exl3 doesn't have yet. I can run UD Q4 at 70+ t/s. I tried configuring TabbyAPI with q4 and offloading, but I only got 20 t/s. Is it just a matter of configuring TabbyAPI for my setup? If so, can speeds be comparable to Strata?

Edit: TabbyAPI has a cpu\_moe\_split\_experts option: "Number of routed experts per MoE layer to run on the CPU, with the coldest experts' weights in system RAM. Unlike `cpu_moe_offload_layers` this splits every MoE layer instead of offloading whole layers, overlapping CPU and GPU compute, with dynamic placement keeping hot experts in VRAM." That's what I used for the 20 t/s. Idk if that's what strata does.

document fetched 2026-10-29 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xhpfmg/is_it_possible_to_configure_tabbyapi_to_be_like