Is it possible to configure TabbyAPI to be like Strata for Qwen 3.8 Flash Next exl3?
retained text
I've read that exl3 quants are even more accurate than Unsloth's (UD). There are exl3 versions of Q3.8FN here: [https://huggingface.co/turboderp/Qwen3.8-Flash-Next-exl3](https://huggingface.co/turboderp/Qwen3.8-Flash-Next-exl3) . TabbyAPI and Exllamav3 support moe cpu expert offloading already. I don't know if there are gguf optimizations that Strata uses that exl3 doesn't have yet. I can run UD Q4 at 70+ t/s. I tried configuring TabbyAPI with q4 and offloading, but I only got 20 t/s. Is it just a matter of configuring TabbyAPI for my setup? If so, can speeds be comparable to Strata? Edit: TabbyAPI has a cpu\_moe\_split\_experts option: "Number of routed experts per MoE layer to run on the CPU, with the coldest experts' weights in system RAM. Unlike `cpu_moe_offload_layers` this splits every MoE layer instead of offloading whole layers, overlapping CPU and GPU compute, with dynamic placement keeping hot experts in VRAM." That's what I used for the 20 t/s. Idk if that's what strata does.
document fetched 2026-10-29 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xhpfmg/is_it_possible_to_configure_tabbyapi_to_be_like