Hy4-Flash is great and finally some model is competing with DSv4-Vision-Exp and Qwen3.8-27B-Uncensored !
retained text
It is going to be my goto model, I guess. Despite being slower, it finishes answers with less tokens. I've ran it on my benchmark, it is far from finished, but it is on the way on topping it:
[https:\/\/beta.locallm.top\/home?apples\_to\_apples=1&include\_unavailable=0&template\_limit=15](https://preview.redd.it/9kvhpxntbg2i1.png?width=904&format=png&auto=webp&s=c628a2f25ab4b520f068d3b0c9f5dcbdd3d703b8)
My 2xRTX3090 + 128 GB DDR4 RAM presets file excerpt (used in llama-server router mode):
[Hy4-Flash-Q3]
load-on-startup = true
hf = AngelSlim/Hy4-flash-GGUF:Q3_K
ctx-size = 131072
ctk = q8_0
ctv = q8_0
no-mmproj-offload = true
fit = off
cmoe = true
moe-cache-mib = 33768
load-mode = none
Which translates into llama-server command
`./build/bin/llama-server -hf AngelSlim/Hy4-flash-GGUF:Q3_K --ctx-size 131072 -ctk q8_0 -ctv q8_0 --no-mmproj-offload -cmoe --moe-cache-mib 33768 -load-mode none`
If you wonder what is the `--moe-cache-mib 33768`, then it is expert streaming (llama.cpp feature inspired by Strata I guess).
I get PP \~60-65 tps, TG \~ 20-24 tps.
Nov 19 00:56:05 builder llama-server[1270042]: [54519] 397.49.817.617 I slot print_timing: id 0 | task 442246 | prompt processing, n_tokens = 4091, progress = 1.00, t = 63.38 s / 64.54 tokens per second
Nov 19 00:56:10 builder llama-server[1270042]: [54519] 397.54.518.918 I slot print_timing: id 0 | task 442246 | n_gen = 100, tg = 21.95 t/s, tg_3s = 22.18 t/s
I am using [https://huggingface.co/AngelSlim/Hy4-flash-GGUF](https://huggingface.co/AngelSlim/Hy4-flash-GGUF) Q3\_K quant and llama.cpp PR [https://github.com/ggml-org/llama.cpp/pull/32517](https://github.com/ggml-org/llama.cpp/pull/32517) , it rebased on the master fine.document fetched 2026-11-19 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1xztpmh/hy4flash_is_great_and_finally_some_model_is