← sources

Hy4-Flash is great and finally some model is competing with DSv4-Vision-Exp and Qwen3.8-27B-Uncensored !

source: reddit (reddit:LocalLLaMA)published 2026-11-18 23:06ingested 2026-11-19 02:39

retained text

It is going to be my goto model, I guess. Despite being slower, it finishes answers with less tokens. I've ran it on my benchmark, it is far from finished, but it is on the way on topping it:

[https:\/\/beta.locallm.top\/home?apples\_to\_apples=1&include\_unavailable=0&template\_limit=15](https://preview.redd.it/9kvhpxntbg2i1.png?width=904&format=png&auto=webp&s=c628a2f25ab4b520f068d3b0c9f5dcbdd3d703b8)

My 2xRTX3090 + 128 GB DDR4 RAM presets file excerpt (used in llama-server router mode):

    [Hy4-Flash-Q3]
    load-on-startup = true
    hf = AngelSlim/Hy4-flash-GGUF:Q3_K
    ctx-size = 131072
    ctk = q8_0
    ctv = q8_0
    no-mmproj-offload = true
    fit = off
    cmoe = true
    moe-cache-mib = 33768
    load-mode = none

Which translates into llama-server command

`./build/bin/llama-server -hf AngelSlim/Hy4-flash-GGUF:Q3_K --ctx-size 131072 -ctk q8_0 -ctv q8_0 --no-mmproj-offload -cmoe --moe-cache-mib 33768 -load-mode none`

If you wonder what is the `--moe-cache-mib 33768`, then it is expert streaming (llama.cpp feature inspired by Strata I guess).

I get PP \~60-65 tps, TG \~ 20-24 tps.

    Nov 19 00:56:05 builder llama-server[1270042]: [54519] 397.49.817.617 I slot print_timing: id  0 | task 442246 | prompt processing, n_tokens =   4091, progress = 1.00, t =  63.38 s / 64.54 tokens per second
    Nov 19 00:56:10 builder llama-server[1270042]: [54519] 397.54.518.918 I slot print_timing: id  0 | task 442246 | n_gen =    100, tg =  21.95 t/s, tg_3s =  22.18 t/s

I am using [https://huggingface.co/AngelSlim/Hy4-flash-GGUF](https://huggingface.co/AngelSlim/Hy4-flash-GGUF) Q3\_K quant and llama.cpp PR [https://github.com/ggml-org/llama.cpp/pull/32517](https://github.com/ggml-org/llama.cpp/pull/32517) , it rebased on the master fine.

document fetched 2026-11-19 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1xztpmh/hy4flash_is_great_and_finally_some_model_is