← sources

What are you using for NVFP4 and Do you like it?

source: reddit (reddit:LocalLLaMA)published 2026-10-08 18:38ingested 2026-10-09 02:39

retained text

What are people using to run nvfp4 on multi-gpu?

>the only thing i can get to run is unsloth and vllm is too slow and takes FOREVER to fucking load. TensorRT-LLM has too many issues, so what are  people using?

And do you like nvfp4 vs q4 qguf for qwen 3.8? apples to oranges is nvfp4 closer to Q6 gguf than q4 gguf is?

>note: im not interested in nvfp4-2-gguf conversion

>it seems like the barrier to using nvfp4 is quite thick

document fetched 2026-10-09 00:45 · canonical https://reddit.com/r/LocalLLaMA/comments/1x0yxrf/what_are_you_using_for_nvfp4_and_do_you_like_it