← sources

Basalt 0.4.0: More models, still as fast - Qwen 27B up to 770 tok/s on a 5090

source: reddit (reddit:LocalLLaMA)published 2026-11-15 17:30ingested 2026-11-16 02:38

retained text

Hi everyone! I wanted to post a quick update on [Basalt](https://basalt.jgulfo.com/). Since last I posted, multi model and router support was implemented, starting with DeepSeek V4.1 Flash (IQ1\_M 90 tok/s conv, 246 tok/s struct on a 5090 + 5060 Ti). Also new: video support, files in chats, and a bunch of fixes and improvements.

Now 0.4.0 is out featuring Basalt's first dense model, Qwen 3.8 27B NVFP4. It's **fast** and accurate, top-1 92.8% vs the full BF16 model. This, in preparation for Qwen 4, so we had a headstart on the dense kernels.

The numbers:

* Up to 776 tok/s on structured output, 262 conversation, 194 on prose. Single stream.
* Prefill, 12,000 tok/s at 4k (slower as context grows)
* 92.8% top-1 vs BF16 (0.0262 KLD)
* Chat, tools, thinking, images and video, up to 262K context

Links:  
  
GitHub: [https://github.com/jesdga95/basalt](https://github.com/jesdga95/basalt)  
Site: [https://basalt.jgulfo.com](https://basalt.jgulfo.com/)  
27B quant: [https://huggingface.co/jesdga/Qwen3.8-27B-Basalt](https://huggingface.co/jesdga/Qwen3.8-27B-Basalt)

Thanks to everyone who has filed issues, sent PRs or posted benchmark rows, special thanks to the sponsors who keep this going, happy to see Basalt gain so much traction in such a very short time!

document fetched 2026-11-16 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1xx1cu9/basalt_040_more_models_still_as_fast_qwen_27b_up