# Qwen3.8-Flash-Next on 12GB VRAM - 65 tokens per second — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/3e9ee121cee266cb

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-09-24 17:30 ingested 2026-09-25 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1wp7zyb/qwen38flashnext_on_12gb_vram_65_tokens_per_second/)

## retained text



```
A while ago I posted 15 tok/s output and 100-120 tok/s prompt processing with the IQ3\_XXS quant on a 12GB RTX 5070 using llama.cpp. Since then I built my own inference engine for this one model and this kind of PC. The same IQ3\_XXS now runs at **\~65 tok/s output** and **\~430 tok/s prompt processing**, and the 2-bit quants run faster still using RCO-GSQ quantization.

**Using:**

64GB DDR5 (5600)  
12GB RTX 5070 SFF (Gigabyte)  
Ryzen 5 7600 CPU  
Windows

**Output (tokens/s) on 128K context:**

Q2\_0 (equivalent to unsloth Q3): 65.1  
IQ2\_XS (equivalent to unsloth Q4): 52.0  
IQ3\_XXS (equivalent to unsloth Q5): 44.8

**Prompt processing (tokens/s) on 128K:**

Q2\_0: 543  
IQ2\_XS: 472  
IQ3\_XXS: 414

Requirements:

Q2\_0 = 37.6GB minimum in RAM+VRAM

IQ2\_XS = 39.2GB minimum in RAM+VRAM

IQ3\_XXS = 47GB minimum in RAM+VRAM

Vision encoder = 0.91GB additionally

You can now one click install and run the engine with low cost hardware (currently only optimized for CUDA).

GitHub: [https://github.com/Niko1221/Strata](https://github.com/Niko1221/Strata)

Model: [https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)
```



document fetched 2026-09-25 00:45 · canonical https://reddit.com/r/LocalLLaMA/comments/1wp7zyb/qwen38flashnext_on_12gb_vram_65_tokens_per_second
