Tried running 748B DeepSeek-V4.1-Flash on a €2k PC
retained text
I bought a used PC with an RTX 3090 24 GB, 128 GB DDR4 and an Intel i9-9900X last year for under €2,000.
With the release of DeepSeek-V4.1-Flash-1017 I decided to try running it on my machine. It is of course the smallest quant available (1-bit), but I was still happy to see it working at **7.2 tok/s**.
The answer has a physics error in it (wondering if someone can spot it), and I don't see much practical use for it. Qwen models actually work pretty well on my setup. Still, it felt really nice to see a 748B parameter model running on such hardware.
Here are some technical details:
* Unsloth `UD-IQ1_S` GGUF, 233 GB on disk
* llama.cpp built from the open PR #30817, CUDA 12.8
* The 110 GB Engram table isn't loaded up front. It stays on the NVMe and only the rows the model needs are read on demand, which is how it fits.
​
llama-server -m DeepSeek-V4.1-Flash-1017-UD-IQ1_S-00001-of-00005.gguf \
-ngl 99 --n-cpu-moe 36 -ot "per_layer_token_embd.weight=CPU" \
-lm mmap -lzm on -t 10 -c 16384 --jinja \
--temp 1.0 --top-p 1.0 --top-k 0 --min-p 0.0 \
--chat-template-kwargs '{"enable_thinking":false}'
Now I only need to decide whether to keep the model or free up the disk space :)document fetched 2026-10-22 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xburkx/tried_running_748b_deepseekv41flash_on_a_2k_pc