RustingStrata vs llama.cpp, DeepSeek-V4.1-Flash Q2 (249 GB) on a single RTX 3060 12 GB with 30 GB RAM
retained text
**RustingStrata now runs DeepSeek-V4.1-Flash (249 GB) on a single RTX 3060 12 GB with 30 GB RAM**
RustingStrata is a pure-Rust GGUF inference engine with CUDA and Vulkan backends. It now loads DeepSeek-V4.1-Flash at IQ2\_XXS (249 GB) and runs it on a 12 GB card with 30 GB of RAM.
The experts stream from SSD through the page cache. A 9 GB LRU cache in VRAM holds the experts that get reused. Everything else (attention, shared matvecs) runs on the GPU.
||RustingStrata|llama.cpp b12116 (`--cpu-moe`)|
|:-|:-|:-|
|Decode|2-3.6 tok/s|0.3-1.5 tok/s|
|Prefill|\~2 tok/s short prompts, \~0.5 tok/s at 1400 tokens|0.5-0.9 tok/s|
|Top-1 agreement with llama.cpp (teacher-forced)|189/192 on short prompts, 21/23 on a 1425-token prompt|\-|
|Sampling, tool calls|not yet|yes|
Same box for both: RTX 3060 12 GB, 30 GB RAM, 12 cores.
Caveats:
* Decode speed varies a lot from run to run with the page cache, because the model is much bigger than RAM.
* Greedy decoding only. The context cap is 4096.
Usage:
strata chat --model $M
strata generate --model $M --prompt "The capital of France is" -n 64
strata serve --model $M --addr 127.0.0.1:8080
`serve` exposes `/v1/chat/completions` with JSON and SSE streaming.
Repo: [RustingStrata](https://github.com/RustingStudio/RustingStrata) (the DeepSeek section is in the README)document fetched 2026-11-10 00:41 · canonical https://reddit.com/r/LocalLLaMA/comments/1xrw6uv/rustingstrata_vs_llamacpp_deepseekv41flash_q2_249