# RustingStrata runs Flint 40B on a 12 GB card: 64 tok/s decode, 1.3x llama.cpp — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/16e6df8eea895c88

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-14 15:51 ingested 2026-11-15 02:36

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xw5nzn/rustingstrata_runs_flint_40b_on_a_12_gb_card_64/)

## retained text



```
I tested new trending Recursive Flint UD-Q5\_K\_XL (27.9 GiB) on an RTX 3060 12 GB with 30 GB RAM, comparing RustingStrata against llama.cpp using the same model, prompt, and generation settings.

||RustingStrata|llama.cpp|
|:-|:-|:-|
|Warm decode speed|**64.6 tok/s** (63.8–65.6)|48.6 tok/s (47.5–49.3)|
|Warm 4K prompt prefill|1.14–1.30k tok/s|0.85–1.02k tok/s|

RustingStrata was about **33% faster once warm**. A fresh load can behave differently because this model is larger than available VRAM and performance depends on which model pages are already cached in RAM.  
  
What made the difference?

The model doesn't fit in 12 GB of VRAM, so many expert weights must be fetched from system RAM during generation. Two changes helped most:

* **Adapt expert placement to the reply**, rather than continuing to prioritize the prompt's routing pattern.
* **Release RAM pages for experts already copied to VRAM**, preserving cache for experts that still need to be fetched from host memory.

About Quality, bc people very often says like "It should give shit output to gain speed", which is not true, in my last posts I already did quality benchmarks and RustingStrata does not lose quality, it gain speed through specific Code for some specific models, in same time llama.cpp lose speed to be "general" engine that can run everything

What I want to know, is do people find Recursive Flint an interesting model? I found this model on OpenRouter and looks pretty interesting, it's better then Qwen3.6:35-a3b, so I think this can be cool modern replacement for this qwen.  
  
Setup:

* **RustingStrata:** 1.2.2, commit `3902978` (working tree)
* **llama.cpp:** b12191, commit `e95426f16`
* Full benchmarks: `docs/BENCHMARKS.md`

Repository: [RustingStrata](https://github.com/RustingStudio/RustingStrata)
```



document fetched 2026-11-15 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xw5nzn/rustingstrata_runs_flint_40b_on_a_12_gb_card_64
