← sources

DeepSeek-V4.1-Flash-1017 on one RTX 3090: Big updates to the Delta Inference Engine v0.93. Same 41 tok/s at 192GB/IQ2 at better quality, usable from 64 GB RAM, first token ~40% sooner, new Unsloth quants at full speed, EXL3, basic multi-GPU, public CI and more

source: reddit (reddit:LocalLLaMA)published 2026-11-05 18:23ingested 2026-11-06 02:39

retained text

Big update to the Delta Inference Engine, you guys asked for more features, more benchmarks and less clickbait. I'm here to deliver

**The short version:** I've added almost everything requested and more. Delta Inference Engine now supports 64/96/128 GB of RAM and fully supports the new Unsloth quants. Unsloth IQ2 runs at the same 41 tok/s (192 GB RAM) using a new native reader for the Engram tables and some clever CPU kernel work. Compared to the old default model on my 99-question benchmark it went from 52 to 57 correct, against 59 for the full model. It's no full deepseek model quant, but they're a lot more usable than you might expect.

**Speed on one RTX 3090 24 GB** (i5-13600K, 4x48 GB DDR5-5600, PCIe 4.0 NVMe, Windows, no WSL). Writing speed, tok/s, greedy, one chat at a time:

|RAM|UD-IQ2\_XXS, 2-bit, 301 GB|UD-IQ1\_S, 1-bit, 233 GB|UD-Q2\_K\_XL, 2-bit, 335 GB|UD-IQ3\_XXS, 3-bit, 395 GB + the default|
|:-|:-|:-|:-|:-|
|192 GB|41|41.0|31.3|28.2|
|128 GB|34.0|40.6|26.6|22.8|
|96 GB|29.3|36.5|22.9|18.3|
|64 GB|21.0|30.6|15.2|11.7|

The 128, 96 and 64 GB rows are the same PC with the rest of the RAM locked away.

**Prompt reading:** long prompts read at up to 740 tok/s at 192 GB and 600 at 128 GB (at \~64K tokens). A short prompt starts answering in about 2 s.

**Benchmark:** 99 hard questions, greedy, 43K token budget:

|Set|UD-IQ2\_XXS (this release)|ds4 Q2 (last release)|Full model, API|Qwen 3.8 Flash-Next, API|
|:-|:-|:-|:-|:-|
|Knowledge (40)|12|9|13|7|
|Maths (14)|10|10|10|8|
|Reasoning (30)|25|23|25|21|
|Coding (10)|10|10|10|8|
|Hard coding (5)|0|0|1|0|
|**Total (99)**|**57**|**52**|**59**|**44**|

UD-IQ2\_XXS is within 2 points of the full API, never scores below ds4 Q2 on any set while running at the same speed, and used 11% fewer tokens across the run.

**Agreement with DeepSeek's API on the next token, at the settings that ship:**

|Model|General text|Code and maths|Agentic|
|:-|:-|:-|:-|
|UD-IQ2\_XXS|91.1%|94.5%|95.9%|
|ds4 Q2|89.9%|93.1%|95.1%|
|UD-Q2\_K\_XL|92.5%|95.3%|97.0%|
|UD-IQ3\_XXS|92.7%|95.9%|97.4%|
|UD-IQ1\_S|89.1%|91.5%|95.3%|

For reference, two hosts of the full model agreed 98.3% and 98.9% in my test. By that measure Unsloth IQ2 is about 20% closer to the full model than the old ds4 Q2.

If you want better quality: UD-Q2\_K\_XL (335 GB) runs 31 tok/s at 192 GB and 26.6 at 128 GB, at 92.5% / 95.3%. UD-IQ3\_XXS (395 GB plus the default) runs 28 and 22.8 at 92.7% / 95.9%.

**What changed since last time:**

* **First token \~40% sooner:** a \~180-token prompt went from about 3.8 s to 2.3 s.
* **Agent turns read \~12% faster**, and tool calls that low-bit models write slightly wrong are now read as tool calls.
* **Unsloth quants at full speed:** the engine reads their Engram tables directly.
* **64 and 96 GB are real options:** SSD read-ahead (+22% at 64 GB), bigger expert memory, and prompt reads in file order (18-20% faster on low RAM).
* **128 GB:** ds4 Q2 went from 33 to 36 tok/s.
* **Default sampling:** the drafter now samples its drafts too, about +3.5% at default settings.
* **First launch** shows which model suits your RAM, and you can pick any other or use your own.
* **A Setup tab** in the chat page changes model, speed vs quality and the second GPU.
* **Public CI:** the release zips are now built by GitHub Actions, so you can see how each one was made.
* **EXL3:** 2.5 bpw is on the menu and the GPU kernel is about 2x faster. In my tests UD-Q2\_K\_XL was both faster and closer to the API, so I still point people there.
* **Basic multi-GPU:** 7% uplift using a 3060 + 3090. It doesn't scale well to 3-4+ GPUs, so use Project Maya for those builds.
* **Fixes:** contexts past 65,535 tokens, `</think>` in prose no longer ends the reasoning, no out-of-memory on the biggest files, a failed picture encoder no longer stops the model, faster prompts after a pause and many more.
* **Support for custom model selection:** just add them to the custom model folder and pick them in the web-ui

**Requirements:**

* **OS:** Windows 10 or 11
* **GPU:** NVIDIA RTX with 12 GB of VRAM or more, driver 580 or newer. 24 GB is the fast path, and AMD GPUs aren't supported yet
* **RAM:** 64 GB or more, 128 GB+ recommended
* **Disk:** a fast NVMe SSD, with 233-395 GB for the model (UD-IQ3\_XXS also needs the default model)
* **CPU:** any x86-64 CPU, Intel or AMD. AVX2 is the fast path

All of this is one PC. It needs testing on other hardware.

Repo: [https://github.com/musicurgy/delta-inference-engine](https://github.com/musicurgy/delta-inference-engine)

MIT licensed. It's a Windows port of Xolotl, built on Strata. Thanks to DeepSeek for the model, Unsloth, nazeshinjite and ISTA-DASLab for the quants, and to the Xolotl and Project Maya communities, whose work and public numbers helped a lot. Claude wrote most of the code and I ran the project.

If you try it, post your GPU, RAM, model and tok/s, good or bad.

document fetched 2026-11-06 00:45 · canonical https://reddit.com/r/LocalLLaMA/comments/1xoq4vq/deepseekv41flash1017_on_one_rtx_3090_big_updates