# Best Hy4-flash quants for their size - Nace Quants using Weight Space Quantization (bonus: 70gb deepseek v4.1 flash quant for coding) — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/7a033baaefb6d5fc

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-18 22:10 ingested 2026-11-19 02:39

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xzseu8/best_hy4flash_quants_for_their_size_nace_quants/)

## retained text



```
hey llamas, its Alish from [nace.ai](http://nace.ai) again. we are an open source ai lab. we build the [drex](https://github.com/nace-ai/drex-decision-models) decision models, served on our [console](https://console.nace.ai), and [kapitsa](https://github.com/nace-ai/kapitsa), optimized inference for apple silicon. We ship a bunch of great quants. In our last [post](https://www.reddit.com/r/LocalLLaMA/comments/1xvh4ag/weight_space_programming_for_quantization_nace/) I promised  more models like Deepseek Flash models built with Weight Space Quantization. So here they are: the new Hy4-flash and the good old Deepseek v4.1 flash.

Hy4 flash is a general-purpose quant, and DeepSeek V4.1 Flash Coding is a specialized one built for coding and agent work.

# Hy4-flash

Hy4 flash GGUF comes in two sizes, both measured against the native Tencent Hy4-flash weights.

[always gotta pick the Nace one](https://preview.redd.it/wgefwg1awf2i1.png?width=1200&format=png&auto=webp&s=d0774338fd7a1a7887426ff17efaa18fb33e9690)

* 107.49 GB (Q3\_K experts, IQ4\_NL Engram): mean KL 0.233.
* 127.22 GB (Q4\_K experts, IQ4\_NL Engram): mean KL 0.133.
* Xylolabs Q2\_K is 125.15 GB and scores 0.402. AngelSlim IQ2\_AH is 135.00 GB and scores 0.509. Atomic Chat IQ2 is 84.70 GB and scores 0.570. Both of our files have 2x/3x lower KL than all three.
* Download, run command on the card: [https://huggingface.co/nace-ai/Hy4-flash-GGUF](https://huggingface.co/nace-ai/Hy4-flash-GGUF)

# DeepSeek V4.1 Flash Coding

It is a specialized quant: DeepSeek V4.1 Flash 1017 (763B parameters), built and tuned for coding and agent work, in a 71.8 GB GGUF.

# Why a coding quant

Several people said they would rather see a quant made for code only. I agree: the future is specialized quants for your use case.

So I built one: calibrated on real coding work, pruned for code, and judged on code text and on coding problems.

Others suggested letting the model write the quantization rules itself, as long as the result snaps to a valid GGUF tensor type. That is now one of the tool classes of our weight space quantization agent (more in the appendix).

# How it's built

Three steps all iterated with the Nace Harness specialized on Weight Space Quantization, all aimed at code.

* Calibration on real coding work: agent sessions, contest solutions and code commits.
* Expert pruning: DeepSeek V4.1 Flash has 384 experts in each of its 40 layers. We keep the 200 per layer that code uses most. They carry about 90% of the model's code routing.
* Mixed precision per layer: every kept expert gets a score, and the 16 layers where code needs it most get larger, higher-precision experts.

https://preview.redd.it/xjtod9mbwf2i1.png?width=3200&format=png&auto=webp&s=cadc51f9ecf9a6d7366f16fbde74ce01ee26c409

# Results

The KL numbers here are on held-out text. Main weights only, the Engram lookup table is separate.

[some stats](https://preview.redd.it/zrcv477dwf2i1.png?width=3200&format=png&auto=webp&s=78f8dd3df01db2677fa739ecb5a6aa5b2e8399f6)

* Size: 71.8 GB for a 763B model.
* KL to the near-lossless Q8 reference on held-out code and agent text: 0.140.
* ISTA Coder, the other coding-specific file, is 72.1 GB and scores 0.231.
* EXL3 at 1.68 bpw (124 GB) scores 0.161.
* The file picks the same next token as the reference 88.4% of the time.

At the same size, the held-out KL is 0.140 against 0.231 for ISTA Coder, which is 39% lower.

LiveCodeBench v6, two seeds with DeepSeek's recommended sampling and a 32K answer limit: the shipped file solves 37 of 38 problems on both seeds. This shows the model's coding ability did not degrade.

https://preview.redd.it/tsmud7rewf2i1.png?width=3200&format=png&auto=webp&s=d93a66eacd13b816b58faab00d9c2e2300c06159

* The 48 GB Mac: it runs on my 48 GB M5 Pro with llama.cpp, experts on the CPU and memory-mapped.
* Download: one repo with the main weights and the Engram table (MIT). Run command is on the card: [https://huggingface.co/nace-ai/DeepSeek-V4.1-Flash-Coding-GGUF](https://huggingface.co/nace-ai/DeepSeek-V4.1-Flash-Coding-GGUF)

If you run either one on a coding or agent workload, I would like to hear which runner version you used and how it went.

# Appendix: in detail

# The pipeline

Four steps: calibrate, prune, allocate bits, assemble.

* Calibrate: calibration text goes through the model to count how often each expert fires on code.
* Prune: keep the 200 most used experts per layer.
* Allocate bits: decide which layers get larger experts.
* Assemble: write a GGUF in which every tensor is a valid llama.cpp type, so nothing needs a custom runner.

# Calibration data

485k characters in three classes: agent sessions from SWE-smith, contest solutions from open-r1 codeforces-cots and code commits from BigCode commitpackft. We cut random 2,500-character windows from each document, so the agent sessions contribute tool calls and their outputs as well as prose. This is the only text that decides which experts we keep and where the bits go.

# Expert pruning

In the case of Deepseek, each layer has 384 routed experts and uses a few per token. On our calibration text the 200 most used experts per layer handle about 90% of the routing for code. For comparison, ISTA Coder keeps 192 experts per layer, and on our text those carry 64% of the same traffic. We rank experts by routing count and keep the top 200.

# Bit allocation

Where the bits go matters as much as how many there are. We use the scoring from Mixture Compressor (MC-MoE, Huang et al., ICLR 2025): every kept expert is scored by how often it fires, how much weight the router gives it, and how much error its rounding adds. 16 of the 40 layers get larger IQ1\_M experts, the other 24 stay on IQ1\_S.

https://preview.redd.it/nh01e2cgwf2i1.png?width=3200&format=png&auto=webp&s=2e9c781e803702e09bbe2da75f608fda7e2f4302

https://preview.redd.it/0bbgiohhwf2i1.png?width=3200&format=png&auto=webp&s=133cba564b41e2ed82859bfb8e0448f3f1b23e2a

# The agent loop

This is the Weight Space Quantization part. An agent edits the quantized file directly, with a measured result deciding every edit. It works with seven tool classes: reading, navigation, quantization, expert, correction, checking and rollback (diagram below). The quantization tools can also apply a rounding rule the agent writes itself, snapped to a valid GGUF type.

The loop has five stages: look at the plan, propose an edit, pre-check it locally, measure it for real, keep or revert.

The local check scores up to 8 edits at once against one layer's output error on cached activations. It takes seconds. The real measurement builds the file and scores KL on development chunks.

https://preview.redd.it/c1oazh2jwf2i1.png?width=3200&format=png&auto=webp&s=e08244920907c798f613b6f1ff32ed1cffe64b88

# Keeping held-out clean

* Calibration (above) decides the experts and the bits.
* The development set, from Nebius SWE-rebench OpenHands trajectories and NVIDIA OpenCodeReasoning (neither is in calibration), is for every choice we made: which file to keep, which agent edit to keep.
* The final set is 200k characters, 55% from SWE-rebench and 45% from OpenCodeReasoning, taken from row groups and a file the development set never read. It was scored once per file, after we had picked the files.

# Concluding remarks

So that's it, more or less, check out the repos, try the models out:  
\- [https://huggingface.co/nace-ai/Hy4-flash-GGUF](https://huggingface.co/nace-ai/Hy4-flash-GGUF)  
\- [https://huggingface.co/nace-ai/DeepSeek-V4.1-Flash-Coding-GGUF](https://huggingface.co/nace-ai/DeepSeek-V4.1-Flash-Coding-GGUF)

This was fun, let me know what other models and niches to touch on next!
```



document fetched 2026-11-19 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1xzseu8/best_hy4flash_quants_for_their_size_nace_quants
