# How long are we gonna depend bunch of model creators (like Qwen, Gemma, NVIDIA, etc.,) for ~100B models? — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/abb75e7e74bc47c9

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-14 12:10 ingested 2026-11-15 02:36

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xw0xeu/how_long_are_we_gonna_depend_bunch_of_model/)

## retained text



```
I think we're fine till 2027 with models(\~100B .... Qwen 4.X & 5, Gemma-5, NVIDIA would release some Medium/Big size 25-200B models). After that it's gonna be critical, as many model creators are increased/increasing model size to 300B+. Future trend probably gonna be around 500B. That size range is not suitable for Unified 128GB Devices or 128GB RAM + 24-32GB VRAM(Q4 won't fit).

Here some models with increased sizes.

* DeepSeek-V4-Flash-0731 (**284B**) - DeepSeek-V4.1-Flash-1017 (**552B** \+ 196B Engram)
* Ling-3.0-flash (**124B**) - Ling-3.1-flash (**560B**)
* MiniMax-M2.7 (**230B**) - MiniMax-M3 & MiniMax-M3.1-Flash (**428B**)
* Step-3.7-Flash (**198B**) - Step-5-Preview (**600B**) (Hope they release Flash too)
* Inkling-Small (**276B**) - Inkling-2-Small (**594B**)
* GLM-4.X (**358B**) - GLM-5.X (**744B**)
* Mistral-Large-3 (**675B**) - Mistral-Large-4 (**1T**)
* Hy3 (**295B**) - Hy4 (**770B**)

So What Are We Gonna Do? What alternate options do we have?

Whenever we're talking about this topic, we get one common answer. Distillation. Do we have any recipes to create distills ourselves easy way? I'm not sure. 

**1. Distillation**

I remember only 3 Distills so far. Please mention in comments if I missed anything.

* DeepSeek-R1-Distills - [https://huggingface.co/collections/deepseek-ai/deepseek-r1](https://huggingface.co/collections/deepseek-ai/deepseek-r1)
* Empero's Distills - [https://huggingface.co/empero-ai/models?sort=created&search=distill](https://huggingface.co/empero-ai/models?sort=created&search=distill)
* [https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B)

**2. 1-bit / 2-bit / Ternary / Bitnet Models**

I see only this option other than Distillation. 

**Category 1** \- High/Decent Quality (Native) Low bit models. Also big/large size models.

* [https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF)
* [https://huggingface.co/AngelSlim/Hy4-preview-GGUF](https://huggingface.co/AngelSlim/Hy4-preview-GGUF)
* [https://huggingface.co/AngelSlim/Hy3-GGUF](https://huggingface.co/AngelSlim/Hy3-GGUF)

**Category 2** \- Other Low bit models. Like Bonsai. 

* [1-bit / 2-bit / Ternary / Bitnet Models - Updates & Tracking](https://www.reddit.com/r/LocalLLaMA/comments/1vryaid/1bit_2bit_ternary_bitnet_models_updates_tracking/) 

Still waiting to see more Ternary MOEs, particularly 100B size. 100B model could come in 20GB size so it's possible to run just with 24-32GB RAM alone on CPU-only inference. And Even 8GB VRAM could make it fly.

  
Any other options do we have? Please share.
```



document fetched 2026-11-15 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xw0xeu/how_long_are_we_gonna_depend_bunch_of_model
