← sources

GitHub - sqliteai/waste: Run the full 2.78-trillion-parameter Kimi K3 model beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.

source: reddit (reddit:LocalLLaMA)published 2026-08-03 00:16ingested 2026-08-03 09:25

retained text

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1vdy1nd/github_sqliteaiwaste_run_the_full/"> <img alt="GitHub - sqliteai/waste: Run the full 2.78-trillion-parameter Kimi K3 model beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine." src="https://external-preview.redd.it/fI1VyFPuuxbtwBA8LVdC2IpRoMvQ-2qIt886spO77vk.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=6528b255d7c47a7e9fc22c17482cf80de2c2b423" title="GitHub - sqliteai/waste: Run the full 2.78-trillion-parameter Kimi K3 model beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine." /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>WASTE is an embeddable inference engine written in C, with no third-party runtime dependencies. It keeps the model trunk in memory, streams selected experts directly from disk, and uses the remaining RAM as a bounded expert cache.</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/ab2377"> /u/ab2377 </a> <br /> <span><a href="https://github.com/sqliteai/waste">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1vdy1nd/github_sqliteaiwaste_run_the_full/">[comments]</a></span> </td></tr></table>

document fetched 2026-08-03 09:25 · canonical https://reddit.com/r/LocalLLaMA/comments/1vdy1nd/github_sqliteaiwaste_run_the_full