# Thoughts about a single framework that dynamically adapts to run any MoE larger than RAM, across everything from phones and laptops to high-end, expensive GPUs — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/290d8f9b0eca1e1f

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-10-28 16:23 ingested 2026-10-29 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xhsr2w/thoughts_about_a_single_framework_that/)

## retained text



```
I am thinking and working on creating a framework that can run any type of MoE (Mixture of Experts) model supported by \`llama.cpp\` on any hardware or operating system.

It would dynamically adapt to the specific characteristics of the hardware, aiming for optimal performance—measured in tokens per second (tok/s)—during both the prefill and decode phases. 

The concept is to build a home server setup using various consumer-grade devices to run local AI—though not in a mesh configuration (at least for now, due to latency concerns).

What are your thoughts on this? What kind of use cases could this enable?
```



document fetched 2026-10-29 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xhsr2w/thoughts_about_a_single_framework_that
