# Research paper: Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts — retained record

> Retained semantic-scholar record from semantic-scholar.

Canonical: https://highsignal.app/data/records/16cc26bce824b7a6

[← sources](https://highsignal.app/data)


source: [semantic-scholar](https://highsignal.app/data/semantic-scholar) published 2025-09-26 00:00 ingested 2026-07-02 08:48

[original source ↗](https://www.semanticscholar.org/paper/1a1a5ecbfc527612fccfec520b185e289684f0e3)

## retained text



```
Query: mixture of experts inference serving
Authors: Naibin Gu, Zhenyu Zhang, Yuchen Feng, Yilong Chen, Peng Fu
Citations: 8
Mixture-of-Experts (MoE) models typically fix the number of activated experts $k$ at both training and inference. However, real-world deployments often face heterogeneous hardware, fluctuating workloads, and diverse quality-latency requirements, while training separate models for each scenario is costly. Considering that MoE models already operate with sparse activation, adjusting the number of activated experts offers a natural path to serving diverse budgets with a single model. Yet, we find that activating more experts $k'$ ($>k$) at inference does not yield the expected gains. Instead, performance degrades rapidly after only a slight increase, a phenomenon we term the \textit{inference-time scaling wall}. Further investigation reveals that this degradation stems from a lack of learned collaboration among experts. To address this, we introduce \textbf{Elastic Mixture-of-Experts (EMoE)}, a novel training framework that enables MoE models to elastically vary the number of activated experts at inference. By simultaneously training experts to collaborate in diverse combinations and encouraging the router to make high-quality selections, EMoE ensures robust performance across inference budgets. Extensive experiments across four MoE architectures (7B--21B) and nine benchmarks show that EMoE significantly expands the effective scaling range to 2-3$\times$ the training-time $k$, while also achieving higher peak performance.
```



document fetched 2026-08-10 07:17 · canonical https://semanticscholar.org/paper/1a1a5ecbfc527612fccfec520b185e289684f0e3
