# Research paper: Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns — retained record

> Retained semantic-scholar record from semantic-scholar.

Canonical: https://highsignal.app/data/records/e64a799557dc8aef

[← sources](https://highsignal.app/data)


source: [semantic-scholar](https://highsignal.app/data/semantic-scholar) published 2026-04-25 00:00 ingested 2026-07-02 08:48

[original source ↗](https://www.semanticscholar.org/paper/ae9d6f475502d77ec26f83a9e5332111b30a9aca)

## retained text



```
Query: mixture of experts inference serving
Authors: A. Bambhaniya, Geonhwa Jeong, Jason Park, Jiecao Yu, Jaewon Lee
Citations: 0
Most recent state-of-the-art (SOTA) large language models (LLMs) use Mixture-of-Experts (MoE) architectures to scale model capacity without proportional per-token compute, enabling higher-quality outputs at manageable serving costs. However, MoE inference at scale is fundamentally bottlenecked by expert load imbalance and inefficient token routing, especially in multi-node deployments where tokens are not guaranteed to be routed to local experts, resulting in significant inter-node all-to-all communication overhead. To systematically characterize these challenges, we profile SOTA open-source MoE models, including Llama 4 Maverick, DeepSeek V3-671B, and Qwen3-230B-A22B, on various datasets and collected over 100k real expert activation traces. Upon studying the expert activation patterns, we uncover various persistent properties across all the frontier MoE models: variable expert load imbalance, domain-specific expert activation where expert popularity shifts across task families (code, math, chat, general), and a strong correlation between prefill and decode expert activations. Motivated by these findings, we propose workload-aware micro-batch grouping and an expert placement strategy to maximize token locality to the destination expert, thereby reducing inter-node communication. Across models and datasets, these optimizations help reduce all2all communication data up to 20, resulting in lower MoE decode latency and better accelerator utilization.
```



document fetched 2026-08-10 07:17 · canonical https://semanticscholar.org/paper/ae9d6f475502d77ec26f83a9e5332111b30a9aca
