Finite mixture models and hidden Markov models (HMMs) occupy central roles in modern statistical inference and data analysis. Finite mixture models assume that data originate from a latent combination ...

Mixture-of-Experts (MoE) has become a popular technique for scaling large language models (LLMs) without exploding computational costs. Instead of using the entire model capacity for every input, MoE ...