About
Welcome to my homepage! My full name is Huy Minh Nguyen, and I am currently a second-year Ph.D student at the Department of Statistics and Data Sciecnes, University of Texas at Austin where I am fortunate to be advised by Professor Nhat Ho and Professor Alessandro Rinaldo. Before that, I graduated from Ho Chi Minh City University of Science with a Bachelor’s degree in Mathematics (Summa Cum Laude).
Email: huynm@utexas.edu
Research Interests
My current research focuses on theoretical foundations for Mixture-of-Experts models. In particular, I try to comprehend the effects of various gating functions (namely softmax gate, top-K sparse softmax gate, dense-to-sparse gate, etc) on the convergence of density estimation and parameter estimation under the Mixture-of-Experts models. Based on insights from these results, I aim to design novel gating functions which help improve the performance of Mixture-of-Experts applications, including Large Language Models. Additionally, I am also interested in Optimal Transport theory.
(*) denotes equal contribution.
Selected Preprints
Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts
Huy Nguyen, Pedram Akbarian, Fanqi Yan, Nhat Ho
Under review [arXiv]
A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts
Huy Nguyen, Pedram Akbarian, TrungTin Nguyen, Nhat Ho
Under review [arXiv]
Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts
Huy Nguyen*, TrungTin Nguyen*, Khai Nguyen, Nhat Ho
Under review [arXiv]
On Parameter Estimation in Deviated Gaussian Mixture of Experts
Huy Nguyen, Khai Nguyen, Nhat Ho
Under review
Selected Publications
Demystifying Softmax Gating Function in Gaussian Mixture of Experts
Huy Nguyen, TrungTin Nguyen, Nhat Ho
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Spotlight [arXiv]
Minimax Optimal Rate for Parameter Estimation in Multivariate Deviated Models
Dat Do*, Huy Nguyen*, Khai Nguyen, Nhat Ho
37th Conference on Neural Information Processing Systems (NeurIPS 2023) [arXiv]
Entropic Gromov-Wasserstein between Gaussian Distributions
Huy Nguyen*, Khang Le*, Dung Le*, Dat Do, Tung Pham, Nhat Ho
39th International Conference on Machine Learning (ICML 2022) [arXiv] [ICML]
On Multimarginal Partial Optimal Transport: Equivalent Forms and Computational Complexity
Huy Nguyen*, Khang Le*, Khai Nguyen, Tung Pham, Nhat Ho
25th International Conference on Artificial Intelligence and Statistics (AISTATS 2022) [arXiv] [AISTATS]
On Robust Optimal Transport: Computational Complexity and Barycenter Computation
Huy Nguyen*, Khang Le*, Quang Minh Nguyen, Tung Pham, Hung Bui, Nhat Ho
35th Conference on Neural Information Processing Systems (NeurIPS 2021) [arXiv] [NeurIPS]
Recent News
- [Oct 2023] I received the NeurIPS 2023 Scholar Award. See you in New Orleans this December!
- [Oct 2023] Our new paper “A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts” is out.
- [Sep 2023] Our new paper “Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts” is out.
- [Sep 2023] We have two papers accepted to NeurIPS 2023, [1] as spotlight and [2] as poster.
- [Jul 2023] We will present the paper “Fast Approximation of the Generalized Sliced-Wasserstein Distance” at the Frontier4LCD workshop, ICML 2023.
- [May 2023] Three new papers on the Mixture of Experts theory are out! See more at [1], [2] and [3].
- [Feb 2023] Our new paper on Mixture Models theory “Minimax Optimal Rate for Parameter Estimation in Multivariate Deviated Models” is out.
- [Jan 2023] Our paper “Hierarchical Sliced Wasserstein Distance” is accepted to ICLR 2023.
Professional Services
Conference Review: ICML (2022), NeurIPS (2022-2023), AISTATS (2022-2024) and ICLR (2024). Workshop Review: Frontier4LCD (ICML 2023).