About

Welcome to my homepage! I’m Huy Nguyen, a third-year Ph.D student at the Department of Statistics and Data Sciences, University of Texas at Austin where I am fortunate to be advised by Professor Nhat Ho and Professor Alessandro Rinaldo. Before that, I graduated from Ho Chi Minh City University of Science with a Bachelor’s degree in Mathematics (Summa Cum Laude). In Summer 2024, I was a research intern at Microsoft AI.

Email: huynm@utexas.edu

Research Interests

My current research focuses on theoretical foundations for the Mixture-of-Experts models. In particular, I investigate the effects of various gating functions (namely the softmax gate, the Top-K sparse softmax gate, the dense-to-sparse gate, the sigmoid gate, etc) on the convergence of expert estimation under the Mixture-of-Experts models. Based on insights from these results, I aim to design novel gating functions and characterize expert networks which help improve the efficiency and scalability of the Mixture-of-Experts applications, including Large Language Models, Multi-modal Learning and Parameter-efficient Fine-Tuning. Additionally, I am also interested in Optimal Transport theory.

(*) denotes equal contribution, (**) denotes equal advising.

Selected Preprints

On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions

Huy Nguyen*, Xing Han*, Carl William Harris, Suchia Saria**, Nhat Ho* *
Under review

Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts

Minh Le*, Chau Nguyen*, Huy Nguyen*, Quyen Tran, Trung Le, Nhat Ho
Under review

Statistical Advantages of Perturbing Cosine Router in Sparse Mixture of Experts

Huy Nguyen, Pedram Akbarian*, Trang Pham*, Trang Nguyen*, Shujian Zhang, Nhat Ho
Under review

Quadratic Gating Functions in Mixture of Experts: A Statistical Insight

Pedram Akbarian*, Huy Nguyen*, Xing Han*, Nhat Ho
Under review

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts

Pedram Akbarian*, Huy Nguyen*, Dung Le*, Pedram Akbarian, Nhat Ho
Under review

CompeteSMoE - Effective Training of Sparse Mixture of Experts via Competition

Quang Pham, Giang Do, Huy Nguyen, TrungTin Nguyen, Chenghao Liu, Mina Sartipi, Binh T. Nguyen, Savitha Ramasamy, Xiaoli Li, Steven Hoi, Nhat Ho
Under review

Selected Publications on the Foundations of Mixture of Experts

Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts

Huy Nguyen, Nhat Ho**, Alessandro Rinaldo* *
Advances in NeurIPS, 2024.

On Least Square Estimation in Softmax Gating Mixture of Experts

Huy Nguyen, Nhat Ho**, Alessandro Rinaldo* *
Proceedings of the ICML, 2024.

Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?

Huy Nguyen, Pedram Akbarian, Nhat Ho
Proceedings of the ICML, 2024.

Demystifying Softmax Gating Function in Gaussian Mixture of Experts

Huy Nguyen, TrungTin Nguyen, Nhat Ho
Advances in NeurIPS, 2023 (Spotlight) .

Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts

Huy Nguyen, Pedram Akbarian, Fanqi Yan, Nhat Ho
Proceedings of the ICLR, 2024.

A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts

Huy Nguyen, Pedram Akbarian, TrungTin Nguyen, Nhat Ho
Proceedings of the ICML, 2024.

Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts

Huy Nguyen*, TrungTin Nguyen*, Khai Nguyen, Nhat Ho
In AISTATS, 2024.

On Parameter Estimation in Deviated Gaussian Mixture of Experts

Huy Nguyen, Khai Nguyen, Nhat Ho
In AISTATS, 2024.

Selected Publications on the Applications of Mixture of Experts

FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion

Xing Han, Huy Nguyen*, Carl Harris*, Nhat Ho, Suchi Saria
Advances in NeurIPS, 2024.

Mixture of Experts Meets Prompt-Based Continual Learning

Minh Le, An Nguyen*, Huy Nguyen*, Trang Nguyen*, Trang Pham*, Linh Van Ngo, Nhat Ho
Advances in NeurIPS, 2024.

Selected Publications on Optimal Transport

Entropic Gromov-Wasserstein between Gaussian Distributions

Huy Nguyen*, Khang Le*, Dung Le*, Dat Do, Tung Pham, Nhat Ho
Proceedings of the ICML, 2022.

On Multimarginal Partial Optimal Transport: Equivalent Forms and Computational Complexity

Huy Nguyen*, Khang Le*, Khai Nguyen, Tung Pham, Nhat Ho
In AISTATS, 2022.

On Robust Optimal Transport: Computational Complexity and Barycenter Computation

Huy Nguyen*, Khang Le*, Quang Minh Nguyen, Tung Pham, Hung Bui, Nhat Ho
Advances in NeurIPS, 2021.

Recent News

Professional Services

  • Conference Reviewer: ICML (2022,2024), NeurIPS (2022-2024), AISTATS (2022-2025), ICLR (2024-2025) and AAAI (2025).
  • Journal Reviewer: Electronic Journal of Statistics.
  • Workshop Reviewer: Frontier4LCD (ICML 2023).