About

I am Huy Nguyen, a final-year Ph.D. candidate in Statistics at The University of Texas at Austin, where I am fortunate enough to be advised by Prof. Nhat Ho and Prof. Alessandro Rinaldo.

My research develops statistical foundations for modern AI, with a particular focus on mixture-of-experts (MoE), multimodal learning, and efficient adaptation of large-scale models, which is organized around three closely connected themes:

Statistical understanding of modern AI architectures. I study how gating mechanisms, expert structures, and routing rules determine statistical sample complexity and expert specialization in MoE. Beyond MoE, I also investigate the statistical properties of self-attention mechanisms, including how different attention formulations affect sample efficiency.
Statistical principles for AI architecture design. I study how these statistical insights can be translated into architectural principles for modern AI systems. This perspective motivates new designs for sparse MoE, multimodal learning, and attention-based models that aim to improve efficiency, scalability, and specialization.
Efficient adaptation of large-scale models. I investigate how the structure and capacity of lightweight adaptation mechanisms affect the efficiency of adapting large pretrained models. My work focuses on methods such as low-rank adaptation and prompt-based tuning, with an emphasis on their statistical efficiency and principled design.

Full Publication List by Topics

Show all publications

[T.20] Characterizing Heterogeneous Rates in Finite Mixture Estimation via Partial Optimal Transport . Under review, 2026
Dung Le*, Huy Nguyen*, Trang Pham, Alessandro Rinaldo, Nhat Ho

[T.19] On the Geometry of Separation in Finite Gaussian Mixtures. Under review, 2026
Huy Nguyen*, Dung Le*, Alessandro Rinaldo, Nhat Ho

[T.18] On DeepSeekMoE: Statistical Benefits of Shared Experts and Normalized Sigmoid Gating. Under review, 2025
Huy Nguyen, Thong T. Doan, Quang Pham, Nghi Bui, Nhat Ho**, Alessandro Rinaldo* *

[T.17] Convergence Rates for Softmax Gating Mixture of Experts. IEEE Transactions on Information Theory 72(2), 1276-1304, 2026
Huy Nguyen, Nhat Ho**, Alessandro Rinaldo* *

[T.16] A Statistical Theory of Gated Attention through the Lens of Hierarchical Mixture of Experts. Under review, 2026
Viet Nguyen*, Tuan Minh Pham*, Thinh Cao*, Huy Nguyen, Nhat Ho**, Alessandro Rinaldo* *

[T.15] Rethinking Multinomial Logistic Mixture of Experts with Sigmoid Gating Function. Under review, 2026
Tuan Minh Pham*, Thinh Cao*, Viet Nguyen*, Huy Nguyen, Nhat Ho**, Alessandro Rinaldo* *

[T.14] Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts. NeurIPS, 2024
Huy Nguyen, Nhat Ho**, Alessandro Rinaldo* *

[T.13] Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective . Under review
Huy Nguyen*, Fanqi Yan*, Pedram Akbarian, Nhat Ho**, Alessandro Rinaldo* *

[T.12] Demystifying Softmax Gating Function in Gaussian Mixture of Experts. NeurIPS, 2023 (Spotlight)
Huy Nguyen, TrungTin Nguyen, Nhat Ho

[T.11] Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?. ICML, 2024
Huy Nguyen, Pedram Akbarian, Nhat Ho

[T.10] Statistical Advantages of Perturbing Cosine Router in Mixture of Experts. ICLR, 2025
Huy Nguyen, Pedram Akbarian*, Trang Pham*, Trang Nguyen*, Shujian Zhang, Nhat Ho

[T.9] Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts. ICLR, 2024
Huy Nguyen, Pedram Akbarian, Fanqi Yan, Nhat Ho

[T.8] On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions. Under review
Huy Nguyen*, Xing Han*, Carl William Harris, Suchi Saria**, Nhat Ho* *

[T.7] Quadratic Gating Mixture of Experts: Statistical Insights into Self-Attention. Under review
Pedram Akbarian*, Huy Nguyen*, Xing Han*, Nhat Ho

[T.6] A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts. ICML, 2024
Huy Nguyen, Pedram Akbarian, TrungTin Nguyen, Nhat Ho

[T.5] Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts. AISTATS, 2024
Huy Nguyen*, TrungTin Nguyen*, Khai Nguyen, Nhat Ho

[T.4] Improving Minimax Estimation Rates for Contaminated Mixture of Multinomial Logistic Experts via Expert Heterogeneity. Under review, 2026
Fanqi Yan*, Dung Le*, Trang Pham, Huy Nguyen, Nhat Ho

[T.3] On Minimax Estimation of Parameters in Softmax-Contaminated Mixture of Experts. NeurIPS, 2025
Fanqi Yan*, Huy Nguyen*, Dung Le*, Pedram Akbarian, Nhat Ho**, Alessandro Rinaldo* *

[T.2] Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts. AISTATS, 2025
Fanqi Yan*, Huy Nguyen*, Dung Le*, Pedram Akbarian, Nhat Ho

[T.1] On Parameter Estimation in Deviated Gaussian Mixture of Experts. AISTATS, 2024
Huy Nguyen, Khai Nguyen, Nhat Ho

Applications of Mixture-of-Experts

[A.9] FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion. NeurIPS, 2024
Xing Han, Huy Nguyen*, Carl Harris*, Nhat Ho, Suchi Saria

[A.8] Mixture of Experts Meets Prompt-Based Continual Learning. NeurIPS, 2024
Minh Le, An Nguyen*, Huy Nguyen*, Trang Nguyen*, Trang Pham*, Linh Van Ngo, Nhat Ho

[A.7] Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts. ICLR, 2025
Minh Le*, Chau Nguyen*, Huy Nguyen*, Quyen Tran, Trung Le, Nhat Ho

[A.6] Revisit Visual Prompt Tuning: The Expressiveness of Prompt Experts. ICLR, 2026
Minh Le*, Anh Nguyen*, Huy Nguyen, Chau Nguyen, Nhat Ho

[A.5] RepLoRA: Reparameterizing Low-rank Adaptation via the Perspective of Mixture of Experts. ICML, 2025
Tuan Truong*, Chau Nguyen*, Huy Nguyen*, Minh Le, Trung Le, Nhat Ho

[A.4] On Zero-Initialized Attention: Optimal Prompt and Gating Factor Estimation. ICML, 2025
Nghiem T. Diep*, Huy Nguyen*, Chau Nguyen*, Minh Le, Duy M. H. Nguyen, Daniel Sonntag, Mathias Niepert, Nhat Ho

[A.3] Learning to Route from Expert Competition: Efficient Training of Sparse Mixture-of-Experts. Under review
Nam V. Nguyen, Huy Nguyen, Quang Pham, Van Nguyen, Savitha Ramasamy, Nhat Ho

[A.2] One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning. ICLR, 2026
Minh Le, Bao-Ngoc Dao, Huy Nguyen, Quyen Tran, Anh Nguyen, Nhat Ho

[A.1] Hypernetwork-Driven Low-Rank Adaptation Across Attention Heads. Under review
Nghiem T. Diep*, Dung Le*, Tuan Truong*, Tan Dinh, Huy Nguyen, Nhat Ho

Optimal Transport

[O.5] Entropic Gromov-Wasserstein between Gaussian Distributions. ICML, 2022
Huy Nguyen*, Khang Le*, Dung Le*, Dat Do, Tung Pham, Nhat Ho

[O.4] On Multimarginal Partial Optimal Transport: Equivalent Forms and Computational Complexity. AISTATS, 2022
Huy Nguyen*, Khang Le*, Khai Nguyen, Tung Pham, Nhat Ho

[O.3] On Robust Optimal Transport: Computational Complexity and Barycenter Computation. NeurIPS, 2021
Huy Nguyen*, Khang Le*, Quang Minh Nguyen, Tung Pham, Hung Bui, Nhat Ho

[O.2] Fast Approximation of the Generalized Sliced-Wasserstein Distance. IEEE ICASSP, 2024
Huy Nguyen*, Dung Le*, Khai Nguyen*, Trang Nguyen*, Nhat Ho

[O.1] Hierarchical Sliced Wasserstein Distance. ICLR, 2023
Khai Nguyen, Tongzheng Ren, Huy Nguyen, Litu Rout, Tan Nguyen, Nhat Ho

Books

[B.1] Handbook of Bayesian Deep Learning.
BayesAI Consortium. CRC Press, 2026 (forthcoming).

Industrial Experience

Morgan Stanley, Machine Learning Research — Summer Associate, Summer 2026.
Microsoft AI — Research Intern, Summer 2024.

Professional Service

Senior Program Committee: AAAI 2027.
Conference Reviewer: ICML (2022–2026), NeurIPS (2022–2026), AISTATS (2022–2026), ICLR (2024–2027), AAAI (2025–2026).
Journal Reviewer: Journal of Machine Learning Research (JMLR), Electronic Journal of Statistics (EJS), IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Transactions on Machine Learning Research (TMLR).
Seminar Co-organizer: StatML@UT, The University of Texas at Austin.

Huy Nguyen