Huy Nguyen

Huy Nguyen

Ph.D Candidate at the Department of Statistics and Data Sciences, The University of Texas at Austin

Publications

(*) denotes equal contribution, (**) denotes co-last authors

Full Publication List by Themes

Show all publications

I - Statistical Understanding of Modern AI Architectures

MoE Theory

Self-attention Theory

Broader Statistical Foundation

II - Statistical Principles for AI Architecture Design

III- Efficient Adaptation of Large-scale Models

Prompt-based Tuning

Low-rank Adaptation

IV - Optimal Transport

Full Publication List by Type

Show all publications

Journal Submissions

[JS.8] On the Geometry of Separation in Finite Gaussian Mixtures. Under review.
Huy Nguyen*, Dung Le*, Alessandro Rinaldo, Nhat Ho

[JS.7] Characterizing Heterogeneous Rates in Finite Mixture Estimation via Partial Optimal Transport. Under review.
Dung Le*, Huy Nguyen*, Trang Pham, Alessandro Rinaldo, Nhat Ho

[JS.6] On DeepSeekMoE: Statistical Benefits of Shared Experts and Normalized Sigmoid Gating. Under review.
Huy Nguyen, Thong T. Doan, Quang Pham, Nghi D. Q. Bui, Nhat Ho**, Alessandro Rinaldo**

[JS.5] Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective. Under review
Huy Nguyen*, Fanqi Yan*, Pedram Akbarian, Nhat Ho**, Alessandro Rinaldo**

[JS.4] On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions Under review.
Huy Nguyen*, Xing Han*, Carl William Harris, Suchi Saria**, Nhat Ho**

[JS.3] Joint Estimation and Feature Selection in Overspecified Finite Mixture of Regressions. Under review, 2026
Abbas Khalili*, Pengqi Liu*, Huy Nguyen*, Nhat Ho

[JS.2] Quadratic Gating Mixture of Experts: Statistical Insights into Self-Attention. Under review.
Pedram Akbarian*, Huy Nguyen*, Xing Han*, Nhat Ho

[JS.1] On Bayesian Softmax-Gated Mixture-of-Experts Models. Under review.
Nicola Bariletto*, Huy Nguyen*, Nhat Ho**, Alessandro Rinaldo**

Conference Submissions

[CS.6] Sparsity Saves Compute, Not Samples: Oracle Inequalities for Dense and Sparse Mixture of Experts. Under review, 2026
Sagar Ghosh, Huy Nguyen, Alessandro Rinaldo

[CS.5] Rethinking Multinomial Logistic Mixture of Experts with Sigmoid Gating Function. Under review
Tuan Minh Pham*, Thinh Cao*, Viet Nguyen*, Huy Nguyen, Nhat Ho**, Alessandro Rinaldo**

[CS.4] Improving Minimax Estimation Rates for Contaminated Mixture of Multinomial Logistic Experts via Expert Heterogeneity. Under review
Fanqi Yan*, Dung Le*, Trang Pham, Huy Nguyen, Nhat Ho

[CS.3] Hypernetwork-Driven Low-Rank Adaptation Across Attention Heads. Under review
Nghiem T. Diep*, Dung Le*, Tuan Truong*, Tan Dinh, Huy Nguyen, Nhat Ho

[CS.2] Improving Weight-Decomposed Low-rank Adaptation with Shared Auxiliary Networks. Under review
Nghiem T. Diep*, Hien Dang*, Tuan Truong*, Tan Dinh, Huy Nguyen, Nhat Ho

[CS.1] CompeteSMoE – Statistically Guaranteed Mixture of Experts Training via Competition. Under review.
Nam V. Nguyen, Huy Nguyen, Quang Pham, Van Nguyen, Savitha Ramasamy, Nhat Ho

Journal Publications

[J.2] Convergence Rates for Softmax Gating Mixture of Experts. IEEE Transactions on Information Theory 72(2), 1276-1304, 2026.
Huy Nguyen, Nhat Ho**, Alessandro Rinaldo**

[J.1] EPEM: Efficient Parameter Estimation for Multiple Class Monotone Missing Data. Information Sciences, Volume 567, August 2021, Pages 1-22.
Thu Nguyen, Duy H. M. Nguyen, Huy Nguyen, Binh T. Nguyen, Bruce A. Wade.

Conference Publications

[CS.25] A Statistical Theory of Gated Attention through the Lens of Hierarchical Mixture of Experts. Advances in NeurIPS, 2026
Viet Nguyen*, Tuan Minh Pham*, Thinh Cao*, Huy Nguyen, Nhat Ho**, Alessandro Rinaldo**

[C.24] One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning. Proceedings of the ICLR, 2026
Minh Le, Bao-Ngoc Dao, Huy Nguyen, Quyen Tran, Anh Nguyen Nhat Ho

[C.23] Revisit Visual Prompt Tuning: The Expressiveness of Prompt Experts. Proceedings of the ICLR, 2026
Minh Le*, Anh Nguyen*, Huy Nguyen, Chau Nguyen, Anh Tran, Nhat Ho

[C.22] On Minimax Estimation of Parameters in Softmax-Contaminated Mixture of Experts. Advances in NeurIPS, 2025
Fanqi Yan*, Huy Nguyen*, Dung Le*, Pedram Akbarian, Nhat Ho**, Alessandro Rinaldo**

[C.21] RepLoRA: Reparameterizing Low-rank Adaptation via the Perspective of Mixture of Experts. Proceedings of the ICML, 2025
Tuan Truong*, Chau Nguyen*, Huy Nguyen*, Minh Le, Trung Le, Nhat Ho

[C.20] On Zero-Initialized Attention: Optimal Prompt and Gating Factor Estimation. Proceedings of the ICML, 2025
Nghiem T. Diep*, Huy Nguyen*, Chau Nguyen*, Minh Le, Duy M. H. Nguyen, Daniel Sonntag, Mathias Niepert, Nhat Ho

[C.19] Statistical Advantages of Perturbing Cosine Router in Sparse Mixture of Experts. Proceedings of the ICLR, 2025.
Huy Nguyen, Pedram Akbarian*, Trang Pham*, Trang Nguyen*, Shujian Zhang, Nhat Ho

[C.18] Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts. Proceedings of the ICLR, 2025.
Minh Le*, Chau Nguyen*, Huy Nguyen*, Quyen Tran, Trung Le, Nhat Ho

[C.17] Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts. In AISTATS, 2025.
Fanqi Yan*, Huy Nguyen*, Dung Le*, Pedram Akbarian, Nhat Ho

[C.16] Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts. Advances in NeurIPS, 2024.
Huy Nguyen, Nhat Ho**, Alessandro Rinaldo**

[C.15] FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion. Advances in NeurIPS, 2024.
Xing Han, Huy Nguyen*, Carl Harris*, Nhat Ho**, Suchi Saria**

[C.14] Mixture of Experts Meets Prompt-Based Continual Learning. Advances in NeurIPS, 2024.
Minh Le, An Nguyen*, Huy Nguyen*, Trang Nguyen*, Trang Pham*, Linh Van Ngo, Nhat Ho

[C.13] On Least Square Estimation in Softmax Gating Mixture of Experts. Proceedings of the ICML, 2024.
Huy Nguyen, Nhat Ho**, Alessandro Rinaldo**

[C.12] Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?. Proceedings of the ICML, 2024.
Huy Nguyen, Pedram Akbarian, Nhat Ho

[C.11] A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts. Proceedings of the ICML, 2024.
Huy Nguyen, Pedram Akbarian, TrungTin Nguyen, Nhat Ho

[C.10] Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts. Proceedings of the ICLR, 2024.
Huy Nguyen, Pedram Akbarian, Fanqi Yan, Nhat Ho

[C.9] Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts. In AISTATS, 2024.
Huy Nguyen*, TrungTin Nguyen*, Khai Nguyen, Nhat Ho

[C.8] On Parameter Estimation in Deviated Gaussian Mixture of Experts. In AISTATS, 2024.
Huy Nguyen, Khai Nguyen, Nhat Ho

[C.7] Demystifying Softmax Gating Function in Gaussian Mixture of Experts. Advances in NeurIPS, 2023 (Spotlight) .
Huy Nguyen, TrungTin Nguyen, Nhat Ho

[C.6] Minimax Optimal Rate for Parameter Estimation in Multivariate Deviated Models. Advances in NeurIPS, 2023.
Dat Do*, Huy Nguyen*, Khai Nguyen, Nhat Ho

[C.5] Fast Approximation of the Generalized Sliced-Wasserstein Distance. In ICASSP, 2024.
Dung Le*, Huy Nguyen*, Khai Nguyen*, Trang Nguyen*, Nhat Ho

[C.4] Hierarchical Sliced Wasserstein Distance. Proceedings of the ICLR, 2023.
Khai Nguyen, Tongzheng Ren, Huy Nguyen, Litu Rout, Tan Nguyen, Nhat Ho

[C.3] Entropic Gromov-Wasserstein between Gaussian Distributions. Proceedings of the ICML, 2022.
Huy Nguyen*, Khang Le*, Dung Le*, Dat Do, Tung Pham, Nhat Ho

[C.2] On Multimarginal Partial Optimal Transport: Equivalent Forms and Computational Complexity. Proceedings of the AISTATS, 2022.
Huy Nguyen*, Khang Le*, Khai Nguyen, Tung Pham, Nhat Ho

[C.1] On Robust Optimal Transport: Computational Complexity and Barycenter Computation. Advances in NeurIPS, 2021.
Huy Nguyen*, Khang Le*, Quang Minh Nguyen, Tung Pham, Hung Bui, Nhat Ho

Books

[B.1] Handbook of Bayesian Deep Learning.
BayesAI Consortium. CRC Press, 2026 (forthcoming).