Updates, papers, awards, talks, and announcements from the MELON Research Group.
Showing 1–8 of 12 results
Our paper on memory- and communication-efficient federated learning was accepted to IEEE Transactions on Signal Processing. Congratulations to our collaborators and students! - [An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data](https://arxiv.org/abs/2509.05213)
Three papers were accepted to NeurIPS 2026. Congratulations to our collaborators and students! - [BROS: Bias-Corrected Randomized Subspaces for Memory-Efficient Single-Loop Bilevel Optimization](https://arxiv.org/abs/2605.10288) - [Accelerating LLM Pre-Training through Flat-Direction Dynamics Enhancement](https://arxiv.org/abs/2602.22681) - [Clapping: Removing Per-sample Storage for Pipeline Parallel Distributed Optimization with Communication Compression](https://arxiv.org/abs/2509.19029)
Eight papers from the group were accepted to ICML 2026, spanning efficient LLM training and inference, AI for Math, and optimization theory.
Two papers were accepted: one to IEEE TPAMI on communication-efficient distributed learning under data heterogeneity, and one to TMLR on domain weight randomization for LLM pre-training.
Kun Yuan will serve as an Associate Editor for IEEE Transactions on Signal Processing.
We are hiring postdocs and undergraduate research interns. If you are interested in machine learning, optimization, and AI systems, please get in touch.
Gave a one-hour invited talk on Memory-Efficient LLM Training via Implicit Structures at the Microsoft Research Asia ACE Talk.
Three papers were accepted to the Journal of Machine Learning Research, on decentralized bilevel optimization, gradient normalization under heavy-tailed noise, and Byzantine-robust distributed optimization.