Welcome to the MELON Research Group

Learn with Optimization, Optimize with Learning.

About the Lab

MachinE Learning and OptimizatioN (MELON) Lab works at the intersection of optimization and machine learning. Our research spans three interconnected areas: optimization theory and methods, efficient large language model (LLM) training, and AI for optimization. These directions inform and reinforce one another: theory guides the design of new algorithms; advances in training enable more capable and reliable LLMs; and LLMs, in turn, offer new tools for discovering, designing, and analyzing optimization methods. Our aim is to create a self-reinforcing cycle that advances both optimization and machine learning.

Research Areas

Optimization Theory

Our research develops distributed, robust, and scalable optimization methods for modern machine learning and scientific computing. We focus on computational, memory, and communication efficiency, with rigorous convergence guarantees for high-dimensional, large-scale problems.

Efficient LLM Training

Our research develops memory- and compute-efficient methods for LLM pre-training and post-training, emphasizing communication efficiency, robustness, and fault tolerance. We also design optimizers guided by the geometry of the loss landscape to improve training efficiency and stability at scale.

AI for Optimization

Our research uses AI to advance optimization theory and characterize the fundamental limits of algorithmic performance. We also harness AI to discover and design optimization algorithms that approach these limits in practice.

See more publications on the Publications page