Spherical Spin Glasses and Stochastic Gradient Descent

This talk will focus on two strands of my recent research – spin glasses and stochastic gradient descent (SGD). While the contexts are different, both draw on tools from high-dimensional probability to study the limiting behavior of large systems.

My spin glass research focuses on phase transitions in the spherical Sherrington-Kirkpatrick (SSK) model. In addition to their physical meaning, these phase transitions can be interpreted in the context of statistics, random matrix theory, and optimization problems. I will highlight one result (joint with H. Le) in which we obtain a limit theorem for the fluctuations of the free energy in the bipartite spherical model at critical temperature. Our work involves random matrix techniques and contour integral analysis.

My SGD research focuses on the dynamics of streaming SGD in the high-dimensional limit. In joint work with C. Paquette, E. Paquette, and I. Seroussi, we characterize the limiting process in terms of its convergence to a high-dimensional stochastic differential equation. Our set-up covers a range of optimization problems including linear regression, logistic regression, and some simple neural nets. For each of these models, we also derive a deterministic approximation of the risk curve, with vanishing error as dimension grows.

Share this Event