Modern Nonconvex Optimization: Parametric and Stochastic Programming Extensions

Many modern applications in data science and operations research can be casted as nonconvex and nonsmooth optimization problems, which usually cannot be rigorously handled by applying heuristics or imposing some naive simplications of the models. These issues will even be more involved if we consider two additional extensions which will be the main topics of my talk.
First is the parametric programming extension which aims to study the solution path of a potentially nonconvex problem as a function of its (scalar valued) parameter. More specically I investigated the parametric programming techniques to enable hyperparameter selections for some sparse statis- tical estimation problems, which can be haunted by a dilemma in the choice of regularizer. Namely the ideal choice, i.e., `0 regularizer, can be computationally prohibitive in large scale, while convex relaxation, i.e., `1 regularizer, can give us unsatisfactory results in practice. The main focus for this part of the talk is on highlighting the advantage of adopting a nonconvex regularizer (capped `1 function) for resolving this critical dilemma, which will be further supported by rigorous analytical and computational results on the parametric programming studies for capped `1.
The second extension is stochasticity. More specically, the second part of the talk will be dedicated to a class of nonconvex stochastic programs motivated by inference of hierarchical Bayesian models when the normalizing factors have no closed form. In this case, nonconvexity and nonsmoothness in the objective will be nested with the intractable integrals. To tackle these complications, I will present an algorithm which combines surrogation with adaptive importance sampling for reducing the variance in approximating the integrals. Through the presentation of some theoretical analysis and numerical experiments, I will verify the superior performance of this algorithm in computation time, stability near convergence and variance in solutions.

Share this Event