Title: Nonconvex Landscape of Policy Gradient Objectives for A Class of Finite Horizon MDPs: Applications in Operations Models

Abstract: We explore policy gradient methods for finite horizon Markov Decision Processes (MDPs) with continuous state and action spaces. Policy gradient methods for Markov Decision Processes (MDPs) do not converge to global optimal solutions in general due to the non-convexity of the objective functions. We identify several easily verifiable conditions to guarantee the global Kurdyka-Łojasiewicz (KŁ) condition for the objectives of policy gradient optimization problems for a class of MDPs. This allows to establish that the policy gradient optimization problems can be solved by first-order methods with a sample complexity in the order of $1/\epsilon$ and polynomial at the planning horizon length to attain $\epsilon$-global optimal solutions. Our results find applications in a host of operations models including the stochastic cash balance problem and multi-period inventory system with Markov-modulated demand.

Speaker Bio: Xin Chen is a James C. Edenfield chair and professor in the H. Milton Stewart School of Industrial and Systems Engineering at Georgia Tech. Prior to this appointment, he was a professor of industrial engineering at the University of Illinois at Urbana-Champaign. His research interest lies in optimization, data analytics, revenue management and supply chain management. He received the Informs revenue management and pricing section prize in 2009. He is the coauthor of the book “The Logic of Logistics: Theory, Algorithms, and Applications for Logistics and Supply Chain Management (Second Edition, 2005, & Third Edition, 2014)”, and serving as the department editor of logistics and supply chain management of Naval Research Logistics and an associate editor of several leading journals including Operations Research, Management Science, and Production and Operations Management.

Share this Event