Online Reinforcement Learning in Continuous Time and Space
DECISION AND CONTROL LECTURE THE GRAINGER COLLEGE OF ENGINEERING | ||
TITLE |
Online Reinforcement Learning in Continuous Time and Space
| |
Sponsor | Decision and Control | |
Date | Wednesday, October 4, 2023 | |
Time | 3:00 PM | |
LOCATION | Coordinated science lab, rm b02 | |
Speaker: Assistant Professor Mohamad Kazem Shirani Faradonbeh Southern Methodist University
| ||
ABSTRACT | ||
A canonical model for decision-making in uncertain continuous environments is that of linear dynamical systems that evolve as stochastic differential equations. Despite versatility, study of online policies for learning from a single state trajectory to control the system, remains immature to date. We focus on this problem and present reinforcement learning policies for stabilizing unknown systems, and for minimizing quadratic cost functions. First, fast and reliable stabilization algorithms that utilize Bayesian learning methods will be discussed. Then, we propose policies that efficiently balance the exploration and exploitation by using randomizations in a fashion similar to Epsilon-Greedy and Thompson Sampling. Theoretical analyses showing regret bounds that grow with the square-root of time and with the number of parameters will be provided, together with experiments for different real systems. Further fundamental limitations of learning-based control will be discussed as well.
| ||
BIO | ||
Mohamad Kazem Shirani Faradonbeh obtained his PhD in Statistics from the University of Michigan in 2017, and his BSc in Electrical Engineering from Sharif University of Technology in 2012. He was a postdoctoral researcher with the University of Florida during 2017-2020, and a fellow with the Simons Institute at the University of California – Berkeley in 2020. Then he was an assistant professor of data science with the Department of Statistics and with the Institute for AI at the University of Georgia until 2023. He is currently with Southern Methodist University.
| ||