Online Reinforcement Learning in Continuous Time and Space

Event Type
Seminar/Symposium
Sponsor
DECISION AND CONTROL
Location
COORDINATED SCIENCE LAB, RM B02
Date
October 4, 2023 3:00 PM
Speaker
Assistant Professor Mohamad Kazem Shirani Faradonbeh of Southern Methodist University
Cost
Registration
Contact
Rhonda Henderson
Email
rrhender@illinois.edu
Phone
217-300-8511



A picture containing text, clipart

Description automatically generated

DECISION AND CONTROL LECTURE

THE GRAINGER COLLEGE OF ENGINEERING

TITLE

 

Online Reinforcement Learning in Continuous Time and Space

 

Sponsor

Decision and Control

Date

Wednesday, October 4, 2023

Time

3:00 PM

LOCATION

Coordinated science lab, rm b02

Speaker:  Assistant Professor Mohamad Kazem Shirani Faradonbeh 

                  Southern Methodist University

 

ABSTRACT

A canonical model for decision-making in uncertain continuous environments is that of linear dynamical systems that evolve as stochastic differential equations. Despite versatility, study of online policies for learning from a single state trajectory to control the system, remains immature to date. We focus on this problem and present reinforcement learning policies for stabilizing unknown systems, and for minimizing quadratic cost functions. First, fast and reliable stabilization algorithms that utilize Bayesian learning methods will be discussed. Then, we propose policies that efficiently balance the exploration and exploitation by using randomizations in a fashion similar to Epsilon-Greedy and Thompson Sampling. Theoretical analyses showing regret bounds that grow with the square-root of time and with the number of parameters will be provided, together with experiments for different real systems. Further fundamental limitations of learning-based control will be discussed as well.

 

BIO

Mohamad Kazem Shirani Faradonbeh obtained his PhD in Statistics from the University of Michigan in 2017, and his BSc in Electrical Engineering from Sharif University of Technology in 2012. He was a postdoctoral researcher with the University of Florida during 2017-2020, and a fellow with the Simons Institute at the University of California – Berkeley in 2020. Then he was an assistant professor of data science with the Department of Statistics and with the Institute for AI at the University of Georgia until 2023. He is currently with Southern Methodist University.