Decision and Control Fall Seminar Series: Bahman Gharesifard || A geometric view of the universal approximation power of deep neural networks

The talk is concerned with the universal approximation capabilities of deep residual neural networks with respect to the uniform norm. We relate the universal approximation problem to controllability of an ensemble of control systems corresponding to sample points on the function to be approximated. We leverage classical Lie algebraic techniques to identify a class of activation functions, encompassing many known ones, that ensure controllability. We then identify monotonicity as the bridge between controllability of finite ensembles and uniform approximability on compact sets. 

Biography: Bahman Gharesifard is currently a Professor with the Electrical & Computer Engineering Department at the University of California, Los Angeles. He was an Associate Professor, from 2019 to 2021, and an Assistant Professor, from 2013 to 2019, with the Department of Mathematics and Statistics at Queen's University. He was an Alexander von Humboldt research fellow with the Institute for Systems Theory and Automatic Control at the University of Stuttgart in 2019-2020. He held postdoctoral positions with the Department of Mechanical and Aerospace Engineering at University of California, San Diego 2009-2012 and with the Coordinated Science Laboratory at the University of Illinois at Urbana-Champaign from 2012- 2013. He received the 2019 CAIMS-PIMS Early Career Award, jointly awarded by the Canadian Applied & Industrial Math Society and the Pacific Institute for the Mathematical Sciences, a Humboldt research fellowship for experienced researchers from the Alexander von Humboldt Foundation in 2019, an NSERC Discovery Accelerator Supplement in 2019, and the SIAG/CST Best SICON Paper Prize 2021. He was a finalist (as an advisor) for the Best Student Paper Award at the American Control Conference in 2017. He received the Engineering and Applied Science First Year Instructor Teaching Award in 2015 and 2017. He has served on the Conference Editorial Board of the IEEE Control Systems Society and IEEE Control System Letters, and is currently an Associate Editor for the IEEE Transactions on Network Control Systems. His research interests include systems and control, distributed control, distributed optimization, machine learning, social and economic networks, game theory, geometric control theory, geometric mechanics, and applied Riemannian geometry.