DCL Seminar: Neil Walton - Decentralized and Reinforcement Learning Approaches to Congestion Control

Decision and Control Laboratory

Coordinated Science Laboratory

“Decentralized and Reinforcement Learning Approaches
to Congestion Control”

Neil Walton, Ph.D.

Head of Probability and Statistics

University of Manchester

Wednesday, March 13, 2019

3:00pm – 4:00pm

CSL Auditorium (B02)

____________________________________________________________________________________________________________________________________________________________________

Abstract:
We discuss two approaches to congestion control at traffic lights. First we look at implementations inspired by Internet congestion control. In particularly we look to compare and contrast the performance of the BackPressure policies and Proportional fair policies. Both policies have the benefit of being maximally stable in switched queueing networks. We discuss some advantages of the proportional fairness in the context of queues which must contain main types of traffic.

In the second part of the talk, we consider on going work on the implementation of reinforcement learning algorithms for traffic control. Here a simulation model is used to train a network under a variety of different loads. This fits a parametrized Markov decision process and then we can look to assess the performance of the resulting policy. We discuss the benefits but also the processing and computational constraints in making this approach a practical replacement for current signal control systems.

Bio:
Neil Walton received his undergraduate ('05), Masters ('06) and PhD ('10) in Mathematics at the University of Cambridge. His research in applied probability principally concerns the decentralized minimization of congestion in networks. He was a lecturer at University of Amsterdam where he held an NWO Veni Fellowship. He then moved to the University of Manchester where he is a Reader in Mathematics and is currently the Head of the Probability and Statistics group. Neil has conducted extended research visits at Microsoft Research Cambridge, the Basque Centre for Mathematics and the Automatic Control Laboratory ETH Zurich. Neil is a Fellow of the Alan Turing Institute where he is co-investigator on the project “Artificial Intelligence for Transport Planners”. He is an associate editor at the journals Operations Research and Operations Research Letters. He has twice won best papers awards at the ACM Sigmetrics conference and he was awarded the 2018 Erlang Prize by the Informs Applied Probability Society.