Decision and Control Lecture Series: Probabilistic Approaches to Transfer Learning for Sparse and Noisy Data Environments

Abstract:

Machine learning (ML) models have thus far been applied to tasks and domains that, while impactful, have sufficient volume of data. For predictive tasks of national security relevance, ML models of great capacity are often needed to capture the complex underlying physics. Such models normally require an abundance of training data to exhibit sufficient predictive accuracy, which might not be available due to (1) excessive expense of computer simulations, (2) prohibitive experimental data acquisition cost, or (3) limited access to classified/sensitive data. To alleviate such difficulties, transfer learning (TL) may be used in which similar data from existing datasets or domains is used. This presentation will outline a novel probabilistic TL framework to enhance the trust in ML models within noisy and sparse data settings. The framework will assess when it is worth applying TL, which ML model to use in TL, and how much knowledge is to be transferred. It relies on extensions of concepts and techniques from the fields of Bayesian inversion, sequential data assimilation, uncertainty quantification, and information theory.­­ Insights will be provided through an application to polynomial-based surrogate model construction, all while highlighting the extent to which TL alleviates sparsity in training data that may jeopardize the reliability of such surrogates.

Biography:

Moe Khalil is a Senior Member of the Technical Staff at Sandia National Laboratories, Livermore, California, in the Quantitative Modeling and Analysis department. He also holds an adjunct research professor position in the department of Civil & Environmental Engineering at Carleton University, Canada. He has been developing and applying Bayesian inference algorithms for machine-learning, parameter estimation, data assimilation, and data-driven model selection, with applications in fluid-structure interaction, combustion modeling, radiation detection, nonlinear structural dynamics, wildfire forecasting, time-series analysis, material modeling, and near-shore wave forecasting for energy harvesting. He currently leads an R&D effort to develop novel probabilistic transfer learning methodologies to aid the training of machine learning and physics-based models in sparse and noisy data settings.