CSL student receives Best Paper Award

Seventeen years ago, CSL and ECE emeritus faculty member Bill Sanders co-founded the International Conference on Quantitative Evaluation of SysTems (QEST). Earlier this month, he and his advisee, Michael Rausch, won the 2020 Best Paper Award at the 2020 virtual conference.

The paper, “Sensitivity analysis and uncertainty quantification of state-based discrete-event simulation models through a stacked ensemble of metamodels,” presents a novel

approach the duo developed for conducting sensitivity analysis and uncertainty quantification of complex simulation models. The new approach uses a specially designed metamodel the pair built using machine learning. Current simulation models are complex, have a large number of input variables, and take a long time to solve. Rausch and Sanders’ method takes a faster approach by using a metamodel, which is a model of a model.

“I work with cybersecurity simulation models and they tend to be long running and slow, and they often have input variables that are uncertain,” said Rausch, a computer science Ph.D. student. “So I developed a better way to do sensitivity analysis and uncertainty quantification, which are some ways to do exploration of the input space.”

The large and slow traditional simulation models make it difficult to run analyses, so Rausch used machine learning to build a fast and accurate metamodel using regressor stacking. The current metamodel is only the beginning of the research’s capabilities.

“Because of this technique I can do those two things [sensitivity analysis and uncertainty quantification] much faster than was possible before, as well as more accurately than other similar metamodel-based methods,” said Rausch. “It’s a really promising technique but we’ve only applied it to one model. We are working on additional research to apply it to other models. I intend to do more extensions where I apply it to other models and fine tune the technique in the future.”

The research is a key contribution of Rauch’s Ph.D. thesis, and has important theoretical and practical implications. A recording of Rausch’s presentation at QEST Conference is available here.