Illinois leads $6.25M initiative to advance multimodal data analysis

An Illinois-led research team has received a $6.25 million Department of Defense Multidisciplinary University Research Initiative (MURI) award to develop a new information theory for data collection, analysis, and decision-making. The research, housed in the interdisciplinary Coordinated Science Lab, aims to speed up and improve our ability to collect and analyze data and subsequently adapt our decisions as new information comes in. Applications range from social network analysis to interactive machine learning with humans in the loop, such as brain computer/robot interfaces (BCI/BRI) or crowdsourcing.

The grant will allow Negar Kiyavash (associate professor of industrial and enterprise systems engineering and electrical and computer engineering) and her team to make advancements related to the non-commutative information structures that are intrinsic to hierarchical representations, distributed sensing, and adaptive online processing. In non-commutative information structures, the knowledge extracted from the data is dependent on the order of operations and direction of information flow.

Non-commutativity is intrinsic to emerging complex sensing and processing systems. The performance of a distributed sensor network depends on the ordering or partial ordering of the sequence of information sharing actions taken across the network. Multiuser brain-computer interfaces provide directed channels of neural communication from human to machine and between humans. To understand human activities depicted in a video, it is necessary to distinguish among different orderings of sequences of gestures and actions.

“In traditional decision-making, you gather all the information and then make a decision,” said Kiyavash, principal investigator on the project. “With adaptive exploitation, we’re fine-tuning our decision-making as we get in new information.”

In a social network, for example, the theory could help marketers understand how better to propagate their advertising messages. It could also help advance applications in other areas, such as recognizing and describing complex human activities from video, fusion of directed information flows, interactive machine learning with humans, and crowdsourcing and networks of brain-computer interfaces.

Kiyavash says the research will position Illinois as the international center of research in this new area; it will establish design principles for extracting knowledge from non-commutative information structures and fusing dynamic information from such structures, and will develop adaptive learning algorithms that manage the complexity and the large scale of the processing required. The research will be built on the theory of random matrices, free probability, and statistical machine learning, from an information theory perspective.

The MURI team includes researchers from the University of California, San Diego; the University of Michigan; the University of Wisconsin; Stanford University; Harvard University; and Princeton University. The award, officially titled “Adaptive Exploitation of Non-Commutative Multimodal Information Structure,” was awarded through the Army Research Office (ARO).