10/12/2020 Allie Arp, CSL
Written by Allie Arp, CSL
Each year at the IEEE High Performance Extreme Computing Conference (HPEC), teams from academia, national laboratories, and industry pit big data graph analysis software systems they’ve developed against each other. In this year’s Sparse Deep Neural Network (SpDNN) Graph Challenge task, members of a CSL-led team were named champions.
The Sparse DNN Challenge draws upon the community’s latest understanding of issues surrounding machine learning and high-performance computing. It is reflective of the emerging sparce computation patterns of modern artificial intelligence solutions and their demanding requirements of current high-performance computing systems.
“We deal with algorithms every day in our research group and sometimes we get really frustrated and think we aren’t good enough with our small group in CSL,” said MertThe team’s winning entry builds on members’ deep expertise in computer architecture and graphics processing units (GPUs), algorithmic design, performance analysis, and rigorous implementation of high performing AI solutions. A key innovation of the new algorithms is the carefully crafted kernels and data layouts that fully utilize the on-chip memory bandwidth, which improves the throughput while reducing the required energy.
“We have been participating in the HPEC Graph Challenge for the past four years on different challenge tasks,” said Jinjun Xiong, IBM researcher and co-director of the IBM-ILLINOIS Center for Cognitive Computing Systems Research (C3SR), with which most of the team members are affiliated. “This year’s winning as Champion in the SpDNN task is really a culmination of our Center’s years of persistence in understanding the computation bottlenecks of modern AI and big graph analytics workloads, in addition to our relentless efforts to optimize systems and algorithms for highest possible performance.”
One unique trait that set the team apart from the rest crowd, Hidayetoglu believes, was the team’s commitment to test“It shows the algorithms we develop are battlefield tested,” Hidayetoglu said. “This international recognition shows we are doing state-of-the-art work. This gives more visibility and publicity to our Center.”
In addition to the influence of C3SR from several students and Professor Emeritus Wen-Mei Hwu, the team also had contributions from partners at IBM and NVIDIA that helped team develop their now award-winning algorithms.
Other team members include Professor Rakesh Nagi, industrial and enterprise systems engineering; Carl Pearson and Vikram Sharma Mailthody, ECE; and Eiman Ebrahimi, NVIDIA. The challenge is sponsored in part by IEEE, MIT, and Amazon.