CSL team develops award-winning cluster computing software
The project, “A General-Purpose Distributed Programming System using Data-Parallel Streams,” was part of ongoing research conducted by Huang, his former advisor Electrical and Computer Engineering Professor Martin Wong, and PhD students Chun-Xun Lin and Guannan Guo.
“This project is focused on developing a new programming system to help streamline the building of high-performance distributed applications,” Huang said. “We wanted our efforts to make an accessible open source system people could use.”
Distributed computing and parallel processing techniques can make a significant difference in many disciplines such as cloud computing, machine learning, and multimedia. An example of distributed computing and parallel processing in action would be aircraft control systems which much operate at various locations (airports) around the country.
As one might expect, programming a distributed application is extremely difficult and requires a substantial amount of software engineering efforts. Huang and his research team created a new programming model that largely simplifies this process. Compared with traditional frameworks, their system can reduce the programming complexity by 2-10x, while still meeting high performance standards.
“We are excited to apply our tool to help DARPA develop new distributed software to create a new era of 24-hour design cycle for Department of Defense hardware systems,” Huang said.
Early this September, the group’s work also stood out at the C++ Conference in Washington. Huang, Lin, and Guo presented a project called Cpp-Taskflow, which was derived from the original project. Cpp-Taskflow focused on a new parallel task programming model for the single multicore architecture. The project aimed to help C++ developers quickly write efficient parallel programs. The group was awarded the Best Poster award for their efforts.
“Winning these awards is great encouragement for us,” said Huang. “Especially since this is a multi-year effort and we have spent a lot of time on the system. This will encourage more in-depth research in this area.”
The project is being supported by a DARPA grant under the Intelligent Design of Electronic Assets (IDEA) program.