4/28/2021 Laura Schmitt, Illinois CS
To achieve the full promise of extended reality (XR) systems, Illinois researchers have launched an open-source testbed and, alongside key industry partners, a new consortium for the broad XR community to come together to accelerate progress in XR systems research, development, and benchmarking.
Written by Laura Schmitt, Illinois CS
To achieve the full promise of extended reality (XR) systems, Illinois researchers have launched an open-source testbed and, alongside key industry partners, a new consortium for the broad XR community to come together to accelerate progress in XR systems research, development, and benchmarking.
Extended reality, the collective name for virtual, augmented, and mixed reality, has the potential to transform many aspects of our lives, including the way we teach, conduct science, practice medicine, entertain ourselves, train professionals, and interact socially.
However, there is an orders of magnitude gap between the performance, power, and usability of current and desirable XR systems.
For example, said Sarita Adve, full immersion requires about 200 Mpixels resolution but the current state-of-the-art delivers about four Mpixels.
“One obstacle to impactful research in this area is the lack of open source full system benchmarks to drive and evaluate innovation,” Adve, a CSL professor, said. “The ecosystem of XR is mostly proprietary and the barrier to doing meaningful systems research in this area is very high.”
Recently, Adve and her students developed a first-of-its kind extended reality testbed that has the potential to democratize XR systems research, development, and benchmarking. The Illinois Extended Reality (ILLIXR) testbed is a fully open-source, end-to-end XR system. Computer architects, system designers, compiler writers, and algorithm and application developers can now use ILLIXR to research, develop, and benchmark their ideas in the context of a complete XR system, and see the impact on end-user experience.
According to CS graduate student Muhammad Huzaifa, the lead researcher on ILLIXR, XR development is technically very challenging because it brings together many different areas as part of one system—computer vision, audio, video, graphics, haptics, and optics—to handle things like hand and eye tracking, scene reconstruction, spatial audio, and 3D displays.
The ILLIXR testbed handles all the components from an XR workflow including perception, visual, and audio pipelines.
“Our end-to-end system can be used by computer architects, system designers, and compiler writers so they can understand how changing one thing in their area affects the user experience,” Adve, the Richard T. Cheng Professor of Computer Science, said.
Adve and Huzaifa recently presented ILLIXR at NVIDIA's GPU Technology Conference (GTC), a large industry conference, while introducing a new academic-industry consortium that brings key players to the table to expedite progress in XR systems research, development, and benchmarking. The consortium includes Arm, Facebook Reality Labs, Micron, NVIDIA, and others.
“ILLIXR’s open-source modular architecture enables the XR research community to address challenging problems in the areas of optimizing algorithms, system performance/power optimizations, scheduler development, and quality-of-service degradation,” said Facebook CPU Architect Rod Hooker.
“Innovating in the XR space requires a deep understanding of the complete software and hardware stack,” said Matt Horsnell, architecture research lead and senior principal engineer at Arm, a leading provider of silicon IP at the heart of billions of devices. “ILLIXR will enable more researchers, including Arm Research, to carry out full system investigations for XR.”
Adve’s team describes ILLIXR in the paper, Exploring extended reality with ILLIXR: A new playground for architecture research,” which is co-authored with four other graduate students and eight current or former undergraduate students. To learn more about the ILLIXR consortium or to join, please visit illixr.org.
ILLIXR Consortium
Adve and Huzaifa recently presented ILLIXR at NVIDIA's GPU Technology Conference (GTC), a large industry conference, while introducing a new academic-industry consortium that brings key players to the table to expedite progress in XR systems research, development, and benchmarking. The consortium includes Arm, Facebook Reality Labs, Micron, NVIDIA, and others.
Learn more at: http://illixr.org.
ILLIXR is funded in part by the Applications Driving Architecture (ADA) Research center (an SRC and DARPA sponsored JUMP center) and the National Science Foundation.