4/29/2021 Aaron Seidlitz, computer sciences
CSL professor Kris Hauser’s team of graduate students will continue working toward the $10M ANA Avatar XPRIZE – a large-scale global incentive competition – after earning one of 38 semifinalist selections. Most recently, this collaboration worked to adapt their creation – a Tele-Robotic Intelligent Nursing Assistant – according to the goals set as part of the ANA Avatar XPRIZE challenge.
Written by Aaron Seidlitz, computer sciences
CSL professor Kris Hauser’s team of graduate students will continue working toward the $10M ANA Avatar XPRIZE – a large-scale global incentive competition – after earning one of 38 semifinalist selections.
Most recently, this collaboration worked to adapt their creation – a Tele-Robotic Intelligent Nursing Assistant (TRINA) – according to the goals set as part of the ANA Avatar XPRIZE challenge.
Hauser's work on this project started as NSF-supported research between collaborators at Illinois computer science and Duke University in 2019. His team has grown to include an augmented reality gaming startup company called VRotors.
TRINA is a robotic avatar that allows the operator to see, hear, navigate and interact with a remote environment. Its development showcases the team’s dedicated effort in artificial intelligence and robotics research.
TRINA’s inclusion in the ANA Avatar XPRIZE semifinals validates it as one of the world’s preeminent efforts in the use of avatars. The reward is a $10 million prize purse that will ultimately recognize a single winner. That project will integrate “multiple emerging technologies to develop a physical, non-autonomous Avatar System with which an operator can see, hear, and interact with a remote environment in a manner that feels as if they are truly there.”
“It’s incredible to be a part of the semifinals of a competition that started with 77 teams from 19 different countries,” Hauser, professor of CS, said. “I’ve long believed that robotic avatars could have the power to connect people even when they can’t be physically present, and the COVID pandemic has provided us with even stronger motivation to develop TRINA’s capabilities.”
The team’s semifinal video submission demonstrated TRINA conducting a remote medical appointment scenario and its safety features. Thus, proving it capable of delivering communication, mobility manipulation and sensing for nursing, in-home care, tourism and maintenance applications.
Hauser and the students’ path to this semifinal stage of the ANA Avatar XPRIZE began about three years ago. After entering the contest, they first cleared a benchmark to become a “Qualified Team” in January 2020.
Then they worked toward becoming a semifinalist selection through a second round of evaluation conducted by the prize’s judging panel.
“We’ve integrated modern robotics with technological advances in virtual reality, low latency networking, and force sensing to let the operator control TRINA naturally,” Hauser, Center for Autonomy faculty member, said. “With a bit of training, almost anyone can drive the robot around, manipulate objects, and communicate with others as if they were actually there.”
In semifinals testing, Hauser and the students need to prove TRINA capable of:
- Participating in a “Testing Run” with up to three judges. One operated the avatar, one served as the recipient at the remote location, and a final judge observed.
- Succeeding through three challenging scenarios conducted by the judges. These challenges included four options – a social interaction, a business interaction, a healthcare activity and a culture/teaching/travel activity.
- Providing a sense of presence in a remote location through interactions with a judge that displayed recognition, understanding, communication, gestures, and shared situational awareness.
Moving forward, TRINA’s team will work toward semifinals testing, which concludes in September. The competition will announce finalists in October, and finals testing takes place in June 2022 – after which a winner will be named.
Under Hauser’s direction, this team of nine graduate students – spanning three departments within the Grainger College of Engineering – has enjoyed the collaborative learning experience.
Their expertise has grown in areas like backend development, semi-autonomous functionality, mechanical components, direct tele-operation, perception, UI and grasping.
"Working on TRINA is an incredibly daunting task. It is a complex system with a huge codebase that constantly requires interdisciplinary work to function properly. As a natural consequence of this complexity, this project is a great avenue for collaborative work – and there is collective joy whenever we see TRINA reach another one of our milestones,” said Joao Correia Marques, Illinois CS PhD student. “We know every little victory is a product of all our efforts as a team, which makes this a truly rewarding teambuilding experience.”