CSL Professor Kesh Kesavadas is working on improving virtual reality in a way that would cut development time in half and allow for realistic training of future medical professionals.

“Most of the current virtual reality technology tends to be fully computer graphics and that takes a very long time,” said Kesavadas, Health Care Engineering Systems Center (HCESC) director and Industrial and Systems Engineering professor. “We have a different approach: mixed reality. We mix 360® videos and a limited amount of computer graphics. We merge them and augment the videos.”

The combination of real-life videos with computer graphics is an entirely new approach that Kesadavas and his team presented at the prestigious IEEE Virtual Reality Conference in Osaka In March. Using their strategy they can develop virtual reality much quicker than traditional methods, which rely on computer graphics alone. The combination also serves well as an educational tool where interaction is key, such as in medical school education.

“The whole concept of medical education is active learning – students observing an operation and then eventually doing the procedure themselves” said Kesavadas. “There’s a lot of time, effort, and resources required to give this experience to medical students. Using our technology, students can have a level of experience that is equivalent to being in an operating room.”

Giving medical professionals experience with a disease or condition before they experience it with a patient is one of the most important opportunities this technology represents. Rather than interrupting patient treatment to show multiple groups of students a medical anomaly, current and future students can experience the rare case through mixed reality. This allows them to go through the steps of how to treat the issue, without delaying or hampering patient care.

Kesavadas was invited to preview the technology at the recent American College of Surgeons annual summit in Chicago. He showcased a curriculum his team developed for nurses and ER doctors at OSF in Peoria that he believes would be easily transferrable to medical schools.

“We have already built a few curriculum, including a module for teaching sepsis care to nurses and ER doctors in the OSF hospital in Peoria,” said Kesavadas. “We also have been creating one for teaching an introduction to medical robotics for medical students.”

Attendees at the summit provided feedback on how the technology could be integrated into their educational programs. Based on their feedback, Kesavadas is working with a startup company, AirV Labs, to translate this research into a commercial product while continuing to improve the machine learning and AI capabilities.

In addition to continuing to improve the technology, Kesavadas and his team wanted to test the viability of their invention in an educational setting. A mixed reality platform was built for the Champaign County Health Department as a way to train temporary food vendors. Previously, potential vendors would come to the county, watch a powerpoint, pass a test, and get a license. With mixed reality, the vendors can actually test their skills.

The group created a food fair that allowed vendors to go through the entire food preparation and serving process. They were able to see what they should and shouldn’t do. The active participation led to higher retention.

“There is all this well-known educational theory that says when you actively participate in learning, the retention is much higher, said Kesavadas. “Reading something about the Grand Canyon and going there are two different things. There are only so many things you can remember from reading, but if you go there it sticks in your mind.”

As mentioned before, medical school is largely based on active learning. Kesavadas believes this is why the medical community responded so favorably to the sepsis curriculum and to future ideas from the group.

“Medical directors of universities had really positive feedback about how this technology will make teaching more efficient,” said Kesavadas. “This concept is the future of medical education.”

This research was funded by the Jump ARCHES endowment and is a collaboration between OSF HealthCare and the University of Illinois at Urbana-Champaign.