‘Explainable AI’ can efficiently detect AR/VR cybersickness

Exposure to an augmented reality (AR) or virtual reality (VR) environment can cause people to experience cybersickness—a special type of motion sickness with symptoms ranging from dizziness to nausea—and existing research to mitigate the severity of the symptoms often relies upon a one-size-fits-all approach.

However, Khaza Anuarul Hoque, an assistant professor in the Department of Electrical Engineering and Computer Science at the University of Missouri, and a team of researchers are working to develop a personalized approach to identifying cybersickness by focusing on the root causes, which can be different for every person.

“Cybersickness is not generic. For instance, one simulation could trigger cybersickness in me while the same simulation may not cause cybersickness for someone else,” said Hoque, who also directs the Dependable Cyber-Physical Systems Laboratory at MU.

“One of the problems people typically face when wearing virtual reality or augmented reality headsets is the user experience can get bad after some time, including symptoms of nausea and vomiting, especially if the user is immersed in a virtual environment where a lot of motion is involved. It can depend on many factors, including a person’s gender, age and experience.”

Hoque said he wanted to concentrate on a novel angle using explainable AI since it has the potential to transform the AR and VR industry.

“Explainable AI is a great tool to help with this because typically machine learning or deep learning algorithms can tell you what the prediction and the decision may be, whereas explainable AI can also tell the user how and why the AI made the decision,” Hoque said. “So, instead of imposing a static mitigation technique for all users, it will be more effective if we know why a particular person is developing cybersickness and give that person the right mitigation that they need. Explainable AI can help us do that without hindering the user experience.”

In addition to watching his own students experience cybersickness, Hoque noticed academic and industry approaches for identifying cybersickness over the past five to seven years often focused on data-driven techniques such as machine learning (ML) and deep learning (DL).

“Such approaches are often ‘black box,’ and thus, they lack explainability,” Hoque said. “I also realized that the explainability of the DL cybersickness models can significantly improve the model’s understanding and provide insight into why and how these AI models arrived at a specific decision. Furthermore, by identifying and understanding what types of important features could lead to cybersickness, we can help designers develop more effective cybersickness detection models.”

Hoque said explainable AI can also help software developers identify the most important features needed to optimize the model for teaching the AI how to identify someone experiencing cybersickness. This is especially important for users wearing stand-alone VR headsets.

This research was recently presented at three conferences for AR/VR research:

• “LiteVR: Interpretable and Lightweight Cybersickness Detection using Explainable AI” was presented at the IEEE Virtual Reality Conference on March 25-29, 2023.
• “VR-LENS: Super Learning-based Cybersickness Detection and Explainable AI-Guided Deployment in Virtual Reality” was presented at the ACM Conference on Intelligent User Interfaces on March 27-31, 2023.
• “TruVR: Trustworthy Cybersickness Detection using Explainable Machine Learning” was presented at the International Symposium on Mixed and Augmented Reality (ISMAR) Conference on October 17-21, 2022.

“I have talked and worked with different groups of people and different countries, and that really helped me look at the same problem but with different angles,” Hoque said. “I think it’s really important for any researcher not to look at a problem from just one perspective, but from multiple perspectives.”