Space Party Rescue

Summary of Approach 

A VR stealth game with sensorimotor interaction, where you must navigate difficult listening situations during a spaceship party, and identify alien words between the sounds of the spaceship engine, music, and party-goers.

• Choose a group to listen in to figure out which speaker is the alien.
• Use special powers  to manipulate the target-to-masker ratio making it easier to understand the target. 
•  Give feedback about your experience to see how hearing ability improves (e.g. resistance to noise), enjoyment, usability, and the impacts of the interaction after repeated practice.

Subject-Matter/Problem Background

Cocktail parties represent the canonical example of a complex hearing scenario [1,2]. Regardless of hearing ability, we can all find difficulty in these scenarios and could benefit from improved hearing [3]. Additionally, 1 out of 5 people in the world suffer from hearing loss [4] and experience increased difficulty in these (important) situations. Some studies have shown that gamified sensorimotor interaction that includes actions that generate modifications on complex sounds can be conducive to improved hearing [5, 6].  Here, we created a space party game to assess and exercise speech intelligibility in complex environments to learn about people’s hearing ability and attempt to improve it.

XR Platform 

VR with Oculus MetaQuest 2

Project Presentation 

Click for Presentation Slides

Research References 

1. Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and two ears. JASA, 25(5), 975–979
2. McDermott, J. H. (2009). The cocktail party problem. Current Biology, 19(22), 1024–1027. 
3. Beck DL, Danhauer JL, Abrams HB, et al. Audiologic considerations for people with normal hearing sensitivity yet hearing difficulty and/or speech-in-noise problems. Hearing Review, 25(10), 28-38.
4. WHO (2021). World report on hearing. 
5. Whitton, J. P., Hancock, K. E., Shannon, J. M., & Polley, D. B. (2017). Audiomotor Perceptual Training Enhances Speech Intelligibility in Background Noise. Current Biology, 27(21), 3237-3247.

Team Members and Contributors

Ayanna Seals - Research Scientist

E. Sebastian L.L.M. - Research Scientist

Min Kwak - XR Designer & 3D Artist

Ai Xia - Game Designer & Producer

Pedro Sodre - Sound Designer & Audio Programmer

Sid Chou - Developer

Mariette Papic - Artist & Storyteller

Chella Quint - Researcher & Author