Holoband
Abstract
HoloBand is a clinical tool for audiovisual music perception training in augmented reality (AR) using the Hololens 2 for the hard-of-hearing (HoH). It consists of two interfaces: a spatial environment in AR for the HoH user and a computer interface to control the training for speech and language pathologists (SLP). We adopt a participatory exploratory research approach to investigate novel ways of conducting timbre discrimination, audio localization, and pitch recognition. HoloBand is evaluated using a focus group with SLPs (n=6) and interpreted through inductive thematic analysis. Hololens 2 shows relevant contextual, visual capabilities to enhance music perception training. However, a closer integration into the clinical workflow and a task-oriented approach is necessary to bring real-life scenarios to the clinical setting.
Collaborating with Rigshospitalet in Copenhagen, we were able to run a series of workshops with a HoH person to design and develop this clinical tool to help train music perception.
The program runs in two parts. An interface was designed for the clinician to control the program which was running on the Hololens 2 worn by the patient. The clinician could then manipulate the postions of multiple instruments playing a song in unison. The could also manipulate an envolope filter to test different ranges of pitch detection, as well as turning the visuals on or off individually.
We were able to run an informal focus group with experts in the field from Rigshospitalet to showcase the application and gain insight into the usefulness of this solution.
In these images you can see the clinicians interface, where they can drag the instruments around the patient (represented by the black dot) as well as manipulate the audio via the cards on the left.
Holoband was published at the AES 2022 conference.
Ivanyi, B., Tsalidis, C., Naylor, S., Tjemsland, T., Adjorlu, A., Kepp, N. E., & Serafin, S. (2022, August). HoloBand: An Augmented Reality Experience to Train Music Perception for the Hard of Hearing. In Audio Engineering Society Conference: AES 2022 International Audio for Virtual and Augmented Reality Conference. Audio Engineering Society.