Alampounti, Chint Lida; (2024) Assessing how vision benefits speech-in-noise perception and the impact of ageing and hearing loss on audiovisual benefits. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text Alampounti__thesis.pdf - Submitted Version Download (7MB) | Preview

Abstract

Investigations of the role of audio-visual integration in speech-in-noise perception have, in the previous decades, largely focused on the benefit provided by lipreading cues. A growing body of evidence however suggests that additional audio-visual processes exist, and that these have the capacity to influence auditory scene analysis. Specifically, audio-visual temporal coherence – the extraction, and integration, of the temporally correlated information of the amplitude envelope of speech and the opening and closing of the mouth – is a potential candidate. Whether audio-visual temporal coherence can aid speech-in-noise, and to what extent, relative to lipreading, remains however largely unknown. The current work examines, across 125 individuals spanning the ages 19 to 85, and with both typical and impaired hearing, the contributions of these two mechanisms to the visual enhancement of speech-in-noise perception. An audio-visual speech-in-noise task (the vCCRMn) was developed, designed to capture both lipreading, and audio-visual temporal coherence-related enhancements of listeners’ auditory performance. The vCCRMn task was employed, along with a battery of accompanying tests, in three experimental groups: younger participants with normal hearing, older participants with normal hearing, and older participants with hearing loss. vCCRMn presented participants with sentence stimuli, containing two key words which participants had to report. These target sentences were presented in the background of two competing (masker) talkers. In audio-visual (AV) and interrupted (Inter) conditions of the task, participants were also presented with a video of a talker. This talker could either match the speaker reading the target sentences (target-coherent conditions), or one of the talkers reading the masker sentences (masker-coherent conditions). In the Inter condition the video of the talker would freeze during the key words – thus, removing the possibility of participants exploiting lipreading cues for identification of the target words. A static-image-with-audio (A) condition was also employed as a control, and baseline for comparing against the video conditions to compute visual enhancements. The image displayed in the condition could also be target- or masker-coherent. Among the target-coherent conditions, participants performed the best in the AV, followed by the Inter, and then the A condition. Thus, participants were gaining visual enhancements from both the AV and the Inter video conditions, compared to the static image condition, with the visual enhancement obtained from the AV condition being the largest. This finding suggested that participants’ auditory performances were at the same time enhanced by both lipreading, and audio-visual temporal coherence cues. Additionally, participant performances were better in the target coherent video conditions, compared to the masker video conditions. Performances in the AV masker video condition were, in fact, even worse than those in the static-image-with-audio condition, suggesting that obligatory dynamic visual cues can impair the listening experience, as well as support it. Finally, participant speech-in-noise performance, and audio-visual benefit were negatively impacted by both ageing, and hearing loss.

Download activity - last month

Export as CSVXMLJSON

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item