Effect of Cross-Modal Perceptual Training on Audiovisual Integration in Older Adults ()
1. Introduction
*These authors have contributed equally to this work and share first authorship.
Our perceptual system continuously receives extensive information through various sensory modalities, including vision, hearing, touch, taste, and smell. To create a unified and coherent perception of the surrounding world, the brain integrates these inputs based on their temporal and spatial proximity, constructing an accurate representation of the environment [1] [2]. When multiple senses contribute to perception, the brain cross-references information across modalities to enhance accuracy. For instance, while watching a movie, the visual system processes on-screen images, and the auditory system interprets corresponding sounds. The brain integrates this information into a singular perception, known as audiovisual integration (AVI), that is both visually and audibly rich [3].
Extensive animal and human studies have proved that integrating information from visual and auditory sensory modalities enhances perceptual accuracy and increases our understanding of the environment [3]-[5]. However, older adults exhibit serious sensory declines, such as reduced visual acuity and increased auditory thresholds, resulting in lower sensitivity to visual and auditory discrimination [6]. Interest in mitigating age-related perceptual deficits has grown due to its close association with falls [7] and cognitive and functional impairments in daily life [8]. To alleviate unimodal sensory decline, older adults expand the audiovisual integration window to accommodate perceptual deficits. Additionally, numerous studies report increased audiovisual integration (AVI) in older adults as a compensatory response to unimodal sensory decline [9]-[11].
Older adults experience difficulty with audiovisual simultaneous judgment (SJ) and temporal order judgment (TOJ) [12] [13], which are associated with a higher risk of falls and varying levels of cognitive decline in daily life. Numerous studies have confirmed that the audiovisual binding window was shaped by SJ training, indicating the plasticity of cross-modal perceptual ability; however, it remains unclear whether AVI itself can be improved. This study specifically aimed to clarify whether cross-modal audiovisual perceptual training designed to enhance hearing and vision perception in older adults would also positively influence AVI.
2. Methods
2.1. Participants
40 community-dwelling older adults (51 - 75 years, mean age ± SD, 59.1 ± 5.9) from Guiyang City were recruited and randomly divided into a training group and a control group. All participants were cognitively healthy, with Mini-Mental State Examination (MMSE) scores above 26, and were not taking any medications with central nervous system effects. Before the experiment, all participants were given informed content approved by the Medical Ethics Committee of the Second Affiliated Hospital of Guizhou University of Traditional Chinese Medicine.
2.2. Apparatus, Stimuli, and Procedure
All participants conducted the experiment in a dimly lit, electrically shielded, and sound-attenuated room at the Psychology Laboratory of Guizhou University of Traditional Chinese Medicine, China. The presentation of stimuli and the collection of behavioral data were managed using E-Prime 3.0 (Psychology Software Tools, Inc., Pittsburgh, PA, USA) during both the training and assessment tasks.
The audiovisual perceptual training consisted of five separate sessions completed over five consecutive days. On the first day, before training, each participant in both the training and control groups was instructed to perform the SJ task and audiovisual detection task, referred to as the pre-training assessment (Pre-test, Figure 1). In the following five days, the training group followed a personalized training schedule, while the control group received no training and remained isolated from the training group. On the final day, after training, participants in both groups performed the same assessment tasks as on the first day, referred to as the post-training assessment (Post-test, Figure 1). All experimental tasks were conducted under the same environmental conditions in our psychology laboratory to minimize the influence of unrelated factors.
Figure 1. The chart for cross-modal perceptual training.
For the perceptual training task (see Figure 2), adapted from previous studies, the visual stimulus (V) was a 10% contrast, 1.5 spatial frequency vertical Gabor with a 5˚ visual angle (52 mm × 52 mm) displayed on a gray 60-cm background monitor (Samsung S19E200). The auditory stimulus (A) was a 1000 Hz, 60 dB sinusoidal tone with a 12 ms duration, presented via a loudspeaker positioned behind the monitor. According to previous studies, the training task was divided into 10 difficulty levels, with stimulus onset asynchronies (SOAs) between the auditory and visual stimuli ranging from 300 ms (level 1) to 30 ms (level 10) in 30 ms intervals. Each level presented 20 synchronous and 20 asynchronous audiovisual stimulus pairs in random order, lasting approximately 2.5 minutes per level. Participants pressed the left mouse button for synchronous stimuli and the right button for asynchronous stimuli. Training began at a medium difficulty level (150 ms SOA), with difficulty adjusted based on participant performance: advancing with three consecutive successful trials (accuracy > 80%), decreasing with a failed trial (accuracy < 60%), or repeating with moderate accuracy (60% ≤ accuracy < 80%). Training lasted one hour per day with self-paced breaks between levels.
The cognitive assessment included the SJ task and audiovisual detection task. For the SJ task, the same experimental materials and paradigm were used, except that audiovisual stimuli with stimulus onset asynchronies (SOAs) ranging from 0 to 300 ms (0, 30, 60, 90, 120, 150, 180, 210, 240, 270, 300 ms) were presented randomly. To balance responses, synchronous and asynchronous stimuli were presented in a 1:1 ratio. In total, there were 600 trials, consisting of 300 synchronous and 300 asynchronous stimuli (15 trials per SOA condition), divided into four blocks with appropriate rest periods between blocks.
For the audiovisual detection task (see Figure 3), the visual stimuli were 10% contrast, 1.5 spatial frequency Gabor patches, including both vertical and horizontal orientations. The auditory stimuli were sinusoidal tones at 1000 Hz and 500 Hz. Audiovisual stimuli were presented as combinations of vertical Gabor patches with 1000 Hz tones and horizontal Gabor patches with 500 Hz tones; no other combinations of auditory and visual stimuli were used in this study. All visual stimuli were presented at the center of a gray-background monitor, positioned 60 cm in front of the participant, while auditory stimuli were delivered through speakers located centrally behind the monitor at 60 dB. The experiment began with a fixation cross (“+”) at the center of the screen for 3000 ms as in previous studies. Following fixation, auditory, visual, and audiovisual stimuli were presented randomly for 100 ms with an interstimulus interval (ISI) ranging randomly from 1800 to 3000 ms. Participants were instructed to respond to each stimulus. In total, there were 300 trials, lasting 12 minutes, divided into two blocks with a self-paced break.
![]()
Figure 2. Schematic depiction of the perceptual training task.
Figure 3. Schematic depiction of the audiovisual detection task.
2.3. Data Analysis
For the SJ task, the probability of simultaneity judgments at each SOA was calculated for each participant in both pre- and post-tests. A two-tailed t-test was then conducted for each SOA. Finally, the individual means at each SOA were averaged to generate the grand average plots (see Figure 4).
For the audiovisual detection task, hit rates and response times (RTs) were computed separately for each participant and analyzed using a 2 (Group: training, control) × 2 (Test: pre, post) × 3 (Stimulus: A, V, AV) repeated-measures ANOVA, followed by post hoc analysis with Greenhouse-Geisser corrections and adjusted degrees of freedom. Statistical analyses were conducted using IBM SPSS Statistics 22.0 (IBM Corp., Armonk, NY, USA), with statistical significance set at p ≤ 0.05. Additionally, audiovisual integration (AVI) was assessed using a race model based on cumulative distribution functions (CDFs) [14] [15].
3. Results
3.1. SJ Task
A two-tailed t-test revealed a significant improvement in perceptual ability in the post-test compared to the pre-test across all SOA conditions (Figure 2, all ps ≤ 0.014).
Figure 4. The mean probability of simultaneity judgment in each SOA before and after perceptual training.
3.2. Audiovisual Detection Task
Hit rates and response times (RTs) are shown in Table 1. A three-factor repeated-measures ANOVA for hit rate revealed a significant main effect of the test [F (1, 38) = 3.864, p = 0.047], indicating higher accuracy in the post-test compared to the pre-test, and a significant main effect of stimulus [F(2, 76) = 23.454, p < 0.001], with accuracy for AV stimuli higher than for A and V stimuli (all ps ≤ 0.011).
Table 1. Mean response times (RTs) and hit rates for each stimulus.
|
|
Pre-test |
Post-test |
|
V |
A |
VA |
V |
A |
VA |
Response times (ms) |
Training |
Mean |
436 |
473 |
375 |
472 |
563 |
363 |
SD |
86 |
140 |
103 |
86 |
122 |
97 |
Control |
Mean |
510 |
537 |
442 |
479 |
528 |
430 |
SD |
81 |
151 |
103 |
99 |
169 |
121 |
Hit rate (%) |
Training |
Mean |
90 |
93 |
96 |
93 |
93 |
97 |
SD |
7 |
8 |
2 |
6 |
9 |
1 |
Control |
Mean |
86 |
92 |
95 |
91 |
95 |
96 |
SD |
8 |
8 |
3 |
7 |
4 |
1 |
The three-factor repeated measures ANOVA for RTs revealed a significant main effect of stimulus [F(2, 76) = 85.702, p < 0.001], indicating faster responses to audiovisual (AV) stimuli compared to auditory (A) and visual (V) stimuli (AV > V > A, all ps < 0.001). The interaction between the group and test approached significance [F(1, 38) = 3.018, p = 0.059]. Post hoc analysis indicated that responses to A and V were faster in the pre-test compared to the post-test, whereas responses for the control group were either comparable or reversed. No other significant main effects or interactions were identified.
Additionally, based on responses to auditory and visual stimuli, the race model (RTrace model) was computed following previous studies [1] [3] and compared with responses to audiovisual stimuli (RTAV, see Figure 5). In the training group, audiovisual integration (AVI) was significantly higher in the post-test (13.87%) compared to the pre-test (9.95%). However, in the control group, AVI remained relatively stable between the pre-test (10.77%) and post-test (9.61%). These findings suggest that perceptual training enhanced AVI.
4. Discussion
To determine whether audiovisual integration (AVI) is influenced by changes in audiovisual perceptual sensitivity through simultaneous judgment (SJ) training, a five-day consecutive perceptual training program was conducted. The results revealed that AVI was higher in the post-test compared to the pre-test for the training group.
Consistent with previous studies showing that audiovisual perceptual training can improve sensitivity in information detection, this study is the first to report enhanced audiovisual integration (AVI) through perceptual training. Age-related
Figure 5. The audiovisual integration for training group (A) and control group (B) before and after perceptual training.
differences in AVI have been extensively studied in both behavioral and electrophysiological contexts. Laurienti et al. [9] were the first to find enhanced AVI in older adults compared to younger adults, a finding subsequently confirmed by numerous studies [8] [10]. They further suggested that this enhanced AVI functions as an adaptive compensatory mechanism to mitigate the decline in unimodal sensory function in older adults.
Using magnetoencephalography (MEG), Diaconescu et al. [11] found that, during audiovisual information processing, brain activity in the medial prefrontal cortex and posterior parietal cortex was significantly higher in older adults than in younger adults after 100 ms of stimulus presentation. Ren et al. [16] observed that older adults specifically recruited the traditional visual cortex to process audiovisual information, as measured through electroencephalography (EEG). They suggested that older adults increase their cross-modal integrative ability to compensate for unimodal sensory decline. Moreover, studies by Wang et al. [17] and Ren et al. [18] demonstrated that global functional connectivity was higher in older adults compared to younger adults during audiovisual information processing, further supporting the compensation mechanism theory. Recent functional magnetic resonance imaging (fMRI) studies comparing hemodynamic differences between older and younger adults during audiovisual information processing revealed that, in comparison to younger adults, older adults exhibited greater activation in dorsal frontal regions, including the middle and superior frontal gyri, as well as in dorsal parietal regions [19].
5. Conclusion
The findings of this study suggest that the enhanced audiovisual integration (AVI) observed in the post-test for the training group may reflect an increased compensatory capacity in older adults. However, further research utilizing electrophysiological and magnetic resonance imaging techniques is necessary to elucidate the underlying neural mechanisms, specifically by comparing age-related changes in AVI before and after perceptual training. This study highlights the potential of targeted perceptual training to reinforce compensatory mechanisms, providing a promising avenue for cognitive interventions aimed at improving sensory and cognitive function in older adults.
Acknowledgements
During the course of this research, we benefited from the support and assistance of various departments, individuals, and external personnel. We extend our heartfelt appreciation to everyone who contributed to this project and offer our best wishes for their well-being and future endeavors. The authors would also like to acknowledge Guizhou University of Traditional Chinese Medicine for their generous support.
Funding
This study was supported by the Guizhou University of Traditional Chinese Medicine Undergraduate Teaching Quality Project Construction Program Fund (2022 Guizhou Province Undergraduate Training Program for Innovation and Entrepreneurship, S202210662067).