Study explores how the brain integrates auditory and visual memories

When a person remembers their friend telling them a funny story, they associate the sound of that friend talking with the appearance of that friend speaking and laughing. How does the human brain form audiovisual memories like this? In a new JNeurosci paper, Emmanuel Biau, from the University of Liverpool, and colleagues addressed this question by exploring brain activity linked to forming memories that integrate sounds and visual information. 

The researchers elicited memories in study participants by presenting them with movie clips of people speaking. They manipulated when sounds and visual information were presented in the movie clips to explore the impact on brain activity during memory recall. Movie clips with speech sounds and lip movements occurring at the same time were linked to oscillatory activity in two brain regions during viewing of the clips. This oscillatory activity reoccurred when participants remembered the movie clips later. But movie clips with speech sounds lagging behind lip movements reduced oscillatory activity during viewing as well as during memory recall.

Says Biau, "We assume that if auditory and visual speech inputs arrive in the brain at the same time, then their chance of being associated in a memory is much higher because they will fall into the same phase of neural activity, which is not the case for asynchronous stimuli."

According to the authors, their work suggests that the oscillatory activity in these two brain regions may play a crucial role in integrating auditory and visual information during memory recall, though more work is needed to confirm this.