It can be difficult to carry on conversation in a crowded public setting, and even more so with any degree of hearing loss. But what if you could amplify only the person you wanted to hear and suppress the rest? What if a computer could do that automatically by reading your brain?
When we focus on a particular person talking, we subconsciously track the gradual modulations in speech volume, which vary from speaker to speaker. This characteristic pattern appears in the brain activity of the listener. And in recent years, researchers have been able to find the signature speech pattern in a brain recording, then identify the voice being listened to, using a technique called auditory attention decoding (AAD).
This fundamental neuroscience is now stepping into the realm of practical medicine. By testing the actual experience of listeners, researchers are taking a step toward one day incorporating attention-based control into hearing aids. In a study published 11 May in Nature Neuroscience, researchers presented listeners with two competing voices, then applied real-time volume adjustments in response to brain activity. The altered audio improved understanding, reduced listening effort, and was simply preferred by listeners.
The study serves as an important proof of principle. “It validates the core idea that brain-controlled hearing enhancement can improve perception, while also making clear what still needs to be solved before this could become practical for patients,” says Inyong Choi, an engineer and psychoacoustician at the University of Iowa in Iowa City, who was not involved in the research.
Amplifying the voices you want to hear
According to the World Health Organization, more than 400 million people worldwide have disabling hearing loss. In the United States, roughly 15 percent of adults live with some form of hearing loss, often related to aging. Hearing deficits can have serious social and mental health consequences. On 7 May, the Advanced Research Projects Agency for Health (ARPA-H) announced a new funding program for hearing aid research with stated goals including neural control or feedback.
Hearing aids are more technologically advanced than ever, says Bharath Chandrasekaran, a neuroscientist who studies hearing and the brain at Northwestern University in Chicago, Ill. But they still tend to struggle in noisy environments with multiple speakers, the sort of challenging situation when people might most want assistance. “That needs a little direction. That’s where this auditory attention decoding helps,” says Chandrasekaran.
The study recorded the brain activity of four subjects with typical hearing with implanted electrodes capable of gathering high-quality electroencephalography (EEG) data originally designed for epilepsy monitoring. Sat in front of a computer, they were asked to listen closely to one talker or another as recordings were played simultaneously from two different audio speakers. The AAD system tracked their attention, then began adjusting volume after a few seconds.
Though there was a range, all four subjects reported greater understanding of what was being said more often when the AAD was turned on. Listening effort, as indicated by the proxy of pupil size, was reduced in the two subjects it was measured for. And all subjects preferred when the AAD was on at least 75 percent of the time.
A panel of 40 participants with hearing loss then listened to the same voices with and without adjusted volume based on the main subject’s EEGs, and also benefited in comprehension and preference.
Researchers also looked at subjects redirecting their attention between the two speakers, both on command and by choice. The system was able to switch to the preferred speaker on the fly in about five seconds. Because listeners are sensitive to delays, the real-time processing of brain data and audio had to work in less than half of a second.
Toward improved hearing aids
“It’s a very big milestone. At the same time, if you think, ‘How can this become a device?’ there are many challenges, actually,” says study co-author Nima Mesgarani, an engineer at Columbia University in New York City.
For example, today’s brain recording technology on the scalp might not provide good enough data for real-time applications. It’s possible that some people with hearing loss would benefit enough from enhanced hearing to justify invasive or minimally invasive procedures that allow for higher-quality brain recording, but this would limit wider use. Also, higher computational requirements might not physically fit into conventional hearing aid form factors.
Experts said they would like to see follow-up research with more participants, including those with hearing loss, and more work exploring non-invasive EEG. More complex listening scenarios could be closer tests of real-life performance, when there might be more than two speakers who move about or speak intermittently—all against a noisy background.
Mesgarani is also interested in how brain recordings could be used with AI to help hearing and communication more broadly.
Early in his career, Mesgarani worked with ferrets as an animal model of hearing. “A good thing about working with humans is they can describe their experience,” he says. In testimonials published alongside the paper, study participants described the experience of hearing sounds modulated in response to their own brain activity.
“It seems almost science fiction,” one participant said.
From Your Site Articles
Related Articles Around the Web