Well-known physicist Stephen Hawking has suffered years of amyotrophic lateral sclerosis (ALS), paralyzing motor neurons. Not only paralyzed, but also lost the ability to speak. He communicated with the outside world using a computer and a speech synthesizer.
Using the switch attached to his glasses, he could choose words on the computer, which he then read the synthesizer. He was able to disarm about a dozen words per minute.
Hope for the Disabled
If Hawking and similarly affected patients had a new speech synthesis system developed by Neural Acoustic Processing Lab scientists at Columbia University in New York, they could communicate much better. (First, however, the number of obstacles will have to be overcome.)
SeeNew neuromorphic processors are the path to real artificial intelligence
Using brain scanning technology, artificial intelligence, and speech synthesizer, scientists have been able to convert brainwaves into comprehensible speech. Such a system will be able to capture people's ideas and produce speech directly from them, eliminating the need for computer control.
Some individuals, whether having ALS or recovering from a stroke, may not have the motor skills needed to control the computer, even by mimicking their faces. New research can open the way for a certain form of "telepathy" for them too.
To create such speech neuroprotection, Neurobiologist Nima Mesgarani and his team created a solution in which they combined the latest findings from in-depth learning with speech synthesis technologies.
SeeIBM teaches artificial intelligence to behave like a brain
Their brain-computer interface, although only a prototype, captured brain waves directly from the brain cortex of the hearing center, which were decoded by the vocoder – a synthesizer of speech with artificial intelligence to create intelligible speech.
Although the speech was very mechanical, almost three quarters of the listener was able to recognize its content, which is a great success.
You can listen to the speech synthesizer here. The test persons attempted to generate the names from zero to nine. Researchers tested different techniques, but the best results were the combination of deep neural networks with the vocoder.
An interesting project has a long way to go
Mesgarani's neuroprotective device does not translate into thoughts in our heads, also called the notion of speech. It is not a real telepathy or a reading of thoughts.
Instead, the system captures the individual cognitive responses of an individual while listening to call records. The neural network with in-depth learning then decodes or translates these patterns, allowing the system to reconstruct speech.
Using a voice synthesizer, instead of a system that can compare and recite pre-recorded words, was important to scientists.
"Because the goal of the research is to restore speech communication to those who have lost the ability to speak, we have tried to learn direct brain mapping and its transfer to speech itself," Mesgarani told Gizmodo.
"It is also possible to decode phonemes (separate sound units) or words, but speech has much more information than just content, such as reproduction and its associated characteristics and style of voice, intonation, emotional tone, etc. That is why research aims to restore the sound itself" , added Mesgarani.
Higher Voice Recognition League
Although voice recognition and voice control systems are well developed and well known, speech synthesis, which we just imagine, represents a substantially higher level.
The brain wavelength itself is a problem, because the regions that generate the bioprocesses and the brain waves during talking and listening are partly overlapping.
SeeArtificial intelligence learns to read thoughts
To make neural networks more than just reproduce words representing numbers ranging from zero to nine, they need to be trained on a large number of brain signals from each participant. The system is specific for individual patients because we all produce different ideas while listening to speech.
According to Professor Andrew Jackson of Newcastle University, it will be interesting to see how decoders trained for one person in the future will be used by others. Time will show whether they will achieve similar reliability and usability as speech recognition systems – such as Google Asistent and Amazon Alexa, used today.