Researchers at Columbia University have devised a clever way of converting thoughts to speech using a handy combination of speech synthesizers and Artificial Intelligence (AI).
This technology effectively combines and "listens" to the brain and discovers patterns of action that can then "translate" into words. Right now his abilities are quite basic, but as scientists know in scientific reports, the possibilities are huge. Not only can it offer us a way to communicate with computers, we can once again offer potentially life-changing solutions for people with limiting speech – such as those with a stroke or living with amyotrophic lateral sclerosis (ALS) as well as the late great Stephen Hawking.
The process depends on hanging patterns of activity that light up our brains when we talk or even think about speaking. Similarly, when we listen to someone else, we speak (or imagine it), there are various other patterns that exist in the brain.
But while previous attempts to "read" brain activity relied on computer models with spectrum analysis and were unsuccessful, this new technique uses Apple's technology for Siri and Amazon for Alex – the AI-supporting vocoder.
Vocoders are a kind of computer algorithm that is able to synthesize speech but must first be trained on the recordings of people who speak. For this particular study led by Nima Mesgarani, chief investigator Mortimer B. Zuckerman, University of Mortimer B. Morgan B. Zuckerman's Mind Brain Behavior Institute, the Vocoder was trained with five patients with epilepsy who were selected because they had been brain- . While patients with epilepsy were asked to listen to people's speech, the researchers watched their brain activity.
Then the experiment actually started. To test whether the algorithm was now able to "read" participants' brain waves, the researchers played records of the same loudspeakers that reflected the sequence of digits between 0 and 9. The brain signals of patients with epilepsy were recorded and passed through the Vokoder. The results of the vocoder were then checked and "cleaned" with AI (neural network). Finally, the robotic-sounding voice repeated the serial numbers.