Electrodes record continuous speech from brain waves

16th June 2015

Nat Bowers

0 0

Researchers at KIT and Wadsworth Center, USA, have been exploring speech produced in the human cerebral cortex and recording brain waves associated with speech processes with electrodes located on the surface of the cortex. For the first time, they have shown that it is possible to reconstruct basic units, words and complete sentences of continuous speech from these brain waves and to generate the corresponding text.

Tanja Schultz, who conducted the present study with her team at the Cognitive Systems Lab, KIT, commented: ”It has long been speculated whether humans may communicate with machines via brain activity alone. As a major step in this direction, our recent results indicate that both single units in terms of speech sounds as well as continuously spoken sentences can be recognised from brain activity.“

These results were obtained by an interdisciplinary collaboration of researchers of informatics, neuroscience and medicine. In Karlsruhe, the methods for signal processing and automatic speech recognition have been developed and applied.

”In addition to the decoding of speech from brain activity, our models allow for a detailed analysis of the brain areas involved in speech processes and their interaction,” said Christian Herff and Dominic Heger, who developed the 'Brain-to-Text' system within their doctoral studies.

The present work is the first that decodes continuously spoken speech and transforms it into a textual representation. For this purpose, cortical information is combined with linguistic knowledge and machine learning algorithms to extract the most likely word sequence. Currently, 'Brain-to-Text' is based on audible speech. However, the results are an important first step for recognising speech from thought alone.

The brain activity was recorded in the USA from seven epileptic patients, who participated voluntarily in the study during their clinical treatments. An electrode array was placed on the surface of the cerebral cortex (electrocorticography or ECoG) for their neurological treatment. While patients read aloud sample texts, the ECoG signals were recorded with high resolution in time and space. Later on, the researchers in Karlsruhe analysed the data to develop 'Brain-to-Text'. In addition to basic science and a better understanding of the highly complex speech processes in the brain, 'Brain-to-Text' might be a building block to develop a means of speech communication for locked-in patients in the future.

A video showing 'Brain-to-Text' in operation is available to view on the KIT website.