This AI can turn brain activity to speech

Science

26 Apr 2019

Now, AI can transform brain activity into realistic speech

AI has wide applications but a group of scientists is using the technology for a noble cause - to give voice to those who can't speak.

They have developed a neural network-backed system that can track brain activity and use it to generate realistic synthesized speech to help people convey their thoughts.

Here's all about the system and its working.

Research

Tech to generate entire sentences from brain activity

Tech to generate entire sentences from brain activity

The speech-producing system, developed by neurologists from University of California San Francisco, revolves around tracking the activity of the brain with a set of stamp-like electrodes and then feeding this information into a machine learning system.

Once the information is fed, the system translates the activity to synthetic speech, producing entire spoken sentences that can be understood by any human listener.

Process

However, this is not translating thoughts into speech

While producing sounds from brain activity sounds a lot like converting thoughts into vocal speech, it's worth noting that this is not the case.

Instead of translating brain signals directly, the researchers use a virtual vocal tract mimicking the exact motions that the brain - region involved in language production - triggers to produce speech.

Love Tech news?

Stay updated with the latest happenings.

Yes, notify Me

Normal speech reverse-engineered to create virtual vocal tract

The researchers asked five patients suffering from epileptic seizures to read sentences from kid's stories. Then, their linguistic experts analyzed those recordings to decode the exact muscular movements involved in the generation of spoken sentences; these movements were mapped to create the virtual vocal tract.

Demonstration

Finally, virtual vocal tract decoded brain activity to synthesize speech

Finally, virtual vocal tract decoded brain activity to synthesize speech

In their study, the researchers demonstrated that their virtual vocal tract, which included two neural network algorithms, produced natural-sounding speech.

One of the algorithms, the decoder, analyzed signals from the brain's speech center to define which muscular movements the brain is signaling.

Meanwhile, the second used these movements to define a synthetic approximation in the form of voice.

Benefit

This could give voice to people suffering from speech loss

Though the system is still novel, the researchers believe it could be improved/perfected to give voice to those who have lost their ability to speak due to injury, paralysis, or neurodegenerative diseases.

If that happens, this would be a major upgrade over current assistive technologies that not just a take a lot of time, synthesizing 10 odd words per minute, but are also error-prone.

We could have a clinically-viable device based on this tech

"We can generate entire spoken sentences based on an individual's brain activity," team member Edward Chang stated, adding that their latest work shows they should be able to employ existing technologies to "build a device that is clinically viable in patients with speech loss."

Share this timeline

Share this timeline

Ask NewsBytes
User Image

Most asked questions

Did the system generate all words clearly?

When this system would come into clinical use?

Are there other systems like this?

Is this technology invasive?

More questions

Did the system generate all words clearly?

Asked 2019-04-26 13:13:08 by Dhruv Verma

Answered by NewsBytes

According to the researchers, some words generated by their virtual speech synthesizer were clearer than the others. But, the system could be perfected in the future.

When this system would come into clinical use?

Asked 2019-04-26 13:13:08 by Chirag Dutta

Answered by NewsBytes

As of now, the system is very new and there's no timeline for when it could come into clinical use.

Are there other systems like this?

Asked 2019-04-26 13:13:08 by Navya Dutta

Answered by NewsBytes

There are many similar projects but most of them are related to translating brain signals directly.

Is this technology invasive?

Asked 2019-04-26 13:13:08 by Shaurya Tata

Answered by NewsBytes

No, the technology is not invasive; it just involves placing electrode patches on the skull of a patient.

Next Timeline