Machine Translates Brainwaves into Sentences

[Ophiolite]

The title is self-explanatory. You can find the article here. The BBC report notes that "Scientists have taken a step forward in their ability to decode what a person is saying just by looking at their brainwaves when they speak. They trained algorithms to transfer the brain patterns into sentences in real-time and with word error rates as low as 3%."

Please note the caveats in the article, but an interesting beginning.

It did strike me that this means the machine is capable of doing what some members at CF are incapable of much of the time.

 

[solid_core]

The title is self-explanatory. You can find the article here. The BBC report notes that "Scientists have taken a step forward in their ability to decode what a person is saying just by looking at their brainwaves when they speak. They trained algorithms to transfer the brain patterns into sentences in real-time and with word error rates as low as 3%."

Please note the caveats in the article, but an interesting beginning.

It did strike me that this means the machine is capable of doing what some members at CF are incapable of much of the time.

I think its not too transferable between people. All brains are different and our neurons are still changing their connections. What the machine identifies as unique patterns for 30 sentences (as the article says it work for 30-50 sentences) in one person, in the brain of other people can look quite differently.

If we both imagine a green house, similar areas of our brains will be activated, but the electrical activity of specific neurons will differ.

 

[Dave-W]

Interesting first steps in this tech. I can see all kinds of application for those who have communication difficulties.

It did strike me that this means the machine is capable of doing what some members at CF are incapable of much of the time.

LOL!!!

 

[Ophiolite]

Interesting first steps in this tech. I can see all kinds of application for those who have communication difficulties.

That was my immediate thought. Obviously the technology is years, perhaps decades away from such applications, but the possibility is there. I had those as disadvantaged as Stephen Hawking in mind.

 

[loveofourlord]

I think its not too transferable between people. All brains are different and our neurons are still changing their connections. What the machine identifies as unique patterns for 30 sentences (as the article says it work for 30-50 sentences) in one person, in the brain of other people can look quite differently.

If we both imagine a green house, similar areas of our brains will be activated, but the electrical activity of specific neurons will differ.

yeah, but experiments I've seen with reading people's memories, and controlling things with the brain, while each person is different, with time you can teach the computer to learn the patterns of the person.

 

[dms1972]

Out of interest how do they measure brainwaves? What are the particular brainwaves they detect? Its an interesting development, but I don't know what its applications might be, and we should be rather cautious about that.

 

[loveofourlord]

Out of interest how do they measure brainwaves? What are the particular brainwaves they detect? Its an interesting development, but I don't know what its applications might be, and we should be rather cautious about that.

I think the main usage for these things is those that can't communicate, I know thats where most of the research is, like those that have locked in syndrome, or hawking or such. It allows them to communicate more readily, the controlling things with the brain have gotten it so while simple they can answer yes or no by moving a cursor on the screen up or down and such.

 

[Dave-W]

That was my immediate thought. Obviously the technology is years, perhaps decades away from such applications, but the possibility is there. I had those as disadvantaged as Stephen Hawking in mind.

I thought of him as well. And I also thought of those who are even less able to communicate, like those in a semi comatose state.

 

[Ophiolite]

Out of interest how do they measure brainwaves? What are the particular brainwaves they detect? Its an interesting development, but I don't know what its applications might be, and we should be rather cautious about that.

The researchers, as noted in their paper in Nature Neuroscience, "recorded neural activity with high-density (4-mm pitch) ECoG grids from the peri-Sylvian cortices of participants, who were undergoing clinical monitoring for seizures, while they read sentences aloud. At each electrode, the envelope of the high frequency component (70–150 Hz, that is ‘high-γ’) of the ECoG signal—that is, the amplitude of the analytic signal in this range—was extracted at about 200 Hz (*reference), and the resulting sequences— each corresponding to a single sentence—passed as input data to an ‘encoder–decoder’-style artificial neural network."

*reference: Dichter, B. K., Breshears, J. D., Leonard, M. K. & Chang, E. F. The control of vocal pitch in human laryngeal motor cortex. Cell 174, 21–31.e9 (2018)

In simple terms, the subjects were a group of patients undergoing tests to determine the cause of their seizures. Those tests involved the implantation of numerous electrodes directly on the surface of the brain. The process ECoG, Electrocorticography, is discussed in this Wikipedia article.

Your caution is entirely valid, but see this for what it is: one of the first steps. It took thirteen years of global research to determine the human genome and cost around $100,000,000. Less than twenty years later the same result can be achieved for $1,000.

 

[Ophiolite]

I thought of him as well. And I also thought of those who are even less able to communicate, like those in a semi comatose state.

A word of caution on this. Keep in mind (accidental pun) that what was being picked up was not the thoughts accompanied the internal speaking of the sentences, but the neuron firing that was activating the speech system (larynx, tongue, lips, etc.)when actually speaking. However it is adaptation and development of that system over time, coupled with harvesting of techniques from other disciplines, that may lead to remarkable achievements.

Note: I should have said in an earlier posts that such advanced technology "is decades, perhaps half a century away" not "is years, perhaps decades away". Sometimes my optimism overrides my good sense.

 

[Dave-W]

A word of caution on this. Keep in mind (accidental pun) that what was being picked up was not the thoughts accompanied the internal speaking of the sentences, but the neuron firing that was activating the speech system (larynx, tongue, lips, etc.)when actually speaking. However it is adaptation and development of that system over time, coupled with harvesting of techniques from other disciplines, that may lead to remarkable achievements.

Note: I should have said in an earlier posts that such advanced technology "is decades, perhaps half a century away" not "is years, perhaps decades away". Sometimes my optimism overrides my good sense.

I understand that. But just as those who learn to "speak" by looking at letters on a computer screen, they can learn to fire neurons as well.

And yes, it is a long time off.

 

[Ophiolite]

I understand that. But just as those who learn to "speak" by looking at letters on a computer screen, they can learn to fire neurons as well.

And yes, it is a long time off.

I agree. I'm just very annoyed at myself for suggesting such an outcome could be achieved in years. I try to choose all my words and structure all my sentences carefully, to deliver a clear and concise message. Getting something wrong is sloppy writing and ultimately discourteous to anyone reading my posts. (I suppose prattling on about my own gerontocratic angst is also discourteous. )

 

[loveofourlord]

A word of caution on this. Keep in mind (accidental pun) that what was being picked up was not the thoughts accompanied the internal speaking of the sentences, but the neuron firing that was activating the speech system (larynx, tongue, lips, etc.)when actually speaking. However it is adaptation and development of that system over time, coupled with harvesting of techniques from other disciplines, that may lead to remarkable achievements.

Note: I should have said in an earlier posts that such advanced technology "is decades, perhaps half a century away" not "is years, perhaps decades away". Sometimes my optimism overrides my good sense.

well I think one study found that by making the motions or thinking the motions you can get simular signals. Cant' remember where it was ages ago.

 

[Ophiolite]

well I think one study found that by making the motions or thinking the motions you can get simular signals. Cant' remember where it was ages ago.

There have been a great many such studies/developments and several will be in progress today. Here are a couple of examples:

Brain-computer interface based on detection of movement intention as a means of brain wave modulation enhancement The title introduces it nicely

Robot Motion Control via an EEG-Based Brain–Computer Interface by Using Neural Networks and Alpha Brainwaves A typical paper on the subject

Analysis and Interpretation of Brainwave Signals This looks like a nicely implemented student effort.

The significant thing about the experiment linked to in the OP is the analysis software that interpreted signals. This showed great promise through its ability to "learn".

 

[loveofourlord]

There have been a great many such studies/developments and several will be in progress today. Here are a couple of examples:

Brain-computer interface based on detection of movement intention as a means of brain wave modulation enhancement The title introduces it nicely

Robot Motion Control via an EEG-Based Brain–Computer Interface by Using Neural Networks and Alpha Brainwaves A typical paper on the subject

Analysis and Interpretation of Brainwave Signals This looks like a nicely implemented student effort.

The significant thing about the experiment linked to in the OP is the analysis software that interpreted signals. This showed great promise through its ability to "learn".

yeah I remember a article years ago about reading memories, where they had someone look at a painting read his mind, and then had him remember it and how the computer tried to recreate it, here is one article on it.

This ‘mind-reading’ algorithm can decode the pictures in your head

it's far from perfect...and some of those are creepy :> but it's a interesting thing they are working on.

 

[Quid est Veritas?]

@Ophiolite

I applaud your measured and sensible reading here. When I saw the title, I was ready to come in here and shootdown misunderstandings and people running too far off of the evidence, but this is unnecessary. This has far too much potential to generate misunderstanding and silliness, especially from those who just half-read the article.

Yes, it isn't measuring 'brainwaves' like EEGs as we conventionally do, but the electrodes are picking up the depolarisation membrane potentials in implanted electrodes. It is measuring movements of speech in the motor pathways, not 'thinking' the words themselves independant thereof, and in this way the ability to learn should be understood. There are only a set amount of sounds possible in human anatomy, and although there is some variance, if we can map how each is formed, we should be able to recreate vocalisations. This is used a limited vocabulary of about 250 words, not without error, but is a surprising result. Preliminary, but encouraging. I think half a century is still optimistic, but who knows?

 

[dms1972]

The researchers, as noted in their paper in Nature Neuroscience, "recorded neural activity with high-density (4-mm pitch) ECoG grids from the peri-Sylvian cortices of participants, who were undergoing clinical monitoring for seizures, while they read sentences aloud. At each electrode, the envelope of the high frequency component (70–150 Hz, that is ‘high-γ’) of the ECoG signal—that is, the amplitude of the analytic signal in this range—was extracted at about 200 Hz (*reference), and the resulting sequences— each corresponding to a single sentence—passed as input data to an ‘encoder–decoder’-style artificial neural network."

*reference: Dichter, B. K., Breshears, J. D., Leonard, M. K. & Chang, E. F. The control of vocal pitch in human laryngeal motor cortex. Cell 174, 21–31.e9 (2018)

In simple terms, the subjects were a group of patients undergoing tests to determine the cause of their seizures. Those tests involved the implantation of numerous electrodes directly on the surface of the brain. The process ECoG, Electrocorticography, is discussed in this Wikipedia article.

Your caution is entirely valid, but see this for what it is: one of the first steps. It took thirteen years of global research to determine the human genome and cost around $100,000,000. Less than twenty years later the same result can be achieved for $1,000.

Can you say what it is one of the first steps toward?

 

[Ophiolite]

Can you say what it is one of the first steps toward?

Towards efficient, effective decoding of brainwaves so that those with disabilities, such as paralysis, can control equipment and communicate.

 

[dms1972]

Towards efficient, effective decoding of brainwaves so that those with disabilities, such as paralysis, can control equipment and communicate.

Can God still heal such? Please don't get angry at my asking. I just wonder do we place too much hope in technology and the medical profession sometimes.

Ambroise Pare said "I treat them - but God heals them."

George Bernard Shaw wrote: "We have not lost faith, but we have transfered it from God to the medical profession"

And don't take that the wrong way, we need doctors and nurses, after all St. Luke in the Bible was a Physician, so I don't think God wants christians to leave the profession, far from it. But imagine what would have happened one hundred years ago during an epidemic - teams of nurses and doctors in some hospitals would have been prayed over and been blessed before going about their work!

I know God can use illness and bring good out of it. I struggle with depression and my nerves, I suppose in a few years they will have a device that will balance my brainwaves and relieve my anxiety - I probably will not be able to afford it though, but maybe that is just as well. Its difficult but maybe I should learn to appropriate scriptures by faith - in fact I do, but if I can just run a current across my brain - would I soon forget God?

 

[Ophiolite]

Can God still heal such? Please don't get angry at my asking. I just wonder do we place too much hope in technology and the medical profession sometimes.

It's a perfectly reasonable question. I don't think those who are in medicine in one role or another place to much hope in it. They are painfully aware of their limitations. Most of us have to face only a handful of deaths, or see a few loved ones disabled, or chronically ill. Many doctors, nurses and carers face this on a regular basis. They know the limitations and that is why they and those seeking technical advances wish to find other solutions, to push those limitations back.

Ambroise Pare said "I treat them - but God heals them."

George Bernard Shaw wrote: "We have not lost faith, but we have transfered it from God to the medical profession"

We could trade pointed and poignant quotations all day, but I'm not sure it would achieve much.

And don't take that the wrong way, we need doctors and nurses, after all Luke in the Bible was a Physician, so I don't think God wants christians to leave the profession, far from it. But imagine what would have happened one hundred years ago during a pandemic - teams of nurses and doctors in some hospitals would have been prayed over and been blessed before going about their work!

I am certain that this is happening in many hospitals today and many where it is not. I suspect there is no difference in the outcome for the patients in the two cases, but if it brings strength to the medical staff and comfort to the patients, it is a good thing.

I know God can use illness and bring good out of it. I struggle with depression and my nerves, I suppose in a few years they will have a device that will balance my brainwaves and relieve my anxiety - I probably will not be able to afford it though, but maybe that is as its difficult but maybe I should learn to appropriate scriptures by faith - in fact I do, but if I can just run a current across my brain - would I soon forget God?

I am sorry to hear of your troubles. This pandemic must be especially difficult for you. I hope you have good support around you.
In theory a device such as you describe may well become practical. The concern over potential side-effects is also valid. That is why it is essential that any experimentation undergo rigorous ethical oversight and public implementation be subjected to broad scrutiny.