Instant Genius - Brain-controlled machines, with Tom Carlson

Episode Date: December 16, 2022

Controlling machines with brain waves sounds like something ripped from a science-fiction book, and yet this is something happening today. We spoke to Tom Carlson about the rapidly expanding world of brain-machine interfaces. Hosted on Acast. See acast.com/privacy for more information. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Ambition comes in all shapes and sizes. At First Citizens Bank, we roll with your goals because we're built for what you're building. Fit for your ambition for Citizens Bank. There's a moment when you start to wonder, what's the right next step? Not about changing who they are, just finding the right kind of support. At Kingsley Manor, life stays expressive, connected, and full of character, shaped by people who have lived interesting lives in our finished yet. So it doesn't feel like a change. It feels like a continuation. Explore your options at canesley manor.org, a non-profit month-to-month senior community within the Front Porch family.

This podcast is sponsored by name, audio and focal. Streaming has made music more accessible than ever, but true listening is about more than ease. It's about quality. British audio experts name Audio, alongside French acoustic specialist focal, combine handcrafted tradition with cutting-edge innovation and high-end materials, delivering digital precision with analog warmth. So you can experience exceptional sound at home. Music just as the artist intended. Visit name audio.com to learn more. From BBC Science Focus magazine. This is Instant Genius, a bite-sized masterclass in podcast form. I'm Alex Hughes, staff writer at Science Focus magazine. This week, I'm joined by Tom Carlson.

He's a professor of assistive robotics at the University College London. He talks through the rapidly expanding world of brain machine interfaces, a technology that can utilise signals from the brain to power wheelchairs, robot arms, and a host of other assistive technologies. He also addresses the possibility of this technology. breaking into the consumer world too. So I think a good place to start is just by asking simply what is a brain machine interface. Okay, well, a brain machine interface is really an alternative way of interacting with the world.

So most of us interact with the world by speaking, by touching things, by moving around in the world. But not everybody can do that. So a brain computer interface exploits your brain activity or your thoughts directly in order to control things in the real world. And without oversimplifying it, how does a brain machine interface actually work? Okay, so there are few components to a brain machine interface. Firstly, we need to be able to record the brain activity to understand what's going on inside the brain. and then we need to decipher some sort of intention, and then we can map that intention to some sort of action

to control something in the world. So maybe let's start with exploring how we can record that brain activity in the first place. You might have heard of things like CT scans or CAT scans that can provide an image of the brain, but this is structural. It doesn't tell us anything about that thought process that's going on in the brain.

So instead, we need to use some sort of method that's functional, that can understand what the functions of the brain are and what thoughts you're having at that time. So you might have heard of things like fMRI, functional magnetic resonance imaging. But these are huge devices in hospitals. They're very, very expensive, and you can't just carry one of those around with you. So instead, we've been looking at devices that are more portable, that are cheaper, that can actually understand what's going on in your brain in real time. And so something we use in my lab is called EEG or electroencephalography.

And what this does is it's essentially looking at the electrical activity of the brain. So inside your brain, you've got billions of neurons, and they're all communicating with each other. And every time one neuron talks to another neuron, there's a little what we call an action potential or a spike of electricity. And all of this electricity adds up in your brain. It's very, very small. But if we put electrodes on the surface of your scalp using something that looks like a fancy swimming cap

with a lot of wires coming out of it, then we can actually pick up this electrical activity to understand what's going on roughly where in your brain. Once we've got that, we can look at classifying different types of thoughts. So we can ask the user to think about specific things, maybe thinking about moving a hand or a leg. And we can use machine learning techniques to build up something we call a classifier. So that when we're looking at this brain activity, we can say, does this pattern of electrical activity look more

like moving a hand or moving a leg. And then once we've got that, we can then take that as an output and we can use it for telling, for example, a wheelchair to turn right or left or stop, or maybe for controlling a prosthetic limb, for example. And you talk about the way that the brain can send a signal for, let's say, moving a leg. Would this look the same in everybody's head or is this different per person? So every person is different, but we have the same kind of basic building blocks within our brain. So the structure is similar, but the exact shape and size of your brain is going to be different. So we can use some of this neurophysiological information in order to look at specific areas of the brain and also specific frequency bands for the electrical activity. And then we can

take an idea of that as a kind of starting point, but then we need to customize this interface to the individual. So there's a certain amount of learning, both from the machine side, so the machine learning, but also from the human side. It's a kind of mutual learning process between the machine and the human in order to get a kind of optimal communication between the two of you. When we talk about this kind of technology,

how far along is it? Is this still very much theoretical or are there some real-life examples of this being used now? So there are lots of different applications of this technology and it ranges massively. So you'll find some commercial brain computer interfaces already available to help with, for example, rehabilitation, particularly after stroke, for example. and you'll find some very low-grade brain computer interfaces targeting the gaming industry, for example. But if we want to be using this reliably as an assistive technology

for people who don't have other ways of interacting with the world, so for example, people with a high-level tetraplegia who can't move their limbs might benefit from using a brain computer interface based on motor imagery, so this imagination of motor type. And that sort of brain computer interface, we've demonstrated in the lab, we've demonstrated in hospitals, but it's not really at a stage yet where it's ready for day-to-day use. You can't just pop into the supermarket and buy this brain computer interface and use it in your daily activity yet. And how would you go about installing these or is this something that needs to be installed or are we still in the stages of caps on your head with sensors? Okay, well, this is a really great question. So the brain-computer interface world is divided into invasive and non-invasive interfaces.

So what I was talking about earlier were non-invasive methods where we can wear something like a swimming cap on your head that has lots of wires coming out of it. And there are also wireless versions of these that are looking a little bit more sci-fi and a little bit more interesting to wear. But then there's also the invasive world as well, which when you talk about installing would need surgically implanting into the brain. And really, depending on the application, you can imagine maybe quite a lot of people wouldn't want to be having an operation where people are embedding electrodes deep inside into the brain. But then for other people that have a chronic pathology, perhaps that's an option for them. So there are different levels of invasiveness as well. There are examples of some sort of one can imagine a sort of brain computer interface, like a deep brain stimulator that's being used clinically since the early 2000s.

And this is particularly used for some severe cases of Parkinson's disease, where electrodes are embedded deep inside the brain, and it's used to regulate some of the brain. to reduce the tremor, for example, that you get associated with Parkinson's disease. And obviously, that needs a huge operation, lots of planning around it, and it's a quite invasive procedure, but that's a good choice for some people. We could imagine similar sort of techniques, maybe not quite so invasive, where we put electrodes on the surface of the brain underneath the scalp. So you could have that there for a long time. There are teams in the US and in France that have

done this with patients recently and showing that they can get some very good recordings. But obviously, you need to go through lots of ethics procedures for this to make sure that it's going to be worthwhile for the patient. And this sort of level is still very much in the research stages at the moment. And when we're talking about these two different methods, what is it that is the most optimal, or is this just more a case of what's best for each individual or in a different circumstance? Yeah, well, everybody's got different goals, everybody's got different needs.

So we need to bear that in mind. I think one of the challenges that we have is looking at the resolution of the brain activity that we can actually capture. So I'm not into football, but given that it's the World Cup at the moment, maybe we can use a football analogy. So if you imagine what I was talking to about earlier

with this EEG, where we're putting electrodes on the skull, and we're trying to detect tiny, tiny little impulses going on between neurons deep inside the brain. That's kind of a bit like watching a football match from a helicopter hovering above a stadium. So without a super zoom lens. Okay, so here, the ball is really small. You can't see it. You can't identify individual players in that football stadium, but you can see roughly where they are.

You can see the motion of the players. So you can kind of infer where that ball is roughly in the stadium. You might be able to guess which team's got control of the ball at particular time. And you might see the crowd go wild at some point and infer that perhaps there's been a goal. But you can't really see that detail. And so that's rather like EEG. It's looking from the outside and trying to understand what's going on deep inside the brain.

obviously the advantage of EEG is that you don't have to have an operation. You can take the cap off when you finish with it. It's not too uncomfortable. And that might be sufficient for some patients, for some level of control. But for more severe cases, where we need to have much more detailed information about the brain, then perhaps we need to go deeper inside. So you want to actually be in that football stadium to understand what's going on. So you can see the players.

You can really see where the ball is. And that's more like E-Cog, electrocorticogram, which sits just on the surface of the brain, or even invasive recordings where we put very thin needle electrodes into the brain, and you can get up really close to those groups of neurons so you can see what's going on. So obviously there you get a richer amount of information about what's happening, but at a cost in terms of you have to have a surgery. And that's not without risks of infection, damage to the brain. it's very expensive. It's going to be in there chronically or for a long time.

And we've spoken mainly about this as a assistive technology. Do you, do you think there's any room for the technology in the A-board? I mean, Elon Musk's Neurilink is picking up a lot of steam and people bring it up as an option a lot. But is there any need for them without it being assistive? That's an interesting question. I like to draw parallels with other assistive technologies here. So you may or may not have heard of environmental control units. These were very bulky, expensive, bespoke, assistive technologies for people with severe disabilities to help them control their environment, control their homes, maybe open and closed curtains, turn the heating on and off and things like that. And obviously now, what I'm talking about is smart homes. Lots of us have

various devices in our homes that we can talk to. We've got apps on our phones where we can control the lighting, the heating, etc. Whether or not we need that is another question, but there are definite advantages to that. So for example, I can now only heat my home when I'm leaving work, coming back, so it's nice and warm when I get there, but not wasting energy during the day, even if my plans are changing. And so kind of drawing a parallel with brain computer interfaces, the technology is incredibly expensive at the moment, but if it becomes a more widespread technology that is used mainstream,

then that would drive the cost down and benefit people that need to use it as an assistive technology, and I'm sure new uses will evolve as well. So I think at the moment we see the main use is kind of like an alternative mode of interaction in gaming, where you can add it as a hybrid interface. So you're not using the brain computer interface as the only method of interaction,

but perhaps you use it to supplement what you're doing with a joystick or keyboard and gives you that extra level of control. But at the moment, it's maybe not as reliable, it's not as fast as other communication modalities. So typing or moving your hands is going to be more reliable and quicker than using a BCI at the moment. And with these interfaces that we've spoken about

that are more permanent, the more invasive options, is this something that needs to be upgraded as time goes on or is it something that is maybe implanted and then it's just left and the technology can handle itself over the years? So as I said before, we've got different levels of invasiveness. The problem is when you stick something inside a human, generally what you create is scarring tissue. And that scar tissue actually changes the electrical character. of the tissue. So you will get a degradation of signals over time, which means that if you're sticking electrodes into the brain and you're wanting to record activity from populations of neurons

around that electrode, over time it will gradually become less than yes, useful. You will lose that kind of high fidelity that we're talking about before. Alternatively, if you're doing something that's sort of minimally invasive where we're putting electrodes underneath the skull, but not penetrating into the brain, so just sitting on top of the dura of the brain, then this doesn't create that same level of scarring. So those electrodes will last longer in the same place and be able to give you that same level of fidelity over a long period of time. So that's on the hardware side of things. And then I guess like everything else, on the software side of things. We can do a lot of upgrading, we can deliver power and everything remotely. So from that

point of view, that wouldn't necessarily need to change. There have been a few studies looking at what we call chronic or long-term implants of electrodes. But typically for these kind of ones that penetrate the brain, we're looking at a matter of months that they would stay in place and still be viable, usually. Whereas ECOG could potentially be in the order of years. But this is still really a very active area of research. And with both the more invasive and the non-invasive methods, what do patients and the participants have tried out? What do they report the sensation to be like of having a brain machine interface?

Okay, well, you can't actually feel what's inside your brain. So I've not tried an invasive interface myself. I've done a lot of non-invasive interfaces and serve my students in my lab. And also we've worked with many patients as well. I think the, so from that point of view, you know, there's a bit of discomfort of wearing the cap or whatever. But the main thing is about getting into the right state of your brain. You can imagine there's a lot of different brain activity going on.

And you have to become quite relaxed, almost like in a meditative state, so that you've got very calm brain signals. And we can really distinguish the ones that we're looking for. from those. I think the other thing that's quite interesting is when we're working with patients who have motor impairments, so they maybe can't move their limbs, we can ask them just to very naturally try to move their limb. Their limb's not going to move, but we can pick up that brain activity as we would normally see it if their limb weren't moving. But when we're working with healthy participants, so here we might be looking at other applications,

areas, if you're asking somebody to think about moving a limb but without actually moving it, that's quite a weird sensation and that takes quite a lot of practice to actually achieve that. When you're using these kind of technologies, is this actually safe to use? I mean, whether or not it's the non-invasive or the invasive methods, because you've spoken about scar tissue and the electrodes and the brain, but just generally using this technology, Yeah, so I guess you've got different thoughts about safety. So obviously there's the whole surgical side. And this has been well established.

We've had implants for decades. We've had cochlear implants. We've got deep brain stimulation that I talked about. And so this is FDA approved in the US. It's approved for use by the NHS as well, for example. But I think what's interesting in terms of safety is actually thinking about what you're doing with that. signal afterwards. So I talked about different application areas. We could use a brain computer interface for maybe sending a text message or sending an email or browsing the internet

or controlling a wheelchair or a prosthetic limb. And so depending on the application, you're going to want some safety guarantees that whatever the devices that you're trying to control, that it's actually going to be doing what you're intending it to do. So, What I've been working with a lot in my group is something we call shared control. And with shared control, it means that the device itself has some level of artificial intelligence or cognitive processing abilities. So it can sense the environment in which it's operating. And it can understand what is a safe action to make actually in that environment.

So, for example, if we're driving a wheelchair, we've developed a smart wheelchair that can understand the environment, it's got lots of sensors on it, a bit like these sort of self-driving vehicles, self-driving cars. But it's not self-driving because there's a human being that's controlling it, just that they're controlling it using a brain-computer interface. And when they're using a brain-computer interface, the sort of decoding that we're getting, as I said before, is maybe understanding if they're thinking about moving a right hand or a left hand or their legs.

But it's not telling me exactly the sort of finger movements or anything like that that they're thinking about. So when we map that to controlling a wheelchair, if we're saying, think about your right hand, make the wheelchair turn right. We're not telling the wheelchair exactly when to turn right or how many degrees to turn right, 10 degrees, 45 degrees or whatever.

But we rely upon a certain amount of intelligence within the wheelchair itself to understand what makes sense in the environment. So if we're going down a corridor, and there's an open door on the right, then the wheelchair can take some responsibility and actually go through that door. But if we're going down a corridor and there's a stairway on the right,

then the wheelchair is going to detect that as a hazard and not allow us to drive down those stairs because that would be dangerous, obviously. Most of the systems currently allow the brain to control machines, but is there a future where it maybe goes the other way and you can use machines to feedback pressure, touch or some sensation that the brain will understand. Yeah, sure. So for many, many years, we've already been doing that using peripheral nerve.

So looking at things like cochlear imparts, we stimulate the auditory nerve. And that's been happening since, I don't know, about the 1960s, perhaps. And more recently, there's been a lot of development saying, okay, if we can do this for hearing impairments, can we do this for visual impairments? So people have been looking at developing stimulators to simulate the retina, and then looking, okay, what if we get closer to the brain and not stimulate the retina, but actually stimulate the optic nerve itself? And then what if we don't just stimulate the optic nerve,

but what if we move in and actually stimulate the visual cortex? And earlier this year, there was a team in the US that's actually embedded in what they call an intracortical visual prosthesis into the visual cortex at the back of the brain. And that's got about 400 electrodes or so. And they're doing experiments at the moment to stimulate directly the visual cortex with this implanted prosthesis.

Now, what you're asking about is can we then expand this to other senses as well, like touch? So obviously a lot of my work has been involving the motor cortex and motor control. And directly behind the motor cortex, the motor cortex is the somatosensory cortex that deals with this touch sensation. So there was a really interesting paper that I read a couple of years back in nature by a team,

I think it was from Seattle. And they've already started some experiments where, again, working with ECOG, so again, that's the electrodes that we put underneath the skull, just on the surface of the brain. They've been using that ECOG electrode array, instead of trying to record from the area. They've been directly stimulating the somatosensory cortex. And there have been some initial, I think with only about four patients or something, but just some initial trials there. And patients were able to actually identify when that area of the cortex was being stimulated. And they sort of described it as not painful, but maybe like a sort of pins and needles feeling that they would get in their fingers. So,

this is happening in the research field at the moment, but it needs refining to turn into something that's kind of useful. I think with any kind of technology, especially ones that are, I guess you could say changing the future, there's always some kind of ethical concerns, especially also when you bring in artificial intelligence and humans, what do you think are the ethical concerns

that need to be addressed with this kind of technology? Yeah, of course, ethics is a huge consideration here. And I think it's something that all of the research teams around the world have to apply to their institutional review boards or their ethics, research ethics committee in order to get approval before even doing these experiments, that's alone thinking further down the line at different applications. And so ethics isn't something that one person can sort of say yes or no to. It's rather a collective. decision by experts in the field, but also the lay public as well are sitting on these ethics boards, to say, what do we think is acceptable? What do we think is an acceptable risk? And when we identify these risks, what sort of mitigations can we be put in so that we can minimize them? We've obviously talked about some of the risks already in terms of safety. but going forward, you know, the considerations about privacy as well. Your brain signals are unique to you, and at the moment, we can decode to a certain level,

you know, movement intention, for example. We can't at the moment decode those sort of thoughts that may be going on inside your brain, you know, sort of deep a cognition. But perhaps that is something. that might be possible in the future. And so we have to be sort of thinking ethically about what the risks are, how likely they are to occur

and what sort of mitigations we can put in place to safeguard against them. A lot of this innovative assistive technologies that have come up in the past have had a very high cost due to the technology that is involved with it. Would you say this is going to be the same issue of this kind of technology

if it reached a practical life stage where it's very expensive for the average person to get hold of it. Well, I think we have to again go back to who the user is, what the purpose is. So if we're talking about assistive technologies that allow people who are maybe in a locked in state or with a very high level of paralysis to engage in activities of daily living, improve their quality of life, to improve their independence. Maybe they can then even work, contribute to the economy. And so, yes, you might be talking about something that sounds a really high cost. We could be talking tens hundreds of thousands of pounds. But over their lifetime and what that enables, that actually might not be such a high cost. If you're thinking about

the average person, as I said before, that kind of link to other technologies that we've seen develop, like environmental control units that have developed into widespread smart home technology, just the sheer volume of people using that sort of technology has really driven down the cost. But at the same time, it does come in terms of different costs, like in terms of the reliability of the system. If you've got something as an assistive technology that you absolutely need in order to function, to carry out your activity of daily living, you need it to be incredibly reliable.

Whereas if you're using this technology to play a computer game, it doesn't really matter if it's not functioning at 100% accuracy, for example. So you'll see that the sort of grade of the technology, the certification costs will be different depending on the actual purpose. But already we're seeing some sort of mainstream BCI technology come out for the gaming industry, which the quality of electrodes is maybe not quite as high as the medical grade stuff. The processing power is not as as good as the medical grade stuff. And we might not be actually able to decode quite as much,

but we might be using some things like artefacts, so muscular activity to decode for, the gaming interfaces. But just by making that more wide stream, that does bring down the cost of technology across the board. And so I think generally it's a good thing. And looking ahead, what is it that you think is the, I guess, the most exciting application of this technology? What is it that you think is going to be the most important thing that happens in this field? Whilst I work a lot in assistive technology, I think rehabilitation is going to be the real strength of this technology. So that's, you know, technology that's used on a more temporary

basis to bring people back to the level that they were at before. We can exploit things like neuroplasticity for rehabilitation. Neuropasticity really comes from the word plastic. And I'm not talking about the material that's kind of threatening our environment here. But the notion of plastic. If we imagine something that's elastic, like a rubber band, when you pull that rubber land, you're applying a stimulus, you're stretching it, it's changing shape, it's changing form. But as soon as you remove that stimulus and let go, it pings back to where it was to start with. So elasticity is not great if we're thinking about learning something. But if we instead think about plasticity and plastic materials, think about something like modelling clay.

If you apply stimulus and you sort of mould that into a new shape with your fingers and you let go, it stays in that new shape. So in some sense, that material has sort of learnt that new shape. You can then do it again. You can change it into a different shape, a different form. And it will remain in that shape you've kind of crafted it into. and that property we call plasticity. That same property we can apply to the neurons in your brain and even your motor and sensory neurons in your body as well. Using a process called Hebbian learning,

that was coined by Donald Hebb, I think, in the late 1940s. And basically what this means is that if we have your net, network of neurons in your brain that are responsible for controlling different parts of the body, for example, or processing sensory information, they need to talk to each other. They need these connections to be made. And what Hebein learning says is what fires together, wires together. So basically, if we're applying a stimulus at one point and we're seeing a resultant movement at another point, if we repeat this many, many, many, many, many times, then gradually these networks of neurons strengthen the connections between each other.

And so what we can use brain computer interfaces to promote neuroplasticity. And we can do this by linking them up. So we've been talking about using them to understand what's going on inside the brain. And we can reward the brain when you're thinking the right thought. So you can imagine after a stroke, for example, and you're trying to retrain how to move a paralyzed arm, you can use that brain computer interface to understand when you're wanting to try to move that arm, but it's not moving,

and link that up with, say, a robotic exoskeleton or some electrical stimulation that might stimulate the muscles in your arm. And we can do that many, many times, but we can kind of have that coherence between when you're actually trying to make that movement yourself and having the success of the exoskeleton or some electrical stimulation system making your arm make that movement. And then in that sense, the brain computer interface is promoting that Hebbian learning and that neuroplasticity, which will improve rehabilitation.

And so in some sense, my dream for the brain computer interface is that it will be there as a temporary thing that will enable somebody to get back to where they were. before the injury, for example, and then they won't need that brain computer interface beyond that point. Thank you for listening to this episode of Instant Genius. That was Tom Carlson, examining the growing industry of brain machine interfaces. The Instant Genius podcast is brought to you by the team

behind BBC Science Focus magazine, which you can find on sale now in supermarkets and newsagents, as well as on your preferred app store. Alternatively, you can consider, come find us online at sciencefocus.com. This podcast is sponsored by Name, Audio and Focal. The texture and emotional depth of music can be lost through digital sources or poor signal. Name Audio believes you can have digital precision with analogue warmth.

Alongside French acoustic specialist focal, name creates high-end audio systems combining innovation with craftsmanship, so you can listen to music. just as the artist intended. Discover more at name audio.com.