The New Yorker Radio Hour - Geoffrey Hinton: “It’s Far Too Late” to Stop Artificial Intelligence

Episode Date: November 21, 2023

The American public’s increasing fascination with artificial intelligence—its rapid advancement and ability to reshape the future—has put the computer scientist Geoffrey Hinton in an awkward position. He is known as the godfather of A.I. because of his groundbreaking work in neural networks, a branch of computer science that most researchers had given up on, while Hinton’s advances eventually led to a revolution.  But he is now fearful of what it could unleash. “There’s a whole bunch of risks that concern me and other people. . . . I’m a kind of latecomer to worrying about the risks, ” Hinton tells The New Yorker’s Joshua Rothman. “Because very recently I came to the conclusion that these digital intelligences might already be as good as us. They’re able to communicate knowledge between one another much better than we can.” Knowing the technology the way he does, he feels it’s not currently possible to limit the intentions and goals of an A.I. that inevitably becomes smarter than humans.  Hinton remains a researcher and no longer has a financial stake in the success of A.I., so he is perhaps franker about the downsides of the A.I. revolution that Sam Altman and other tech moguls.  He agrees that it’s “not unreasonable” for a layperson to wish that A.I. would simply go away, “but it’s not going to happen. … It’s just so useful, so much opportunity to do good.” What should we do? Rothman asks him. “I don’t know. Smart young people,” Hinton hopes, “should be thinking about, is it possible to prevent [A.I.] from ever wanting to take over.” Rothman’s Profile of Geoffrey Hinton appears in a special issue of The New Yorker about artificial intelligence.  Sam Altman, the C.E.O. of OpenAI (which created ChatGPT), spoke with David Remnick on this episode of The New Yorker Radio Hour.  New Yorker Radio Hour listeners, we want to hear from you.  We have a few questions about the show and how you listen to it. The survey takes about twenty minutes, and your feedback will help us make our podcast better.  Take the survey here.

This is the New Yorker Radio Hour. I'm David Remnick. So the way the brain works is this. Neurons get inputs, and if they get enough input, they go ping. And the input to a neuron either comes from the senses, but for most neurons, it's input from other neurons. And so a neuron's receiving these pings from other neurons. Jeffrey Hinton has been thinking about how brains work for a very long time. Hinton is a computer scientist who's been called the godfather of artificial intelligence, AI.

For decades, he worked on building computers that would work in a way analogous to the human brain itself. It's an approach known as neural networks. This was an obscure and seemingly fruitless effort for a long while, but eventually it paid off beyond anybody's imagination. That work on neural networks led to incredibly intricate machines

like Dolly, which will take your prompts and make you a beautiful picture. Or chat GPT, which in the last year put AI on everybody's radar, well, that was a future that nobody expected. These are machines that learn and perhaps even think. What strength are associated with each incoming ping, so that it can decide if it got enough input for it to go ping. And that's all there is.

That's all you need to know to know how the brain works. It's very clear that an AI revolution is at hand, here and now, and it's going to reshape our world profoundly. But Jeffrey Hinton, the foremost pioneer of the first pioneer. of neural networks has come to have concerns about what AI can do. Very serious concerns. Joshua Rothman, the New Yorker's ideas editor, recently talked with Hinton in-depth, and will hear some of their conversation today. Josh, the New Yorker, has just published an entire issue on artificial intelligence, and at the very center of it is your profile of Jeffrey Hinton.

Why is he so important? Why is he such a crucial figure? So he's followed the arc of the tech from the very beginning all the way to now. He's 75. And during a period of time when nobody thought this technology would work, he continued to work at it. And he believed in it. And he's ultimately been proven right. And he's now said that he's scared about the tech that he worked on for his whole life. He doesn't regret what he did.

But he says we need to be realistic about what's been invented, which is a machine that can think the way we can. Tell me a little bit about what it's like to spend time with Jeffrey Hinton. He's very emotionally rich as well as intellectually rich personality as you portray him. What was Hinton actually like as a person? A delightful person. He's a little bit from a prior world. He's not a Silicon Valley techno overlord.

He's not an eccentric egomaniac. He's a highly intelligent, basically humble person who's worked on this technology for a long time, who got used to being a regular, a regular computer science professor until he was in his 60s when this technology really started to take off. I was pretty intimidated by Jeff. Our first interaction, he sort of gave me a quiz on various subjects and philosophy of mind, I think to sort of like confirm that we were going to be on the same on the same wavelength. And I have to say I'm a basically a regular Joe.

I don't really understand like I did a Khan Academy course on linear algebra. I did some things to get ready. You prepare to do this piece by taking a course online in linear algebra? I did. Wow. The strangeness of AI is so weird. You know, it combines physics and math and neuroscience and computers. It's like a weird discipline in psychology

and ideas about learning and all this kind of stuff. What is AI and what are its implications because they seem so varied, so vast, to some people so scary and to many, many other people, so filled with possibility. So I think the question of what it is

is a little bit of a contested one. But the best way for me to understand it as a mere mortal is to go back to the beginning. The way this all started was with the idea that, you know, our brains are powered by neurons that are connected in a network. And that created, that led to a field called neural networks, a field of computer science, in which computer scientists would create networks of simulated neurons inside computers. Back in the day, in the 60s, 70s, 80s, you couldn't do that.

That was impossible to simulate in a computer. But you could build small networks, and they could learn small things. They could learn to recognize handwritten digits, for example, like, say, in a zip code on an envelope. But over the last many decades, computers have gotten bigger and bigger. They've gotten literally a billion times faster. The number of neurons that can be simulated inside a computer has grown by that scale. Now, these neural networks, they're not yet as complicated as the ones in our heads, but they're really complicated. and they're capable of doing something

that certainly looks from the outside to many people like reasoning. Jeff Hinton thinks that they're understanding just like we do, their reasoning just like we do, we should take their mental lives seriously, as it were, and we should take their intelligence seriously. You spent quite a lot of time with him at his house on an island in Lake Huron. Let's hear some of your interview. A lot of people struggle to understand how an AI mind is similar to or different from a human mind.

And they can't decide or they don't know whether to think of today's AIs as similar to the computer programs they've used their whole lives or similar to the people that they kind of. inverse with. Like, how do you think about that? So I think today's big AI, things like chat, GPD4, they're much more similar to people than they are to computer programs. So computers were designed so people could program them. That is, they'd do exactly what you said. And they didn't have things like intuition. But now if you look at what it took to make computers good at chess, you had to give computers intuition. A computer had to be able to look at a board and think, oh, So that will be a good move the way a grandmaster does.

In fact, they're better than grandmasters of that now. And using neural nets, we could get computers to learn intuitions. And that's very different from logical reasoning. And when you're an expert in a domain, to begin with, you have to do reasoning, a real expert in a domain can just do it intuitively. That's what doctors call clinical experience. They just look at this patient and they know what they've got. And they didn't do a lot of reasoning.

just obvious to them. But if they're good doctors, occasionally it turns out the patient hasn't got that, and they'll use that to revise their intuitions. It sort of seems like when I write, I'm a writer, when I write, I'm the one doing it from the top down using my intelligence. It's not like driving a car or riding a bike. It's something I'm choosing to do, and those are my ideas coming out of my ego, as it were. And that's what I think of when I think of, like, high-level intelligence is like me writing an article, writing an email or something. It doesn't necessarily seem like it's connected

to the world of learned intuitive behavior. But if you think when you're writing, suppose you're halfway through a sentence, and now you have to choose the next word, so you choose, there'll be a word that comes to mind because it sort of fits nicely there. How did you decide,

Why did it come to mind and how did you decide it fitted nicely? You have no idea. You can retrospectively, you can make up a story, which has probably got some element of truth, but definitely not the whole story. And really what's happening is, these big patterns of neural activity that you've learned are in effect implementing analogies with lots of different stuff you know

so that that word seems right there. just the process of selecting the next word when you're writing, which you might say is just you doing autocomplete, involves more or less everything you ever learned. To do autocomplete properly, you have to understand how the world works, and that's what you're doing. You're just an autocomplete device. So Josh Hinton is teasing you here.

He's saying that your excellent writing is just autocomplete. At least I hope he's teasing because he's wrong. But what are the implications of, a machine with intelligence? What could it bring to society in the most positive sense? And what should we fear? So on the positive side, there's ways in which these digital minds are different from ours, and I should say usefully different from ours.

So if you think about what chat GPT is, I think a lot of us have used chat GPT at this point. One of the striking things is it seems to have ready access to a huge amount of knowledge. like more than we do. It doesn't mean that it's smarter than us exactly. But if you ask it to translate something between languages or to try to solve an equation or to discourse on the history of economics, like it can do that.

Because these artificial intelligences are really good at working with huge amounts of data. Is this capacity because a machine at artificial intelligence can have in its head, as it were, all of Google, all of Google translate, and then begin to work with it, whereas our minds do not have that capacity. Is it a capacity question? Yeah, it's partly a capacity question, as I understand it. I mean, I think it's different. Like, you think about what our minds are doing right now, for example, you know, you and I are having this conversation,

we're moving towards a deeper level of mutual understanding, all the while where maybe there's some part of our brains, thinking about what we're going to have for lunch. You're a very busy man. I'm sure you have a lot going on. You're thinking about all that stuff too. And AI also, and this is something that Jeff talked about in my piece that I thought was fascinating. You know, if I learn something, how do I communicate it to you? I have to write a 10,000 word article for the New Yorker magazine. And you have to read it and try to stay awake during it. If an AI learned something and it wants to communicate it, it just downloads the information and it can be uploaded into another AI. The first thing that comes to mind is AIs have mastered a level of conversation and of not just

responding to what you say, but of understanding your intent. There's a piece in the issue that's about this. That's about how AIs is affecting coders. One of the ways that chat GPT is very powerful is that if you're sufficiently educated about computers and you want to make a computer program and you can instruct chat GPT in what you want with enough specificity, it can write the code for you. It doesn't mean that every coder is going to be replaced by Chad GPT, but it means that a competent coder with an imagination

can accomplish a lot more than she used to be able to. Maybe she could do the work of five coders. So there's a dynamic where people who can master the technology can get a lot more done. So there's economic consequences that are real, and then there's this sort of bigger fear, a sort of science fictional fear, which Hinton shares. There's a whole bunch of risks that concern me,

and other people have talked about these much more than I have. I'm a kind of latecomer to worrying about the risks, because very recently I came to the conclusion that these digital intelligences might already be as good as us, They've able to communicate knowledge between one another much better than we can. So that's what made me feel I needed to talk out about the existential threat that these things will get to be smarter than us and take over. And that's because an AI can just copy its learned knowledge out of itself

and give it directly to another AI. So you can have, say, 10,000 different copies of the same knowledge, of the same neural network. each can be looking at different data and when one copy learns something from one part of the data it can convey it to all the other copies that haven't seen that data simply by telling them how to update their weights these synapse strengths inside. Now you and I can't do that because my brain's wired differently from your brain

and if you told me the synapse strengths in your brain it wouldn't do me any good. Right. How does that relate to these set of risks that are? Okay, so that relates to the existential threat that these things will become smarter than us, and not just a little bit smarter, but a lot smarter, and will also decide to take over. They'll decide to take control. That's the existential threat.

And why would they decide to do that? A very senior official in the European Commission, who I was talking to, said, well, people have made such a mess of things. Why wouldn't they? Computer scientist Jeffrey Hinton. He's speaking with the New Yorker's Joshua Rothman, and will continue in a moment.

This is the New Yorker Radio Hour. This is the New Yorker Radio Hour. I'm David Remnick. One year ago, the future arrived, loudly. ChatGPT launched at the end of last November, and it was all anybody could think about for a while. Suddenly, artificial intelligence wasn't just a tool for advanced science research,

but it was entering all of our lives, right down to your kids cheating on their homework. Our current issue of The New Yorker is all about this explosion in artificial intelligence. the mind-boggling advances, and some of the terrifying possibilities. And as part of that project, our ideas editor Joshua Rothman sat down with the so-called Godfather of AI, Jeffrey Hinton. We'll continue with that conversation now.

Now, Hinton has spent a lifetime helping to teach machines how to learn, and now he believes that he's succeeded almost all too well, and he's scared of what may happen when machines are smarter than people and have their own ideas about what to do. Suppose it's a chess playing computer. It wants to win the game. It doesn't have anywhere inside it an ego which thinks I want to win the game. It's wired up in such a way that it's trying to win. So I think the idea they don't have intentions and they don't have goals is just wrong. It is the idea that people in charge of these systems will give them goal. will start us down this path?

I mean, is that what we're envisioning? Where would the goals come from that would start the whole problem? Okay, there's two sources of worry, and they're very distinct, and they have quite different solutions. So one worry is bad actors. You can probably imagine Putin giving an autonomous lethal weapon in the goal of killing Ukrainians, and you can probably imagine he wouldn't hesitate.

So that's the bad actor's scenario. But there's another scenario which is, if you want a system to be effective, you need to give it the ability to create its own sub-goals. Like, if you want to get back to the US, you're going to need to get to an airport. And so you have a sub-goal, get to an airport. That's a sub-goal. It's created in order to achieve a bigger goal. So if you give an AI some goal and you want it to be effective, is going to work by creating some sub-goals that will allow it to achieve the goal.

Now the problem is, there's a very general sub-gold that helps with almost all goals. And the AI will certainly realize this very quickly. The very general goal is get more control. If I just get more control, it's going to help with everything. And these AIs are going to realize that, and pretty soon they're going to realize, well, if these are my goals, best things are, to do is stop humans interfering and just get on with it and do it a sensible way that these stupid humans don't understand. Whatever goals they do have were given to them by us. And a big

question called the alignment problem is, can we give them goals such that they do useful things for us and they never, ever want to take over? Nobody knows how to do that. It's no use thinking we could air gap them so they can't actually pull levers or press buttons, because they could simply convince us to do it, because they're much more intelligent than us. And I mean, is that a technical problem? Yes. And then it's also a governance problem. It's a technical problem, a governance problem.

But we don't even know how to solve the technical problem, even if we could do the governance. Even if we could make a law that said, you're not allowed to make an AI that can go wrong. We wouldn't know how to do that yet. Exactly. Imagine that you're not a central figure in the history of machine learning, but you're just a regular person. And now some of the world's biggest companies are saying, we've developed this technology. It promises all sorts of benefits that you don't really want. You can drive your car. You can do your job. You can do everything you need to do in your life. We don't know how to control it. It might take your job or it might take over.

It might help solve some scientific problems that you don't care about. I think that regular person might just say, why don't we just unplug it? I mean... Why don't we just stop the development? Why don't we just unplug it? We don't need this. We don't need... We already have intelligences, human beings.

We don't need artificial ones. It's not unreasonable to say we'd be better off without this. It's not worth the risk. just as we might have been better off without fossil fuels. We'd have been far more primitive, but it may not have been worth the risk. But it's not going to happen. Because of the way society is, because of the competition between different nations, no one nation could stop it.

If you had a very powerful world governance, if the UN really worked, possibly something like that could stop it. although even then it's just so useful. It has so much opportunity to do good like in medicine. You just aren't going to stop it. It's also got so much opportunity to give advantage to a nation by our autonomous weapons. I mean, the US is already developing autonomous weapons. So yes, that might be a sensible move, but it's far too late. Yeah. So my last question in this vein is, you know, what should we do?

I don't know. It would be great if it was like climate change, where someone warning about climate change could say, look, we either have to stop burning carbon, or we have to find an effective way to remove carbon dioxide from the atmosphere. And if we, one of those two is essential. And you know what the solution looks like. It's just a question of the political will to actually implement something because it's going to be painful. Here it's not like that. I have no advice. All I'm doing is just warning

that this may well be coming. Smart young people should be thinking hard about is it possible to prevent it ever wanting to take over? Josh, we know the names Bill Gates, Steve Jobs, Alan Turing, but only very recently if we learned the name, Jeffrey Hinton.

And we've learned it, A, as somebody who is a great innovator in AI, known very commonly as the godfather of AI, but now also as the apostate of AI, why has he only emerged now? I mean, it's like the first question you want to ask is, like, it's 2023. So how come in all the decades you didn't freak out before? And he told me, you know, first, no one thought this would work. as quickly as it did. If you rewound the tape to the 80s, the 90s, the 2000s,

people thought, you know, maybe in 50 years' time, he thought maybe in 100 years' time you would reach the place where we currently are. And so his view was like, why worry about it? You know, there's plenty of time to try to sort this out down the road. A lot of people who work in AI have mentioned this to me. It's that when you start using Chad GPT or another modern AI, the first thing you notice is all the ways it's bad.

So the first thing you see is the way it's not really human or the mistakes it makes, the things it's not capable of. And your early impressions are, gee, you know, like nice, it's a lot of window dressing. But the more time you spend, the more you get impressed. I mean, it reminds me a little bit. I have two small kids. I have a five-year-old son.

And when you spend a lot of time around one kid, you notice every little improvement in their mind. You're always noticing things they're, learning. But if you just visit with friends who have a kid, you don't, you're not impressed. You say that's a little child. Josh, in this issue of the magazine, along with your profile of Jeffrey Hinton, there's a piece by A.L. Press about AI and facial identification, which seems extremely fraught, quite dangerous. What's the issue at stake there? Well, the issue is an AI that provides quote-unquote evidence

to the police could very well be wrong. And the people who are using these systems might not fully understand how they work or what their limitations are. But once a computer system that appears objective, appears powerful, once it makes a judgment, it can be pretty hard to contravene the judgment. It's a problem in policing.

It could be a problem in medicine. Imagine you're a doctor. An AI system delivers a diagnosis. you think it's probably wrong, but are you going to go up against the computer and go on the record and say, I disagree with the neural network, with billions of artificial neurons running in a huge data center?

You might not say that. And so the larger question is, as these systems get more useful, as they get more integrated into real-world contexts, and as real people have to start being responsible for them, have to start either disputing what they think or acting on what they think, and what the systems think,

it's going to require a level of literacy and of nuance around the technology that we're just really not equipped to have at the moment and that we need to start developing if we don't want really negative consequences. John, recently I had a conversation on this show with Sam Alton and one of the people behind ChatGPT. And in a very almost bland way, he said that AI could put millions and millions. of people out of work, which would cause the government to have to come around and start giving us all universal basic income while the machines do the work and the thinking. And, and, and, he delivered this all in a very, I have to say, matter-of-fact way. Is this a crucial concern for Jeffrey Hinton? He certainly worried about what will the young people of today do for work? how will the world of work be transformed?

Well, I guess can I back up and say one thing broadly? Sure. There's huge disagreement about this. You know, there's, no one has a crystal ball. And technologists obviously are in love with what they've made. They really see the potential. They live in the future. That's why they do what they do.

They're in love with it, or they stand to make an immense fortune? Yes, and they stand to make immense amounts of money. And I think Hinton is a little different from Altman. Because he doesn't have any real financial skin in the game. He's not a businessman. He's an academic researcher fundamentally, although he worked for Google for a period. And walked away, as I recall, with tens of millions of dollars, now? Yeah, he sold a company to Google in the early days of the current AI boom for $44 million.

But it is something, job loss is something that Hinton is very worried about. But I think he's more worried about, I guess you could just say chaos. his view is the technology is out there bad people will learn how to use it and the next period of history will be destabilized by this technology and there isn't an obvious way to solve the problem Isn't that a little easy

we've been through this now with the internet Arguably He didn't say this to me but this is an argument many people make The best path to understanding and controlling AI is simply to keep working on it That sounds a little fatalistic

Yeah I think he's I think the word he would use is a stoic I think he sees his position as a realist one. And he thinks that these are problems that aren't going to be worked out through people sitting around and talking about them. They're problems that are going to have to be worked out through the actual development of the technology. And a lot of people say one thing and do another thing.

So it's kind of like they say the technology can't be stopped. And then at the same time, they say, let's sign a letter proposing that we stop it for six months. There's a sort of wanting to have it both ways. And I don't mean this to say that they're being cynical. I think these are just two impulses when you look at a technology like this. One impulse is to say... You're talking about people like Sam Altman, to be clear. One impulse is to say, you know, we can control this.

And the other is to say, you know, what do we mean by we? There's endless researchers around the world. This technology is available. visible. Kinton's view, it's alarming. It freaked me out personally. It also struck me as at least consistent. And I think if you look at the history of what happened with, say, nuclear weapons, we do now have a regulatory regime around them, but that took a while to build. And bad things had to happen first. Yeah, and you also have a Russian leader who, if he wants to, can threaten to use them in Ukraine.

Right. So we've never really solved that problem. Do the benefits of AI outweigh the existential threat to humanity as Hinton himself has posted? Before I wrote this piece, I think I had an idea of what AI was, which was that it was just statistics. It was just number crunching. And learning more about the history of the field and learning more about the innovations that Hinton helped create and that other people helped create. I feel like this technology is incredible. I guess my overall feeling is, people are really smart. If we can build it, we can hopefully control it in some way.

I left the piece even more impressed by what AI can do, by what it really does. I think it's worth it. I think some critics of AI or skeptics of AI feel that the, that the, that the Endeavour is somehow anti-human, anti-human, like saying that what the AI does and what we do is the same, for example, is sort of to diminish what the mind does, the human mind. Right. If you want to be mystical about it and think that humans have some mystical special property that a machine could never have, obviously it does diminish that. I don't believe humans have mystical properties that machines couldn't have.

I believe we're just wonderful and very complicated machines. I shouldn't even use the word just. We're wonderful and very complicated machines. And trying to understand how these machines work gives us much more insight into what we are. It tells us about our true nature. I think this is giving us enormous insight into the kinds of machines we are.

And it's clearly a huge revolution. I mean, the industrial revolution was when we could replace physical labor with machine labor. And now we can replace intercourse. intellectual labor with machine labor, and it's a revolution of at least the same scale. Do you think we'll have to think differently about what's valuable about ourselves and what makes us unique? That question has an assumption in which is what makes us unique. Maybe we're not.

You can read Joshua Rothman's profile of Jeffrey Hinton in The New Yorker. And you can find my earlier conversation with Sam Altman of the company that runs chatGPT at New YorkerRadio.org. I'm David Remnick, and thanks for listening today. See you next time. The New Yorker Radio Hour is a co-production of WNYC Studios and The New Yorker. Our theme music was composed and performed by Merrill Garbes of Tune Arts, with additional music by Louis Mitchell. This episode was produced by Max Walton, Adam Howard, Kalalia,

David Krasnall, Jeffrey Masters, and Louis Mitchell, with guidance from Emily Boutin and assistance from Mike Cutchman, Michael May, David Gable, and Alejandro Decker. We had additional help this week from Jared Paul. The New Yorker Radio Hour is supported in part by the Cherina Endowment Fund.