The New Yorker Radio Hour - Should We, and Can We, Put the Brakes on Artificial Intelligence?

Episode Date: June 2, 2023

Sam Altman, CEO of OpenAI, which created ChatGPT, says that AI is a powerful tool that will streamline human work and quicken the pace of scientific advancement  But ChatGPT has both enthralled and terrified us, and even some of AI’s pioneers are freaked out by it – by how quickly the technology has advanced.  David Remnick talks with Altman, and with computer scientist Yoshua Bengio, who won the prestigious Turing Award for his work in 2018, but recently signed an open letter calling for a moratorium on some AI research until regulation can be implemented. The stakes, Bengio says, are high. “I believe there is a non-negligible risk that this kind of technology, in the short term, could disrupt democracies.”  New Yorker Radio Hour listeners, we want to hear from you.  We have a few questions about the show and how you listen to it. The survey takes about twenty minutes, and your feedback will help us make our podcast better.  Take the survey here.

This is The New Yorker Radio Hour, a co-production of WNYC Studios and The New Yorker. Welcome to The New Yorker Radio Hour. I'm David Remnick. Every technological revolution has frightened people, particularly people who've got something to lose. When Gutenberg began printing with movable type, religious and political authorities wondered how to confront a population that had new access to information and arguments to challenge their authority. So it's not surprising that artificial. intelligence is now causing grave concerns because it will affect every one of us. Perhaps the biggest nightmare is the looming new industrial revolution, the displacement of millions of workers, the loss of huge numbers of jobs. Congress has a choice now. We had the same

choice when we face social media. We failed to seize that moment. What is surprising is that some of the very same people who have been racing to develop AI now seem deeply alarmed at how far they've come. In March, not long after ChatGPT began captivating and terrifying us all at once, over a thousand technology experts signed an open letter, calling for a six-month moratorium on certain AI research. And some of those experts say that unchecked AI could be as dangerous to our collective future as nuclear weaponry or pandemics. So we're going to talk today about AI.

How could it change the world and how concerned should we be? I'll start with Sam Altman, the CEO of OpenAI, the company that's been releasing ever more sophisticated versions of ChatGPT. Years ago when the Internet was in its earliest stages, we were surrounded, or at least I felt surrounded, by a sense of internet euphoria. And anyone who raised doubts about it was considered a Luddite or ignorant or a charmingly fearful person passed his cell by date.

Now, with the rise of AI, we're hearing alarm for many quarters. So what I want to try to accomplish here is to have a rational discussion that at once gives a factual picture of where we are, where you think we're going, and at the same time airs out the concerns. So let's just start with the most basic thing. You've been working on AI for nearly a decade. How did you get into it? and what were your expectations?

I mean, this was what I wanted to work on from when I was like a little kid. I was like a very nerdy kid. I was very into sci-fi. I sort of never dreamed I'd actually get to work on it, but then I went to school and studied AI. And one of the memorable things I was told was the only surefire way to have a bad career in AI

is to work on neural networks. And we have all these other ideas, but this is the one that we've proven doesn't work. In 2012, there was a paper put out, one of the authors was my co-founder, Ilya Sudzegovir, which was a neural network doing an amazing thing that performed extremely well in a competition to categorize images. And that was amazing to me, given that I had sort of assumed this thing wasn't going to work. After that, a company called DeepMind did something with beating the world champion at Go. At the end of

2015, we started Open AI. One of our first big projects was playing this computer game called Dota 2. And I got to watch that neural network, that effort, that system sort of grow up. Number one, we truly, genuinely no parlor tricks had an algorithm that could learn. And it got better with scale. It took us a while to discover this current paradigm of these large language models, but the fundamental insight and the fundamental algorithms were all right from the beginning of the company. So GPT suddenly appeared on the scene. So GPT suddenly appeared on the scene and you have talked a lot about its potential and at the same time you've well let's put it this way you freaked a lot of people out what do you see as its potential and do you understand why people

are unnerved about it first of all even the parts that i don't agree with about what people are freaked out about i empathize with to the degree we are successfully able to create a computer that can one day learn and perform new tasks like a human even if you're not even if you don't believe in any of the sci-fi stories, you could still be freaked out about the level of change that this is going to bring society and the compressed time frame in which that's going to happen. Well, let's slow down for a second. What does this imply in the much broader sense about what change is coming down the road? In concrete terms. I think it means that we all are going to have much more powerful tools that significantly increase what a person is capable of

doing, but also raise the bar on what a person needs to do to be sort of a productive member of society and contribute. Because these tools will do eventually, they will augment us so powerfully that they'll change what one person or one small group of people can and do-do. A lot of writers I know have naturally gotten very interested in chat GPT, and they somehow think it's going to eliminate them. I have to admit, I've used your latest version of chat GPT to try and emulate my writing. And without being overproud about it, it kind of didn't. What came out was like an encyclopedia entry with nouns that were subjects that I was interested in. So tell me where this, where chat GPT is in its development now, but should I basically pack it in a couple of weeks when chat GPT is all the better?

We get excited about where things are, but we also try always to talk about the limitations and where things aren't. And maybe a future version of GPT will replace bad writers. but it's very hard for me looking at it now. Every time I talk to someone like you, I say, this is, you know, this is really not it. Yeah. You think we're being defensive. No, no, no, I think you're right.

I think in the sweep of emotion about chat GPT and this new world, it is so easy to say the writing is on the wall. There's going to be no role for humans. This thing is going to take over. And I don't think that's going to be. I don't think we are facing this, you know, total destruction of all human jobs in the very short term. And I think it's difficult and important to balance that with the fact that some jobs are going to be totally replaced by this in the very short term. What jobs do you think will get eliminated pretty quickly in your view?

I think a lot of customer service jobs, a lot of data entry jobs get eliminated pretty quickly. So this may be useful. the thing that you do right now where like you go on some website and you're trying to return something and you like chat with somebody sitting on the other side of a chat bot and they you know send you a label blah blah blah that job i think gets eliminated also the one where you call and you know talk to someone that takes a lot longer but i think that job gets eliminated too but i don't think that most people won't work i think for a bunch of reasons that would be unfulfilling to a lot of people some people won't work for sure. I think there are people in the world who don't want to work and get fulfillment in other ways and that shouldn't be stigmatized either. But I think many people, let's say, want to create, want to do something that makes them feel useful, want to somehow

contribute back to society. And there will be new jobs or things that people think of as jobs that we today wouldn't think of as jobs in the same way that maybe what you do or what I do wouldn't have seemed like a job to somebody that was like doing an actual hard physical job to survive. As the world gets richer and as we make technological progress, standards change and what we consider work and necessity and a whole bunch of other things change to. So I think that's going to happen again with AI. I realize that some of this draws on your essay that was published a couple of years ago, Moore's Law for Everything. You suggest economic policies like a universal basic income. taxes on land and capital rather than on property and labor. And all those things have proven

impossibly difficult to pass, even in the most modified form. How would they become popular in the future? I think this stuff is really difficult, but A, that doesn't mean we shouldn't try, and the way things that are outside the Overton window eventually happen is more and more people talking about them over time. And B, when the ground is shaking, I think is when you can make radical policy progress. So I agree with you. Today, we still can't do this. But if AI stays on the trajectory that it might, you know, perhaps in a few years, these don't seem so radical. And we have like massive GDP growth at a time where we have a lot of turmoil in the job market. Maybe all this stuff is possible.

And the more time up front we have for people to be studying ideas like this and contributing new ones, I think the better. I believe we have a real opportunity to shape that if you take something, a good that has been super expensive and limited and important and make that easy to access and extremely cheap. I believe that is mostly an equalizing force in the world. And we're seeing that with chat chbt. One of the things that we tried to design into this, and I think is an exciting part of this particular technological revolution, is anyone can use it. You know, kids can use it, old people can use it,

people that don't have familiarity with technology can use it. You can have a very, like, you know, cheap, cheap mobile device that doesn't have much power and still get as much benefit out of this as someone with the best computing system in the world. My dream is that we figure out a way to let the governance of these, systems, the benefits they generate, and the access to them, be equally spread across every person on Earth. This is the New Yorker Radio Hour, and I'm talking today with Sam Altman, the CEO of Open

AI, which developed ChatGPT and GPT4. Sam, talk to me about artificial general intelligence, which seems to be a step even past what we've been talking about. I think it's a very blurry line. I think artificial general intelligence means to people like very powerful artificial intelligence. It's sort of shorthand for that. My personal definition is systems that can really dramatically impact the rate that humans make scientific progress

or that society make scientific progress. Other people use a definition like systems that can do half of the current economically valuable human work. Others use a system that can learn new things on its own. That latter point is the thing that creates anxiety, isn't it? That it's a system that can operate beyond the bounds of human influence. Well, there's two versions of that. There's one that causes a lot of anxiety even to me, and there's one that doesn't.

The one that doesn't, and the one that I think is going to happen, is not where an AI is off right in its own code and changing its architecture and things like that, but that if you ask an AI, a question that it doesn't know the answer to, it can go do what a human would do, say, hey, I don't know that. I'm going to go read books. I'm going to go call smart people. I'm going to go have some conversations. I'm going to think harder.

And now I'm going to have some new knowledge stored in my neural network. And that feels fine to me. Definitely the one where it's off, like writing its own code and changing its architecture. Very scary to me. AI systems have already generated skills that its creators didn't expect or prepare for. Learning languages. It wasn't programmed to learn, figuring out. how to code, for example. So the worry is that AI could break free from its human overseers and wreak havoc

of one kind or another. The fundamental place that I find myself getting tripped up in thinking about this and that I've noticed in others too is, you know, is this a tool or is this a creature? And I think it's so tempting to project creatureness onto this because it has language and because because that feels so anthropomorphic. But what this system is, is a system that takes in some text, does some complicated statistics on it, and puts out some more text.

And amazing, emergent behavior can happen from that, as we've seen. That can significantly influence a person's thinking, and we need a lot of constraints on that. But I don't believe we're on a path to build a creature here. Now, humans can really misuse the tool in very big ways. And I worry a lot about that, much more than I worry about currently the sci-fi-esque kind of stuff of this thing, you know, wakes up and loses control.

Sam, you've had quite a few conversations lately with lawmakers. You testified in front of a Senate subcommittee, and that was widely reported. But before that, you had a private meeting at the White House. Tell me who was there and what was the conversation about? It was a number of people from the administration led by Vice President Harris, and then the CEOs of four AI or tech and AI companies. And the conversation was about as we go heading to this technological revolution, what can the companies do to help ensure that it's a good change and help sort of reassure people that we're going to get the things right that we're able to get right and then we need to in the short term. And then what can the government do? What are the kinds of policy ideas that

might make sense as this technology develops? One area in particular that I am worried about in a short term is provenance of generated content. We've got an election next year. The already image generation is incredibly good. Audio generation is getting very good. Video generation will take a little bit longer, but we'll get good too. I'm confident that we as a society with enough time can adapt to that. You know, we've learned when Photoshop came out, people were really tricked for a little while and pretty quickly people learned to be skeptical of images and people would say, oh, that's Photoshopped or that's doctored or whatever. So I'm confident we can do it again, but we also have a different playing field now and there's sort of Twitter and these telegram groups and however else this stuff spreads.

there's a lot of regulation that could work, and there's technical efforts like watermarking images or shipping detectors that could work in addition to just requiring people to disclose generated content. And then there's like education of the public about, you've got to watch out for this. Ultimately, who do you think was the most powerful people in the room? The people on the government side

or the people heading the tech companies? That's an interesting question. I think the government certainly is, more powerful here in even the medium term, but the government does take a little bit longer to get things down. And so I think it's important that the companies independently do the right thing in the very short term. But you understand that, again, years ago, tech and, you know, the cover of wired and all, there was a kind of euphoria attached to technology that in the past several years. does it? No, it doesn't feel that way at all. Not because I relish it, but the public images of places

like Facebook and Google are not what they were. And I think trust in those companies to get things right. So when we hear about a conversation at the White House between the vice president and her colleagues and the heads of tech companies, we want to intensely know what is going on what the conversation is like and what it's leading toward who's in charge, it would be really good to know the details of that. The right answer here very clearly is for the government to be in charge, and not just our government. I think this is one of these places where, and I realize how naive this sounds and how difficult it's going to be, we need international cooperation. The example that I've been using recently is I think we will need something like the IAEA

that we had for nuclear for this and it's going to atomic weapons obviously and atomic energy and I think that's so difficult to do it requires international cooperation between superpowers that don't get along so well right now

but that's what I think the right long-term solution is given how powerful this technology can grow I'm actually optimistic that it's technically possible to do. I think the way this technology works, the number of GPUs that are required, the small number of people that can make them and the controls that could be imposed on them to say nothing of the energy requirements for these systems. It is possible to internationally regulate this. So I think the government has got to lead the way here. I think we need serious regulation

from the government, setting the rules. I think it's good for the tech companies to provide input, say where we think the technology is going, what might work technically and why. won't, but the government, and really the people of the world, have got to decide. Sam Altman, thank you very much. Thank you. Sam Altman is the CEO of OpenAI, which created ChatGPT. We're going to continue on the risks and benefits of AI in just a moment. This is the New Yorker Radio Hour.

This is the New Yorker Radio Hour. I'm David Remnick. We're talking today about the promise and the danger of artificial intelligence. computer scientist Joshua Benjio began working on AI in the 80s and the 90s, and he's been called one of the godfathers of AI. Benjiio focused specifically on neural networks. That's the idea that software can somehow mimic how the brain functions and learns. The brain itself is a kind of network. Now, at the time, most scientists thought this would never really work out, but Benjiio and a few others persevered. their research led to advances in voice recognition and robotics and much more. In 2018, Benjiio received the Turing Award, kind of the Nobel Prize of Computing,

alongside Jeffrey Hinton and another colleague. ChatGPT is also built on the foundation that Benjiio helped to build. It's a neural network. But Benjiio, instead of celebrating this remarkable achievement in his field, has had quite a different reaction. So in March, a group of viewers, very prominent people in tech, signed an open letter that said that all AI producers should stop training their systems for at least six months. And you signed that letter. Even Elon Musk,

who's not known for his overweening sense of caution, also signed the letter. Please tell me how that letter came about and what was the motivation. We saw the unexpected and rapid rise of the abilities of AI systems like chat GPT and then GPT4. we didn't ask to stop every AI research and development and deployment, only those very powerful systems that are of concern to us. I believe there is a non-negligible risk that this kind of technology in the short term could disrupt democracies. And in the coming years with advances that people are working on,

could yield to loss of control of AI, which could have potentially even more catastrophic impacts. So I just spoke to Sam Altman, and I asked him about what seems to be the most frightening concern of all, that an AI entity could basically become a sentient creature that could rewrite its own source code and somehow, as if in a horrifying science fiction movie,

break free from human control. All of men assured me this is very unlikely. What do you think? Did he say it was unlikely with the current systems or in the future? The current systems, to be sure. Yes, so I agree with him. But what about in the future? For the future, yes, there is a real risk.

It's a risk we don't understand well enough. In other words, you can see experts like my friend Jan Lekin saying one thing, and other experts like Jeff Hinton and I saying the opposite. The scenarios by which bad things can happen haven't been sufficiently discussed, studied. A lot of people talk about AI alignment. In other words, the fact that you may ask a machine to do something and it could act in a different way that could be dangerous. There is an alignment problem of a different kind between what's good for society and the general well-being of people and what companies are optimizing, which is profit under constraints of being

legal. It's actually interesting because I find that as an inspiration to better understand what can go wrong with AI. So you can think of corporations as special kind of intelligences that are not quite completely artificial because there are human beings in there, but that can behave in a similar way. We try to bring corporations back into alignment with what society needs, with all kinds of laws and regulations. And in particular in the case of AI, I think we need a regulatory kind of framework that's going to be very adaptive because technology moves.

moves quickly, science moves quickly. We don't want Congress or parliaments and other countries to be the ones dictating the details. They want to assign some more professional body that are not politicians, but they are experts to find the best ways to protect the public. Well, how would you and Jeff Hinton and others describe a very bad outcome? What is the scenario that you envision is at least possible and unpredictable and dangerous. Imagine that in a few years, scientists figure out how to build an AI system that would be autonomous and could be dangerous for humanity because it would have its own goals that may conflict with ours. And maybe we even also have figured out how we can build safe AI that wouldn't behave

like this. The problem is we have that choice. And maybe those scientists in the labs would choose the good AI solution. But there could be somebody anywhere on the world if they have access to the required compute, which right now isn't that much. You can take... So think about chat GPT. You don't need to retrain it.

You just need to give it the right instructions. Anybody can do that. What is the scenario that you see in specific terms as a possibility that you are trying to prevent? There's an organization called Auto-GPT, which arose. just in the last few weeks or months that made it possible to turn

something that has very little or no agency like CHAPT into something that actually pursues goals that the human would type, but then creating its own sub-goals, you know, to achieve those. It's increasing the chances of an AI system becoming, like, dangerous for humanity because they are like connecting, for example, that system to the internet.

It could ask people to do things for money through existing facilities for this. If we had, instead of chat GPT, something that's smarter than humans, which may arrive in as few as five years, I don't know, then that could become catastrophic. You've raised the idea of AI being exploited in military. use, how should the military use artificial intelligence, if at all? What are the dangers? Well, the danger is first that we're putting a lot of difficult moral decisions in the hands of machines that may not have the same understanding of what is right and wrong as we do. you may know about the story of the Russian officer who decided not to press on the button

in spite of the instructions that would have led to probably catastrophic a nuclear exchange because he thought it was wrong and it was a false alarm right if we build AI systems with agency and autonomy and they can kill because their weapons, it just makes the likelihood of something really catastrophic happening larger. But then there's just like simple, let's say, Putin wants to destroy Western Europe and take advantage of AI technology to do it in a way that might not be possible otherwise. If AIs are embedded into the weapons, the military, then it just gets easier to have large-scale, dangerous impacts.

I have to ask you, you've been working in this field. for many years. Why is it suddenly... Decades. Decades. Suddenly everybody's very concerned about it. There have been rumblings about it over the year, not only in the field, but beyond the field.

It's exploded, this level of concern. What happened? And why wasn't it foreseen a little earlier? Well, it was foreseen by some, as you said, kind of a marginal group. So there's the fact that most of us in AI research did not expect that we would get to the level of competence that we seem to see in the chat GPT and GPT4.

We expected something like this, that level to come maybe in 10 or 20 years, and the human level intelligence to come maybe in 50 years. I mean, a horizon for risk just got much shorter. If you're working on a topic, it's more psychologically confident. to think that this is good for humanity than to think, oh, gee, this could be really destructive. And we have these natural defenses as part of the problem with humans. We're not always rational. Is there a possibility that AI leads to an even greater disparity, social disparity, income disparity?

What prevents a scenario where the benefits of AI are concentrated among a very small slice of the population and vast numbers of people are experienced dislocation, unemployment, and actually get poorer. In general, if you think about what AI is, it's just a very powerful tool. If you just think about having very powerful tools, it can clearly be used by people who have power to gain even more power.

What prevents that tends to be governments, you know, taxation, for example, and services offered by governments to everyone and so on. so as to balance things out. Are you concerned that the warnings coming from Jeff Hinton, from Steve Wozniak, come across to some people as the warnings of an old guard complaining about a new generation of scientists? No. I mean, my students are concerned. And there are young people who are concerned.

I think that the battle that is shableness. shaping up, in a way, has a lot of points in common with the concerns and the requirement for policies about climate, climate change. And a lot of young people are fighting to preserve the interests of future generations. I think something similar is at stake with the eye. One of the confounding things about confronting the climate emergency is the requirement for coordinated international effort. With AI, you not only have that, but you also have, I think it's fair to say, a level of understanding of the basics of AI that's very low. In other words, people can understand dried up rivers, raising temperatures,

rising sea levels and all the rest, the complications of AI and predicting those complications are even more complex, don't you think? So I think what may bring countries to this international table that is needed indeed is their self-interest in avoiding catastrophic outcomes where everyone loses. So a good analogy is what happened

after the Second World War between mostly the U.S. and the USSR and to some extent China, to come up with agreements to reduce the risks of nuclear or Maged. I think in good part, thanks to these international agreements that it has been okay. So the comparison is to arms control, nuclear arms control? Yes. It's not exactly the same thing, but I think it's a good model. Mr. Benjillo, thank you so much. I appreciate your time. My pleasure. Thanks for having me.

Joshua Benjio is the scientific director of the Montreal Institute for Learning Algorithms. I'm David Remnick, and that's the New Yorker Radio Hour for today. Thanks for listening. I hope you'll join us next time. The New Yorker Radio Hour is a co-production of WNYC Studios and The New Yorker. Our theme music was composed and performed by Merrill Garbess of Tune Arts with additional music by Louis Mitchell. This episode was produced by Max Walton, Brita Green, Adam Howard, Calalea, Avery Keatley, David Krasnow, Jeffrey Masters, Louis Mitchell, and Gophane and Putabuele, with guidance from Emily Boutin and assistance from Harrison Keatline, Michael May, David Gable, and Alejandro Decker.

The New Yorker Radio Hour is supported in part by the Cherina Endowment Fund.