Making Sense with Sam Harris - #94 — Frontiers of Intelligence

Episode Date: August 29, 2017

Sam Harris speaks with Max Tegmark about his new book Life 3.0: Being Human in the Age of Artificial Intelligence. They talk about the nature of intelligence, the risks of superhuman AI, a nonbiological definition of life, the substrate independence of minds, the relevance and irrelevance of consciousness for the future of AI, near-term breakthroughs in AI, and other topics.   If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.

Thank you. of the Making Sense podcast, you'll need to subscribe at SamHarris.org. There you'll find our private RSS feed to add to your favorite podcatcher, along with other subscriber-only content. We don't run ads on the podcast, and therefore it's made possible entirely through the support of our subscribers. So if you enjoy what we're doing here, please consider becoming one. Today I am speaking with Max Tegmark once again. Max is a professor of physics at MIT and the co-founder of the Future of Life Institute, and has helped organize these really groundbreaking conferences on AI. Max has been featured in dozens of science documentaries, and as I said, he's been on the podcast once before. In this episode, we talk about his new book, Life 3.0, Being

Human in the Age of Artificial Intelligence, and we discuss the nature of intelligence, the risks of superhuman AI, a non-biological definition of life that Max is working with, the difference between hardware and software, and the resulting substrate independence of minds, the relevance and irrelevance of consciousness for the future of AI, and the near-term promise of artificial intelligence. All the good things that we hope will come from it soon. And we touch other topics. And this is a conversation that Max calls the most important conversation we can have. And I more or less agree. I would say that if it isn't now the most important conversation we can

have, it will one day be. And unlike most things, this topic is guaranteed to become more and more relevant each day, unless we do something truly terrible to ourselves in the meantime. So, if you want to know what the future of intelligent machines looks like, and perhaps the future of intelligence itself, you can do a lot worse than read Max's book. And now I bring you Max Tegmark. I am here with Max Tegmark. Max, thanks for coming back on the podcast.

It's a pleasure. So you have written another fascinating and remarkably accessible book. You have to stop doing that, Max. I'm trying to stop. This is really, it's a wonderful book, and we will get deep into it. But let's just, kind of the big picture starting point. At one point in the book, you describe the conversation we're about to have about AI as the most important conversation of our time. And I think that, to people who have not been following this very closely in the last 18 months or so, that will seem like a crazy statement. Why do you think of this conversation about our technological future

in these terms? I think there's been so much talk about AI destroying jobs and enabling new weapons, ignoring what I think is the elephant in the room. What will happen once machines outsmart us at all tasks? That's why I wrote this book. So instead of shying away from this question, like most scientists do, I decided to focus my book on it and all its fascinating aspects because I want to enable my readers to join what I, as you said, think is the most important conversation of our time and help ensure that we use this incredibly powerful technology to create an awesome future, not just for tech geeks like myself, who know a lot about it, but for everyone. Yeah, well, so you start the book with a fairly sci-fi description of how the world could look in the near future if one company produces a

superhuman AI and then decides to roll it out surreptitiously. And the possibilities are pretty amazing to consider. I must admit that the details you go into surprised me. We're going to sort of, I guess, kind of follow the structure of your book here and backtrack out and talk about fundamental issues. But do you want to talk about for a moment some of the possibilities here where you just imagine one company coming up with a super intelligent AI and deciding to get as rich and as powerful as possible as quickly as possible and do this sort of under the radar of governments and other companies? Yeah, I decided to indulge and have some fun with the fiction opening to the book, because I feel that the actual fiction out there in the movies tends to get people, first of all, worried about the wrong things entirely. And second, tends to put all the focus on the downside,

and nothing almost on the upside. In my story, therefore, I want to drive home the point, first of all, that there are a lot of wonderful things that can come out of advanced AI. And second, that we should stop obsessing about robots chasing after us, as in so many movies, and realize that robots are an old technology, some hinges and motors and stuff, and it's the intelligence itself that's the big deal here. And the reason that we humans have more power on the planet

than tigers isn't because we have stronger muscles or better robot-style bodies than the tigers, it's because we're smarter. And intelligence can give this great power. And we want to make sure that if there is such power in the future, it gets used wisely. So yeah, so walk us through some of the details. Just imagine a company, let's say it's DeepMind or some company that does not yet exist, that makes this final breakthrough and comes up with a superhuman AI and then decides, what struck me as fairly interesting about your thought experiment is to think about what a company would do if it wanted to capture as much market share, essentially, with this asymmetric advantage of being the first to have a truly universal

superhuman intelligence at its disposal and to essentially try to achieve a winner-take-all outcome, which given how asymmetric the advantage is, it seems fairly plausible. So walk me through some of the details that you present in that thought experiment, like going into journalism first, which was a surprise to me. I mean, it makes total sense when you describe it, but it's not where you would think you would go first if you wanted to conquer the world. the goal to quickly take over the world by outsmarting people has actually gotten a lot easier today than it would have been, say, 500 years ago, because we've already built this entire digital economy where you can do so much purely with your mind without actually having to go places.

You can hire people online. You can buy and sell things online, and they start having huge impact. And the farther into the future, something like this were to happen, I think the easier it's going to be as the online economy grows even more. I saw this cartoon once online, nobody knows you're a dog. And there's this cute little puppy, you know, typing. But certainly online, nobody knows if you're a superhuman computer. this cute little puppy, you know, typing. But certainly online, nobody knows if you're a super human computer. Now, how do you go make a lot of money and get power online? In the movie

Transcendence, for example, they make a killing on the stock market. But if you really want to make a lot of money, and you want to still be in control of your super intelligent AI and not just let it loose. There are a lot of these tricky constraints, right? Because you want to have it make you money, but you at the same time don't want it to cut you out of the loop and take power over you. So the team that does this in the book jumps through all sorts of hoops to manage to pull this off. And producing media has this nice property that the thing that they keep selling is a product which

can be generated using intelligence alone, but it's still easy enough to understand that they can largely check and validate that there's no breakout risk by them pushing all that stuff out. Whereas if they were selling computer games, for example, that ran on computers around the world, it would be very, very easy for the AI to put some malicious code in there so that it could break out. Well, let's talk about this breakout risk, because this is really the first concern of everybody who's been thinking about what has been called the alignment problem

or the control problem. How do we create an AI that is superhuman in its abilities and do that in a context where it is still safe? I mean, once we cross into the end zone and are still trying to assess whether the system we have built is perfectly aligned with our values, how do we keep it from destroying us if it isn't perfectly aligned? And the solution to that problem is to keep it locked in a box. But that's a harder project than it first appears.

And you have many smart people assuming that it's a trivially easy project. I mean, I've got people like Neil deGrasse Tyson on my podcast saying that he's just going to unplug any superhuman AI if it starts misbehaving, or shoot it with a rifle. Now, he's a little tongue-in-cheek there, but he clearly has a picture of the development process here that makes the containment of an AI a very easy problem to solve. And even if that's true at the beginning of the

process, it's by no means obvious that it remains easy in perpetuity. I mean, you have people interacting with the AI that gets built. And at one point, you described several scenarios of breakout, and you point out that even where every other grown-up on Earth died, and now you're basically imprisoned by a population of five-year-olds who you're trying to guide from your jail cell to make a better world. And I'll let you describe it, but take me to the prison planet run by five-year-olds. Yeah, so when you're in that situation, obviously, it's extremely frustrating for you, even if you have only the best intentions for the five-year-olds. You know, you want to teach them how to plant food, but they won't let you outside to show you.

So you have to try to explain, but you can't write down to-do lists for them either, because then first you have to teach them to read, which takes a very, very long time. You also can't show them how to use any power tools because they're afraid to give them to you because they don't understand these tools well enough to be convinced that you can't use them to break out. You would have an incentive, even if your goal is just to help the five-year-olds to first break out and then help them. Now, before we talk more about breakout, though, I think it's worth taking a quick step back

because you talked multiple times now about superhuman intelligence. And I think it's very important to be clear that intelligence is not just something that goes on a one-dimensional scale like an IQ. And if your IQ is above a certain number, you're superhuman. It's very important to distinguish between narrow intelligence and broad intelligence. Intelligence is a word that different people use to mean a whole lot of different things, and they argue about it. In the book, it just takes this very broad definition that intelligence is how good you are at accomplishing complex goals,

which means your intelligence is a spectrum. How good are you at this? How good are you at that? And it's just like in sports, it would make no sense to say that there's a single number, your athletic coefficient, AQ, which determines how good you're going to be winning Olympic medals. And the athlete that has the highest AQ is going to win all the medals. So today what we have is a lot of devices that actually have superhuman intelligence and very narrow tasks. We've had calculators that can multiply numbers better than us for a very long time. We have machines that can play Go better than us and drive better than us, but they still can't beat us at tic-tac-toe unless they're programmed for that.

Whereas we humans have this very broad intelligence. So when I talk about superhuman intelligence with you now, that's really shorthand for what we in Geek Speak call superhuman artificial general intelligence, broad intelligence across the board so that they can do all intellectual tasks better than us. So with that, let me just come back to your question about the breakout. There are two schools of thought for how one should create a beneficial future if we have superintelligence. One is to lock them up and keep them confined, like you mentioned. But there's also a school of thought that says that that's immoral if these machines can also have a subjective

experience and they shouldn't be treated like slaves, and that a better approach is instead to let them be free, but just make sure that their values or goals are aligned with ours. After all, grown-up parents are more intelligent than their one-year-old kids, but that's fine for the kids because the parents have goals that are aligned with what's best for the kids, right? But if you do go the confinement route, after all, this enslaved God scenario, as I call it, yes, it is extremely difficult, as that five-year-old example illustrates. First of all, almost whatever open-ended goal you give your machine it's probably going to have an incentive to try to break out in one way or the other and

when people simply say oh i'll unplug it you know if you're chased by a heat-seeking missile you probably wouldn't say i'm not worried i'll just unplug it we have to let go of this old fashioned idea that intelligence is just something that sits in your laptop, right? Good luck unplugging the internet. And even if you initially, like in my first book scenario, have physical confinement, where you have a machine in a room, you're going to want to communicate with it somehow, right? So that you can get useful information from it to get rich or take power or whatever you want to do. And you're going to need to put some information into it about the world. So it can do smart things for you, which already shows how tricky this is. I'm absolutely not

saying it's impossible. But I think it's fair to say that it's not at all clear that it's easy either. The other one of getting the goals aligned, it's also extremely difficult. First of all, you need to get the machine able to understand your goals. So if you have a future self-driving car and you tell it to take you to the airport as fast as possible, and then you get there covered in vomit, chased by police helicopters. And you're like, this is not what I asked for. And it replies, that is exactly what you asked for. Then you realize how hard it is to get that machine to learn your goals, right? If you tell an Uber driver to take you to the airport as fast as possible, she's going to know that you actually had additional goals

that you didn't explicitly need to say. Because she's a human too, and she understands where you're coming from. But for someone made out of silicon, you have to actually explicitly have it learn all of those other things that we humans care about. So that's hard. And then once it can understand your goals, that doesn't mean it's going to adopt your goals i mean everybody who has kids knows that and uh finally if you get the machine to adopt your goals then how can you ensure that it's going to retain those goals as it gradually gets smarter and smarter through self-improvement most of us grown-ups have pretty different goals from what we had when we were five.

I'm a lot less excited about Legos now, for example. And we don't want a super intelligent AI to just think about this goal of being nice to humans as some little passing fad from its early youth. It seems to me that the second scenario of value alignment does imply the first of keeping the AI successfully boxed, at least for a time, because you have to be sure it's value aligned before you let it out in the world, before you let it out on the internet, for instance,

or create robots that have superhuman intelligence that are functioning autonomously out in the world. Do you see a development path where we don't actually have to solve the boxing problem, at least initially? No, I think you're completely right. Even if your intent is to build a value line AI and let it out, you clearly are going to need to have it boxed up during the development phase when you're just messing around with it. Just like any bio lab that deals with dangerous pathogens is very carefully sealed off. And this highlights the incredibly pathetic state of computer security today. I mean, and I think pretty much everybody who listens to this has at some point experienced the blue screen of death, courtesy of Microsoft Windows or the spinning wheel of doom, courtesy of Apple.

And we need to get away from that to have truly robust machines, if we're ever going to be able to have AI systems that we can trust, that are provably secure. And I feel it's actually quite embarrassing that we're so fl are provably secure. And I feel it's actually quite embarrassing that we're so flippant about this. It's maybe annoying if your computer crashes and you lose one hour of work that you hadn't saved. But it's not as funny anymore if it's your self-driving car that crashed or the control system for your nuclear power plant

or your nuclear weapon system or something like that. And when we start talking about human level AI and boxing systems, you have to have this much higher level of safety mentality where you've really made this a priority the way we aren't doing today. Yeah, you describe in the book various catastrophes that have happened by virtue of software glitches or just bad user interface where, you know, the dot on the screen or the number on the screen is too small for the human user to deal with in real time. And so there have been plane crashes where scores of people have died and patients have been annihilated by having, you know, hundreds of times the radiation dose that they should have gotten in various machines because the software was improperly calibrated or the user had selected the wrong option.

And so we're by no means perfect at this, even when we have a human in the loop. And here we're talking about systems that we're creating that are going to be fundamentally autonomous. And, you know, the idea of having perfect software that has been perfectly debugged before it assumes these massive responsibilities is fairly daunting. I mean, just how do we recover from something like, you know, seeing the stock market go to zero because we didn't understand the AI that we unleashed on the Dow Jones or the financial system generally? I mean, these are not impossible outcomes. Yeah, you raise a very important point there. And just to inject some optimism in this, I do want to emphasize that, first of all, there's

a huge upside also, if one can get this right. Because people are bad at things, yeah. In all of these areas where there were horrible accidents, of course, the technology can save lives and healthcare and transportation and so many other areas. So there's an incentive to do it. And secondly, there are examples in history where we've had really good safety engineering built in from the beginning. For example, when we sent Neil Armstrong, Buzz Aldrin, and Michael Collins to the moon in 1969, they did not die. There were

tons of things that could have gone wrong. But NASA very meticulously tried to predict everything that possibly could go wrong and then take precautions so it didn't happen, right? There wasn't luck that got them there. It was planning. And I think we need to shift into this safety engineering mentality with AI development. Throughout history, it's always been the situation that we could create a better future with technology as long as we won this race

between the growing power of the technology and the growing wisdom with which we managed it. And in the past, we by and large used the strategy of learning from mistakes to stay ahead in the race. We invented fire, oopsie, screwed up a bunch of times, and then we invented the fire extinguisher. We invented cars, oopsie, and invented the seatbelt. But with more powerful technology like nuclear weapons, synthetic biology,

super intelligence, we don't want to learn from mistakes. That's a terrible strategy. We instead want to have a safety engineering mentality where we plan ahead and get things right the first time, because that might be the only time we have. Let's talk about the title of the book. The title is Life 3.0. And what you're bringing in here is really a new definition of life. At least it's a non-biological definition of life. How do you think about life and the three stages you lay out? Yeah, this is my physicist perspective coming through here, being a scientist. Most definitions of life that I found in my son's textbooks, for example, involve all sorts of biospecific stuff,

like it should have cells. But I'm a physicist, and I don't think that there is any secret sauce in cells, or for that matter, even carbon atoms that are required to have something that deserves to be called life. From my perspective, it's all about information processing, really. So I give this much simpler and broader definition of life in the book. It's a process that's able to retain its own complexity and reproduce. Well, biological life meets that definition, but there's no reason why future advanced self-reproducing AI systems shouldn't qualify as well. And if you take that broad point of view of what life is, then it's actually quite fun to just take a big step back and look at the history of life in our cosmos. 13.8 billion years ago, our cosmos was lifeless, just a boring quirk soup. And then gradually, we started getting what I call life 1.0, where both the hardware and the software of the life was evolved through Darwinian evolution.

So for example, if you have a little bacterium swimming around in a Petri dish, it might have some sensors that read off the sugar concentration and some flagella, and a very simple little software algorithm that's running that says that if the sugar concentration in front of me is higher than the back of me, then keep spinning the flagella in the same direction, go to where the sweets are, whereas otherwise reverse direction of that flagella and go somewhere else. That bacterium, even though it's quite successful, it can't learn anything in life. It can only as a, learn over generations through natural selection.

Whereas we humans, I count as life 2.0 in the book, we are still, by and large, stuck with the hardware that's been evolved. But the software we have in our minds is largely learned, and we can reinstall new software modules. Like, if you decide you want to learn French, well, you take some French courses and now you can speak French. If you decide you want to go to law school

and become a lawyer, suddenly now you have that software module installed. And it's this ability to do our own software upgrades, design our software, which has enabled us humans to take control of this planet and become the dominant species and have so much impact. Life 3.0 would be the life that ultimately breaks all its Darwinian shackles by being able to not only design its own software, like we can do a large example, but also swap out its own hardware. Yeah, we can do that

a little bit with humans. So maybe we're life 2.1, we can put in an artificial pacemaker and artificial knee, cochlear implants, stuff like that. But there's nothing we can do right now that would give us suddenly 1000 times more memory or let us think a million times faster. memory or let us think a million times faster. Whereas if you are like the super intelligent computer Prometheus we talked about, there's nothing whatsoever preventing you from doing all of those things. And that's obviously a huge jump. But I think we should talk about some of these fundamental terms here, because this distinction between hardware and software is, I think, confusing for people. And it's certainly not obvious to someone who hasn't thought a lot about this, that the analogy of computer hardware and software actually applies to biological systems or, in our case, the human brain. So I think you need to define

what software is in this case and how it relates to the physical world. What is computation? And how is it that thinking about what atoms do can conserve the facts about what minds do? do can conserve the facts about what minds do? Yeah, these are really important foundational questions you asked. If you just look at a blob of stuff at first, it seems almost nonsensical to ask whether it's intelligent or not. Yet, of course, if you look at your loved one, you would agree that they are intelligent. And in the old days, people by and large assumed that the reason that some blobs of stuff, like brains, were intelligent and other blobs of stuff like watermelons were not, was because there was some sort of non-physical secret sauce in the watermelon that was different. Now, of course, as a physicist, I look at the watermelon and I

look at my wife's head, and in both cases, I see a big blob of quarks of comparable size. It's not even that they're different kinds of quarks. They're both up quarks and down quarks and with some electrons in there. So what makes my wife intelligent compared to the watermelon is not the stuff that's in there. It's the pattern in which it's arranged. And if you start to ask, what does it mean that a blob of stuff can remember, compute, learn, and perceive, experience these sort of properties that we associate with our human minds, right? Then for each one of them, there's a clear physical answer to it.

For something to be a useful memory device, for example, it simply has to have many different stable or long-lived states. Like if you engrave your wife's name in a gold ring, it's still going to be there a year later. If you engrave Anika's name in the surface of a cup of water, it'll be gone within a second. So that's a useless memory device. What about computation? Computation is simply something, a system, when a system is designed in such a way that the laws of physics will make it evolve its memory state from one state that you might call the input into some other state that you might call the output. Our computers today do that with a very particular kind of architecture with integrated circuits and electrons moving around in two dimensions. Our brains do it with a very different architecture with neurons firing and causing other neurons to

fire. But you can prove mathematically that any computation you can do with one of those systems, you can also implement with the other. So the computation sort of takes on a life of its own, phone or a Mac laptop, because all you're aware of is how the information in that program is behaving, not this underlying substrate. which is one of the most intriguing aspects of intelligence is a system where the computation itself can start to change to be better suited to whatever goals have been put into the system. So our brains, we're beginning to gradually understand how the neural network in our head starts to adjust the coupling between the neurons in such a way that the computation actually does is better at surviving on this planet than winning that baseball game or whatever else we're trying to accomplish.

So to come back to your very original question, what's the hardware here and what's the software? I'm calling everything hardware that's made of elementary particles. So basically stuff is the hardware here and what's the software, I'm calling everything hardware that's made of elementary particles. So basically stuff is the hardware, whereas information is made of bits as the basic building block. And the bits reside in the pattern in which the hardware is organized. So for example, if you look at your own body, right,

you feel like you're the same person that you were 20 years ago, but actually, almost all your quarks and electrons have been swapped out. In fact, the water molecules in your water in your body get replaced pretty regularly, right? So why do you still feel like the same guy? It's because the pattern into which your particles are arranged stays the same. That gets copied. It's not the hardware that gets retained. It's the software. It's the patterns.

Same thing if you have life. If you have a bacterium that splits into two bacteria, now there are new atoms there, but they're arranged in exactly the same sort of pattern as the original one was. So it's the pattern that's the life, not the particles. Well, there's two things I'd like to flag there beyond you having compared both of our wives favorably to watermelons. No offense, I love watermelons. No offense, I love watermelons. No one will get in trouble for that. Let's just focus for a second on this concept of substrate independence, because it's, again, it's highly non-intuitive. And in fact, the fact that it's non-intuitive is something that you

make much of in the book in a fairly arresting passage. The idea is that it is the pattern that suffices to make something a computation. This pattern can appear in anything that it can appear in, in principle. So it could appear in a rainstorm or a bowl of oatmeal or anything that could conserve the same pattern. And there is an additional point you made about the universality of computation, that a system that is sufficient to compute information to this degree can be implemented in another substrate that would suffice for the same computations and therefore for the same range of intelligence. This is the basis, as you put it, for why this is so non-obvious to us by virtue of introspection. I mean, because the mind doesn't feel like mere matter on your account

because it is substrate independent. Yeah, I think you summarize it very well there. And it might be helpful to take another example, which is even more familiar. Think of waves for a moment. We physicists love studying waves. And we can figure out all sorts of interesting things about waves from this nerdy equation I teach at MIT called the wave equation. It teaches us that waves attenuate like the inverse square of the distance.

It teaches us exactly how waves bend when they go through doors, how they bounce off of walls, all sorts of other good stuff. Yet we can use this wave equation without even knowing what the wave is a wave in. It doesn't matter if it's helium or oxygen or neon. In fact, people first figured out this wave equation before they even knew that there were atoms for sure. It's quite remarkable. And all the complicated properties of the substance get summarized in just a single number, which is the speed of those waves. Nothing else matters. If you have a wave that's traveling across the ocean, the water molecules actually don't. They mostly just bob up and down, yet the wave moves and takes on a life of its own. So this also shows that, of course, you can't have a wave without a substrate. You can't have a computation or a conscious experience

without it being in something. But the details of the substrate don't really matter. And I think that is the fundamental explanation for what you eloquently expressed there. Namely, why is it that our mind subjectively feels so ethereal and non-physical. It's precisely because the details of the substrate don't really matter very much. If you, as some people hope, can one day upload your mind into a computer perfectly, then it should subjectively feel exactly the same way, even though you don't even have any carbon atoms at all now, and the substrate has been completely swapped out. You've introduced a few fundamental

concepts here. You've talked about computation as a kind of input-output characteristic of physical systems. And we're in a circumstance where it doesn't matter what substrate accomplishes that. And then there's this added concept of the universality of computation. But then you also in the book introduce a notion of universal intelligence. And intelligence, again, as you've defined, is the ability to meet complex goals. What's the word universal doing in the phrase universal intelligence? In physics, we know that everything we see around us...

If you'd like to continue listening to this conversation, you'll need to subscribe at samharris.org. Once you do, you'll get access to all full-length episodes of the Making Sense podcast, along with other subscriber-only content, including bonus episodes and AMAs and the conversations I've been having on the Waking Up app.

The Making Sense podcast is ad-free and relies entirely on listener support. And you can subscribe now at SamHarris.org.