Making Sense with Sam Harris - #151 — Will We Destroy the Future?

Episode Date: March 18, 2019

Sam Harris speaks with Nick Bostrom about the problem of existential risk. They discuss public goods, moral illusions, the asymmetry between happiness and suffering, utilitarianism, "the vulnerable world hypothesis," the history of nuclear deterrence, the possible need for "turnkey totalitarianism," whether we're living in a computer simulation, the Doomsday Argument, the implications of extraterrestrial life, and other topics. If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.

Thank you. of the Making Sense Podcast, you'll need to subscribe at SamHarris.org. There you'll find our private RSS feed to add to your favorite podcatcher, along with other subscriber-only content. We don't run ads on the podcast, and therefore it's made possible entirely through the support of our subscribers. So if you enjoy what we're doing here, please consider becoming one. Welcome to the Making Sense Podcast. This is Sam Harris. Today I am speaking with Nick Bostrom. Nick is someone I've been hoping to get on the podcast for quite some time. He is a Swedish-born philosopher with a background in theoretical physics and computational neuroscience and logic and AI and many other interesting intersecting topics. But officially, he's a professor of

philosophy at Oxford University, where he leads the Future of Humanity Institute. And this organization is a research center which is focused largely on the problem of existential risk. And today we get deep into his views on existential risk by focusing on three of his papers, which I'll describe in our conversation. We talk about what he calls the vulnerable world hypothesis, which gets us into many interesting tangents with respect to the history of nuclear deterrence and the possible need for what he calls turnkey totalitarianism. We talk about whether we're living in a computer simulation. He's the father of the now famous simulation argument. We talk about the doomsday

argument, which is not his, but it's one of these philosophical thought experiments that have convinced many people that we might be living close to the end of human history. We talk about the implications of there being extraterrestrial life out there in the galaxy, and many other topics. But all of it is focused on the question of whether humanity is close to the end of its career here, or near the very beginning. And I hope you'll agree that the difference between those two scenarios is one of the more significant ones we can find. Anyway, I really enjoyed talking to Nick. I find his work fascinating and very consequential, and that's a good combination.

And now I bring you Nick Bostrom. I am here with Nick Bostrom. Nick, thanks for coming on the podcast. Yeah, thanks for having me. So you are fast becoming, or not too fast, it's been years now that I've been aware of your work, but you are becoming one of the most provocative philosophers I can think of. And really, it's wonderful to read your work. And I want to introduce you, but perhaps to begin with, how do you view your work and what you focus on at this point? How do you summarize your interests as a philosopher?

Well, that's always been a challenge for me. Broadly speaking, I'm interested in big picture questions for humanity and figuring out which direction is up and which is down. That is, out of all the things you can be pushing on or pulling on in the world, which ones would actually tend to make things better in expectation? Yeah, and then various kind of sub-questions that come out from that ultimate quest to figure out which direction we should be heading in. Yeah, so when I think about your work, I see a concern that unifies much of it, certainly, with existential risk. And I don't know if this

is a phrase that you have popularized or if it's just derivative of your work, but how do you think of existential risk and why is it so hard for most people to care about? It's amazing to me that this is such an esoteric concern and you really have brought it into prominence. Yeah, I introduced the concept in a paper I wrote back in the early 2000s. The concept being that of a risk either to the survival of Earth originating in talent life or a risk that could permanently and drastically reduce our potential for desirable future developments. So in other words, something that could permanently destroy the future. I mean, even that phrase, I mean, you really have a talent for coming up with phrases

that are arresting and, you know, it's such a simple one. It permanently destroy the future. You know, there are probably more people working in my local McDonald's than are thinking about the prospect of permanently destroying the future. How long have you been focused on this particular problem? And again, why is it, there's something bewildering about trying to export this concern to the rest of culture, even to very, very smart people who claim to be worried about things like climate change. Why is existential risk still such an esoteric concern? Well, it's become less so over the last few years.

There is now actually a community of folk around the rationalist community, the EA community, you know, various academic centers. is an effective altruism yeah and about not not just these but but kind of radiating out from these a number of individuals that that are quite interested in this so i think the comparison to the mcdonald's restaurant would no longer be true now maybe it was true several mcdonald's years ago why it is well i guess you could ask that or you could ask why it's no longer the case. I mean, I don't know that the default should be, if we're looking at academia, that questions receive attention in proportion to their importance. I think that's just kind of a poor model of what

to expect from academic research. So one can ask why it's changed. I mean, on one level, you're asking people to care about the unborn if the time horizon is beyond their lives and the lives of their children, which seems on its face probably harder than caring about distant strangers who are currently alive. And we know we're already pretty bad at that. Is that a major variable here? Sure. It's an extreme case of a public good, right? So generally, in a simple model of the market economy, public goods tend to be undersupplied because the creator of them captures only a small fraction of the benefits. The global public goods are normally seen as the extreme of this. If all of humanity

benefits from some activity or is harmed by some activity, as in maybe the case of global warming or something like that, then the incentives facing the individual producer are just very dissociated from their overall consequences. But with existential risk, it's even more extreme, actually, because it's a transgenerational good in the sense that all future generations are also impacted by our decisions concerning what we do about existential risks. And they are obviously not in a position in any direct way to influence our decisions. They can't reward us if we do things that are good for them. So if one thinks of human beings as selfish, one would expect the good of existential risk reduction to be undersupplied.

Like it could imagine if somehow people could go back in time that future generations would be willing to like spend huge amounts of money to compensate us for our efforts to reduce X risk. But since you can't, that transaction is not possible, then there is this undersupply. So that could be one way of explaining why there's relatively little. And there was something about what you said about it's harder to care.

It's a little strange that caring should be something that requires effort. If one does care about it, it doesn't seem like it should be a straining thing to do. And if one doesn't care, then it's not clear why one should be, what motive one would have for trying to strain to start caring. It's a framing problem in many cases, so that there's a certain set of facts, let's say the reality of human suffering in some distant country that you have never visited, you have no connections there, this information can just be transmitted to you about the reality of the suffering. And it transmitted one way, you find that you don't care, and transmitted another way,

the reality of it and the analogy to your own life or the lives of your children can be made more salient. And so we know that, we know, you know, through the lives of your children can be made more salient. And so we know that we know, you know, through the work of someone like Paul Slovic, we know there are moral illusions here where people can be shown to care more about the fate of one little girl who's delivered to them in the form of a personalized story. And they'll care less about the fate of that same little girl plus her brother, and they'll care less still if you tell them about the little girl, her brother, and the 500,000 other children who are also suffering from a famine. And you just get this diminishment of altruistic impulse

and the amount of money they're willing to give to charity and all the rest, and it goes in the wrong direction. As you scale the problem, people care less. So we know we have some moral bugs in our psychology. Yeah. So the original paper about existential risk made the point that from a certain type of ethical theory, it looks like existential risk reduction is a very important goal. If you have a broadly aggregative consequentialist philosophy, say if you're a utilitarian, and if you work the numbers out, the number of possible future generations, the number of individuals in each of those that can live very happy lives, then you multiply that together. And then it looks like even a very small chance in the probability that we will eventually achieve this would have a very high expected value.

In fact, a higher expected value than any other impacts that we might have in more direct ways on the world here and now. So that reducing existential risk by, you know, one thousandth of one percentage point would be, from this utilitarian perspective, worth more than eliminating world hunger or curing cancer. Now, that, of course, says nothing about the question of whether this kind of utilitarian perspective is correct or is agreeable to us. But it just notes that that does seem to be an implication. I'm definitely a consequentialist of a certain kind, so we don't need to argue that point. But one thing that's interesting here, and this may be playing into it, is that there seems to be a clear asymmetry between how we value suffering and its mitigation, and how we value the mere preemption of well-being or flourishing or positive states.

So that suffering is worse than pleasure or happiness is good. You know, I think if you told most people, here are two scenarios for how you can spend the rest of your day. You can spend it as you were planning to, living within the normal bounds of human experience, or I can give you one hour of the worst possible misery followed by another hour of the deepest possible happiness. Would you like to sample the two extremes on the phenomenological continuum? I think most people would balk at this

because we think, we have a sense that suffering is, on some level, you know, in the limit, is worse than any pleasure or happiness could be. And so we look at the prospect of, let's say, you know, curing cancer and mitigating the suffering from that as being more important ethically than simply not allowing the door to close on future states of creativity and insight and beauty. Yeah, I think one might want to decompose different ways in which that intuition could be produced. So one might just be that for us humans, as we are currently constituted, it's a lot easier to create pain than to create a corresponding degree of pleasure. We might just evolutionarily be such that we have a kind of deeper bottom

than we have a height. It might be possible, if you think about it in biological term, in a short period of time to introduce more damage to damage reproductive fitness more than you could possibly gain within the same amount of time.

So if we are thinking about these vast futures, you want to probably factor that out in that you could re-engineer, say, human hedonic systems or the hedonic systems of whatever inhabitants would exist in this future so that they would have a much larger capacity for upside. Right. And it's not obvious that there would have a much larger capacity for upside. Right. And it's not obvious that there would be an asymmetry there.

Now, you might nevertheless think that even in some sense, equal amounts of pleasure and pain, and it's a little unclear exactly what the metric is here, that there would nevertheless be some more basically ethical reason why one should place a higher priority on removing the negative. A lot of people have intuitions about equality, say, in economic context, where helping the worst off is more important than further promoting the welfare of the best off. Maybe that's the source of some of those intuitions. Actually, there's one other variable here, I think, which is that there is no victim or, you know, beneficiary of the consequence of closing the door to the future. So if you ask

someone, well, what would be wrong with the prospect of everyone dying painlessly in their sleep tonight, and there are no future generations, there's no one to be bereaved by that outcome. There's no one suffering the pain of the loss or the pain of the deaths even. So people are kind of at a loss for the place where the moral injury would land. Yeah, so that is a distinction within utilitarian frameworks, between total utilitarians who think you basically count up all the good and subtract all the bad. And then other views that try to take a more so-called person-affecting perspective, where

what matters is what kind of happens to people, but coming into existence is not necessarily a benefit. And now I would say some kinds of existential catastrophe would have a continuing population of people who would be experiencing the bad. If you might say the world getting locked into some totalitarian, like really dystopian totalitarian regime, maybe there would be people living for a very long time, but just having much less good lives than could have existed.

So in some scenarios of existential catastrophe, there would still be inhabitants there. Yeah, no, I think it's pretty clear that destroying the future could be pretty unpleasant for people who are along for the ride. Now, I'd like just to harken back a few minutes ago, like on the general premise here. So I don't see it so much as a premise, this utilitarian view. I mean, in fact,

I wouldn't really describe myself as a utilitarian, but more just pointing out the consequence. There are various views about how we should reason about ethics, and there might be other things we care about as well, aside from ethics. And rather than directly trying to answer what do we have most reason to do all things considered, you might break it down and say, well, given this particular ethical theory, what do we have most reason to do given this other value or this other goal we might have? And then at the end of the day, you might want to add all of that up again. But insofar as we are trying to reason about our ethical obligations, I have a kind of normative uncertainty over different moral frameworks. And so the way I would try to go about making decisions from a moral point of view would be to think, I have this moral parliament model. It's a kind of metaphor,

but where you try to factor in the viewpoints of a number of different ethical theories, kind of in proportion to the degree to which you assign them probability. It's kind of interesting. When I'm out and about in the world, I usually have to make the case for utilitarianism, or at least you should consider this perspective. People are scope insensitive. the case for utilitarianism, or at least you should consider this perspective. Like people are scope insensitive. You should look at the numbers. If this thing has millions of people and this one only has hundreds of people being affected, clearly that, and yet when I'm back here at the headquarters, as it were, I usually am

the one who has to kind of advocate against utilitarian perspective because so many of my friends are so deeply dyed in the wool utilitarians yeah and so narrowly focused on x-risk mitigation but i feel that i'm always the the odd one out well you know i would love to get into a conversation with you about meta-ethics some other time because i you know i i think your views about the limits of consequentialism would be fascinating to explore. But I have so much I want to talk to you about with respect to X-Risk and a few of your papers that I think, let's just table that for another time. In fact, I don't even think we're going to be able to cover

your book Superintelligence. I mean, maybe if we have a little time at the end, we'll touch it. But I should just want to say that this book was incredibly influential on many of us in arguing the case for there being a potential existential risk with respect to the development of artificial intelligence and artificial general intelligence in particular. And so it's, you know, this is something, the reason why I wouldn't cover this with you for the entirety of this conversation is I've had several conversations on my podcast that have been deeply informed by your view. I mean, I've had Stuart Russell on, I've had Eliezer Yudkowsky on, and basically every

time I talk about AI, I consider what I say to be, you know, fairly derivative of your book, and I often remind people of that. So my audience will be familiar with your views on AI, even if they're not familiar with you. So if we have time, we'll get back to that. But what I really want to talk about are a few of your papers. The first is the vulnerable world hypothesis. Maybe I'll just name the papers here that I hope will cover. Vulnerable world, the second is, are you living in a computer simulation? And the third is, where are they? Which is your analysis of the Fermi problem, asking where is the rest of the intelligent life in the galaxy?

Let's start with the vulnerable world hypothesis. What do you mean by that phrase? Well, the hypothesis is, roughly speaking, that there is some level of technological development at which the world gets destroyed by default, as it were. So then what does it mean to get destroyed by default? I define something I call the semi-anarchic default condition, which is a condition in which there are a wide range of different actors with a wide range of different human recognizable motives.

But then, more importantly, two conditions hold. One is that there is no very reliable way of resolving global coordination problems. And the other is that we don't have a very extremely reliable way of preventing individuals from committing actions that are extremely strongly disapproved of by a great majority of other people. Maybe it's better to come at it through a metaphor. Yeah, the urn. The urn metaphor. So you can kind of think of the history of So you can kind of think of the history of technological discovery as the process of pulling balls out of a giant urn,

the urn of creativity. And we reach in and we get an idea out, and then we reach in and get another out. And we've extracted throughout history a great many of these balls. And the net effect of this has been hugely beneficial, I would say. This is why we now sit in our air-conditioned offices and struggle not to eat too much, rather than to try to get enough to eat in large parts of the world. But what if in this ball, in this area, there is a black ball in there somewhere?

Is there some possible technology that could be such that whichever civilization discovers it, invariably gets destroyed? Just to add a little color here, Nick. So in your paper, you refer to this as the urn of inventions. And we have been, as you say, pulling balls out as quickly as we can get our hands on them. And on some level, the scientific ethos is really just a matter of pulling balls out as fast as you can and making sure that everybody knows about them. We have this norm of transparency in science, and we have pulled out, thus far, only white or gray balls, and the white balls are the ones, or the technologies, or the memes, or the norms, or the social institutions that just have good

consequences, and the gray ones are norms and memes and institutions and, in most cases, technology that has mixed results or that can be used for good or for ill. And nuclear energy is a classic case where we can power our cities with it, but we also produce fantastic amounts of pollution that's difficult to deal with. And in the worst case, we build weapons. So I just want to give a little more context to this analogy. Yeah, and I guess most technologies are in some sense double-edged, but maybe the positive predominate. I think there might be some technologies that are mainly negative if you think of, I don't know, nerve gases or other tools.

But what we haven't so far done is extract a black ball, one that is so harmful that it destroys the civilization that discovers it and um what if there is such a black ball in the urn though i mean we can ask about how likely that is to be the case we can also look at what what is our current strategy with respect to this possibility and it seems to me that currently our strategy with respect to the possibility that the urn might contain a black ball is simply to hope that it doesn't. So we keep extracting balls as fast as we can. We have become quite good at that, but we have no ability to put balls back into the urn. We cannot uninvent our

inventions. So the first part of this paper tries to identify what are the types of ways in which the world could be vulnerable, the types of ways in which there could be some possible blackball technology that we might invent. And the first and most obvious type of way the world could be vulnerable is if there is some technology that greatly empowers individuals to cause sufficiently large quantities of destruction. Motivate this with a, or illustrate it by means of a historical counterfactual. We, in the last century, discovered how to split the atom and release the energy that

is contained within some of the energy that's contained within the nucleus. And it turned out that this is quite difficult to do. You need special materials. You need plutonium or highly enriched uranium. So really only states can do this kind of stuff to produce nuclear weapons. But what if it had turned out that there had been an easier way to release the energy of the atom? What if you could have made a nuclear bomb by baking sand in the microwave oven or something like that?

So then that might well have been the end of human civilization in that it's hard to see how you could have cities, let us say, if anybody who wanted to could destroy millions of people. So maybe we were just lucky. Now we know, of course, that it is physically impossible to create an atomic detonation by baking sand in the microwave, but before you actually did the relevant nuclear physics, how could you possibly have known how it would turn out? Let's just spell out that, because I want to conserve everyone's intuitions as we go on this harrowing ride to your terminus here, because the punchline of this paper is fairly startling when you get to what the remedies are.

So why is it that civilization could not endure the prospect of what you call easy nukes? the prospect of what you call easy nukes. If it were that easy to create a Hiroshima-level blast or beyond, why is it just a foregone conclusion that that would mean the end of cities and perhaps the end of most things we recognize? I think foregone conclusion is maybe a little too strong. It depends a little bit on the exact parameters we plug in. And the intuition is that in a large enough population of people, like amongst every population with millions of people, there will always be a few people who, for whatever reason, would like to kill a million people or more if they could, whether they are just crazy or evil, or they have some

weird ideological doctrine, or they're trying to extort other people or threaten other people. Just humans are very diverse, and in a large enough set of people, for practically any desire, you can specify there will be somebody in there that has that. So if each of those destructively inclined people would be able to cause a sufficient amount of destruction, then everything would get destroyed. Now, if one imagines this actually playing out in history, then to tell whether all of civilization really would get destroyed or some horrible catastrophe short of that would happen instead, would depend on various things. Like just what kind of nuclear weapon, would it be like a small kind of Hiroshima type

of thing or a thermonuclear bomb? How easy would it be? Could literally anybody do it in five minutes? Or would it take some engineer working for half a year? And so depending on exactly what values you pick for those and some other variables, you might get scenarios ranging from very bad to kind of existential catastrophe. But the point is just to illustrate that there historically have been these technological transitions where we have been lucky in that destructive capability we discovered were hard to to wield you know and maybe a plausible way in which this kind of very highly destructive

capability could become easy to wield in the future would be through developments in biotechnology that maybe makes it easy to create designer viruses and so forth right that doesn't don't require high amounts of energy or special difficult materials and so forth and there you might have an even stronger case like so with a nuclear weapon like one nuclear weapon can only destroy one city right where the viruses and stuff potentially can spread so yeah and we should remind people that we're in an environment now where people talk with some degree of flippancy about the prospect of every household one day having something like a desktop printer that can print DNA sequences, right? That everyone becomes their own bespoke molecular biologist, and you can just print your

own medicine at home, or your own genetic intervention at home, and this stuff really is, you know, the recipe under those conditions, the recipe to weaponize the 1918 flu could just be sent to you like a PDF. It's not beyond the bounds of plausible sci-fi that we could be in a condition where it really would be within the power of one nihilistic or otherwise ideological person to destroy the lives of millions and even billions in the wrong case. Yeah, or send us a PDF or you could just download it from the internet. So the full genomes of the number of highly virulent organisms are in the public domain and just ready to download. So yeah, so I mean, we could talk more about that. I think that I would rather see a future where DNA synthesis was a service provided by a few places in the world where it would be able,

if the need arose, to exert some control, some screening, rather than something that every lab needs to have its own separate little machine. Yeah, so these are examples of type one vulnerability, where the problem really arises from individuals becoming too empowered in their ability to create massive amounts of harm. Now, so there are other ways in which the world could be vulnerable that are slightly more subtle, but I think also worth bearing in mind. So these have to do more about the way that technological developments

could change the incentives that different actors face. We can again return to the nuclear history case for an illustration of this. And actually, this is maybe the closest to a black ball we've gotten so far with thermonuclear weapons and the big arms race during the Cold War led to something like 70,000 warheads being on here, trigger alert. So it looks like when we can see some of the archives of this history that have recently opened up, that there were a number of close calls. The world actually came quite close to the brink on several occasions, and we might have been quite lucky to get through.

It might not have been that we were in such a stable situation. It rather might have been that this was a kind of slightly blackballish technology and we just had enough luck to get through. But you could imagine it could have been worse. You could imagine properties of this technology that would have created stronger incentives, say for a first strike, so that you would have crisis instability. If it had been easier, let us say in a first strike to take out all the adversary's nuclear weapons, then it might not have taken a lot in a crisis situation to just have enough fear that you would have to strike first for fear that the adversary otherwise would

do the same to you. Yeah. Remind people that in the aftermath of the Cuban Missile Crisis, Yeah. Remind people that in the aftermath of the Cuban Missile Crisis, the people who were closest to the action felt that the odds of an exchange had been something like a coin toss, something like 30 to 50 percent. And what you're envisioning is a situation where, which you describe as safe first strike, which is, you know, there's just no reasonable fear that you're not going to be able to annihilate your enemy, provided you strike first. That would be a far less stable situation. And it's also forgotten that the status quo of mutually assured destruction was actually a step towards stability. Before the Russians had, or the Soviets had, their own arsenals, there was a greater game theoretic concern that we would be more tempted to use ours because nuclear deterrence wasn't a thing yet.

Yeah, so some degree of stabilizing influence, although of course maybe at the expense of the outcome, maybe more so. If you'd like to continue listening to this conversation, you'll need to subscribe at You're welcome. The Making Sense podcast is ad-free and relies entirely on listener support. And you can subscribe now at SamHarris.org.