Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas - 111 | Nick Bostrom on Anthropic Selection and Living in a Simulation

Episode Date: August 24, 2020

Human civilization is only a few thousand years old (depending on how we count). So if civilization will ultimately last for millions of years, it could be considered surprising that we've found ourselves so early in history. Should we therefore predict that human civilization will probably disappear within a few thousand years? This "Doomsday Argument" shares a family resemblance to ideas used by many professional cosmologists to judge whether a model of the universe is natural or not. Philosopher Nick Bostrom is the world's expert on these kinds of anthropic arguments. We talk through them, leading to the biggest doozy of them all: the idea that our perceived reality might be a computer simulation being run by enormously more powerful beings. Support Mindscape on Patreon. Nick Bostrom received his Ph.D. in philosophy from the London School of Economics. He also has bachelor's degrees in philosophy, mathematics, logic, and artificial intelligence from the University of Gothenburg, an M.A. in philosophy and physics from the University of Stockholm, and an M.Sc. in computational neuroscience from King's College London. He is currently a Professor of Applied Ethics at the University of Oxford, Director of the Oxford Future of Humanity Institute, and Director of the Oxford Martin Programme on the Impacts of Future Technology. He is the author of Anthropic Bias: Selection Effects in Science and Philosophy and Superintelligence: Paths, Dangers, Strategies. Web site Oxford web page Wikipedia Amazon author page Talk on the Simulation Argument

Post-partum, depression, or PPD. Consistently sad. Loss of interest. Numb. Alone. Irritable. Empty. Hopeless.

Anxious. Exhausted. These aren't even all the symptoms. And it's different for every woman. PPD is a real medical condition. And there is treatment available. Trust what you're feeling. Talk to a doctor.

And visit treatppd.com to learn more. Sponsored by Supernus and Biogen. Struggling to see up close, make it visible with Viz. Viz is a once daily prescription eye drop to treat blurry near vision for up to 10 hours. The most common side effects that may be experienced while using Viz include eye irritation, temporary, dimmer, dark vision, headaches, and eye redness. Talk to an eye doctor to learn if Viz is right for you. Learn more at Viz.com. Hello, everyone. Welcome to the Minescape podcast.

I'm your host, John Carroll. And today's guest is someone I had in mind right from the beginning as a wonderful guest for Minescape as soon as I started the podcast. It's taken a while for us to work it out and get it to happen. but I'm very happy to have Nick Bostrom as the guest on today's podcast. Nick, of course, is relatively famous in the public sphere as a philosopher because he's one of the driving forces behind the simulation argument, the idea that it is more likely than not that we human beings and anyone else in our observable universe are actually simulated consciousnesses

part of a program being run on a computer designed and maintained by a far more advanced civilization. But he didn't start there. Nick did not first start with the simulation argument. He got there from his thinking in philosophy. Some of his first work was on the anthropic principle. Cosmologists, of course, know the anthropic principle as trying to figure out the selection effects that we should impose on cosmological parameters, given the fact that we have to live in a part of the universe where human beings can live. But the anthropic principle is not just for cosmologists. There's a famous version of it, or I should say a famous application of, of it called the doomsday argument that goes back to John Leslie, Richard Gott, and other people. The idea is, our technological civilization is not that old, right? I mean, maybe 500 years old, a few thousand years old, depending on what you count as technological civilization.

But the point is, let's imagine you're hoping that our civilization at its technological peak is going to last for millions of years. And then you say, well, you know, the population is only growing. So it's actually extremely unlikely to find ourselves as people who live right at the beginning of our technological civilization. And therefore, people like Leslie and Gott and others have argued, the probable lifespan of our civilization is not that long. It's measured in thousands of years, not in millions of years. So this seems a little bit presumptuous, right? Like how can we decide the future lifetime of our civilization without getting out of our armchairs in some sense?

That's the philosophical problem that people like Nick Mostrum and others have attached their thought processes to, and it leads us to think about what a typical observer is like, and therefore, as we'll get to in the podcast, could typical observers actually be simulated agents rather than biological ones? This is a really fun podcast. I think it's very important stuff. Nick is now at the Future of Life Institute at the University of Oxford, where he also very famously worries about artificial intelligence becoming super. super intelligent and doing bad things to the world. So we'll talk a little bit about that. But mostly today in this conversation, we're about the philosophical underpinnings. We're about how to think about these problems. And I think that the conversation we have will be very helpful to all of us when we do so.

So with that, let's go. Nick Bossroom, welcome to the Mindscape podcast. Thanks for inviting me. So we know that you're very interested in the future of humanity, you are literally the director of a place called the Future of Humanity Institute, which sounds like an awesome responsibility to have. But there's different ways in which one can approach this issue of the future of humanity. You can very specifically say, well, this technological change will have this certain impact,

and I know that you probably do that at the institute. But there's another angle one can take, which is more strictly philosophical. And you come from a philosophy background. There's this angle that we can just use general principles of reasoning and very, very meager data that we have to make grandiose pronouncements about, you know, the possible future development of humanity. And what I'm thinking of are things like the doomsday argument that you didn't originate, but I know you've had a lot to say about. So for those of us in the audience who are not familiar with the doomsday argument, why don't

you explain that a little bit? Yeah, okay. So by context, it falls within the broader category of anthropics reasoning about observation selection effects, which then comes up in different areas. I mean, I know in the foundations of quantum physics and cosmology, these kind of methodological questions become important. But the doomsday argument is one particular application of this style of reasoning, a controversial one. And I, at the end of the day, am rather doubtful about its soundness. But for what it's worth, it goes something like this. Well, So it might be easiest to explain it via an analogy.

So let's consider a simple analogy first. Let's suppose that there are two urns and they are filled with balls. And you know that in one of these two urns, there are 10 balls, numbered 1, 2, 3 up through 10. And in the other urn, there are a million balls numbered 1 through 10. a million. Now, somebody flips a coin and collects one of these urns at random and they place it in front of you there. And they ask, what's the probability that this urn in front of you has 10 balls? So you say, oh, 50% right? It's easy. Now, let's suppose you then read into this earn and you pull out the ball and you look at it. It's number of it. It's number

seven. So then you've got to update, right? So you say it's a lot more likely that I would get ball number seven if there were only 10 balls in the earth than if there were a million balls in the earth. So you can just use base theorem. You conditionalize on this evidence and you get a posterior that overwhelmingly attaches probability to the 10 ball urn hypothesis. So all of that is uncontroversial. That's just elementary probability theory. Now, the doomsday argument is the idea that you should apply similar reasoning to different hypotheses about how long the human species will survive, how many humans that will be in the future. So instead of these two different hypotheses about Earn, now consider two different hypotheses about the total number

of people that will ever have been born. So to keep things simple, let's suppose there are only two hypotheses. Either there will be 200 billion in total or 200 trillion in total. Now, corresponding to this prior probability of 50% of the earned, now we are supposed to use some empirical prior about the different things that could cause humanity to go extinct. So you have some views about the risks of nuclear wars and meteors or what not. And maybe you say, let's suppose there is a 10% chance based on those normal considerations that humanity will go extinct within the next century when there may have been about 200 billion

people. And a 90% chance that maybe will survive for a long time and millions of years will, there will be 200 trillion. And corresponding now to the idea of reaching into the urn and pulling out ball number seven, in the case of the doomsday argument, you're supposed to consider your own birth rank, your own place in the sequence of all humans who have been born. And that is roughly number 100 billion. That's more or less how many humans have come before you. And so that just as in the urn case, we're pulling out such a low number, we're supposed to increase the likelihood that there are only few balls in the earns. So similarly here, you having this finding that you were born so early, it's supposed

to increase the likelihood that the total number of humans will be 200 billion rather than 200 trillion. It would kind of make you more typical. If your number were roughly 100 billion out of 200 billion is no big surprise about that, whereas it would kind of be surprising and extraordinary to think you were in the very earliest small fraction of one, the first point. percent of all humans that will ever have been born. So that's the structure of the doomsday argument. I think the critical part of that is this idea that you should reason as if you were a random sample from the set of all humans that will ever have existed.

That's right. Yeah. If you do accept that, then, you know, the rest follows pretty straightforwardly. But it does seem, I mean, just at glance. And to be very honest, let me put my own cards on the table. I have some opinions about this kind of reasoning, but they're pretty mild. I go back and forth and I keep changing my mind, so I'm interested to see where we go here. But clearly the point of view of the skeptic here is going to say, how in the world can you reach any conclusions about the far future rate of humanity just without leaving your armchair, right?

I mean, this seems like you've sneaked in some innocent seeming assumptions to do a lot of heavy lifting. Indeed, yes. So this was, I think, the universal reaction when the doomsday argument was initially presented. It originated, I think, from a physicist, Brandon Carter, and then was kind of written up by a philosopher John Leslie in the 90s properly and put into the spatial framework. And most people thought, I've got to be something wrong with it. There's no way you could, you know, cut through all the fog up on no ability about different future risks of wars and new technologies and all that and derive this very striking conclusion just from this seemingly very meager piece of evidence.

But then when it came to trying to explain what was wrong with the argument, that this agreement stopped. And where many people were very confident that I knew precisely what was wrong with it, but they all seemed to have a different. explanation. And many of those critique and objections could quite easily be seen to be flawed. If we try to apply the same kind of critique to, in other cases, it would produce absurd results. So it's actually quite tricky to, if the doomsday argument is false, it's non-trivial to say why it's false. Right. Well, maybe you can explain a little bit more, and you can be judgy. You can give your own opinions here about this step that says, I should reason as if I am a typical, randomly selected observer from the history of all humankind. I think that's clearly the place where we should interrogate what's going on to maybe see that we haven't sneaked something in.

Yeah, so you might wonder why on earth would we think that. I mean, it's not as if there were some, I don't know, a time. I'm traveling stork that sort of picked up a human at random and then dropped them into the year you were born. So why would one believe something like that? Well, it seems that a methodological principle similar to that is kind of necessary in order to make a lot of perfectly reasonable science work out. So I mean, in your field of cosmology, these days what I call big world hypotheses are quite widely embraced. This would be hypothesis according to which the world is so big and locally varied that you would expect all possible observations to be made by some observer somewhere. Yeah. Just if not for the other reason just because of sort of local thermal

fluctuation. You have the balsman brains and you have all kinds of local circumstances that just by random accidents would be different. And if you have. an infinite or maybe to bracket different issues that come up when we talk about infinity is just an enormously but still finite large universe you wouldn't have

all these different theories that we might have about cosmology and about the value of different critical constants all of those theories predicting that whatever you predict would in fact be predicted would in fact be observed yeah let me I'll just say

very quickly I think that just so everyone knows, this is an open question in cosmology. I mean, there are absolutely the possibilities on the table, the universe is infinite. There's an infinite number of observers of all different kinds.

And there's a possibility on the table that the universe is finite, and there's not that many observers. We just don't know right now. So it's important to keep up. We do think that by looking through the telescopes or building big accelerators, we can get some useful evidence

that potentially could have some bearing on these questions. And without introducing something that looks a little bit like this typicality assumption. I call it a self-campling assumption, the idea that you should think of yourself as though you were randomly sampled from some reference class. It's very hard to see how those observational predictions would be produced that you could untest.

But if you have something like that, then you could say that the vast majority of October are likely to not, be these weird, rare, diluted freak observers who happen to see some extremely unusual local fluctuation, but most would be typical. And then the theory predict that the theory that matches what we observe would be one that said that what we observe actually reflects the typical conditions throughout the universe. And you can kind of then see how you would be able to connect these big world theories to observation in a way that seems kind of commonsensical.

Alternatively, you could try to argue for this self-sampling assumption by thinking instead of a simpler thought experiment. So just imagine that you start with an empty world. You have a hundred rooms, cubicles. And in each of these hundred cubicles, you create one observer. And then on the outside, you paint maybe 90 of these rooms, blue and 10 red. And now you find yourself having been created in this world, you're told about this whole setup, and you have to guess what color is the room you are in. At some point, you're going to exit the room and you can find out the true answer.

It seems very likely that in this case, you should set your credence to your room being red, equal to the fraction of all the observers that are in red rooms. This would match. If everybody bet this way, you would maximize your expected winnings. And you could consider limiting cases if instead of, say, 90% of the rooms being read, what if 99% were? As you approached the limit, if all the rooms were read, you could just use logic to deduce that your room is red. And it seems that the probability should sort of gradually approach probability one as you move to a situation that is more and more similar to the case where all the rooms are read.

So there are these different both scientific applications that seem to require the self-sumpling assumption and also these thought experiments where it seems intuitively plausible that that's the correct way of reasoning. I guess I'm on board with the second one with the thought experiment red rooms. then the worry is that that's not a good analogy to the real world. I'm not sure I quite got the first one there. I mean, yes, I can imagine cosmological scenarios where there are many, many observers. There seems to be a little bit of an extra leap to say, and I should assume that I'm typical within them or that there's some properties that I'm randomly sampled from.

Could you maybe say a little bit more about this? Are you claiming that there is empirical evidence for this kind of reasoning or just that it is the only logical thing to do? I'm saying that it looks as though one needs this methodological principle in order to get the result that our ordinary practice of trying to test these different theories we have make sense. So let's pick something more specific. So the temperature of the cosmic background at this stage of cosmic evolution,

And what is it, 2.7 Kelvin or something? Yeah, 2.74. Right. So that's what we think. And we think we have evidence for that in that people have measured it and stuff like that. There's various observational data. But now consider a different theory. It says that actually the cosmic background is 3.1 Kelvin.

So we think, okay, well, that's a possible theory. We have strong reason against it. But now suppose that we combine both of these hypotheses about the temperature of the background with the claim that the universe is so big that whichever of these hypothesis is true, there will be some observers who observe either value, because locally that would have been some statistical fluctuation. In a sufficiently large universe, there will be a sort of little bubbled where the temperature just happens to be a bit lower or higher.

And so it looks then that these two claims together imply that there will exist some observers who will see, who will measure a value of 2.7 Kelvin, and there will exist some observers who will measure a value of 3.1. both the hypothesis that in general the average value is 2.7 and the hypothesis that the average value is 3.1 will predict this, that there exists these observers who make either observation. So then the question is, if they both say that both observations are made, what can we conclude from the fact that, well, we've made this observation of 2.7? It seems to be perfectly consistent with both of these different hypotheses. And yet we think intuitively, it obviously favors the hypothesis of 2.5. Yeah. Okay. And so there I'm saying that, well, if you add in the methodological principle that we should think most likely our observation is kind of what an average observer, a typical observer, a random observer would do. Then you get the probabilistic inference to come through. And you can conclude that, yeah, it's possible. The temperature is really 3.1. And we were just this very rare observer who saw something very unusual. But most likely, with overwhelming probability, the average temperature is 2.7, as indeed we observe,

because that's what almost all observers would see if the real temperature were actually 2.7. I guess there's a tiny bit of daylight here between this way of casting it and what I think is the traditional cosmologist's way of casting it. And to be honest, let me be very clear. As a cosmologist, I know that typical cosmologists are not philosophically very sophisticated and they don't think about these things too hard. They just get the right answer and move on with their lives. But there's the way of saying, you're casting it as sort of if the universe were very, very large and there were lots of observers, and we were a typical observer in that ensemble, whereas I think most physicists would just say, let's imagine we're at a typical place in the universe, regardless of how many observers there are, right?

Even if the universe is not that much bigger than what we can observe in our horizon. Do you think that's an important difference, or basically the math works out to be the same in those two justifications? Well, I think there is an important difference in principle. I think as it might for various applications kind of come out to more or less the same result, if you think that observers are sufficiently uniformly distributed. But if you have a different application where you are trying to evaluate different hypotheses, say, about where observers are. Like, if you think about the galaxy and where there would likely be planets conducive for the evolution of intelligent life and so on, then it could start to be important that you

focus not on spatial temporal regions, but actually on counting observers. It could be more likely that you would find yourself within a certain small region of space time. which is dense, say, in planets likely to create intelligent life, than with it's a much larger region of spacetime, where observers would be more scarce. Okay, I mean, I know that to get back to the doomsday argument, because I do, I want to be, I know that philosophers like to, like, list all the possibilities,

but I want to at some point actually get an argument that one is correct or incorrect. There is another counter argument, not just saying we're not typical observers, but there's a counter argument that says, well, if your two scenarios are 200 billion people or 200 trillion people in the history of humanity, because there are more observers in one of those scenarios, I am more likely to be in that universe. I should sort of give that theory a boost in my prior probability

just because there are more observers there. And that sort of cancels out the fact that I am early on. And therefore, even though I'm unlikely to be early in that big universe, that big universe is also more likely, and therefore I can say nothing about which of these two universes I live in. Yeah, that is one of the most important possible responses to the real estate argument. And this initial idea that I explained earlier, the idea that you should think of yourself as a random sample from all the uptabers that exist. in some reference class, I call that the self-sampling assumption. And what you just mentioned is what I would call the self-indication assumption. And it's roughly the idea that the very fact that you find yourself existing,

you have been born into the world, it gives you evidence that a lot of observers probably exist in this world. In some sense, it's as if that would then have been sort of more slots for you to have been born into it. Right. And as you say that, yeah, it does turn out that if you do accept this self-indication assumption, then that exactly cancels out the probability shift that the doomsday argument says we should make in favor of there being fewer.

It's like you first register the fact, oh, I exist. That gives me increased evidence that probably there will be a lot of people existing by the self-indication assumption. Then you notice, oh, I'm really early. That makes it more likely that there will be fewer observer in the future, but those two shifts exactly cancel out. So that has that neat feature. You get rid of the doomsday argument in one fell swoop. And that might in fact be the strongest argument for this indication assumption.

It does have some counterintuitive implications of its own, though. So there is what I call the presumptuous philosopher thought experiment. I love that phrase, by the way. I use that in many, many talks and writings because I really think it captures something that we should be worried about. So please explain what it is. Well, it's the idea that it seems somewhat of an open question at the moment, whether the universe is finite or infinite. or, I mean, if it's finite, then just how big it is.

We know it's very big, but is it like very, very, very big or very, very big? That seems like something you can't just sit and answer in your armchair. You have to actually, you know, build cosmological models and measure the expansion speed and try to evaluate inflationary cosmologies. But if you accept the self-indication assumption, it does. like you would have overwhelmingly strong evidence for concluding that out of two hypotheses, one of which postulate that there are many orders of magnitude and more observer, that that hypothesis is true. So consider one, suppose the universe is finite, but yeah, now,

according to one hypothesis, there are trillion, trillion observers on a, on a, on a rival hypothesis there are a trillion, trillion, trillion observers. And let's suppose the physicists say, well, this is like interesting. The super-duper symmetry considerations show that one of these two possibilities is true. And now we just need to run this experiment and it will definitively tell us which of these is correct. And we just need, I suppose you just need $20,000 to build this very simple machine, like a trivial thing in comparison.

Right. Then the presumptuous philosophers says, no, no, no, no, it's not worth wasting 20. thousand dollars, I can just tell you what the answer is. Of course, it's the trillion, trillion, trillion observer hypothesis that's true. That's a trillion times more likely than the other one by the self-indication assumption comes directly out of the self-indication assumption. And so the objection is that we don't really accept that. It just seems like crazy that, I mean, about as crazy as the idea that you could rule out hypotheses where the human species will last for a very long time just by reflecting on your birth rank.

So here instead, you're ruling out hypotheses where there are slightly fewer, but still a huge number of people in the universe, just by considering the fact that you exist. It also seems a little bit like an overreach. So good. So it does seem like in either case, it just be super duper clear because I know that when I was learning this, this all confused me very quickly.

So the two options on the table are, you know, give theories prior probabilities by how elegant or reasonable they seem, and then within each theory, assume your typical observer. And then the other is assume your typical observer, but boost those theories that have a lot of observers in their prior probabilities because observers like you are more likely to appear there. And you're making the argument that either one of these seems to give us leverage over the world that goes beyond what we should be able to get sitting from our armchair. So what is your recommendation? I mean, how should we think about these questions?

Well, I'm a kind of reluctant to embrace either of these. And I mean, in what you found a lot in the early literature on this, I mean, when people were starting to discuss the dooms argument, is that they would usually just pick one of these alternative or rather usually they would reinvent it, not realizing that other people. And then I would kind of just be either ignore the kind of argument on the other side or be oblivious to it and they would just feel very pleased. So they might invent the self-indication assumption and say, ha, I've refuted a doomsday argument, but then not try to address or reflect on the fact that it on its own have

these other counterintuitive implications. So, I mean, I think it's unclear what we should do. I mean, it certainly seems worth exploring whether they're from third alternative. So you could kind of say that the self-indication assumption says, think of yourself as a random sample from all possible observers, whereas the self-sampling assumptions that think of yourself as a random sample from all actual observers that exist. Yeah. But maybe what you should do is to say, think of yourself as a random sample from all actual observers within some reference class that might not be identical to the set of all observers.

Like maybe you don't need to think of yourself as randomly samples from all observers, including ones that will exist far into the future if humanity survives. But maybe from somewhat narrower class of observers. So for example, if you think these future humans, they would be very different from us. Maybe they will be post-human. They will certainly be in an epistemically very different situation from, I mean, they will know among other things that the human species survived the 21st century. So maybe they will be so different that we shouldn't think of ourselves now as a random sample from some set that includes those.

Maybe they are just too different from us, just as we don't think of ourselves as a random sample from all physical objects that includes rocks and windows and trees. Like if we do that, then I mean, then you would block the doomsday argument. You say, well, we can't rule out these futures where there's going to be a lot of future pervers as long as we say they are different from us in such a way that they fall outside our reference class. You could try to go down a path like that. And I mean, in my doctoral dissertation back in the 90s where I developed a theory of anthropics,

I explore the possibility of whether you could relativize the reference class so that different observers should use different reference classes. And think of themselves as if there were a random sample from some different reference classes, depending on which observer it was and which time it was. And it might then be possible to avoid both the counterintuitive implications like the doomsy argument and the ones like the presumptuous philosophers that come if you accept the self-indication assumption. Well, my own, what I'm tempted to think, and I haven't really completely nailed this down myself yet, is that maybe we're just, it's just a mistake to think of ourselves as typical observers in some class that is much bigger than me. In other words, you know, I know a lot of things about my non-typicality already. Like, most people are not theoretical physicists. You know, there's plenty of obvious ways in which I'm not a typical observer.

And maybe I should judge cosmological scenarios on the likelihood that observers exactly like me should arise, but not go beyond that at all. And therefore, draw no conclusions on the basis of how many alien life forms or post-humans that might exist. So that, I think, is two. narrow. So if we go back to the example of the cosmic background, whether it's 2.7 or 3.1 Kelvin. So on both of these hypotheses, we imagine in a big enough world, there would be some observers who would be seeing 2.7 when they run some measurements. And that would be some that would be seeing 3.1. But if you only include in your reference class, observers who were exactly like you in the same mental state with the same evidence,

then that would only include ones that saw 2.7, since that's what you're seeing, right? Yeah. Yeah. So in that case, it would be true that on both of these different theories, the 2.7 theory and the 3.1, 100% of all the observers in that reference class would be seeing 2.7. Well, right, but I'm suggesting that we can judge theories on the basis of whether or not the likelihood that they predict any observers that would predict, that would see exactly that already. In other words, it's sort of the old evidence versus new evidence issue. Like, I don't want to forget that I already know, I'm an observer who sees the CMB with 2.7 degrees. I can judge theories on the basis of whether there should be people like me in them, but I can't say that those people are typical. Right. So in this case, both of these theories predict that there would be people seeing 2.7, in fact, predicted with probability.

Oh, I'm sorry, yes. Right. So you're comparing, maybe I misunderstood. There's a sort of a small universe where the universe is 2.7 everywhere and a large universe in which the CMB is usually 3.1, but in some places it's 2.7. Is that what we're comparing? Yeah, or compared two large universes where the average temperature in one is 2.7 and the average temperature in the other is 3.1. But both are large enough that there would be pockets of different temperatures. Right, yeah. But both of both big world theories predict that there will be some observers seeing 2.7, some seeing 3.1. What I disagree about is kind of what the average observer or the vast majority of observers will see. That's right. So I think the bullet I would like to bite is, in those cases, where both universes are big enough that even though the averages are different, it is very likely that an observer just like me exists in both of them.

I cannot judge between them. That's what I would actually accept that conclusion. That seems to be the least presumptuous thing I can do. Yeah, except I think the universe probably is like that. It's big enough to think that all of these, I mean, yeah, different finite-sized brains and brain states that humans could occupy are instantiated. And I still believe that we have, that we gain some useful information about the layout of our universe from doing astronomy. Tires matter. They're the only part of your vehicle that touches the road. Tread confidently with new tires from tire rack. Whether you're looking for expert recommendations or know exactly what you want.

want. Tyraq makes it easy. Fast, free shipping, free road hazard protection, convenient installation options, and the best selection of Yokohama tires. Go to Tyraq.com to see their Yokohama test results, tire ratings, and reviews, and be sure to check out all the special offers. Tyraq.com, the way tire buying should be. Yeah, no, I think that's true. Actually, I mean, let me steer the conversation in that direction a little bit, because in the philosophy literature, we have these discussions about the future of humanity and Boltzmann brains and stuff like that. But the down-to-earth working cosmologists, some of them do appeal to anthropic reasoning to explain, for example, the value of the cosmological constant or other parameters that we observe, you know, the fine-tuning of the fine structure

constant that allows for atoms in chemistry and life. Do you think that this kind of reasoning, do you think that the working scientists who use that kind of anthropic reasoning are on the right track? Yeah, well, I mean, you have to look at it application by application, but the general idea that you might initially be puzzled by the apparent fine-tuning of our universe, that there are a number of different parameters and constants that seem to have values that permit intelligent life to exist, obviously, but also to be such that had the value been very different, no observers would have existed. It would just have been a highly diluted hydrogen gas or some other degenerate state. I think it's a right to initially be struck by that. That's like,

huh, that's kind of weird. And then also to recognize that one possible explanation of that is that there is an ensemble of universe with a wide distribution of different parameter values. instantiated within that ensemble and then apply anthropic reasoning to say that this ensemble theory nevertheless predict that what we should observe is this apparently very fine-tuned universe where the fine-tuning is kind of apparent in that the ensemble as a whole ideally would not need to be very fine-tuned and nevertheless all observers would find themselves in the universe that was fine-tuned and that would then constitute an explanation for the fine-tuning that we seem to see.

I think that line of reasoning is basically sound. Do you think that we can go beyond that to actually say, to make a calculation and a prediction? I mean, the thing that Stephen Weinberg, for example, tried to do back in the 80s with the cosmological constant was to say, let's imagine there is a smooth distribution of values of the cosmological constant, and the value affects how many galaxies are produced.

Therefore, let's ask what an average observer might see, and he predicted more or less the right value. Are you on board by the right value, I mean the value that was empirically discovered 10 years later. So do you think that's that kind of valid inference? Yeah. So I think that there is a piece of methodology that it's needed in order to go from some theory about what exists. what galaxies there are out there and what planets and what observer. To go from some kind of objective model of the world to some observational prediction

about what you or I are likely to see. And this is where the anthropics comes in as this little piece of methodology that tries to bridge that gap. In a sense, it's a methodology about how to reason about indexical information. That is information that has to do with where you are, what time it is, who you are, as opposed to the kind of objective structure about what exists. And so that kind of methodology can then be used in combination with different hypotheses

about the objective structures out there to derive different. predictions that we can test. Can we make, sorry. No, go ahead. I mean, can we make, aside from the cosmological constant and cosmological things, can we use this kind of reasoning, in your view, to reason about the existence of intelligent civilizations elsewhere, the Fermi paradox kind of thing?

Yeah, well, so I've always thought that the Fermi paradox is not very paradoxical. we don't see any signs of intelligent life. That's true. But I don't know what, I mean, for there to be a paradox, there has to be some sort of argument for X and then some other argument for not X that both seem persuasive. And then we are left with this conflict that we don't know how to resolve. But here it just seems that there is an argument for one.

We haven't seen any aliens. I'm not sure what the argument would be that we should be surprised about that. I mean, we certainly know that there are a lot of planets. Yeah. But that doesn't mean that was a high likelihood that aliens would result. I mean, because there are a lot of steps between having a planet and having life, let alone intelligent life. And for all we know, those steps might be very improbable or there might be one improbable

step. So maybe just going even to the simplest replicators, maybe that just required an astronomical coincidence. The right amino acids might, like a hundred of them might just have to have bumped into each other in precisely the right way to create something that could get self-replication going. For all we know, in biochemistry, that's perfectly possible that that could be such an improbable step. Now, you might then say, well, it's something. something we should be very reluctant to do to postulate something that improbable because we exist. I was going to say that. That would be extremely improbable, and our existence seems to conflict with that story.

So that's where this anthropics would come in. Well, it would, if there was only one planet and we existed on that planet, that would be evidence against the idea that there were some extremely improbable step in evolutionary history. But if there are gazillions of planets and only the ones where intelligent life are in the end observed, then it might not be so surprising that we find ourselves on a planet where intelligent life observed, even if there are some extremely improbable steps in going from one planet to intelligent life. If there are not planets, it's like enough lottery tickets. It's not surprising you win the lottery, even if every ticket has only a one in a million chance of win.

if you bought 10 million lottery tickets. Right. Okay. Okay. So, I mean, I'm actually very sympathetic to that point of view. I think that people tend to say, well, there's a lot of planets and how small could the probability of life forming be? And, you know, the answer could be, it could be really, really small. Yeah, it could be really small.

The Drake equation must be one of the most overhyped in all because it gives this appearance of some rigorous scientific grasp that you could use to calculate how. I mean, aliens there are, but then there are like some, some parameter values there that we are not just uncertain about, but the value might be one, 100% probability, or it might be 10 to the power of minus 1,000. We just have no clue. But you seem, yeah, and so I completely 100% agree with that, but you do seem more sympathetic to the doomsday argument kind of reasoning than to the presumptuous philosopher that follows from the self-indication. assumption, the boosting that we would give to theories with lots of observers. So is there some combination of doomsday-like arguments and there's no other aliens out there that seem highly technologically advanced empirical observation that lets us conclude,

wow, we're probably doomed. We're probably going to destroy ourselves within a few generations. Well, I mean, it gets complicated. I mean, just to be clear, I mean, I think there are aliens out there. I just think they're, like, far away. So, I mean, if the universe is infinite, then certainly that would be aliens out there. Right. But maybe not within our light cone.

Yeah, I mean, so the doomsy argument is kind of counterintuitive and surprising, but maybe you could sort of persuade yourself to accept it. There is a thought experiment that has the same structure as the doomsy argument, but maybe takes the counterintuitiveness one level up. So consider what I call the Adam and Eve thought experiment. So imagine that the world was created and that there were initially two people, Adam and Eve. And then whether there were going to be additional people, whether there was going to be a whole human race coming into existence at the later time. maybe depends on the choices that Adam and Eve makes.

And so let's say that they have, they might be tempted by carnal desire. They were. We know from the books that they were. So yeah. But let's suppose that they thought, if we do create a whole race of humans, that would be very bad,

that we were not supposed to do that. And so we definitely would not want to create billions of humans. We might know that that would be the inevitable result if Eve has a child as a result of our thing. But we're not sure. I mean, obviously they could carnally embrace. And Eve might not get pregnant, right? But I might think, let's suppose it's one in ten pants that she would bear a child as a result, based on just normal facts about the human reproductive system.

So they say, well, it's not worth risking it. 10% of creating this disaster, we are not going to do it. But now that's supposed there is this snake that slithers up and whispers. Well, okay, so you're either going to be come pregnant or not if you carnally embrace. But really, think about what the chances are. If you do become pregnant, then you would be the first lot of billions and billions of humans. an extraordinarily unusual position in the distribution of all observers. And by the self-sampling assumption, the probability that you would be the first two,

if there were going to be billions of humans, would be what? Well, one in several billion. Yeah. Extremely improbable. So when you conditionalize on the fact that you were among the first two, you can basically disregard that hypothesis. And conversely, if it doesn't get pregnant, you would be completely normal. You would be the third two out of two people, probably one.

So updating on this, then they could become extremely confident that this act would not produce what seems intuitively like they should think would have about the 10% chance of resulting. That seems counter-adjifty. You could go further. Let's suppose that Adam is a bit lazy. He doesn't want to go out hunting. and he thinks it would be really convenient if like a wounded deer just happened to limp by the cave in the next 10 minutes. You know, save him a lot of effort. So then it seems he could form the firm intention that if a wounded deer does not appear, they will produce offspring and that will then go on to create billions of people.

and then suddenly he would then have strong evidence to be fully convinced that a wounded deer would appear. And that also it seems kind of hard to think that that would be the rational thing for him to believe. So I think those are maybe even more counterintuitive consequences of accepting the self-samping assumption with an unrefitted reference class. And therefore we should not accept it? it? Well, it certainly would count against accepting it, right? Yeah. Now, you have to then look at what the alternatives are. If you don't accept it, what do you do then? Do you accept the self-indication assumption? Well, then the presumptuous philosopher, do you instead go down the path maybe of relativizing the reference class,

like I was alluding to earlier? Maybe, although that also, you know, has its own possible problems. and in general I think it's an area where there is still some murkiness and on clarity. I think it goes quite deep these methodological questions and we don't get it might be that one of these answers is correct but I'm not sure we have understood enough yet to be justifiably very confident in any of them. Okay, that makes perfect sense. So in other words, specifically for the doomsday argument, it's something we should think about and maybe be worried about and maybe found institutes to try to avoid doomsday. But it's not like a good idea independently. Yeah.

But you don't think that we understand it well enough to just say yes, that's the correct conclusion. That's right. Okay, very good. But I can't let you get away without, you know, saying we can extend this idea. of self-sampling of typicality among the sense of all observers, you have famously said, well, let's include in the set of all observers that we might be typical among observers that are being simulated by some higher-level intelligence and live their whole lives out in a computer. So that leads us to the simulation argument. Is that right?

Yeah. So why don't you tell us, let's imagine that there are people, it's unlikely, but that there are people in the audience who don't know what the simulation argument is. Well, the simulation argument tries to show that one of three possibilities obtains. And one of those is that there is a very strong convergence where virtually all civilizations that are current stage of technological development go extinct before they reach technological maturity. So that's one possibility, something that could be true. A second possibility is that there's a very strong convergence amongst all technologically mature civilizations in that they all lose interest in creating computer simulations with conscious people and tester simulations,

if you want. And the third possibility is that we are almost certainly living, in a computer simulation. And so the argument involves some simple probability theory and stuff, but the basic idea is very possible to graph just intuitively, which is supposed that the third possibility does not obtain. So at least some civilizations at our current stage eventually reach technological immaturity, even if it's just one in a thousand.

And suppose the second possibility also does not obtain. So at least some reasonable fraction of those who do become technically mature still are interested in using some non-trivial fraction of the resources to creating ancestor simulations. You can then show that there would be many, many more people like us living in simulations than would we have lived in original history, just because if you estimate the amount of compute power that the mature civilization would have and you compare that to estimates of the cost of creating simulations of conscious beings like humans. You just see that even by devoting a tiny fraction of 1% of their compute power for just

like one minute, that they could create thousands and thousands of runs of all of human history. And so if you reject the first two possibilities, you are then forced to conclude that the vast majority of people with our experiences are simulated. And then I claim conditional on that, we could think we are probably one of the simulated ones. The anthropic stuff comes in only in this third last step in going from most people with our kinds of experiences are simulated, to therefore we are probably simulated. Right.

So, and you buy this. This is one that you're willing to stand behind this argument. Yeah. Right. And so because of that, let me just push back on it a little bit. I'm very open-minded about this. I'm agnostic. I really don't know.

Can you say a little bit more about how we know how much compute power it takes to effectively simulate a reasonable consciousness? I mean, I can at least imagine that we understand, once we understand better what that would take, if we understand, you know, the efficiency of the brain, et cetera, we could argue that to truly simulate human consciousness requires almost as many atoms as a brain has, a order of magnitude. And you're clearly assuming it takes a lot less. Yeah, a lot less. So we have to, I guess, first be clear on what the success criterion is for having successfully created one of these ancestor simulations.

So it's not that you would create a simulation that behaved exactly like the original in as much as every microscopic behavior would be captured perfectly, and you could use it as a perfectly reliable predictive model. There might be all kinds of random stochastic events in human brains that via butterfly effects eventually have big implications on our behavior, like maybe a single element. particle moved, you know, one plank length might, might a day later make you say something different than you would otherwise have said. But it's rather that it's close enough that you couldn't tell from the inside, whether you were the simulated one or the non-simulated one.

Right. And so to do that, I think it would be sufficient to capture a human, mental phenomena at a computational level, more or less something that would have maybe neurons and maybe synapses. And then the properties of a synapse might be represented by some reasonably sized vector, like a thousand values to represent the synaps. But certainly nothing that would go down to having to keep track of where every atom is at any given point in time or anything. I think that would be vast overkill. Okay. There are different ways you could come at it as well.

You could look, for example, if you have estimates of the human brains processing power, estimates of, say, our sensory perception, like how high resolution does a screen have to be for us not to detect pixels? and you could, another kind of line of argument for why this could be feasible to do is you could, like creating, so in a sense you would have to not just create the simulated brains, but also some sort of environment for them to experience. And so you might think that would be hard. But then you think even our own humble little three pound.

organic brains manage every night to create a kind of virtual reality simulation that sometimes seem pretty realistic to the person dreaming. And if they can do it, like without training, then presumably a post-human civilization with planetary-sized nanocomputers would be able to do this without breaking a threat. Yeah, no, I'm on board with the environment. I think you can trick people into thinking of their environment is realistic with rather low amounts of sensory input. But it's the brain and the connectome I'm less sure about. I mean, we have 85 billion neurons and they're connected in complicated ways.

And so I guess I'm just a little wary when people, I think people leap a little bit too easily into imagining how easy it would be to simulate a human consciousness. I mean, one of the, I didn't want to bring this up, but one of the other ways, that the argument could fail as if it's just impossible to simulate human consciousness on a computer. I think that we're both on the side that it shouldn't be,

but there are definitely people who would disagree, right? There are, yeah. I mean, so the simulation argument assumes, I call the substrate independence thesis. Yeah.

Which a lot of the people accept, I mean, I think in philosophy of mind and amongst computer scientists and physicists, I think a majority opinion would be that what's necessary for conscious phenomena to arise is not that some specific material is being used like carbon atom, but rather that there is a certain structure of a computation that has to be performed. Yeah.

So, I mean, the paper in which I presented the simulation argument just makes that assumption. Sure. Okay. And then you can look for arguments for that. elsewhere in the literature. Yeah, so I do still worry that simulating consciousness is harder than we think, even though it should be in principle possible.

But the other worry I have is that if I take seriously some version of the self-sampling assumption, I just say some version because I'm unclear on what version it would be, not because it is intrinsically unclear, but isn't there a prediction then that you would make that since it's easier to do low-resolution

simulations than higher resolution ones. Most observers should find themselves living in the lowest possible resolution simulations, the clunkiest versions of reality. Well, there are kind of two sides to the equation. So there is the cost of a simulation. And other things equal, yes, the lower the cost of running a particular simulation, the more of those simulations you'd expect to be run. But the other side is the benefit, like that's like the people creating the simulations might have different reasons for creating them.

And it might be that some of those reasons, maybe the most common reasons, would require something more than the minimal level of resolution. And then you have most observers of our kind living in higher than the minimal level of resolution simulations. Yeah, maybe. I don't know. I mean, I think that when we start doing these simulations, we'll start doing them at pretty low resolution. It becomes a little fuzzy to me once we think of the practicalities of actually doing this. So that's why I am agnostic about it. But you would go so far as to say you think that we probably are in a simulation right now? I tend to punt on that question.

Pregnant pause. If a trick your journalist sometimes how, by just saying nothing for a while, you usually get the subject to kind of say more than they wanted to say. But I'm not falling for it. Well, okay. I mean, it is, but that's fine. I will let you punt on it, but let's, you know, to get onto the record, the idea that we can't punt forever, right? You know, the idea behind these arguments is that they're supposed to. there's supposed to be a right or wrong version of them.

And I'm very happy to sort of punt, you know, provisionally say, well, we just don't know yet, but presumably it's knowable. I mean, how should we get better at this? How should we figure it out? Yeah. Well, I mean, so I believe in the simulation argument in that the disjunction between these three hypotheses. Then the question arises, how should we apportion our credence? between these three alternatives.

And, I mean, more than one of them could be true. Yeah. And I mean, as a first cut, it saves we have quite a lot of uncertainty about these matters in general. So probably each one of them should have some non-trivial amount of probability. Beyond that, though, I mean, that doesn't imply they should each have exactly a third probability. You might kind of think one of them deserves the alliance share. that then becomes a more a less clear-cut issue,

where the original simulation argument is silent, and you need to bring in some additional evidence or arguments or speculations, which I think, yeah, one definitely wants to do, but then one takes like a step beyond the original simulation argument. I guess, let me put it this way. Are there predictions you would make on the basis of the

hypothesis that we live in a simulation? Are there things that we should expect to see about reality if that were true? Well, so if we conditionalize on the simulation hypothesis that we are in simulation, then does that have any observational consequences? Any predictions following from that? And I think, yes, but they are kind of probabilistic in nature. So to start with, I think there are certain possible observations that would be extremely low probability otherwise that at least become conceivable if we are in a simulation. It means it takes something trivial. So if you are in a simulation, you could imagine a window popping up in front of you at some point saying you click your in simulation, click here for more information. That would be extremely strong evidence

for the simulation hypothesis. Yeah. other things like say an afterlife might seem if we are in a naturalistic world not simulated that that would be more of a stretch in a simulation it seems like a perfectly natural thing that may or not depending on just how the simulation is set up and what the simulators have in mind but there would be no impediment to running the same mind repeatedly in different simulations or environments. Another type of implication would be via the simulation argument itself. So if the simulation argument is right that at least one of these three possibilities is true, then if we get evidence that the simulation hypothesis, that the third hypothesis is true,

that might then lower the probability of the other two hypotheses. We've already satisfied the requirement that at least one of them is true. And so the others might still be true, but that would be less reason to believe it. So the probability of those might go up. Then, yeah, you might predict as well that further insight into, I don't know, neuroscience and hardware design and nanotech would tend to not reveal information suggesting that simulations are infeasible to build. That doesn't follow with logical necessity because you could imagine that the simulation would have a different physics than the universe in which the

computer running the simulation. Nevertheless, I think that would be in other things equal. implication in that direction. And a whole host of other maybe more fanciful possibilities as well that just seemed kind of hard to reconcile if we live in basement level physical reality. In a simulation, you could imagine the simulators

acting more or less like gods able to intervene and shape it in ways that might not make sense if you were thinking of the universe as just this blind equation of evolving particles according to some simple differential equations. But if you thought of it as being interactive with that system and then some kind of intelligent, purposeful designer slash whatever role they are playing, that was kind of interacting with

this, then it might make it less unlikely that there would be phenomena that that that they might introduce into the world that that that would otherwise be kind of weird. Well, could we talk to them? Could we attract their attention somehow? You know, I mean, I figure typically when we do simulations of large-scale structure in the universe, we don't necessarily pay attention to what every single particle

in the mesh is doing. And likewise, they probably don't pay attention to every single planet among the billions of billions in our observable universe. So would it be wise or possible to, you know, raise our hand a little bit and say, hey, simulators, we've reached self-consciousness. Why don't you say hi? Yeah.

I mean, the cost of keeping track of what we are doing would be, I think, small compared to the cost of running the simulation in the first place. So I think in a wide range of scenarios, they would be easily able to monitor and see whatever the most significant things that were coming out of human civil, or possibly ever. that I might keep track of every thought. Then it depends on the purpose of this, right? That like whether this information would be relevant for what they were doing or which information. Well, it's good to think about it.

I mean, I like whether or not I believe it, I do think that it is an option we should keep on the table and maybe, I don't know if it affects how we act in the world. But, you know, it definitely is something that, you know, cosmological, thinkers should have as one of their things on the table. And it leads to, I think, you know, the last topic I want to get to just very quickly is you've been talking recently about artificial intelligence. And certainly I would imagine that if you grant that we could in principle simulate pretty convincingly human or human-like intelligence, then why not have things that are similar but different? You know, completely artificial intelligence is maybe ones that are much smarter than us. And so you wrote the provocatively titled book, Super Intelligence.

You know, what should be, we've had some talk on the podcast already with people like Stuart Russell and Max Tagmark, but, you know, what should the person on the street keep in mind about the prospects for truly super intelligent AI? Yeah, I mean, I think it's coming unless we sort of managed to destroy ourselves before by some other means, which unfortunately cannot be completely rule out. But since the book came out, I think progress in AI has been quite impressive and things seems to be coming together. And so what the book really tries to do, though, is not so much describe the current state

of place play in AI or predict the timelines. I mean, there's like one or two chapters in the beginning, but the book really then focused on the dynamics that would arise if and when you do attain something comparable to human level of general intelligence in machines. And I argue that you would probably then fairly shortly thereafter have superintelligence, things that radically outperform us across all cognitive. domains. And that if you think about the implications of having machine superintelligence, it really would be the last inventions we'd ever need to make in as much as the super intelligence would then be much better at inventing. And you would get the kind of telescoping

of the future. So all of those science fiction like technologies that maybe humanity would produce if we had 20,000 years of working on the problem, maybe we will have perfectly realistic virtual realities, have colonies on Mars, and we'd have cures for aging and all kinds of other things that these are consistent with the loss of physics, just very hard to do. But if you had this science and technology being developed at super intelligent time scales, then all of those things might happen quickly. So this, what otherwise might seem like a far future could happen quite shortly after you have superintelligence. And all in all, And this seems to make this maybe the most important transition in all of human history.

Such that if you think that there's even some reasonable probability, this may happen in our lifetime, let's say, then that should make it a very high priority to better understand it. And in particular, to see whether there are things we could do to increase the chances that things will go well in this transition to the machine intelligence era. And so a lot of the work that I've been doing and other researchers here at the Institute have focused on AI, both working on some technical issues related to AI alignment, how to design algorithms that would make it possible to get arbitrarily smart systems to actually understand human goals and values and play some beneficial role. also at the governance level, thinking about how the world might then, assuming we solve the alignment problem, increase the chances that this powerful technology is used for some beneficial purpose, rather than to wage war against one another or press one another.

And a bunch of ethical questions as well that arise when we are starting to think about the possibility of advanced digital minds. We had an interesting conversation on the podcast with John Danaher about automation and what it means for employment going forward. And his argument was that robots will basically take over essentially all of our jobs and that's a good thing. Do you think that super intelligent AI will help us solve the problem of needing to work for a living? Is there sort of an economic and social structure angle to this kind of stuff? Yeah, yeah. I mean, that that that would happen, certainly for all their jobs that are kind of functionally defined. So, I mean, if you think about the job of being an Olympic athlete or something like that, so you're using Bolt and you can run past it and anybody else and you get paid for doing that. it doesn't directly follow that he would lose his job, even if we could create like a bipedal machine that can run faster because in a sense, the task is intrinsically defined in terms of a human doing certain things.

We might just have a basic preference for certain things being done by human. you know, just as some people pay extra for some products to be made by hand rather than by machine or be made by indigenous people or some favored group. But setting aside those types of jobs than the others, yeah, I think once you have artificial general intelligence at the broadly human level, you could automate a whole bunch of them and the rest when you also sort out the the robotics part of it which I think yeah you'd

you'd be able to do most of the things that humans need their bodies to do find motoric control and stuff with using I think the AI part is kind of the key part in robotics as well and you could get by with a fairly clunky robotic body

if you had enough intelligence to operate it well yeah I mean okay There's clearly many, many things we could talk about here. But let me just have a final question, fairly open-ended, so you can choose what direction you want to go in. But you are the director of the Future of Humanity Institute. So I have to ask you about what you actually would predict for the future of humanity. What I would like to say is, what do you think things are going to be like 50 years from now?

But I'll let you choose whatever time scale you're most comfortable talking about. Yeah. So we often try to, I think, separate out the time scale questions from the other questions. So some of the things one might be thinking about with respect to the future can kind of be explored without making specific assumptions about when they will happen. You could, for example, say if and when, you might say that there's this set of technology. capabilities, they look like they are physically possible. There is some trajectory that eventually will be there. We are not sure how long it will take,

but we might be able to say that if and when we do get those capabilities, here is a bunch of things that would enable us to do. And one might then think about how that would play out strategically, how it would interact with other things. Yeah, okay. So I find it like if you take some intermediate timescale, like a few decades. I find it very hard to predict what the world would be like then, because there are certain radically transformative technological developments that I think will happen.

I don't think that will happen next year. I do think that will happen within the next 100 years, but if you're asking me in 20 years, I'm kind of, that's very unsure, whether, say, the AI revolution will have happened by then or not. So I would be then in an epistemic superposition regarding what the world would look like that. But that doesn't mean I have no opinions. It just means that they kind of bifurcate into two broad classes of scenarios. Well, I think that's fair. I mean, I think let's put the timescale aside.

I mean, are there, what are the sort of conceptual changes you would most want to have people be appreciative of when they think about what the future will bring? Well, I guess the most important might be the meta-level one, that it is a topic. about which it's possible to think hard and do better or worse. Like I think traditionally the future has been more... It's been a free-for-all. It's like you sort of feel... I think people feel they can just make stuff up. You can relax.

It's like the land of fantasy and fancy. And it's almost like a projection screen where you can display our hopes and fears. and tell some morality tale, but actually trying to get it right, I think that just hasn't been the driver of much thinking up until more recently. And now, and over the last, you know, a couple of decades, there's actually started to be developed a set of concepts and that enable us to start structure our thinking about the future,

I think, in a much deeper and more incisive way. And we don't have all the pieces of, but we have some important pieces. We've actually covered a few of them in our conversation. I think the simulation argument is one of these pieces, a clue. It doesn't tell us exactly what would happen, but it narrows down the range of possibilities to three. I mean, if you accept it, right? There is a doomsd argument, which may or may not be sound, but if it were sound, certainly that would be an important clue.

We've talked about machine intelligence. if you think that that is going to happen, then that looks like it's a pivot point. And you could then broadly divide humanity's future into pre-imposed superintelligence. And most of what happens pre-superintelligence might mainly be important insofar as it impacts how this transition to the machine intelligence era goes. That certainly I'm oversimplifying here, but if you accepted something even vaguely like that, it would radically simplify the task of thinking about the future. Because now, instead of almost anything possibly being really important and relevant to think about,

now it's a much smaller set of developments that really could be pivotal in this sense. The concept of an existential risk, I think is another one of these that helps us, it's like a lens to sort of see certain structural elements of the human condition on its future. you know, questions about whether there are extraterrestals and stuff, like, they could be relevant as well. And so we've already covered a few of these. And there's a bunch of other concepts and ideas and arguments like that, that together makes it now the case almost, I think, that the hard thing is to conceive of even one coherent future that satisfy all of these constraints or one strategic picture that tells us what we should. should do that meets all of these criteria. So it's not as if there's this space where you could just make anything up and the difficult thing is finding some way to choose between them. Now it's

more that there are all so many constraints that it's hard even to figure out one thing that kind of fits them all, which I think is a big change compared to, say, futurism in, I don't know, in the 70s and 80s. But I think that the implicit message in what you're saying is that to best prepare for the future, people should listen to the Minescape podcast, because we We've talked about many of these issues. A good start, yes. It's a very good start. All right.

Nick Bostrom, thanks so much for being on the podcast. Thank you. Thank you, Sean.