Into the Impossible With Brian Keating - Are We Living in a Simulation? Nick Bostrom (2022)

Episode Date: September 17, 2024

What if everything you know is just a simulation?  In 2022, I was joined by the one and only Nick Bostrom to discuss the simulation hypothesis and the prospects of superintelligence.  Nick is a Professor at Oxford University, where he heads the Future of Humanity Institute as its founding director. He is the most-cited professional philosopher in the world aged 50 or under and is the author of some 200 publications, including Anthropic Bias (2002), Global Catastrophic Risks (2008), Human Enhancement (2009), and Superintelligence: Paths, Dangers, Strategies (2014), a New York Times bestseller. With a background in theoretical physics, computational neuroscience, logic, and artificial intelligence, there is no one better to answer this question than him! Tune in.  — Key Takeaways:  00:00:00 Intro 00:00:44 Judging a book by its cover 00:05:22 How could an AI have emotions and be creative? 00:08:22 How could a computing device / AI feel pain? 00:13:09 The Turing test  00:20:02 The simulation hypothesis  00:22:27 Is there a "Drake Equation" for the simulation hypothesis? 00:27:16 Penroses' orchestrated objective reality  00:34:11 SETI and the prospect of extraterrestrial life  00:49:20 Are computers really getting "smarter"? 00:53:59 Audience questions 01:01:09 Outro — Additional resources:  📚 Superintelligence by Nick Bostrom: https://a.co/d/fYC5DH0  — ➡️ Follow me on your fav platforms: ✖️ Twitter: https://twitter.com/DrBrianKeating  🔔 YouTube: https://www.youtube.com/DrBrianKeating?sub_confirmation=1  📝 Join my mailing list: https://briankeating.com/list  ✍️ Check out my blog: https://briankeating.com/cosmic-musings/  🎙️ Follow my podcast: https://briankeating.com/podcast  — Into the Impossible with Brian Keating is a podcast dedicated to all those who want to explore the universe within and beyond the known. Make sure to follow so you never miss an episode! Learn more about your ad choices. Visit megaphone.fm/adchoices

Number one, possibility is amongst all the civilizations at our current stage of technological development, almost none attain technological maturity. The second possibility is there is this strong convergence among all civilizations that do become technologically mature. They all just lose interest in using those immense technological powers for the purpose of creating large numbers of detailed simulations with conscious digital beings in them. And then the third possibility is we are living in a computer simulation. Any sufficiently advanced technology is indistinguishable from magic.

Open the pod bay doors. Welcome everybody to a very exciting, super intelligent conversation with none other than possibly the most requested guests that I have not yet had on in any form. And that's Professor Nick Bostrom, joining us from Oxford. How are you today, Nick? A pleasure. That's really great. to meet you virtually and we'd be together if there weren't, you know, existential risks

threatening the world. But I want to begin by doing what you're never supposed to do, which you're always told not to do, which is to judge a book by its cover. And this is a segment that's very popular on this podcast. Because, you know, really, what else do you have to go on? If you see a book, you only can look at its cover. Maybe you know the author. Maybe you know somebody that endorsed it. So I want to talk about your New York Times, number one, best-selling book called Super Intelligence. And I want to ask you, how did you come up with the title, the subtitle, and the very, very evocative and interesting cover design, at least here in the U.S., which is a very wise and observant bird known as the owl. So, Nick, how did you come up with the title

and the cover design for this wonderful book? Well, I mean, the title is pretty obvious. The whole book is about superintelligence, and so I keep it simple. There is a sub-tileged. There is a sub-trial. path, dangerous strategies, since it's a book with academic pretensions, I feel it needed the double-barreled approach. It also fleshes it out a little. The cover was a bit of a struggle, actually, with a publisher to get it the way I wanted. The owl picture itself is from some artwork that we got permissions for only, apparently for the English-speaking version, which was a mistake because there are all these translations with various mangled owl-like creatures on the cover.

So that's a lesson for next time. Make sure you get the rights worldwide. And then I added these little, there are some snowy trees in the background, and I think you can actually see some computer code in there as well in the trees that's a little, Easter egg in that code. It's probably going to remain an uncracked mystery. And some red squares, I think, right? You can see the... Like a Conway life game.

Yeah, you can read it many ways. Pixels in the color of blood or, yeah, Conway or it's open for interpretation. Very good. So, yes, I always find it interesting, although my audience gets mad at me because half the time I ask an author, you know, how'd you come up with the title? They say, oh, is the publisher's idea, including Sabina Hassenfelder, who I just had on recently. And she has a new book about existential physics, including some aspects of your work that she refers to. And not with the greatest of enthusiasm. Unfortunately, she is very convergingly.

We love her around here. but she's negative about some of these prospects. We'll get into the topics of it, of course. But I wanted to begin because that book has a very powerful and long-lived impact, at least on culture and so forth. But I want to, instead of delving into some of the stuff you're most renowned for in many ways, and I should state, actually, I'm going to have my computer do it. Computer, who is Nick Bostrom?

According to Wikipedia, Nick Bostrom is a Swedish-born philosopher at the University of Oxford known for his work on existential risk. Computer. Stop. So I have a computer here, a computer assistant, which can do many, many things, including computer, turn on the plug. Now, this turned on the power supply that is in parallel run to power my computing device. So the question is, will the computer commit suicide if I pull, ask it to terminate itself?

What do you think will happen, Nick? If I ask the computer to turn on very power supply. Does it have an internal battery? Like, is it a laptop? No. Well, I guess there's only one way to find out. Computer, pull the plug on yourself. Plug doesn't support that.

Ah, all right, now it's right. Computer, open the pod bay door. I'm sorry, Dave. I'm afraid I can't do that. Oh, my God. Also, I'm not how, and we're not in space. But, Nick, one of your most fascinating aspects of your bio, I'm going to read that you got your Ph.D. at the London School of Economics. Your dissertation was selected by the late Robert Nozick, a towering figure in economics and many different academic scholarly realms. You won the outstanding dissertation. You're included in his book on outstanding dissertation. But I want to go back to your undergraduate days because at King's College, you studied astrophysics and general relativity.

And in doing so, I want to sort of harken back to your undergraduate days. And that is to bring up this guy. So this is a guy, it's pretty famous guy. His name's Albert Einstein. And Albert said, and I quote, he said, the happiest thought of his life. Do you actually, do you know what the happiest thought of Albert Einstein's life was, Nick? I think I might have heard it.

I don't remember it, though. fill in. It was that a person in free fall would experience no gravitational force field. So he would not experience gravitational fields. And he called that the happiest thought of his life. We hear a lot of talk about AI doing physics, your fellow Swede, Max Tagmark, talks about the mathematical universe and the dangers of what he calls Life 3.0. I'm less nervous about the impact of AI because I would like to understand, how could a computer have a happy thought, first of all?

And second of all, how could it do things like Max hopes it will do, create new laws of physics, etc., but how could it do it by visualizing its own behavior as a body in free fall? Aren't these things preventing AI or computers from doing anything remotely like what Albert did 100 years ago or what we might do today? I mean, the lack of a body? Yeah, sensation and also happiness and optimization in terms of some internal driven metric. Insofar as a body were necessary, I guess one could just give it a body like a robot or something, and then that problem would be solved. Of course, we don't know exactly how it would manage to have such a happy thought as Einstein had, But then again, we don't know exactly how Einstein managed to have that either.

But whatever the computational mechanisms are that allow our brain to achieve general intelligence and occasional sparks of indignity, I think similar algorithms would work if implemented on a biological substrate. So instantiating that. And similarly, could you instantiate feelings of pain as a corrective mechanism, a a way to, you know, what humans react to, stimuli both positive as Einstein did in that example, but also negatively. I mean, could you imagine a situation where, you know, thinking, you know, computing device

could come up with some sort of, you know, physical analog to pain out of blow a capacitor here or there? Stop, thanks. I'm not great. Would that be useful, though, is my question. Not could we do it. We could give it, you know, some body or something. But would that be useful? In other words, what is the role of pain and pleasure in the human creative process?

Well, certainly the functional analog of pain and pleasure are used in reinforcement learning. You give positive or negative rewards that then increases or decreases the propensity to repeat the same behavior that led to the reward. So that's one of the main paradigms in the current machine learning. Now, when we speak of human pain and pleasure, we tend to add to that functional role, also the idea that there is a phenomenal experience, that there is a sort of subjective awareness that then has moral significance in the case of suffering, something that it is important to try to reduce in the world. I think that current AI systems probably lack that, but that there is no principled important. possibility in having digital minds also possessing these phenomenal experiences. And I think we will create systems that have that. Of course, there are different theories about exactly what the extra ingredients are

that would be required for a computer to be conscious. But according to all the main contenders, if you have the global workspace theory, you have attention schema theory, you have higher order of thought. theory, for example. On each of those, it would be possible for a suitably sophisticated digital minds also to satisfy those criteria and therefore to have phenomenal experiences. So another thing that comes up quite frequently and just I'm a lay person in this domain, obviously, but we hear a lot about computers dominating humans at games like chess and go. And in addition to the very useful things,

that they can do, like, you know, compute all possible foldings of proteins and laundry in my kid's bedroom or something. But my question is not can they beat us at these games, but can they invent a game that is, that, you know, could stand the test of time the way that chess, chess is 2,400 years old at the minimum, maybe even twice as old. There are computers now you can buy, you know, a layperson can buy, and computer playing programs, not even deep mind. So to what extent do we really?

have much to fear. We invented these domains, and they're playing in it and they're doing well, they're beating us. Is that so surprising? I mean, my kid is better than me at basketball. Should I fear that? Or should we really start to worry when a computer invents a game like chess? That would seem to be a watershed moment rather than just beating us. Well, there might be several watershed moments. It wasn't that long ago since we didn't have computers at all, right? I mean, it's less than 100 years. And so we've come a long way since then, and then for a long time, they couldn't beat humans

in chess, and then they could. And then in Goa, now they can fold proteins, and they can do a whole bunch of other things. And at some point, I will be able to invent new games as well. Maybe initially within some confined parameters, like you could have procedurally generated content, like some computer games have that kind of can design new levels, say. and then maybe the span of their creativity will increase from that point on. I don't see it as a kind of in-principle stumbling block. I think if you're talking about the full range of human ability to invent new paradigms and such,

that might be AI complete, meaning that by the time AIs have that capability, they will also have the ability to do all the other things that we should. humans can do so. And at that point, you might already be in the singularity or in the transition to superintelligence. Right. So when I think about superintelligence and your wonderful book, in the book, which we should eventually make a detour where you define all these terms, because it would be like when I had David Chalmers on and I asked him, you know, to give me some forbearance and define the hard problem of consciousness. I said it's like asking the best band in his country, ACDC, not to play back in black.

And for me not to ask you to define things like superintelligence or the simulation hypothesis would be like me, you know, not asking Abba to play Dancing Queen. So we're going to have to do that at some point. But before we do that, you mentioned, yeah, computers are only 100, I mean, digital modern computers are about 100 years old. What's coming up on its 75th anniversary is the so-called Turing test, which Alan Turing gave the name, the imitation game. And in 1950, when he wrote that seminal paper, he talked about the advent of digital computers making predictions by the year 2000. They would pass the imitation game. Now, I'm not up to speed on many things in this field, but to my knowledge, there isn't anyone that's fully or any machine digital computer, at least, that's passed. the Turing test, is that right?

Yeah, so there are different versions of the Turing test. And it turns out it matters quite a lot exactly how you define it for how hard it is. So you could have some very rudimentary version of it, where some random person that doesn't know much about computers, just does something for a few minutes, and it's quite easy to trick such a person. You just, you program by hand something with a few canned answers. So that has happened a long time ago. But if you have the full-blown version where you have experts interrogating the system for two hours,

then that's something that milestone hasn't yet been passed. And indeed, that might be another example of a Turing complete task because you really would need to be able to sort of exhibit science of general intelligence across all domains in order to pass such a test. Now, in your book, Super Intelligence, you make a claim that by 2100, so 100 years exactly after Turing thought the Turing test would be passed, that we would have superintelligence. So, as I said, I can't resist. Probably before. I mean, it might well be long before then. I think there's a lot of uncertainty about time scales.

Right. And that we should therefore think in terms of a smeared-out probability distribution over many arrival dates. But I mean, progress has been, I think, faster than expected since the book came out in 2014. What surprised you about the progress or that pace of progress? In other words, why didn't you make it 2022 or something? What's been the progress that's most impressed you? Well, I mean, I didn't predict 2100. And I just reported the, like, a survey of AI experts back then and some other sources,

like, and then indicated that there was a wide distribution and that there was some non-trivial. I think the only main claim regarding timelines in the book release wasn't much about timelines, but was that that was like a significant and non-trivial probability that it would happen within the lifetime of many people alive today. when the book was written. Since 2014, we basically had the whole deep learning revolution where a number of things just seemed to be clicking into place. And the same small set of techniques have turned out to first crack computer vision.

And then AlphaGo was a big milestone where you combine deep learning with reinforcement learning. And then we have continued to see a parade of dramatic advances. Most recently, these large language models that are producing text output and now also visual output that is really quite impressive. Yeah. Yeah, I've been, you're referring, of course, to Dolly 2 and some of the GPT3 engines. I wonder, you know, so last year I had the honor to make the first ever audio book of Galileo's, any of Galileo's books.

I did it with Carlo Rovelli and Frank Wilczek and Fabio La Giannati and many other physicists. And it was so exciting to me because I had to get the actual text of this trialogue between these three characters that, you know, set the stage for the scientific revolution, you know, the 17th century. And in this book, I have the actual words. And so at one point, my thought was to just dump all those words into, you know, GPT3 and get this creature that I would call Gallo AIO. But I wonder if that's too simple-minded. In other words, if I just have all this text and it's reflective of his personality, and he was actually one of the best writers in human history, according to Einstein and others. But, you know, is that, is this the deep part of what you just talked about? Is that sufficient or is it merely necessary?

If I just took every word that Einstein thought, could I come up or wrote down or maybe even spoke? Could I ever get approximation of what he'd be like? And if not, maybe why not? So whether from, say, the text or a lot of normal human, you could, in theory, infer what the mind must have been like that wrote those words? Yeah, I mean, I suspect, yeah. Yeah, it is. I mean, we don't know. My guess would be that if you were a superitalus,

you could get pretty amazingly far in that direction from relatively few signs, like, yeah, the corpus of a text, combined maybe by just observing in general the distribution of human minds and knowing how brains work, and then kind of the specific information

you had about the person was maybe just their text out but I think you could probably interpolate some pretty good idea of what the generating mind behind that would have been like you said this place was steps from the water we just haven't found the steps yet how much did we save

enough enough to get lost Or you could book a stay with Hilton. Welcome to your oceanfront room. Just steps from the water. The Hilton sale is on now. Book on Hilton.com or the Hilton app and save up to 20% to get the stay you expected.

When you want savings, not surprises. It matters where you stay. Hilton for the stay. Yeah, so that seems to me to be one of the, maybe it's, yeah, this notion of Turing completeness that would maybe not be met, but it would still be a useful. I mean,

something that's not Turing complete is still pretty useful, right? I mean, I have these computer programs that I put my kids in front of, you know, and they'll ask it, how was your day?

And how do you feel about that? And, you know, why is your dad such a bad dad? You know, it'll do all these things that entertainment. It doesn't mean, in other words, you have to solve the,

you know, the Turing, the imitation game in order to have something of great use, obviously. So I want to turn, you know, to this,

to this notion of, you know, the simulation hypothesis, which you're intimately connected with. And as I said, you know, not asking you to define it would be like not asking Abba to sing dancing queen. So I wonder if you could, in your 30-second manner that I've heard you do it in the past, if you would please indulge me with some forbearance and repeat what you mean by that. Well, 30 seconds will be challenging. But yeah, the simulation argument tries to show that at least one of three possible

actually obtain, although it doesn't tell us which one. So what are those three? Well, number one possibility is amongst all the civilizations at our current stage of technological development, almost none attain technological maturity. So they all go extinct before they develop the full range of technologies that we know are physically possible. Second possibility is there is this strong convergence among all civilizations. that do become technologically mature, they all just lose interest in using those immense

technological powers for the purpose of creating large numbers of ancestor simulations, detailed simulations with conscious digital beings in them. So those are two, right? And then the third possibility is we are living in a computer simulation. The simulation argument kind of demonstrates that at least one, is true by assuming that the first two are false and then seeing that if those first two possibilities did not obtain, we would have strong reason for believing in the third. So that then suggests you can't reject all three, and so at least one of them has to hold.

And it's safe to say that there's been tremendous interest and attention given to it. And when I talked to David Chalmers about his most recent book on Reality Plus, I believe, he called it, a wonderful book. I'll have a link to it in the links above. That, you know, he basically suggests the likelihood is extremely high. In other words, it's not just that there are these three possibilities. They all have equal weight. There's actually, in his mind, a great deal of evidence for it.

And I started to ask him, and I don't know if I fully understood his answer, but in some sense, we in astronomy have, you know, the Drake equation for talking about similar things. the possibility there could be aliens. We could have been aliens. We'll get to aliens later, by the way. But, you know, we could have been seated here on Earth by aliens. So I started talking about this concept of Drake equation and some way of codifying the probability. And I always say the Drake equation makes me a little bit angry because it's really just a product of six different numbers.

And the really interesting things, as you know, as a scientist, is not the numbers themselves, the central value, but the error bars that you assign on it. Right. Yeah. I mean, it's probably like the most overhyped equation in all of science. More than E equals MC squared? Well, I mean, E equals MC squared is like a deep, profound revolutionary insight that took immense genius to arrive at, right? I mean, the Drake equation, as you said, is just like, yeah, multiplying a bunch of numbers.

So I can know the Chalmers equation, the analog thing. But I wonder if you could walk through, like, the individual, you know, kind of probabilities of these different outcomes that one of which should obtain. Is it, you know, kind of proper to think about it in terms of a proper Drake equation where you're assigning likelihood based on some Bayesian prior confidence intervals on all these things? And then we can assess the individual challenges to each one of the input priors, as well as our precision on how well we understand them. So can you, Could you assign some previous knowledge to these three different conjectures and then ask which one is, if not, you know, definitively more probable is at least most likely to be true?

Well, I guess the thing with the civilization argument is that any of the three possibilities would be striking if true. So if the first one were the true one, then it means we are basically doomed, right? We will, contrary to what we might hope and expect, not actually develop these technologies that we know to be physically possible. And that's the more striking if we think that we are actually pretty close to being able to do this. So if you are amongst the people who, for example, think superintelligence might well be developed in this century, then if the first hypothesis were true, right, the first disjunct, that would mean not only that we would have to go extinct soon in this century

before we reach superintelligence or otherwise sort of stop all technological progress, which already would be pretty surprising and alarming, but also that the same would hold true for basically all other technological civilizations throughout the cosmos. So it's not as if that would be a shock, that would be like a major, big revelation about the world. That would be very surprising. The second possibility also would be kind of surprising if you think that there's this immense convergence where everybody would just refrain from using powerful computers that they could by allocating just a tiny percentage of a small number of their vast numbers of computers

for just, you know, a few minutes produce enormous quantities of these answers to simulations. And I wouldn't even do that. Not just would our descendants not do that, but again, it would have to be a pretty universal pattern throughout the universe. So that also would be striking. What could cause such a convergence in behavior? And then the simulation hypothesis is, of course, striking. And so I think that's what gives simulation arguments its oomph,

even before you roll in any further surmises you might have about how the probability should be apportioned as between these three alternatives. And in terms of, as I mentioned before we started recording, I spoke with your colleague there, Sir Roger Penrose, about his theory of what he calls, orchestrated objective reality, which involves very complex, you know, detailed predictions about how way functions actually collapse. And they do so in connection to another one of his great, you know, kind of triumphs involves what's called the wild curvature, which is a derivative of an analog function within, you know, continuous three dimensions of space and one of time in GR.

And but I wonder, you know, and I asked him about his thoughts on superintelligence and the simulation hypothesis. And he was very not sanguine because as you probably know, he doesn't believe the brain is a computer in a sense. And so I wonder, is that a valid objection? In other words, I say there aren't, you know, the ultimate computers aren't even quantum computers. They're, you know, well, there are quantum computers, but consciousness is a manifestation of

a, you know, of this very, very highly non-trivial classical problem that, uh, that does involve at some level, um, you know, how a wave function actually collapses and what causes it to collapse. And then it's instantiated in terms of these microtubules inside of the brain. So what, how would you react to that? What if these ultimate computers that the ultimate, you know, simulators will, will come up with or have nothing to do with, uh, with an advanced, you know, MacBook Pro. Yeah. So I don't buy this claim of quantum consciousness.

I don't particularly think that quantum phenomena are important for how we humans achieve our high-level cognition. I think the human brain is basically too noisy to really take advantage of quantum superpositions. If he were right about that, Then what would have to be assumed in order for the simulation arguments still to hold

would be that at technological maturity, you were able to construct extremely powerful quantum computers such that you would be able to make a lot of simulations of minds in the future, even though each of those minds that you would have to simulate would require harnessing quantum computation. I mean, it does seem like if the human mind naturally evolved to take advantage of those quantum phenomena, then presumably a technologically mature civilization could also take advantage of those phenomena and probably do it on a larger scale, or at least with great parallelism. So that I had to guess it would still be feasible to create astronomical numbers. of human-like experiences, even under those assumptions.

But as I said, I don't buy the assumptions. When you look at these kind of arguments, I guess one question that naturally arises is, yes, if these processes, people tend to put a lot under the rug, as we say, saying something is quantum, right? So that allows for a lot of either sloppy thinking or, you know, in some sense, a lot of latitude. But thinking about processes that are continuous versus discrete, I do note that, you know, Turing in his paper, a lot of people forget this,

he was only talking about digital computers. Now, that could have been because those are the only kinds of computer, I mean, analog computers existed, right? They've been around for thousands of years. But I don't think he was necessarily making a distinction between quantum computers, which, you know, people have thought about. But where do they fit in? I mean, it seems to me, in most cases, quantum computers, at least today, are the best devices ever made for studying quantum computers.

They can do other things, and colleagues of yours like Scott Aronson and others have done great work and kind of forecasting what they could do. But do you, is there sort of hype with that, not Drake equation level hype? but are people basically extrapolating from, you know, hearing this magical word quantum to this enormous infinite power, which is not really the intention? In other words, is quantum, you know, is a quantum computer really the ultimate evolution that's necessary in any way to unlock the simulation, you know, arguments?

No, no, I don't think it's necessary either for the simulation argument or for, indeed, developing, machine super intelligence. I think if we really got quantum computers to work, they would accelerate certain types of computation. That would be useful, for example, as he said, for simulating quantum systems, also for cracking certain cryptographic codes.

And no doubt people would find other uses for them. Possibly, some uses that might be helpful for developing artificial general intelligence, although it's not completely clear how. But it's not needed. In fact, if we look at the current main types of algorithms that are used in machine learning, they are quite insensitive to precision and a very sort of digital approach seems to work very well. Like initially, people were using, you know, 32 or 64-bit float-te,

numbers to represent the parameters of these large neural networks. Turns out that's overkill and you don't need that kind of precision. Like 16-bit numbers are perfectly fine. And you can kind of discretize things quite a lot and it still works. You save a bunch of computing power and you don't really lose that much in performance. And to me, the human mind also looks like a system which has to be robust to noise. It can't be too fuzzy about, you know, the. 10th decimal in some particular setting in some neuron because like biological systems are

noisy things we go around all the time you know sometimes you do exercise and you heat up by one degree and then you put like you you can't rely on those excessively fine gradations i think and so that kind of forces you to use a slightly more robust paradigm where exactly where you carve up the bits doesn't really, you know, it's not critical for the functionality. So my wife has a question, which is if the simulation argument is correct, why are there so many Kardashians? Well, why wouldn't there be? I guess it's like these, you know, people that, like Shirley Maclean here, yeah, people always think that they, they're reincarnated, but they're always like super interesting backstories, you know, that they were the maid of Marie Antoinette or

something. You know, it's never like I was one of the guys who put the axles on some wheel that ran off a donkey cart, putting a pyramid block in place, you know, 2,900. Anyway, I guess it's the bias towards, you know, kind of people thinking of themselves with more importance. I mean, putting kind of the similarity bias, so to speak. But, you know, that was just. a joke, really. But in seriousness, there is... But you could ask a sort of related question, right? So you could ask, is the human history that we observe more or less interesting than you would expect a typical history of some human-like intelligent species to be

on some random planet that were human-like creatures? Like, do you think on most planets things would just be a lot more boring or are we sort of in the middle of the distribution? I mean, I don't know that there is anything particularly suspect if we just look at how things, I mean, it might be kind of a little bit surprising that life evolved at all and stuff like that, but once you get to the rise of Homo sapiens, then it doesn't seem that there's this amazing sequence of coincidence. within the past few thousand years that are just like that looks astronomically unlikely unless somebody rigged it to be like that. It doesn't, I mean, it's as far as we can tell, like to be pretty typical, right? Of what one would expect given the start.

Yeah, I mean, given the uncertainty in the prior distribution, given that we only have one example, you know, I always like to ask my colleagues, you know, do you believe there's life on other planets? And if they answer either way, affirmative or negative, I always just say, you shouldn't be talking about belief at all. Like, you should be talking about, like, what evidence do we have? We have no evidence for life on other planets. We have no evidence for simulation, but that doesn't mean... We do it.

We do have a lot of evidence for planets, though. We have evidence for planets, but, yeah, well, actually, let's go there. So, I mean, that's the starting point. Yeah, so, well, there's some that say you don't even need planets, but let's say you have, you need a planet, right? So I've been to Antarctica twice, and I've spent a couple months probably my life there. It's a big, flat, white, dead place that's always trying to kill you. Not unlike Sweden.

No, I'm just kidding. Sweden's a lovely place. I've been there too, although I don't think I'll go back because of several of my books have been critical of the Nobel Prize. Sweden doesn't try to kill you. It tries to build your character. Oh, you should work for the tourism board, although you don't live there. How do you know that I don't? Well, you'd be a great ambassador, Nick. That's for sure. But people always say, you know,

oh, you bet. So I've been there twice. You know how much life there is there? There's almost no life there. And it's one seventh of all the continents on Earth. And my argument's kind of just tongue in cheek. In other words, it shouldn't just be capacity, you know, that, oh, there's a capacity for life. It should be, you know, are there ingredients for life that are mandatory and how likely are those to originate? Avinicio, not just that there's many, plenty of places for it to exist. In fact, I have a counter example. If you say there's a lot of planets, I say, well, there's at least eight in our solar system alone. We're actually looking for a planet number nine, as you know. But there are these objects called meteorites, which you can get, if you subscribe to my

newsletter and you subscribe to Nick's newsletter. We'll put a link to Nick has a wonderful newsletter that he updates on occasion. I'll put links to his newsletter on his web page below. But if you sign up for my newsletter and you live in the U.S., you win a meteorite, every 100 people will win a meteorite selected at random. Now, these meteorites traverse the solar system, and they sometimes crash into places like Argentina and other places. And some of them have crashed into places like Antarctica, and they've been claimed to have evidence for microbial life form respiratory processes. This was discovered in 1996. And it's never been refused. And the question is, well, this brings up what's called panspermia, which sounds dirty, but it's not.

But it's one of these things that argues that life could transport its way around the galaxy very easily, just floating on meteorites. It doesn't solve the origin of life problem. But given that's true, and given that life on Earth emerged four billion years ago, you know, how surprised should we be in a Bayesian sense that we don't see life on Mars? in other words, and you hope we don't see life on Mars. Maybe we can go there. You wrote an article that you hope we don't see alien life. Why is that? Why would you say something so seemingly contradictory,

at least to my fellow astronomers that are always looking for water and life? I'm like, go down to the campus. Well, yeah, I mean, more specifically, I think it would be bad if we found on Mars independently evolved life. So life that wasn't the origin of life on Earth, nor was it like life from Earth that had come there. but if it had evolved independently, I think that would be bad news,

because that would be evidence that life is common in the universe. I mean, if it's happened twice in our solar system, it's got to be all over the place. And since we haven't seen any advanced life, I mean, unless you believe in the UFO, like it looks empty there, right, insofar as radio transmitting similar. So then there's going to be this great filter somewhere between having primitive life on some planet and having

advanced life like our current civilizations, let alone what we will become if things go well. And then there are basically two possibilities that Great Filter could either be behind us in our evolutionary past, and that would be fine, and we would have made it through. But the other possibilities that it's ahead of us and that we still have to confront this great filter, and that would be very bad news, because by definition, the Great Filter there would be some transition that almost nobody makes it through. And so we should... They don't call it great for nothing, right?

Right. So since there are these two conditional on finding independently evolved life, there's got to be this great filter. It could be behind us and before us. We don't know which one, but some of the probability would then be on it being ahead of us. So it would be bad news for our own future prospect, because of my time, well, be this great filter.

It would increase the chance that there is a great filter in our future. In our future. But yeah, I mean, I think there is life in the universe, though, it's just probably very far away or quite likely very far away. I mean, at least if you think the universe is infinite, then there are like infinite and many planets there, then it would be a virtual certainty, right? That it would exist on...

Unless there's a huge barrier to it. And I think... Well, even if there were a huge barrier, like, if there were a finite probability, and you have like an infinite number of roles of the dice, you would get any possible outcome, not just a few times, but in fact, infinitely many times. Sure, sure. But right.

But I think, you know, more important than that is, you know, it's great to talk about, yeah, the number of galaxies, hosting planets, hosted, you know, hosted around stars is truly astronomical, just in our observable universe. And now we're learning more about the... Wow, well, that's a big difference, yeah. Right. So I don't like... What we care about is not like, is there life inside of this galaxy behind me, M-51, the... World Book gallery, you know, that's never, it's totally, you know, disconnected causally from any of our hopeful lifetimes, right?

Well, I mean, you might care about both. Like, so observable universe is you care about because, well, A, things that are in the observable universe, you know, maybe they will, we will observe them, or they could interact with us or influence us or like, so that they have special sort of relevance to our prudential interests. But what happens outside the observable universe could also be of interest. you might just care about it for its own sake. For the same reason, you might care about people on the opposite side of the world, you want things to go well for them, even if you're never going to meet them.

You just think it would be... But also, from a theoretical point of view, our beliefs about what is going on outside the observable universe might also indirectly influence what you should believe about other aspects of what's going on inside the observable universe, since it's all part of the same physical existence. So there might be sort of theoretical implications as well. Yes, although it's confusing to me, you know,

it becomes sort of infinitely solipsistic because it's not clear if you can have a different universe if the laws of physics, A, old as many people believe they wouldn't necessarily maintain from multiverse, you know, universe to universe is Max Tagmark, your fellow suite has talked about. But more than that, it's not clear that even the laws of logic or, you know, Hilbert spaces and why would they be guaranteed to, you know, even the concept of such a thing is not really decidable, right? Because you can't just say, well, the speed of light would be 20% higher in universe 64 over there,

but it'll be. But how do you know that modus tollens would hold? I mean, are these not things that could be open? If you're saying infinitely many things can happen, then why shouldn't the laws of logic fail? Well, I think they have a different status as arguably analytic truths that might not be violated in any possible world. You could have different systems of logic. You could have a trivalent system of logic and stuff like that. I think they possibly are to be placed in a different category than the laws of physics that certainly one could imagine being different than they are.

And maybe they are different in some other real universes out there. but even if there were not such universes in actual physical existence where the loss of physics were different, even if we're just talking about our standard Big Bang cosmology with the loss of physics as we know them, you could still have a situation with infinitely many planets. If it does have kind of an open or flat universe with the simplest topology, it just goes on as it were forever, and there's a part of it that we can see. the observable universe, light has had time to get there. But beyond that, there is no reason to think there are not more galaxies just continuing out in perpetuity.

What do you make of those that suggest that, like the multiverse, which is what you're describing, that the simulation hypothesis argument is not falsifiable, and therefore, in their words, I'm not saying this, it's not part of the domain of science, which is admittedly very paparian demarcation, dialectical, but what do you make of these criticisms? You can't falsify the simulation hypothesis because it could all be simulations and turtles all the way down. How do you react to those kinds of criticisms? Yeah, I think there are certainly possible observations that would increase or decrease the probability of the simulation hypothesis. For example, take the limiting case where a big window pops up in front of you saying you're in a simulation,

take care for more information or something like that. That would be pretty strong evidence. I don't know, though, Nick, because I talk about, you know, I'm a practicing Jew, and we make note in our Torah and the Bible, the Old Testament, whatever you want to call it, just a few days, weeks after the Jews allegedly witnessed the splitting of the sea in Egypt, allowing them to traverse on their way to go to the land of Israel, that they missed Moses for 40 days. He was about six hours late from returning from communicating with God and getting the Ten Commandments.

And in that six hour period, 40 days after witnessing this miracle among all miracles, they made a golden calf. Now, that, you know, I think, in other words, I think if somebody said they saw this click here button, just as people say about the Jews, they were hallucinating or there was a wind and there was some mass delusion, or it didn't happen, I think there are a lot of ways that people would reject, you know, quote-unquote observational eyewitness evidence, but certainly that, but maybe even physical, quote-unquote, evidence, because, you know, it could all be a brain in a jar. So, a brain in a jar is not that different from the simulation argument. I mean, I think then you have to go into the, like, estimating the cost of producing brains and jars versus creating simulated minds in virtual realities, where I argue that, it would be a lot cheaper for a technologically mature civilization to run the whole thing in silica. So it would just be a more efficient way to generate.

And then you would generate more of them. So even if there were some brains in jars, which that might well be, that would be more people with our experience that would be in digital simulations. And so we should expect if we're one of those two kinds, we would more likely be one of, the kind that has many more instances. I saw. But yeah, no, I mean, certainly if it were some random person reporting that they had seen a big window popping up, like, I agree with you that we should be rather skeptical of that.

I mean, in fact, I do get a lot of reports to random people emailing that I've seen all kinds of things, right, pixels in their bathroom mirrors or whatever. But I think that these kinds of reports, you would be likely to see whether or not we are in simulation and are much more likely to be explained. Even if we were in a simulation, the most likely explanation for those kinds of reports, which will be the normal psychological explanation that people have had a visual hallucination, or they lie, or they misremember, or they misinterpret,

like all of the standard things. There's no reason why not all of those would also apply to a simulated civilization. It's peak pollination. season and my business is scaling fast. To keep the nectar flowing, I need a phone plan with top priority data speeds. That's why I chose GoogleFi wireless. My connections stay strong even when the hive is buzzing. Plus, unlimited plans started $35 a month. Now that's a deal that doesn't stay. Explore GoogleFi Wireless plans today. Plus taxes and government fees. Google Fi Wireless is not subject to data traffic deprioritization during times of high network usage.

something called E-ROM's Law, or E-ROM's Law, which is Moore's Law spelled backwards. And it states that the number of new drugs approved per billion dollars spent on research and development gets cut in half every nine years. So it's the opposite of Moore's Law, which computing power doubles every 18 months or whatever it is. I've heard it said that these computers, they have more terraflops or whatever, but they're not actually by the higher order metrics of, you know, H-indexend. or some publications or some other metric, they're not actually, they're kind of saturating in terms of the net scientific output, not the throughput and instantaneous capability.

Is that not kind of an obstacle or a great filter, if you like, for the simulation hypothesis itself? So what is saturating exactly? The number of either the final product like distillation, final computer simulations, final nuclear web bomb simulation, the amount of the, because there's so many more end users at heart who want to use the technology,

there's also like hackers that are trying to exploit it. In other words, there are other uses rather than just the pure, you know, intended use of the computer as a silicon, you know, device. There's more uses for it. And so it gets,

it's saturating in terms of its final output number of publications. Let's just use that. A higher order statistic rather than just a lower order, but still important, terraflops. Wouldn't that happen? In other words, you get better and better. Now you really want to simulate more and more. And so now the tax on the computing hardware is going up also exponentially. And so you have this ratio of two exponentials that saturates. I'm not sure exactly. I mean, if you look at, I don't know, like the number of video frames

that humanity is producing. I'm sure it's growing rapidly. And the number of photographs taken is, I think still increasing the amount of compute that is being devoted to artificial intelligence applications has been growing a lot faster than Morse law over the past years. How to assess the output of that, of course, that's a kind of qualitative judgment. You could have specific benchmarks where you can plot progress, like on classifying dog and cat images, what's the error rate? And you can see in specific domains like GoPlaying capability or just playing capability and you do see improvements there.

It tends to saturate if you're measuring at the error rate because it has sort of a lower bound of zero or sometimes intrinsically in the dataset, there is like a minimum possible error rate. So as it approaches that, it has to saturate. But then there are new domains to conquer. And my take, if you just look qualitatively at AI achievement impressiveness in some sort of just guts level sense of, do you get the sense for wow when you're looking at what's being achieved each year? I don't have a sense of that having stagnated in AI in the last few years.

I mean, from the start, of the deep learning revolution. It seems like every year or two, we have some other major intuitively impressive achievement. Most recently, I guess, the alpha folding of all these proteins and then the large language models

and then these sort of Dali-style systems and recently some kind of reasoning systems built on top of the large language models as well that are now solving kind of math problems. There's a codex that is like a code writing assistant. All of these things, just at the gut level, I think,

feels pretty impressive and with no particular sense of us having plateaued. I think we will seem to be in the midst of something very rapid unfolding of potential there. Maybe it will possibly stagnate at some point before maybe it gets going again, but I don't I think we are in the midst of an stagnation in AI currently. Mm-hmm.

I wonder, I know it's getting late there, Nick. Do you have time for one more segment about risks in the future? Well, I could take one more question. Okay. All right, I'm going to parlay this into a question regarding risk, but also the simulation hypothesis. So we've had COVID. I see kids coming around, not me personally, thankfully, but I see kids on my campus, biking around, not wearing a helmet, using their phone and then wearing a mask. And I guess the question that one of my audience members is asking is that

given the super intelligent concept of a singleton, which is not a simpleton, but a singleton in which any entity that has complete world domination such that can eliminate all threats to an existence, it could be a government or super intelligent AI. And once it arises, it cannot be replaced because it will systematically destroy any opposition. Now, I want to ask you, given that COVID is kind of the first ever, you know, worldwide, fully documented video, audio, you know, virtual reality, in any possible way. It's kind of the first of a, you know, premise interpowers. Is this sort of pointing at perhaps a government like the CCP or something like that, that they are approaching it? And if they did, would it not behoove whoever's not,

you know, in the cusp of being a singleton to destroy that to minimize risk? So I know it's a multi-part question involving COVID, Chinese government, and so forth. But if you felt there was a chance for folks to, or for government like CCP to come up with this, one of my listeners, JT. I, Alexandria, is asking what would be, or sorry, Joseph Billing, is asking, should we kill the Singleton while it's still a newborn? Well, the concept of Singleton is a somewhat more abstract one. So it's a world order in which at the highest level of organization, there is a single decision-making entity such that global coordination problems are solved. So that can't be sort of two single tons on Earth, right, by definition. It's a kind of a global world order.

And so it's to some extent a matter of degree. We have various weaker forms of international collaboration. But they are far from perfect. So we still have global coordination problems that we haven't solved, that results in the fact that we have militaries that are preparing to kill the people in the other militaries, that we have global warming, that we have overfishing, that we have a whole bunch of problems that arise fundamentally from our ability to get along with one another. within nations we have as it were the analog of singletons but within a limited jurisdiction so yeah so the communist

party in china would be a singleton in china the u.s. government would be kind of a singleton in the US and it's almost a definition of a state that it's an entity that has a monopoly of violence within its territory you know give or take but that's approximately the case but we don't have any analogous thing at the global level. The different organizations we have are very weak, like the UN is weak, the international trade organization. There are these kind of weak overlapping

world economic forum. Yeah, yeah, they're very weak. They may be more than zero, but they are less powerful over the globe than the United States government is over the United States or the Communist Party is over China. And so

now it's an interesting question whether we, to the extent that we have some choice in the matter, should favor or disfavor the emergence of a single song. That is some kind of state-like, and it doesn't need to be a state-like entity. You said it could be a super intelligent. It could also be in theory other kinds of things, like a kind of moral code, universal moral code that was self-enforcing. So it's an abstract concept that could be instantiated in many different ways.

Should we be in favor or opposed to? this. I think in theory, a singleton could be very bad or very harmful. The concept itself doesn't determine that. I think it's quite likely that there will eventually be a singleton by no means certain. And on the margin, should we favorite or disfavorite? Very hard to say. I mean, I kind of tend to think that a lot of the biggest existential risks will arise from the fundamental fracturing of humanity that the possibility of conflict or of one part of humanity doing something that is against the interest of another part, new weapon systems, new arms races, new kinds of risk taking that you're doing competition with other superpowers. If that's the biggest source of

existential risk, then it's possible that having something that reduces that would be net positive, even though certainly it is the case that the single ton itself would be a another big source of existential risk. Like, if it turns bad, then, you know, it's all the eggs in one basket. So it's a question of weighing these two different kinds of risk. But I think the risks from conflict looks plausibly dominant over the risk of oppression. But, I mean, that's a hard judgment call. And last question, if you'll indulge me for one second.

Elon Musk has talked a lot about the great filter, super AI intelligence and stuff But he's also talked a lot about underpopulation and depopulation. What's your stance there? Should a young person have children right now in your... I don't know if you have children, but should somebody ethically have children? I think seems fine. I mean, I think there are, to the extent that you want to do the morally optimal thing, there are many things you could do besides having children,

but I certainly don't think there is any moral reason to refrain from it. I worry less about population decline because I'm thinking probably before that has, it's a kind of slow and gradual thing that the world will be dramatically transformed before that has had a time to play itself out. And in the longer run, I think that would be powerful evolutionary selection for higher fertility that would eventually counteract these cultural trends, most likely. We already see selection, say, in favor of reduced female education stuff, have higher fertility, fundamentalist, many fundamentalist communities have higher fertility. And there will be all kinds of other partially inheritable traits that will impact fertility, and those will be on the positive selection. So over longer timeframes, like if we're talking hundreds of years and stuff,

then eventually I think some group will emerge that either for biological reasons or more likely for cultural reasons, just have managed to find some way to retain high fertility even in a sort of advanced technological context. Interesting. So humanity does have a future in addition to your institute, which you direct as founder. Nick, I want to thank you so much. Your work is always so inspiring. It makes me think. It makes me turn different ideas in my head.

My audience loves your mind, and we hope to have you back someday. Let's be fun. Thank you for having me. Thank you. All. Pay off your home, travel for life, drive a Ferrari. In celebration of the world premiere of the Monopoly Big Board Buckslot machine by Aristocrat Gaming, Yamava Resort and Casino at San Manuel is giving one person a $1.6 million dream package.

The biggest prize in Yamava's history. Club Serrano members can earn daily instant prizes and secure a spot in the finale May 29th. Don't pass go and own it all. Only at Yamava, celebrating its 40th anniversary. You win? Details at Yamava.com must be 21-20. Please gamble responsibly.

Monopoly is a trademark of Hasbro. Hasbro is not a sponsor of this promotion.