Into the Impossible With Brian Keating - Nick Bostrom: Superintelligence (#256)

Starting point is 00:00:02 Welcome, loyal, into the impossible listeners. Or maybe you're just simulated listeners. Maybe I'm simulated. Who's to know? All I know is that this interview with Nick Bostrom was a bucketless conversation, which we went deep talking about the most important topics in all of science, perhaps, the nature of existence. Whether or not we are simulations depends on whether or not you take seriously what's

Starting point is 00:00:34 called superintelligence. We talked about that from his phenomenal 2014 book of the same name. We also discussed a wide variety of topics, including whether or not the Earth should fear depopulation more than fearing aliens, or perhaps what is called ominously the Great Filter. We talked about a tremendous variety of subjects ranging from the risk of super intelligent AI to whether or not humanity has witnessed its first global scale phenomenon. for the first time documented by high-performance computing devices, cameras, and humans. If you assume that we are human and not just simulated. If we're simulated, then it really shouldn't be such a big deal to get my kids to fold their laundry.

Starting point is 00:01:20 If these alpha-fold computer algorithms and machines can do the same thing, why can't my kids fold their day? All right, I'm going off on a tangent. Sorry about that. But this was really a delightful discussion with Nick Bostrom. Originally born in Sweden, along with past guest, Max Tagmart. And future guests, the Abba group, now hopefully we can get on some musical entertainment that's a little bit younger than Abba. But he is currently a professor at Oxford University where he heads the Future of Humanity Institute as its founding director.

Starting point is 00:01:50 He's published hundreds of publications. He has a wonderful website. You'll hear us talk about what troubles him most about the research, the interpretation, whether or not we could ever have counter examples to falsify perhaps his theories about simulations and others. Are we reaching the base level of simulation and whether or not we could ever get down to the ultimate reality, whether it be Silicon or in Vivio? Took some audience questions. It reminds you, you can ask audience questions anytime on my website, briankeating.com slash podcast. And you can also leave comments on my YouTube channel, Dr. Brian Keating, or on Twitter, Dr. Brian Keating, or even Instagram. Some of the only social media devices and platforms for which the ultimate, simulators will have to atone, although that might be true of Kim Kardashian and all the proliferation

Starting point is 00:02:40 of Kardashians as well. So now sit back, relax, and enjoy this ride into The Impossible with myself, Brian Keating and Nick Bostrom. Enjoy. Any sufficiently advanced technology is indistinguishable from magic. Open the pod bay doors, please, hell. Welcome everybody to a very exciting, super intelligent conversation with none other than possibly the most requested guests that I have not yet had on in any form. And that's Professor Nick Bostrom, joining us from Oxford. How are you today, Nick? A pleasure. It's really great to meet you virtually, and we'd be together if there weren't existential risks threatening the world.

Starting point is 00:03:29 But I want to begin by doing what you're never supposed to do, which you're always told not to do, which is to judge a book by its cover. And this is a segment that's a segment that's It's very popular on this podcast because, you know, really, what else do you have to go on? If you see a book, you only can look at its cover. Maybe you know the author. Maybe you know somebody that endorsed it. So I want to talk about your New York Times number one bestselling book called Super Intelligence. And I want to ask you, how did you come up with the title, the subtitle, and the very, very evocative and interesting cover design, at least here in the U.S., which is a very wise and observant bird known as the owl.

Starting point is 00:04:10 So, Nick, how did you come up with the title and the cover design for this wonderful book? Well, I mean, the title is pretty obvious. The whole book is about superintelligence, and so I keep it simple. There is a subtitle, Path, Danger, Strategies, Since It's a book with Academic Pretensions, I feel it needed the double-barreled approach. It also fleshes it out a little. The cover was a bit of a struggle, actually, with the publisher to get it the way. I wanted. The owl picture itself is from some artwork that we got permissions for only,

Starting point is 00:04:54 apparently for the English-speaking version, which was a mistake, because there are all these translations with various mangled owl-like creatures on the cover. So that's a lesson for next time. Make sure you get the rights worldwide. And then I added these little There are some snowy trees in the background, and I think you can actually see some computer code in there as well in the trees. That's a little Easter egg in that code. It's probably going to remain an uncracked mystery. And some red squares, I think, right? You can see the...

Starting point is 00:05:40 Yeah, like a Conway life game. Yeah, you can read it many ways. Is pixels in the color of blood or yeah, Conway or it's open for interpretation. Very good. So, yes, I always find it interesting, although my audience gets mad at me because half the time I ask an author, you know, how'd you come up with the title? They say, oh, is the publisher's idea, including Sabina Hassanfelder, who I just had on recently. And she has a new book about existential physics, including some aspects of your work that she refers to. and not with the greatest of enthusiasm.

Starting point is 00:06:20 Unfortunately, she's very convergingly. We love her around here. But she's negative about some of these prospects. We'll get into the topics of it, of course. But I wanted to begin because that book has a very powerful and long-lived impact, at least on culture and so forth. But I want to, instead of delving into some of the stuff you're most renowned for in many ways, and I should state, actually, I'm going to have my computer do it. Computer, who is Nick Bostrom?

Starting point is 00:06:51 According to Wikipedia, Nick Bostrom is a Swedish-born philosopher at the University of Oxford known for his work on existential risk. Computer. Stop. So I have a computer here, a computer assistant, which can do many, many things, including computer, turn on the plug. Now, this turned on the power supply that is in parallel run to power my computing device. So the question is, will the computer commit suicide if I pull, ask it to terminate itself?

Starting point is 00:07:21 What do you think will happen, Nick? If I ask the computer to turn out very powerful. Does it have an internal battery? Like, is it a laptop? No. Well, I guess there's only one way to find out. Computer, pull the plug on yourself. Plug doesn't support that.

Starting point is 00:07:39 All right, now it's right. All right, computer, open the pod bay door. I'm sorry, Dave. I'm afraid I can't do that. Oh, my God. Also, I'm not how. And we're not in space. But Nick, one of your most fascinating aspects of your bio, I'm going to read that you got your PhD at the London School of Economics. Your dissertation was selected by the late Robert Nozick, a towering figure in economics and many different academic scholarly realms. You won the outstanding dissertation. You're included in his book on outstanding dissertation. But I want to go back to your undergraduate days because at King's College, you studied astrophysics and general relativity.

Starting point is 00:08:26 And in doing so, I want to sort of harken back to your undergraduate days. And that is to bring up this guy. So this is a guy. He's pretty famous guy. His name's Albert, Albert Einstein. And Albert said, and I quote, he said, the happiest thought of his life. Do you actually, do you know what the happiest thought of Albert Einstein's life was, Nick?

Starting point is 00:08:51 I think I might have heard it. I don't remember it, though. It was that a person in free fall would experience no gravitational force field. So he would not experience gravitational fields. And he called that the happiest thought of his life. We hear a lot of talk about AI doing physics, your fellow Swede, Max Tagmark, talks about the mathematical universe and the dangers of what he calls Life 3.0. I'm less nervous about the impact of AI because I would like to understand how could a computer have a happy thought, first of all? And second of all, how could it do things like Max hopes it will do, create new laws of physics, etc., but how could it do it by visualizing it's,

Starting point is 00:09:38 its own behavior as a body in free fall. Aren't these things preventing AI or computers from doing anything remotely what, like what Albert did 100 years ago or what we might do today? I mean the lack of a body? Yeah, sensation and also happiness and optimization in terms of some internal driven metric. Insofar as a body were necessary, I guess one could just give it a body like a robot or something, and then that problem would be solved. Of course, we don't know exactly how it would manage to have such a happy thought as Einstein had,

Starting point is 00:10:18 but then again, we don't know exactly how Einstein managed to have that either. But whatever the computational mechanisms are that allow our brain to achieve general intelligence and occasional sparks of indignuity, I think similar algorithms would work if implemented on a biological substrate. So instantiating that. And similarly, could you instantiate feelings of pain as sort of a corrective mechanism, a way to, you know, what humans react to stimuli,

Starting point is 00:10:51 both positive as Einstein did in that example, but also negatively. I mean, could you imagine a situation where, you know, computing device could come up with some sort of, you know, physical analog to pain out of low-capacitor, here or there. Would that be useful, though, is my question, not could we do it?

Starting point is 00:11:18 We could give it, you know, some body or something, but would that be useful? In other words, what is the role of pain and pleasure in the human creative process? Certainly the functional analog of pain and pleasure are used in reinforcement learning. You give positive or negative rewards that then increases or decreases the propensity to repeat the same behavior that led to the reward. So that's one of the main paradigms in the current machine learning. Now, when we speak of human pain and pleasure, we tend to add to that functional role,

Starting point is 00:12:01 also the idea that there is a phenomenal experience, that there is a sort of subjective awareness that then has moral significance in the case of suffering, something that it is, important to try to reduce in the world. I think that current AI systems probably lack that, but that there is no principled impossibility in having digital minds also possessing these phenomenal experiences. And I think we will create systems that have that.

Starting point is 00:12:32 Of course, there are different theories about exactly what the extra ingredients are that would be required for a computer to be conscious. But according to all the main contenders, if you have the global workspace theory, you have attention schema theory, you have higher order thought theory, for example. On each of those, it would be possible for a suitably sophisticated digital minds, also to satisfy those criteria and therefore to have phenomenal experiences. So another thing that comes up quite frequently and just I'm a lay person in this,

Starting point is 00:13:10 in this domain, obviously, but we hear a lot about computers dominating humans at games like chess and go. And in addition to the very useful things that they can do, like, you know, compute all possible foldings of proteins and laundry in my kids' bedroom or something. But my question is not can they beat us at these games, but can they invent a game that is that, you know, could stand the test of time the way that chess, chess is 2,400 years? years old at the minimum, maybe even twice as old. There are computers now.

Starting point is 00:13:45 You can buy, a layperson can buy, and computer playing programs, not even deep mind. So to what extent do we really have much to fear? We invented these domains, and they're playing in it and they're doing well. They're beating us. Is that so surprising? I mean, my kid is better than me at basketball. Should I fear that? Or should we really start to worry when a computer invents a game like chess?

Starting point is 00:14:07 That would seem to be a watershed moment rather than just beating us. Well, there might be several watershed moments. I mean, it wasn't that long ago since we didn't have computers at all, right? I mean, it's less than 100 years. And so we've come a long way since then, and then for a long time, they couldn't beat humans in chess, and then they could. And then in Gaul, now they can fold proteins, and they can do a whole bunch of other things. And at some point, I will be able to invent new games as well, maybe initially within some confined parameters, like you could have procedurally generated content, like some computer games have

Starting point is 00:14:44 that kind of can design new levels, say, and then maybe the span of their creativity will increase from that point on. I don't see it as a kind of, in principle, stumbling block. I think, like, if you're talking about the full range of human ability to invent new paradigms and such that might be AI complete, meaning that by the time AIs have that capability, they will also have the ability to do all the other things that we humans can do. So at that point, you might already be in the singularity or in the transition to superintelligence. Right. So when I think about superintelligence and your wonderful book,

Starting point is 00:15:32 in the book, which we should event, eventually, you know, make a detour where you define all these terms because, you know, it would be like when I had David Chalmers on and I asked him, you know, to give me some forbearance and define the hard problem of consciousness. You know, I said it's like asking his, you know, the best band in his country, ACDC, not to play, you know, back in black. And, you know, for me not to ask you to define things like superintelligence or the simulation hypothesis would be like me, you know, not asking. asking Abba to play Dancing Queen.

Starting point is 00:16:09 So we're going to have to do that at some point. But before we do that, you mentioned, yeah, computers are only 100, I mean, digital, modern computers are about 100 years old. What's coming up on its 75th anniversary is the so-called Turing test, which Alan Turing gave the name, the imitation game. And in 1950, when he wrote that seminal paper, he talked about the advent of digital computers having a predict, making predictions by the year 2000, they would pass the, imitation game. Now, I'm not up to speed on many things in this field, but to my knowledge,

Starting point is 00:16:42 there isn't anyone that's fully or any machine digital computer at least that's passed the Turing test. Is that right? Yeah, so there are different versions of the Turing test. And it turns out it matters quite a lot exactly how you define it for how hard it is. So you could have some very rudimentary version of it where some random person that doesn't know much about computers I just does something for a few minutes and it's quite easy to trick such a person. You just program by hand something with a few canned answers. So that has happened a long time ago, but if you have the full-blown version where you have experts interrogating the system for two hours, then that's something that milestone hasn't yet been passed. And indeed, that might be

Starting point is 00:17:30 another example of a Turing complete task because you really would need to be able to be able to to sort of exhibit science of general intelligence across all domains in order to pass such a test. Now, in your book, Super Intelligence, you make a claim that by 2100, so 100 years exactly after Turing thought that Turing test would be passed, that we would have superintelligence. So, as I said, I can't resist. Probably before, I mean, it might well be long before then. I think there's a lot of uncertainty about time scales. Right.

Starting point is 00:18:11 And that we should therefore think in terms of a smeared-out probability distribution over many arrival dates. But I mean, progress has been, I think, faster than expected since the book came out in 2014. What surprised you about the progress or that pace of progress? In other words, why didn't you make it 2022 or something? What's been the progress that's most impressed you? Well, I mean, I didn't predict 2100. I just reported the, like, a survey of AI experts back then and some other sources, like, and then indicated that there was a wide distribution and that there was some non-trivial.

Starting point is 00:18:54 I think the only main claim regarding timelines and the book release wasn't much about timelines, but was that that was like a significant and non-trivial probability that it would happen within the lifetime of many people alive today or when the book was written. Now, since 2014, I mean, we basically had the whole deep learning revolution where a number of things just seem to be clicking into place and the same small set of techniques have turned out to, you know, first kind of crack computer vision and then AlphaGo was a big milestone where you combined deep learning with reinforcement learning. And then we have continued to see, you know, a parade of dramatic advances.

Starting point is 00:19:47 Most recently, these large language models that are producing text output and now also visual output. That is really quite impressive. Yeah. Yeah, I've been, you're referring, of course, to Dolly, and some of the GPT3 engines. I wonder, you know, so last year I had the honor to make the first ever audiobook of Galileo's, any of Galileo's books. I did it with Carlo Rovelli and Frank Wilczek and Fabiola Giannati and many other physicists.

Starting point is 00:20:23 And it was so exciting to me because I had to get the actual text of this trialogue between these three characters that, you know, set the stage for the scientific revolution, you know, the 17th century. and in this book, I have the actual words. And so at one point, my thought was to just dump all those words into GPP3 and get this creature that I would call Gallo-A-I-O. But I wonder if that's too simple-minded. In other words, if I just have all this text and it's reflective of his personality, and he was actually one of the best writers in human history, according to Einstein and others.

Starting point is 00:21:00 But, you know, is that, is this the deep part of what you just talked about? Is that sufficient or is it merely necessary? If it just took every word that, you know, Einstein thought, could I come up or wrote down or, you know, maybe even spoke? Could I ever get approximation of what he'd be like? And if not, maybe why not? Your summer starts now with Memorial Day deals at the Home Depot. It's time to fire up summer cookouts with the next grill, four-burner gas grill, on special buy for only $199.

Starting point is 00:21:34 And entertain all season with the Hampton Bay West Grove seven-piece outdoor dining set for only $499. This Memorial Day get low prices guaranteed at the Home Depot. While supplies last, pricing invalid May 14th or May 27th, U.S. only exclusions apply.

Starting point is 00:21:48 See Home Depot.com slash price match for details. So whether from, say, the texture last foot of normal human, when you could in theory infer what the mind must have been like that wrote those words. Yeah. I mean, I suspect, yeah. Yeah, it is. I mean, we don't know. My guess would be that if you were a super italus,

Starting point is 00:22:13 you could get pretty amazingly far in that direction from relatively few signs. Like, yeah, the corpus of a text. Combined maybe by just observing in general, the distribution of human minds and knowing how brains work and then kind of the specific information you had about the person was maybe just their text dot

Starting point is 00:22:39 but I think you could probably interpolate some pretty good idea of what the generating mind behind that would have been like. Yeah, so that seems to me to be one of the maybe it's, yeah, this notion of Turing completeness that would maybe not be met, but it would still be a useful. I mean, something that's not Turing complete is still pretty useful, right? I mean, I have these computer programs that I put my kids

Starting point is 00:23:08 in front of, you know, and they'll ask it, how is your day? And how do you feel about that? And, you know, why is your dad such a bad dad? You know, it'll do all these things that entertainment. It doesn't mean, in other words, you have to solve the, you know, the Turing, the imitation game in order to have something of great use, obviously. So I want to turn, um, you know, to this, to this notion of, you know, the simulation hypothesis, which you're intimately connected with. And as I said, you know, not asking you to define it would be like not asking Abba to sing, dancing queen. So I wonder if you could, in your 30 second manner that I've heard you do it in the past, if you would please indulge me with some forbearance and repeat what you mean

Starting point is 00:23:51 by that. Well, 30 seconds, we'll be challenging. But yeah, there's a simulation. This is a The simulation argument tries to show that at least one of three possibilities actually obtain, although it doesn't tell us which one. So what are those three? Well, number one possibility is amongst all the civilizations at our current stage of technological development, almost none attain technological maturity. So they all go extinct before they develop the full range of technologies that we know are physically possible in. Second possibility is there is this strong convergence among all civilizations that do become technologically mature. They all just lose interest in using those immense technological powers for the purpose of creating large numbers of ancestors, detailed simulations with conscious digital beings in them.

Starting point is 00:24:49 So those are two, right? And then the third possibility is we are living in a computer simulation. The simulation argument kind of demonstrates that at least one is true by assuming that the first two are false and then seeing that if those first two possibilities did not obtain, we would have strong reason for believing in the third. All right. So that then suggests you can't reject all three, and so at least one of them has to hold. And, you know, it's safe to say that, you know, there's been tremendous interest and attention given to it. And when I talked to David Chalmers about in his most recent book on Reality Plus, I believe he called it, a wonderful book. I'll have a link to it in the links above, that, you know, he basically suggests the likelihood is extremely high. In other words, it's not just that there are these three possibilities.

Starting point is 00:25:45 They all have equal weight. There's actually, in his mind, a great deal of evidence for it. And I started to ask him, and I don't know if I fully understood his answer, but in some sense, we in astronomy have, you know, the Drake equation for talking about similar things. The possibility there could be aliens. We could have been aliens. We'll get to aliens later, by the way. But, you know, we could have been seated here on Earth by aliens. So I started talking about this concept of Drake equation and some way of codifying the probability. And I always say the Drake equation makes me a little bit angry because it's really just a product of six

Starting point is 00:26:20 different numbers. And the really interesting things, as you know, as a scientist, is not the numbers themselves, the central value, but the error bars that you assign on it. Right. It's probably like the most overhyped equation in all of science. More than E equals MC squared? Well, I mean, E equals MC squared is like a deep, profound revolutionary insight that took immense genius to arrive at, right? I mean, the Drake equation, as you said, is just like, yeah, multiplying a bunch of numbers.

Starting point is 00:26:48 So I can know the Chalmers equation, the analog thing. But I wonder if you could walk through, like the individual, you know, kind of probabilities of these different outcomes that one of which should obtain. Is it, you know, kind of proper to think about it in terms of a proper Drake equation where you're assigning likelihood based on some Bayesian prior confidence intervals on all these things? And then we can assess the individual challenges to each one of the input priors as well as our precision on how well we understand them. So can you, could you assign some previous knowledge to these three different conjectures and then ask which one is, if not, you know, definitively more probable, is at least

Starting point is 00:27:38 most likely to be true? Well, I guess the thing with the civilization argument is that any of the three possibilities would be striking if true. So if the first one would, we're the true one, then it means we are basically doomed, right? We will, contrary to what we might hope and expect, not actually develop these technologies that we know to be physically possible. And that's the more striking if we think that we are actually pretty close to being able to do this.

Starting point is 00:28:14 So if you are amongst the people who, for example, think super intelligence might well be developed in this century, then if the first hypothesis were true, right? The first disjunct, that would mean not only that we would have to go extinct soon in this century before we reach superintelligence or otherwise sort of stop all technological progress, which already would be pretty surprising and alarming, but also that the same would hold true for basically all other technological civilizations throughout the cosmos. So it's not as if that would be a shock.

Starting point is 00:28:59 That would be like a major, big revelation about the world. That would be very surprising. The second possibility also would be kind of surprising if you think that there is this immense convergence where everybody would just refrain from using powerful computers that they could by allocating just a tiny percentage of a small number of their, fast numbers of computers for just, you know, a few minutes produce enormous quantities of these answers to simulations. And I wouldn't even do that. Not just would our descendants not do that, but again, it would have to be a pretty universal pattern throughout the universe. So that also would be striking. What could cause such a convergence in behavior? And then the simulation hypothesis is of course striking. And so I think that's what gives civilization arguments. It's oomph, even before you roll in and if other surmises you might have about how the probability should be apportioned as between these three alternatives.

Starting point is 00:30:07 And in terms of, you know, as I mentioned before we started recording, I spoke with your colleague there, Sir Roger Penrose, about his theory of what he calls orchestrated objective reality. which involves very complex, you know, detailed predictions about how way functions actually collapse. And they do so in connection to another one of his great, you know, kind of triumphs. It involves what's called the wild curvature, which is a derivative of an analog function within, you know, continuous three dimensions of space and one of time in GR. And but I wonder, you know, and I asked him about his thoughts on superintelligence and the same I was very not sanguine because as you probably know, he doesn't believe the brain is a computer in a sense. And so I wonder, is that a valid objection?

Starting point is 00:31:02 In other words, I say there aren't, you know, the ultimate computers aren't even quantum computers. They're, you know, well, there are quantum computers, but consciousness is a manifestation of a, of this very, very highly non-trivial classical problem that does involve at some level, you know, how a wave function actually collapses and what causes it to collapse. And then it's instantiated in terms of these microtubules inside of the brain. So how would you react to that? What if these ultimate computers that the ultimate, you know, simulators will come up with or have nothing to do with an advanced, you know, MacBook Pro?

Starting point is 00:31:41 Yeah. So I don't buy this claim of quantum consciousness. I don't particularly think that quantum phenomena are important for how we humans achieve our high-level cognition. I think the human brain is basically too noisy to really take advantage of quantum superpositions. If he were right about that, then what would have to be assumed in order for the simulation argument still to hold would be that at technological maturity you were able to construct extremely powerful quantum computers such that you would be able to make a lot of simulations of mind. in the future, even though each of those minds that you would have to simulate would require harnessing quantum computation. I mean, it does seem like if the human mind naturally evolved to

Starting point is 00:32:45 take advantage of those quantum phenomena, then presumably a technologically mature civilization could also take advantage of those phenomena and probably do it on a larger scale, or at least with great parallelism. So that I had to guess it would still be feasible to create astronomical numbers numbers of human-like experiences, even under those assumptions. But as I said, I don't buy the assumptions. When you look at these kind of arguments, I guess one question that naturally arises is, yes, if these processes, people tend to put a lot under the wool under the rug, as we say, saying something is quantum, right?

Starting point is 00:33:30 So that allows for a lot of either sloppy thinking or, you know, some sense, a lot of latitude. But, you know, thinking about processes that are continuous versus discrete, I do note that, you know, Turing in his paper, a lot of people forget this, he was only talking about digital computers. Now, that could have been because those are the only kinds of computer, I mean, analog computers existed, right? They've been around for thousands of years. But I don't think he was necessarily making a distinction between quantum computers, which, you know, people have thought about. But where do they feel? fit in. I mean, it seems to me, in most cases, quantum computers, at least today, are the best

Starting point is 00:34:10 devices ever made for studying quantum computers. They can do other things. And colleagues of years like Scott Aronson and others have done great work and kind of forecasting what they could do. But do you, is there sort of hype with that, not Drake equation level hype, but are people basically extrapolating from, you know, hearing this magical word, quantum to this enormous infinite power, which is not really the intention. In other words, is quantum, is a quantum computer really the ultimate evolution that's necessary in any way to unlock the simulation arguments? No, no, I don't think it's necessary either for the simulation argument or for indeed developing machine superintelligence. I think if we really got quantum computers to work, they would accelerate certain types of computation.

Starting point is 00:35:09 That would be useful, for example, as you said, for simulating quantum systems, also for cracking certain cryptographic codes. And no doubt people would find other uses for them. Possibly, some uses that might be helpful for developing artificial general intelligence, although it's not completely clear. here, how. But it's not needed. In fact, if we look at the current main types of algorithms that are used in machine learning, they are quite insensitive to precision and a very sort of digital approach seems to work very well. Like initially people were using, you know, 32 or 64 bit floating numbers to represent. the parameters of these large neural networks. Turns out that's overkill and you don't need that kind of precision. Like 16-bit numbers are perfectly fine.

Starting point is 00:36:09 And you can kind of discretize things quite a lot and it still works. You save a bunch of computing power and you don't really lose that much in performance. And to me, the human mind also looks like a system which has to be robust to noise. It can't be too fuzzy about the 10th decimal. in some particular setting in some neuron because like biological systems are noisy things wiggle around all the time you know sometimes you do exercise and you heat up by one degree and then you can't rely on on those excessively fine gradations i think and so that kind of forces you to use a slightly more robust paradigm where exactly where you carve up the bits doesn't really

Starting point is 00:36:57 you know, it's not critical for the functionality. So my wife has a question, which is if the simulation argument is correct, why are there so many Kardashians? Well, why wouldn't there be? I guess it's like these, you know, people that like Shirley McLean here, yeah, people always think that they, they're reincarnated, but they're always like super interesting backstories, you know, that they were the made of, of Marie Antoinette or something. You know, it's never like I was one of the guys who put the axles on some wheel that ran off a donkey cart, putting a pyramid block in place, you know, 2,900. Anyway, I guess it's the bias towards, you know, kind of people thinking of themselves

Starting point is 00:37:48 with more importance. I mean, putting kind of the similarity bias, so to speak. But that was just a joke, really. But in seriousness, there is. But you could ask as. sort of related question, right? So you could ask, is the human history that we observe more or less interesting than you would expect a typical history of some human-like intelligent species to be on some random planet that were human-like creatures? Like, do you think on most planets, things would just be a lot more boring or are we sort of in the middle of the distribution?

Starting point is 00:38:27 I mean, I don't know that there is anything particularly suspect if we just look at how things, I mean, it might be kind of a little bit surprising that life evolved at all and stuff like that, but once you get to the rise of Homo sapiens, then it doesn't seem that there's this amazing sequence of coincidences within the past few thousand years that are just like that looks astronomically unlikely unless somebody rigged it to be like that. I mean, it's as far as we can tell, to be pretty typical, right, of what one would expect, given the start. Yeah, I mean, given uncertainty in the prior distribution, given that we only have one example,

Starting point is 00:39:14 you know, I always like to ask my colleagues, you know, do you believe there's life on other planets? And if they answer either way, affirmative or negative, I always just say you shouldn't be talking about belief at all. Like, you should be talking about, like, what evidence do we have? We have no evidence for life on other planets. We have no evidence for simulation, but that doesn't mean... We do it. We do have a lot of evidence for planets, though.

Starting point is 00:39:35 We have evidence for planets, but yeah. Well, actually, let's go there. So, I mean, that's the starting points. Yeah. So, well, there's something that say you don't even need planets, but let's say you have, you need a planet, right? So I've been to Antarctica twice, and I've spent a couple months probably my life there. It's a big, flat, white, dead place that's always trying to kill you.

Starting point is 00:39:56 Not unlike Sweden. No, I'm just kidding. Sweden's a lovely place. I've been there too, although I don't think I'll go back because of several of my books have been critical of the Nobel Prize. Sweden doesn't try to kill you, it tries to build your character. Oh, you should work for the tourism board, although you don't live there. How do you know that I don't? Well, you'd be a great ambassador, Nick. That's for sure.

Starting point is 00:40:20 But people always say, you know, so I've been there twice. You know how much life there is there? There's almost no life there. And it's one seventh of all the continents on Earth. And my argument's kind of just tongue in cheek. In other words, it shouldn't just be capacity, you know, that, oh, there's a capacity for life. It should be, you know, are there ingredients for life that are mandatory and how likely are those to originate ab initio? Not just that there's many, plenty of places for it to exist. In fact, I have a counter example.

Starting point is 00:40:51 If you say there's a lot of planets, I say, well, there's at least eight in our solar system alone. looking for a planet number nine, as you know. But there are these objects called meteorites, which you can get if you subscribe to my newsletter and you subscribe to Nick's newsletter. We'll put a link to Nick has a wonderful newsletter that he updates on occasion. I'll put links to his newsletter on his webpage below. But if you sign for my newsletter and you live in the U.S., you win a meteorite. Every 100 people will win a meteorite selected at random. Now, these meteorites traverse the solar system and they sometimes crash into places like, Argentina and other places. And some of them have crashed into places like Antarctica,

Starting point is 00:41:32 and they've been claimed to have evidence for microbial life form respiratory processes. This was discovered in 1996. And it's never been refuted. And the question is, well, this brings up what's called panspermia, which sounds dirty, but it's not. But it's one of these things that argues that life could transport its way around the galaxy very easily, just floating on meteorites. It doesn't doesn't solve the origin of life problem. But given that's true, and given that life on Earth emerged four billion years ago, you know, how surprised should we be in a Bayesian sense that we don't see life on Mars? In other words, and you hope we don't see life on Mars. Maybe we can go there. You wrote an article that you hope we don't see alien life. Why is that? Why would you say something

Starting point is 00:42:19 so seemingly contradictory, at least to my fellow astronomers that are always looking for water and life? I'm like, go down to the campus. Well, yeah, I mean, more specifically, I think it would be bad if we found on Mars independently evolved life. So life that wasn't the origin of life on Earth, nor was it like life from Earth that had come there. But if it had evolved independently, I think that that would be bad news because that would be evidence that life is common in the universe. I mean, if it's happened twice in our solar system, it's going to be all over the place. And since we haven't seen any advanced life, I mean, unless you believe in the UFO, like it looks empty there, right, insofar as radio transmitting civil. So then there's going to be this great filter somewhere between having primitive life on some planet and having advanced life like our current civilizations, let alone what we will become if things go well.

Starting point is 00:43:16 And then there are basically two possibilities that great filter could either be behind us in our evolutionary past and that would be fine, then we would have made it through. But the other possibilities that it's ahead of us and that we still have to confront this great filter. And that would very bad news because by definition, the great filter would be some transition that almost nobody makes it through. Right. And so we should call it great for nothing, right? Right. So since there are these two conditional on finding individuals, independently evolved life, there's got to be this great filter.

Starting point is 00:43:48 It could be behind us and before us. We don't know which one, but some of the probability would then be on it being ahead of us. So it would be bad news for our own future prospect because of my time well be this great filter. It would increase the chance that there is a great filter in our future. In our future. But yeah, I mean, I think there is life in the universe, though, it's not just probably very far away or quite likely very far away. I mean, at least if you think the universe is infinite and there are like infinite and many planets there, then it would be a virtual certainty, right, that it would exist on.

Starting point is 00:44:23 Unless there's a huge barrier to it. And I think, well, even if there were a huge barrier, like if there were a finite probability and you have like an infinite number of roles of the dice, you would get any possible outcome, not just a few times, but in fact infinitely many times. Sure, sure. But right. But I think, you know, more important than that is, you know, it's great to talk about, yeah, the number of galaxies, hosting planets, hosted, you know, hosted around stars is truly astronomical, just in our observable universe. And now we're learning more about the...

Starting point is 00:44:54 Well, that's a big difference. Yeah. Right. So I don't like it. What we care about is not like, is there life inside of this galaxy behind me, M51, the World Book galaxy. You know, that's never, it's totally, you know, disconnected causally from any of our hopeful lifetimes, right? But, well, you might care about both. Like observable universe is you care about because, well, A, things that are in the observable universe, you know, maybe they will, we will observe them.

Starting point is 00:45:20 Or they could interact with us or influence us or like, so that they have special sort of relevance to our prudential interest. But what happens outside the observable universe could also be of interest. One, you might just care about it for its own sake. But for the same reason, you might care about people on the opposite side of the world. things to go well for them, even if you're never going to meet them, right? You just think it would be, but also from a theoretical point of view, our beliefs about what is going on outside the observable universe might also indirectly influence what you should believe about other aspects of what's going on inside the observable universe, since it's all part of the same

Starting point is 00:46:03 physical existence. So there might be sort of theoretical implications as well. Yes, although it's confusing to me, you know, it becomes sort of infinitely solipsistic because it's not clear if you can have a different universe if the laws of physics, A, old as many people believe they wouldn't necessarily maintain from multiverse, you know, universe to universes, Max Tagmark, your fellow suite has talked about. But more than that, it's not clear that even the laws of logic or, you know, Hilbert spaces and why would they be guaranteed to, you know, even the concept of such a thing is not really decidable, right? Because you can't just say, well, the speed of light would be 20% higher in universe 64 over there. But how do you know that modus tollens would hold?

Starting point is 00:46:54 I mean, are these not things that could be open? If you're saying infinitely many things can happen, then why shouldn't, you know, the laws of logic fail? Well, I think they have a different status as arguably analytic truths that might not be violated in any possible world. You could have different systems of logic.

Starting point is 00:47:22 You could have a trivalent system of logic and stuff like that. But I think they possibly are to be placed in a different category than the laws of physics that certainly one could imagine being different than they are. And maybe they are different in some other real universes out there.

Starting point is 00:47:53 But even if there were not such universes in actual physical existence where the loss of physics were different, even if we're just talking about our standard Big Bang cosmology with the loss of physics as we know them, you could still have a situation with infinitely many planets. If it does have kind of an open or flat universe with the simplest topology, it just goes on as it were forever,

Starting point is 00:48:21 and there's a part of it that we can see, the observable universe. Light has had time to get there, but beyond that, there is no reason to think there are not more galaxies just continuing out in perpetuate. You said this place was steps from the water. We just haven't found the steps yet.

Starting point is 00:48:39 How much did we save? Enough. Enough to get lost. Or you could book a stay with Hilton. Welcome to your oceanfront room. Just steps from the water. The Hilton sale is on now. Book on Hilton.com or the Hilton app and save up to 20% to get the stay you expected.

Starting point is 00:48:58 When you want savings, not surprises. It matters where you stay. Hilton for the stay. What do you make of those that suggest? that like the multiverse, which is what you're describing, that the simulation hypothesis or argument is not falsifiable. And therefore, in their words, I'm not saying this, it's not part of the domain of science, which is admittedly very Paparian demarcation, you know, dialectical. But what do you make of these, you know, criticisms? Like, you can't falsify the simulation

Starting point is 00:49:29 hypothesis because it could all be simulations and turtles all the way down. How do you react to those kinds of criticisms. Yeah, I think there are certainly possible observations that would increase or decrease the probability of the simulation hypothesis. For example, take the limiting case where a big window pops up in front of you saying you're in a simulation, take care for more information or something like that. That would be pretty strong evidence. I don't know, though, Nick, because I talk about, you know, I'm.

Starting point is 00:50:03 I'm a practicing Jew, and we make note in our Torah and the Bible, the Old Testament, whatever you want to call it, just a few days, weeks after the Jews allegedly witnessed the splitting of the sea in Egypt, allowing them to traverse on their way to go to the land of Israel, that they missed Moses for 40 days. He was about six hours late from returning from communicating with God and getting the Ten Commandments. And in that six hour period, after 40 days after witnessing this miracle among all miracles, they made a golden calf. Now, that, you know, I think, in other words, I think if somebody said they saw this click

Starting point is 00:50:39 here button, just as people say about the, are they were hallucinating or there was a wind and there was some mass delusion or it didn't happen, I think there are a lot of ways that people would reject, you know, quote unquote, observational eyewitness evidence, but certainly that, but maybe even physical, quote unquote, evidence because, you know, it could all be a brain in a jar. So do you? Well, a brain in a jar is not that different from the civilization argument. But I mean, I think then you have to go into the, like, estimating the cost of producing brains and jars versus creating simulated minds in virtual realities, where I argue that it would be a lot cheaper for a technologically mature civilization to run the whole thing in silicon in silica.

Starting point is 00:51:22 So it would just be a more efficient way to generate. And then you would generate more of them. So even if I were some brains in jars, which that might well be, that would be more. people with our experience that would be in digital simulations. And so we should expect, if we're one of those two kinds, we would more likely be one of the kind that has many more instances. But yeah, no, I mean, certainly if it were some random person reporting that they had seen a big window popping up, like, I agree with you that we should be rather skeptical of that. I mean, in fact, I do get a lot of reports to run people emailing that I've seen with all kinds of things, right, pixels in their bathroom mirrors or whatever. But I think that these kinds of reports, you would be likely to see whether or not we are in simulation and are much more likely to be explained.

Starting point is 00:52:20 Even if we were in a simulation, the most likely explanation for those kinds of reports, which will be the normal psychological explanation that people have had a visual hallucination or they lie or they misremember or they misinterpret, like all of the standard things. There's no reason why not all of those would also apply to a simulated civilization. I wonder if you've heard of something called E-ROM's Law or E-Rooms Law, which is Moore's Law spelled backwards. And it states that the number of new drugs approved per billion dollars spent on research and development gets cut in half every nine years. So it's the opposite of Moore's Law, which computing

Starting point is 00:53:01 power doubles every 18 months or whatever it is. I've heard it said that these computers, they have more terraflops or whatever, but they're not actually by the higher order metrics of, you know, H index or some publications or some other metric. They're not actually, they're kind of saturating in terms of the net scientific output, not the throughput, instantaneous capability. Is that not, you know, kind of an obstacle or a great filter, if you like, for the simulation hypothesis itself. So what is saturating exactly? The number of either the final product like distillation,

Starting point is 00:53:41 final computer simulations, final nuclear web bomb simulation, the amount of the, because there's so many more end users at heart who want to use the technology, there's also like hackers that are trying to exploit it. In other words, there are other uses rather than just the pure, intended use of the computer as a silicon device, there's more uses for it. And so it's saturating in terms of its final output, number of publications. Let's just use that, a higher order statistic, rather than just a lower order, but still important, tarifflobs. Wouldn't that happen? In other words,

Starting point is 00:54:15 you get better and better. Now you really want to simulate more and more. And so now the tax on the computing hardware is going up also exponentially. And so you have this ratio of two exponentials that saturates. I'm not sure exactly. If you look at, I don't know, like the number of video frames that humanity is producing. I'm sure it's growing rapidly. And the number of photographs taken is, I think,

Starting point is 00:54:43 still increasing the amount of compute that is being devoted to artificial intelligence applications has been growing a lot faster than more slow over the past years. how to assess the output of that. Of course, that's a kind of qualitative judgment. You could have specific benchmarks where you can plot progress, like on classifying dog and cat images, what's the error rate? And you can see in specific domains like go playing capability or just playing capability

Starting point is 00:55:19 and you do see improvements there. It tends to saturate if you're measuring at the error rate, because it has sort of a lower bound of zero or sometimes intrinsically in the data set there is like a minimum possible error rates. So as it approaches that, it has to saturate. But then there are new domains to conquer and my take, if you just look qualitatively at AI, AI achievement impressiveness in some sort of just guts level sense of, do you get the sense for wow?

Starting point is 00:55:56 when you're looking at what's being achieved each year. I don't have a sense of that having stagnated in AI in the last few years. I mean, from the start of the deep learning revolution, it seems like every year or two we have some other major intuitively impressive achievement. Most recently, I guess the alpha fold folding of all these proteins and then the large language models and then these, of Dali-style systems and recently some kind of reasoning systems built on top of the large language

Starting point is 00:56:34 models as well that are now solving kind of math, math problems. There's a code-writing system, like all of these things just at the gut level, I think, feels pretty impressive and with no particular sense of us having plateaued. I think we rather seem to be in the midst of some something very rapid unfolding of potential there. Maybe it will, it will possibly stagnate at some point before maybe it gets going again, but I don't think we are in the midst of any stagnation in AI currently. I wonder, I know it's getting late there. Do you have time for one more segment about risks in the future?

Starting point is 00:57:18 Well, I could take one more question. Okay. All right, I'm going to do, I'm going to parlay this into a question regarding risk, but also the simulation hypothesis. So we've had COVID. I see kids coming, not me personally, thankfully, but I see kids on my campus biking around, not wearing a helmet, using their phone, and then wearing a mask. And I guess the question that one of my audience members is asking is that given the super intelligent concept of a singleton, which is not a simpleton, but a singleton in which any entity that has complete world domination such that can eliminate all threats to it exists,

Starting point is 00:57:56 It could be a government or super intelligent AI. And once it arises, it cannot be replaced because it will systematically destroy any opposition. Now, I want to ask you, given that COVID is kind of the first ever, you know, worldwide, fully documented video, audio, you know, virtual reality, in any possible way, it's kind of the first of a, you know, Primas into Paris. Is this sort of pointing at perhaps a government like the CCP or something like that, that they are approaching it? And if they did, would it not behoove whoever's not, you know, in the cusp of being a singleton to destroy that to minimize risk? So I know it's a multi-part question involving COVID, Chinese, you know, government and so forth. But if you felt there was a chance for folks to or for, you know, government like CCP to come up with this, one of my listeners, JT. I Alexandria is asking what would be, or sorry, Joseph Billing, is asking, should we kill the singleton before, you know, while it's still a newborn? Well, the concept of a single-thon is a somewhat more abstract one.

Starting point is 00:59:02 So it's a world order in which at the highest level of organization, there is a single decision-making entity such that global coordination problems are solved. So that can't be sort of two single tons on Earth, right? By definition, it's a kind of a global world order. And so it's to some extent a matter of degree. We have various weaker forms of international collaboration, but they are far from perfect. So we still have global coordination problems that we haven't solved that results in the fact that we have militaries that are preparing to kill the people in the other militaries, that we have global warming, that we have overfishing, that we have a whole bunch of problems that arise fundamentally from our ability to get along with one another. Within nations, we have, as it were, the analog of

Starting point is 01:00:03 singletons, but within a limited jurisdiction. So yeah, so the Communist Party in China would be a singleton in China. The U.S. government would be kind of a singleton in the U.S. and it's almost the definition of a state that it's an entity that has a monopoly of violence within its territory, you know, give or take, but that's approximately the case. We don't have any analogous thing at the global level. The different organizations we have are very weak, like the UNIS week, the international trade organization. There are these kind of weak overlapping-

Starting point is 01:00:36 World Economic Forum. Yeah, they're very weak. They may be more than zero, but they are less powerful over the globe than the United States government is over the United States or the Communist Party is over China. And so now it's an interesting question whether we, to the extent that we have some choice in the matter should favor or disfavor the emergence of a single song. Like that is some kind of state-like, and it doesn't need to be a state-like entity. You said it could be a superintendence. It could also be in theory other kinds of things, like a kind of universal moral code that was self-enforcing.

Starting point is 01:01:16 So it's an abstract concept that could be instantiated in many different ways. Should we be in favor or opposed to this? I think in theory, a singleton could be very bad or very harmful. Like, the concept itself doesn't determine that. I think it's quite likely that there will eventually be a singleton by no means certain. And on the margin, should we favorite or disfavor it? very hard to say. I mean, I kind of tend to think that a lot of the biggest existential risks will arise from the fundamental fracturing of humanity, that the possibility of conflict

Starting point is 01:01:59 or of one part of humanity doing something that is against the interest of another part, new weapon systems, new arms races, new kinds of risk-taking that you're doing competition with other superpowers. If that's the biggest source of existential risk, then it's possible that yeah having something that reduces that would be net positive even though certainly it is the case that the singleton itself would be another big source of existential risk like if it turns bad then you know right it's all the eggs in one basket uh so it's a question of weighing these two different kinds of risk but i think the risks from conflict looks plausibly dominant over the risk of uh oppression but i mean uh that's a hard judgment

Starting point is 01:02:45 call. And last question, if you'll indulge me for one second, Elon Musk has talked a lot about the great filter, super AI intelligence and stuff. But he's also talked a lot about underpopulation and depopulation. What's your stance there? Should a young person have children right now in Europe? I don't know even know if you have children,

Starting point is 01:03:04 but should somebody ethically have children? I think seems fine. I mean, I think there are, if to the extent that you want to do the more on the optimal thing, there are many things you could. do besides having children, but I certainly don't think there is any moral reason to refrain from it. I worry less about population decline because I'm thinking probably before that has, it's a kind of slow and gradual thing that the world will be dramatically transformed before that has had a time to play itself out. And in the longer run, I think that would be

Starting point is 01:03:42 powerful evolutionary selection for higher fertility that would eventually counterfeit. interact these cultural trends, most likely. We already see selection, say, in favor of reduced female education and stuff, have higher fertility. Fundamentalist, many fundamentalist communities have higher fertility. And there will be all kinds of other partially inheritable traits that will impact fertility, and those will be on the positive selection. So over longer timeframes, like if we're talking hundreds of years and stuff, then eventually,

Starting point is 01:04:18 I think some group will emerge that either for biological reasons or more likely for cultural reasons, just have managed to find some way to retain high fertility even in a sort of advanced technological context. Interesting. So humanity does have a future in addition to your institute, which you direct as founder. Nick, I want to thank you so much. Your work is always so inspiring. It makes me think. It makes me turn different ideas in my head. My audience loves your mind and we hope to have you back someday. Let's be fun.

Starting point is 01:04:48 Thank you for having me. Thank you. Any sufficiently advanced technology is indistinguishable from magic. Well, that's a wrap. That was tremendous. That was really, as I said, a bucketless conversation. I hope it don't kick the bucket for quite some time. Maybe we'll get them back.

Starting point is 01:05:05 Maybe we won't. At any rate, I want to thank you for submitting questions. Remind you, you can submit questions to me via my mailing list, Briancadding.com slash list, where you might win a little piece of proto-solar system, planetary debris, a meteorite, if you're among the 100 lucky winners, on the next run, printing, mailing of meteorites that will go out soon. Otherwise, I want you to subscribe to my YouTube channel where you can see the interview and some background footage. We do different things on the audio podcast versus the video podcast. Give you two bites at the impossible Apple so that

Starting point is 01:05:36 you can enjoy whatever your listening mode. But if since you aren't listening to this, as I know, because this is on my podcast-only format, I want to ask you to do a favor last done for me, a very loyal listener who goes by the name of J-Bro 22 in the US of A who wrote headline, I love this show. And he went out to say, as an amateur, this show feeds my need for more knowledge and context, and it's giving me better perspective and information than other sources. It's challenging yet accessible. The only problem is that, as a newer subscriber, my reading list is growing faster as I catch up

Starting point is 01:06:08 to all the content. Well, thank you, Jay Bro. That means the multiverse to me. I hope you will all out there do the same favor as, Jay Bro, you can do that now on Audible, where you get Audible Podcasts, so do that there, as well as on Apple Podcasts. And reviewers and publicists and all sorts of folks, look at those reviews in the number count, which is about 430, 440 by now, in the USA, and have 570 worldwide ratings. And they look, you know, should we have this author like Nick Bostrom? It took two years to get them on the show.

Starting point is 01:06:37 And they look at subscriber count, review counts, things like that. So if you want to hear phenomenal conversations like that, please do, leave a quick rating in a review anywhere you get your, podcast and subscribe to my channel and my mailing list so we can help grow this multiverse of minds throughout the known limits of the human mind. I want to thank you for going into the impossible. Stay tuned. Remember, you can always leave a comment. We're going to have an AMA episode not too long from now. There's a little recording icon on my website, bryankeating.com slash podcast. Also sign up there for my mailing list. So I do hope you'll do that and indulge me these delightful replies. I love hearing from you. Looking forward to new episodes, including

Starting point is 01:07:15 Dr. Sean Carroll with his wonderful new book, The Biggest Ideas in the Universe, and many, many more. But for now, stay tuned to this space and enjoy the rest of your time and have a magical, magical week. Thanks, everyone. Ambition comes in all shapes and sizes. At First Citizens Bank, we roll with your goals because we're built for what you're building.

Starting point is 01:07:54 Fit for your ambition for Citizens Bank.

Into the Impossible With Brian Keating - Nick Bostrom: Superintelligence (#256)

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.