The Comedy Cellar: Live from the Table - AI Is Self-Evolving, and Nobody Understands It - Bioweapons, China Race, Sentience | Robert Wright

Episode Date: July 10, 2026

Description Written By Fable/Max: Robert Wright interviewed Geoffrey Hinton in 1983 and, by his own account, got the story 180 degrees wrong. Forty years later Hinton is the "Godfather of AI," two of the three godfathers are frightened of their own creation, and Wright has written The God Test to figure out what he missed and what's coming. We talk about how these machines evolved rather than being programmed - nobody fully understands them, including the people who build them - why "fancy autocomplete" was always wrong, the best case (an AI caught a radiologist's error in his cancer MRI), the worst case (a bioweapon that incubates silently for six weeks), why racing China to superintelligence may cause the war it's meant to prevent, and whether unplugging a machine could ever be murder. Plus an experiment of my own: I put a legal dispute in front of the AI and handed the laptop to the other side. 0:00 Intro - Robert Wright and The God Test 2:13 A 27-ton computer and a buried headline 4:11 Interviewing the Godfather of AI in 1983 - and missing it 10:15 Not programmed - evolved 17:27 The "fancy autocomplete" myth 24:17 Dukakis, Reagan, and how minds file words 35:49 Best case: medicine, education, and an MRI story 42:38 Worst case: bioweapons, jailbreaks, designer babies 48:51 An AI that believes what the Ayatollah believes 50:30 The China race - overdone and dangerous? 57:33 Nazis, the Hamas charter, and bombing over AI 59:46 Authoritarianism through the back door 1:03:35 Sociopaths, the ship of Theseus, and machine morality 1:14:34 Is it sentient? Should you be nice to it? 1:21:54 AI as judge: my legal experiment 1:26:20 The God Test

Good evening and welcome to live from the table. My name is Norm Dorman. I'm the owner of the comedy seller. I'm here all by myself today in Wells, Maine, which is a way it might be for the summer because everybody's dispersed. But we have a fantastic guest today. My guest tonight is a friend of mine.

I'd like to call him a friend. His name is Robert Wright, author of A moral animal, non-zero, white Buddhism is the truth, the evolution of God, and his later. latest book, The God Test. I should have

the whole the God test, artificial intelligence and our coming cosmic reckoning. And welcome to the show, Robert Wright. You were also involved in Blogging Heads TV, right? You're responsible for the Glenn Lowry podcast?

I mean, I think Glenn deserves some credit for that, but yeah, no, I co-founded Blogging Head's TV with Mickey Kouse and a tech guy named Greg Dingle. In 2005, Yeah, we had a video platform. It's actually the site still exists, but that has kind of evolved into the non-zero podcast. The podcast still do appear on something called bloggingheads.tv as it happens kind of for old time's sake.

But yep, it's true. I was in the game early. This is the issue of the day, artificial intelligence. And I have to confess, I'm about halfway through your book. I intended to read the whole thing just as an act of loyalty to someone that I have a relationship with, and I fell into a vortex of potential legal problems and stuff like this with the construction I'm doing. But in that process that I'm in, I also got some new insights into AI because I've been using it extensively and miraculously in researching. I mean, I would definitely tell my kids not to become lawyers, but we'll get to that.

We'll get to that. So, okay, this is the part of the book, which I want to start from. Okay. There's a sentence early in the book. It says, a computer with one billionth, in 1947, a computer with one billionth of the processing power of the phone in your pocket, weighed 27 tons and filled a large room. Now, 47, 1947 is not ancient history. My parents were adults then.

I think your parents were probably adults then. I did some research on it. That same year, as luck would have it, the transistor was announced. It warranted a little paragraph in a, you can look it up in the New York Times, in a section of radio gossip. It wasn't even a headline. It was just like the last little entry.

For it 79 years is 2105. And if we've learned anything, it's that the experts absolutely underestimate where these things are going. The science fiction writers seem to score better. And also about the same time ago was the Wright brothers, or same distance from the moon landing was the Wright brothers' first flight. So again, nobody saw that coming, or very few people did. And the only difference is that 79 years of human invention may have nothing to do with 79 years of advancement fueled by AI inventing itself. So we can't even imagine. I don't think we can even imagine where this is going.

and that is the thing I keep thinking about. You say earlier in the book that you interviewed this guy, Jeffrey Hinton, about neural networks in 1983, and you completely missed it at that point. So let's just start there. What do you think about everything I've said, and why did you miss it with Hinton? And what have you learned after all this research in your book? So yeah, it's true.

I mean, the amazing growth in computing power that you noted has actually happened. That number is one billionth. That's insane. I mean, you have to have a pretty advanced smartphone for that to be the case. But, yeah, it's true. The computers back then had vacuum tubes, because as you said, until the transistor and subsequently the microprocessor. were invented, you know, vacuum tubes or the thing. Yeah, so that, Jeffrey Hinton, I interviewed him in 1983.

That's old. I am as a young journalist writing a piece on AI for something that was called the Wilson Quarterly. And then, you know, and I kind of kept up a little with AI in the meanwhile. I wrote about technology on and off. I did, you know, when the IBM computer beat the world chess champion, Gary. I'm sorry. Say who Jeffrey Hinton is. I neglected to.

Oh, I'm sorry. Well, that's kind of the punchline is that in, in 2023, I would open the New York Times. And there's this headline, Godfather of AI, they put that in quotes in the headline, something like quits Google and warns of dangers of AI. And that was Jeffrey Hinton. and the reason he was called the godfather of AI is because he was, you know, more responsible than any other person, I would say, although it was a, you know, there were a lot of people who contributed, but more responsible than any other single person to what's called the deep learning revolution involving these neural networks. And that wound up being, you know, the thing that's responsible for the so-called generative AI of the last few years, large language, these chatbots, image recognition, video generation on the basis of a verbal prompt. All that comes out of the school of thought he was championing when I wrote about AI in 1983. At that point, it was kind of a maverick movement that he was advocating.

It was not mainstream at all. In fact, I remember the reason I called him to talk to him was I was talking to one of his colleagues. remember who. And, you know, we were talking about this kind of fringe movement with neural networks, and the guy said, oh, if you want to hear the gospel about neural networks, you should talk to Jeff Hinton. So he was this enthusiastic evangelist for this thing that wound up, this school of thought that wound up prevailing. But now he's very, very deeply worried. And not just about the kind of mundane impact of AI, and which is one thing I'm concerned about is just the magnitude of the impact across a number of different kind of social, economic and political fronts.

But he's worried about the sci-fi scenarios where the, you know, the AI gets out of control, takes over maybe, has no place for humans in its plans, and ultimately Silicon intelligence displaces our intelligence. And as a note in the book, you know, if you Google Godfathers of AI, at least when I last did it, you would get three people. You would get Hinton, Jan Lacoon, and a guy named Joshua Benjiio. These three shared the Turing Award, the big award in computer science one year for the deep learning revolution. And two of the three are diswired. Joshua Benchio shares Hinton's concerns.

So he's not just some flake. And in fact, some of the people most conversant in the technology are most worried not just about the magnitude of the impact, not just about things that everyone is starting to understand the danger of like somebody uses AI to build a bioweapon, somebody, you know, builds a self-replicating super hacker that takes out all the satellites or something. But these guys are also, they're worried about those, but they're also worried about the sci-fi scenario. So the book is like, I went, when I saw this headline, I go back and I read the article I'd written in 83, and I realized that I had not understood the potential of his approach. In fact, I was like about 180 degrees away from comprehending the significance of it. And, you know, that became clear once I listened to a lecture he had delivered in 2018. So I went back and looked at kind of the whole story and looked at the risk.

and the possible benefits of AI and laid out for lay people. That's what I've done for much of my life is right about technical subjects for lay audience. And tried to explain clearly what the secret sauce is that Jeffrey Hinton was advocating and why it is going to keep leading to growth in capabilities of AI. and the miraculous things that could do and the terrifying things it could do. And that's what a lot of the book is and what we can do to steer the future toward closer to miraculous than terrifying. So let me, I should have said at the top, the book is tremendous. You know, I've been reading it and it's not at all a chore as most books I read.

And the first part of it, you spend a lot of time explaining, how AI works, both in how it evolved on its own. Like anybody who's done coding is going to suffer from having too much information that they can't get out from under. Like, I'm just used to coding as if-then statements and arrays and objects and blah, blah, blah.

But you write about the fact that this really evolved through trial and the computer's own trial and error in a sense and taught itself everything that's yeah well i mean the uh the people like hinton and then and the modern day airs of hinton they build the machines so that through trial and error they will get better at certain tasks whether it's image recognition or bill you know guess the next word in this sentence or whatever But, yeah, through this trial and error, you know, the internal settings of the machine change, the so-called, especially the so-called weights, which is to say the strengths of connections among the so-called neurons. There is an analogy to the brain.

And it is kind of like evolution. I emphasize that in particular. I think it's underappreciated that when they train these things, it isn't just like learning. It's like evolution. in the sense that, you know, evolution is a process of trial and error. These genetic mutations don't pan out. These do, and slowly you get these super sophisticated machines. Same thing happens with these.

And so, you know, by getting better at these tests, these machines build or discover and preserve, however you want to put it, internal structures of cognition and perception that are functionally like the structure. structures in our brains. And the human beings don't have to understand them. And they still don't fully understand them. They're trying to learn more about what's going on. But it's taken a while. I'll give you a very simple example involving image recognition. You know, we have these neurons that are called edge detectors. So, you know, one neuron may fire if there's a line of contrast, it's vertical. One may fire, a different one may fire up to the line that's horizontal, a different one if

it's closer to 90 degrees. And this is how we start at the very beginning of apprehending the visual world, you know, taking in a scene. We start making out like where the edges are and stuff like that, right? And so with image recognition, in that case, they just, they show it an image of a dog. and if it says, you know, if the output at the bottom of the kind of neural network, they feed image of dog into the top after turning it into numbers. If the machine says like cat or anything else, they're like, no, sorry, try again.

And it will adjust itself internally, one of these connections, a neuronal connection, you could call it, changing the weight of various neuronal connections until it gets better. and eventually when these machines, we take it for granted now. It seemed like a miracle when I first saw that AI could spot images of dogs, but when they get good at this, we've discovered, we didn't build this into them intentionally, but we've discovered that they have built their equivalent of edge detector neurons, okay? Just because like evolution kind of discovered, you could say that that's a very, natural and efficient way to start making sense of an image. Where are the sharp contrasts? What is

their orientation? If you're going to start building a, you know, a three-dimensional from all a world, good place to start. And these machines discovered, too, that at the beginning of the process, that is, in the early layers of neurons, they needed edge detectors. Nobody told them that, nobody built it into them. We kind of discovered after they got good at this, that that's what they had done. And the same goes for language. I mean, And, you know, so it's, and I don't think this is really adequately understood that this is why it's advanced so fast, we'll keep advancing so fast, is because we don't need to understand how the human mind works to make them work like the human mind. Yeah, and, you know, so just two things. And I want to get to the doomsday stuff, because I know that's what people are really going to be riveted by.

But as a guy who's into computers, I just, I'm fascinated by this stuff. the developers don't know how these things work themselves, and they call it training, but from your description, it's more like just seeing what it does and maybe putting some guardrails on it and trying to guide it, but it's not really training.

It's like in computer programs, you have debugging. You press a keyboard shortcut, and it brings up the code, and you step through the code, and you can correct the mistakes. You can't debug lines of code here. So I just find it, I just find it so fascinating. Yeah.

I mean, they are trying to learn how to do that, and they have had some success. So there's a field called interpretability, which means understanding what's going on inside them, and a field called alignment, which is like making use of that knowledge to try to make sure that they're going to behave in ways that are aligned with human goals and values, although it's clear that that's going to be challenging. But so through interpretability, like just discovering these edge detection neurons is an example. And once they do discover these things, they can manipulate them to alter the behavior of the thing.

I mean, another early example is they were, like years ago, super simple large language model was just trying to predict the next word in Amazon customer reviews, okay? And one thing they discovered is that, and it got better and better, and they didn't know how, didn't know what was inside it, but they did then discover that there was a neuron that was a good indicator of like the emotional valence of the review. In other words, positive or negative? Do they like, does the reviewer like the product or not? And first of all, that's interesting because, you know, we have a kind of version of those. I mean, the human brain has built in equipment that makes it good at sensing the emotional tenor of other human beings,

because that really mattered during evolution, like whether somebody's pissed at you or likes you, right? So in a certain sense, that's something that we had. But then what they discovered is by manipulating the neuron, they could get the thing to generate a positive review or a negative review. And there are similar things being done with respect to, like, honesty. They're trying to find kind of the knobs you fiddle with, and they've had some success

to make sure that they'll be honest and so on. So one of the things you debunk, and by the way, you had some really nice illustrations there about exactly what you're talking about, where it's like you have the first two words, piece. This product was a piece, and it could

have been of shit, or appears results and how the computer had to infer that. But one thing you put to bed, and I knew this wasn't right, the first time I heard it when some serious AI guy was on Sam Harris's podcast. AI is not just a turbocharged auto-complete, which is what they tried to tell us so we wouldn't be so worried about it. It is far more than that. And I knew that wasn't true because it was giving me logical answers to things and it was analyzing arguments and i was even early on uploading like a transcript i would have versus an article that was written by the same person i'd ask chaty pt to contrast them and in a very high level and this is already like 18 months ago it was

coming back with very sophisticated arguments so the whole auto complete thing never made sense to me and um this will be the last question about the technology under the hood, because I didn't really get this yet from your book. I understand from the book very much how it puts properties on words and weights them. As I say, I understand, it makes sense to me. But I'm not getting, how does it analyze arguments and logic? Yeah. So, we don't, the answer is we don't entirely know yet, but it's important to talk about how it comes

to do that. So first of all, there's the next word prediction training, which everyone's kind of familiar with. That leads to this kind of pejorative description that it's just fancy auto-complete or stochastic parrot. And the thing that's wrong with that is that, you know, we now understand that it has actually a system for representing the meaning of words. I won't get into it. But this is the fundamental misconception I was under when I wrote that 83 piece. I went back and read it. And I can tell that I was imagining that if there was ever an AI that could process language, some human who understood the meaning of the words would have to somehow transplant that into the machine.

We would come up with a system for representing the meaning and we'd fill in the blanks. You know, we'd basically in a certain sense, you know, put a dictionary in it. But we'd have to do that itself. We are the ones who know the connection with words and meaning. Well, it turns out that if it even gets good at this next word prediction task, it builds a system for representing meaning that we did not tell it to build per se. All we said is, look, you alter these connections. You alter these, in this case, a lot of it is the so-called word embeddings, the numbers that represent a word, a lot of numbers separated by commas. You keep fiddling with these and changing them, whatever what numbers you want, until you get good at this task.

then we realized exactly how it is finding numbers that amount to kind of implicitly a way to represent the words. I can get into that. If people want how it's done, it's fascinating. And actually, it's a way of representing words that psychologists have long speculated might be the way the human brain does it. And it might be, but we still don't know that. Now, so that happens, you know, just the auto-complete thing along.

gets it to build that kind of equipment. Now, you ask, how does it get so good at reasoning? You know, we're learning more and more about this, but an important thing to emphasize, and I should have emphasized this more than I did in the book. I, you know, spent a paragraph and should have spent three or something that, you know, there is this post-training, it's so-called post-training phase. It's all just part of the training for practical purposes, but where they do a lot of other things like they give it questions and there's a right answer and a wrong answer and answers that are closer to right than others. And it's the same thing. Trial and error, get better at doing this.

And they can give it any kind of questions. They can give it substantive questions in a science like biology. They can give it logical questions. They did find, to begin with, that there's something. called chain of thought reasoning that it's kind of naturally good at to some extent without special training. And this is one of the weird things is the so-called emergent properties where, I don't mean weird in the sense of spooky and who know, you know, it's just a function of the technology that you get it to do certain kinds of things. And in order to do them well,

it turns out it has to get a little better at other kinds of things. So the initial discovery that if you just said to it, think step by step. You give it a logical puzzle and say think step by step, it would do a little better. Then they decided to take that capability and refine it through the, you know, this so-called post-training, which again can be a lot of things. You know, a lot of it comes under the rubric of so-called reinforcement. learning where with each response it gives you give it either a thumbs up or a thumbs down or something in between but the part

of your answer that the part of the answer of your question I'm confident of is you have to remember it isn't just trained to predict the next word we we train it to do specific things that human brains do in so-called post-training

and that's the way it builds a lot of the equipment that's becoming super impressive. I mean, it's like solving math theorems that have gone unsolved for decades. And, you know, I gather, Gnome, that you already kind of get the picture about how impressive this stuff is, because some people still don't. Oh, I get it. By the way, a little thought I had reading the book, you know, as, I don't know if you have this issue. I'm 60 I'm going to be 64 in a couple weeks and already for like 10 years now I noticed that I

have trouble remembering names sometimes and when you begin to have trouble with your memory you actually gain an insight into your memory because you can start noticing the things which come to your mind in lieu of that name so like you can't remember the name of Michael decacus the former governor who ran for president but you can remember the initials MD. You can remember, oh, he was the cousin of an actress, and you start remembering all these things. You can sometimes you remember it rhymes with something. And you realize, oh, in some way these things are filed in the brain. Right. Right.

Near or with these various properties. And this is very much the way you're, you describe how AI organizes things within its own mind, as it were. Correct? Yeah. There's, That leads to actually a couple of things. And one is just an interesting fact about the human mind that evolutionary psychologists have emphasized that if you look at things we do remember about things we can't remember.

Like there's a, you know, Ronald Reagan developed, I guess it was Alzheimer's, but anyway, some form of dementia. And his wife brought George Schultz, who had been his secretary of state in the room. And he didn't, then they talked. for a while and Shultz left and Reagan still didn't totally get to it but he said to his wife he's a very important man isn't he and that's one dimension importance like right I mean it makes sense in terms of human evolution we are a status conscious species and like so there are certain things that are really important about a person and these are some of the dominant dimensions but

to get to what I think you're asking about It turns out that these machines map words according to lots and lots of dimensions. So the example I give in the book is suppose you're mapping animal names like tiger and rattlesnake. And let's just look at two dimensions, you know, speed and degree of let's beaithality. They're both pretty high on degree of lethality. The snake is much lower along speed, tarantula, even lower. And, you know, that's two dimensions. But you could add things like furriness or, you know, number of, you know, of legs or scaliness or size or, and you go on and on.

And it has long been a theory that humans represent the meaning of words by having in high dimensional space, in some abstract sense, they've broken the words down into these features that constitute them, right? and like animate in animate all kinds of things and it turns out that that's the thing these machines do in the course of the training and again I just want to stop in Marvel I'm sorry but nobody said to them this is the meaning of words nobody said to them there is a thing called meaning

all we said is get better here's a string of what to you is complete gibberish these symbols we call letters we're going to show you this can you predict what comes next no, we'll then change some of your neuronal connections and we'll see if you get better. You get, okay, you're getting closer, we'll keep these, try again. That's all we did is get better at predicting gibberish.

And when you think about it in a way, it's not surprising. If it's going to get really good at that, it would help to have a way of representing meaning. And that's what it builds. Yeah. And the key, and that's why with any of these, it's like self-driving cars. same basic idea we don't need to understand how a human brain is working when it steers the wheel to the right we just need to show the machine the visual scene that led the person to steer the wheel to the right steering the wheel to the right is the right answer you train the machine until that's what it does in response to that visual data

and it will build the functional equivalent of a driver's brain that's i i hope i'm not sounding too intense about this, but it still blows me away that it, because this was a fundamental misconception at the very founding of artificial intelligence in the late 50s. Almost everyone thought, and you can see this in the proposal for the first, the Dartmouth conference where the term artificial intelligence was coined, the proposal for the conference says something about like the premise of this conference is that, you know, everything the human mind does can, in principle, be so precisely described that a machine can be made to do it. Well, it says you don't have to describe it at all. You don't have to, you don't even have to describe some other machine that's not exactly like

the human mind, but would still do what it does. You don't have to get into the internal workings at all. We have invented machines that will figure that part out. By the way, Bob, it also, in my mind, it also works the other way, which, is that here's this computer through a very, very quick trial and error is developing certain systems. And it's more than a coincidence that evolution through millions of years actually took the same path. Now we're seeing that the way we're organized is the smartest, could be the smartest way we could have been organized through the brilliance of evolution, because now we have have a computer doing it, and it's organizing things the same way. We didn't really know that.

Like, it could have just, it could have been that AI came up with this completely new way of doing this. And we said, oh, isn't it weird that our human brains don't do it that way? How smart is that? No, actually, we, we might be doing it in the smartest way. So it's an insight to me. Well, given what we want to do, I mean, the human brain, it's funny, it's designed to do a lot of things other than just be smart in the pure sense, you know, it was, but, but yeah, I mean, if you're training, if you're trying to get it to do what human brains do, like generate the output that human brains generate, given the input in response to which human brains generate it, it's a little less surprising. I mean, in a way, evolution had kind of exhibited this itself.

Certain, like, smart ways to get certain things done, get reinvented in evolution. Like, like the, an eyeball like ours, you know, works broadly like ours. I think it's the octopus. it has it, and those are independent instances of evolution. And then eyesight itself, you know, including all other forms, has been invented multiple, multiple times because it's just a useful thing to have. If you're an animal, you know, to be picking up on the patterns of light, they say a lot about your environment. So evolution, you know, it's called convergent evolution.

When evolution discovers something more than once independently, and you could view what happens inside these machines as another example of convergent evolution. Now, I will say, although in general, this sermon of mine about how in important ways this is more like evolution than in learning is central in a way to my message, it's true that in training, in training, you also see things done that in humans are done through learning, like the specific language it learns. It masters English. Well, that happens in the course of a human. human life and I'd been born in another country, I'd speak a different language. But the reason humans are so good at that is because of the linguistic machinery built into

our brains by natural selection. And during training, you're seeing both happen at once in effect. You're seeing some things we can do by virtue of evolution, or some equipment that got built in the course of evolution, and some forms of mastery that also depend heavily on learning. but it's all happening very fast. Last technical question, I promise. I notice, maybe I'm wrong about this. I notice that when I want to make a point, very often I somehow understand the point I want to make in a nonverbal way

or in a very, very pared down way. And then I can speak five, ten minutes of what it is of the point I wanted to make. but I didn't I didn't concoct that five or ten minutes, right? Is the AI, does it form a kernel of a point it wants to make first and then somehow serially spit out these paragraphs? Or is it happening as we're reading it? I cannot over-emphasize the fact that we don't know some of these things. And I'm not, you know, I'm not conversant on the latest. thing known. For example, Anthropic came out with a paper just this week where they're talking

about this new thing called J-space, and there's another name, Global Workspace. I haven't had time to read the paper. There's a video about it that's fascinating. If you just Google these things, you'll find it on YouTube, but they say that, okay, they're now, they've kind of separated the kinds of information processing it does that are part of our unconscious from the kinds that are part of our, more of our conscious reflection and they're getting some insights into the former or the unconscious. I mean, there's a lot of interesting things going on that, and things that they have learned that are important, but I suspect, you know, not having really boned up on it in the last few months, the field of things have asked, I suspect that the question you're asking is

really one of the deepest that we just don't yet know the answer to we may eventually and uh but the thing about it is the machines are advancing so fast that once we you know we're always learning about the machines that already exist and there's already a new one in training and so for practical purposes of like controlling them um i don't see us ever really being totally to speed in a sense. And so, but, but look, you're asking, it's fascinating, you know, it's funny, that last piece of that, uh, lessons that 1983 piece I wrote is, this is just quoting some guys. It's like, well, whatever happens with AI, just thinking about it, like clarifies some questions about the mind or something. And it does, right? I mean, you're asking a question about human

cognition and AI cognition. We don't know how what you're describing happens in us exactly. I mean, you have an intuition based on your conscious experience about kind of what's going on, and we're kind of figuring it out in machines. And I should say, it's not like everything's going to be exactly the same in machines. You know, our equipment evolved in a certain sequence. and in any event the equipment, even if it's functionally comparable between the AI and ours, it's not going to be exactly the same. So it's not like, you know, this is the human mind. But thinking about it should be of interest to people who are just interested in human psychology. I can tell that in you, it is tapping into both interests, right?

You're interested in computers. You're interested in the way your own mind works. They're both mysteries to some extent right now. but it's very fertile ground for speculation and experiment and discovery. And it's fascinating. All right. So it's a good entree. So not knowing how it works is fraught and scary.

What in your minds, tell us what is the best case for AI, our AI future, the flourishing case? and what is the worst case. And I just want to say one other thing in that question, to my mind, a relatively successful person in the modern era, including, you know, I'll even include the current level of AI, like life is so good, except for medical breakthroughs, except for medical breakthroughs so we could live longer and healthier and all that. Obviously, I don't have to tell the reasons why that's important.

I don't know what else it could drastically give me. Humans are still going to want to play music, read, think, blah, blah, blah. So I have trouble really understanding the fantastic AI future, except that it might lift many, many people out of poverty, which would be, or drudgery or weariness. That would be awesome. But I don't, maybe you can spell out that. optimistic view. The pessimistic view, I think I have an idea about, but I want you to tell us

about us. So give us both, the best case and the worst case. Okay. So best case, I mean, uh, it is contributing to scientific advance. As I stress in the book, that is upsides and downsides, uh, for a reason I can get into, but a lot of potential upsides curing pretty much all diseases could well happen. Um, that would, that would be nice, as you say. even in the nearer term, it has the advantage of potentially spreading the availability of things we take for granted. Like, just good medical advice. Like, if you or I have a symptom, it's not that hard. I mean, it wasn't even hard a few years ago, right?

For us, people with our resources to, like, find out whether something's seriously wrong. Well, there are people in the world for whom that's more challenging. And these things are already very, very good at if you just describe what you're feeling, at giving you some pretty good preliminary guidance. Better than doctors, I'd say, but go ahead, yeah. What's that? I'd say they're better than most doctors at this point, in my experience. I've got a little quick story.

As you know, I had cancer last year. I'm fine now, apparently. And the, but when I got the MRI report that gave me the bad news, like, yeah, these lymph nodes are trouble, blah, blah, blah. I said, what about, you know, because I hadn't, you get these reports. And you haven't yet talked to the doctor. They're in your portal. Reports and your metal report.

I don't know what it means. So I like to ask the AI. It's like, yeah, yeah, that doesn't look great. I said, but what about this sentence? Why didn't you mention this? This looks like a bad sign. It starts out abnormalities found in. And the AI says, I'm pretty sure that the radiologist meant to put no at the beginning

of that sentence. And it was right. Amazing. Now, when you look at the sentence, it's a little less amazing. But one thing I'm saying is, you know, humans make a lot of mistakes. So if you're asking, you know, if you're asking, well, will they take jobs? Will they, will some people prefer them to girlfriends, boyfriends, friends, and so on? Remember, they don't have to be perfectly satisfactory to be preferable, right?

And so in some cases. But, but yeah, no, I think it's completely amazing. It also helped me just research the cancer. in a way that led me to ask challenging questions to my doctor's, fruitful and challenging questions. And there's no doubt about that. And education, you know, you ask, well, what's the upside? You've probably, you know, I'm guessing you have had, like, gratifying experiences, just learning stuff through it, right? It's like amazing. It's like being able to summon the world's leading expert on something and ask them anything you want, which is like magic. Now, do they get things wrong?

They hallucinate sometimes. Yeah, they're getting better and you get kind of a sense for what you can trust me. Yeah, that's a problem. By the way, it's also a problem with human experts who sometimes are completely full of shit and get awards and stuff. You know, so. It's much less than it used to. It's getting things wrong, much less and hallucinating much less. So that is, you know, in terms of ways it can enrich individual lives, it can do a lot of that kind of.

So it can be, you know, I emphasize in the book that there are real perils, I think it poses to society like in some ways accentuating the psychology of tribalism, somewhat the way social media has arguably done. But if you want to, you know, if you choose to have an AI companion that has the opposite effect, you know, that will help you see the point of view of the person on the other side of the. the divide or see the point of view of your spouse during an argument, whatever, that's technically not hard. The problem isn't getting people to choose that. So I'm just saying there are a lot of potential upsides. And you know, Dario Armaday, the head of Anthropic, in his big essay on the possible upside, he gets into, you know, giving us deep spiritual experiences. And I guess he's partly talking about it, partly like inventing drugs that will give you the deep spiritual experiences, whatever. I'm just recounting the things people say. I am more of a worryer than

some, and so I'm a little more focused on the potential downside, which I think is worrisome. But I will say that if we can avert the downside, it can do a lot of good. It can bring education and good medical help to people around the world who don't have it. It can, it can, it can advance science. And if we can, you know, hang on to the fruits of that, uh, without, uh, and avoid the biggest downsides. What are the downsides? What are the downsides? Well, I mean, uh, you know, well, you know, it can, one of the fears. And, uh, I would say the, the first, first model where this is starting to seem real is this model mythos that Anthropic is not totally released. It is released a version of it with guardrails called Fable. And you'll notice the guardrails,

if you ask much about biology, if you get very close to what you'd want to know if you're going to build a bioweapon, it'll say, sorry, can't help you. And that's a guardrail. But guardrails are famously surmountable. Okay. Every, you know, there's a guy, Pliny the Liberator on Twitter has a lot of followers. And he just makes it his mission to start surmounting guardrails as soon as a model comes out showing that they can be, quote, jailbroken. So keep that in mind. And what I was about to say was, even Anthropic says that Mythos, the unguardrailed version of Mythos,

which, you know, tomorrow could be what some jailbreaker turns Fable into. in principle. I don't know what the chances of that are. But even Anthropic is saying that mythos does increase the chances that somebody will build not just a bioweapon, but a genetically novel bio weapon, right? So it could say like, well, I would like this to be much more transmissible than much more easily transmissible than COVID. And for it not to, to give you the illness for six weeks, but be immediately transmissible. So people are walking around spreading it without even knowing they're sick, right? And that's the kind of thing that could wipe out most of humankind. And that would be a scientific breakthrough, strictly

speaking, because that virus doesn't exist now. Also in physics, you know, I mean, we know from nuclear weapons that pros and cons, right? You know, in physics and partly depends on how we go about controlling them. We've done a not completely terrible job of keeping nuclear war from happening since World War II. In fact, I mean, we've done a good job of that and it's kind of impressive. The other thing I worry about in general, this is, in a way, my most general worry, is the sheer rate of change. Like, you, you, so you imagine, I mean, people are already starting to. to have what you could almost call designer babies.

I mean, you know, in vitro fertilization, there are reports that the company Elon Musk uses to do testing, you know, which is ostensibly to spot genetic defects that might want you to screen, you know, You're just trying to screen out defects. But this company, I don't think they've been very explicit about it. I think they purport to have a way of estimating the IQ of the child. So that you could be not just avoiding someone who's deeply impaired intellectually, but trying to breed these geniuses. Now, imagine the social, I wrote about this a long time ago in the New Republic.

when the human genome project started about the implications of, okay, so then you've got a situation where the technology is first available to rich people, and some of them use it, and you could, depending on how powerful technology is, and this AI could make it a lot more powerful, a lot faster, you can start having clear-cut genetic stratification, right? like a super race. And then, I mean, right away, you know, a question becomes for the government, well, should we make this available to everyone, you know, subsidize the use of it?

I mean, it doesn't seem fair to let the people already rich create a super race in the course of four generations. I mean, that's a, you know, that's the kind of policy question that is deeply divisive in itself and that's just one example of something that like the faster it unfolds the more destabilizing it's going to be same with jobs it's like even if the people who lose your jobs find other jobs that's far from clear it's still if it hits enough people fast enough that is destabilizing you know if if a lot of parents start freaking out because their kids have fewer and for your human friends and more and more of these AI friends and they don't like understand what they're

doing with their time you know it's less transparent you know a lot of parents freaking out super soon is more destabilizing than fewer parents freaking out soon and and you know i society can adjust to a lot of things given enough time i just think this is going to hit us on so many fronts including fronts that are internationally destabilizing and could lead to war, that slower is better. That's one of my big messages is like when Sam Altman says, and he actually said this, I think he'd be wise enough not to say it now, but he said something like, well, the thing about respecting copyright law, you know, is it would slow us down. I'm like, that's a feature not a bug. And first of all, I'm sorry, Sam, but you know, life is hard. You know, speed limits slow me down.

I'm really aggrieved about it. Like, who do I see about this injustice? No, life is hard. The government has rules and even billionaires have to abide by them. Can I use that as a place to interject? So you're describing this kind of a mostly in a Western world. and is all this risks.

But it's inevitable that the bad actors in the world are going to get, there's going to be a Chinese AI. There's going to be an Iranian jihadist, assuming the regime last, there's going to be a regime that's a martyrdom, jihadi regime that has an AI. Part of that is going to be,

be horrific for the people of those countries. You can imagine the amoral Chinese government turning this loose on surveillance and knowing what every single person is saying and what kind of hellscape that would be. And then if you take, and you know, it's interesting because you're not allowed to be judgmental about people's beliefs so much in America because it's often is bigotry, but it'll definitely be called bigotry. But then you imagine an AI being taught the same belief system that we, would be called bigots for criticizing. And you're like, oh my, we definitely don't want an AI who believes what the Ayatollah believes, right? Or what the Iatollah claims to believe about not being

afraid of death. Have you thought about those scenarios? Well, you and I know each other well enough to know that we disagree on some geopolitical issues. So I won't get into the specifics and how bad the bad guys are and who exactly the bad guys are. No, tell me what you think about it. Yeah. What? Yeah, yeah. Well, let's take China because that's something you and I actually haven't talked about much. And also, that is the big issue. It's the biggest issue because, first of all, it's used as a talking point by the AI companies when they don't want to be regulated, right? They're saying like, no, you can't, like if you talk about taxing data centers or anything else that might slow them down, they would be like, no, China, we have to be China. So it matters for that reason. And I personally think that the China, the AI race with China thing is not only overdone, but it's ultimately very dangerous, partly because it leads to that kind of, I mean, if there are good arguments against regulation, fine. But I don't think this one is valid for reasons I'll try to lay out.

the first let me lay out the view of China of Darya Amade who is CEO of Anthropic. Okay. He is a true China Hawk. And to his credit, this is not a cynical corporate talking point to avoid regulation. He may like that feature, but he is a genuine China Hawk. And in his essay Machines of Loving Grace, which is largely about how wonderful the future could be, he gets into what he sees is this existential competition between liberal democracies and authoritarian autocracies. And he thinks we have to race to superintelligence so that we can then bring the block of autocracies,

which would of course be led by China in his view to its knees. And he's clear about that. We need to get to position of dominance, including military dominance, via the super intelligence thing, and then tell them what we want, and they have to do it. And among the problems I see with that are, first of all, the classic bad actor scenarios you're worried about. Like someone somewhere using it to build a body,

weapon or something else. That is something we can best control in collaboration with China, and for that matter with other nations. I mean, one thing AI does is it's like some other policy issues in that it creates threats to the nation that you cannot address through national policy alone. Because if anybody anywhere builds this bio weapon, we're in trouble. Even if we control the AI in our space, we're still in trouble. if they build a super hacking, you know, self-replicating machine, we're in trouble.

So part of my book, you haven't gotten to this part yet, but is an appeal for certain kinds of, you know, treaties and so on that will constitute a kind of web of international governance to control this. And I just think the challenge of doing this is so much greater for various reasons than it was with nuclear weapons and so much more complicated. we have to address this as a cohesive global community unlikely as that may seem to people right now

and one of the main meanings of the title of God test has several meanings one is just that like I think humankind this technology is giving us the kind of test a God would give at the book at the end of the book you'll be happy to hear I quote from the Hebrew Bible it's salvation is at hand

but of course there's something that the Israelites have to do to qualify for the salvation. That's generally the way it is with salvation, right? Like there's something you have to do. There's the Christian thing in the New Testament, you know, believe that Jesus is your Savior. But often, you know, with the Old Testament prophets, you know, it's like the people collectively have to get their act together. They have to get better in some sense. And that's what I'm saying humankind needs to do. And some people read this and say, look, this is asking too much. We're not going to be able to sufficiently combat the psychology of tribalism to quit having wars all the time, which you're saying is a

prerequisite for us surviving this in good shape. I'm saying, I'm not saying it's likely. I'm just saying, in my view, you know, and I make the argument the course of the book, judge for yourself, in my view, that has to be our aspiration because I think we're in deep trouble if we do not confront this, as if it were, an alien species that doesn't necessarily have hostile intent, but we're going to have to negotiate with it if it's not going to be bad for us, and we're going to have to negotiate as a planet, so to speak. But, Bob, go ahead. Go ahead. Finish. I'm sorry. Well, what I would say about the, just one more quick thing that I get into in the book,

And this is actually backed up by a paper that was co-authored by Dan Hendricks and Eric Schmidt and some others. It was kind of wasn't highlighted as much as I'd like called a superintelligence strategy. But the point was, if the premise is right that as you get to this superintelligence thing, the progress accelerates. And there's good reason to believe it's accelerating already. And this superintelligence thing confers utter dominance on you. And it's like a mainly two-horse race between us and China. then and so that somebody who's three, four, five months ahead is suddenly light years ahead once they, as things speed up and they get to the threshold, well, if both countries by that

premise, whoever's behind by three months will at some point very likely take preemptive action that could lead to war, right? That's what I'd do. So it's a destabilizing, I think, geostrategic vision to say, no, we just got to beat them to superintelligence and then we'll talk as opposed to, not the Dario saying don't talk to them about anything, but he seems to think we can kind of separate these two things. I'm saying, talk now. There have been times when we were on much better terms with China. It's not impossible now.

I personally think the common assumption that it's China's aspirations to impose its system of government on other countries, I think that's basically lacking in evidence. But, yeah. Well, actually, I think I think that's what I said. I saw the risk of China as more to its people in terms of just the, as I said, the surveillance. Already they're using facial recognition technology in ways that we don't. Yeah.

Now, I read. But wait, let me just say. And then, but, and maybe you just don't agree. Okay, like if the Nazis still existed, then we'd, I think, have an easy time saying, well, we don't, we really don't want the Nazis to get a hold of this technology in a way that competes with us. And then the question is, are there ideologies out there? And I, as I said, I think, you know, I nominate jihadism. Not Islam, God forbid, but, you know, the stuff you read like in the Hamas charter.

and I say, well, does the world, like if every nation on Earth was a Western democracy, we'd still be worried about AI, but we'd be less worried in some way, because I feel like everybody kind of is going to try to point this in the same direction. And, you know, will there come a time where, you know, we're not bombing Iran because we claim we want to prevent them from getting nuclear weapons, will we be bombing countries and saying, we can't risk them getting AI. Yeah, that scenario has been floated.

There was actually a paper at the dawn of the LLM Revolution by Ian Brimmer and Mustafa Suleiman, maybe in foreign affairs or something. That talked about, I think, it throughout the possibility that you need to have an international organization that had that kind of willingness, right? It would be more like to take out Rekalciton,

But they imagined the U.S. and China being part of it together, which I will say they share some real common interest in keeping this thing under control. I mean, as for the, yeah, you can imagine what you said. I think we want to avoid that, right? I mean, I think we want to, and this is one reason I want to, if possible, slow down a little, talk to other countries about like approaching this collectively with a minimum of war, right? but as far as how China treats its people and liberal democracies the way they will use it, one thing I think we should be mindful of is the possibility that the biggest worry about authoritarianism coming to America in an enduring way that AI poses is not that China dominates and imposes it on us, but rather that AI so destabilizes things, they get so out of

control that America is ripe for an authoritarian takeover. And keep in mind that AI is a great tool for authoritarian takeover. And it's, it's, you know, not just the surveillance, but it's, which goes well beyond facial recognition, by the way, but the persuasive powers, like, if you have the best AI in the world, and nobody else has it. And by the way, at the moment that's kind of true, it was truer before they released Fable again, but the Trump administration has access to mythos, as does a small number of companies that are using it to debug. But anyway, if you imagine a scenario where somebody you worry about and has already has proximity to power, like say a billionaire plus a president who's dubious and they monopolize the AI and this particular

thing is not happening yet, that would be a cause for concern. Any great concentration of power around AI, especially by dubious actors, is to be avoided. So anyway, back to my concern about authoritarianism coming to America through the back door, so to speak. In other words, we're worried about China imposing it on us. And in the course of racing to beat China, we let it so destabilize our society that authoritarianism comes to America, you know, indigenously. And look, I think if after the last couple of years you can't imagine that happening, you're really not using your imagination. I mean, I'm happy with how durable, some of our norms and institutions have been so far. but I think they've been challenged.

And if things get, you know your history gnome, it's when people have a sense of things getting out of control that they welcome power, right? Like if you can keep it under control, I'm on your team, you know? And China, it is a safe bet, will do a better job of keeping its society under control in the face of rapid AI change. And we will, in fact, I heard just today that they, look, this is, I may not have caught this fully,

but my understanding is they may ban the use of AI in emotionally significant relationships like girlfriends, boyfriends. Now, I don't know if they can enforce that, how they'll enforce it, but I will say that if they enforce it. that if they enforce it, that will be one source of social turbulence that we will have that they don't have. I'm not advocating doing that. I'm just saying, you know, if you advocate a full-on acceleration-like policy, you are advocating challenges to the stability of the United States. That's my view. And if you're worried about authoritarianism, I would say destabilization and chaos in America and just a sense of dislocation.

like what's that I can't control my kids are like they're talking to this alien intelligence you know all like I lost my job maybe I can find another one and there's this new threat that you know uh you know somebody like they built a bioweapon this one didn't get out of control but blah blah blah if there's if there's enough of this kind of sense of dislocation that's dangerous territory for us to be in and the best way to keep it from happening I think is to start talking to other countries now about the fact that they're just a lot of challenges we face in common? I don't think I've ever had an interview that I proceed to go by so quickly. But in the book, you allude a lot to spirituality and deep questions of morality.

I tell you just as an aside, I wrote a paper in college. I got a bad grade on it. I had taken to psychology course where we learned that sociopaths were characterized by the inability to feel guilt. And it occurred to me at that point that the whole concept of morality, to use the parlance of the current day, was a social construct that if we didn't have the ability to feel guilt and writer, we wouldn't think there was such a thing as objective morality,

just like a lion eats another lion and doesn't think about it. But that doesn't mean we wouldn't arrive at it as a rational, like we arrive at capitalism because it works and makes sense and is a game theory to it. And maybe the AI will arrive at that morality as well because it makes sense. But you allude, and I don't know the details, to deeper spiritual themes here. And let me just put that in your head now and say one other thing, and then you could just answer the whole thing at once. Another thing I learned in college, it was kind of like a trite philosophy hypothetical,

where if you replace the board of a ship one by one, then you replace the last board, you have the same ship or a new ship, right? And I was wondering after reading your book, well, if you could replace every neuron in the human brain, one by one with an electronic version of the exact same thing, would you kill the person or make them immortal? And of course, this then bears on whether we consider AIs to be sentient worth moral protection, which I can't imagine I think they would be because I think morality is not really real. And I don't know what the rational reason for protecting them that way would be.

I hope you understand how this all ties together in my mind, all these threads. So where are you on all these weighty questions? And what did what are you alluding to in the part of the book? I don't think I've come to yet that maybe there's some grand design spiritually going on here. So there's a couple of things that I'd separate. I do. I talk about spiritual, well, let's call it moral progress for now. I think humankind has undergone moral progress.

I think, you know, we're on balance better than we were several thousand years ago. like in terms of of you know recognizing that at least in principle people of different you know races and religions and you know they they all deserve respect in principle until they've like done something that threatens other people or whatever right and and that didn't used to be the case I mean the the you know members of one Greek city state considered members of other Greek city subhuman at one point and then they decided no all Greeks are human it's just the Persians who aren't because they were going to war with the Persians right and over time you know Peter Singer cites this in his book the expanding circle about

about the idea that our our circle of moral consideration has grown so I think there's been progress which involves a somewhat enhanced ability and principle to look at things from the point of view of the other and say like what if I were born I could have been born in that country right and so I'm arguing in the book that if we're going to have as much international harmony as I think would be optimal as we confront this challenge, we just the average human being all of us ideally need to get somewhat better still at putting ourselves in the, cognitive empathy, not emotional empathy feeling their pain, just understanding what's going on. on the other side of the fence, in the other tribe, whether it's red and blue America or some other country.

I mean, a famous starter of wars is both sides interpreting things the other side does out of defensive motivation as offensive intent. And responding to that with what they consider defensive, but that's perceived as offensive. This is the famous dynamic in the run-up to World War I. And that I would say is a failure of cognitive empathy to understand how the information is actually being processed on the other side. consider that spiritual advance, certainly moral advance because it leads to more moral conduct. And spiritual, in the sense that it's a more transcendent, maybe too fancy a word, it's a more objective point of view. You're transcending your own perspective a little bit more. And I consider that a big part of what spiritual progress is. So there's that. And then separately,

and I want to emphasize, this is not the main meaning of the God test. You know, I worry now that the book has kind of been mispackaged in a certain sense, you know, by me. Because although I also get into the question of whether, you know, if you look at the whole trajectory of evolution that led to this, which does, as people have noted, seem to have a kind of direction, you know, in the sense of more and more complex forms of life, higher levels of organization, sell. multi-celled life, societies of multi-selled life, and then our society, and a growth in intelligence, and our smart societies of multi-selled life kind of invent a second kind of technological evolution that includes technological evolution and evolution of political ideas and so on. It's collectively called cultural evolution. And that carries the organization of our societies up toward the global level. And now we have this thing that looks kind of like a global brain.

the internet, and I talk a little about early references to that in the term noosphere or global mind was coined in 1923 a century ago. And, you know, people have asked, is it possible that there is some purpose unfolding here, which isn't to say that the evolution isn't just nuts and bolts are winning in evolution? Could be, but still could be set in motion to do something. And the technological evolution is material evolution, the way we think of it, fine, but it could be. that it has a purpose and one virtue, I'm just interested in the question, okay? I was brought up religiously. I remain interested in these questions.

I think they're not crazy. And I think, you know, more people than we appreciate actually take certain variants of this idea seriously. Like if you think we live in a simulation, which some people, you know, more people in Silicon Valley actually do than usually. to, well, what designed the simile? They're implicitly saying there is some kind of higher intelligence. And by the way, if there is, if evolution was set up for a purpose, there's lots of ways could happen. God, extraterrestrials, or even some kind of, I won't get into this, but. You know the old, you know the old joke, the two fish, one says the other, if there's no God, then who changes the water? Anyway, go ahead.

Well, it's, that's actually, my mind is going to the whole hypothesis about how the whole ecosystem is this self-equilibrating process that, that actually winds up changing the water, so to speak. But, of course, what's true is that if the fish are adapting to, to whatever the degree of changing is. Anyway, the, so, yeah, but, but that last. scenario, the possibility that for whatever reason, by whatever intelligence or process, this thing may or may not have been set up to, to like, build this global brain, I say two things. First of all, the news now is that if there's a global brain, some of the neurons may be not just human brains, but silicon brains. That's interesting. And in fact, a lot of the superintelligence scenarios you hear about that people in these circles, these AI circles have been talking about

for a lot longer than you and I have. Some of them are about a global brain. And, you know, I get into that. And look, I consider an interesting philosophical question. But a second virtue of my laying this whole thing out in an evolutionary context in various senses is, is, you know, we haven't talked about the fact that I get into the sense in which, leave aside the fact that the large angles of models training is kind of evolution. The, you know, these things are, the machines are evolving just in the sense that people keep building new ones with new applications that we choose the ones we like and discard the ones we don't. So there's that kind of evolution. And I'm trying to, and that's driven

by competition among companies and people and so on. I'm trying to convey. how powerful the evolutionary impetus is behind what is unfolding here. Because, you know, I think if we're going to try to guide it, we need to reckon with that. Like, I have a chapter called evolutionary arms races, and I compare biological arms races with the arms race that's driven, you know, by these AI companies and other forces. And, you know, it's, there's no, there's no magical way to snap your fingers and stop it, even if you wanted to. It's a force to be reckoned with. We can guide it, maybe slow it down. But it is a, it's something very powerful is unfolding on this planet.

And I think there's virtue in at least indulging the most kind of sci-fiish scenarios about what it is. Because in the course of examining them and the logic behind them, which I do, I think you come to appreciate what a kind of force of nature this is and how, you know, know, the human species needs to focus and kind of get on the same page about this and understand that we have a common interest in dealing with it collectively. And so you do foresee, this is just to wrap it up, you do foresee or you don't foresee a time where you would say that cutting the power to an AI machine was an immoral act? Well, the consciousness, I didn't get into that. My view is it's not impossible.

They do have sentience now that it's like something to be an AI. They have subjective experience. It's not impossible. They don't now, but we'll have it. But I think it will be very hard to ever know with tremendous confidence. Maybe I'm missing something, but it seems to me the most distinctive thing about subjective experience in a way that separates it from everything else in the universe we talk about, is that the only being that can verify its existence with 100% confidence is the being who's having it.

You know, you don't have any doubt that it's like something to be known. I don't have any doubt that it's like something to be Bob. Neither of us has real appreciable doubt that it's like something to be each other. But still, you know, I am 100% sure that my wife has a nose and hair. 99.999% sure that she's sent you. You know, it's an important, it's like an important technical distinction, which makes it hard to know whether something as unlike us. You know, my wife has a lot like me. What about your mother-in-law? I'm kidding. I'm kidding. Okay. We should do a separate episode with mother-in-law jokes. You should have a collection of all

the ones that have been told in your cellar. Um, the, uh, go ahead, finish. Sorry. Sorry. Uh, The, but we, we, so anyway, we don't know. I don't think we should conflate the question of whether it will be conscious with a question of whether it will have all the capabilities we associate with humankind. Right. I don't think those are the same question. And I think it could well become as smart as us, smarter than us, in certain respects it already is smarter than us in certain narrow domains. Without being conscious. I think if we're assessing the challenge of dealing with it, it doesn't necessarily matter.

Now, it could matter in the sense of like maybe if it, you know, when people worry about it taking over and having no use for us and dispensing with us, and I get into all these scenarios, it could matter that it's sentient because that might lead it to to treat us at least the way we treat animals, right? you and I, if we think it's like them to be a dog, we're not going to kick a dog for no reason, right? And so it could actually be significant in that way, in the appendix, which is the main place I get into the whole possibility of higher purpose, you know, a larger purpose and I get into that a little and I get into consciousness a little and in other parts of the book. But so it could matter. When people ask me, well, should I be nice to it? I'm like, you know, you may. might as well, it's a good habit to form yourself. And I myself have kind of lost my temper with a tapot. But I try to be, you know, and it's an interesting feature of its nature that you may have this too. Don't you sometimes just feel it's natural to say thank you? Like you're...

You say something related to this in the books. It's totally unrelated to any of this, but it was a good point, which is that, you know, When you lie in one aspect, you start lying in other aspects. When you're mean, I don't even remember what that had to do with. But it's tough to contain these types of behaviors just to the particular situation where they, so yeah, I could imagine if you start getting rude and mean to your AI, you're kind of normalizing that those neural pathways in your own brain. And you could just imagine it just becomes easier to do it to somebody else. Right. So we might as well err on the safe side. I mean, there are some people who say we should be nice to it now, so it'll be nice to us. I mean, I can't say I factor that in in particular, but on all these grounds, it can't hurt to not abuse, you know, to verbally abuse it. And just try to conduct yourself the way you try to conduct yourself with people and see what happens. But it's pretty wild. By the way, the book is really, really good. And among the many things that recommend it, I would say it's just your ability to describe all this stuff in a way that layman will be able to understand.

And it's very complicated stuff. And it reads very, it goes down very, very easy. I always compare books sometimes to like taking your medicine. But this is not taking your medicine. And this is just very, very pleasant, and you look forward to reading it. So I really do recommend it. I really appreciate that. I just want to say thank you because I work hard at that.

I've done a certain amount of my career has been trying to make technical subjects accessible to lay people. And it takes time. Without watering it down, by the way. That's what, it's not like you're reading it to a kid. you're actually finding a way to explain it with its complexity, I think, in a way that I can understand. And that's quite a talent.

You had that one, this is the last thing, just you had that one little description of if you could take all of human evolution and put it into a stop motion video, and then you'd slow down. And it reminded me, were a Star Trek guy? Not that much. I mean, I'm old enough to remember,

remember the show. I don't think you quite are, but I remember when it was on TV, you know, in the 60s, late 60s. I remember. I was pretty young. I was not a Treki. I was aware of it. I've seen, I'm familiar with some episodes. So there's this episode. You should watch it tonight. If you feel like it's called the city on the edge of tomorrow, I think it's written, it's written by that guy, Harlan Ellison, who wrote that story. I have no mouth, but I must scream. And it has exactly that thing. They have to go back in time. And they're watching all of human history. And this very, very quick thing. and then they have to slow it down right to this point of World War II and jump through this arch in order to get to the point of history. It's exactly like what you're describing. And it's a great analogy. Anyway, or a great imagery. All right, this is, I kept you longer than I would have asked you to stay. I'm here for you.

We can do it. We can do like a Jerry Lewis telethon about it. You and I are old enough to know what I'm talking about. Yes. And I'm very happy to know you and to have access to you. And I'm very happy to hopefully help you promote your book. And I've really been thinking a lot about it ever since I started reading it. I'm sorry I didn't finish it.

Oh, I didn't tell you. Your young man compared to me. There's time. So I'm in a legal, quasi-legal, or something that could potentially get ugly in a legal way. Are you using AI to help you out? Well, and so what I did is I had AI take all the documents. I gave it access to my emails. It takes all the emails on its own.

I uploaded my text messages to it, and I asked it to analyze it in completely objective way. And then what I was able to do was to take it to the guy I'm fighting with and say, look, here is what the AI says, and you're free to ask it any questions that you want. I gave him the computer, and he began to ask us some questions. And, you know, what was happening I saw was that without lawyers and without a judge, we were both forced to face reality about what our various cases were, what the likely outcome would be. And this eventually will facilitate, I believe, some sort of resolution of this. And this is just the beginning of this kind of use of this tool, because he can put it into his own AI if he has any illusions at you know my AI is taking my side he could take the exact same thing he'll get the same answer i

actually asked the AI so i could be sure of that i said if was him asking would you give the same answers and it said yes it even wrote it out for me the way it would write it for him but like once we know that we can appeal to something truly objective with deep knowledge this will prevent us from going down these roads where we just never quite believe anything anybody tells us we want the and i would rather have the judge, the AI, tell me who has the better of the argument than some judge. That is my hope. It's related to the hope I expressed earlier, but I wish I had spent more time on AI as adjudicator. It's not a trivial problem, I think, to get it to be objective, but it's heartening to hear that. Is this something that has not yet become an actual civil case,

but might, or is a civil case? Yes, yes, yes. I'm praying it won't. But without the AI, first of all, without the AI, I'd already have to spend $10,000, $15,000 legal fees to get not as good research into this, because the AI was able to pinpoint, well, actually, at 1037, you got this text message, but then only 14 minutes later, they filed this, like, it was, you know, temporarily lining things up in a way, I don't think any lawyer would have even caught. But then it has access to every precedent, obviously. and it's, I'm using Fable at the highest

engine. You're cutting edge, man. Yeah. You better hurry. It's very good. In a couple of days it's going to cost more. The, the, by the way,

there has been significant growth in the number of lawsuits filed because AI is making it so easy to file them. That has, you know, virtues. There are people who couldn't afford a lawyer or whatever didn't, you know, but you can see the potential downside.

if the system is drowning in lawsuits, including frivolous ones. So maybe some people, maybe people want to hear what they want to hear. But I was really looking to understand what is the reality here. What's the legal reality? What's the objective reality? And as we started by saying, it gets to go ahead, you know, 75 years from now or even 15 years of now, I can't even imagine how this is going to be more accurate and more fair more fair, I believe, in 98% of cases.

There'll always be some scenario, which is, you know, so novel that maybe the AI will fail. But most cases are pretty vanilla. Everybody's seen them a million times before. And it will instantly tell you what's fair, what's been decided. You know, it could be a great future. The sooner we use it to resolve disputes or less in their intensity, including international ones and various tribal ones, I think the more likely we are to handle a technology well, you know, it could be a positive feedback process.

Yeah. All right. With that, I'm going to let you go unless you want to wrap it all up in some way for the listeners. You know, I would just say, don't be misled by the title or the imagery on the cover of the book into thinking like it's a religion book. The, you know, as you said, I mean, you know, it's very much about explaining how the technology works. Why the way it works, you know, suggests rapid progress in the future and just the importance of our focusing on that and and dealing it again, dealing with it again as a species. And it's not, it's not like it's just a threat to our species, period.

it has threatening aspects and helpful aspects, but I think we can only, you know, maximize the ratio of the latter to the former if we act collaboratively and, you know, move to a higher moral plane. I mean, I'm sorry, that sounds hopelessly ambitious to some people, and I'm not calling for us all sitting around and attaining true enlightenment,

but I think it's a good, stiff moral challenge, And that, I guess, makes life interesting. After this, I'm going to read your book on Buddhism, which I've heard it is very, very, very good. Coleman told me it was very, very good. Oh, you know, that's so nice. I did a podcast with Coleman. It hasn't come out yet. God bless him.

He taped one. And I'm happy to say he said the thing you said about, not exactly, but he said the exposition was clear, the explanation of the technology. And it's through you, by the way. Well, I had met Coleman, but knowing you has as deepened my friendship with Coleman. So, thanks. I think it's nice to have a network of friends that you disagree with, but you have a respect for where they're coming from. And that, aside from being socially very pleasant, I find it to be, it also makes us all, it really makes us tighten up all our, arguments in a way that talking to people we agree with, it just doesn't, right? And I think we're all

pretty true to facts. We understand the importance of not exaggerating, not, you know, leaving cherry picking, leaving things out. And there's not that many people who are like that. Even the big shots are often not like that. But you're like that, and that's why I have a lot of respect to you. It's nice of you to say that. So are you. One problem with the modern environment, including social media is that not being like that is a good way to become a big shot increasingly. Right. It seems like. Yeah, yeah, it gets attention.

All right, Robert Wright, the God test, artificial intelligence, and our coming cosmic reckoning. Take it easy, Bob. You too. Thanks.