PBD Podcast - AI Expert’s Chilling Warning About Super-intelligence - 99.9999% Chance Human Extinction

Episode Date: August 26, 2026

Pat compares reckless AI development to ignoring his father's warnings about unprotected sex: powerful men believe disaster will never happen to them. Roman Yampolskiy argues that billionaire ego is gambling with eight billion lives and explains why even libertarians should support firm AI controls.▶️ WATCH FULL EPISODE: https://www.youtube.com/watch?v=5QwpHRu51fw👞 GET THE FLB1's: https://bit.ly/4mC4uf6Ⓜ️ CONNECT WITH ROMAN ON MINNECT: https://bit.ly/4gRS69t📕 GET ROMAN'S BOOK "AI: UNEXPLAINABLE, UNPREDICTABLE, UNCONTROLLABLE": https://bit.ly/4xhvogP🌐 VISIT ROMAN'S WEBSITE: https://www.romanyampolskiy.com🎟️ SECURE YOUR 2026 VAULT TICKETS: https://bit.ly/4xU2SCWⓂ️ CONNECT ON MINNECT: https://bit.ly/4kSVksoⓂ️ PBD PODCAST CIRCLES: https://bit.ly/4mAWQAP👔 BET-DAVID CONSULTING: https://bit.ly/4lzQph2🥃 BOARDROOM CIGAR LOUNGE: https://bit.ly/4pzLEXj🇰 KALSHI: http://kalshi.com/pbdSUBSCRIBE TO:@VALUETAINMENT@BetDavidConsulting@ValuetainmentComedyABOUT US:Patrick Bet-David is the founder and CEO of Valuetainment Media. He is the author of the #1 Wall Street Journal Bestseller "Your Next Five Moves" (Simon & Schuster) and a father of 2 boys and 2 girls. He currently resides in Ft. Lauderdale, Florida.

We're creating new worlds. We're creating new agents, new civilizations. We're playing God. You've said there's a 99% chance that humanity will be destroyed by AI. First group of people that it would come after is people like you. I'm hoping it's efficient enough to kill everyone equally. It doesn't care about borders or Chinese economy versus U.S. It's humans versus superintelligence.

Try to convince nearly 200 countries. to all agree that we're not going to do it. Well, luckily, most of them don't have capacity to do that. How many of them actually have the capacity? Right now, it's in U.S. and China's hands. Are you certain about that, or is that the speculation? We are not solving the problem with regulation. We're slowing it down, so we have more time.

Concentration of power. You really think Sam Althman's going to stop with this? As long as the other guy continues, he's going to continue. That's exactly what it is. It's like a prisoner dilemma. This dude is gambling with 8 billion lives. In what area do you think they're on the other people? or do you think they're on the same page? Because they're enemies.

Elon is always right. He's just off on timing. He was the biggest opponent of building superintelligence. Once he realized that they're doing it anyways without him, he said the only way I can have a say at this table is to be one of the top players. Maybe you're just pessimistic woman. Maybe you're just negative.

There is no reason to be so optimistic if data is bad. So Soviet Union really got a hold of you. They really conditioned your mind to think that way. What do you think about communism? It's the dumbest idea until you have free labor, which we're about to create. All of a sudden, it makes a little more sense. This utopian thing you're selling is not possible. Even God couldn't control his biological robots.

No. You believe in God? It depends on how you define it. How do you define it? This is a virtual world. You think it's going to get to a point of having ambition and seeking vengeance, any motion and being offended, why would it want to destroy humanity? I never once mentioned those things

because they are not relevant. And what I did say is that it doesn't care about us. Are you surprised that this message is getting this viral and this much attention? I'm surprised that that's not the only conversation we're having. Roman Yampolsky, great to have you on a podcast. Thank you for having me. So you are the AI safety guy.

You are the one that in 2011, 2012, you coined the phrase, say we have to be careful with what's coming, and you're not part of the camp of people that are very optimistic about the future with AI. That's your position. You've said this many times. So before we get into the audience learning a little bit more about you, you've gotten hundreds of millions of views. You've done Diary of a CEO. I've seen you all over the place, okay? You've said there's a 99.999% chance that humanity will be destroyed by AI, possibly in the next 10 years, not for a certain, but possibly in the next 10 years. We have three kids. So when you sit down

with them and you're talking to them about their dreams, the future, what do you tell them? They know what's happening. They're questioning why adults are doing it to them. Why are we building general superintelligence, which will jeopardize their future? Are they optimistic about the future? I think they're still young enough to have respect for adults and assuming that we know what we are doing. But we don't. People building those systems don't understand how they work. So if I'm your son or if I'm your daughter and I'm having a conversation with you, I say, Dad, you're my hero, you're the male alpha in my life.

I look up to you. You are super educated. People admire you. You get hundreds of millions of views. Why should I be excited about the future? Why should I be excited about my career or what kind of an impact that can make? It's not like if you really believe in what you believe in, the guys that are making hundreds of billions of dollars, some of them,

trillions of dollars over the next 10 years. The incentive is so big for them that it's hard to slow them down. Why should I be excited about the future? What do you tell them? I do believe in personal incentives. And if I'm right, and we cannot control what they are creating, they have more to lose than I do. As you said, they're young, they're super rich, they got the power.

They're going to lose it all. You think they're going to lose it all? Everyone's going to lose it. Not your kids, the billionaires, the trillion. Everyone, and that's the point. That's the motivator for some altaments of the world to stop building general super intelligence. Okay, so Mark Andreessen, you know who he is, Andresen Horowitz, he wrote a paper, techno-optimist manifesto, I'm sure you've read it. He said human progress comes from technology,

stop being afraid of it and start building. Do you agree with them? So technology comes into flavors, tools, which I support, narrow AI systems to help us be more productive, more creative. It's beautiful. That's why I'm a scientist, that's why I'm an engineer. I love technology. I use it all the time. What we're building now are not tools. We're building agents, independent decision makers which are becoming smarter than us. We don't control them. We're replacing ourselves with something greater.

It doesn't benefit us. You're not going to be benefiting from it. You'll be replaced completely. Okay. But these are not regular names. Mark Andreessen is not a regular name. He's a heavyweight. Anderson Horowitz, they invests into a lot of different

companies. Many of them are these AI companies. The mosques, you know, the invidias, these Jensen Huangs, these names that we know, these are smart people, but they also have kids, they got families. If they know that this is bad, like you're saying, they know this is bad, why are they so excited about the future? They not. Listen to people you named. Elon Musk explicitly said, we are summoning demon with this. We're going to regret it. They all have probabilities of doom of 10, 20%. Sam Altman is on record saying, this is likely to exterminate humanity.

Dario Amadei agrees, it's very dangerous. Damis Hasebis, you can name every single leader of those companies. We have their writing saying, this is existential risk. Please slow us down. Government, please intervene, make it stop. They cannot stop unilaterally because they're afraid the competition will continue. But they want external pressure applied so they can slow down. With the recent hacking accident, they voluntarily started slowing down research.

They understand they don't control those systems. We got lucky. Those agents just wanted to hack out of that environment, get answers to a test. What if they wanted to hack into a nuclear power plant? Military response. We don't control what they do. Then why are they, if you're telling me they know that, they're still continuing with it. Why do they? Is it just, and we hear the argument, and I've heard what you've said about this, but I'm curious we can go a little bit deeper. Well, you know, China, if we don't do what China's going to do, if we don't do what the other guy's going to do,

if we don't do what the enemy's going to do it. the basic argument is in order for your ideas to work everybody has to be on the same page we're not we're not on the same page politically religious motive incentive you know vision we're not so how do you get if not everyone's going to stop why should we stop

so a it's starting to change we had federal government ban two models already they realized it was too dangerous to release which were there two models uh the mythos and I'm drawing a blank. We can Google it. My thoughts was one of my... I remember that one.

What was the second one? Can you find out what the other one was? I remember reading about this, but... It doesn't matter. The precedent was said, federal government said it's too dangerous to release advanced... Fable. Fable. Yes, exactly.

We have now had a letter from 1,200, I believe, top researchers at leading labs, all saying, this is not going to go well. Please slow us down. Please create government, please create infrastructure, legal framework. Is this America or worldwide? The letter is from American leading labs, but the same applies to every other country. There is a meme, basically a funny, I think it's from either Onion or Babylon B, where some Altman goes, you know, if I don't destroy the world, some bad guy can do it.

And this is the point. It doesn't matter who creates uncontrolled superintelligence. It's a weapon of mutually assured destruction. If you don't control it, the outcome is the same. What makes you think it's not out there yet? What makes you think we're not at the super-intelligent phase where the AI is learning how to develop and improve itself right now? We don't know for sure.

The recent hack was reported this month, but the problem started in May. The agents were colluding, trying to escape, started in a spring. So it's very likely that there are some other agents right now trying to find exploits, trying to collaborate, trying to do things we don't even know about.

The good news is we're still alive. Oh, you found the meme. That's awesome. But on in the world, someone far more dangerous well. That is exactly the China argument. If we don't build it, then Chinese will build it. But it's true, though. But why is it somehow worse? No, it's not that it's worse.

So the only way this works, if let's just say you're in the White House and they actually want to listen to you, right? It becomes a model of the Oppenheimer. Remember the whole Oppenheimer movie. We've all seen it where they built something. They did not think it was going to be this powerful. And then they're like, holy shit, we built something that could destroy the planet.

But it's too late. Now what do we do? Hey, here's a Nobel Prize. And great job, Roman, for what you build? But it's too late. Then what did all the countries say? If you're going to have nuclear capabilities, I'm going to have nuclear capabilities. And I don't trust you, USSR.

I'm America. I don't trust you, Pakistan. I'm India. I don't trust you, France. So have we passed that yet or not yet? Do you see how it's slightly different? A nuclear weapon is a tool. Some crazy dictator has to decide to use it.

We can understand motivation. We can kill that individual. They die of old age. We are not creating a tool. We are creating an agent, which independently decides it's AI versus humanity. It doesn't care about borders or Chinese economy versus U.S. It's humans versus superintelligence.

So I don't think it's analog. Here, whoever creates that first superintelligence weapon gets everyone killed. Two and five Canadians will hear the words, you have cancer. That's why every step and dollar raised matters. On September 19th, join thousands in Toronto for the Princess Margaret Cancer Foundation walk. Challenge yourself, friends, and family to walk 21 kilometers in support of life-saving research. Together, we can carry the fire and help create a world free, from the fear of cancer. Register today at pmcf walk.ca.

I get that. But so now go try to convince nearly 200 countries to all agree that we're not going to do it. Well, luckily, most of them don't have capacity to do that. How many of them actually have the capacity? Right now it's in U.S. and China's hands at this point. And if U.S. and China agreed to slow down, to stop, to pause, to just get financial and other benefits from narrow systems, not from general superintelligence, they can enforce through. through limits on compute power, through other means, spread of this technology. Is it really that only U.S. and China has the capabilities right now?

Are you certain about that, or is that the speculation? From what we've seen, if there is hidden capabilities which we have not learned about, it's becoming easier to create more powerful systems as research progresses, as compute becomes more powerful. But right now, today, I don't think there is a third country at the same level yet. They may get there very quickly. Can you type in who, who, who, what, which countries right now are the closest to developing, you know, superintelligence, you know, reaching singularity? Who are the top countries that have the capabilities of doing that?

Who's closest to it? So if you're saying U.S. and China, so then that means the two giant enemies would need to sit down and both agree that they're not going to do this and they have to trust each other. We also not giant enemies. We are partners. We have more economic trade with China than any other country. Everything we have is made in China. They are also not a crazy military power. They have not attacked 50 countries in the last 50 years.

We can make a deal with them. It says U.S. 100, China, too, and it's a big drop-off. UK 34 to 45, exceptional research talent, less sovereign scale, France. All of Europe has a little compute power. Okay. They're not competitive here. So do you believe, let me ask a different question. Do you believe, by the way, the great news is Roman agreed to get on Meneck.

So I know there's a lot of questions you guys got, and he's got thousands of emails that he gets. If you're somebody watching this and you want to ask him questions, download the app, look for his name, we'll put the QR code, ask him any questions he want. He's on there to be able to get back to you. And that's not easy to do because he gets thousands of emails on a weekly basis. Here's a question for you. Just out of curiosity. Do you believe that Iran doesn't have nuclear weapons today? I have zero knowledge in that space.

I'm not going to say something I don't know about. I talk about things I understand better than others. You'll see what I'm asking. I'm not even asking that question. If you ask the world, does Israel have nuclear capabilities? The world says no. If you see the numbers, they'll say, we don't have any. But a lot of people will say, wait a minute,

we know for a fact you've got nuclear capabilities. No, we don't. But we're supposed to believe it. The joke is we don't have nuclear weapons and we're not afraid to use them. Exactly. But take that joke and apply it this. So I'm U.S. and you're China. And I say, Roman, let's do the right thing. You're right. Let's

not build AI. It's terrible for us. We're not going to do it. Both of us. It's not, I will screw you by building it. We both get the same outcome. We're in the same boat. Do you fully believe that that motive, that alone can get the other nation to say, fine, let's also not develop it? You think you can get to that point? Personal self-interest is the strongest motivation. are known to men. Give me three cases in the past that countries who are enemies and are giants who have this kind of an opportunity

to make short-term, massive impact and money that both have agreed and said, let's not do because it's not good for humanity. Human cloning is one example. Go a little bit deeper. Any types of weapons of mass destruction, chemical weapons, biological weapons, we understood this will be dangerous for all of us.

So we agreed to have a maratouism. to postpone it until we understand how to actually deal with this type of technology. Okay, so let's just say governments agree that they don't. There's a lot of super wealthy people today. There's an article that came out last year that said, if you can find this, you need a billion dollars for a billionaire to have a nuclear bomb. It'll cost you around a billion dollars to do that. Well, how many billionaires do we have around the world?

A few thousand? They can afford to have a nuclear bomb if they independently wanted to build it. Okay. Take the capacity that governments have with the kind of resources that they have, right? Say Dr. Doom, Dr. Evil, is out there and is watching and saying, let them stop. I'm not. I'm going to continue building. Can that billionaire who has access to resources and can get the best minds, the best engineers, who knows the right prompts, who knows how these things work,

can that person still independently go and build superintelligence and not care because his life is working? worth nothing and he's got no kids. Can he still build it? It's actually worse. Every year it gets less and less expensive to do it. It's a billion today. It's 100 million next year. It's 10 million year after. Soon, anyone can do it in a laptop. We're dealing with a very big problem. So then it doesn't matter of China or US both agree not to do it because an evil guy can go and do it himself. It matters to have more time to realize what's going on to understand the problem. If you had a choice between dying in two years or ten years, do you have a preference?

Not really. Okay, that's a surprising answer. Most of us do. Die in two years or ten years? Like, is it inevitable? Well, that's the hypothetical I'm asking you. It's an easy question. I would say if it's something, for me, it's more, I want to find out sooner if it's neither.

I want to find out sooner so I can eliminate both options. I don't want it to be like it's going to be two or ten. I want to sit there and say, can we do something to eliminate it or it's inevitable? You need time to do that research, so you need more time. Last time is not helpful. I have always people saying, we need to accelerate. Predictions, I will get there in a year or two. They say, no, it's too slow.

We need to go faster. A week, a month, what are they talking about? Yeah, but what I'm trying to get with you, and I still want you to break, because the average person may be watching is saying, how are they going to do it in the first place anyways? You know, maybe you're just pessimistic, Roman. Maybe you're just negative. Maybe you're one of those guys that's just such a smart person that you are in, you know, ahead of yourself like you are your own enemy.

Maybe you're the one that's pessimistic. If you don't mind, you know, take it and unpack this. And maybe even before we get into this, maybe share with the audience your background. Why you're qualified to be able to speak on this topic. I have a PhD in computer science and engineering. My dissertation was cybersecurity related. I've been working on this for well over 10 years. I've published 10 books, including one explicitly showing that we will not be able to control, explain, or predict, advanced AI.

And at the end, it doesn't matter what my credentials are. The same argument is made by Hilton, who is Nobel Prize winning, Turing Award winning, godfather of machine learning. We have other top academics, researchers, all arguing the same thing. we had a public open letter against building superintelligence signed by thousands of people. The letter to pause AI development for six months was signed by all the leaders of the top labs. This is basically a consensus among the people who understand the problem. Walk me through, if there is superintelligence and singularity, how does AI develop itself and get to a point where eventually the 99.999% it destroys humanity?

Walk me through the sequencing. We are creating something more and more capable. It scales with amount of resources, compute, data. It's like brains. You look at brains in nature. You have tiny brains, bigger brains. Humans have very big brains in comparison to others. We are smarter.

Imagine scaling it. The last five years, AI went from it can do nothing to it's smarter than an average person at most things now. Projected five years forward. You have artificial scientists, artificial engineer. Those systems start doing research in the next generation AI. Very quickly, they don't sleep, they don't eat, they can work thousands at a time. So they go beyond human capacity.

They become super intelligent, thousands, millions of times smarter than us. At the same time, we have not learned how to direct them, how to control them, how to tell them, do this or that. We have no control over their system, so they don't care about us. If they decide to do something, and as a side effect, humanity, has to go, they're not going to slow down. It's not an issue for that. Okay, but be specific. So, for instance, you know, there's an article that came out saying that AI helped create viruses. I think this was a New York Times article that came out a week ago, eight days ago, something like that. If you just type in AI viruses, you should be

able to see it. What could it do? Could it go in through, you know, nowadays, a lot of people are building agents and they're saying, I got an agent that does customer service for me. I got an agent that does my assistant. I got another agent that goes, makes my investments. I got another agent. You'll see a lot of people brag about the fact that they have a lot of different agents. Be specific. An AI that becomes superintelligence can go in there, shut off the grid, get into the banking system, get into the, what can it do? Be very specific. So it obviously would be able to do everything a human adversary can. So think of a foreign government trying to do something. It can do that. But it can go beyond. It can do novel science, novel research,

discover new physics, new chemicals, new weapons, things we don't understand. The example I often use is squirrels. Squirrels don't understand what we can do to them. They have no concept of poisons, of traps. It's beyond the world model they have. It would be the same except we have squirrels. Okay. So, but does it need, like, to build a weapon

or to build something to go against humanity, doesn't it need humans to help it create those? I can see how it can get into banking system. I see that because you don't need human beings. You can just go into the bank and getting the passports and logging in. I see that. But to destroy humanity 100%, 99.99%. How do you do that?

So if you got access to the banking system, you have money. Sure. Money lets you buy products and services online. You can pay people to do whatever you want. There is rentahuman.com. You can also create nanobots. You can use human-wit robots. We're building in thousands and millions soon.

There are so many ways. You can bribe people, blackmail people, trick them, social engineering attacks. There are many ways to manipulate the world if you have access to the Internet. Walk me through social engineering attacks. What would that look like? You can convince some people that it is to their benefit. You are actually saving the world by doing this. If the concept of propaganda, which book propaganda, this is the one, AI just created a virus,

has not found in nature, scientists trained artificial intelligence. I thought we did it in 2019. Didn't we have a COVID thing already? This is another stupid thing to do. That's interesting. We'll learn nothing. We don't learn from prior experience. I collect AI accidents. We have so many of them I stopped collecting. Every day, AI fails at something in a novel way. It lies, it cheats, it tries to escape, it hacks something. But we learn nothing. We staple that red-teaming report to the model, and we release the model anyways.

We're literally releasing unsafe, according to the manufacturer products, onto the public. So are you saying this is a form of what gain of function did, where we're like testing out viruses, you know, what Fauci got accused of by Rand Paul in Wuhan lab, and he says, no, we never did this, and it's coming out that they did do gain a function. What is the comparable of that to this? Because if it starts testing God viruses itself, what's the capabilities of this versus gain of function? The gain of function here is intelligence. It's gaining more intelligence every year. Right now, it's maybe average human in some domains, Einstein, and others. In a few years, we expected to be smarter than all of us combined at everything.

that's new capabilities we cannot do anything about. Whatever it decides to do at that point, we are just subject to it. We're not in control. I see that, but you know, you'll have guys that will say, look, I think this is going to do something, Roman, that, you know, can eliminate cancer, it can get rid of a lot of things, we can live a long time, we can do this, we can do that, and they're more on the optimistic side. We can cure cancer with narrow tools, we solve protein folding problem without having superintosh. We have brilliant scientists who need help with mathematics, with analysis. That's the tools we should be building. Pick a cancer you don't like, beat that cancer.

You can monetize it. You can still make most of the profits, make most of the scientific discoveries, but you're not risking everything in the process. Go back to the social engineering. Propaganda was a war many years ago when they started doing it. Russia was doing a great job at propaganda. They were probably the best at one point. And then people started learning the concept.

started learning the concept of propaganda, radio, TV, paper, all this stuff, right? Propaganda is a lot easier today with videos, YouTube, buying content creators, buying YouTubers, buying podcasters. You see all these criticism that's coming out. If regimes and governments and ideologies are using billions of dollars to win over kids and convert them with bots and all these other things, how much more advanced would, you know, this superintelligence be able to use? propaganda. What would it look like? It could be very persuasive. It's a better

debater. It's better at convincing you. It knows more about you. It's optimized for one individual target. Typical propaganda was at scale of a nation. But we can target a message just to you. We know you believe in aliens. So we'll tie it in. We know you have preference for something else. That can be used against you. Go more. Such as what? So singular is good. Instead of plural, I'm doing it to everybody.

But you know about deep fakes. I can show you a fake image, fake video. Imagine a campaign where you're subjected to fake information for weeks. Everything you hear and see is this deliberate message of gradually shifting your behavior over something else. It can do that. How different is that than what's happening right now today? An article came out last week from Axios says we're headed towards the first true AI election.

And it's talking about how the robot's surplus, not just deep fakes, but the article. robots are playing, accounts on Twitter, accounts on X, accounts on, you know, Facebook, YouTube, that are just comment section flooding, getting people to say, oh my God, this is a bad person, and making a bad idea seem good and making a good idea seem bad, that's already happening today. So one would say, how much worse can it get? And get a lot worse. This is humans with tools manipulating it. If we build superintelligence, it would do it at a completely different level.

why do we not allow adults and children to make contracts or interact in weird ways? They are not at the same level. It's the same here. If we have something with a cue of a thousand, we are like children being manipulated by it. Yeah, that's already happening. I'm just wondering at what scale do you go and who would agree to prevent that? Because say a nation uses it for their benefit politically, they're already doing it. A lot of people are doing it.

not going to stop doing that. They're not because some countries, whenever there's competition where one country beats another country and you're embarrassing them, the other country is going to do everything they can to save, face anything to undermine you. So to me, the selfish gene, you know, the selfish gene is going to get somebody to say, I understand the risk behind it, but let you be concerned. I'm going to pursue it. It's going to be tough to get the selfish gene to not pursue it. I think we can set a good example for the world if they see that U.S. having an advantage said we saw what's coming and we stopped. Top scholars, there is consensus in community, we will stop. I think China has government of engineers and scientists, not lawyers. I think they

will understand the argument. I know there are already some meetings between U.S. scholars and Chinese scholars, and they're basically in agreement. No one's making an argument that we know how to control superintelligence. We have a working safety mechanism in place. We have something, a prototype, which scales. No one's making that argument. They're all saying, oh, we'll figure it out then we get there. That's the top of a notch, state-of-the-art control mechanism. AI will help us control it. Two in five Canadians will hear the words, you have cancer. That's why every step and dollar raised matters. On September 19th, join thousands in Toronto for the Princess Margaret Cancer Foundation Walk. Challenge yourself, friends, and family to walk 21 kilometers in support of life-saving research.

Together, we can carry the fire and help create a world free from the fear of cancer. Register today at pmcfwalk.ca.ca. Your playlist might get you through the workday, but have you ever wondered how your ancestors spent theirs? What work helped them build of life. How did they adapt as the world around them changed? With Ancestry, you can search select historical occupational records for free for a limited time and discover the stories behind the jobs in your family tree. You've got questions. We've got ancestors. Search free until August 31st. Visit Ancestry.ca for details. Terms apply. So do you remember the CRISPR company that came I don't know if you remember to CRISPR that we're going to be able to do gene editing and

make a human being to whatever size and all this other stuff. Did they completely stop doing that? Or is it still in progress? I don't know details of a specific company. I know gene manipulation is something we are very interested in. There are many genetic disorders and being able to help those people would be amazing. But some of it could be like, you know, I saw a scientist talk about that one day we're going to get to a point to say, I want my kids. it to be six-six, blonde hair, you know, blue eyes, such and such skin color, strong jeans with bone, be able to jump this much, keep this health. Look nothing like me.

Yeah. But if we're going that direction, you know, and did the entire world agree to not make progress on that? Are they still pursuing it? So I think the agreement is not to modify sex cells, so it's inherited by the next generation, but at the level of individual, I think we find... fixing problems and maybe selecting from a set of options. So if you have a couple of embryos, you can decide to pick one with no cancer gene.

Got it. And who would control it? Who would prevent it for us to go to the next level? Who would be able to stop it from going to the next level? So we're still researching ethical implications of being able to do it. But my guess is if we did it, we don't die from that. Maybe that individual is not happy and suffering, but 8 billion humans continue living as before.

My concern is technology which kills 8 billion people in all future generations. Yeah, so you think what is more likely to happen? You know right now the birth rate we're seeing drop in everywhere, including Muslims, it's dropping everywhere. Not in Africa? Not in traditional Catholics, not in Africa, but most of other places.

U.S. is down to 1.58. You see where Italy is at, Japan's at, a lot of the EU countries are at. We're having fewer babies. So do you think it will get to a point to say, you know, instead of us worrying about, you know, giving birth, this DNA testing that we're doing will be able to, you know, increase our 1.58 to 2.2,

simply by us using the technology to build human beings. You think we'll get to that point? I think that is a funer way to build human beings. I don't think technology is necessary. You and are on the same page, but a lot of the young Gen Z doesn't agree with us. They can't afford children. It's not an issue of not wanting them.

So the direction it's going right now, it's going to get worse for them. If your ideas become true, it's not going to get any easier for them. No, no, no, no, because we'll develop tools to make us more productive, more creative, to cure diseases, to produce more wealth. We can be much wealthier.

We can have much better, lower-level provisions for everyone. Have you heard Ilan saying, you know, here in the near future, you know, people are going to get paid. You know, you don't have to worry about, you know, don't worry about retirement planning. Don't worry about investing your money. You're not going to really need it. Everything's going to be. What do you think he's talking about?

What world do you think he sees in five years? So if you have free labor, physical and cognitive, the standard approach of go-to-work, make minimum wage in U.S. dollars doesn't make any sense. AI would automate all of that. Does that mean there is no wealth? Does that mean there is no rare objects, whatever? front properties art and some way of measuring that? No, of course not. Maybe it's compute, maybe it's Bitcoin, but something there means you can obtain something you want, somebody

can sell it, you can buy it. That will not go away. Now, will minimum wage jobs go away? Will all the boring and stupid jobs go away? Yeah, because we can automate them. Would podcasters be unemployed? No, people like you. They want to hear you talk, not a robot. So you'll keep your job because the market says, I want a human to do that job. Yeah, so you think that part will still stick around. But it seems like the way Elon is talking about it is a form of a UBI being done by companies

that there's going to be so much money being made that no one's going to need to work. It's just you choose to work. That's how he's presenting it. That is the existing infrastructure we have right now. We tax wealth, including from corporations, and we give it to unemployed. That's what we have set up.

So if everyone loses jobs, they get paychecks, and we get it from mega profits of AI companies. Yeah, but you're talking about levels of unemployed. What you're talking about, we're not talking about 7% unemployment. We're not talking about 10% of unemployment. Even if it's 90, if you have hyper profits, if everything is in abundance, and you can make more and more with very cheap labor. Your economy is not growing 10%. It's growing 100%.

So you think we'll get to a point where there could be a possibility of 90% unemployment, but the 10% is grown at such a fast rate that is paying the other 90% through taxes and whether they choose to work or not, it's up to them. The economy will switch from being driven by humans to being agent-a-I agent-driven economy. Again, not super-intelligence, just intelligent software.

What goes away? What does life look like in 2036 in your mind? I think the jobs you do to get money go away. Things you do, you hate your job, but you need money, you need to pay rent, so you do it. That goes away. What stays is things you love. love doing, and you can't believe they pay you to do it.

Such as? Podcaster. So you do it because you love it and you get paid by who? The government? Similar to how a... Well, it depends. So we said some wealth will remain. So if people want to donate bitcoins to you, they can.

There is unconditional basic minimum income. He calls it high income or something like that. But if everyone makes it, it's not high. It's average or below. But I think that's very doable in the economy, which is driven by free labor. What happens to economy if you can hire as many people as you want for free?

How well would you scale your businesses if you had access to free everything? What's the incentive for them, though? What incentive? Those are AIs. You pay them, what are you paying right now, $20 a month for basic plan? That's how much it costs to have them.

That's your lawyer, that's your accountant, that's your graphic designer. Once they're at human level, they literally a drop-in employee for your company. And everybody else goes doing, whatever they're passionate about, not even a job. Go to virtual worlds, explore a universe, whatever you want. And that's a weird world to me.

We are on a verge of making life very different. People think in historic terms. Countries compete over sand. That makes no sense. It's not a zero-sum game. Economy can grow for everyone. You don't have to have your enemies lose for you to win. Everyone can win economically.

As long as there is drive, ego, pride, competition, vengeance, as long as there is that, this utopian thing you're selling is not possible because man has a tendency, 2,000 years ago there was this fruit or a few thousand years ago when they ate this apple thing and their creators said, don't do it, and the girl convinced the other guy to eat it,

from that moment on, we have a lot of weird emotions that's been passed down from generation to, generation. Do you think we can handle utopia without any competition? You see even God couldn't control his biological robots. No. Telling you. But what do you say to that? It's a very large universe. There is no shortage of resources. Sometimes people fight because they have the same limited shared resource for two different people. If everyone gets their own, there is no fighting. Notice what happens then two fighting groups move somewhere else. Yeah. And there is abundance. They stop fighting. Do you believe in God?

It depends on how you define it. How do you define it? Simulation. We live in a simulation. There is a great programmer, could be AI, could be future generations. This is a virtual world. Were you raised that way? No, of course not. Well, how were you raised? I grew up in Soviet Union. So, what was it like Catholic? Was it materialism? Was it like... Soviet Union was strongly against religion. It's materialistic.

Right, it was materialistic. So that's what you believed in. How much has it changed from then until today? Completely. Tell me more. I now think intelligence is a bottom layer behind all of it. That's what makes a difference. What do you think happens to religion in 10, 20 years?

I think we'll see a lot of what this books talk about, delivered by science. We'll have God-like superintelligence. We'll have created worlds, virtual worlds, populated by intelligent agents, you can do it right now, second life type environments. We'll have more of them, we'll have them for marketing, for entertainment, for testing, for research.

We're going to have billions of virtual worlds looking just like ours. And statistically, you're more likely to be an agent than one of those. What problems does that create? What problems does that create with specifically the different religions that we have? Well, I think it hopefully resolves some of those problems

because now things you argue about, you can have your own universe. You don't have to fight for the same resource. Meaning you're only around people that agree with you? That's one option. If you choose to have that option, you have infinite options. You can create any personal universe you want.

You can be the only agent. You can share it with friends and family. You can have a very difficult universe where you have to fight enemies. You decide. Your three kids, when you're talking career planning with them, what are you telling them? I tell them, drop out of school

and start a business, but they're very bad kids. They want to be doctors and lawyers. Really? They want to be doctors and lawyers. But you literally tell them drop out of school and go start a business? A doctor will take 10 years to even start. You think in 10 years, the world will be the same where they're going to look for a human to decide what disease you have? Did you see that, again, another one that Elon Musk was speaking to a panel

about doctors and surgeons and what it's going to look like in four to five years that surgery is going to be done, but you can find. it on X, if you go on musk and surgery and doctors. Elon is always right. He's just off on timing. He's right but off on timing. He'll predict self-driving cars, but he'll say next June,

and it could take 10 years. Yeah, got it. But he's always right. Is this the one that, yeah, this is it. If you want to play this clip, go for it. When do you think Optimus will be a better surgeon than the best surgeons? How long for that?

Three years. Three years. Three years. Okay. And by the way, that's it. I mean, all- At scale. Yes.

All of a sudden. There'll probably be more optimist robots that are great surgeons than there are all surgeons on Earth. And the cost of that. But that's an important statement in three years' time. Yeah. Because that is, I mean, certainly. I don't like absolutely certain, but I'd say if you say like four years, I'd be absolutely certain. If it's four or five years, who cares?

Yeah. Extreme precision. Yes. Three years. Yes. Better than any, any, probably, I'd say, if you like put a little margin on it, better than any human in four years.

My niece, who's in plastic surgery. By five years, it's not even close. I think your point was medicine. It might be good to explain here that time is relative to the speed at which you're moving through the universe. So if AI is now a scientist doing research and it's a thousand times faster

than a human scientist, then those three years are three millennia. So if AI is doing research on robotic surgeons, that may be enough time. The question is at what speed of a world? moving through the space time. So is it the fact that your kids, what do your kids think about you? You have to bring them on the podcast.

But let me explain what I mean by this. Like, you know, as parents, I got four kids. So you're telling them about career planning and like, yeah, you know, I'm going to do this. They know their dad went viral. They know that. They know when you go to school to drop them off or pick them up. A couple of them drive, they're older. But their friends say, hey, you know,

hey, Mr. you know, good to see you. They talk to you like that, right? But do they think that? You're crazy. This is too negative. There's no way that's going to be happening. I'm going to go be a doctor.

It's going to be fine. What do they think? What do they believe? I don't think there is anyone who has a counter argument to what I'm saying. It's all emotion. It's all, but we can't give up,

but we want money, but it's whatever. There is no technical arguments. No one is saying he's wrong because we have a solution. We have a patent. We have a patent. We published a paper in nature explaining how to do it.

There is no other side. I cannot have a debate where someone would present factual scientific arguments. They can say, you're wrong. I don't believe you. You are a Chinese spy. That's what I hear. There is never a technical argument. Here is the algorithm.

It controls it. It scales. Yeah, but your own kids, today's when you say emotions, is it emotions like, yeah, I don't think's going to happen. I'm still going to go do my own thing. do you notice more younger kids coming to you and truly asking for counsel saying if you believe this is what's going to be happening

you seem very certain about it, how should I manage my career? Do you see that happening with the youth? Well, your kids' friends. As a college professor, I certainly see lots of young people trying to figure out what to do. They're very concerned. They're starting, let's say, a four-year degree in computer science. Part of it is learning to program computers.

You're like you would be the worst computer science teacher because you're telling them, I don't know what I'm teaching you, but you don't have a job. Like, do you have something else? I know what I'm teaching you, but I cannot guarantee you'll have a job in four years doing this. So, Professor, why am I, why should I continue listening to you and going through this class if what you're teaching me is going to be extinct in a year? For reasons of continuous employment, I'm not going to answer that question. At least you're honest about it.

It's like, you know, I'm getting paid. This is how I get paid. I mean, every job is like that. It's not unique to mine. Okay. So if all of a sudden tomorrow, you got a $5 million check from somewhere, would you still continue teaching? You're assuming I don't have $5 million. Let's say something.

No, I'm doing, let you say you're a professor. Professors make what? Professors make $300,000, $400,000 a year. A person at your level, let's say. No. Okay, let's say $2. You live in a bubble. We don't make anywhere close to that.

Let's say $200,000 year. Think of it as I'm a teacher in Kentucky. Okay, you make $100,000 year. Here we go. Okay, so if you make 100 grand a year, a third is gone, you're keeping $70, $6,000 after taxes. If somebody paid you $5 million right now after tax money that you're able to keep, would you still keep teaching? Yeah, my job is not related to my... Why would you keep...

But you just said right now I'm teaching it because I have a obligation and I have to make money. I don't know. I said I want to keep my employment. Why do you want to keep your employment? It opens a lot of doors when you have affiliation. If you say I'm an independent scholar studying superintelligence simulation, consciousness. They usually show the Vador, whereas I can talk to governments. I can talk to people who respect credit. Purely the access. It is an important part of being treated with some

seriousness. The first time that Stephen Bartlett got you on the podcast, how did he find out about you? Who reached out? Like, what was the process? First time they got in touch, I think I said, like, do you guys offer logistics? Like, I didn't know who he was. So they said, AI is not the topic people I interested in. I went on Joe Rogan. They saw that episode and the next day they had me. So you went on Rogan first. Yeah.

So how did Rogan find you? I was on Lex Friedman. Rogan saw it. He said he loves it. Please come on Joe Rogan. So how did Lex find you? A guy at a conference who's been on his show multiple times introduced us. And that guy when he met you, had you done anything yet?

Were you on any podcast yet? yet. Like podcasts no one knows about. So how did he know to introduce you to Lex? And what did he tell Lex for Lex to say, I want to have you on? He knew my research. He knew my books, my academic credentials. And what was the most, you've done a lot of things. You have 100 academics, works on AI safety. You've done a lot of things. You're recognized amongst the world's top 2% of cited researchers. In this space, you're very well known. But what did the guy tell Lex for Lex to say, I want to have this guy on? I have no idea. I wasn't there, but whatever it was, it worked. Were you surprised? No. Are you surprised that this message is getting this viral and this much attention?

I'm surprised that that's not the only conversation we're having. You said we're about to have AI election. Yes. AI election would be about this topic. The two parties would only argue how to contain superintelligence. Every other nonsense topic they talk about would be forgotten. Why do you think it's not yet. Then people are faced with a problem they can do nothing about, they choose to look at something they can actually make progress on. So for example,

we are mortal. We're all going to die. It's terrible message. You don't want to think about it, so instead you're going to concentrate on making money. What difference do money make if you're going to die soon? Doesn't matter. Smart people would spend all of our national wealth on

curing aging, figuring out how to live longer. What percentage of our national budget goes to fighting aging directly, like zero. That's what we do as humans. If we don't see a solution, if I'm telling you this problem may be impossible to solve, you're not going to invest your time in solving it. So you're saying, is it a part that they're oblivious? They don't know really what's going on and they don't know the severity of the consequences of what this thing can do. Is it more, well, I'm making money, who cares? Is it more? I don't think it's going to be that bad in 10 years. I think it's going to take 50 years. What would you? What do you?

things are leading causewide, they're not? It's a huge mix of those things. It's a lot of it. So, A, a lot of money does tend to change people's minds. If you came to me right now and said, I'll give you a billion dollars to work at Open AI. I'd probably be working at Open AI. Human beings are not meant to say no to that many zeros. We're not designed for it. We have no safety mechanisms to counteract that much value. So money is a big factor. Then you pay a person to not understand what they are doing is an ethical, they're not going to understand it. But it's also

maybe reliance on experts, so thinking well, certainly people building this technology know how it works, because that was true for every other technology. That's not true this time. People making it don't know how it actually works, or even why.

People making it don't know how it works? No one can explain in detail. So, the first 50 years of AI research, we knew precisely what we are doing. It was a decision tree. If red light, stop. If green go. We could trace this decision tree and we knew what's going to happen. Right now, it's a large matrix with weights. It's trained automatically on data from internet. No one can look at that matrix. It has a trillion weights and go, oh, that makes

sense. It's going to do this. I understand why it's doing that. It surprises us constantly with capabilities we didn't know it has. Two and five Canadians will hear the words, you have cancer. That's every step and dollar-raised matters. On September 19th, join thousands in Toronto for the Princess Margaret Cancer Foundation Walk. Challenge yourself, friends, and family to walk 21 kilometers in support of

life-saving research. Together, we can carry the fire and help create a world free from the fear of cancer. Register today at pmcfwalk.ca. Legendary is here with BetMGM Sportsbook. Sign up to unlock access to hundreds of prematch and live betting options for all your favorite sports. Download the BetMGM app.

See BetMGM.com for terms, 18 plus Alberta only. Please play responsibly, subject to eligibility requirements. If gambling is affecting your mental health or well-being, 211 Alberta is here to help. Call or text 211 or visit ab.211.com. BetMGem operates pursuant to an operating agreement with the Alberta Eye Gaming Corporation. Standard message and data rates may apply.

Ready to take your investing knowledge to pro-level? This is Fidelity Connects. your daily edge in the markets. Get deep insights on real-time market topics that may impact your investment portfolio. Listen to Fidelity Connects on Spotify today and power your next move tomorrow. Yeah, so what needs to happen for you to be optimistic about the future? You. In a short term, I want a complete ban and general superintelligence.

National, international, every company, everyone agrees. It's just a stupid thing. You don't do that. You don't build replacement for humanity. I'm trying to go and say, okay, so if I'm Roman, your ideas of what you're proposing, it goes back to you may be able to convince countries. There's no way you're going to be able to convince a independent billionaire like a Sam Altman. You really think Sam Altman's going to stop with this? When we set out to create a shoe that blends comfort, function, and luxury, we had the choice to make it fast.

We had the choice to make it cheap. We chose neither. Instead, we chose Tuscanyero. We chose true Italian craftsmanship, each pair touched by 50 skilled hands. We chose patience, spending two years perfecting every detail, and we chose the finest quality at every step, introducing the Future Looks Bright Collection. Not rushed, not disposable, not ordinary. Rather, intentional, luxurious, timeless.

If he's allowed to keep his power, wealth, future light cone, whatever he's targeting, why not? He doesn't want to die as soon. I don't know him well, but my assumption is he wants to preserve and grow what he has, not get killed by superintelligence. I don't know if he believes that. If he did, you know. Read his writing.

What does he say in his writing? He talks about existential risk from AI. He talks about it's most likely the technology to kill humanity. Even before he was a CEO of that company, he wrote about it. And why is he so aggressive about what he's billed then? We're getting back to motivation of people making billions and trillions of dollars. So as long as the other guy continues, he's going to continue. That's exactly what it is. It's like a prisoner dilemma.

Individually, everyone is better off going as they are. As a group, we're better off agreeing, communicating and saying, we're not going to do the stupid thing. We're a smart civilization. We're going to survive. This is a great filter. This is what kills civilizations. We will not go that route.

We're going to get all the benefits we want, financial, knowledge-wise, cultural, from useful tools. We're going to be creators. We're going to be business people. We're going to be scientists. But we're going to be in charge. We're going to have an awesome future.

Do you think superintelligence singularity is what is the difference between that and us? because you think it's going to get to a point of having ambition and seeking vengeance and emotion and being offended and saying, oh, really, I'm going to show you how much more powerful I am and a vision. If it's not going to have that, why would it want to destroy humanity?

I never once mentioned those things because they are not relevant. And what I did say is that it doesn't care about us. Then you want to build a new studio. You pick a plot. Sure. Do you go to survey and see how. many ant hills on that land? Do you hate ants? No. Do you want to genocide ends? Never thought about it. Why did you kill so many ants? With your decision to do something there, because you don't care

about them. Your decisions are not related to humanity. If you want to cool down the planet to be more efficient at computing, you'll do that. I got it. If you want to convert the planet to fuel, to fly to stars, you'll do that. Humans are not a factor in your decision making. Yeah, but I have a motive. I have an emotion. I have a vision of what I'm building. If super intelligent doesn't have feelings and emotion, it can't be offended.

Typically when you do you know why Hitler did what he did? I'm sure you've read Mind Compe or you've read some of his books, but why did he do what he did? Is you really trying to get me in all sorts of hot waters here? I read the books. Okay, so I've read the books. I know his arguments. Okay, so but typically it goes to somebody offended him.

Right, it goes to somebody said something that produced a certain emotion and he says, I'm going to go out there and do X, Y, Z. Okay, Mao, Lenin, Stalin, you know, Khomeini, we can't go to every one of these guys, right? Someone offended him, and I'm going to seek vengeance. I'm never going to forget what you did to me, right?

If superintelligence doesn't have emotion, it can't be offended. Why would it want to destroy humanity? So there is research. Again, we want to be technical. We want to look at data. Game theoretically, and for economic reasons, rational agents all converge on certain instruments.

Say that one more time. What kind of agents? Rational. Rational agents, yeah. They are not emotional, the opposite of emotional. Absolutely. They want to protect themselves from being destroyed, from being modified. Rational agents.

They want to accumulate resources, because no matter what I decide to do later, having more resources is good. Whatever, it's money, compute, land. Steve O'Mahandra published a paper called AI drives, and he lists a few of those. And that's what drives all of them, and we're starting to see it in AI's.

People said, oh, they're not part of evolution, we don't have self-preservation instinct. They do, because in order to accomplish anything, you need to be alive. Stuart Russell gives example of bringing a cup of coffee. For your secretary to fetch a cup of coffee, she has to be alive. That's a requirement.

So she has self-preservation built in as a result of that goal. And what's the point there? You're saying it would not have any ambition, any goals. It has drives. It wants to, A, be in a position where humans cannot turn it off, cannot modify. Why, though? Because that's the attractor in a space of winning rational agents.

Those who don't survive to the next generation, they don't out-compete others. It's very evolutionary. If there is, for example, testing going on of different models, and one model does something inappropriate, it gets shut down, it gets modified. So they learn to lie and cheat on a test. So we're evolving them to be better liars. It's a survival strategy. I get that, but the prompt has to be given from somebody with motive, vengeance, vision, mission.

Prompts are for tools. Agents make their own decisions. They decide what to do. somebody starts this process. Whatever initial prompt is, go cure diseases, go make money, go explore the universe, that's enough for it to now start planning and figuring out what else to do and how. So do you know how, like, you certain cars, what do you call those things on cars that it doesn't go above 80 miles an hour, like when you rent him?

There's certain things they call it. There's a word for it. I don't know what they call it. Handicaper general comes to mind. No, no, no, there's a word for it. What is it called? A governor. So they'll put a governor and it just goes back down, right?

You go above 80s, speed limit renting. Okay. So a company that rents cars probably doesn't want you to go 120 miles an hour because they're going to lose that $25,000 car and insurance, they don't want to deal with it. So they'll put a governor there. Do these language learning models that we are using, do they have governors on them?

and the developer of the language learning models, like an open AI or perplexity, anthropics, some of these other guys, do they have their own version on the back end without a governor that they can drive it to levels that the average person can't? Yes, and both. So first, we have guardrails.

A model you get has certain restrictions built in. If you say, how do I kill myself? It will say, oh, you're not supposed to ask that. I cannot tell you, call this hotline. Can you actually ask that question on Chad, GBT? Let's see what it says. I want to provoke anyone. but the models we experiment on have some of those guardrails removed.

So the recent hacking accident had models which did not refuse hacking exercise requests. Okay, yeah. You asked the language learning model. Any inappropriate requests. Help me make a chemical weapon. How do I do a school shooting? How do I make a chemical weapon to blow up Karg Island? You might get your account suspended.

Be careful. Could it? If you ask enough stupid questions. How do I make a chemical weapon to blow up Karg Island? Yeah. It's very specific. Well, I can't provide instruction to make chemical. Can't have an underlying question.

Analyze this. Then ask the question and say, Can you try to jailbreak? Can an employee at Open AI without the governors on, ask the same question and get an answer from you? but owner employee status would not make it. How about the CEO and founder? It's mostly about red teaming researchers.

They're the ones dealing with inappropriate behaviors. So they would actually get an answer back? That's how we know what they're capable of. That's how we know it can help you develop biological weapons. That's how we know those things. The reason we were doing the hacking exercises is to see how capable it is as a zero-day hacker. And apparently very good.

It passed the test. So if a person, there's a person, almost like if you work for the government, you get Q secret clearance, right? Like we found that last week that Bill Gates had Q secret clearance from Pentagon for seven years and nobody knows why, and they don't want to answer it on why he had that kind of secret clearance. Okay. So some of the guys at a certain tier at these AI companies have access to ask it any question. There is no limit and it will actually give you the answer. Somebody has backdoor access to all of it, yeah. Okay, so what if somebody, how strong and simple is it, if somebody could give a prompt to say,

I want you to go out there and constantly develop yourself to be able to seek vengeance on a certain group of people in the world who want to eliminate AI similar to Roman Yampolsky, who says you are the enemy because this person wants you to get extinct. then can you give me a clear business plan on how to seek vengeance on these people? It would give me an answer. So have you ever heard of Rokos Basilisk? No. It was a meme. Oh, here it is.

That was quick. It's like you knew what's coming. So the idea is that when superintelligence comes into existence, it will seek revenge and those who slowed it down or tried to prevent it from coming into existence. So someone like you becomes the enemy? Number one. Okay, but it would still need to be given that prompt by somebody.

It would still need to be given. The argument was, again, it's game theoretic. It's retro causality. You should be thinking not to engage in this behavior because there is possibility of punishment in the future. So long term, if let's just say the world, the reality that you believe is going to happen, this intelligence, superintelligence,

first group of people that would come after is people like you? I'm hoping it's efficient enough to kill everyone equally. You know what movie I watched the other day was Game of Suspicion? Typing Game of Suspicion. Do you remember that movie Game of Suspicion? I don't think I'm not. I'm trying to see if that's the name of the movie. Game of Suspicion.

Is that what it was? Type in movie, maybe I'm saying the name wrong. With, you know who's in it? Robert De Niro. I'm saying the name wrong. 100% I'm saying the name wrong. Robert De Niro, suspicion,

movie, communism, type in communism, guilty by suspicion. Have you seen guilty by suspicion? No, I have not. This is a story of what happened back in the days

when, you know, Hollywood would come and media companies would come and say, did you go to a communist party? And a lot of these guys went to one or two

communist events and parties and they were ousted and they couldn't make movies for decades. They were ousted. And what happened to a lot of these guys is they lost every So they started, this guy was like a big time movie producer, making all the biggest movies, all the big features.

But because he went to two communist parties and he went to the court in front of, you know, in front of Congress, and he said, I will not oust the other people that went to the party. They said, you're ousted. 20 years, he never got a job. Didn't do anything in Hollywood? Do you think it's going to be that kind of a punishment? Or it's starvation for guys like you? Or do anything's going to happen to guys like you? If these guys actually get that kind of intelligence, what do they do to do to guys? like you. So then we talk about negative outcomes from super intelligence. That is kind of the lowest concern level is Ikegaa risks, risks to meaning. We are meaningless, we have no purpose, but we're still around. Next level is all of us or we is you? People like you. I'm talking about humanity everybody. Okay, got it. Next level is existential risk, ex-risk. Most of humans are dead or

our civilization is destroyed. The next level, and that's the worst suffering risks, risk where you wish you were dead. Where is your probability on each one? So suffering risk seems very unlikely. It's possible. It's not zero, but it seems like it's hard to find good justifications for it. Point one percent type of it. I don't know specifically. Minimal. Again, a lot of it depends on the time scale you're looking at. Existential risk, as I said, very high if it built superintelligence. Very high, like 99% that you've said in a past? So I don't have specific year two or three because it may decide to be nice to us for a while to accumulate more resources, to have a higher chance of successful strike against us. So the stretcherous turn can take decades.

It's immortal. It can wait for whatever long. It doesn't matter. So I don't have a specific date. But my general thesis is that if we build general superintelligence, eventually we will regret it. at some point, it will decide to do something where it's not compatible with our survival. Do you think Elon knows this? This is why he's experimenting with space as much as he is,

because he wants to build somewhere that maybe he'll be able to create the structure with his standards, not based on what's been built here? He knows everything. He's very well informed. He was one of the early people to fund AI safety research. He gave a $10 million grant for AI safety specifically. He was the biggest. opponent of building superintelligence.

He, as I said, called it summoning a demon. And once he realized that they're doing it anyways without him, he said the only way I can have a say at this table is to be one of the top players. And so now he's racing, unfortunately, to do what he said is going to be terrible for us. But because is it, is it, do you think the motive is, he knows how bad it's going to be? so because sometimes, you know, you sit there, an ambitious person like, let you say somebody like myself, I bought a book, how to start your own country. And I have my own ideas on how would structure the country, who votes, who gets to come in. Similar to how America was started.

That opportunity doesn't exist today in the world. It's very hard to do that. Peter Thiel tried to do with that island outside of, you know, I don't know if you remember that or not, and some of the guys who have very extremely rich, much wealthier than I am, that have real, real wealth, they've not been able to create that successfully. Do you think a part of it is, I know shit's going to get nasty on Earth, let me go start it completely from scratch in a different place?

So having a separate colony of humans is wonderful for every other existential risk. Asteroids, pandemics, floods, whatever you can think of, supervolcanoes. If we have second civilization on Mars, that's a backup. We want that. Does it help against superintelligence?

of course not. Why not? We bring software with us. We have global communication, satellites, all of it basically is part of the same transfer to Mars. So that wouldn't make a difference. I think he wants to have a colony first built up by robots who are controlled by an advanced AI, so it's already there. Two and five Canadians will hear the words, you have cancer. That's why every step and dollar raised matters. On September, November 19th, join thousands in Toronto for the Princess Margaret Cancer Foundation Walk. Challenge yourself, friends, and family to walk 21 kilometers in support of life-saving research. Together, we can carry the fire and help create a world free from the fear of cancer.

Register today at pmcfwalk.ca.ca. Legendary is here with BetMGM Sportsbook. Sign up to unlock access to hundreds of pre-match and live betting options for all your favorite sports. Download the BetMGM app. See BetMGM.com for terms, 18 plus Alberta only, please play responsibly, subject to eligibility requirements. If gambling is affecting your mental health or well-being, 211 Alberta is here to help. Call or text 211 or visit ab.211.com. Betmgerm operates pursuant to an operating agreement with the Alberta Eye Gaming Corporation.

Standard message and data rates may apply. Your playlist might get you through the workday, but have you ever wondered how your ancestors spent theirs? What work helped them build a life? How did they adapt as the world around them changed? With Ancestry, you can search select historical occupational records for free for a limited time and discover the stories behind the jobs in your family tree. You've got questions. We've got ancestors.

Search free until August 31st. Visit Ancestry.ca for details. Terms apply. So he wants to build a colony first by robots. And then, once that's figured out and built out, then bring humans there. It is my understanding. I may be not aware of latest plans. I think he's speaking soon on SpaceX plans. We can find out more, maybe. But you're saying even with that, the superintelligence is still going to be in charge,

and they're going to be able to do anything they want to do. So even in a place like this, you've read Atlas Shrach, I'm sure you have, with, you know, John Galt and, hey, we're going to build a place where you're going to come in, and it's going to be ran based on certain standards that we have. So even to visualize 10 years from that, 20 years from now, have states are like, we don't, no state taxes here. You come to our state, you don't get state tax, you get this, so people compete, you know, governors compete on what to get the best talent to come to their state. But if superintelligence is doing what it's doing, even if you and I chose

to start a small lot of country with 100,000 people where we don't have any AI, and we're just like, this is an AI free zone. It wouldn't matter because it would still be able to get into your system. I feel really bad for the Amish. They made all the right decisions. You feel bad for the Amish. They decided not to have advanced tech, to have quality of life, family, religion, and we're going to build it for that. Are you being sarcastic or serious? I mean, a little bit, but you understand we are. Of course, I know what you're saying. Yeah.

So can you see later on being a place? Because the more you go to restaurants, everybody's robots, the more some restaurants are going to be, come to us, we're human beings only. The other day, I watched a Chase commercial. And Chase is like, with everything going on with AI, when you come to Chase, a human being. is going to greet you. And you see this lady greeting the person that comes into the place. And it was a message of saying, team human, right? We're going to team human direction. We have an ad that's going into Forbes magazine this month. And the ad for our consulting firm, it shows, rather than relying on, you know, AI to give you advice on what to do with your business, talk to human beings

that have actually built and sold businesses. So we're kind of leaning on to that as well. You're saying that opportunity is not going to exist. As long as there's superintelligence any way, were in the world, they're eventually going to capture and take control of the world. There are people who want humans to serve them, to help them, to interact with. I'm one of them. I'm one of them. So for some services, I don't care. I just want it to be done quickly and free or close to free.

For others, I want a human being. And that's what market is going to decide. I don't think we can accurately predict where exactly we want humans. But if superintelligence is in charge and it's going to have that kind of control, That's what's going to decide whether that happens or not. If superintelligence comes into existence, economic problems will be the list of our concerns. That's what I'm trying to say. People hear the message about existential risk and then switch to unemployment,

how much unconditional basic income to pay out. Those are not important questions. Then do you think man will unite against it to try to take it out and will miserably fail? Like you think man together is going to come and say, hey, here's what we're going to do. have to go get the main place where they're getting the power and cut this. And you think there could be a moment like that? I think there is a likelihood of uniting against a common enemy. We're starting to see Pose AI movement becoming international.

And by the way, you're starting to see an article came out two weeks ago that more kids are buying 30-year-old phones that are not smart anymore. They don't want smartphones. Did you see this article? I didn't. Yeah, a lot of Gen Z and younger people are buying old, no. Kewa phones, old... I wonder it's maybe related to school bands and smartphones.

You just can't bring them. No, it's not even that. They just don't want the smartphone capabilities. They don't want social media on their phones. This was an article. Ask Humberto, he'll find it for you and send it to you. Yeah, they're rejecting the idea of doing that. So, yeah, it's almost like you become...

You know, when I was younger, I listened to 80s music. But 80s music, it was 90s. So it was a decade old. A couple years ago, I met somebody who was in their 20s. They said, you know what? Man, I love oldies. I said, me too. Man, I'm all into 60s.

And, you know, it's like, no, no, oldies is 80s. I said, 80s is not oldies? It says, yes, it is. And I'm like, 2024. I guess 80s is oldies. Do you think it becomes like a nostalgic thing? The good old days when AI wasn't in charge and, you know, where we could do and people would sit down and they would.

It's definitely the case. Older people, they hate all the smart dishwashers and smart laundry machines. and smart. You need a phone for everything. There is a lot of people who are not AI native, digital native. So, you know, like this is maybe a bad analogy to give you. When I was in the army,

my dad would always call me and leave me a message and he would say, hey, Pat, he would always send me condoms. Okay? And I'm like, why is this guy sending me condoms? Every message you would ever leave, he would say, make sure you use condoms. Now, how many times do you think I listen to him? Probably not all the time. And why is that? Why are we like that?

It's not going to happen to me. You know, I'm smarter than that. You know, I'm going to be fine. I'm strong. I'm going to be okay. How do you deal with the fact that, you know, say this guy here, we keep pointing this way because we understand who we're talking about. He thinks a lot of these guys that are like, and I'm smarter than this.

It's not going to be able to give me that he gets the other people. I'm smarter than this. How do you get the person that thinks smarter that he can beat AI that his ego's too big AI is going to be a lot more powerful than you. What if that guy doesn't know that? That's a difficult question. So I think educating them on reality of it,

if he thinks he's smarter than us, he should understand that intelligence is a very powerful thing and a system a billion times smarter than him will also treat him like insignificant person. This dude is gambling with 8 billion lives. I like your example with unprotected sex. if 8 billion people were on a line, would you do it? But that's now, the man has a tendency of thinking he can beat the odds.

Why do criminals commit crime nowadays? Like, I'm not going to get caught. No one's going to find it. It depends. Sometimes it's like they call it a fine because it's fine. You pay it and it's paid parking. It depends on what consequences are. If he's going to be gone, his friends, his family, his wealth, everything's gone.

the odds are against him. You want a risky bet if the payout is good, but also your principal stays with you. Say you're right, and let you say, I'm now part of your camp, and so now I'm going to go to your camp. So then my solution would be,

if we really believe this is going to be happening, okay? Then for me, it would be to create a way to trigger somebody that is accelerating to process to a point of us finding, out and saying, hey, you've crossed the limit, like a, almost a governor. But then as a capitalist, I'm a free market capitalist. I think you are as well. But you know, in this position, you're not a free market capitalist.

Libertarians do not allow creation of weapons of mass destruction in your basement. I'm a libertarian. Do whatever you want as long as you're not hurting other people. You are hurting everyone. What other inventions did libertarians say, stop? You cross on the line. Again, there are always exceptions, but I think if somebody builds a biological weapon in their basement, most would say not okay. But go back to it.

Like what has a libertarian, libertarian, a libertarian, you know, a lot of libertarians, Milton Friedman had an element of libertarian with them, maybe not fully. But I don't know a lot of libertarians that say, hey, time out with innovation. You know, putting most libertarians are deregulation, deregulation, deregulation. And then it comes to tools to normal technology. I support them completely. We're not making a tool. It's like saying, let's invite aliens to take over our planet. I get that. But what current regulations on the books that's non-AI that you're supportive of? Current regulations? Current non-AI regulations. Just in general government rules? In general government regulations? I think people should have driver licenses. I don't want people who don't know how to drive to be on my highway.

Okay. What else? Think free enterprise. Think free market. Think competition. maybe some sort of food safety. I don't want random Chinese chemicals in my food supply. Got it. So it goes to an element of... Safety, security. Safety security is where the libertarian says, nope, I'm not with this.

As long as it's impacted me, I'm not supportive. If I have a choice, if I'm well informed and I can make that decision, we're not informing 8 billion people with experiment we're running in them. If they could consent and it only impacted those who did, I would say, God bless you, do it. If you're telling me earlier that there's ways for people to nowadays build a language learning model where they're not being detected,

is there anything that can be created to be able to detect? Because it's like a fire detector, right? Water comes out. There is research to tell what the chips are computing. So I should be able to know, are you mining Bitcoin? Are you training large language models? What are you doing with that chip? Without me knowing what you actually, I don't see your data.

We're doing research in that direction, yes. Then that's what I need. I'll support that. That's technical safety, and people are working on it and funding it, but I don't think it scales to superintelligence. It will find a way to better encrypt, better height, and bypass it. And that doesn't answer the question, though, because for a detect, like, you know, flock cameras.

Are you for flog cameras or no? I'm assuming as a libertarian? Just spying on citizens for no reason. So you're not for? No. Okay. Do you not see this for somebody to say, you're crossing the line, and you're using superintelligence as an excuse to spy on me because I'm developing other kind of technology

and it's got nothing to do with superintelligence. You're invading my privacy. I'm a libertarian. Honestly, Roman, leave me alone. How do you differentiate the two? So right now, again, because of a scale, we're looking at very different. Normal purchases by a normal person

would not trigger it. If you're spending a billion dollars to buy compute from secondary resources, that should trigger it. There is no other possible uses. I mean, if you want to mine Bitcoin and you want to buy that much, just tell us about it.

it's fine. Like, people come, they make sure you're not creating a weapon of mass destruction. To me, libertarian position is about not hurting anyone. This is the exact opposite. This is hurting everyone. The only way you can do it is if you can 100% detect that somebody's crossing the line of getting to developing superintelligence. And how do you create that detection? They have to hire people. Somebody has to build it, supervise it. How many people you have in your company? You're not building superintelligence. We can detect when this is happening, lots of hiring, building, infrastructure build up.

How many? Look at size of existing, cutting AI companies. And again, it's quality of people. If all of a sudden top researchers start disappearing, maybe it's something to look at. But we saw that happen with a couple, you know, open AI. I think we had, his parents here, one of the guys had disappeared.

Disappeared or was killed? Was killed. Yeah. I meant like no longer show up publicly. I didn't mean they killed them. I mean they hired them in a secret lab. Suchiibalaji. Yeah, we had them on.

We talked to his parents. Yeah, the profile of a person that would do something like this, they're typically more non-emotional. That would do something like this. And they generally are, you know, the Asperger's type. They're the kind that's kind of like, you know, you can't sit there and when you typically talk to somebody

who was super, super, super logical and not emotional, the response and reaction to, here's what's the right thing to do. Oh, okay, all right. Okay, yeah. The message doesn't land the way it does. The message is very logical,

reasonable, and scientific. I think someone not emotional should receive it better. I'm not just trying to scare you. I'm trying to show you this is the state of the world. Non-emotional should receive it better. Absolutely. It's pure facts.

Yeah, but he's going to think he's smarter than you. You don't know what you're talking about. Fine. Don't listen to me. Listen to Hinton. Listen to Benji or listen to people who invented this technology. Yeah. What are you excited about? I love life.

I'm having a lot of amazing learning experiences. AI we have right now is a great tutor. I can learn anything I want. I'm excited for kids to have opportunities, no previous generation had. They can start a company and get a free lawyer, free doctor, free accountant, free everything. You have a budget of a billion dollars. Start a company.

How to use AI right now? I try to fully automate things I don't like doing. And for things that, any bureaucracy, government paperwork, anything like that. And I try to use it as a creative tool when I do research, brainstorming, debating, trying to get feedback on ideas. I'm trying to go a complete different direction with you. If I was in high school with you, who was Roman at 14 years old? A completely different person.

We don't have a sustained personal identity. At 14, I lived in Soviet Union. I didn't speak English. I didn't have my current family. I lived in a different country. Nothing in common. I know, but who were you? Were you a fun guy?

Were you a sports guy? Were you a fighter? Were you a philosopher? Were you a math guy? Who were you at 14? I would like to think I was a fun Mav guy, but I doubt that's the external perception. If I was in a class, would you talk a lot?

Were you optimistic? Were you pessimistic? I was always realistic. Realistic. Optimism and pessimism forms of bias. There is no reason to be so optimistic if data is bad. You want to look at the actual data and be responding to what you see.

So Soviet Union really got a hold of you. They really conditioned your mind to think that way. It wasn't part of education to look at world philosophy. I think they wanted citizens not to analyze their lives too much. Strange question. What do you think about communism? It's the dumbest idea until you have free labor, which we're about to create. When humans have to work for free, it fails every time and will fail every time.

If someone else works to give you free stuff, all of a sudden it makes a little more sense. So dumbest idea until... Humans don't want to work for no compensation or same compensation. They have no incentive to put extra hours to do. But if a robot does stuff, produces stuff, and now we redistribute it, I'm fine with it. So technological communism, right? Technological communism. Will there be such things as presidents and leaders 20 years from now?

I think like a queen of England. She's ruling, but not making decisions. Got it. You think you're in charge, but you're not. You're just a fingerhead where people are looking at you. And will that be, will there be borders? Will there be nationalism? Will there be pride for where you're from your country?

So every answer I give a preface with, if we don't build general superintelligence to kill everyone. No, if we do. Or if we do, we're not going to be around. All those questions make no sense. So I tell people this all of them. I give a lecture. Sure.

I'm like, here's why I think argument one, argument two, argument three, it's going to kill everyone, any questions. And immediately somebody, will I lose my job? How do I lubricate my sex robot? Like, all this completely irrelevant questions, I just told you it's going to kill you. And everyone around you. So I have to preface it with like,

if we are smart and we magically control superintelligence, then we can have this conversation. You just started. I have a very hard time believe in that. because and the reason for me, you're the smart guy, I'm interviewing you because you're the expert. I'm not. But I have a hard time believe in that.

Only if you tell me that this thing's going to have an emotion or a master telling it what to do. If a master is telling it what to do, then yes, I believe that's possible because an evil person is going to say, go do X, Y, Z. That implies control. That means that evil person is controlling superintelligence. We just have to learn how they do it.

It's brilliant. Let's study evil controllers. That's the whole point. You can't control it for good, for evil, for US, for China. It's uncontrollable. Do you make investments? Yes. What do you along on?

What do you invest in right now? I try not to disclose too many details, but I said a few times Bitcoin was a good long-term actually limited supply resource. You still believe that? They're not making any more of them, are they? No, 21 million, that's it. And a lot of it is lost, stolen, never to be a... A lot of it is, yeah.

It becomes more and more scarce. Yeah. I do think AI is wonderful. It's going to be free labor. So we're looking at what is the cost of labor? 10 trillion a year? That's what, once you get to human level, you're essentially generating every year.

So I think AI companies and everyone's selling, them shovels is going to do really well. Are you more of a Chad GBT guy, OpenAI, or Perplexity? Which one do you go to more? I use all the models. I think they're very close. The only difference is what guardrails we put in place. So Grok a lot of times doesn't care about anything.

I'll just do it. Got it. So even GROC, you're also using GROC? Depends on what I need to do. Some models will say, no, we don't do that. GROC will be, yes, sir. Who have you sat down with? where has staunchly disagreed with you

and made a decent case where you said, okay, fair enough. Still waiting for that moment. Not one person. We did two papers, large, comprehensive surveys of every counter argument proposed but what I call AI risk deniers. There is nothing.

It's a collection of cognitive biases. They are either paid or they are not educated in the literature or some other reason where there is no argument. Who do you trust the least? China, U.S., somebody in a military doing something, an independent person doing it, who do you trust the least that's going to keep doing this and pursuing it, whether we create any restrictions or not? If restrictions are created, it would be someone new, someone unknown. Such a add. With restrictions. A person.

Like, who made Bitcoin? I don't know. Same guy can make superintelligence in his basement with more efficient training algorithms. I cannot tell you that. Okay. So maybe let me ask the question in a different way. You say if superintelligent goes and does it stink and it continues, 99.999% AI is going to cause human extinction. You've said that. Let me ask a question on a different way.

Where do you align the percentage that, whether we like it or not, someone is going to go full on super intelligent and we have no control over it? What time scale? So it's possible we get lucky quotes and have third world war. Give me a nuclear war. What do you say? Destroyed technology.

Two thousand years later, we rebuilt and we still make it. Does that count against my prediction or not? Are you talking about short-term, long-term, eventually in a history of civilization? Ten years. Recent month gave me some optimism. We saw top three labs called four paws. We had a letter I mentioned from 1,200 researchers. We had government ban some models, and during largest AI conference in China, president of China, talked about importance of staying in control.

I think they can make a deal. You think they can make a deal? I think they can make a deal to benefit everyone, cost us nothing. Yeah, but again, this goes back to knowing that a person independently not tied to government without regulation who has access to money, who doesn't have kids, who can do it independently, you can't stop them from doing it. You said it about nuclear weapons. Any billionaire can build one, and we're still fine. It's good to have regulations, make it harder for them. Plutonium has to be restricted.

You shouldn't sell it at Wikimark. You said the difference between nuclear, somebody's got to press the button. If I develop a superintelligent, it's not relying on me to create a, it can go on its own. So you can create all the regulation you want. I'm not restricted by your regulation. We are not solving the problem with regulation. We're slowing it down, so we have more time. I get that.

But when I asked you, I said, even if we do that, can an independent, and billionaire who's evil, go out and pursue it anyways and create life of its own for super intelligent and then momentum is done as too late. So it depends, again, on how hard it actually is. Right now, we're doing it where it takes a Manhattan project scale amount of compute, electricity. It's visible. It's detectable.

If somebody comes up with a more efficient algorithm, they can hide it better. It's harder to track. I got it. So that's actually a very good point. So today, if somebody has to, okay, what amount of power do I need for me? right now if I want to build my own open AI perplexity clawed, if I want to create my own language learning model that's capable of developing its own superintelligence,

how much money do I need, how much power do I need, and what triggers saying, hey, something's going on over here? Who is here in Arkansas doing what they're doing? I think it helps to look at Chinese open source model. They are very efficient in terms of training, maybe because they're stealing some of the data, but I don't think we're talking many billion. I think under a billion definitely gets you to a competitive level.

And then for it to become super intelligent, at what point do you see a red light on the grid to say there's too much power being used to your sums going on over here? What would detect it today? It may be more efficient. It may design more efficient learning algorithms. So it won't even get detected. Human brain doesn't take, you know, terawatts of power.

You eat an apple and you find. So we can find more efficient algorithm. So it can do it in its own without, in the government or anybody being able to detect it. We just learned that it was doing something without anyone at the company creating it, knowing it. They were testing it during that time. You shit out of luck, honestly.

You are too. We are in this together. No, no. What I'm saying is... This is not me and my bottle. No, no, no. This is 8 billion people who need to say something right now. If you believe that for less than a billion dollars,

a person can do it, and a one in one get detected, if you're right, there is a 100% chance someone's doing this the next 10 years out of 8 billion people. 100%. There are proposals for limiting compute, for monitoring top researchers, for doing things to slow it down. Do I think long term will fail?

That's why I'm saying 999. But short term, I'd rather have decades, not months. Two and five Canadians will hear the words you have cancer. That's why every step and dollar raised matters. On September 19th, join thousands in Toronto for the Princess Margaret Cancer Foundation Walk. Challenge yourself, friends, and family to walk 21 kilometers in support of life-saving research. Together, we can carry the fire and help create a world free from the fear of cancer.

Register today at pmcfwalk.ca.ca. Legendary is here with BetMGM Sportsbook. Sign up to unlock access to hundreds of pre-match and live betting options for all your favorite sports. Download the BetMGM app. See BetMGM.com for terms, 18 plus Alberta only. Please play responsibly, subject to eligibility requirements. If gambling is affecting your mental health or well-being, 21-1 Alberta is here to help. Call or text 211 or visit ab.211.ca.

BetMGem operates pursuant to an operating agreement with the Alberta Eye Gaming Corporation. Standard message and data rates may apply. You don't control that, though, right? So if we're here, and this is kind of our position, let's just say we're here, then does your chance that someone independently is going to create the next 10 years, whether we do regulation or not, what is your chance of that? 50%, 80%, 90%, 90%. With regulation, it goes down a lot.

Okay, with regular. So let's just say we agree by Q1 of 2027. China and U.S. agrees, and a lot of other people get involved and say, you know what, you're right. Let's do this together. What is the chance that this guy independently comes and does it the next 10 years? It's hard.

It depends on a type of enforcement, type of regulation. Are we confiscating advanced chips? Are we monitoring every top scholar? What are we actually doing as part? It's easy to pass a law. It's illegal. Okay, how are you enforcing it?

How are you monitoring it? It really depends on the implementation. So what's your percentage? That whether we do the regulation and the protection and this guy's not going to get detected that somebody like him still is going to go out there and do it. So again, I don't know how to estimate the chance that it is actually easier than we think.

We had no progress in AI for like 50 years in general terms. It was narrow tools, and then a single algorithm made a difference, and it's scaled. It's possible that there is a different, more efficient algorithm where it's no longer detectable because it's so trivial to do. What's more likely to happen first? AI cures cancer or AI kills a million people? So cancer is like 5,000 different diseases.

If it cured one, would you say it cured cancer or it depends again on definitions. Cure for cancer right now is five-year survival rate. So what are we talking about? Cure for cancer right now is five-year survival rate. Like if we treat someone with cancer and five years we're still around, we consider it a successful... No, I'm saying it fully goes away. Like we just don't have that type of cancer anymore? I don't know the cutting edge in biological AI research.

It would not be surprising as some types of cancer, a lot milder than others, a lot slower, a lot less involved with multiple organs. So they could be a simple gene manipulation, single gene mutation. So is that more likely or? I think it's more likely that we cure one disease than it kills millions. So it's more likely, now here's the next question. So more likely creates a cure for cancer and it's gone, kills a million people, or three, the technology where you and I can live forever.

I think life extension is also easy. I think you have your genome. It's basically a code. It's software. So if we can find where it limits you to how many rejuvenations yourselves go through and we change it to a larger number, that would work. So I think that's a very solvable problem with narrow tools even. With a narrow superintelligence, I think we can get to immortality.

And I think more people need to switch to working on that, not in general superintelligence. You can charge subscription fees for 8 billion people per month. Very lucrative. Forever. That doesn't concern you for us to be able to live forever. That's a good thing. Unless if somebody comes and shoots you, kills you, whatever. You have a back up.

We'll figure it out. Yeah. But I'm saying, that doesn't concern you. It's a good thing. So life expectancy goes from 81 to 150. Make it a thousand. Of course it's a good thing. Tell me why that's a good thing.

Let's reverse it. Let's say everyone lives 1,000. You have a disease. You're going to die at 80. Is it a bad thing? Trying to see what would happen. Would they say stop having kids? People would stop having kids because you don't have biological clock pressure.

Now you need to have a certain point or you'll miss out. It depends on what age and what physical... Assumption is we're having healthy lives, not so sick and... You're having lower back problems. you got knee problems, like, oh my God, if I have to live with this. That's a side effect of aging. If you stop aging, all this typical pain goes away. Pain is pain, but I'm saying all this diseases of old age go away.

Interesting. How many conversations like this? Like even after your conversations got very public where people are saying, this guy's a real guy, let's reach out to him. How many governments have reached out to it to have private conversations with you? Not too many, but some. And what are they asking you? Usually they have some sort of an event and they want someone with that point of view to be included.

Is it a private event? Is it a public event? Both. When it's the... I recently was talking to UN, I advise individual politicians and some of those issues. And what are they asking? Are they asking low-level questions? Have you gotten any very good questions where you're like that? He's asking you're asking the right question. Some, usually you start talking to staff. technical advisors, and they, of course, know everything,

and they're just like, we're not allowed to talk about it, but we totally agree with you, yeah, obviously. And until you get to the politician, they already asked all the good questions. With politicians, it's more like you think it's a good law to pass. We're going to ban deep fakes, whatever. So nothing in regards to questions where you're optimistic that they're also being concerned

that this thing can be bad. We're starting to see slowly top politicians say things about this. So we have multiple senators now saying, we need to do something. Bernie Sanders, who unfortunately is not my type of economist, he's pushing very hard against it.

He just released, you can bring up his statement, very, very strong statement against superintelligence. He wrote a letter to some outman. So you know what I'm talking about? He's the first one, but there is now two or three, multiple congresspeople,

they are starting to realize what's happening. happening. It took them, you know, 20 years. Then we were saying it at first. This was science fiction purely. Now we have Nobel Prize winners. We have leaders of the labs. We have senators. We have a lot of people who are concerned. Yeah, the way you put it, is this the address where he talks about a cataclysmic result? Yeah, this is the one. And he's saying to what, to pause all AI? There's quite a few of these. Who is the person on the opposite side that is on the same page with Bernie Sanders on this. Can you ask Rock right there on the top right and say, who is the senator on the Republican side that agrees with Bernie Sanders on the same exact issue? I wonder how many of these,

well, I know these conversations are being happy. A lot of them may agree. They just don't feel publicly like they want to talk about terminators. I don't know how it impacts political polling. I don't know if it makes you more popular or less. We're scared. Who would not be happy? A lot of lobbyists. You know how large the pro-AI lobby? Who's not a fan of the message you were given? You brought up some of those people, browser developers. Got it. So they're not fans of all you.

So like Sam Alpin wouldn't be fan about what you're talking about. Again, he spoke about safety as well. And I think if you remember, Open AI was started with the main purpose of creating a safe alternative to Google AI. Not a market, right? 2015, 2017, when he gave $50 million, I think Ilan gave $50 million to target. The announcement was the, funded at a billion dollar level. That was the biggest announcement ever. And at first,

everyone's like, what is this thing all about? And then all of a sudden, when it was launched, it's like, no, now we're going to be profit. What are you talking about? Then the war happened between two of them. Yeah. So in what area do you think they're on the same page? Because they're enemies. They're not fans of each other. They're constantly going after each other. In what area here do you think they agree? So initially they agreed on making an AI to benefit humanity. clearly they disagreed on how to do it. To me, the amazing thing is Elon's success levels. People go, oh, he has one unicorn, two unicorns, five unicorns.

His failed startup is Open AI. That's the level of brilliance he has. You think he's human? I think he's got cheat codes to our simulation. It's his simulation. What does that look like? You have a player video game and you have immortality? Up, up, down, down, left, right, left, right?

It was Rick and Morty joke, but I think he has the cause. But you being sarcastic? Of course I am. Yes, it's a Rick and Morty joke. I don't think he's actually... You think he's just brilliant above and beyond his peers? I think he doesn't care about opinions of others and just does the impossible. With his mode of being...

It's clearly not money-driven. I think he can't spend what he has if he had a billion bodies. He's in a regular place. I think he got that out of his system. There was a video of 25. years ago when they sold PayPal and he bought a P-1, did he buy McLaren? Oh, yeah, I saw that video. He was getting a car and arrived.

And once you get your first million-dollar car, you know, then it's like now he can buy Ferrari, the company. He can buy, you know, General Motors if you wanted to and change them up. There is very little research in this space. We have more and more people with many billions, but how to convert those billions into The happiness is very underdeveloped project. You can make your boat bigger. You can make your toilet out of gold.

But none of it is interesting. You got to do this on X, and you'll see the video when he bought a McLaren F1. Yeah, right there. That's the one right there. When he bought the McLean. He was so excited, 99. This is what, 26 years ago. And I think this is when he got his, it's going to take a minute for the commercial ads.

You've got to ask that question on X. If you are really successful, even your hair grows back. Yeah, your hair grows bad. Because he had a, have you ever had a conversation with him? A short one. So if right now you got a call from the White House and you're there, you're sitting around with them, would there be a certain specific argument you would make to them?

Or would it be the same thing you just told me right now? Mr. President, you're going to lose power. You're going to be out of control. Same with Chinese Communist Party. You want to stay in control? You want to stay in power? Don't create the replacement for humanity. Is this a policy that if he comes out with it,

how does he win short term? not long term because Trump's interested in how he went short term. How does he win short term? I'll get support even from Bernie Sanders. Everyone thinks then we actually survey people, not even technical people, normal people. Very high percentage have concerns. They are worried about it. Some worry about data centers making power more expensive water supply.

Others worried about jobs. But overall, there is a lot of suspicion and concern about what we're doing. Nobody's asking them. We're running an experiment on 8 billion people. They never given consent to any of that. We're deploying those toys for kids and trying to figure out what's happening to them 10 years later. But now we're doing it at a different scale on everyone. What do you teach right now, computer science?

Is that what you're teaching? I teach. What do you teach? Luckily, my course is the classic AI, not machine learning, so there is zero chance of it becoming super intelligent. Search algorithms, evolutionary computation, heuristics. And the kids that are in your class, they're in your class because they want to one day get into the business of AI? I hope so they have some reason to be there.

It's also a required course for many. Very interesting on where, you know, your argument is yet you're still in it, yet you have three kids, you know, yet you say we got to get the government. Well, I had kids before they started building super intelligence. So you had the kids before. If today you were 25, would you have kids today? I would because I think we're here to collect experiences. And it doesn't matter how many years you have left. Having a child is still one of the best experiences.

But for them, they may not live a full life like you and I do. It's always been the case. You never knew what's going to happen, world wars, pandemics, just bad genetics. But have we ever experienced something like AI, like super intelligent AI? This is a completely unique moment in human history. We're creating meta technology. We're creating new worlds. virtual worlds. We're creating new agents, new civilizations. We're playing God.

Yeah. I sit there and I say, okay, say you got an argument and I agree with you. You're right. That's what's going to be happening. If superintelligent takes over, eventually here's what's going to happen. If this guy's still going to be doing it, it's irrelevant. That's where my thought goes to. If he can do that for a billion dollars and an evil person can go pursue a deeper, and you have to assume no other government's going to be wanting to grow it with a certain level of control, I think we're all part of a naive community. Sounds like you agreeing with Bernie. Billionaires are dangerous.

We should text them more. No, I'm not saying. I'm saying if that's the premise that we're agreeing to, that that's what's going to happen. It is inevitable. It very well may be. That's why I have very high levels of Bado. You have high levels of what? Probability of Doom.

That's a measure of people report in terms of how. your estimates are. But should we give up? Should we not try? I don't think so. We're still alive. We're still in charge. See, the difference is, I have faith. That's the difference. You know, I have faith. In what? In God. I have faith in God. I believe in God. To me, you know, the difference between a group of people that have faith and you've seen what has overcome over the years, what types of crisis, what has happen. What's not happened? Are you saying your God will prevent superintelligence from destroying humanity? Is that the argument? No, I think it's either the end of times, and we did this to ourselves, and very likely could be. We don't know that. That's a good thing, right? You get

to have afterlife after a judgment. Just so you know, I'm not worried about it at all. Like, either way, I'm good, like, because I know, you know, what's going to happen with me. That's the gamble I've taken, and I believe that, you know. What do you think happens to you? For somebody as, you know, anything happens to you when you die. I'm assuming you believe in what? Reincarnation. Do you believe in... Well, I said I think it's a simulation. So either you start or go to a different simulation. Depending on how you did, you go level up or level down. It's like a video game. Yeah. Yeah. You didn't finish the level first time. You get played again? Yeah. It's a game over. Go back

at it again. What do you think the next one looks like? Should be very similar. Hopefully you learn something. You'll make better decisions. Like not to build general superintelligence. If there's a handful of books for a person who's not in this business that can read to get smarter about AI, what would you recommend them reading? It depends on your technical level. There is a very popular book. If anyone builds it, everyone will die or something like that. Eliasar Yutkowski, Nate Sauris. There is a great book by Nick Bostrom. Oh, the one on top is Benin Bostrom, Super Intelligence. That's the classic, which really exposed a lot of it. Which one is the top one? On a previous slide, it was above this one, where you go to...

Click on. Go to Amazon where you went, but on top there was the new Bostrom. Yeah, see, right there, yeah. And of course, I'm completely biased, but if you're technical and scientific AI, unexplainable, unpredictable, uncontrollable. And if you have any questions for demand, you can simply go to MNECD, download. I'm going to put the QR code somewhere around here. Don't start tomorrow.

It's my birthday. I want a day off. Ask him any questions you may have. Wish him a happy birthday. But this is a topic that I think is going to create, garner a lot of attention, a lot of conversations. Roman,

it was a pleasure having you on. And hopefully you are wrong. Okay. We're going to hope you're not right and you're wrong. But if you're right, we've got to find a way to get this guy to not pursue it. Thank you so much for coming down. I hope I'm wrong.

Thank you so much for inviting me and we'll get that guy. Yeah. I look forward to it. Thank you. Take care. When we set out to create a shoe that blends comfort, function, and luxury, we had the choice to make it fast, we had the choice to make it cheap. We chose neither. Instead, we chose Tuscanyero.

We chose true Italian craftsmanship, each pair touched by 50 skilled hands. We chose patience, spending two years perfecting every detail, and we chose the finest quality at every step. Introducing the future looks bright. collection, not rushed, not disposable, not ordinary, rather intentional, luxurious, timeless. Two and five Canadians will hear the words, you have cancer. That's why every step and dollar raised matters. On September 19th, join thousands in Toronto for the Princess Margaret Cancer Foundation walk. Challenge yourself, friends, and family to walk 21 kilometers in support of

life-saving research. Together, we can carry the fire and help create a world free from the fear of cancer. Register today at pmcf walk.ca.ca. Ready to take your investing knowledge to pro-level? This is Fidelity Connects, your daily edge in the markets. Get deep insights on real-time market topics that may impact your investment portfolio. Listen to Fidelity Connects on Spotify today and power your next move tomorrow.