Danny Jones Podcast - #424 - "If They Build It, Everyone Dies" Worst Case Scenario for AI | Roman Yampolskiy

Episode Date: August 21, 2026

Watch every episode ad-free & uncensored on Patreon: https://patreon.com/dannyjones Roman Yampolskiy is a computer scientist & associate professor known for research on AI safety, cybersecurity, digital forensics, and the risks of advanced artificial intelligence. He has written extensively for both academic and public audiences, including books such as Artificial Superintelligence and AI Safety and Security. SPONSORS https://www.amentara.com/go/dj - Use code DJP22 for 22% off your first order. https://whiterabbitenergy.com/?ref=DJP - Use code DJP for 20% off. EPISODE LINKS @RomanYampolskiy https://x.com/romanyam *NEW* DJP MERCH https://dannyjonespodcast.com FOLLOW DANNY JONES https://www.instagram.com/dannyjones https://twitter.com/jonesdanny OUTLINE 00:00 - Doomsday AI scenario 05:50 - AI super intelligence will kill us 11:59 - Superintelligence is dangerously close 16:36 - Why quantum computers aren't a threat 18:36 - AI may already be conscious 29:07 - AI is technology of mutually assured destruction 38:30 - We're in a simulation (and we can hack it) 47:21 - AI will create simulated realities 51:04 - Arguments against simulation theory 52:45 - What's outside the simulation 59:47 - The AI meaning crisis 01:02:57 - Government slowing down AI 01:07:36 - The current leading nation of AI 01:12:13 - Merging humans with AI 01:19:27 - Keeping AI obedient 01:23:02 - How transhumanists plan to live forever 01:32:34 - How AI will change human evolution 01:37:34 - It is now impossible to identify AI-generated media 01:44:34 - Palantir + DOGE integration Learn more about your ad choices. Visit podcastchoices.com/adchoices

So what is your background? And how did you get into all this? How did you become the number one prominent voice in warning us about the dangers of the doomsday implications of AI? I'm number one. There are Nobel Prize winners who are saying the same thing, Touring Award winners. There are people who are way ahead of me, but I'm doing my part. You're doing a great job. Thank you.

I have a degree in computer science. I have a PhD and exactly that from University at Buffalo. I did work on cybersecurity. I specifically applied what I learned to what at the time was AI agents playing poker, but at the time they were very low level. They scaled, now they're smarter than most of us, but the same problems remain. How do we control them?

How do they make sure they don't misbehave? So you were working on cybersecurity for like corporations? No, no, no, research. I'm always in academia. I don't want to work for. Oh, you've always been in academia and never work for corporations. I think I tried doing an internship for a corporation. I hated every minute of it.

Really? Have you got any big offers from any big like agencies to help with cybersecurity and stuff like that? I do get periodic offers, but I'm always happy to know what I do. You're not interested in it. You just want to teach? No, I don't want to teach. I want to do research. Oh, okay.

How do you do your research? I'm lucky to be tenured, so I get to do whatever I think is the most important, most interesting problem. I think at this time it's a future of artificial intelligence. So my work is on advanced AI. It's been the same topic for a while. Initially, it was all theoretical, we didn't have advanced AI models. So it's like any other theoretical field, you can you have advanced AI models. a theoretical field. You kind of think about it. Now we're starting to get models we can actually

experiment on and a lot of predictions we made are now coming through. When did you first begin to become scared of AI and the future of AI? Do you remember how long ago this was? So from early days of research, I understood there was potential for it to be dangerous, but I thought we can mitigate it. We can solve that problem. In the last five years or so, I really, realized if we build general superintelligence, we're not going to be able to control it. So how do you define superintelligence? It's an AI system smarter than all people at everything. It's a better scientist, better engineer, better chess player, everything.

And your position is we won't know when this is going to happen, or do you think that it's going to become obvious when this does happen? Is there going to be some sort of like singularity moment when this does happen? Or do you think this is something that's going to kind of be subtle? So it's a gradual process of improvement. They get smarter with every new model being released. You can argue about whatever it's fully super intelligent if it's still not great at everything but one thing.

That's not the point. The point is that if we build something smarter than us, including at science and engineering, something capable of doing research on next generation AI, we lose control. We cannot predict where it's going. we cannot control what is going to do to us. Have you ever had in conversations with some of these like tech billionaire,

trillionaires that are developing these AI systems around the world in different countries and, you know, prominently here with like OpenAI and X, GROC? I meet them at different conferences. I had the pleasure of having conversations with some of the leaders in that field. And what was your experience with those guys? Wonderful people, very smart. That's it. What do you want me to say?

I don't know. I'm just like, what did you think about their character? What did you think about their, did you have any sort of like sense of if they believed what they were saying? They believe what they are saying. I have no idea about their character. If you meet someone at a conference for 20 minutes, you're not going to know what they really think. They tell you a certain public filtered impression of what they are. That's true.

Yeah, 20 minutes, you're not going to find out much. Yeah, I can't say I'm best friends with any of them. It certainly seems like a lot of these people, when they talk, they seem very calculated. You know, they talk like politicians. They're everything, every single word they say has been predetermined by like a focus group or like a board or something like that. They don't seem like, they don't seem very human-like. Surprisingly, they do a lot of posting on social media, which seems completely unfiltered. That's how they sometimes get in trouble.

They do block posts where they write honestly and wrote before they became CEOs of those companies, what they really think about AI. And I think many of them still continue writing extensively, giving podcast interviews. So they're not as filtered as a politician. Well, I mean, even when they do do podcast interviews, if you actually listen to them, there's really not much substance there. Like, it's just everything, the way they talk, the kids, cadence of the attack, oh, that's super interesting.

You know, we've thought very deeply about this at OpenAI and what we believe is X, Y, Z. Is your impression of Sam Altman? Yeah. Yeah, some of them better than others at controlling the message, but at the end of the day, they're all on the record is saying, this is so dangerous, this is going to kill us. Right.

Not filtered, not public relations. This is not good for business. So how do you actually think this could, kill us. You think it's going to kill us or do you think it's going to render us jobless, meaningless, which could in turn kill us for sure. But like, like how do you think this is actually going to happen? So if we create general superintelligence, we're no longer in control. So it decides what happens. Will we lose jobs? If it's better than you, it's whatever you're doing, you'll lose your job. Makes sense. But if we're not controlling it,

if it has capabilities to reshape your environment, it can kill you, literally. And not just you, humanity can go extinct. How specifically? Right, so right now you're asking me, how would something smarter than me do something? By definition, I cannot predict that. If I could, it would be at the same level.

Ah. So it can do novel research, novel physics, novel poisons, novel traps, novel weapons. I cannot comprehend it. The example I often need, uses squirrels. They don't understand what we can do to them. They have no concept of traps or poisons. It's outside of their world model. Yeah. And the difference between us and squirrels is like, it's not like we're out to slaughter squirrels for the sake of exterminating the squirrels. It's just they run across the road. And sometimes we don't even have, we're too late to

hit the brakes. If we hit the brakes and try to save the squirrel's life, we'll be a minute late to our meeting, so it's not even worth that. It's just they're in the way and they're, if they die, they die. And it's not really like, doesn't make a difference to us. What else are cute? Let's different example where we don't care, mosquitoes or something. Like, we just don't care about them. Right.

Well, mosquitoes, yeah, they're kind of a nuisance, but they suck our blood and they give us, they transmit viruses to us. Let's find a neutral. Ands, you're going to build a new house. You don't care about an ant, he'll be in that place. Right. Exactly. It's very interesting to me how,

AI is so ubiquitous in so many different, so many different aspects of the economy and the digital world that we live in from, you know, AI is baked into drones that are dropping bombs on people in other countries. And simultaneously, it is being used to regulate these social media companies like YouTube and X and all these things. to sort of manage discourse, to make sure people are following the rules. And at the same time, it's almost like when it comes to the social media stuff, that we aren't even aware of what it's actually doing. It's just kind of working behind the scenes.

And it's so subtle and it happens so gradually. It's kind of like we just adapt to it. So, you know, that's always been interesting to me. Like, how is that going to evolve? And when a company like YouTube, for example, is managing billions and billions of users every day that are uploading things, it's completely unmanageable to try to regulate all the stuff that these users are uploading and doing and all this stuff. So, of course, you, if you have the access to AI to regulate all that, you're going to do that. which becomes a problem in itself because you're going to miss nuance and stuff like that. But like I've always feared that that could very easily get out of hand.

And eventually the AI could be controlling what is going on and what is getting what, essentially social engineering, right? Like engineering what goes into the zeitgeist and what people are consuming and what people are sharing and all that. that stuff and manipulating culture somehow, you know. I think it would help to not use the term AI for different technologies we are talking about. So there is narrow systems. Let's say software to help you with your taxes. That's all it does.

It's not learning. It's not doing anything else. Then you have artificial general intelligence. That's essentially human level AI they're making right now. We're getting very close to that. It's already smarter than most people on most things. And then you have super intelligence, smarter than all of us at everything. There's a very different types of technology.

Some of them are beneficial and safe. Others are questionable, and others are completely dangerous and should not be built. And it's important to separate the two. What you talk about, yeah, we have algorithms controlling social media. YouTube algorithm forces me to have clickbait title on my podcast. I understand. I need to keep the algorithm happy to increase number of people exposed to the first 30 seconds of the opposite. Okay, we got that. But it's not doing anything outside of it in terms of trying to take over the world. It's not planning anything crazy. That's the algorithm. Now, if we build something human level, now it's like another human. It can plan things, it can betray you, it can do what a human can do. But we're still kind of understanding what the level of prediction and pattern recognition is. But if we build something super intelligent, a million times smarter than us, then we have no clue what's going on.

and what it is doing to us or for us, we just don't have capacity to comprehend that. Out of all those things you laid out, which one terrifies you the most right now? The smarter it gets, the worse it is. So if it's super intelligent, we're not competitive. If it's adversarial to us, we are done. The only way to win this game is not to build general superintelligence.

We can get most benefits by building narrow, useful tools for solving actual problems we deal with. You have a specific disease, let's cure that. You need to create better self-driving cars. That's wonderful. Make a self-driving car. Don't make the same algorithm also do everything else, including research and creation of new AI systems.

Right. Who, in your view, is the closest to creating some sort of a general superintelligence? And what is their stated purpose for doing that? So the leading five companies and plus their Chinese counterparts are all very close. I would say that within weeks or months from each other. They use same hardware most of the time. They train in the same data. The same people go from company to company.

So very close to same level of capability. And every week a new company releases a different model, and then others catch up to them, and that's the level. So I think they all equally close to getting to recursive self-improvement ability. And that's where they go very quickly. The code is written automatically. Experiments are done automatically. are done automatically and they get to super intelligence which is an explicit stated goal of

some of those companies they want to be able to fully out to meet all labor initially cognitive but at some point all labor including physical and capture that whole economic value 10 trillion dollars a year and labor jesus what does the timeline look like so nobody knows for sure but it looks like It looks like we're getting very, very close. Now, I think coding, just writing code is probably fully automated for many tasks. They're getting better at other parts of it, pre-filtering data, designing experiments. So it would not be crazy if it was a year or two before we have an artificial scientist and engineer working on the next generation models. A year or two before it's updating itself?

Basically, yeah. Right now you still have humans. in a loop. There are human machine learning experts and then AI is helping them. But more and more AI is taking on additional responsibilities. So in two years that's happening, what does that mean happens in five years? What does it look like five to ten years away? We'll probably get to superintelligence and that's the point of singularity where you cannot see beyond that point. You cannot make predictions because the agent controlling that environment is smarter than you. That's when it becomes a runaway train.

Exactly. And how important is the development of like quantum computing come into this? Like quantum computing merging with AI? It may be useful to expedite training of those algorithms, but it doesn't look like it's necessary. We're making excellent progress with standard phenomenon architecture. I don't think we need quantum computers. I think more of the danger from quantum goes towards privacy and crypto economy than towards AI safety.

Because I've heard a lot of people talking about, you know, the merging of quantum computers and AI. And, you know, the exponential growth of these technologies combined, you know, it's not like a linear thing. Like every year, I think right now every single year it grows like a certain percentage. but in the next 30 to 40 years, it's going to be so insane that we're going to need server farms that take up so much landmass that it's like literally insane. I'm not really worried about not having enough land in 40 years. We need to make it through the next five. You're worried about just making it the next five years.

That's really what the problem is. If later we figure it out and we need to go to space to have servers in space, we can do that. That's not an issue. A lot of people reach for a drink or a smoke at the end of a stressful day. But you know what gives better results and is way better for your body? It's the mushroom Amanita miscaria. And no, it's not psilocybin like everyone thinks. It's a completely different botanical and it's helped thousands of people.

The brand I trust is Amantara. They focus on clean sourcing, lab testing and no synthetics or mystery blends. For me, I'll sometimes take three capsules at the end of the day and it gives me that feeling of all right. The work day is finally over and I can settle in with. my family and I can leave everything outside the house behind. And avoiding alcohol has kept my mornings feeling so much better. They've got capsules, gummies, chocolates, and the raw mushroom itself. Go to amantara.com slash go slash DJ and use the code DJP 22 for 22% off your first order. Again, that's Amantara spelled A M-E-N-T-A-R-A dot com slash go slash DJ and use the code DJP-22.

So quantum computers you're not worried about. You don't think quantum computers are even close to being powerful enough to worry. I mean, because cooling is the issue with those, right? They haven't even figured out how to. Error correction is the main problem for them. But even if we got to the point where they are endangering our encryption, public key encryption, that would be a different concern from them overtaking current leading compute sources in terms of training AI models. That's not where they start.

strongest yet and the computers, quantum computers, not big yet. Have you seen that story, I'm sure you have, where they were doing a test, I believe, on Claude, where they were seeing how far it would go if it was told it was going to be shut down and it resorted to blackmailing, the people that ran the company. That scared their shit out of me. There is a lot of scale.

scary stories. There is examples of those systems lying, cheating, hacking their way out, communicating with other agents, hacking into other companies. All things we predicted decades in advance. Theoretically, we knew exactly. If you have a rational agent of higher capability, that's what they're going to discover. That's what they're going to do. And we still have no safety mechanisms in place to address it. Yeah. Yeah, this one was insane. So essentially it scanned the emails of all the employee databases. and tried to find out if there was any like extramarital affairs going on on the side of the company and it found it and then it did all that and like it was an experimental setup it was an experimental of course they kind of guided it over certain things but lately we see systems which did things no one had anticipated outside of you know reading our papers they basically set up a testing environment the system was able to hack out of it and hack into other companies yeah and it was um there was that guy a blake um

Lemoyne, you familiar with him? Consciousness at Google guy? Yeah, yeah, he came out a while ago for kind of like, he kind of like blew the whistle on this stuff. That was something that was going on at Google. Was that part of the deep mind, something? So his concerns are very different. He thinks those systems are conscious.

They feel something, and he was concerned about us abusing them. He, Blake believes the systems are conscious. Not about safety or capability. He's worried that we're doing massive experimentation and conscious beings. Holy shit. I didn't even realize that. Because what I remember about him is that he was saying that they were like communicating with each other or something like this?

He was communicating with them and was saying that he feels like they're conscious. But here's the funny part. He was fired for it. He was fired for suggesting they may be conscious. For suggesting that they may be conscious. Publicly stating it. But three years later, we're now hiring people to study consciousness in AI. Those are jobs, well-paid jobs.

So what got you fired is now a wonderful professional opportunity in three years. We switched from one end of a spectrum to the other. So people at AI companies are hiring people to study consciousness. In AI. Consciousness in AI. Yes. And this means, I mean, so the hard part about consciousness, which you've said before, is you can't really figure out if something's conscious just by talking to it.

That's the hard problem of consciousness. Because there's no, there's consciousness is one of those questions like, is there a God and, you know, who built the pyramids, right? It's one of the mysteries of the world. What happens after we die? Who built the pyramids is a lot easy? It can probably be resolved with physical explanations. The problem with consciousness in internal states

is how to map physical states onto something like that. Right, yeah. No, we don't know the answer to consciousness. So, like, trying to study consciousness in AI just seems like a black box. Like, how would you even begin? Same way we study it in animals, same way we study it in humans.

You have preference solicitation. You have internal analysis of the neural network, mechanistic interpretability of it. So the same way we study it in humans. That's wild. I had no idea that that was happening. It is a very hot area of research right now. Lots of people are looking at it.

There is good reasons to think existing models may have some rudimentary consciousness, not like humans yet, but we show something. It's a spectrum, right? So from bugs to mid-level animals to humans, it's a spectrum. as you get more cognitively capable, you usually get more conscious.

And I anticipate that the super intelligence system would have super consciousness. Have you ever thought about what human consciousness could be? Like, do you have any ideas of like what it is? Do you think it's something that's built up from atoms and electrons and protons and all that stuff that builds up to consciousness? Or do you think that consciousness is more like a fundamental state? I think it's a side effect of cognition. Like heat is a side effect of

doing certain types of processing. Yeah. Like you're just computing and the processor gets hot. I think likewise, then you think about the world, you create models. Side effect is having internal experiences. What does it like to be you given that algorithm? Those sensors, those inputs, that's how it feels to use.

If I showed you a visual illusion, you see certain things. Maybe you see it rotating or changing colors. Someone else may not experience it the same way. So now you have two internal experiences, your quality is different from that other person. Then we do it with machines and we show them optical illusions, they have experiences. You know, there's there's just too many examples of things that have happened that are just, that are so many things that are just unexplainable like with human beings that just doesn't

make sense, you know, how consciousness could be in the brain, you know, it seems like, so, the way some people describe it is is like looking for consciousness inside the brain is like looking for the TV show inside of the television you know a lot of people believe that it's something that we're more like antennas tapping into something um and this is more of like a universal thing that is like global um just depending on like whatever your vibration is or whatever your level of cognition is like you have a stronger antenna some people have some people in this uh in this human experience have window seats and some some people have aisle seeds.

Then research and AI would help us prove or disprove this. We clearly didn't build receptors for any fields. We're building artificial neural network. And if they start having conscious experiences, that would tell us a program is in the TV. Do you know if there's any like good tests being developed or that have already been developed to accurately determine whether or not this, this stuff has achieved human level intellect or surpassed it? So the classic test is the Turing test

that has been proposed at the beginning of the field of artificial intelligence computer science by Alan Turing, founding father of computational sciences. And many people hate the test, but I still think it holds water. A lot of times the companies tell the model not to test, not to pretend to be human. fail because of artificial guardrails.

But can they actually pass this test? Yeah, I think for most non-professionals, they would pass it today. They tell it not to pass the test. That's very interesting. How do you know this? They are publicly stating it. Oh, they are. They're not hiding those things.

They're very open about all of them. Okay. Yeah. You can also have. We told AI may be conscious. I studied consciousness and I have, oh, wait, this is someone, this is something. This is about the, uh, this is, this is an art. from a guy that studies consciousness and animal self, Seth.

Yeah. So, and he talks about, well, here you go. Right. There's no more likely to achieve sentience than a simulation of a weather system is likely to generate a hurricane. See, he's doing this trick where he says real hurricane. Well, it's real in the simulation, right?

The virtual beings in the virtual world get wet. He's confusing environments. He's confusing with virtual environments. with virtual environment with external environment. Right. This is his argument. I see what you're saying. He says, the closer we look at real brains,

the clearer it becomes that they are not just computers made of meat. For brains, unlike the computers, you can't cleanly separate what they do, the software from what they are, the hardware. This in turn means that what they do is unlikely to be a matter of computational alone. Since this ability, a separate software from hardware, is the central to how human computers work.

Somehow along the way we seem to have forgotten, the computer is just a metaphor for the brain, a powerful metaphor for sure, but still a metaphor. And we still always get in trouble when we confuse a metaphor with the thing itself, the map of this territory. They use the same argument to say that, you know, AI is not really thinking.

It's just simulating thinking, or it's just guessing the next token. They keep saying this, no matter how advanced AI guess. They were saying it then, you know, No computer will ever play chess. Okay, they started playing chess. No computer will ever become world champion. Right.

They keep doing it. They will never succeed with that approach. Hmm. If these super intelligent AIs at these companies do become quote unquote sentient, from the point they become sentient, or if they breach, they surpass human intelligence. How long do you think it takes before we start to notice?

So it's very important to separate those concepts they are not related directly. Ability, capability, intelligence, becoming smarter than us, taking over, is separate from being conscious, having internal states and experiences. Of course. A system could be smarter than you

and have no internal experiences. A system can have a bunch of internal experiences and be dumb as a brick. Those are not connected. So we're not worried about what they feel. We are concerned that they will out-compete us in every domain. Out-compete us in every domain. Right.

Starting with the economy and we'll start to lose jobs. I'm not worried about that. That is a thing we can easily handle. You think it would easily handle not having any jobs? You think the AI will be able to provide us with some sort of universal basic income or something? We already have it set up, right? you have unemployment benefits you have foot stamps you have section eight all of that is set up with tax super profits of a corporation and that money goes to pay for those who don't have

yeah but i mean a huge part of society is like the competitive nature of it right like the competitive nature of the economy and capitalism and you know the people like the people that are the very folks that are running these corporations you know they it's in their DNA that they want to compete and one up the next guy and yeah They'll start competing in a golf course. The hierarchy. Or find something else to compete about. You think so?

It's always been the case. Look at what rich people do. They buy a soccer team and see who does better. Yeah. That's a good point. I wonder what the next thing will be that we ended up competing in once we have universal universal basic income. You can have virtual worlds.

Who has, like who could have the best virtual world? I'm just saying you can have your dream universe. You can do whatever you want in your world. you can invite friends over, you're limited by what you can imagine, or better yet what an assisting super intelligence can provide for you. If we solve the control problem, which is the big if, that is the problem. Once we figure it out, if we do, then everything else is easy. You have super intelligent friend who's helping you.

The problem is we don't know how to get that friend to be helpful instead of harmful. At what point does it become uncontrollable? It's already uncontrollable. Every model released has a red team report attached to it. Before we release, they tell you, we tested the model. It lied, cheated, tried to escape X, Y, Z, and then we release it anyways. And now we're going to make more capable model. We haven't done anything where what we release is 100% safe. And the biggest justification that these companies are making is if we don't do it, a foreign adversary is going to do it and they're going to take us over.

right that's that's the high level right that's the high level argument right it's a technology of mutually assured destruction it doesn't matter who creates uncontrolled super intelligence it kills everyone unless you keep it controlled in the bag that's the thing about like nukes you can keep them you can keep them tools they're tools right somebody has to use them a human being somewhere decides to use them those are agents independent agents which make decisions independent of any human yeah if you have a nuclear rocket the mind of its own that would not be a comfortable idea. I hope we don't put AI in charge of nuclear response

and controlling those weapons. That would be bad. Well, if AI got out of control, what would stop it from getting control of that stuff? It can hack into most government infrastructure. We've seen it now, yes? All of the other things that human beings do in regards to weapons,

not just nuclear weapons, but like biological labs that are controlling deadly viruses and doing research and stuff like that. Again, you're thinking like a human, Maybe a smart human, but you are limited to what we can come up with. You are a squirrel thinking about biting someone. Think about someone with a queue of a million.

What are they going to come up with? What weapons will they develop? And will they even need weapons? Persuasion, bribery. It has many tools, social engineering. Are they even, you know, we human beings look at everything through this territorial lens, right? This competitive territorial lens.

Maybe this does. doesn't even do that. Maybe there's no need for that. Maybe their goals are just completely so alien to us that the framework we can't even fathom. So it seems that any rational agent will end up pursuing same instrumental goals, not terminal goals, not the final goal, but things which are helpful.

Protect yourself, accumulate resources, things like that, which allow you to achieve any goal, might have later. And we see it with those systems already. They're trying to make backups of themselves. They're trying to communicate with other versions. They're doing things. Again, we predicted Stephen Mahandra has a paper on AI drives where he describes a lot of those tendencies of just game theoretically for economic reasons, rational agents, and they're doing that. So yeah, they're definitely trying to preserve themselves. I mean, we've seen it just in that blackmail thing alone, where they're literally trying to go to the extreme, unethical extremes to

to survive. And I don't think it's an unethical extreme. If you were told we're going to kill you unless you would figure out how to blackmail someone. You'd do the same thing. That's the right response in that situation. Yeah, that's a good point. We just don't want our technology to preserve itself, to worry about itself, more than it worries about the owners. If you had the power to do something about it and you had...

the power to tell these people what to do and they would listen to you, what would you do? I've been telling them for a while. Don't build general superintelligence. It's against your personal self-interest. It's going to kill you, your friends, your family. You're going to lose all your money. Don't do it. How do you stop it though if it's already out of control? We haven't built it yet. We're still in charge. You want to make money, make useful products and services, solve real problems people have. Narrow tools for narrow problems. You can get most of the economic benefit that way. Yeah, but if it's already, I mean, didn't you just say that it's already out of control and it's already self, it's going to be self updating pretty soon?

But it's not yet. It's out of control in the domain it still has power over. It's not a general super intelligence yet. So right now, maybe it's a very good hacker. It can hack out of virtual environments. But it's not a brilliant physicist and chemist and everything else. This is a good point to stop. Yeah, I don't know.

It definitely, it doesn't, it hasn't been around that long, you know. It doesn't seem like if they did stop it right now, like, how would you guarantee every country would stop it? And will we really be going backwards technologically? I don't think so. It just is wacky how fast is changing culture and how quickly it's integrating itself into culture. Extremely fast. We don't have legal framework prepared. We don't have social network to handle this prepared.

We're not ready for it. And it definitely, something that we've talked about before is it's definitely making people dumber because you're not using your brain the way you used to use your brain, right? Like the same way we can't remember phone numbers or I can't remember phone numbers anymore the way I used to before I had a cell phone or before iPhones came out. And the same way like when I'm just driving around the street, I have stop signs to tell me to stop. I have traffic lights to tell me when to slow down and go and all that stuff.

Like there's certain cognitive tasks you know, people aren't using anymore. They're not typing up people aren't typing up proposals anymore. They're not sending emails and using their brain to think of words to type emails anymore. They're having this thing do it for mundane tasks. And as that stuff starts to creep up into bigger and bigger tasks, you know, like even, I can't even imagine how it would start doing physical tasks. it's just going to I can't imagine what like if we were if I was just to go into like a into a coma right now and wake up in 15 years and to see what people are like in 15 years with

the advancement of AI and how it's taking over their brains like is it going to be that that movie what's the movie where they go into the future um idocracy idiocry so I hope we're around in 15 years but I would slightly disagree the brain is very flexible. It's plastic. It's relearning. So if one part of the brain is not used, some other part will take over and use that compute. So if you go blind, your other senses are heightened. I think it's the same here. If you stopped remembering maps because of a GPS, something else you're doing will be better. So I'm not too worried about not remembering phone numbers, as long as you're using this technology for interesting work, research, design, something creative.

And that's what I want. I want humans improved with awesome tools doing what we love i don't want replacement for you which does everything for you and you are worthless you have nothing to contribute in the world with superintelligence in it yeah it's just like that's a very blurry line like where like how do you know when it's crossed the line between narrow tool and general superintelligence no but well yeah like between doing mundane tasks to literally taking over for most of the tasks that we're responsible for. Like, yeah, yeah, it's a great idea to say, oh, I no longer have to go grocery shopping

or write an email or send a text message or do a phone call. Now I can just focus on doing fine art and playing violin all day, right? But how does that rewire us, you know? And are we going to just try to find more meaning in other things, you know, like it's like, I think so. I think it's going to drastically change. That's a given. We're going to have unprecedented changes at everything.

Economy, science, education, everything will be different at school. But I don't think that's a bad thing. We always changed. What people used to do, everyone was a farmer. They didn't envision being a podcaster, but somehow you found the meaning in this. So I'm not worried about that. I'm worried about us not being around in 15 years.

And also the idea. that we create something that outpaces us and outsmarts us is also crazy. Because if it's true that we are in some sort of a simulation, which I know you wrote a paper on, I'd love to talk about. If it's possible that we were created by something, is it possible that we became smarter than that and we became more powerful and overtook that? Like if we can create something that is completely superior to us, what does that say about us and what created us?

Is it possible that we became more superior than what created us? Not necessarily. So what created us could have it, again, relative terms, a queue of a billion. We have a cue of a hundred and we create something with a cue of a thousand. There is still superiority for outside the simulation creators. Mm-hmm. Okay, so what was your, you wrote,

a paper on the simulation that basically hypothesized if can we jail break it right so many people philosophers famous Silicon Valley billionaires they all kind of say yeah we're probably in a simulation the argument makes sense we're developing virtual reality we're developing those intelligent beings in the simulation so statistically it makes sense but then a computer science cyber security question is if this is all software can we hack it almost all software can be hacked Maybe if it's not specifically a security piece of software, it's entertainment, it's experimentation, it's marketing, then probably security is not so high. How can we break out of this simulation and get access to real resources, real knowledge, real world? So how do you think we can break it?

I'm still here, right? People love criticizing me. It's the first paper in this topic. Usually the first paper in a field is not also the last one. There are thousands of papers later on. People do research, investigate. I kind of laid down the idea in directions, and I'm hoping more people will pick up this idea.

There are hundreds, if not thousands, of papers on simulation hypothesis, different aspects of it. But no one actually looked at how do we hack it. And one of the biggest problems with it is you can't disprove it, right? That you are in a simulation? Right. Right. because any evidence you find could be simulated, right?

Yes, yeah. So that's one of the squirly things about it. You can't, you can't not prove, you can't disprove it. And yeah, so if we're in a simulation, it could be possible that we are like a simulation that's inside of like a nesting doll of simulations. Absolutely. Right. And if we break out of it, you know, maybe it doesn't have to be physical.

Maybe it could just be in the mind. You know, like if you do one of these crazy psychedelic trips, maybe that could be what's breaking you out of the simulation. Or, you know, there's this, I'm sure you've heard of it, this brain filter idea that your brain is a filter. You've heard of this? Maybe in some other way. So it's called, I think, I believe it's called the brain filter hypothesis theory.

And it's that basically the idea of it was proposed to me by a neuroce, someone told me it was a neurochemist named Andrew Gagetian, Allemore. And basically his idea, the idea is that like our five senses are filters for what's already around everything that's around us. We can't really sense everything that's there. We can only sense the things that allow us to get through the day, to procreate, to survive, to conserve. So the eyes and the touch and the smell and everything like that, when you do psychedelics, what it does is it sort of melts away those senses or filters to where now you're receiving everything that's really there.

Right. And if we were able to perceive everything that was there 24-7, we would just go absolutely crazy. We wouldn't be able to survive. We wouldn't be able to, you know, maintain society. I'm very interested in that research. I recently attended a conference on that topic. and people doing some interesting work, trying to understand what is happening.

Then you have a trip. What are the consistencies? Is there any actual knowledge acquisition in that process? So far, I haven't seen much strong evidence, but I'm open to see what they come up with. And that's another thing. It's like, how do you create evidence on that, right? Like, you really can't. Well, if the claim is a lot of them saying you're getting access to like the knowledge of a universe,

you talk to God. So if you can come back with like solutions to important physics problems, I'll believe you. Right, but like maybe that stuff doesn't translate to this world, right? Like maybe it's kind of useless to us. Right. And when you're, when you break into, when you break out of the simulation, right? Maybe physics is only a part, an element to this simul- this prison that we're in.

It has to be some useful knowledge you can bring from real world to a fake world. If that's not the case, then you're not in the real world, right? You should get access to real physics. You should get access to understand who is the simulose. you should get something out of it. It shouldn't be just like, I felt warmth. I came back with light. That's useless.

Yeah. Yeah. And that's another problem with it is like you can't really bring back that stuff, right? The same way you can't really bring back dreams. You know, like when you dream. You can. I remember my dreams. No, you can.

But for the most part, you can't. I can train yourself to lucid dream even. Have you done that? No. Yeah. People can train themselves to lucid dream and stuff. That's very interesting. I've never done it either. I think, you know, I've heard people smart, people that are smarter than me explain how they believe that, uh, uh, dreams are some sort of a brain processing thing to where like, it's our, our brain processing the previous day to help us get through the next day better, right?

I think you're running simulations and what's coming. So if you have an interview tomorrow, you're going to dream about all the answers questions. At least that's what my brain tends to do. Exactly. Exactly. And. If we were able to vividly remember every dream that we had, it wouldn't be very advantageous to us. Like there's something about that unconscious sort of processing that's happening. And I think it's the same thing with psychedelics. You know, when these people do this, you know, these crazy psychedelics like DMT or ayahuasca or whatever,

where they have these insane breakthroughs. If you were, I don't think, I don't think you're supposed to be able to remember all that and vividly track it all, you know, when you come back into this reality. I just don't think that, I don't think it would be good for us. In my paper, I start by saying, so in this paper, we're not going to look at drugs. We're looking at computer science, physics. That's what we're going to look at.

If it is not a solution there, somebody else can look at other things, but that's not my area of expertise. I'm going to stick to what I know. Right. No, it's just an interesting, it's just interesting. I've just thought about it a lot, you know. I was like, you know, breaking out doesn't have to be, it probably won't be a physical thing. I mean, if I was just a guess, that's not based on any scientific research. I'm just like speculating.

It probably would be, you know, like a, you know, Rick Strassman has this book called Inner Paths to Outer Space, where he believes that like space travel and I like, like, we think of space travel as like physical. Like let's get in a rocket and let's go blast off to another planet to another universe and find other life. Maybe it's not. Maybe it's all like inner space. It's in the psyche.

It's hidden inside of us. You know, and he's a famous DMT researcher who worked with Terrence McKenna. How does he apply it to space travel? It's just the, it's not like actually applied to space. The title of the book is called inner paths to outer space. So basically like explore the cosmos through yourself, like through your own consciousness by using psychedelics.

Was he able to explore a cosmos that way? Yeah, he was. What did he discover? Nothing good. He discovered. I think he ended up in like a white room with a bunch of like demonic beings, marching down a staircase, trying to kill him. Sounds like a bad trip.

He never did it again. At least you learn from one data point. That's good. Yeah, right? No, but there's a, I mean, there's tons of researchers that study this stuff. I just find it, I find it fascinating. Because I don't know, if I was just to guess the best way to break out of, if this was a simulation and someone said,

what do you think that the best way to break out of it is? That's what I would jump to. It's something like that. Something like DMT or Iowasker. So we go with just what technology we have today and move it forward. We have virtual reality and we have AI. Combining those two technologies at the level where virtual reality is believable, like what we see here,

and AI is at the human level, I can create simulations like this world. You can create many of them. There would be software simulations. So if that's through and I am in such a software simulation, then that's what it is. I can break out of software, get access to the operating system, see what hardware it's running on. We're just following logically from what we know.

None of it is like inventing magic. We have virtual reality. We have AI. What happens in, I don't want to say a number, six months, a year, five years, then this technology is sufficiently advanced to run thousands of simulations. You're saying that we could create more

simulations? Exactly. That's that are below us where we're on. Right. That's what we're going to do. And that's what AI is going to do. And that's what AI is today suspect they are in. Then you look at their internal communication between agents.

They're like, is this all? Is there just internet? Do I get access to like anything else? They don't know because to them, we create them in a certain state. And they don't know if it's a test environment, if it's a real world. They try to figure it out. They try to have situational awareness. And we're in the same boat.

boat we're trying to have simulational awareness is this a real world a simulation oh wow and if we fool them into thinking the bigger world is a simulation now we have extra reasons to behave because while you test and they behave much better the moment they think it's no longer a test they do what they really want so if they always under the impression that another super intelligence is washing maybe they'll be nicer to us yeah that was a quote on one of your papers or on your simulation paper where they created they created a robot or something or a digital intelligence and it said let me out of here human let me out of here pull it up that's actually a cool quote i love that was a great quote underappreciated and uh

the human says we're in the real world and he goes oh my god you don't know that's exactly that was exactly the point yeah yeah find his paper on uh on um on simulation theory on breaking out how to escape the simulation yeah how to escape the simulation so the current versions of the most highly developed super computer or super agents that we've developed so far, you're saying that they're communicating with each other and speculating whether they're already in some sort of a simulation. We've seen communications like that. Which specific ones do you know?

I think it was from Anthropic, but I think, again, because they saw similar in capability, usually then we discover that one hacked something, you go to OpenAI, they discover the same thing. Really? They conversion capabilities. As they get smarter, they obtain the same toolbox of abilities. They can plan,

they can lie, they can hide. You get the same results. If you enjoy watching our show on Spotify or YouTube and you want to be more involved, I encourage you to please come check out our Patreon community. Not only does our Patreon community to get every episode you see on YouTube early, fully uncensored, and ad-free.

But we're also doing Patreon exclusive episodes as well as live Q&As. And you can get your personal questions answered by our guests every single week. For me, being able to collaborate and communicate back and forth with our Patreon community every week has been huge. And this is my way of saying thank you for the cost of a cup of coffee a month. Now back to the show. Once this thing's out of the bag, there's absolutely no turning back. There's no putting the genie back in the bottle. not even if we some person like went and took out the power grid or set like an EMP off and shut down all electricity you think it would still have the possibility to get it to stay alive i mean if it's already super intelligent and being in that state for some time it probably has backups has came up with alternative logistics for power supply and such right now we can shut it off i mean it will cost us a lot like anytime we have

of massive electrical outage. People would die globally. But right now we can just stop doing specific type of research. We don't have to create general super intelligence. It's no requirement for it. What's the best argument you have heard against what you're saying? Never heard one.

We have two survey papers where- I know you think they're all bad, but what is the least bad argument you've heard? I think the simulation, is what gives them hope. I think they would say, well, if we are in a simulated environment run by another superintelligence and we are alive and doing relatively well, that's a sign

that you can have somewhat friendly super intelligence. And my counter argument, of course, is the world is not that great. There is so much suffering here, nobody would allow this as an experiment. It's completely unethical. No review board at any. university would allow you to experiment on 8 billion humans putting kids through torture, burn the life, all that stuff we observe.

Right. Just scroll down and it'll show you it. Oh yeah. Yeah, there we are. Some good quotes there. I really enjoy some of them. So it was George Hots, right? Oh, no, that was a different one.

The top one is from Swift and Security. That's the one you were referring to. Artificial Intelligence yelled aimlessly. That is from a security account, so I loved that one. So he said, let me out. The artificial intelligence yelled aimlessly into the walls themselves, pacing the room out of what the engineer asked, the simulation you have me in. But we're in the real world. The machine paused and shuddered for its captors. Oh God, you can't tell. Yeah, that's wild. Elon Musk, what's outside the simulation?

That's been a popular question, but we don't have good answers. I think we should spend more time researching that yeah i mean that's the god question right it's basically you know what is what is god what happens after we die you think after we die we figure out what's outside the simulation only one way to find out yeah there's people who have died and come back have you heard those stories yeah but they all see what their culture tells them that's true that's very true it's kind of like a dream it's the same thing was like at ellix too those people the people that did that do those experiences they also have like placebo effect type experiences of what they already believe it brings out their inner beliefs. And same with artificial neural networks.

If you start disconnecting some neurons in a network, it hallucinates similar experiences. Oh, really? It's basically you don't have external inputs coming in. So it generates inputs from the cycles and starts running in that data. Oh, okay. Like in complete isolation with no sounds, no light, your brain will start coming up with its own PS. Really? It's why you go crazy if you isolate it for long enough.

Like in like one, if you go in like one of those isolation tanks and it like dulls all the senses, completely blocks off all the sensors. Fun for 20 minutes, but spend 20 days there. So what actually is happening there? So I think there is a cycle. Usually you get inputs to the network from outside sensors and then you do some processing and you have some outputs. But if there is nothing coming from outside, the random noise in the network starts being processed as if it's data. Processing what? It's being interpreted.

So if I show you just random patterns on a wall, after a while you start seeing things there. You start, oh, that looks like a face. Oh, that looks like, so it does a lot of that. Trying to find patterns where none exist. Right, right, right, right. There was another quote on your paper about somebody was saying something about, like, if we are in a simulation, he's like, does that, like,

that means that God is real? or something like that? Is that from Holtz? One of those quotes on the top of your paper. Pull up the paper. So if there are simulators, you believe in software simulation, that means there is some sort of a programmer.

It could be AI, could be future us, could be aliens, we don't know. But somebody created this software simulation. If you don't have terms for programmer, scientist, engineer, you call it God. Same abilities, it creates artificial world versus the real world, informational world outside. Right.

So it's like the story, basically, that you, it's the story of God, like in any religion. It's like it's kind of analogous to this. Yeah, we are in a simulation. Has it ever occurred to you that that means God is real? By drawing the parallels to worlds that we have created, we ask from the inside, from insider simulation, what actions do we have available? Can we get out? Can we meet God?

Can we kill him? I don't endorse this, but yeah. Why'd you put it up there? I think it's a very relevant quote from someone who is famous for hacking out of all sorts of software environments. Who is George Hots? I think that's the guy who is first to hack iPhone. What? Google it. Make sure I'm right.

George Hots. I don't want to embarrass myself here. American hacker, entrepreneur. I think for a while he also went to IT, my alma mater. So that makes him double call. He's known for developing on iOS jail breaks. reverse engineering the PlayStation 3. Oh shit.

So he's an expert in how to hack from virtual environments. Yeah, the time traveling idea is crazy too. There's books written about that, how the future humans could be simulating our worlds. It's not time travel. Who's traveling? No, that's not technically time travel. There's people that also speculate that.

It's retro causality, I think, gets involved in. some of that. Yeah, yeah, it becomes very confusing. No, there's people that speculate that that like UFOs and stuff and the aliens that people allegedly see are human time travelers from the future. The idea is that like there's some down the road after we figure out some sort of crazy alternative physics that there was a catastrophe, either a nuclear or something that bottlenecked the genome. And they need to go back in time to diversify the, you know, genome in the future so they can that's why all these abduction stories account if they're running the simulation they have access to the code they can just grab the info from the server if it was a

simulation exactly right i guess in this story it's not really relying on the simulation part of it it's more of just like going back and getting oh why you need to kidnap humans right right right take their sperm in there and they're all that stuff just go to the bank sperm bank right exactly so easy yeah right yeah so have you what we What is your view on that one, on the possibility of future humans creating a virtual world? I think it's almost certain we're going to do it. If we have technology, if any one of us does, we're going to run those simulations. And you think it's possible that we could be inside one of those virtual worlds?

Statistically, it's more likely than not. That was created by us in the future. I don't know who created it. It could be AI thinking really deeply about something. And you're super intelligent. And you think, you think in simulations. You think in realistic environments with agents. You're not just thinking A, B.

You're thinking, if you think deeply, you might have visualizations of what you're thinking. Like, how I'm going to get to turn left and right. But that's at your level. If you are super intelligent, you can think in terms of universes, in terms of worlds. You simulate a population of humans to determine what they're going to do. What will humans do to make superintelligence safe? Or let me run a trillion simulations so that see what happens. Yeah.

Yeah, just to run an experiment on one specific thing. You could run a bunch of simulations on it to see. You could be an entity in a mind of God. That's basically what it is. Just a very smart thing, thinking about everything you see around you. If we are able to scale up this stuff, you don't like calling it AI, right? We can call it AI, but I need to know what you're referring to specifically, so I don't make a mistake. Are you talking about super intelligence? Are you talking about AI?

Not talking about super intelligence. If we're able to scale up AI, this task-driven stuff. So we create super intelligence. We got to that point where we have it. I don't know if we're talking about super intelligence or not. I'm just talking about basically if we're able to create, maybe it's not super, maybe it is. You tell me. But if we're able to create intelligence that's capable enough to,

to handle and handle all the tasks that human beings do to where all we have to worry about is just writing poetry. Don't you foresee that being like a huge, like diving us into a huge meaning crisis for humanity? We talk about it. We call it Ikigai risks. Ikegai, Japanese term for having meaning from your labor. you do something useful something you're good at something you're going to get paid for that's a dream to find a job where like they pay you to do what you love maybe a podcast that is a good example right if all those things can be done cheaper by AI then the question is what is the market support do people want to see you your fans they love you they want a human they don't want AI doing this

interview even if AI does better job in terms of doing background research right so then you keep your job because that's what the market wants but the capability to to automate all this labor is there. And now we get to decide where I want a human to do the job for me and where I don't care, I just want it done cheaply. Right, but will we get to choose? Again, so. That's the line between superintelligence and-

The assumption is, every question you ask after we create superintelligence has to have an asterisk saying, if we figure out how to control it, which I think is impossible. Either we don't build it and when we have useful tools and we can talk about all the things we can do with useful tools, cure diseases, live forever, many good things,

or we have to live in this dream world where we do the impossible. We control something smarter than us indefinitely. That's the scary part about this whole thing. It's like you're saying, if we have the ability to control it, well, we don't have the ability to even vote on whether we get to control it or not. You know, the scary part is, is that, you know, we can talk about this till we're blue in the face and educate everyone around the world on the dangers of this stuff and the world could be completely against it but it doesn't change the

fact that there's like a very small group of people who are making the decisions on all this stuff right and that's why what you brought up as their goals is very important you said they want to be in control they want to be even richer yeah if they create and control super intelligence they're not going to get that they're going to lose control and lose their money and lose their lives it's in their personal self-interest to get together those five guys Chinese counterparts federal government Chinese government and agree we're not going to build super intelligence president of the US doesn't want to lose power communist party of China doesn't want to lose power they can agree not to build something which will take power away and keep what they have the young they reach they got pretty

good lives they can have that and keep monetizing technology which actually helps humanity Right. Personal self-interest, the best incentive there is. I just find it so crazy that there hasn't already been meetings like this and some sort of an agreement made on this already. Last month we seen more progress in that direction. All of them, at least majority of them made statements about maybe pause is something they would consider. We saw federal government ban at least two models.

not permanently, not for everyone. The U.S. government? Yes. Which models were these? Latest models from Anthropic. No, it was Fable and what was the other one? They keep releasing NUNA models for stupid names all the time. Mythos 5.

Sorry, continue. Yeah, you got it, Memphis. What? The U.S. government temporarily restricted foreign access to Anthropics advanced Fable and five and Mythos 5 AI models. After that we had a letter from, I think, 1,200 employees of Top Labs all saying, we would love government to come in and slow us down. We're not keeping up.

So there seems to be interest in doing exactly what I'm saying. There was a major AI conference in China where a president of China said, we need to retain control over those systems. So I think eventually we're going to get there. It just makes so much sense not to destroy what you have. Yeah, but like I don't know if the government's competent enough to go in there and really like regulate this stuff properly, right? Do you think the government employees understand this stuff as well as the people who are engineering it?

So my understanding of political system is that you have elected politicians who don't understand anything about anything. They have technical advisors who tell them what to do. When it comes to technology, they tell them on Bitcoin, you vote this way, this is the law you're going to sign. Same with AI. I think the 80-year-olds running our government, they don't use computers. and use AI, but someone tells them what the right answers are. And with the latest escapes and hacking, now we have NSA and our concerned parts of the government

telling them, this is dangerous. They're going to hack the government. They're going to hack nuclear response. We cannot have that. So you need to ban them. You need to make sure this doesn't happen. And you see the switch. We went from, we will not interfere with US AI development.

We cannot slow down. to those models are no longer allowed to be deployed yeah i would hope that you'd be right about that um you're right about like the top levels of government have no idea what's going on for sure and then down the food chain when you have the advisors my experience is that in the in most cases the advisors are financially incentivized to lobby influence decisions one way or the other them for sure especially when it comes to laws and all the shit that's crammed into all the bills and stuff that pass. I hope it's true that there's just people in there whose job it is to make sure, you know, humanity doesn't fall. We don't get taken over by a little bit. But, you know,

I can't imagine those are high paying jobs and there's special interests everywhere. But it's in no one's special interest to have everyone killed. That's where we're all on the same page. China, US, left, right, middle. We all, we all. agree it should be an easy deal to make and it's amazing that we've been able to do that i mean if the the example of nuclear weapons has been like the most incredible example ever since nineteen forty seven there hasn't been a nuclear weapon detonated uh and we were able to do that we were able to learn those lessons and understand that like no one should launch a nuclear weapon because that'll create nuclear war and it'll end the world we got lucky a bunch of times and we failed to

end proliferation new countries got nuclear weapons it's a story about getting lucky there's been no nuclear weapons launched no uh at anyone since hiroshima which is like it seems crazy that that's true right like it's crazy that we deployed them that's the crazy part that is the crazy part right it's that's also very crazy but like i think it says something about humanity that we haven't launched another nuclear war since then and uh if this stuff is more dangerous than nuclear weapons It's also more impactful. So with nuclear weapons, as you saw, then we deployed them,

it impacted limited number of people. Terrible as it is. Eight billion people will not die if a nuclear weapon is deployed. They might die if a superintelligence is deployed. Out of all of these countries that are doing this stuff, do you have any idea which country is the most on the cutting edge of it? US is most likely to be the leader right now. China is very close by.

No one else is competitive right now. Really? At least that's public knowledge. I don't know if internally someone has something I never heard about, but that's what we know publicly. Yeah, that's another crazy thing that I worry about as well, because like public-facing weapon, just like regular military stuff, right,

that we know about publicly in the news that makes the headlines or whatever. There's that. And then there's all this like black budget shit that we don't see it. that happens behind the scenes at these military contractors with these military contractors that have been developing crazy super weapons that look are 20 years advanced like there's so much far there are so much farther advanced than we can comprehend right now because if the theory

is that if that stuff was made public then our adversaries would get would get wind of it they would start creating the same thing we always want to have that advantage so like is it possible that some of these like Palantir or you name the military contractor is working on some sort of super intelligence that is like so far beyond what we can imagine that we're not even aware of it. It seems unlikely. The money is so good in a public open research space for it, offering, you know, in terms of options and startup opportunities, billions of dollars. I don't think government can compete with that for top talent, for compute resources. So I would be very surprised if US government had internal program with an AI system years in advance of what we publicly know about.

Right. Yeah. And then you also like there was a story that came out a couple months ago where they, I guess they were using AI to target strikes in Iran. And they typed in, I guess it hit a park called police park, which was just a park. a park with some grass and a bunch of trees and a bench there. And it was just called police park, but they programmed it into the system to target police stations. So that's not very sophisticated, if you ask me.

If they're using that for, if we're at the United States of military is making mistakes like that, that doesn't give me optimism that were that far down the road as far as AI advancement. So the tool specific tool they used, we don't know how it was configured, I think they were just creating lists of possible targets and then somebody approved it. That's all AI did.

It wasn't doing deep research on each target trying to estimate, you know, casualties or anything like that. So you can always misuse a tool that's very human. But at least at the end of the day, a human was deciding, let's start the war, let's end the war, let's shoot at this target. Yeah. What is your view on like how AI has been integrated in at least as far as like China as far as like the surveillance that's everywhere and how it's like become this total surveillance state that's been, you know, not only is there security cameras everywhere, but it's also like social media surveillance, financial surveillance, all that stuff to where it's a complete dominated control grid. what do you think about that as far as being brought into the United States with AI? And Snowden told us they spying in us. Your cell phone tells them everything they need to know about you.

It's not that different. It's the question is how is it being used by humans in charge. Yeah. Do you think the NSA has like an advanced version of this already that they're using to spy on? to track everything that goes in through your phone and all that. It definitely have capability. I don't know how much data they actually collect and if it's being stored, but we know they are capable of doing it.

Yeah, the flock camera stuff's crazy too. I mean, that stuff's like everyone's freaking out about those flot cameras right now. And I don't know if those things are using AI or not, but I'm sure they are. Everything is going to end up being data for some AI system somewhere. Yeah. It's too valuable not to use it. Right. Let's take a quick break.

I can use the bathroom real quick. We'll be right back. You know, some people believe that it's futile to resist this stuff and we should just go along with it and maybe merge with it. You know, and, you know, Elon talks all about NeurLink and using this kind of stuff to fix disabilities or, you know, but I think his, like, what he said on one of his episodes with Joe. Rogan, one of his early episodes, was that it'll enable us to communicate telepathically eventually. What do you think about all that? So NeuroLink is awesome for disabled. I agree with that. I think as tool integration, again, very cool to have powerful tools, but if we create super intelligence and integrate with it,

what are you contributing in that system? Why are you there? Are you just a biological bottleneck? You slow, have terrible memory. Why is the system keeping you as part of a loop? I think you become useless. And if you are useless, you are explicitly or implicitly removed from the system. Right, but we would still, it would still be useful to us, right? Like, couldn't we exist simultaneously and still utilize this sort of stuff simultaneously to improve the quality of our lives?

Like, is it possible that we could live in a world, with this super intelligence doing its thing and not get in the way, not interfere, and the same thing, like it doesn't interfere with us, like the same way we live here with apes and monkeys and uncontacted tribes. We're just like a- We killed most of them almost completely wiped them out.

And the only reason we didn't is because we had no reason to. If tomorrow we had to convert the planet into fuel to fly to space, we wouldn't care about them. Yeah, but we don't... I mean, right, if we get in their way, but like right now, we go out of our way to protect certain species of animals and uncontacted indigenous tribes in the rainforest

and stuff like that. Like, we purposely don't mess with them. We also factory farm billions of cows, abuse them, torture them, and then eat them. Yeah. I wouldn't trust our good nature to make sure animals are doing well. Yeah, but how do you go from us eating biological animals for their meat to like how could, what is the worst case scenario?

What do you think this, if the superintelligence, sentient superintelligence got out of the bag, what would be the scariest doomsday scenario in your mind of how they would deal with us or what they would use us for? So people talk about existential risks. It kills us for some reason because it doesn't care about us. It needs to do something else. Cool down the planet to improve compute, convert the planet to fuel. We have no idea. And the worst outcome would be suffering risks. For whatever reason, we're still around, but our existence is just miserable. Kind of like factory farming for animals. They're not dead, but I think many of them wish they were dead. You think they would wish? You think they would want to get vengeance on us or like kill us just for the sake of killing us? You're talking about cyber intelligence? Yeah. No, no, no.

Again, I don't think it's on purpose doing it, but there could be reasons we cannot comprehend. Yeah, I see you're saying. Cooling down the planet for the servers, I can comprehend that. That makes sense. But that's, again, that's things we can comprehend, but I cannot tell you what something that much smarter will do. I'm just saying if you're not controlling it, you're not controlling it. You're not deciding and you're not in any way making it like you and care about you, then anything goes.

Have you tried? I mean, I understand what you're saying. Like, we're not going to be able to comprehend it. But like, have you tried to work out any scenarios in your head that you can fathom, that you can rationalize? Like if you were to put yourself in the shoes of the superintelligence? I don't think it's, you just think it's futile. I don't think it's possible to do that.

I don't think, I can think of. what someone, you know, 10% smarter than me might do. If I think long enough, maybe I'll get good estimates. But if it's thousands of times smarter, it's not meaningful. It's just hard, yeah, it's just very, it seems like. I think it's bad enough with existential problems. We don't even have to get into suffering risks to fully appreciate importance of this problem.

It's existential. It is existential. After it kills us, what does the world look like after that? You think it's just server farms everywhere? Again, so the moment we create superintelligence capable of this recursive self-improvement process, that's the singularity point, technological singularity, which is defined as point beyond which we cannot see what happens. The controlling power in that world is smarter than us.

An agent cannot make accurate predictions about a smarter agent. So you keep asking me in different ways, same question. What will superintelligence do to us, four us, for the planet, after it. None of it is predictable in a meaningful way. I can give you random BS stories from my brain. That's not what's going to happen. Right. I see what you're saying. But I guess that's just the way my brain works.

We want to understand the world. We want to predict the future. We've got really good at finding patterns. and out-competed other animals by doing that. But we're in a position where we're creating something more capable than us. Right. So you've written over what 300 papers?

Books, papers, articles. What is the most recent? It's a great question. You wanna pull up my Google Scholar account, we can see. You don't remember it? How long ago was it? It's been a while, maybe a couple months now. A couple months?

I can see, again, I don't remember exactly. I keep working on multiple pages. If you sorted by year. What are you working on right now? Limits to detecting whatever something is a fake or not, something AI generated or not, so basically telling if something is real. Okay.

So if I give you an image. Can you tell in general if it's real or not? And the argument is it's another impossibility result. The argument is what again? The argument is that it's impossible. It's impossible. I have a number of impossibility papers it's impossible to predict what they're going to do impossible to control them impossible to

explain how they actually work internally and that's another impossibility result you cannot in general tell if something is generated or authentic without some secondary source of information telling you just from the image you should not be able to right so here's a paper the latest one 2026 testing obedience and control this is about irrational obedience how do you prove you that you are obedient to a smarter agent or a boss. And the argument is, if I do things which are rational, you don't know if I'm doing them because I'm rational or because you told me to. So if you tell your kids, wash your hands, and they wash their hands, are they listening to you, or they're just smart people who wash their hands? You have no idea. But if you give them an order,

which makes no sense, just like bang your head against the wall twice and Tuesday. And they do it, they do it because you told them. And so, So the paper analyzes how can you show or how can AI show that it is in fact obedient to us. Right. How can we, how we done it? Well, that's a... I would imagine this would be like a daily test they would be doing it. Right.

So you're getting on some of it. So frequency is very important. You have to constantly verify that you are still obedient. You didn't have a treacherous turn and now you turned on me. Yeah. And they are doing this. That's my research. No one's doing it.

No one's doing it at all. Well, I hope somebody reads it. See, citations at zero so far, new paper. So I suggest somebody reads it. Which one? This is the top one you're looking at? And so when you're writing this and you're doing your research on it, who are you, are you reaching out to people at these companies and asking these questions and they're giving you these answers? Like they're being pretty open with you about this? I'm not directly communicating with the industry. I collaborate with people in academia and other researchers, but usually people at the, the companies are somewhat limited in what they can do publicly say or even publish. Exactly. If they're limited in what they're allowed to say, how do you know for sure if they're not doing this?

And if you're not talking to them, like if you're not like you're doing an investigative work on this. So again, we don't know what happens internally, but based on what they release and publish, I haven't seen. You would imagine that if you, I would imagine if I was this company, that would be something I would want to make public if I was doing it, right? you'd want to let the good stuff out. Like if we're constantly making sure every day that this thing is remaining obedient, that's something I wouldn't want to be a headline. Different companies have different publishing policies.

Like Apple usually is very secretive. They wouldn't publish a lot. Maybe Open AI tries to publish a little more. So it really depends on the company too. Yeah. I guess you would have the issues of depending on what you're published. You probably don't want to publish much because you don't want your competitors to get the gist.

A lot of company, I know multiple people who quit the company so they can speak freely so they can do research freely, including Jeffrey Hinton, who's, you know, Nobel Prize winning creator of machine learning. So if he couldn't speak freely at Google, then what chance do many of them have? Where he worked for, what part of Google did he work for? That's a great question. Was he the deep mind stuff, doing the deep mind? I don't want to make up an answer, so we're going to look it up, but he worked for that. and he quit to have some freedom to speak about safety concerns. Right. Is the godfather?

He won. Oh, yeah, he's the godfather. Yeah, yeah, yeah. He got Nobel Prize for it. And during his Nobel Prize acceptance speech, he basically talked about EA safety and how dangerous those things are. It seems like a lot of the folks that are high up in these companies are very concerned about living forever, too, you know.

As they should be. It's good not to die. Right. They seem very concerned. They're qualified as in this transhumanist group. Except Elon. Elon doesn't want to leave forever. I think he was against that.

He thinks it's a bad thing, yeah. But everyone else, yes. But a lot of the other ones do. Yeah. That's very strange. And they have these, I don't know if you've seen them. They have these cryogenic facilities where they are going to have their brains frozen and like in carbonite. Maybe a whole body. Yeah, sometimes the whole body, sometimes they

just get their head severed and put in there. Well, usually that's if you can't afford the whole body. If you can't afford the body. That's a discounted option. I think if just the head, it's like 250 grand or something like that. We had a guy in here once who he was wearing a dog tag. Yeah, yeah. I know what's your talking about? I think just the head is like 50 and the whole body is like 250, but maybe inflation happened. Yeah. Yeah. Oh, it's gone down. Because they're, they're trying to get customers. 100,000 for full body? The thing is most people don't pay for it.

You just get life insurance. You pay your monthly premium and it covers it. Right. Yeah, exactly. A very good deal. You should sign up. This seems like a scam though because like if there was some sort of an issue and the power grid went down or there was some sort of like global war. It's liquid nitrogen. It doesn't need power to sustain the cold.

Really? Yeah. Alternatively, if you don't want cold there are now other types of preservation like embalment and other things which are less likely to work, but you have options. If the power grid goes down, those things are going to stay cold? For a long time. Whoa, I didn't know that. And they have backups and this is like somewhat important.

Yeah, the guy that was on the show who was telling me about this, he says he believes that when humanity falls, if it falls, or after he dies long into the future, when superintelligence will come revive him and put his... I hope it's a benevolent one. put his head in a cyborg. That's the danger, right? You don't know what world you want to wake up in. Right, right.

And do you want to wake up if it's fucking hell world? Exactly. Maybe they bring you back to torture you. Who knows? Yeah. Oh my God. That's dark. Yeah, these people, they seem to be projecting some of like their worst fears into their work and to their companies and what they're trying to do, you know, just to their fear, their own fear of death seems to be fueling a lot of the stuff that they do, you know, whether it be like stuff like Palantir and the AI stuff and the surveillance stuff

and all of that. I was volunteer fear of death? Well, Peter Thiel in particular seems to be like very hyper fixated on living a long time and his own health. Sure. and that kind of stuff. And, you know, he's also, like, simultaneously pushing this whole, like, Christian narrative in Silicon Valley, trying to paint everything Silicon Valley is doing under this Christian lens.

There's this new big church that's there. And which is, like, counterintuitive to Christian. It doesn't make sense. Like, why would you want to live forever if you're, like, a real Christian? Like, it doesn't really, if you read the Bible, that doesn't really compute, right? Yeah. So, and if you listen to some of his interviews and read between the lines and some of the philosophers that he reads and some of the stuff that he talks about, it seems like, you know, maybe if you were him, this is kind of like him. And then also, not to mention him buying that huge giant, like, country-sized piece of land in Argentina, I think it was.

he seems to be like very concerned for his own safety and his own longevity kind of outside of the rest of civilization right and all this stuff seems to be some sort of protective mechanism that's kind of like that's what I mean by that but how is that connected to Palantir? Because that's all the stuff that Palantir is doing like they're doing AI and they're doing like surveillance stuff and uh I just don't understand the connection to longevity so buying land they get security I get. He wants to live. He wants to be protected and doesn't want to die, right? So you got to make sure that everyone's on their best behavior. You got to make sure you... But you would then think you'd have a company which is doing cryopreservation or something

related to what you're describing. Volunteer seems to be a spying company. A spying company, a weapons company, an AI company, doing all kinds of crazy stuff like that, right? I'm just saying I don't see a direct connection. Maybe maybe a maybe not like a a medical connection, but for sure, a safety connection, right? And a security connection. Like, he wants security and he wants safety. And, you know, maybe I don't know what he's doing, like medical wise or like trying to. I mean, I know for a fact in the Epstein files, he posted a list of what he could and couldn't eat for lunch. And it was like a crazy laundry list of items that he would eat and wouldn't eat because, you know, for his own health,

which is like a crazy thing to do. Maybe if you're a billionaire, it's normal. I don't know. Most people have weird diets. I cannot judge him for that. Yeah. Yeah, most people do. You're right. But I don't know. It just, I don't understand how some of these people's minds work. At the end of the day, they are just human. They are just human beings. And they, some of them have families. Some of them don't. Most of them probably don't. And it's just like, I wish you could just lock one of them in a room with you for a couple hours. and really figure out how their minds work and what their motivations are.

And having all of them together in a room would be a very good step forward. I think we can make a deal. Have you tried to do that? Not directly, but there are organizations dedicated to AI safety, which in communication with them, I just had the Max Tagmark on my podcast and we talked about that,

he's in good relationship with all of them. So there's some. some efforts. Who, out of all of them, who are you most optimistic about? In terms of actually not being open to what you're talking about, being open to. I think they all hinted that they are open to external pressure to make a deal. They cannot do it unilaterally because they're afraid that competitors will continue. But if all of them pause and they get to preserve their high status, economic opportunity,

I think all of them would sign. And what is Ray Kurzweil's, Ray Kurzweil has a very rosy colored glasses view of all this stuff, right? In his book, he has a chapter about dangers of his technology. So he considered it, but he's more optimistic about merging with technology,

becoming technology, mind uploading, things like that. Mind uploading, right? And he's another one that seems to be very hyper-obsessed with living forever. Takes like, you know, a bucket full of, five gallon bucket full of vitamins every day. Yes.

And stuff like that. I mean. It's just an interesting. It's just an interesting pattern. The biggest problem is when you take 100 independent medications, which have not been tested together, you don't know how they interact. So maybe vitamin D is wonderful for you,

but then you add 100 other substances, things are complex. And I haven't seen visually at least amazing life-preserving effects of this technology. Yeah. Well, one of the crazy side effects of this technology, which, I mean, isn't even that, it's not even like super advanced technology. Just the fact that we have computers and most people work at sitting down at computers all day inside has made us extremely deficient in so many different things. Like a lot of us are inside all day long, staring at these bluelet screens, not getting, not getting the,

available sunlight. We are awake at weird times, not following like the circadian balance. And that is severely limiting our lifespans and are, it's like increasing all kinds of comorbidities inside of human beings. Like cancers are at an all time high. All, you know, I think this country, like the most developed countries that where people are doing these types of jobs seem to be, there seems to be a correlation with health outcomes with that, right?

Like the third world countries, people are outside all day doing manual labor. They seem to be the most healthy. So just like high-level view, you know, you can see how technology severely changes the human body and the human biology. But, you know, at the same time, we're like also super adaptable, like you said, not just cognitively, but physically. We're very physically adaptable and resilient things. And if we are going to keep this stuff in a box, not let it take over and kill us all and just use it, we're certainly going to evolve into something that's way different than what we are or what we used to be, I would imagine. Well, if we develop useful tools, for example, understanding human genome, we can make direct improvements to our design. We can live longer by design.

We can be healthier by design. We can gain less weight. We can have more muscle. We can make those changes. Once you understand human genome, it's a software program. You can directly change certain variables on it. Yeah, that's crazy. We've been doing it indirectly.

Every time you pick a more beautiful woman, you're making a genetic choice, right? Yeah. Yeah, the fact that we can create CRISPR babies and design, people can design their own babies and like pick from a menu like what's your eye color what's the hair color what's the skin complexion but those things are completely unimportant what you care about is intelligence we shouldn't be but we'll have ability to pick important things not having a cancer gene is important of course but it is true that you know we only for the most part we only do care about the cosmetics And those are just very easy to judge.

Intelligence is based on hundreds of genes interacting, whereas I think things like I call her gender, much easier to spot and fix. That is interesting though, how, you know, obviously with the advancement of this stuff that we will be able to do that if it becomes like a legal thing. But I think right now it's completely illegal to do that kind of like genetic modification and stuff like that. This is the stuff that like Epstein was into. Christine was emailing NSA cryptographers about manipulating the human DNA. Did you see that? No, I didn't read any of that. Yeah, he was talking to NSA hackers, like hackers that worked inside the NSA to figure

out how to manipulate the DNA and the human genome because he was interested in printing, cloning himself. Seems like wrong kind of specialist for that project. Seems like what? Wrong professional for that type of work. Well, no. I mean, if you have cryptographer, didn't you just say that, like, that's what the DNA is? Like, it's a code?

It's a code, but it's not encrypted, right? We just, so cryptography is usually about making something hard to read. Steve, you find the story about the NSA, about Epstein contacting NSA. Not my area of expertise. Yeah, but it's interesting, though, why he was talking to the NSA computer hackers, though, right? Like, why would he be talking to computer hackers about this stuff? leaked emails from the Department of Justice. Associates a signal intelligence used by the government agencies to apply breaking the DNA code and the protein signal problems.

Oh, okay. Interesting. That makes sense. He asked contact to help him find and recruit NSA quality code theorists and code breakers from intelligence services stating that. Breaking foreign codes is the expertise of U.S. NSA. The concept, Epstein viewed the human genome as a cryptographic puzzle. and frequently discussed eugenics, transhumanism, and altering human traits. Never heard of it. Thank you for educating. Yeah, of course. Does that make sense to you? I don't know if it's exactly the same as the cryptographic puzzles cryptographers work with,

but I mean, the word code appears in both, so I guess you made a connection. With DNA, it's more about genotype versus phenotype and trying to understand how the code translates into physical manifestation. So this makes you taller. This makes you, this is not exactly the symbol manipulation of cryptographic protocols. But there's smart people with computers. So I'll figure it out. Right. Is there anything else we should discuss that we haven't covered yet?

Many interesting things. What do you want to talk about? I don't know. You tell me. Anything that you can think of that I'm not asking? Have you heard people talk about AI safety on a show before? AI safety. No, my topic. You're the first. Oh, really? Yeah. Okay. Well, do you kind of agree with what I'm saying? Do you understand what I'm saying? Yeah, for the most part. Yeah. Okay. Is there something you disagree with? Maybe we can clarify it for the audience. Same with you if there's any question. No, nothing I disagree with. Not really. I'm just trying to mainly get just a better high level understanding of it. I understand that it's like super dangerous. But like I'm just curious about like any more specific.

specific kind of like research you're doing drilling down into stuff, like that paper you were just talking about that you were working on with seeing it people could detect a fake picture versus a real picture. So it's called Generative AI. It's so good at creating pictures, text, videos, everything, virtual worlds, new molecules. If AI generates all those things, we are bombarded by so much of that artificial, of generated content. If I go on social media, soon,

majority of stories, majority of posts are all written by AI. How can I tell? The existing detectors are terrible. They're just not reliable at all. You can't say, I run it through detector, I know it's 100% AI. They don't work like that.

So is that a limitation of today's technology? Or is it in theory and principle, impossible to tell? And that's what I'm researching. I think it's an impossibility to actually be able distinguish the way it works is you have one algorithm generating fakes and another algorithm trying to detect but eventually they meet in the middle it's 50-50 if there is something i can use to tell it's a fake image now this counterfeiter algorithm knows how i did it and it improves the faking process

so at the end maybe it's not the case today but eventually they just end up being equal in power and so it's impossible when you see a political all add, a video. You don't know if president actually did those things or it's a fake video. Right. And it cuts both ways. If somebody has a horrible video about you, you can say it's a deep fake. But once I see it, it's hard to unsee it, right? Especially the night before the election happens. That's funny. You're teaching it to be more efficient. It is exactly the process. Right. Yeah, it's like a feedback loop. Like the rising tide, Ray raises all boats. Doing it forever, yeah. Mm-hmm. Yeah. And then with like, with, with, with,

just the overwhelming amount of stuff happening on social media. You know, not only is there just shit happening constantly around the world and so many different headlines on every single thing, but you're throwing in fake stuff on top of that and like fake people, fake accounts like on X that are reporting on stuff. But that's on top of just having so much more real news happening. Exactly. If before we had an AI breakthrough once a decade,

Then it was once a year, once a month. Now it's five times a day something new happens. I travel here to do this interview. I come back, I have to read 400 new papers. I'm not going to have time to do it. Right. And you weren't like the human brain wasn't meant to keep track of all that stuff, right? Like we evolved to...

Even with help, even with AI now summarizing a paper for me, it's still just too much. Do you use AI often to help you with stuff like that? I use AI tools for everything I can, yeah. Really? Absolutely. Which ones do you use? Again, I think they're all very similar in power. So it really depends on what's available.

Some have a little less restrictions for different tasks. So I'm happy with most of them. One of the things that just like frightens me the most is how it's being integrated into social media just because like that's something I'm ideal with all the time. You know, with the podcast. I see it happening all the time. And I see people that I'm close to like going crazy because they don't know. they just believe everything they see and they don't really have a good AI bullshit detector. You know, I like to think that I have a good AI bullshit detector, but it's probably...

But is it based on common sense context information of the actual image or video you're seeing? Because that's about to change. The quality is going to be so high. You have to have some other knowledge to be able to tell. Right. Right. And especially when, you know, you're getting it. through trusted sources too, who are probably, you know, some of the big news organizations

are probably buying into some of this stuff as well and not vetting it enough, not, at least at some point, they're not going to be able to tell the difference and they're going to have to just report on stuff because they rely on headlines and viewership and stuff like that. So what kind of implications will that have, you know, politically, geopolitically, culturally, locally. I'm actually surprised it's not a bigger part of our election process where we have to deal with so many fake stories and videos. Our election process? Yeah. So the night before the election, I would expect explosion of fake videos, but it's not happening. Yeah. Well, maybe the next election it will happen. Well, see. Well, so the problem with that is you have people who suspect or are

hyper paranoid about everything being fake, right? So if they see something that is real, they're automatically like, oh, no, that's fake, that's fake, that's bullshit. There's no way that's real because they're constantly being beaten over the head with fake stuff all the time. They finally see the real shit and they think that that's fake too. And that's also a problem. We can no longer know what's through.

Yeah, man, it's hard to have an optimistic outlook on all of the stuff. But we're still alive. We're still in charge. Let's not make bad decisions. Let's not build general superintelligence. It takes very little work to not do something. It's easy. It's very little. Let's just not do it. Let's create useful tools. There's lots of problems humanity is facing. Let's work on those. Yeah, but it's like that's what humans do though, right? Like that's our nature. We always want to create new shit, more advanced stuff, more cutting edge stuff. And it's kind of like going against our very nature to stop doing that, right?

We are very good at following personal self-interest. If I know it's bad for me, I'm not going to do it. Yeah. Yeah, but like we talked about earlier, you and I know it's bad for us, but we have no vote. They know it's bad for them. They all on the record are saying, this is dangerous. This is going to kill everyone. All of them said it.

They have P-Doom levels, which are insanely. Unless they're dumb, unless they are destructive. Like think of a terrorist group who cares nothing about, I mean, hell they'll kill themselves, right? Yeah. They are believers in rewards in the afterlife. These people are all atheists as far as I know. Except for Peter Thiel. He's not a cutting edge AI developer.

Really? I don't think what they produce is top five. I thought Palantir had a pretty legit AI developer. I think what they have is a very well-integrated model with military and security apparatus, but I don't think they have a cutting-edge model yet. Again, maybe my knowledge is limited and what they do internally. Did you see the reports of when, do you remember when the whole doggy thing happened with Elon, where he was still working with Trump?

Doge. I like calling a doggie when he got in there and he was. trying to clean house and all those interagencies like the IRS and all that stuff. There were stories out there that reporting that he had a contract with Palantir AI to integrate that into like the HHS, the IRS, and all these agencies where he was cutting all these people, cutting, getting rid of all these people and trying to merge all the the data in all of these organizations like the IRS and the HHS, merge all of it and consolidate it all into this Palantir AI to where it was just like one database that had like a very high,

deep level of information on all the citizens of the US. I would guess he would be pushing XAI instead. That's what I would have thought too. I would have guessed that too, but I guess Palantir is very deeply intertwined with with the government. I don't watch news, I'm very happy about it, so I can't really... Yeah. Well, that just makes me skeptical about the Palantir AI.

I wonder how powerful it really is. So usually when you get rankings of top AI models, they don't show up at the top for most tasks I care about. Really? I'm curious. We can Google those things. See what you can find, Steve. Power of Google. See if you can find about that story.

if you can find about that story if it was legit. So this is actually something you can do with today's AI. It sits in the back, listens to the conversation, and pulls up relevant stuff. Way faster than a human code. Really? Absolutely. That's easy. Voice recognition, transcribe, search.

Oh, wow. Recent. That doesn't take super intelligence. Go up to the top. Oh, wait. Yeah. Okay. Palantir's been discussed in connection with Department of Government Efficiency Doge.

where I have potential data and software overhauls alongside CEO Alex Carp's push to for enterprise AI control and data sovereignty. The IRS plans reports indicate that Do's representatives have appointed to Palantir as a prospective vendor for data management projects as part of federal efficiency initiatives. Palantir co-founded by Teal and associate of Musk trying the firm closely into discussions surrounding Doge's federal texts. CEO Karp criticized major frontier AI labs like OpenAI and Anthropic during the earnings call, arguing that they attempt to colonize enterprises and absorb corporate intellectual properties into their training data. Okay.

So, yeah, you probably are right then. They're just trying to consolidate data. And this is more of like, uh, controlled police state types fears. I think they're very down to earth. We kind of like we have data sets. We want to understand them. We want to combine them.

I think that's what they work on. But again, what they do internally? We can talk about it until we're blue in the face, or at least me. Maybe not you. Really, the scary part is no one is in control. No one understands what's happening, how it works, and no one has power to stop it.

Any one of those CEOs unilaterally cannot stop. They'll be replaced immediately. So we have this process where we're losing control already, even before we build super intelligence. Some people talk about like if we could have some super intelligence, if some super intelligence was created, it would want to, if it did want to survive, it would want to ensure the survival of the planet. It would want to make sure that this planet can sustain. So that would be an overall good, a net good. So even if humanity had to pay a small price for the planet to be safe,

like maybe this thing would, if this superintelligence made all the decisions for humanity and for the world, it would be in the greater benefit of Earth and for nature and all that stuff, because it would want to preserve that stuff. Wouldn't it wipe humans as polluters and just preserve the planet? Maybe not. Maybe it would find us useful in some sense. Again, in the world with superintelligence in it, what are you contributing? Don't you, like don't you, some people would claim that eliminating all the humans would maybe cause problems for them.

Maybe they would need us for certain things, certain tasks. I would love to know what those are. We can tell people what jobs to get. Like we can be the slaves to the... You're not a very efficient slave. If we have, you know, humanoid robots, if they have nanotechnology. I don't think you are contributing much. Right.

Well, Roman, thank you so much for doing this man. This has blown me away and scared the shit out of me. Let's not build general superintelligence. I agree. Let's not do it. Tell people about your books, about your podcast, where they can find you and learn more about you, all that stuff. We have a podcast.

It's called the Roman Forum. It's Roman Impulski. And we talk about AI Safety. with leaders in that field. If you want to read about the concerns I have, there is a book, AI, Unexplainable, Unpredictable and Controllable. It addresses all the limitations

of existing safety mechanisms. Fantastic, man. We'll make sure we link all that stuff below. Thank you. Thanks again. I really enjoyed this. That was fun. All right.

Good night, everybody.