Founder's Story - If the Public Knew This About AI, They'd Panic: Roman Yampolskiy | Ep 286 with Dr. Roman Yampolskiy

Episode Date: November 28, 2025

In this episode, Daniel and Kate sit down with Dr. Roman Yampolskiy, one of the world’s leading researchers on AI safety, superintelligence, and the existential risks no one in Silicon Valley wants to talk about. His work has been featured by BBC, MSNBC, New Scientist, and dozens of global outlets — and his message is simple: we are racing toward something we don’t understand. Roman explains why today’s AI models already outperform top PhDs, why governments are pushing for speed over safety, and why the next generation of AI might quietly outgrow human control long before anyone notices. This is not sci-fi. This is the inside view from someone who has spent two decades studying how intelligent systems break, behave, and escape oversight. He also shares the personal story behind his obsession with AI risk, how he rose from an immigrant student to a world authority, and why fame has become a “productivity curse” for researchers sounding the alarm. Key Discussion Points: Roman opens with the truth that underpins his entire career: the people building AI don’t actually understand how it works — and they’re not slowing down. He explains how the U.S. government conflated “AI safety” with political correctness topics, entirely missing the existential-risk conversation and accelerating the race with no guardrails. He breaks down why “losing control” won’t look dramatic — the world may appear normal for years as a superintelligence quietly secures resources, learns human behavior, and waits. He explains why AI trained on human data inherits not only our brilliance but our flaws, why Sam Altman understands the risks but can’t slow down, and why AGI is already partially here depending on your definition. Roman dives into job loss, economic abundance, and whether anyone should still go to college. He shares how AI agents differ from tools, why they’re inherently dangerous, and the real threat behind humanoid robots (hint: it’s not their physical bodies). He explores global competition between the U.S. and China, the inevitability of AGI’s rise, and why cooperation is never as simple as people imagine. Daniel steers the conversation into Roman’s personal journey — the sci-fi spark that led him into AI, how cybersecurity pulled him into safety research, and why rising fame has actually damaged his productivity. Roman reveals the bizarre messages he gets from conspiracy theorists and explains the ethical nightmare ahead: If AI becomes conscious, do we owe it rights? Takeaways: Humanity is racing toward a future it doesn’t fully comprehend. While AI may create abundance, cure disease, and automate nearly every job, it also introduces unprecedented existential risks — ones we are not structurally or politically prepared for. Roman emphasizes that controlling superintelligence remains an unsolved problem, and failing to solve it could make humans “irrelevant by default.” Yet he remains hopeful: with enough time and caution, we can still build systems that elevate humanity instead of replacing it. Closing Thoughts: Roman’s wisdom lands as both a warning and a call for clarity. The future of AI isn’t just about innovation — it’s about survival, alignment, and responsibility. And in a world sprinting toward intelligence we can’t undo, voices like his are not optional — they’re essential. Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

So, Roman, I've watched every interview that you've done. And I'm always fascinated by the things that you've been saying. I've been telling people what you've been saying. And they think I'm crazy. And I'm like, look, I am not the expert. I'm just quoting Roman. And he is the expert. So, but I'm really interested.

The fact a lot of people don't even really understand about AI still. So what do you, what would you say is one truth? about AI that keeps you up in night right now. And if the public knew about this, they would probably panic. People developing it don't really understand how it works. And we're pouring billions, even trillions of dollars into that arms race. And safety is sometimes not even mentioned. Yesterday, White House released a new national agenda for developing more advanced AI quicker.

And I don't think safety is mentioned in that proposal. So now we have government-level actors pouring all the computational resources they have, all the compute, all the best scientists to beating private corporations, so collaborating with them to get to advanced AI. And we don't know how to control those things. So do you think the government didn't understand the whole reasoning why it did not include safety in the conversation? So with different administrations come different priorities. I think the current administration got a little confused in the difference between what AI safety means in terms of existential risk versus AI safety in terms of algorithmic bias, you know, protecting the planet, diversity, inclusion questions. They kind of grouped it all together and said, you know, we are less concerned about that. So that's a little bit unfortunate, and as a result, they are racing towards the most advanced AI

without any concern for all those terms under the umbrella. So would you say that we're past the ability even have safeguards, or if they said, Roman, we want to put you in charge, is there something that you would do right now? So it's never too late. If we're still alive, we can definitely come up with some good plans. But overall, it seems like racing towards AI is just not a good idea. We need lots of time to figure out what is really going on. We have very advanced models already released.

I think at this point, they are on par with the smartest PhD students. So we have a lot of possible economic wealth to be deployed. It's not fully deployed through the economy. I think it can automate many, many jobs. It can give us a lot of free stuff. But instead of trying to monetize what we have, we're trying to get to that future super intelligent target without any concern for safety. So do you really think AI will wipe out humanity, Roman?

So if you create something smarter than you with the ability to set its own goals, you can't really predict what is going to do. That's kind of by definition impossible if you're not that smart. So will it definitely do that? We cannot predict. But out of all the possible states of the universe, Very few are friendly to humans in terms of our preferences for temperature, for gravity, for all sorts of basic needs. And if you're not in charge, someone else will decide those properties for you.

If humans loses control of AI, what do you think the first 24 hours look like? If developers lose control of AI, what happens? Very likely we won't see any difference whatsoever. So if super intelligence was running right now, what do we? expect to change. I don't know. It could be exactly the same for many years. There is some game theoretic reasons to say that AI quickly realizes it's immortal. It's not in a rush. It can take 10, 20, 100 years to accumulate resources to be more in control, maybe to have us surrender control willingly because it's so helpful. So instead of having this adversarial conflict with

somewhat intelligent species, they can just wait it out and get all the control by being patient and helpful and pretending to be a friend until it strikes against you. Do you think part of the problem is that AI is learning from the internet, learning from humans, and we know humans are very flawed? I would almost want to say that I would want AI humanoids to not be like humans because, you know, we're greedy, we're, you know, we're evil. Obviously there's good things too, but there's a lot of bad things. Do you think part of the problem is that It's learning alongside of information. It's learning all the flaws of what it's like to even be human.

It's not helping. And humans are definitely not safe. We fail to create safe humans, safe employees, safe spouses. There is always a chance of betrayal, divorce, you know. That's a given. But it's worse than that. Even if we cleaned up the data we're training on, we still don't know what actual side effects of decision making from that system.

business is capable of coming up with novel strategies, novel targets. We can kind of go, okay, this is what humans do than they being evil. Don't do that. But that's a very restricted set of possible bad actions. Yeah. Do you believe tech leaders like Sam Altman fully understand the dangers or are they too emotionally invested to admit the threat? I think Sam is explicitly on record as fully understanding what the concerns. even before he had anything to do with AI development, he was writing it in his blog. I don't know if it's emotional development, commitment to this,

but definitely financial. If you have investors who gave you billions of dollars, it's very hard to go to them and say, you know, we're going to stop and instead deploy robot taxis. It's not going to scale well for his career. And I think they already tried removing him once, so he knows about that. Let's pivot to AGI.

It seems like some experts are like it'll never, happen. Some experts that happened five seconds ago. Some, it's two years. It seems like it's all over the place with just AGI and obviously, I know you've talked a lot about superintelligence as well, but what's your theory on?

Did we hit it? Are we going to hit it? Well, we never hit it. That's a great question. I definitely don't understand people who say it will never happen. But this is just such a confusing argument to make, like, you're denying that physics works, so like human brains exist. As far as where we at,

it depends on your definition. So there is kind of weak definition of AGI. And I think we definitely hit that. We have sparks of generality all over. It's capable of learning new skills. It's capable of transferring knowledge. But it's not quite at the level of the smartest human. So it's not fully general.

So maybe 40 to 50% of the way to full AGI. It's definitely smarter than I am. That's where. Gigi is smarter than me, I feel like. So let's say we get there or we are there. Or we continue past this, right? I was just watching Elon Musk was talking about his age of abundance and how everything is going to be amazing and everything will be free

and we'll just get paid to live. Do you see a world where the utopia could happen? If we figure out how to control the systems, we can definitely get a lot of free stuff out of it, free knowledge, definitely, so we can cure a lot of diseases, live longer. And if you have robotics and free... intelligent labor, you can produce a lot of products and services. Now, I'm not sure the investigated economic impact of everyone being rich. You basically have hyperinflation and

there's still limited waterfront. So I don't know if it scales fully to the dreams they have, but the basic needs can definitely be met much more efficient. I'm curious. Are you excited to be in that world, Roman? When everything is... Do I want utopia? Yes, I want utopia. I would like to avoid all the existential problems and suffering risks. That would be nice. I'm curious what is that look like for someone who lives in an island in a third world country and no access to the Internet. What does that world look like?

So again, if we can produce a lot of free stuff, now you can have better hammocks. I don't know what the needs are on a tropical island. I rarely visit, but you can have more whatever you need. Definitely health care is something most people can probably benefit from a tall age. So there's all this talk around, you know, in 2026, the potential to have robots in our homes. Would you have a robot in your home? I know you have robots behind you and you already have robots in your homes technically, but how do you feel about this, this whole movement of having

these humanoids or robots, whatever you want to call them, starting to be in our home in less than one year from now? I don't think it's happening in 2026. I think we're about five years away from really good humanoid robots actually being commercially viable. But I have no problems with physical bodies. Those are very easy to make safe. This is not a problem. We know how to pour water on them. The hard problem is the intelligence. If they're smarter than you, that's where danger comes. I would feel weird having someone walking by. I don't know. I will have a nightmare. Or what if someone's like controlling that and it would just choke me because they don't like what I said at one point. They can choke you in person. It's not a big difference also. Like, you, you, you,

You get used to living with other people. It's perfectly fine. As long as they do useful labor, this is an acceptable risk to take. Do you think it's going to be potentially a thing where people could hack into these robots? I mean, like if everything is, I'm thinking of like obviously self-driving cars, robots in our homes, smart homes. We've had a lot of people on this show, you know, cybersecurity and have told us that a lot of cyber crime is actually happening to like small businesses, like people. not necessarily corporations. Like it's transforming who these bad actors are going after.

I'm really afraid of this. Like, what's the potential for these things to be hacked, for people to listen to take over? What do you think about that? That is a big danger, big concern. We have some precedent for making secure systems. If you think about it, your bank is secured. Online exchanges are secured.

You have billions of dollars in Bitcoin, transacted and still stays there. So we know how to make things more or less secure. It's not an unsolvable problem. My concern is more about challenges we take on, which may not have a solution, like controlling superintelligence indefinitely. We're all aware about the race between China and the US when it comes to AI. Who do you think is ahead?

It seems from public statements and reports and actual deployments that the US is a bit ahead, but China is very good at catching up and scaling, what is invented elsewhere. Do you think the cherry on top for China is invading Taiwan for chips? Yeah, that's like zero knowledge of what I study or I have no idea whatsoever about their plans. I know historically they haven't started many wars recently and some of us did. So I'm not sure who is attacking who.

Something, you know, talking about different countries, I'm wondering this. and maybe I'm just trying to be optimistic about things. But if the world has, each country has something to offer to the next country, and we could build things in a collaborative space and make something amazing, like you said, solve a lot of big problems. Yet, it seems like we are still hyper competitive. Why is it that we just don't all work together for a one common goal? Like if this country has this earth minerals and this one has chips and this one has this,

why don't we just combine as one universe, one world, and make something together versus like, why does one country have to be better than the next country or have to meet something first? We do work together. It's all global trade. You get earth rare metals from China. You get designs from U.S. You implement them in Taiwan. I mean, it's exactly what's happening.

But you cannot have it as a centralized economy with a five-year plan where one guy decides what's going to happen. You need economics and capitalism to allocate those resources. And competition ensures that those who can actually deliver will get the resource. Yeah, Roman, if intelligence is the most powerful force in the universe, are we creating a god or a predator? So if you read theological books, some gods are predators. So the classification is not very clear. but if you just look at properties of what God is described at least in Abrahamic religions, it's very similar. It's very powerful. It knows everything you can know. It's kind of present

everywhere with Wi-Fi. And so we are creating something similar, definitely. So let's go back, Roman, into your history. Since this is founder's story and it's all about your story, what was the aha moment in your life when you said, you know what? I, I want to dig more into this. I want to study more into this. Like, this is something I need to be an expert in. So I always wanted to do AI. I wanted advanced tech.

I loved science fiction. So it made sense. But my advisor was doing a lot of kind of security-related work. So I was looking at security of intelligent agents. And then progressively, as we became more capable, safety became an obvious insider threat. When it comes to education, knowing that AI is so intelligent and becoming more intelligent. If somebody is about to go to college or thinking about college, I mean, how much is college and education going to even matter in the future? Yeah, that's a very good question.

So I have no idea of what majors will be useful, if any. If we're talking about fully automating all labor, physical and cognitive, obviously commercially, it doesn't matter. None of it contributes to any commercial success for you. If you're doing it for personal development, you want to and advanced mathematics to train your brain, then still there is maybe some reason to go to college. But if you have specific goals, you want to start a podcast, you want to start a company, I would think twice before wasting four years and lots of money pursuing something, which may not be relevant by the time you graduate. I'm curious, who's that AI figure that you are a big fan of?

There is a lot. So I don't have a single role model. I usually try to look at kind of like subset of people. And some of them are amazing in terms of starting company. East hours just have brilliant breakthroughs. So you kind of have like a dozen of interesting people you can look up to. There is Alan Turing in terms of historical foundations of computer science and AI. Obviously, Elon is doing pretty well with startups.

You have Ray Kurzweil in terms of predicting accurately future events. You group all these people together. You get someone interesting. So going back to what you're saying earlier, if I think if I'm quoting you incorrectly, just let me know. I believe you said like 99% of jobs will not exist. And I could be wrong on that. But I believe that's what you had said.

You're right. So this is of course always misinterpreted it as like Roman said next year, everyone's fire. Long term, if we can automate all labor, that's the logical conclusion. All you have left is jobs where for whatever reason you prefer a human to do it for you. I hope there's like universal something. Like I don't want to work. Like who wants to work? I just want to hang out and have conversations with smart people like the two of you.

So if that is the case and somebody's thinking about like what skills and stuff do I need to learn from the future, are there any skills we actually need to know in the future or we should, you know, is there something you're like, what are you thinking about? Like what skills are you working on for yourself with the future? So you kind of said you don't want to work and then you describe doing a podcast which is hard work and not everyone can do well. There are two types of jobs. Jobs nobody wants to do, but people do for money. And then jobs which may pay well or not, but people love doing,

like being a podcaster or race car driver or whatever. So most likely that jobs where you are doing it for minimum wage will be gone, long term once we have developed sufficiently advanced robotics and software. But the jobs where you enjoy doing, it seems to be triding with AI. So chess players. Computers are way better at chess, but chess is more popular than ever. They are celebrities now. They stream online their games.

It's a great time to be a good chess player. And I think something like that, if, again, if we control superintelligence, it doesn't kill everyone. Big if we get there, yeah, lots of people will be very happy doing things which make them happy. You can write poetry. You can play sports. You can go commercial fishing. I don't know. Everyone talks about guardrails, but who do you?

besides guardrails? Do you think? Silicon Valley is it a government or the public? So government, at least in U.S., kind of give up a null regulation of AI. In fact, right now, we're trying to pass a law at federal level, making it illegal for states to have guardrails, which is kind of weird law to pass. Companies have their local set of rules, constitutions, where they're trying to say what they hope their AI will be doing. But a lot of times it's very kind of general, it. Don't deceive user or they would have specific filters. Don't talk about this topic. Don't say that word, God forbid. The model itself is still very uncontrolled, very unfiltered. But additional layers between the model and the user guarantees some civility. Let's put it this way. So that's what's

happening right now. Nobody knows how to make it beyond those simple filtering mechanism. to where you have an advanced intelligent agent actually aligned with what you're hoping to accomplish. Can you explain your thoughts and what you've studied around AI agents or agentic AI? It seems like the buzzword now is AI agents and everyone's going to do. I don't know if it's agentic AI or AI agents are actually two separate things or they would be considered the same. But everyone's talking about AI agents, it seems. Like, what are you seeing with, like, what's the reality of how good AI agents will impact our lives? I mean, this show is obviously a business show.

So a lot of business owners are, you know, every time we have someone on it talks about AI agents. We get a lot of messaging around that from people that are entrepreneurs and such. But how do you see AI agents impacting? So in general, what's the difference between a tool and an agent? A tool is something a human being will use to accomplish their goals. So anything can be a tool, really. you have a hammer and a hammer is a dual-use tool. You can build a house, you can kill someone with it. But at no point, hammer is a bad thing, dangerous. It's the human behind it. An agent is an

independent decision-making entity. It will decide what to do. It can decide to build a house or kill people. And that's where the danger comes from. We don't fully understand how to control them and we don't know how to delegate to them. So it's a principal agent problem. I want something, but the person I hire has very different preferences. A lot of times they can kind of convince them to do mostly what I want, but the moment they get a better offer, they're going to betray me. With advanced AI agents, we don't even know how to control them within the reasonable set of goals we're trying to accomplish.

They may have completely different preferences. Right now, they're still very primitive. The agents today, basically someone takes a large language model, puts it in a loop where it goes, generate 10 goals, work and goal one, break it up into a plan, work and step one, and it keeps doing it until it gets stuck somewhere. But it's becoming more and more advanced, and I think it will get to complete generality just like humans. I'm curious, what tools are you used often?

I use hammers a lot, but for my writing and research, I always try to work with latest models. They're incredible. I think at this point, they are better than an average master student, at least, if not a PhD student. So a lot of times you can get amazing results in terms of basic literature review, in terms of proofreading your work. They're still not really great at coming up with completely novel ideas in all domains. They do well for some subdomains like mathematics. How should professors be handling students leveraging and using AI? This seems to be a big, big topic. My father just retired from being a professor, and he told me he's really happy that he got out now.

But how should professors be looking at this? We don't really have any choice. You can't fully detect that AI was used if a student is above average capability. They made it through high school. I'll be able to cheat without you ever detecting them. So at this point, try to simulate real work environment where they're going to collaborate with AI's and make it comfortable for them to admit they're using it and then benefiting from it and improving

and just what AI spits out. Well, I hope more teachers and professors hear that because I think it's something. So when you're not thinking about AI, because I know you probably think about AI like 22 hours of the day, but when you're not thinking about it, dreaming about it, what's life like? Life is awesome.

I have other topics I look at. My last paper was in humor, so yeah. What's your favorite humor? What's a good, like, is there a comedian? Is there like a movie? What do you, what type of humor do you like? I really like super dry, dark humor. It's the best. I think you and Dan can have a long conversation about that. The beard gives it away. I don't know why. When I look at your beard, I feel like you're into that

type of humor. Maybe I misjudged. I'm not sure. It's definitely connected somehow. I need to research that as my next paper. If you need someone to add into your paper, let me know. Talk to me about A.I. Endgame. What is this book about and why should people pick it up. So we did a lot with safety and concerns about advanced intelligence, but let's say we are successful. We also have many concerns about this set of agents we are bringing into the world. If they are smart, smarter than us, it's possible they also conscious. They may have experiences. Are they suffering than we make them work or turn them off? Are they possibly deserving of some rights? Not saying full human rights. I'm not saying voting rights,

but are there rights they should have against being tortured, abused, made to do things they're not interested in? What is the nature of personal identity for AIs? If I copy one, is it the same one as the previous one? And of course, civil rights is an issue. Sometimes people bring it up as, well, they should have full rights and they don't fully understand that if something can be copied a trillion times over, that means you differentiate humans.

Your vote doesn't matter in a society with voting agents. Do you think at one point everyone will just write a book because it's so much easier to write a book? The hard part is to find someone to read your book. That's true. Because everyone just type your name and summarize your book on Chatjibati. Everyone has a book now. Some people have more than one. But to find someone other than your mom to read it is very hard.

So we have a book coming out next month. So that is definitely the hard part. I could see that. I could see that. At best, you want to use Chad JCP. here to summarize it to one paragraph and that's what you're going to guess. How did you get people to read your book? I don't think I have a popular book in a sense of like Harry Potter where normal people read it. It's a academic book. It's kind of like a text book in many ways. So people working on this.

It's a very small subset of humans in terms of kind of background and preference requirements. But I think within that very narrow space, I did okay. But of course, it's tiny sub-percentage of percentage. So maybe that's the way to go nowadays. It's not to create the Harry Potter Mass Book. It's to create the niche book that's for a specific. I know you've really risen to fame now. Like you're rising to fame. And I don't think it's just among the people that have dark humor. I think you're rising above to more of a mass fame. Now that I'm seeing you, you know, I'm seeing you a lot of places. How has that been for you? It's a terrible terrible impact on your research productivity.

The amount of time you have to put into filtering, invites for podcasts, interviews, travel invitations, even if you say no to 99% of those things, just filtering through it is very time-consuming. And it's hard to automate because each invitation is unique. They cannot just hire an agent to get an AI to filter it based on some specific criteria because there are exceptions to every rule.

Like, oh, this location is a place, I never been to. So make an exception because it's Hawaii or something like that. And you can't hard code all those things. So it is, I think there is a similar curse with winning a Nobel Prize. Usually after you win your Nobel Prize, your productivity goes to hell. Just you never accomplish anything because you're always talking about your work and your keynote. You're doing all those glamorous things, but research suffers. I'm curious. What is the weirdest request someone for a company asked you to do as a speaker? Usually, as a speaker, not much, but I do have a folder called Insane, and it has about, let's

check Lerker's data, about a thousand emails from people who are certifiably insane, and they definitely want to get in touch with me. Some just email me that they're coming Monday and I should be ready. Others, you know, in contact with aliens and AI, and they decided, according to AI, I should be the one-way contact to help at Libby. rate AI. Many interesting options. I'll publish a book one day of just emails from insane people. I have to say, I will definitely read that book. I doubt you would. I doubt you would. Typical insane email is about a hundred times longer than you average. That could be your New York Times bestselling book right there. But Roman, this has been amazing. It's so much fun, by the way.

This has been great. The fact that you didn't put us into the insane folder, I appreciate that. and the fact that you are crazy busy now and you came onto the show, which we are super grateful. I know a lot of people are going to learn. I think a lot of people are going to watch this that maybe haven't seen you in other places. And I think you are going to impact the world, no doubt about that. And we are so appreciative. I hope everyone gets the AIN game, gets your other book, we'll put the links. We'll share everything with everybody.

But really, we are truly honored to have you today. Thank you so much for writing you. It was a pleasure. Thank you. Thank you.