Your Undivided Attention - Why Superintelligence Won’t Cure Cancer

Episode Date: April 30, 2026

One of the most common arguments you hear from company executives racing to develop super-intelligent AI is that it will cure cancer. It’s an incredibly powerful and seductive promise.  If superintelligent AI really can cure cancer, then anyone who stands in the way of it, anyone who wants to slow it down — even because of its serious risks — is essentially letting people die. In fact, the biggest risk would be going too slowly. But what if a superintelligent AI isn’t actually capable of solving cancer in the way it's been described? What if we're being sold a false promise to justify a dangerous race? That’s exactly what our guest this week argues is happening. Dr. Emilia Javorsky is a physician, public health researcher, and director of the Futures Program at the Future of Life Institute. She's worked across scientific research, clinical trials, tech startups, and AI policy.   Emilia recently wrote a paper titled “How AI Can and Can't Cure Cancer,” in which she argues that the promise of superintelligence curing cancer falls apart under scrutiny.  Emilia lost a parent to cancer, so her criticism of this promise comes from a place of real concern, not cynicism. It also comes from her belief that AI can be really revolutionary for medicine, if we build it the right way. Your Undivided Attention is produced by the Center for Humane Technology. Follow us on X: @HumaneTech_ and subscribe to our Substack.RECOMMENDED MEDIA How AI Can and Can’t Cure Cancer by Emilia JavorskyThe Emperor of All Maladies by Siddhartha Mukherjee RECOMMENDED YUA EPISODES Decoding Our DNA: How AI Supercharges Medical Breakthroughs and Biological Threats with Kevin Esvelt Forever Chemicals, Forever Consequences: What PFAS Teaches Us About AI Big Food, Big Tech and Big AI with Michael MossCLARIFICATIONS: Emilia’s claim that “the doubling rate of medical knowledge has gone from 50 years in the 1950s down to 73 days” comes from an oft-cited 2011 paper from the NIH. However, this paper does not include any methodology for arriving at this claim. Emilia stated that we have yet to cure any complex, chronic disease in humans. However, we have been able to cure Hepatitis C, which is considered a complex infectious disease, and we have managed to effectively cure some types of Leukemia Correction: Tristan incorrectly paraphrased a quote from Charlie Munger about incentives. The actual quote is “The basic rule of incentives is you get what you were owed for. So if you have a dumb incentive system, you get dumb outcomes."     Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

I think this is a false choice we're forced to make quite often in the discourse. It's like we either get our cancer cures and then we have to take on the risks of unemployment, extinction, X, Y, and Z. There's another path here where we get our cancer cures and we don't take that on, right? Like there's a different option on the table that I think often kind of gets pushed aside. Hey, everyone, and welcome to your invited attention. This is Tristan Harris. One of the most common arguments you hear from people racing to super intelligent AI is that it'll be able to cure cancer.

It's incredibly powerful. It will do all these wonderful things like that will help us cure cancer. It may help us to eradicate tropical diseases. We are working to build tools that one day can help us make new discoveries and address some of humanity's biggest challenges, like climate change and curing cancer. I think one day maybe we can cure all disease with the help of AI. not help with cancer, not improve treatment, but cure cancer. Now, that's obviously an incredibly powerful and seductive promise.

And everybody listening to this right now likely knows someone who's died of cancer. It kills almost 10 million people per year. I lost my mother to cancer in 2018. This is a very personal topic. And that's why this promise is so potent and why we need to examine it. Because of the technology really can cure cancer, than anyone who stands in the way of it, anyone who wants to slow it down,

even because of the serious risks, is essentially letting people die. This is the idea of the invisible graveyard you hear about from the accelerationist. Think of all the people that we might be able to save by racing forward. In fact, the biggest risk is not going fast enough, they argue. But what if it isn't actually capable

of solving cancer in the way it's been described? What if we're being sold a false promise to justify a dangerous race? and just to make a handful of people incredibly wealthy and powerful and avoid regulation. Our guest today argues that this is some of what is happening. Dr. Amelia Javorski is a physician, public health researcher, and director of the Futures Program at the Future of Life Institute. She's worked across scientific research, clinical trials, tech startups, and AI policy.

And she recently wrote a paper called How AI Can and Can't Cure Cancer, in which she argues that the promise of superintelligence curing cancer falls apart under script. and that we can't use this false promise to justify the peril that we're currently facing. We're going to link to that in the show notes. This is a deeply personal conversation for both of us. Emilia also lost a parent to cancer. So hear her criticism of this promise as coming from a place of real concern and not just cynicism. It also comes from the belief that AI can be really revolutionary for medicine,

but not in the way we're building it today. So Amelia, welcome to your undivided attention. Thank you so much for having me, Tristan. So first I'll just say, Amelia and I are friends, and she's an incredible ally in this work. We were at the South by Southwest conference really this year, and you talked about the work that you've been doing on AI and cancer, and I was struck by how personal this is for you, since you lost a parent to cancer. Before we can get into your arguments, can you just talk about your experience of that and how it shaped your thinking? Yeah. So when we hear the promise of AI and cancer, it triggers in all of us a personal experience because all of our lives, have been touched by some sort of loss to cancer. And for me, it was deeply personal that I lost my

father to cancer. I lost my father to cancer, you know, over a decade ago. And when I sat down to write this essay and really think about examining the ASI to cure cancer promise, I went back through the medical literature to see how much progress had been made since the time my father passed to where we are today. And the reality is the survival rate is almost exactly. the same as it was over a decade ago. And so the problem of progress in oncology is probably one of the most urgent of our time and one of the most noble things we can deploy capital in service of solving and our talent in service of solving. But I think it's really important to examine whether putting that capital into a race to superintelligence is the best way to save the lives of our loved ones.

Yeah, I mean, having lost my own mother in 2018, Aza co-hosted this podcast, he lost his father to pancreatic cancer. I just want to establish, I think, goes without saying, anybody who has this in their family with a loved one wants to accelerate anything that will save their life, anything that has a chance. And yet there's so many issues that you find out about, you know, there's all these new things that are coming to market,

but then they're not actually even available to your spouse, your loved one when they get this. And so I think one of the things we're going to talk about is, you know, there are many ways technology can help advance biomedical science, but is the specific path of building super intelligent AI that is reasoning with a massive data center across everything? Is that the specific vehicle that'll get us there? And you wrote this essay that I really want to encourage people to check out. Why did you write this essay? What was the kind of motivating purpose here? Yeah. So in addition to the personal experience with, loss in cancer. Having a background as a clinician and having gone to medical school,

you also experience it from the other side, the frustration of providers about how limited of a toolkit they have to actually help people. And encountering it sort of over and over again day in and day out, having to deliver news of loss to families. And so for me, this is deeply personal to me both in terms of my life but also in terms of my career. And also in sort of a parallel hat that I've worn in this AI policy conversation for the better part of a decade now, have seen these two worlds, which is biomedical innovation and the ASI race. And to me, hearing over and over and over again, AI is going to cure cancer. Like we must build ASI because it's going to cure cancer. And yet that promise going entirely unexamined, just kind of being taken at face

value, that if we want to save lives and if we want to cure cancer, that this is the thing that we have to do. And I strongly believe that that is not actually the best way to start saving lives today. And for listeners, ASI is artificial superintelligence, which is an AI system that is more intelligent and powerful than all of humanity's intelligence combined. You are not anti-AI. for cancer, you just think there's a totally different approach we could be taking. And first, we have to understand the problems with our current approach and then give people the hope that there actually is a totally different way we could be applying AI that would actually get us to the outcomes that we're all looking for as opposed to false promises to sell

investors and keep pumping up your data centers. Yes, I'm incredibly excited about the potential for AI and this general moment that we're in for progress in oncology. I remain really hopeful and excited about what the future has ahead. For me, that's sort of three ingredients, which is one, supporting all of the AI tools that are being developed in specific areas of oncology that are making things go faster, cheaper, better, unlocking new capabilities. The exciting research that's happening in biology, so there's really exciting science that's happening that's sort of discovering totally new ways to think about the problem. And so figuring out how do we support those scientists doing that good work

and getting their discoveries out of the lab and into the clinic faster. And then thinking about how can we actually realign and redesign the system that we had and identify where the parts are in the current system that are either holding up progress or even taking it in the other direction, right? And so I think that kind of tripartite approach is one that makes us well-suited to make a lot of progress in oncology in the next decade. But part of the reason I wrote this essay is because I'm worried that the current approach isn't doing those key things that we need to actually move the needle

and that our resources are being placed in areas that are not going to deliver the benefits that we hope for. Yeah, I'm just brought back to my memory of going to Senator Chuck Schumer's AI Insight Forum. It was this historic event where they invited all the CEOs, Elon, Jensen, Mark Zuckerberg, Sam Altman, Bill Gates, all in one room. And A's and I were there with a handful of civil society groups. And I remember talking to some of the Senate staff up beforehand before we went up there. And one of the things you heard from, I think the Senator Mike Rounds,

was just because they had family with cancer, that there's this thing, you and I've talked about it, that people kind of turn these puppy dog eyes of like, but it could cure cancer. And there's this hope of like, well, that literally would just eclipse any other reason to slow down, right? If it's life and death, we do anything to save that person. Let's just steal man for a second.

So why would they say it could cure cancer? I mean, it seems intuitive. AI understands language, patterns in language. So just the same way you can understand patterns in text and generate chat GPT essays. You could understand patterns in DNA and understand immunoncology. Let's just see a moment for a second why people believe. Because it's not like it's wrong, but it's seductively false and kind of an optical illusion, almost like a magic trick.

So we hear a lot about the ways that AI is helping advance progress in medicine in kind of the here and now, which it is. and it is going to be instrumental in doing so, but it's not chat GPT that is unlocking that progress. It's scientists building bespoke models off of highly curated data sets to actually solve a specific problem, whether that be drug design

or whether that be predicting toxicity, the list goes on. So I think one piece to start is the AI will cure cancer promise, surfs a little bit on the AI progress that's already being made with tools and smaller models and kind of bundling that, as evidence as to why ASI will help solve the problem because if the AI could get so much better, imagine how much better results we could be getting, right? So that could be an image of a mammogram for breast cancer or it could be blood test results, right? And then getting sufficient measurement

of that phenomenon into a data set. And so can we generate a data set that captures all of the variability that we see in when we measure that phenomenon that's sufficiently representative. And then can we apply intelligence to unlock insights that previously humans did not see or were unable to do at scale? And so in medicine, we're seeing this happen across many domains where we have good data. So when we talk about early detection of breast cancer, AI is amazing at that because we have lots of great images that are high quality and curated by human radiologists of what is and what isn't breast cancer. So in that domain, AI does very well when it has the data to work with

and that data is sufficiently representative of the phenomenon that we would like to study. Right, so we have lots of mammograms, we have lots of results that confirm whether that mammogram did have a cancer or not, which means you can train a more and more accurate model. That one's solid. Correct. So what are some of the other narrow AI applications that words helping? One area we're hearing a lot about is AI being able to predict whether a new drug is going to be toxic or non-toxic.

And that's because we have extensive libraries of existing compounds that we know whether or not those caused problems or adverse events when they were put into people. So the AI can take a look at a new compound and say, okay, based on all of my knowledge of everything else that's either safe or unsafe, what do I think this will be? Do I predict this to be more likely to be safe or unsafe? And that's called computational toxicology,

and AI is doing a great job at that. we're hearing a lot about AI for drug design, right? Being able to really just lean into the chemistry part of biology, even more so than biology itself, to design new molecules, to design new drugs. So that also is, I'd say, an area that's quite exciting. And then there's clinical AI. So AIs that are actually being used in the operating room

when they're excising tumors, right, and trying to figure out if they have a margin or not. And that's because there's imaging databases of what a margin looks like. right, that an AI can look at and say, okay, I think we've got it or we haven't gotten it. So I would just highlight those three examples. And each of those are not being developed within large companies. They're all being developed either by small startups or even academic institutions.

Whereas the ASI promise is saying, let's just digest everything. Let's take all knowledge and put it into one big giant model and see what insights it can derive from that model. Right. And so the idea here is the more. and more data we put into this, the more and more capable systems we can make. And one day we'll make a system that is more capable than humans, and then thus will be able to do types of reasoning or types of insights that humans would not really be able to do or discover, and assuming in that set is a cure for cancer.

Right. So this is like if I read not just the entire internet, but all biology textbooks had access to every science lab, had a robot arm doing lots of studies, plus integrating it with the GPT7 trained data center, you know, with Sam Allman's Stargate cluster, that's just combining so much information that it's going to magically find all the needles and all the haystacks. Kind of that vision of ASI finding cures to cancer, right? Correct, yes. What actually is cancer? So this is where the AI to cure cancer piece breaks down is what is cancer and what is a cure?

And those are two actually really fuzzy terms, even for the experts. in the arena. So when we think about cancer in the early days, the way you thought about cancer is like, there's some cell, it gets a mutation, it goes rogue, and it makes a tumor, right? And that was the original sort of simplistic understanding of cancer. And as our understanding of oncology has gone on, there's been these papers that have come out called the hallmarks of cancer. And as we find new biology and new ways to measure things, we're getting further and further away from that simple explanation of one cell with a mutation that goes rogue and makes a tumor. It's actually a much more complex disease involving the immune system and the blood supply,

and even within one tumor, different things are happening in different parts of that tumor. And so the story of cancer has been, as we've pushed science forward, we've uncovered more and more complexity to the disease, not less. So there hasn't been sort of a march towards a simplifying or unifying hypothesis. It's been a march towards an ever more complex. an individualized type of disease. So fundamentally, when we think about the complexity of cancer, it is sort of a shadow self. And there is a book I highly recommend folks read called The Emperor of All Malities that really

delves into this problem of why this is the most complex disease of all, because it is something that is co-evolving with us, it's dynamic, it's complex, and it's highly individualized. So compared to other things like treating the flu or treating high-evolving. blood pressure, right, which are more static biological processes relative to cancer. Like, this is really the big one in terms of complexity. Okay. So let's go back to the promise made by CEOs. You have Dario Amadai from Anthropic who talks about compressing 100 years of biological progress into five to 10 years, like creating what he calls a country of geniuses in a data center that are all dedicated to that.

And that's obviously a really compelling idea. I mean, just to go into that thought experiment, imagine the last 100 years of scientific progress. Just like see that in your mind's eye, all of the things that we got over the last 100 years. Now imagine that coming in the next 10 years, scientifically. That's like magic. This is sort of the science accelerator button.

It's what leads to J.A. Cotra to say, this is why AI is like 24th century technology crashing down on 21st century society. But what is the problem with this argument of 100 years of biological progress? I would say there's three main problems with that argument. The first one is in science, we actually have been accelerating knowledge and intelligence. We have an oversupply of human scientists relative to what we can actually resource in terms of experimentation. So the doubling rate of medical knowledge has gone from 50 years in the 1950s down to 73 days by some estimates. We have an oversupply of scientists relative to number of lab benches and pipettes and people. we can resource. And despite that acceleration and knowledge, we've noticed that therapeutics approved to actually help people have remained markedly flat. We actually haven't made commensurate progress.

So the intelligence that we've gained hasn't really been coupled to actually moving the needle on saving people's lives. This is very interesting because it's like the promises that we just have more intelligence, that intelligence is essentially the bottleneck for why we don't get more progress in biology. But you're saying we did get an explosion of intelligence. in the form of new biological data, the amount of medical data we got, and the number of actual people that are sitting at lab benches, and yet it hasn't resulted in that. So you argue, though, it's not only wrong, it's actually dangerous.

Can you speak to that? Yeah, so there is a danger to waiting and hoping that some future genie is going to solve a problem, which is in some ways the essence of what the ASI promises. It's sit, wait, hold tight, don't do anything in the here and now, In the future, there's going to be a cure for all of these problems. The reality is people are dying today, right? People need solutions today.

We need to actually be unblocking progress and moving the needle today. So there's the temporal piece of this where it's like people who have cancer don't have time to wait on the future, right? Even if that were to be true. The second piece of this that's really important to think about is we don't live in a world of infinite capital. If we lived in a world of infinite resources and we're, we've lived in a world of infinite resources, and one bucket wasn't coming out of another, then there's a different argument to be made. But we're seeing that biotech is at a 10-year low

in terms of venture funding of new ideas. And venture funding is really where you see the new breakthrough, exciting, high-risk types of projects that really can move the needle for patients. We're living in a time where we're reducing our investments in sort of basic science, in science infrastructure, in data collection. And so the essence here is if we're going to take money away from doing the things we know will unblock progress,

then we better be really confident that that is actually the fastest way to save lives. Can you speak to the amount of resources that are currently going into accelerating ASI versus how much is going into, let's say, cancer research? If you look at the amount of money going into building ASI and the infrastructure associated with that, that's an unprecedented amount of money in terms of investment in a technology. In 20, 26 alone, they're looking at $540 billion plus, right? And if we want to compare and contrast that to, let's say, the National Cancer Institute, which is a pretty good barometer of what are we investing in the public in the basic science and understanding and moving the needle in oncology, that's only $7.2 billion, right? So it is a fraction of the amount on

of annual spend that we're spending on actually solving the problem of curing cancer is opposed to an ASI spend. So essentially, we're putting half a trillion dollars into a genie that people think or are selling the idea that it'll magically solve all of our problems from climate change to cancer. Compared to $7.2 billion, 7.2 billion versus half a trillion is the gap. Not just that we're not making progress in the cancer side, we're actually robbing billions of dollars away. Instead of getting 10 years of scientific progress, it's almost like we're losing 10 years of scientific progress because all the money is going towards this genie rather than going towards things that would actually

unlock progress. I'm just wondering, though, if listeners would, at this point in the conversation, believe that the genie won't actually address these things? Because all of what we're saying depends on whether that is true or not. So let's break this down for listeners. So I think the AI for science promise gets all kind of bundled into one. And can't who gets put into that, along with physics and along with manufacturing and along with chemistry. But it's really important to break those out because physics and biology are very different phenomenon. And physics is a domain where in math is similarly where we're seeing this correlation between capabilities and progress in those sciences where we have basic rules. Like we know the laws

of physics, we know the rules of physics, we know the rules of math. But for biology, there are no first principles to work with. There are no actual rules. rules of the road to feed to an AI to learn and to model from and to analyze. And people say, well, you have physics. Everything's physics at the end of the day, right? You have physics, you have everything. Well, that's simply not true in biology. And it's infeasible, even using classical physics, never mind quantum physics, to simulate even like a week or a minute of a human's biology if you covered the entire Earth in GPUs. Right. So you're not saying that AI couldn't massively accelerate physics or math.

Correct. So we could hit a button, and it's already true, by the way, just for listeners, Paul Erditch, who was a mathematician in the 1940s, he laid out these math problems in the 70s that had never been solved. And just recently in the last few months, AI has actually made progress and solved those math problems. It's now winning gold in the International Math Olympiad. It is generating new physics.

So you could actually, just to put listeners through this, using the raw rules of physics that we know, you could re-derive everything up to quantum physics with just an AI doing that. Like, that's mind-blowing. So you're endorsing that AI could do that, but you're making the distinction that in biology, you have these emergent effects. It's the complex adaptive nature of biology that's different from other systems that makes it so hard to model. And then you gave a quote in there of how much computation it would take to simulate. You said it was what one week or one minute of the human body would take more than the GPUs on planet Earth and more than the time in the universe? Correct, yes. That's crazy. Okay, let's take the example of COVID. So, you know,

we had this COVID vaccine. Basically, there was a Operation Warp Speed to figure out how could we take something that was a new disease, a new virus, and we did develop something with super fast deployment. And I think ASI has thought it to be Operation Warp Speed for everything. Like in nine months, we could have cures for everything, because that's what this magic genie in a box is going to do. Could you distinguish why was that possible with COVID that's not? possible with cancer. Yeah. So COVID is used as the sort of case study of how could we prevent or cure something. And I think it's worth taking a step back and having the perspective that we've actually yet to cure any complex chronic disease in humans. So we've done a really

great job with infectious diseases, which are not actually targeting the human. It's targeting the bacteria or the virus. And we're making a lot of progress with some genetic diseases where it's of a single bug in the code is causing the disease. But diseases that are complex, we still have yet to cure one. So never mind cancer, but pick anything, diabetes, Alzheimer's disease. We have some ways to manage them, but we actually haven't cured them. So the COVID example is not a great example for prevention and cure. It's also not a great example of drug development in general. So when we think about infectious disease, that is a very easy study to run a clinical trial on. Because in order to determine whether something new, be it a vaccine or a drug works, you have to figure out, does it actually work in people?

Right. And so when you're dealing with something like COVID, from when you get exposed to when you show symptoms, you're talking about seven to 10 days, right? That's very different than something like cancer or Alzheimer's disease, where these are processes that are really decades long from when they sort of start to finish in the disease. And most of the trials in those domains, you really need to follow people for five or six years to actually understand. Is this moving the needle in a significant way to solve this in patients? And the third thing about COVID is that story of like, oh, well, we had COVID and science went and guns were blazing and we got there in less than a year, ignores the fact that the science had already started 10 years earlier, right? So scientists were hard at work at developing MRNA technology for over. a decade before COVID started and doing the safety testing and doing the regulatory submissions.

And so when COVID hit, there was already a decade of science and investigation and inquiry to build on to actually take that forward quickly. So maybe just to sort of summarize, COVID had unique advantages because there was one easy recruitment from the general population because it was a shutdown in the whole world. People would actually want to volunteer for this. Two, clear rapid outcomes. you could test whether something worked in weeks, not years, compared to cancer, which requires years of follow-up,

harder recruitment, and the disease also is heterogeneous. You have so many different variations of the disease, whereas COVID is much more similar. Yeah. So what makes Alpha-Fold difference? So alpha-fold people will remember is what I think Demis Hasav has got the Nobel Prize for because it accelerated what, like, decades of research that would have taken a single PhD, their whole PhD,

to get one protein, and now we got, like hundreds of millions of them or something like that. What distinguishes AI that's accelerating that and protein folding versus the broader cures to cancer? So the alpha-fold story is the poster child of AI for science and AI in biology is evidenced by it being an incredibly significant breakthrough to solve protein folding, something that has stumped humans for decades.

But as much as it is an AI story, it is a data story. And that is the piece that I think often gets lost. It's thought of as an AI breakthrough, but what actually enabled intelligence to unlock insights. And that's where we find the story of the protein data bank. So this was a database curated by scientists all over the world, over decades, that as they started to figure out what the structure of a protein was, and you have your sequence and your structure, they started uploading all of those images, all of that data of what the structure of the protein looked like and its sequence.

And so when you went to solve the problem and say, like, where could, you know, increasing AI capabilities or my new AI techniques that I'm playing with to develop new models be significant? It's areas where you have this, right? I want to understand how a sequence results in a structure. And then there's a database where there's curated sequences and structures over decades. And before we move on, I think we should explain what protein folding is and what it has to do with, you know, medical. interventions in general. Can you just explain that protein folding? So one of the reasons that protein folding is so significant in terms of the science, what does that actually mean for patients, is when we design new drugs or develop new drugs, they're designed to target a specific protein in the body.

And so think of it a little bit like a lock and a key, right? If you want to go home and put your key into a lock, it has to be open and the key has to be the right size and fit there and open up. And so we don't really know when we look at new targets, whether that keyhole is blocked, whether it's open, whether it's the right shape and size. And that's what protein folding and solving that problem has enabled us to do, is to understand in advance, okay, I have the key and I can get to that lock. So the piece, I think, of the alpha fold story that gets lost is like, yes, there were new AI techniques and models built specifically to solve that problem.

But what enabled AI to solve that problem was having that data, those two pieces of the puzzle that it needed to actually derive, well, what is the relationship between these two things, sequence and structure? So we had the right data sets that we could actually find the patterns, whereas with cancer, you have someone whose disease is progressing over a decade, and we don't have all the data of what's happening at each interim step for every patient available in some database to look at everything we were doing and changing their health habits, what they were eating differently, what drugs they were taking.

So we don't have that basis, that library in the same way that we did for protein. folding. Correct. And I wish we were in that world, Tristan, where that was the data standard of where the gap was and what we needed. But it's so much more crude than that. And I think that's something people don't realize. Like, we don't even have a national sort of data comments of cancer genetics and imaging data and things that scientists could learn from that's interoperable and people can work with. Just the simple things that we already collect in clinic. And I think this is a piece that Silicon Valley gets wrong about medicine too, is really overestimating the data that we have and the strength of that data in representing what's actually happening in a patient.

Amelia, you said something in other interviews and talk about how there's a difference between curing cancer readily in mice versus in humans. What is that? Yeah. So it is probably the best time in human history to be a mouse in the sense that we can cure cancer in mice. We've done a really great job of that. the years, and we have a lot of drugs that are able to do that. The problem is when we take those

things that look good in mice, it looks like it's curing the cancer, it looks like it's going to be safe, this looks like it can actually get where it needs to go in the body, and then test them in humans, it falls apart, and they don't work. And 90 plus percent of the things that are going to cure cancer or save the life of a mouse are not actually going to move the needle at all in a human being. And so that's the piece that I think is a missing link, which is from what we know, in the lab bench, does it actually work in the bedside, right? Does it work for the patient? And that gap is something that we've yet to bridge.

So something like a cure for cancer, I think you've just shown, is not constrained by intelligence as the core bottleneck. But it does seem definitely constrained by systems. So what are the ways that our human systems, our FDA approval processes, or intellectual property laws, or grant making, or funding, that are getting in the way of treating disease that superintelligence won't be able to get?

around? So I think a fundamental assumption most people have is that if a drug looks promising to treat a disease, that that means we'll get it to patients and it'll make it through the FDA and it'll make it to be able to actually help people. But I think it's important to look at the graveyard of things that have already failed due to misaligned incentives in our current system. So I think the really great place to look at this is in antibiotics. So there's been many companies that have discovered new antibiotics, including ones that have been AI discovered, that look really promising. And the data looks really good. And you start to take them into clinical trials. And the clinical trials look really good. And you're like, this is so exciting. Like, this is working. We have a new

therapy. There's a huge unmet need. Problem is, it doesn't meet the financial requirements to actually make it over the finish line and go through the FDA. And so antibiotics are something you only take once, right? You're not taking them every day. You take them when you get an infection. and by nature of the antibiotic resistance problem, like you don't want to use them too much, right? You want to use them sparingly. You want to use the new stuff only when you have to, only when the other stuff has failed.

And so what that means is there isn't really a viable business model, right? This is not going to be a billion dollar a year project. Then why do I bother sort of take it through the FDA? Because FDA processes take billions of dollars to make it through phase one, phase two, phase three, just for people to track that. I mean, this aligns with something ASEN and I have said that it's really, really AI is just forcing us to confront the ways that our systems have not been aligned for this.

People talk about aligning AI, but can you have aligned AI inside of a misaligned system? Can you have advancements in biology inside of a system that has for many different reasons, corruption and incentives and revolving doors and poor FDA regulatory approval processes? Because as you said, we're about to get an explosion of new molecules and new drugs. But if we don't have a process they can deal with, we're also about to flood that system and then jam up the gears. because now there's so much more trying to make it through a system that also wasn't terribly working perfectly well at the beginning. Yeah, there's two things I want to say here.

So on the AI side, we're rapidly scaling and flooding the system with new molecules that we want to test. But we can't scale people. We can't scale the number of patients in a clinical trial. We can't scale the number of tumor specimens that come from a patient to test, right? And so that's not actually a scalable model. And you really need to understand how are we going to allocate this precious resource of patients, of samples that we have that are actually limited and we can't create more of. And similarly with these diseases that take time to actually test out whether something is working or not,

we can't compress time. You can't scale time. You can't make a pregnancy go faster. There's certain fundamental things in biology that just need to. to take time to understand on that iteration, is this working or not working? Even when we flood the system, we have to examine what kind of system are we introducing this technology into and what does it incentivize? And while we call it the health care system, it's actually not a system where the incentives are aligned with keeping people healthy or preventing disease. It's a system where you make

more money as a provider or a hospital based on more care that you give. That's totally decoupled if that care is effective or not, or what the outcomes are. It's just the more care you give, the more money you get. Right. Just to link it back to Cht's frames, we always reference Charlie Munger. If you show me the incentive, I'll show you the outcome.

And while humans wield technology, incentives wield humans. And I see your core warning is that if you deploy AI optimization into a system without fixing the incentives of that system, you're just going to supercharge the misalignments of that system. And to give examples of this, like insurers optimize for denying claims, right? They make money when they don't pay out. And so they always find a way to sort of make it difficult

in subtle, subtle ways to just like have you settle for half the amount of your insurance claim and just like not have to fight back the itemized list. You know, United Healthcare deployed an AI system to process claims that was reportedly denying them at a massive elevated rate because of the AI system. This is somewhat what you're talking about. You know, hospitals optimize for volume, not for outcomes.

You know, under a fee-for-service model, hospitals and doctors get paid for delivering more care, not better care, more procedures, more tests, more visits, regardless of whether the patients get healthier. And so across the board, if we really want a better world, AI should be focused on how do we change the bad incentives of all these systems, because that is what is going to unleash the better world that we really want to get to. And this is where I think the opportunity and the peril for AI and the healthcare system

really exists, because there are ways that we could leverage these AI tools that we have today to complete. redesign the system and redesign the structures and enable new ways of incentivizing the things that we actually want, right? So I think the example of United in healthcare, like, there's so many middlemen in health care that are not actually the person taking care of the patient, right? And if you look at from health insurers to pharmaceutical benefit managers, the administrative waste estimated in health care is somewhere between 30 or 40 percent by some estimates. It's up there, right? You know, this is a national crisis, our health care crisis at the moment and our

spending on health care. Like, why are we spending all that money where AI could do those administrative tasks and that money could be routed back to actually taking care of people and giving them care or lowering the costs of care? So I think there are ways we can reimagine health care with AI, with a better system, with better incentives that get us where we actually want to go. But we have to be proactive and mindful about that. We can't just, you know, let AI lose. in the world that we have because we're just going to get more of the things that we already have, which we know in health care is not things that we want. Yeah.

Totally. Okay. So we've just sort of established that intelligence isn't the bottleneck, that cancer is a different kind of disease. It's not receptive in the same way that accelerating physics or specific molecules for infectious diseases interventions. Now we should get to, so how would we change these perverse incentives that we have just been outlining?

And what will we do differently in our investments in AI rather than build? the genie that won't actually develop the cancer drugs? I think step one in making progress in this domain is the data piece of the story. Measurement in data. How do we better measure our biology? How do we better capture and understand what cancer is, what's happening in an individual? And collecting that data with the state of the science that we have at scale. And an example that I think is really prolific has been the work that has happened in the United

Kingdom with their UK. BioBank Project. So this was a project where they followed 500,000 people and they're still going over 20 plus years using actual sort of modern state-of-the-art measurement techniques. So this isn't the simple blood test when you go to your doctor and you get the paper readout and there's like 30 things on it. This is measuring thousands of things in the body. This is taking all kinds of imaging of the body. And we're starting to see these headlines like AI can predict Alzheimer's 10 years earlier, right? And that is actually a story of just normal AI machine learning methods applied to this prolific data set that has come out and required decades of investment.

And actually measuring just the baseline. Like, what does healthy look like? Like, we still don't actually know that question because all of our data is when people present to a system that are sick, right? And so I think that is a great example of, like, public infrastructure investment in data collection that is clinically relevant to help us. bridge that gap of like the mouse to the human, right? How do we know if something works in a human being? How can we better predict that? It's going to start with measuring and studying people at the end of the day. I think there's the piece of AI investments in general, and I would argue we should be investing a lot more in AI just in medicine and in tool development. And there are so

many areas that this is really exciting for AI to discover new biomarkers, right? New things in your blood that you can start to see. Well, is someone. working or not or is a surrogate of a disease that can help accelerate therapeutic development, helping to detect things earlier. We use the AI in mammography example. And so those are all like AI tools that need to be built that are going to actually like unblock progress in oncology that we're just not investing in because that money is going into building the ASI promise.

So we could be building tools that take the cost of getting through the FDA processes from billions of dollars down to even just hundreds of millions or something like that, therefore allowing many more smaller, medium-sized startups and businesses to even make it through that process. We could be building, you know, data commons that collect more brain scans earlier for the early detection of Alzheimer's. We could be doing more toxicity prediction. I heard you say earlier as well of, you know, are these drugs going to create more toxicity or not? More pre-screening, more prediction. There's a bunch of places where narrow AI can actually really, really help cancer.

So this whole conversation I want people to hear, you're not anti-AI. you're not anti-technology. You're actually for applying it in a totally different way that'll actually achieve outcomes as opposed to supercharging bad incentives that lead to bad outcomes. Yeah, fundamentally, I'm super bullish on the promise of AI and oncology and medicine in general.

It's just the right kind of AI development that's targeted to actually solving the problems and unblocking the things that are holding up our ability to move science forward. I would also add to that landscape, Tristan, on the AI on the manufacturing side of things. So, like, how do we actually make drugs at scale? How do we do quality control?

Looking at something like CAR-T therapy, which is a cell-based therapy to help treat cancers in patients that's very individualized. And because it's individualized, it's very expensive to make. Right now, it's upwards of $400,000 to access the therapy. Now, if we could use AI to help us find out cheaper ways to manufacture that, bring down the cost, be able to make the drug in more places closer to the patient. Now, way more patients can actually access this because it's no longer cost prohibitive.

So you can use AI in that way to democratize access through bringing down the costs of manufacturing a new drug. Yeah. And so basically you're talking about just bringing down the cost of individualized treatments, which are currently very expensive because you have to make one per person that you're trying to treat. I'll just note that Josh, our podcast producer's father, was saved by CART therapy. And so everybody who has someone in their family, my mother almost was thinking about using

R-T therapy but did not, you know, where that cost is prohibitive. Imagine a world we're just trying to bring down the cost of this thing, rather than building a genie that's not actually going to uncover these brand-new cancer treatments. I just really feel like there's this mind-upending, sort of turning the world upside down, framing to everything that you're saying. Like, it should feel really crazy to people that we're currently putting half a trillion dollars into the genie that's actually not going to give the cancer treatments. It's crazy.

This should feel just insane. And if we just redirected even a third of that investment to accelerating all these other applications and all this other updates to the governance and regulatory design and data commonses and narrow AI applications and better harnessing the existing geniuses that are not in the data center

that are sitting over abundantly in labs at universities without access to the tools, like there's a much better, more beautiful world that our hearts know actually is possible if we were just applying this technology, and the regulatory interventions very differently. So to me, this conversation is actually very optimistic, but it's optimistic by puncturing a hole in this false promise

that is being sold to us to really shield the companies from essentially any kind of regulation or slowing them down to do this other thing they want to do, which is build this ring of power and kind of own the world and build a god and make trillions of dollars from AGI. Yeah, absolutely. I mean, it's absurd to me, the situation that we're in, that there's so much we could be doing

that we are not actually doing, and we're doing all of the wrong things, right? And investing unprecedented sums of money into the wrong things. Like, if we are actually serious, the thing we want to do with AI is to cure cancer, and the goal is curing cancer. We need to say, what is the fastest way to achieve that goal?

Where do we put our dollars to get that goal? And there's so many places we could put our dollars that get us there a lot faster. You know, we've been talking about whether superintelligence would actually be a genie that would solve cancer. Well, let's talk about whether superintelligence is actually controllable or safe when you have it basically already demonstrating all the hell 9,000 behaviors and the early warning signs and warning shots of deception,

hacking computer systems, not caring about the longevity of humanity, disobeying shutdown commands. Why don't we just make that thing a million times more powerful? Does that sound like a good idea? We haven't even talked about the other side of the balance sheet of whether any of this is worth it. Yeah, I was just going to see what you're done. I think there's like a framing to this in the conversation, which is a lot of what we're discussing is whether ASI or a different approach that is a more AI tools and systems redesign approach is the most effective way to cure cancer. And that's just looking at the upside piece of it. But there's also a requirement to complete the risk benefit analysis and say, like,

what are the benefits of these potential technologies, but also what are the risks? And that's where you see a lot more divergence between these two perspectives, because we know there's a lot of systemic risk. risks with ASI development, right? With the tools and the system redesign approach, there aren't those risks, those systemic risks. And so what you end up with is being able to sort of have your cake and eat it too, where you get the benefits of AI in progress without taking on the risks. And I think this is a false choice we're forced to make quite often in the discourse. It's like we either get our cancer cures and then we have to take on the risks of unemployment,

extinction, X, Y, and Z. There's another path here where we get our cancer, and we don't take that on, right? Like, there's a different option on the table that I think often kind of gets pushed aside. This is actually just such an obvious other path. This is the narrow path. We can have narrow AI systems that are narrow and specific and tool-based, not general, inscrutable, and controllable systems that are way more powerful than us that carry these risks unnecessarily.

We don't have to do that. So, Amelia, thank you so much for coming. Thank you, guys. Your undivided attention is produced by the Center for Human, We're a nonprofit working to catalyze a humane future. Our senior producer is Julia Scott. Josh Lash is our researcher and producer. And our executive producer is Sasha Fegan.

Mixing on this episode by Jeff Sudaken, and original music by Ryan and Hayes Holiday. And a special thanks to the whole Center for Humane Technology team for making this show possible. You can find transcripts from our interviews, bonus content on our substack, and much more at humanetech.com. And if you liked this episode,

We'd be truly grateful if you could rate us on Apple Podcasts or Spotify. It really does make a difference in helping others join this movement for a more humane future. And if you made it all the way here, let me give one more thank you to you for giving us your undivided attention.