a16z Podcast - a16z Podcast: AI and Your Doctor, Today and Tomorrow

Episode Date: June 13, 2019

with Eric Topol (@EricTopol) and Vijay Pande (@vijaypande) Artificial intelligence is coming to the doctor’s office. In this episode, Dr. Eric Topol, cardiologist and chair of innovative medicine at Scripps Research, and a16z’s general partner on the Bio Fund Vijay Pande, have a conversation around Topol’s new book, Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again. What is the impact AI will have on how your doctor engages with you? On the nature of the doctor's visit as a whole? How will AI impact not just doctor-patient interactions, but diagnosis, prevention, prediction, medical education, and everything in between? Topol and Pande discuss how AI’s capabilities for deep phenotyping will shift our thinking from population health to understanding the medical health essence of you, how the industry might respond and the challenges in integrating and introducing the technology into today’s system—and ultimately, what that the doctor’s visit of the future might look like.

Hi and welcome to the A16Z podcast. I'm Hannah. This episode is all about how artificial intelligence is coming to the doctor's office, how it will impact the nature of doctor-patient interactions, diagnosis, prevention, prediction, and everything in between. The conversation between A16Z's general partner Vijay Ponday and Dr. Eric Topal, cardiologist and chair of innovative medicine at Scripps research, is based on Topal's book, Deep Medicine, and touches on everything from how AI's deep phenotyping can shift our thinking from population health to understanding the medical health essence of you, how the industry might respond, the challenges in integrating and introducing the technology into today's system, what the doctor's visit of the future might really look like and ultimately how AI can make

healthcare more human. Before we talk about technology and we talk about all the things that are changing the world or making huge impacts, it's interesting to think like, what should a doctor be doing? And how do you see that? Yeah, that's really what I was, pondering, and I really did this deep look into AI. I actually didn't expect it to be this back-to-the-future story. But in many ways, I think it turns out that as we go forward, particularly in a longer-term view, the ability to outsource so many things with help from AI and machines, I think is going to get us back. It could. It could.

That's the big if to where we were back in the 70s and before. What was better then was that doctors were spending much more time with us, right? Exactly, that gift of time, the human side, which is the center of medicine, that's been lost. The big business of health care and all of its components like electronic records and relative value units and all this stuff basically has sucked out any sense of intimacy and time. And it's also accompanied by lots of errors. But of course, it's not a gimmie because administrators want more efficiency, more productivity. I think you put this on Twitter where it's like some kid drew like a drawing

of going to the doctor and the picture was the doctor with their back turned working on a computer and that is what happens too much but then but yet we're talking about technology coming in so how does this all work out that more technology means less computer? Well I think that is a kind of fundamental

of the problem of doctors not even making eye contact and as a child to draw that picture how unnerving that was a hurt, her trip to the pediatrician, natural language processing can actually liberate from keyboards. And so it's already being done in some clinics and even in the UK in emergency rooms. And so if we keep that up and build on that, we can eliminate that whole distraction, doctors and nurses and clinicians being data clerks. I mean, this is ridiculous.

So the fact that voice recognition is just moving so fast in terms of accuracy and speed is really encouraging. Alexa is a very basic version of voice recognition, but you're talking about something much more sophisticated now, something where they're actually doing NLP, they're doing transcriptions, the doctors and have to take notes. If you were more sophisticated, you could put an ontology onto this such that you're not just getting like a transcript of what's going on, but that you have very machine learning friendly, organized data.

Exactly. So the notes that are synthesized from the conversation are far better than the notes that you would get an Epic or CERNOR where 80% are cut and pasted and they're error-laden. So, I mean, just as we Google AI published in JAMA of their experience, I think it's really going much faster because the accuracy of the transcription and the synthesized note is far better than what we have today. And it exceeds professional medical transcriptionist in terms of accuracy. Yeah, and so I'm imagining like what the doctor visits like then.

And so we've got maybe NLP, so the doctor doesn't have to be transcribing and not interacting with Epic. Who knows what's in the back end, but doesn't even matter anymore. Right. Presumably you've got like billing and all that headache for, you know, like a PCP, all that's a huge headache. Oh, that's all part of that conversation because he say, well, you know, Mr. Jones, we're going to have you have the lab tests for such and such. And then we're going to, you know, get this scan and it's all done through the conversation. We could bring in other technologies, right? We've thought about, you know, just how imaging or other type of diagnosis comes in.

And we've seen all these cool things about how machine learning can improve this. But then also it's funny because at the same point, and you bring this up in the book, that over-diagnosing also can be very difficult. Like it was very stunning where you talked about how the incidence of thyroid cancer is going up, but the mortality's went flat. Exactly. And so I guess the challenge will be is how can we bring these technologies, technologies in so doctors can do the things they should be doing, not the things they shouldn't be doing.

Right. Well, I think there is getting arms around a person's data, this whole deep phenotyping. So no human could actually integrate all this data, not only the whole electronic record, which all too often is incomplete, but also, you know, pulling together sensor data, genomic data, gut microbiome, all the things that you'd want to be able to come. up with not only better diagnoses, but also the better strategy for prevention or treatment. So I think what's going to make life easier for both doctors and for the patients is having that data fully processed and distilled. In the book, I tell the story about my knee replacement and how it was a total fiasco. Part of that was because my orthopedist who did the surgery wasn't in touch with my congenital condition. And so that whole whole. hopefully is going to be something we can transcend in the future.

And this is one thing that computers can do very well as logistics and coordination. There's tons of cases where you might, like thyroid cancer, we're just talking about maybe you have to bring in an endocrinologist in addition to an oncologist. And it's shocking that often there's no discussion there, there's no communication. But yet the challenge in my mind is how does computer magically know things that we can't do right now? Yeah, well, that's, I think, where the complementarity, the synergy between machines and people is so ideal, because we just have early satiety with data, whereas deep learning has insatiable appetite. And so that contrast. But we have, as doctors and humans, we have just great contextual abilities, the judgment, the wisdom experience, and just the features.

and just the features that can basically build on that machine processing because we don't ever want to trust an algorithm for a serious matter. But if it teased it up and we have oversight and we fit it into that person's story, that I think is the best of both worlds. Well, here I'm really curious because what is the ground truth? Because generally we don't trust an individual person as like knowing everything either. Right. So the ground truth would be like a second opinion or a third opinion or a fourth opinion or even like, you know, a board to look at something. And that would be what I think most people would view as the ground truth. The ground truth when it's applied to training an algorithm, of course, is knowing that it is the real deal, that it is really true. And I think a great example of that is in radiology. Because radiologists have a false negative rate of 32%. Oh, pulse negative.

And that's the basis for most of the litigation in radiology, which is over the course of a career, a third of radiologists get sued, mostly because they miss something. So what you have are algorithms with ground truths that are trained on hundreds of thousands of scans so that whether it's a chest x-ray, a CT scan, or MRI, whatever it is, that it's not going to miss stuff.

But then, of course, you've got that over-read by the trained radiologists. So there are, you know, I think that's an example of how we can use AI to really rev up accuracy. Somebody is going to need to interpret the algorithms or sort of be on top in that, in a sense, the doctor is freed from the stuff the doctor shouldn't be doing, like the BS accounting or the typing or all these things. And that's not the best use of doctor's time. And it's funny because it seems like the best use of doctor's time is in understanding what these tests would mean, whether it be an A. test or cholesterol or whatever and in how to communicate with the patient. Yes.

And so that seems like a theme running through the book. So for the first half, in terms of the understanding, what do you think that's going to look like? I mean, one of the things I've been constantly wondering about is whether there'll be a new medical specialty. Like, you know, because you don't have radiology without like CT or X-rays, right? So presumably you don't have radiology 100 years ago. Do you have AIology or something like that?

Saurab Jah, who is a radiologist at Penn. he and I penned a JAMA editorial about the information specialist, radiologists and pathologists, their foundation is reviewing patterns and information. But what's interesting is this is an opportunity for them to connect with patients because they don't, you know, right now, radiologists never sees a patient. Radiologists live in the basement.

The pathologists look at the slides, that group of pathologists. But they actually want to interact with patients, and they have this unique insight because they're like the honest brokers. They don't want to do a surgery. They want to give you their expertise. And so I think that's what we're going to see, a pretty substantial change. And as you're touched on this new specialty, it'll look different than the way it is today. In that case, it almost seems like everybody is better off.

The doctor is better off because the pathologist is not just looking at slides, but actually he's dealing with patients. And presumably the patient's better off because you have these false negatives. get found. The pathologists, you know, they have remarkable discordance when they look at slides. And to be able to have that basically looked at as if hundreds of thousands of them were reviewed, getting back to your second, third, and end the opinion, to get that as input for them to help and consult with a patient, I think is really a bonus. We're talking about radiology here in pathology, but, I mean, we could sort of think about this as an issue for diagnosis in general.

And there's one thing you point in the book, I think really beautifully, you know, you said once trained, doctors are pretty much wedged into their level of diagnostic performance throughout their career. And that's kind of an amazing thing is that doctors, I guess, go through CP and so on. But like, you only go through med school once, and that's an intense process you learn a lot, but you can't go through med school all the time. No, it's so true. And that gets me to, you know, Danny Kahneman's book about thinking fast and slow. And the system one, that is the reflexive thinking that happens automatically versus what we want are reflective thinking, which is system two, which takes time. And it turns out that if a doctor doesn't think of the diagnosis of a patient in the first five minutes is over a 70% error rate. And actually, that's how much time is the average with patients.

So we have the problem, as you've alluded to, of kind of a plateauing early on in a career, but we also suffer because of lack of time with this system one thinking. The thought is that the machine learning is reflecting what system two would look like because it's trained from doctors sort of doing system too. Exactly. You know, it brings in the integration of what would be the ground truce of thousands for that particular data set. So I think it has a potential.

And, of course, a lot of this stuff need validation, but there's a lot of promissory studies to date that suggests that's going to be very possible. When I think about what ML or artificial intelligence AI could do, there's like two axes. There's ones like just scale, like the fact that you can scale up servers on Amazon trivially, much more than you could scale up human beings.

We could scale up 1,000 servers right now or 10,000 servers right now. Right. I don't think we could call 10,000 doctors and get them here. The other thing you could do, you know, is that you could sort of talk about another axis is like sort of intelligence or capability. And you're talking about both in a sense,

that you can scale up, not just the fact that you could have like a resident or a med school student sort of doing things and having a lot of them. But you, in addition to that, you have, in a sense, a doctor through AI that's in diagnostics that's better than any single doctor. Right.

I think that's really what is going to be one of the big early changes in this. new AI medical era, is that diagnosis is going to get so much better. Right now, we have over 12 million serious errors a year in the United States. And they're not just costly, but they hurt people. Yes. So this is a real opportunity to upgrade that. And that's a much more of a significant problem than most people are real life. Well, and so so far we've been talking about things that feel like the sci-fi fantasy version of stuff, right? I mean, like, because we've got like this doctor of sorts through diagnosis that can do what no single doctor could do

presumably at scale it's doing this at lower cost um it's allowing human beings that do the things they should be doing um does there any dystopian turn here or how is this how there's no shortage of those and so how could how could this go wrong and what can we do to prevent well i mean i think one of the things we've harped on is that you've got to have human oversight we can't trust an algorithm absolutely because for any serious matter because if we do that and it has a glitch or it's been hacked or it has some kind of adversarial input it could hurt people at scale so that's one of the things that we got to keep an eye on for example if an algorithm gets approved by the fda oftentimes it's these days it's it's an in silico retrospective sort of thing yep and if we just trust

that without uh seeing how it performs in a particular venue the people particular cohort of people. These are things that we just shouldn't accept blindly. So there's lots of deep liabilities and it runs from, of course, privacy, security, the ethics, how many aspects that are not ideal about this, but when you think about the need and how much it could provide to help medicine, I think those are the trade-offs that we have to really consider. What we should be doing now and what people could be doing now to anticipate this? Or do you think it's like too early? is, I mean, because people have these algorithms now. And what do we see in one year versus five years versus ten years?

Well, it is rolling out in other parts of the world. You know, I just finished this review with the NHS. And that was fascinating because they are really going after this. They are the leading in the world force in genomics. And now they want to be in AI. So they already have emergency rooms that are using, that are liberated from keyboards. And they are going after this. And this, of course, in the middle of Brexit.

So that's kind of amazing. But China is really implementing this. You could say, well, maybe too fast because out of desperation or need. But one of the advantages that we don't recognize with China, not just that they have scale in terms of people, but they have all the data for each person. Yeah, that's right. And we have no data for each person. Basically, our data is just spread around all these different doctors and health systems. Nobody has all their data.

and that is a big problem because without the inputs that are complete for like you personally yeah yeah then what are you going to get out of that so this is a this we are at a handicapped position in this country and the other thing of course is we have no strategy as a nation whereas china UK and many other countries they are developing or have developed planning and strategy and put in resources here as a nation we have zero resources in fact we have proposed cuts to the same, you know, granting agencies that would potentially help. Well, yeah, so what should one do at that scale? Like, you know, there's various things people propose, you know, is this something to have

a new National Institute of Health, you know, in this area? Is there, I mean, I think, when I think about the government playing a role, I think I want them to try to help them build the marketplace and set the rules, but we have to be careful that we don't put too much regulation as well. I mean, what are, what do you, I mean, when you say we don't have a strategy, what's, What's missing? What should we be doing? Yeah, well, we have no national planning or strategies. How is AI not only for health care, but in general, how is it going to be cultivated and made transformative?

The experience I had in the UK was really interesting because there, they not only have the will, but they have a whole wing of the NHS for education and training. You just think about it. We already talked about professions within men. medicine that are going to have a morph of their daily function. So we're not well prepared. Who should take the lead to them? One of the problems we have that you're touching on is our professional organizations haven't really been so forward thinking. They mainly are centered on maintaining reimbursement

for their constituents. The entities like NIH and NSF and others could certainly be part of the solution. What you want to do here, I think, is to really. accelerate this. We're in the middle of an economic crisis in health care, which is in the U.S. the worst outlier. I mean, we're spending over $11,000 per person, and we have the worst outcomes, life expectancy going down three years in a row. Childhood mortality, infant mortality, maternal mortality, the worst, people don't realize that. Then you have the UK and so many other countries that are at the $4,000 per year level, and they have outcomes that are far superior. So if we use this, we could actually reduce inequities. We could make for a far better

business model, paradoxically, but we're not grabbing the opportunity. Maybe there's another solution we could think about which you also point to in the book, which is what if, what can we do to drive through consumer action? For instance, a lot of our health care is sick care, right? What happens when you get sick? That's almost all of it. What about what can we do to stay healthy? First thing I think of is diet and lifestyle, right? That could go a long way in so many diseases, how many things that we deal with. So actually you touch on diet. Before we even talking about like diagnosing whether you have cancer, should we be diagnosing what should be having for lunch? I couldn't agree more that that should be a direction. We have had this so naive notion that everyone should have the same

diet. And we never got that right as a country. But now we know without any question that people have an individualized and highly heterogeneous response. That's not just through glucose spikes. If you and I ate the exact same food, the exact same amount, the exact same time, our glucose response would be very different, but also triglyceride response would be different. And they don't track together. So what we're learning is if you get all this multimodal data, not just your gut microbiome and sensor data and your sleep and your activity, your stress level, and what exactly you eat and drink, we can figure out what would be promoting your health. We're not there yet, but we're seeing some pretty rapid progress. What's intriguing to me is that in cases, there are cases now,

especially, let's say, just glucose, where you can take technology to develop for type one or type two diabetics. And now I'm not diabetic, but like I actually had the sort of, sort of, I was about to say joy, but at least the intellectual intrigue of having a C.E. G.M on me for two weeks. Yeah. And so I got to play all these experiments. Right. Right. Like, so I tried white rice is or brown rice. Yeah. How's ice cream? You know, wine versus scotch. You know, all the important questions one has to figure out. Exactly. And actually, it was actually a surprise to me that how, for instance, I did not spike on ice cream. Spiked on brown rice. Yeah. I'm not sure I'm prepared to go on their ice cream diet just yet. And I don't think you would prescribe that either, right? But I think

the idea is that it's just different for everybody, right? So maybe you spike on ice cream, I don't. And that, you know, what's I think been kind of so annoying about nutrition is that we hear all these conflicting things. But perhaps part of the reason why we're hearing these conflicting things is that it is so individual. Exactly. And that it's so complicated and such a fundamental data science problem that it probably

takes something like machine learning to figure it out. Well, I think that's central. If we didn't have machine learning, we wouldn't have known this. And only, you know, thanks to the group and the wise Institute in Israel, they cracked the case on this. So, Iran Segal's work? Yeah, Aaron Segal. And now it's been replicated by many others and it's being extended.

What would be promoting your health? And right now it's, you know, these proxy metrics like your glucose or your lipids in the blood. But eventually we'll see how outcomes and prevention can be fostered by your diet. It's really kind of mind-blowing how difficult data science problem that seems nutrition is. Yeah, the problem, VJ, is a number of levels and the sea of data. I mean, we're talking about terabytes of data to crack the case for each individual. So it's not even just your gut microbiome of the species of bacteria and their density. But now we know it's the sequence of those bacteria that are part of the story.

Then you have, of course, these continuous glucose every five minutes for a couple of weeks. That's a lot of data. Besides that, you've got, you know, all your physical activity. your sensors for stress, your sleep data, and then even your genomics. So when you add all this together, this is a real challenge.

No human being could assimilate all this data. But what's interesting is not only at the individual level, but then with thousands of people. So take everything we just talked about, multiply by thousand or hundreds of thousands. And that's how we learn here. And so what I think is the biggest thing about the AI underappreciation is the things that we're going to learn that we didn't know.

Like, for example, another great example is when you give a picture of a retina to international retina expert and you say, is this from a man or a woman, the chance of them getting right is 50-50, but you can train an algorithm to be over 97, 98% accurate. And there's so many examples like that, like you wouldn't miss polyps in a colonoscopy, which is a big issue, or you would be able to see your potassium through your smart watch level in your blood without any blood. And then the imagination just runs wild as far as what you could do when you train things. And so training your diet with this torrent of data, not just from you, but from a population, is, I think, a realistic direction. And what I think is interesting about this is that it's. something where, A, you know, we don't need the AMA or NIH or anything else to get involved

in terms of diet. Right. And B, actually, people want to take care of these problems because I think most people are motivated. We just don't know what to do. Right. And so many aspects of it. Like, you know, now chronobiology is really this hot topic.

That's about your circadian rhythm. And should you eat only for eight hours during the day? Well, certain people, yes. But, you know, the whole idea that there's this thing for. for everyone. We got to get over that. That's what deep phenotyping is all about to learn about the medical health essence of you. And we haven't had the tools until now to do that. Okay, so there's a ton of data. But like a lot of it seems kind of subjective, right? I mean, did I sleep well or not? How do you sort of overcome the fact that not everything is quantitative,

like, you know, my cholesterol level? Well, as it turns out, that was kind of old medicine where we just talked about your symptoms, but new medicine is with all sorts of objective metrics. So a great example of this is state of mind or mood, and that's going to be a transformative for mental health. Because now everything from how you type on your smartphone to the voice, which is so rich in terms of tone, intonation, to your breathing pattern, to your facial recognition of yourself. I mean, there's all these ways to say, you know, Vijay, you're really depressed. You know that you're depressed. So the point being is that you have objective metrics of one's mental health as a cardiologist.

For all these years, I'd have these patients, they'd come and tell me, I feel my heart's fluttering. And I would put in the note, the heart's fluttering. That was so unhelpful. Now I can say, well, you know, you should be able to record this on your phone or if you have a smart watch, And when your heart flutters, just send me the PDF of that. And we have the diagnosis. That is real world, no longer subjective, the whole different look, really. And by the way, the patient who has a fluttering, who records their cardiogram, they don't have to wait for me.

They already have an automated read from AI that's more accurate than a doctor. There is something very anecdotal about the doctor visit. It's not right there in the moment. Yeah. Right. It's a one-off. Yeah, it's a one-off. And so it's a funny thing because people wonder about, let's say, the knock on a wearable

will be that it's not like an 8-point EKG or something like that. But on the other hand, it's there with you all the time. Yeah, no, exactly. And then there's this contrived aspect of going to see the doctor where a lot of people find that very stressful. And when we talk about white coat hypertension, we don't even know what normal blood pressure is because we need to check that out in thousands, hundreds of thousands of people in their real world to find out what's normal.

We've already had this chaos of the American Heart Association saying that they changed the blood pressure guidelines on the basis of no data. Speaking of lack of objective metric. Well, so one other area that I thought was really intriguing and to me this was almost paradoxical, the concept of AI being useful for empathy. Because I would have thought, like if we're thinking

about the things that a computer is good at, like multiplying numbers, So that's going to be something like they're going to beat humans of that any day. I would have thought, though, that empathy would be the one, like the last bastion of what we're good at and what the computers good at. But how does AI get the empathy? Because we started the conversation with us about how that's a key part of what a doctor does. Well, like, what can AI do there? Well, we are missing that in a big way today.

And how do we get it back? Well, you know, I think how we get it back is we take this deep phenotyping. we do deep learning about the person, and that's all outsourced with oversight, you know, for a doctor or a clinician. Now, when you have this remarkable improvement in productivity, in workflow and efficiency and accuracy, all of a sudden you have the gift of time. If we just lay down as we, as doctors have over decades, for administrators to go ahead and just drive revenue and basically have no consideration for patients or doctors, we're not going to see any growth of empathy. We're not going to see the restoration of care in health care. But if we stand up and if we say that time, all that benefit of the AI part, the machine support, And by the way, that's also at the patient level.

So the patients now, with their algorithmic support, they're decompressing the doctor load, too. Yeah, yeah, they're doing some of it before the doctor. A lot of simple things, ear infection, skin rashes, and all that sort of stuff that's not life-threatening or serious. But that's bypassing a doctor potentially almost completely. So between this fly-wheel of algorithmic performance enhancement, if we stand up for patients, then we have all this time to give back. Once we have time to give back, then we tap into why did humans go into the medical profession in the first place? And the reason was because they want to care for their fellow human being, but they lost their way. And now we have the peak burnout and depression and suicide in the history of the medical profession.

And by the way, not just in the U.S., you know, in many parts of the world. And how are we going to get that back? Because it turns out if you have a burnout doctor, you have a doubling of errors. And it's a vicious cycle. You have errors? And they get even more depressed. Yeah. More depressed.

So we have to break that up. And I think if we can get people so there's time together and that real reason why the mission of health care is brought back, we can do this. It's going to take a lot of activism. It's not going to be easy. And it's going to take a while. But if we don't start planning for this now, it's not going to happen. Yeah. How do you think that changes for, you know, how do you, how do you,

become a doctor. I mean, getting into med school and all the training is really difficult. What does the future of medical education look like? Like right now, pre-med degree is a lot of like biology and chemistry, not too much effort in psychology or empathy or in statistics or in machine learning. What does that look like in the future? I think we're missing the mark there. We continue to cultivate brainiacs who have the highest MCATs. We have the highest MCATs, scores and grade point averages and oftentimes relatively low on emotional intelligence. So we have tilted things. We want to go the other way.

We want to emphasize who are the people who have the highest interpersonal skills, communicative abilities, and who really are the natural empathetic people. Because a lot of that brainiac work is going to be machine generated. And so, you know, it's something that we are to start to lean in that. that direction. Yeah. Now, and it's intriguing because I think there's a chicken and egg problem here because I think first this has to be put in. Yeah, often in these any big changes, there will be resistance. Who's going to be fighting this? The resistance we have to anticipate is going to be profound. One of the problems is that the medical profession, it may not be ossified, but it's

very difficult to change. The only changes that ever occurred rapidly, like adoption of robotic in surgery, or because it enhanced revenue. None of these things are going to enhance revenue. They're actually going to potentially be a hit. We have all these interests that this is going to challenge. Like, for example, we could get remote monitoring of everyone in their home instead of being in a hospital room unless they were needing an intensive care unit. Now, do you think hospitals are going to allow that to happen?

Because they will be gutted. and then they won't know what to do with all their facilities. So the American Hospital Association is not going to like this. So I'll be del's advocate here. They're not going to like it revenue-wise. Would they say that would put patients in danger because they don't, obviously, you know, at home you don't have what the hospital has? Well, you know what's the interesting thing is?

I don't know if you can get in more danger than going into our hospitals. One in four people are harmed. Yeah, you mean sepsis and infection? The main thing, those are comial infections from the hospital. Yeah, yeah. But other medication errors and other things, the comfort of your own home, you can actually sleep, you'd be with your loved ones, the convenience. But most importantly, just think at the difference in expense. You could buy

years of broadband data plan for one night in the hospital, which is $5,000 on average. It's amazing. We have the tools to do that now, but you're not seeing it being seriously undertaken because of the conflicts. You know, so if you think about how all this has to actually happen, You know, we talked about what's possible, but like if you get to nuts and bolts, it's interesting to think who's going to do it? Because if you take just a pure data scientist who doesn't understand the medicine, I don't know if that would be enough, right? But also, I don't know if you could take a doctor that doesn't understand data science, right? And so is it going to be teams of commingled groups that get this together? Because there will be iterations between the data science and the biology and the clinical aspects that have to,

have to come one after the other to be able to make these advances? We need machines and people to get the best of both worlds. So in the book, that example of how we cracked the potassium case between Mayo Clinic cardiologists and a live core data scientists. And what was amazing about that experience to review with them was that the cardiologist thought you should only look at one part of the cardiogram, which historically known this so-called QT interval, because it was known to have something to do with potassium. But when that flunked, and the algorithm was a farce, the data scientist said, well, why are you so

biased? Why don't we just look at the entire cartogram? And by the way, Mayo, you only gave us a few million cartograms, and why don't you give us all the cartogram? So then they nailed it. So the whole idea is that the biases that we have that are profound. But when you start debiasing both the data scientists and the doctors, the medical people, then you start to get a really great result.

One of the scariest story I saw was that this algorithm was getting cancer, no cancer, right, with crazy high accuracy, like AUC of 1.0, like never making a mistake. And it turned out that there was some subtle difference between like a high Tesla magnet and a low Tesla magnet and that the patients who were very sick to start off with were always getting one type of scan and that a human being couldn't tell the difference but that the machine was picking up some signal not of whether it was cancer, no cancer,

but whether they were getting like the fancy measurement or the less complicated one. Or another great example is like there's a classic example where they're, I think, predicting tumors and they had rulers for the size of the tumor on all the tumor ones. And so really ML was a great ruler detector. Yeah. The whole idea that as a pathologist, we can't see in a slide the driver mutation.

Of course. But you could actually train the algorithms. So when the pathologist is looking at it, it's already giving you what is the most likely driver mutation. It's incredible. And that does get me to touch on the deep science side of this, which we are recognizing is way ahead of the medical side. The ability to upend the microscope, you don't have to use fluorescence or H&E. You just train, so you forget staining. The idea that you used to be hard to find rare cells, just train the algorithms to find

the rare cells. I mean, we're seeing some things in science, no less in drug discovery, in processing cancer and sequencing data, and certainly in neuroscience. It's a real quiet revolution that's much further ahead than on the medical side because there's no regulatory hurdles. And you make a good point because I think it's tempting to. just try to do what the human can do better or what the human can do better now, try to do as well. But now you're talking about doing things that no human being could do.

Yeah. Imaging plus genomics, where the genomics read out, let's say, or whatever, the blood assay is the gold standard. I don't want to predict what the pathologist would say. I want to predict the biopsy. I want to predict the blood or whatever, the true gold standard is behind it. Right. And if you're training on the best labels, you can do things that no human being could do. Well, you know, this may be the most important point is that.

we have to start having imagination because we don't even have any idea of the limitless things that we could teach machines because I'm getting stunned almost on a weekly basis I said I never would have thought of that and so just fast forward here we are in 2019 what's it going to be like a few years of all the things when the Mayo Clinic told me they could look at a 12 lead

cardogram for millions and be able to say this person's going to get atrial fibrillation in their life with X percent probability. I said, really? And they've done it. And so I never would have expected that. Yeah. Well, and that's a really fun point because,

and you could either think of two ways, that the human beings aren't being imagined of or what does imagination mean for an algorithm? Right. Well, if we get into heavy into unsupervised learning, we're a bit limited by the annotation and the ground truth going back to that. You can only imagine.

things when you have those for supervised learning. But, you know, as we go forward, we'll have more of those data sets to work with and we'll be better at going forward without, well, with federated data sets and unsupervised learning. So the opportunities going forward are pretty enthralling. Yeah, well, the unsupervised learning is interesting because you can finally just, you know, and for those aren't familiar with the term, it's kind of like trying to find the clusters to sort of not have the labels, but so see the lay of the land. And that's interesting because no human being can sort of, especially in a high dimensional space, like visualize that and see that. And so that's one thing. But the second thing is that if you just throw all of the data in

and maybe have the algorithm to make sure that it's not overfitting, that it's not trying to find an overly complicated story, almost like, you know, these conspiracy theories are like human beings overfitting for the moon landing being a hoax or something like that when there's a simpler explanation for things. If you keep it to a simple explanation, the computer can try everything. Yeah. So like you talk about it could look at the whole. whole cardiogram. It could look at things that we don't look at because either we're expert enough to know that couldn't possibly write even if it is. Or we just don't have the time. It reminds me sometimes these algorithms almost like children and that kids just don't

know until they'll try things. And that's where imagination and creativity often comes from. I couldn't agree with you more. So we've been spending a lot of time talking about diagnosis, but prediction is another thing that is really important. I'd call that a real soft spot. in AI. And I told the story of my father-in-law, who kind of was my adopted father in the book, about how he was on death's door. He was about to come to our house to die. And he was resurrected. But any algorithm would have said he was a goner. And so the idea that at the individual level, you could predict accurately, whether it's end of life or when you're going to die, or in the hospital, this is how long you're going to stay or you're going to be readmitted,

all these things. We're not so good at that. We can have a general sense from a population level, but so far prediction hasn't really panned out nearly as well as classification, diagnosis, triage, that kind of stuff. And I still think that that's one of the shakier parts, because then you're going to tell a person about a prediction. You know, we're not very good at that. We talk to people with cancer and we tell them, you know, their prognosis, it's all over the place in reality.

And so, you know, the question is, our algorithms are algorithms really going to do better? Are they just going to give us a little more precision, maybe not much? Is there enough information to ever predict anything like that? Well, that's a part of the problem, too, is that the studies that have been done to date, things like predicting Alzheimer's, predicting all sorts of outcomes you can imagine, they're not with complete data. They're just taking what you can get, like what's in one electronic health record, one system, rather than everything about that person. So maybe it will get better when we fill in the holes. I always think about what would be the interesting challenges to work on.

That's like one of the most interesting ones. I think it is because there you could improve the efficiency if you knew who are the people at the highest risk and who you want to change the natural history, what their algorithm is predicting, if it's something that's an adverse outcome. that eventually will probably get there, but it isn't nearly as refined as the other areas. But if you combine all these things together, you know, this thing where you're monitoring your body every five minutes and your diet and your exercise and your drugs and you have all this longitudinal data, that's something that no one's ever had before. Yeah, well, you're bringing up a big hole in the story, which is multimodal data processing. We are not doing it yet. You know, like for a perfect example is like in diabetes, people have a glucose sensor and the only algorithm they have tells them that the glucose is going up or down. That's pretty dumb.

Why isn't it factoring in everything they eat and drink and their sleep and activity and the whole works? Someday we'll have multimodal algorithms, but we're not there yet. Well, so let's go back to where we start, you know, a visit to the doctor in the future. And like the good news is that the doctor doesn't have to do it. in either typing or recording, AI's sort of figuring out the diagnosis and that the doctor has all the time now to actually be empathetic and communicate, which is great. But is that now all that's left? No, no, not at all because human touch. So when you go to see a doctor, you want to be touched. That's the exam part of this. People, when they get examined for their heart and you don't

even take off their shirt, they know. They know there's a shortcut going on here. Yeah, that's interesting. They wanted to have a thorough exam because they, they know that that's part of the real experience. And so what we're talking about is the exam may change. Like, you know, for example, I don't use a stethoscope. I use a smartphone ultrasound and do an echocardogram, and I show it to the patient together as we're doing it in real time, which the person would never see. And, By the way, they wouldn't know what Lubbub looks like, or sounds like, but you sure can show them. So the tools of the physical exam may change, but the actual hands-on aspects of it and the interaction with the person, the patient, and that that's the intimacy. And we've lost that too.

You know, the physical exams have really gotten very much a detraction from what they used to be. I mean, we need to get back to that. That's what people want when they go see a doctor. Yeah, people have deprecated exams because essentially they said they weren't of value. But it sounds like what was being done was not the part that needed to be done. When you're dealing with analog tools and they can be so superseded by the things we have today. And when you're sharing them with the patient, so here's what you have. And then you send them the video files or the metrics so they can look at when they get home and get more familiar with their body.

It's not only the physical exam that happens instantaneously in the encounter, but the ability to have that archive data that people get more, they learned about themselves. That's all part of that awareness that's important. And you know, you talked about back to the future. There might be another sci-fi analogy. I think there's some Star Trek episodes like this where actually the group that has the highest technology is the one where the technology is invisible. Yeah. And it sounds like that's what the, all of this is going to be in the background. That's right.

But that you really are interacting with a person and this person now has just these powers that they couldn't have before. I'm with you all the way. Well, thank you so much. This has been fantastic. I really enjoyed it.