Plain English with Derek Thompson - How AI Could Help Us Discover Miracle Drugs

Episode Date: September 13, 2024

We may be on the cusp of a revolution in medicine, thanks to tools like AlphaFold, the technology for Google DeepMind, which helps scientists predict and see the shapes of thousands of proteins. How does AlphaFold work, what difference is it actually making in science, and what kinds of mysteries could it unlock? Today’s guest is Pushmeet Kohli. He is the head of AI for science at DeepMind. We talk about proteins, why they matter, why they’re challenging, how AlphaFold could accelerate and expand the hunt for miracle drugs, and what tools like AlphaFold tell us about the mystery of the cosmos and our efforts to understand it. If you have questions, observations, or ideas for future episodes, email us at [e-mail]. Host: Derek Thompson Guest: Pushmeet Kohli Producer: Devon Baroldi Learn more about your ad choices. Visit podcastchoices.com/adchoices

A restaurant's best dishes tell stories. Their flavors embed themselves in our memory like song lyrics or lines from a movie. So much so that a little slice of a restaurant's story can become part of our own. I'm Danny Chow and this is ShiftMeal, a new video podcast from The Ringer where we're sharing a bite and chopping it up with chefs and restaurant people during their off hours. Look out for episodes of Shift Meal on Ringer food starting September 18th. Today's episode is about the search for new miracle drugs and the future of medicine. So a few years ago, I was having this off-the-record conversation with a scientist who said something to me that was so simple and memorable and compelling that I knew in that moment I would eventually make it into some article or essay or podcast, but I never had the perfect opportunity to recapitulate his observation until today. He said the history of medicine has been a kind of journey to the center of the self.

And you can tell the multi-thousand-year history of medicine as a kind of three-part history. In part one, which stretches back tens of thousands of years until maybe the beginning of the 20th century, medicine has largely been finding miracle drugs in nature. Almost all of the earliest medicines were derived from plants, animals, bacteria, fungi, leeches, for better or worse. Aspirin, for example, was originally derived from a bark, white willow bark. Quinine comes from a tree. Morphine is derived from opium poppy. Penicillin, the first truly effective antibiotic, comes from a mold.

Even the first vaccine in human history against smallpox comes from cowpox. Hence the name, vaccine from the Latin vaca, meaning cow. In the 20th century, we went from phase one, call it the early natural phase of medicine, to phase two, the biotech phase, where scientists learned to design medical products based on the proteins and molecules that are inside of our body. that we naturally produce. We went from canvassing the outdoors for molecules to canvassing the body. We synthesized insulin. We learned to engineer cells in the lab that we could pump into sick people's bodies

to help them fight diseases, so-called monoclonal antibodies. For example, one of the most famous and most profitable drugs developed in the 21st century is called Humira, which treats rheumatoid arthritis, Crohn's disease, and a lot more. It is a monoclonal antibody. Maybe the most famous drug of the moment is OZempic, which mimics our gut hormones to suppress our hunger, and also it seems to encourage dopamine moderation. These drugs were not developed by scraping bark off of a tree

and rubbing it on our bodies. They were built by first looking inside of our bodies. But now, the scientist said, we're at the cusp of a third turning, a third revolution in medicine, which he called the age of programming. The mRNA vaccines that were so effective against serious disease with COVID,

they engineer our immune system to target the spike protein of the coronavirus. CRISPR allows us to edit our own genome. The future of science looks a lot like engineering. But a good engineer needs to be able to see what they're working on. And for the large part, we can't actually see functioning parts of the body very well. That means we need new tools to look inside the human body,

understand how it works at the smallest level, see the tiniest Lego blocks of our biology. Proteins. If you open a biology textbook and you look under the section where they define proteins, you'll probably find something like this. A protein is a big molecule composed of chains of amino acids that play crucial roles in the structure of function and regulation of cells and organisms. All true and very boring.

I think a stickier way to think about proteins is to think of them as a link, a critical link, between your genes and your body, your genotype, and your phenotype. For example, take the case of keratin, a tough protein that helps to form tissues in, among other things, your hair. Your genes code for the production of certain kinds and amounts of carotin. Your body makes a certain type and amount of carotin protein. And the type and amount of

carotin produced by the body, based on your DNA, determines various aspects of your hair phenotype. Is your hair wavy? Is it straight? Is it strong? Is it brittle? Is it thick? Is it thin, the link between DNA, which we cannot see or feel, and the visible and felt human body is proteins. The problem with proteins is that their function is largely determined by their shape, and understanding protein shape has been one of the toughest challenges in medicine. In the last few years, however, Google DeepMind has released several versions of a technology called Alpha Fold, which can predict the structure of proteins from their amino acid sequence.

If the future of medicine in this third age, the age of programming, the age of engineering, requires that scientists understand how the human body works at the smallest level, then the invention of AlphaFold and similar tools might do for science in the next century what the microscope did in the first scientific revolution, it could allow us to see biology more clearly. And with that new understanding, a whole host of new drugs could be unlocked. Today's guest is Pushmeet Koli.

He is the head of AI for science at DeepMind. And today we talk about proteins, why they matter, why they're challenging, how AlphaFold could accelerate and expand the hunt for miracle drugs. And what tools like AlphaFold and other AI tell us about the mysteries of the cosmos and our efforts to understand it? I'm Derek Thompson.

This is plain English. Pushmeet Koli. Welcome to the show. Thank you, Rick. Pleasure to be here. I'm grateful you're here. I've wanted to have this conversation with you for a while. And I would like to start at the highest possible level.

before we dive into the details of proteins and artificial intelligence and the weird and hopefully wonderful future of AI and medicine. So, life. The universe is just a random distribution of atoms in space, and some of those atoms form inert rock,

and some of those atoms have gotten together and somehow formed entities that live and strive and procreate and fight and laugh and even understand themselves in relation to to their environment. And this fact that conscious life emerges from some of the same simple elements that would otherwise be cold rock might be one of the most profound mysteries of the universe.

To you, what is so complicated about life? So thanks, Derek. I mean, like, you explained it really well, right? It's such a very interesting thing that you and me, are having this conversation. And if somebody was to chemically sort of analyze us, we are essentially water. A majority of our bodies is just water.

We are just bags of water floating around. And so what is it that makes me sort of be able to speak what I'm speaking for you to be able to make sense of it, think about it, rationalize it? And so the magical stuff in this bag of water are these small, tiny things called proteins. These are the machines of life. They allow us to see the world. They allow us to reason.

They control how energy is transformed in our body. It governs all the processes of life. And yet, there are only 20,000 of the world. these basic building blocks, 20,000 proteins in the human body, and that's for everyone. So these are the Lego blocks. They're composed of amino acids, and they sort of build every single tissue in our body. And so how does from an embryo, we are able to create such a complex system that can do, that can exhibit so many different properties. And how do we go from understanding what these molecules do, how they interact, and how can we explain the behavior

that we exhibit? That's a key grand challenge of life. And to give people a sense of some of these behaviors and functions of proteins that they might have heard of, hemoglobin, transports oxygen and red blood cells, insulin regulates blood sugar, collagen provides structure and some tissues, keratin provides structure to hair and nails and skin. What's so interesting about proteins is when you look at them up close, as I have in some of my reporting on your work, they look like little balls of string. Like if you cut maybe six inches of string and you rolled it between your fingers and you put it on a flat surface so that the string knot sort of loosened a little bit, what you've built there is very close to many of the protein structures that I've seen.

You said that these proteins are built out of very basic stuff, and that you can, it's important to understand the shapes of these proteins. Why does the shape of the protein have such significance? So the shape is so important because it governs the function. of those proteins. Like you gave the example of hemoglobin, right? If there's mutation in any of these proteins, even a single mutation, for instance,

can get you a disease like sickle cell anemia. So there are these mutations called mis-sense mutations in your genome, which lead to changes in the proteins that is expressed. and that changes the function of the protein. So it's really important to understand what is the function, what is the structure of that protein? Because it governs, it gives us hints as to what the function would be. And also these proteins interact with other proteins to form larger complexes.

And so it's really important how they geometrically align and interact with each other. And that's why protein structure prediction has been a grand challenge of biology. Can you give me a sense before we dive into the work of Google DeepMind and AlphaFold, of where protein science was maybe up to five years ago? It seems like for many decades, scientists had been working really hard to understand the amino acids that build these proteins, and from that understanding of the sort of recipe of these proteins, build a kind of protein data bank from which we could help determine or devise the ultimate shape of these proteins. Can we talk a little bit about what the state of protein science was before the arrival of

AlphaFold? Okay. So if you sort of look, if you go around and try to sample biological life, different organic matter and so on and try to figure out how many proteins you see, you will see millions hundreds and millions, even billions of these proteins in the world. I told you, like, for humans, we have 20,000 proteins, but look at all the species, the diversity of life that we have on the planet, and you get billions of these proteins.

And scientists have been trying to figure out what is the structure, given a sequence of a protein, what is its structure? And for each of these proteins, finding the structure is an extremely time-consuming process. You need to sort of isolate the protein. Then you have to crystallize it using, and then you try to find the structure using X-ray crystallography or cry-oem. And before AlphaFold came out, even finding the structure of a single protein might take a PhD

worth of work, right? And for some heart proteins, their structure was unknown. For instance, like there are 20,000 human proteins, but only a very small fraction of the human proteome of that 20,000, for those we knew the structure. For most of it, we did not know the structure of those proteins. So AlphaFold, which is DeepMind's protein folding prediction engine. It was called when it came out science journals breakthrough of the year,

There's several AI experts I've talked to who say that it's the most important breakthrough in artificial intelligence to date. At the highest level, because we're going to spend the rest of this show talking about the implications of Alpha Fold for medicine, for our understanding of the world. What did AlphaFold accomplish? What did it do? So AlphaFold tried to solve protein structure prediction in a computational way. So instead of basically going through the lab, and trying to find the structure through experimental beans, we said, is it possible for us to somehow build a computer program that given the description of a protein through the sequence

of amino acids that composed it, can we take a sequence of amino acids and directly predict the structure of that protein? Can such a program exist? And that is the problem that our team started working on in 2017. We came up with Alpha Fold 1. Our first generation, which we showed was much, much better than any other system in 2018. And in 2020, we got to a point where it was reaching a point where a section of the community sort of felt that the problem has, for many practical purposes, solved in the sense

that these predictions, these structures can now be used for many important biological tasks. A lot of times the follow-up question here immediately goes to implication. Okay, what can we do with this? But I actually want to start with something that in a way is the inversion of that question. Why did you think that protein folding was a solvable problem in the first place? Right? Like people at Google DeepMind could have taken any challenge. You could have said, we want to predict hurricane wind speeds in Florida based off of weather patterns near Africa. We want to predict football win averages because if we built a prediction engine that allowed us to predict the outcomes of sports games, we could make a heck of a lot of money for a parent company alphabet by essentially building this super betting thing.

But you picked protein folding, which I imagine has to do with both the significance of protein. these are the small engines of life, but also I have to imagine it has to do with probability that this was a solvable challenge. So the first half of this, the fact that proteins are important to life, that's obvious.

What's not obvious to me is why you thought this problem was solvable in the first place. Yeah, so that's a very good question. So there were a number of conditions that we were, while selecting the problem we were thinking about. The first thing was that it needed to be a root node scientific problem.

The fact that if we solved this problem, there would be a lot of downstream implication. And protein folding definitely qualifies on that dimension because if you understand proteins, it has implications for drug discovery, sort of disease understand, synthetic biology, many different things. The second bit was that intelligence does not happen in a vacuum. It doesn't just emerge. Like, you're not in a room and suddenly you get intelligent. You need experience. So you need high quality data.

And this is where the protein structure community had been very visionary in investing in resources where whenever the community found the structure of a protein, whenever they found a new structure, they carefully deposited it in a protein database called the PDB. And so we had very high quality training data that we could use for training such a model. And then the final thought of thing is that with machine learning and AI systems, these systems are really powerful in the fact that they can memorize things very easily. So what needs to happen in this particular case is you want to make sure that you're not fooling yourself. there is a way to assess the model such that it can't fool you.

And in the context of protein structure prediction, there was a competition or there was an assessment, a biannual assessment called the, it's like the Olympics of protein structure prediction, which was held for almost two decades. And for every two years, they would assess different protein structure. prediction models, and they would do it in a very interesting way. So in that year, whenever somebody discovers a new protein structure, instead of sharing it with the scientific community, they will basically, they will hide it. And they will share it with some organizers who know the structure, but no one else knows

the structure. And then every submitting team will be asked to predict structures. for these proteins, for which the answer is known, but only by a few experimentalists. So there is no way to sort of memorize or sort of train on the test set. And so this was important as well. So we had a very high impact problem. We had good training data for it. And then we had a very good means of validating whether we have actually solved the problem or not.

So in this case, it is a little bit like the implication of generative AI that more people might be familiar with, which is that AI like Gemini and ChatGBT, BT, are trained on this massive corpus of digitized text, whether it's Reddit or the complete works of Shakespeare and the Bible and everything else that's on the internet. And from a kind of computational mapping of the relationship between words in all of these sources can predict. new sentences from in response to prompts. So you take this small bank of information, you make interpretations or inferences from it, and that allows you to predict new protein shapes in this respect. I just wanna ask because people are familiar, I think, in Chachby, T, and Gemini

with the concept of hallucinations. So how do you know that you're predicting the right protein shapes. I mean, the cost of nonsense in Chachibit is very small. I ask it a question. It hallucinates. It lies to me. I move on with my life. Or maybe I tweet it out and hope that I get some credit from people who think that the mistake is funny. But the cost of nonsense in medicine is very high. How do you guard against Alpha Fold, quote unquote, hallucinating? Yeah. So that's a very important question. And so in the context of AlphaFold, not only was the model very, very accurate, but it had another property. We trained it to predict the

uncertainty associated with its predictions. And where it really sort of shown, where it can, where whenever it was right, it knew it was right. And whenever it might have made a mistake, it said, yes, I might have made a mistake. And that was really an important feature of it so that you can, you can really sort of figure out where does the model have good confidence. And finally, as I mentioned, there was this evaluation, the KASP assessment with which you can really test, does the model actually generalize to completely unseen protein structures? What effect is this having in medicine? So typically when before alpha fold, medicine and drug discovery was done in a structurally impoverished sense.

You would have targets and those targets, you were very careful in which targets you want to find structures for and each structure might cost you sort of anywhere from a quarter of a million dollar to a million dollars. So you were very sort of, you didn't want to do a lot of structural characterization experimentally. And then with Alpha Fold 2, now you have a lot of different structures that you can find out. And this has really both accelerated the discovery cycle as well as made it much more accurate. So you are seeing, and there have been cases reported, where people have looked at a liver cancer sort of drug and have been able to, from start to finish, being able to propose a candidate in a matter of days and weeks, which would have earlier taken

sort of months or years. Can you go inside that example and explain to me exactly how someone working on a liver cancer drug candidate could use a protein structure predicting technology to accelerate the discovery and design of this kind of cancer therapy? Yeah. So in the context of a cancer drug, for instance, maybe there is a certain pathway, a cancer pathway, and there's some target, some target protein, that you want to sort of inhibit or you want to disrupt some pathway.

So there's a target, and you want a sort of a molecule to bind to that, to that sort of protein so that its behavior changes. Earlier, you would not know the structure of that protein. So you are just sort of developing these compounds and trying to see which one would hit. But now you know the structure of what you are trying to target, and that gives you an ability to sort of really figure out computationally what kind of small molecules you might, or what kind of sort of biological.

you might want to design to bind to that target. And this is not just in the context of liver cancer. Like if you take, for example, neglected tropical diseases like dengue or Leshminiasis, shagas, there are proteins associated with these diseases. And again, you want to sort of design compounds and small drug molecules which bind to those proteins. if you have the structure of the protein, you can do that in a very informed way. Embedded in what you just said, I think, is a really important point and a very interesting point that I want to understand a bit more. You said at the top that proteins are essential

to life, but they're not just essential to human life. They're essential to all life. And some of the proteins that are most useful in the drug discovery process are not human proteins. They're proteins of bacteria, proteins of viruses, which are authors of our destruction, rather than things that keep us alive. My only entry into this world is that I know the coronavirus is called coronavirus because of the crown or corona around it. It has this prominent spike protein that the mRNA vaccines were exceptional at identifying and then training the body's immune system to seek and destroy. I understood that AlphaFold was very good at predicting the structure of the 22,000 human proteins. But in order to be useful in the way that you're describing, it also has to be relatively successful at predicting the structure of, say, virus proteins or bacterial proteins, proteins of enemies of the human body.

Can you help me understand how successful Alpha Fold is at predicting these non-human proteins, if that makes sense? Yeah, and that's exactly right. Alpha Fold is actually not just applicable for human proteins, the 20,000 proteins. in the human protein, it can be applied on any protein that you find in nature, whether it comes from a bacteria or a virus and so on. It has different levels of accuracy, but it can be applied on any of these proteins. And with regards to SARS-CoV-2 and COVID, we were developing alpha-fold-2 at that time when SARS-Sorpe-2 was happening. And the structure of the spike protein, you would remember, very quickly.

quickly sort of came up because there was a lab which because of the similarity to SARS 1, people were able to structurally characterize the spike. But the structures of the accessory sort of proteins within the virus, people did not know what was the structure. And with Alpha 4-2, we were able to very quickly figure out the structure, make predictions for the structure of those necessary proteins and then share them with the scientific community. And Later on, once those structures were actually experimentally sort of verified, we saw that AlphaFold 2 had actually given a very, very good prediction about off the structure of those proteins. The metaphor that I'm about to employ might not be useful.

And if it's not useful, feel free to just discard the premise of the question and answer something else. But it seems to me like the way you're describing AlphaFold and its relationship to our understanding of the coronavirus spike protein is that when we understand the shape of proteins that are our nemesis, so to speak, on viruses and bacteria, it's a little bit like getting an x-ray of a lock. But then we also have to design a key that fits into that lock, right? The same way that the MRI vaccines... Okay, okay. So I'm on the right track with the metaphor, but this is the part of the metaphor I don't understand. So let's say that with COVID, with SARS-CoV-2,

we found a way to x-ray the lock that we wanted to pick. But then we needed to design a vaccine that was the perfectly shaped key to fit inside that lock. I get that we have the technology to x-ray the lock. Do we also have the technology to develop the perfect key once we understand the shape of that lock? Yeah, and so this is a very timely question because just last week, in fact,

we released Alpha Proteo, which is our protein binding system. It designed proteins to bind to any target. So for SARS-CoV-2, you can now have a protein and you can say, okay, or given any sort of target, you can say, design me a new protein that binds to this. And by binding it might inhibit it or inhibit its function or sort of do something else to that protein. So we have tools like that now which can leverage the protein to design specific binders. So I see, we developed the x-ray technology for the locks, and now we've just recently, as a days ago, developed this new technology to design keys. And certainly at the conceptual level, that sounds

incredibly promising. We've talked mostly here about proteins on viruses and bacteria can you give me an example of how alpha-fold can be used when we're directing it on human proteins, on one of these 20-22,000 human proteins in our body? What's a way that someone working with human proteins can use alpha-fold in order to accelerate or expand their drug discovery process? Yeah. So, again, like in many of the different, diseases that we get are due to something going wrong in our cellular machinery.

So sometimes certain proteins are overexpressed, they accumulate, and that causes problems, right? The body is like a fine-tuned machine where proteins are, every protein is doing its sort of thing. and if there is any problem, like if certain proteins sort of are overexpressed, then that's the problem. It can cause you a disease. If certain proteins are not expressed, that causes a disease. If there are more proteins, you might want to inhibit their function. So you understand the structure of that protein. If you know the structure of the protein, you can now design a drug to interact with this protein and inhibit its function.

so that you can change the function of that protein. No technology is perfect. And this is just the 1.0, 2.0 version of AlphaFold. What are the most important limiting factors on this technology? If I want an honest assessment, what's the most important limitation here? Yeah. So one of the key sort of limitations of AlphaFold is that it makes predictions about a full protein, but it is not very sensitive to small changes that might happen in the

protein. For example, if there's a single point mutation in the protein that might change its sort of shape, then alpha-fold might not be sensitive, might not pick that up. We have other models which can sort of say that this might be a problem, but we are not in the position to actually predict the structural change that will happen to the protein under these mutations at the moment. If we're thinking about the various ways that Alpha Fold could be used in the near future, we've talked about, number one, the idea that you can drug proteins of viruses. Number two, we've talked about the ways in which we can help identify flaws in protein creation in the human body. Is there any way that we've done?

We can, that you have thought about, ethically, taking proteins that seem to do something quite wondrous in nature, outside of human bodies, and pull it inside of human bodies to accomplish something critical. I'm imagining here an example in modern drug discovery, a very prominent example of modern drug discovery, which is the GOP1 drug revolution. The revolution that's led to OZempic and all these other medicines that's upbound, this is a revolution that, to a certain extent, arguably began with an investigation of the venom of Gila monsters, where the saliva of Gila monsters was studied.

It included a peptide, so just a little string of amino acids, that seemed to help Heli Monsters suppress their appetite after a meal so that they could just eat once and be okay for two months, that peptide was synthesized in a lab, yada, yada, yada, yada, we now have hunger suppressing drugs like OZempec that came from the study of a peptide in a friggin lizard. Are there other ways that you understand that we can use a technology like this to inventory proteins in the natural world that synthesized and introduced into our bodies could essentially allow us to do somewhat superhuman things. Like in a weird way, Ozenpik is a superhuman drug to the extent that it was, it functions in a similar way that a lizard saliva functions. Yeah. So that's,

that's very interesting. And the point is, as we are finding the structures and then the functions of these proteins across not just the human proteome, but the wider sort of natural sort of protein universe, we are finding more and more interesting use cases of this. Now, one of the non-discovery use cases that was very interesting, and one of the initial ones that we looked at, was using alpha-fold for designing enzymes. for decomposing plastics. Now, that's a very, sort of, that's not a drug discovery. And you would think, well, protein structure prediction,

we have been talking about human health and so on. But the same technology can be used for designing enzymes for decomposing plastics. Like the same organic chemistry is involved, and you have an understanding of proteins is very useful in that context. So while sort of I am not aware of example, examples where other proteins have been used in the context of human sort of changing human behavior in cells. But these things definitely can be used in that way. I have a thought about artificial intelligence like AlphaFold that might be a little bit too

much grad student with Sativa at 3am. But the nice thing about having an expert on this show is that you can tell me whether this thought points us in any useful direction. One way to think about what generative AI has accomplished is that we've used math to build a map of human language such that we can talk to a machine, ask it questions, and get back answers that speak in our own language well enough

that we feel understood. And some people find that astonishing and some people think it's overrated, but what's undeniable is the accomplishment itself. We mapped a language. It seems to me that AlphaFold has mapped a language as well. You take 20 amino acids. They create sequences of several hundred in various combinations, and those sequences create proteins of which there are millions, and the interaction

of these proteins, these amino acid sequences, create life. This really is not so different than 26 letters combining in sequences to produce words and sentences that number in the millions, and then from those words and sentences, a nearly immeasurable number of unique ideas can be communicated, the sum of which we call language. I am fluent in one of these languages. I'm fluent in English. I am not fluent in protein. But the fact that we can build a machine

that is passively fluent in a language no human being speaks is really thrilling because it seems possible to me that there are more languages in the universe than the number that humans are fluent in. There are languages embedded in the fabric of reality,

that can be pulled out by some kind of language superpower, whether it's genetics or cellular functioning or cancer or neuroscience, these processes might be best understood as languages that humans don't speak yet. Do you think about your own work as a scientist at all like this? Is there value in thinking about exploration at the scientific frontier as the discovery of languages that the universe speaks in which we are not fluent. Absolutely. I think that's absolutely the right way to put it. In fact, part of sort of biology and biological research is about translating, doing language translation. If you think about,

we started with the analogy of how as humans, we are a bag of water. But, If you think about it, how does that bag of water created is created? Our DNA is like a recipe book. And there is a coding part of the DNA which encodes which 20,000 proteins will be expressed. And then there's a non-coding part of the DNA which is like the recipe, which says, like you mix so and so and this is how you express things, right? And every life form that we know comes with that, like many life forms come with that DNA form. And we want to understand how to decipher that language.

If there's a change, if there's a mutation in the coding part or the loan coding part, what will the effect on the organism be? So this is very much a language translation problem, but the alphabet is different. The amount of data that we have is very limited. The sentences, you're not sort of translating a short paragraph. You're translating something very, very large. Our training is quite long compared to the normal sort of text that you would expect to translate. and the semantics of it is also quite sophisticated.

One sort of alphabet change here or there might have quite different effects in the overall functioning of the organism. In thinking about what languages we might be able to translate next, I was thinking about this relationship between existing data and progress in artificial intelligence. So one reason I think we've been able to computationally map English is that there's just so much, or I should say language, is that there's so much damn language on the internet.

We can teach Gemini and chat GPT how language works because there's just trillions of words on the internet that we can feed it in pre-training. Similarly, you mentioned that, Alpha Fold is able to predict protein structure in part because we have this wonderful protein data bank that we have been building that AlphaFold could ingest before making its own predictions. What are the data banks that we could theoretically tap next or in the next five to 10 years that you think would allow us to become fluent, so to speak, in some new feature of biology?

Yeah, so I think this is a very active area of research. Understanding the behavior of cells, understanding the behavior of sort of systems at the whole sort of organism level. So you need a lot of... data, but it's also the data has to be, in some sense, it has to have high quality. It has to have expressity and it needs to be sort of expressive. Like if you repeat the same thing many times, that doesn't, that's not a rich experience. A rich experience is basically, it comes from the diversity that is implicit in the dataset.

So I think as genetic sequencing becomes more ubiquitous, we will be able to generate more data, and that will give us more information about our DNA, how sort of perturbations and how mutations in the genome affect things. But then at the cellular sort of level, we need to collect a lot more data about the behavior. of how cells behave under different environmental conditions. And those are the types of data sets that the field currently has not been able to generate and is actively working towards. And I'm hoping that we will see a lot more advances in that area in the next few years. You mentioned genetics.

And there we have an enormous amount of data with, you know, G-WAS studies and the incredible declines in the price of genetic sequencing. Can you tell me where we are right now in our ability to, quote, read the book of the genome and understand it? Like, it seems to me that we don't know yet what sequences of genes produce all kinds of of phenotypes and characteristics. Like if I gave you or I gave an AI, my whole genomic sequence, it wouldn't be able to read that and tell me, here, you have this genetic predisposition for schizophrenia or right-handedness

or that you're likely to become a tenor or a bass as a singer or you're more likely to become a world-class marathon runner or a world-class memory champion or, you know, you'll die young of this disease or live old. Can you help me understand where we are in terms of our literacy when it comes to reading our genes? Yeah. So I think the challenge is also that first of all, our DNA is extremely long, right? You're not talking about a few, a paragraph. You're talking about multiple volumes, billions of tokens, right, for each instance,

just to describe one instance, right? There are billions of tokens. like at the base resolution. And then the other issue is that when you're interested in predicting phenotypes, like phenotypes like the ones that you were sort of mentioning, like longevity or some other sort of thing, they are a function of both what is in the genome, but also a function of the environment.

Right? And that makes the prediction problem, ill-post. If you just give me the genome, I wouldn't be able to predict it because the environmental factor is not visible.

But there are certain things that we can do. There are certain phenotypes which are not informed by environmental factors, like how genes express and so on.

And so we are at that point where we are trying to at least make predictions about those phenotypes at the tissue level, right? How gene expression happens, how proteins are expressed in different tissues, and so forth. And then we are making sort of, we are seeing quite significant advances in that area. And at some point of time, as we sort of are able to capture these, effects, we'll be able to move forward on these other phenotypes where environmental conditions will be able to model the environmental conditions and their effects as well.

Right. And theoretically, I don't want to take this concept too far. But there are a range of diseases and characteristics that have different levels of environmental determinism. and it would be incredibly interesting if we could reach some kind of certainty gauge by looking at the human genome and saying, you have this percent chance of developing this condition relative to environment. Or some things like eye color are probably much less sensitive to environmental pressures, whether other things like personality might be much more sensitive to the environment. And it would be fascinating if you could develop some kind of technology for not only, identifying the genetic predisposition, but also assigning to it a kind of environmental

sensitivity score that would really allow scientists of the future to read into our genome what truly is there. Yeah, I think that's a very interesting sort of question. We have models today, in fact, in last year, we released a model called Alpha Miscence, which makes predict. predictions about what is the effect of mutations in the coding region of the genome? So these are mis-sense variants, where the change in the genome results in a change in the protein that is expressed. So what is the effect of that change? Is it a benign change? Nothing happens.

Or is it a pathogenic change? that somehow it will affect the fitness of the entity concerned. So at this point, we actually are able to train these models, and we have been able to show that these models are very, very good in then a zero-shot, in a zero-shot way, generalizing to various sort of applications of the model. to see whether this particular mutation will have a specific phenotype or not. Well, that would be absolutely wonderful if there was, you know, new ways of reading our genome and predicting characteristics

and human outcomes at the level of disease and body function. It's an incredibly interesting frontier of science, and I really appreciate you helping the understand it better. Prushmeet, thanks so much. Thanks, Jadik. Thank you for. listening. Today's episode was produced by Devin Boraldi. Our summer schedule for plain English for the next few weeks will be one episode a week on Fridays. We'll see you next week.