Instant Genius - How science can help us predict the future

Episode Date: August 15, 2024

The future can be scary, but what if there was a way for us to understand it a little better? Tom Chivers believes there is. His new book Everything Is Predictable explains how Bayes Theorem, a statistical model, can explain the world around us and, in some cases, help us predict the future. Learn more about your ad choices. Visit podcastchoices.com/adchoices

This podcast is sponsored by name, audio and focal. Streaming has made music more accessible than ever, but true listening is about more than ease. It's about quality. British audio experts name audio, alongside French acoustic specialist focal, combine handcrafted tradition with cutting-edge innovation and high-end materials,

delivering digital precision with analogue warmth. So you can experience exceptional sound at home. Music just as the artist intended. Visit name audio.com. to learn more. Hello, I'm Alex Hughes, and this is the Instant Genius podcast, a bite-sized masterclass from the BBC Science Focus magazine. There are countless theories, explanations and beliefs that can help us understand the world

around us. But what if there was one that could allow us to predict the future? Of course, as great as this would be, it just isn't possible, but there is a theory that gets us close, Bays theorem. originally theorised in the 1800s, it is still going strong today, and many believe it to be the closest we have to predict the future. So how does it work? And what can it actually be used to predict?

We spoke to Tom Shivers, the author of Everything is Predictable, to learn more about this leading statistical theory and how it can be used in everyday life. So back in the 18th century, a statistician came up with a theory of probability that at the time was, I guess, arguably not that well regarded. In 2024, you've wrote a full-length book on this topic. It's become a commonplace theory in the world's statistics. Obviously, a lot's happened in that time. Can you explain what Bayes' theorem actually is and sort of how it's become used today?

Yeah, sure. Okay. So if you did probability at school, you'll have done things like, I don't know, how likely are you to draw three aces out of a pack of cards? Or how likely are you to roll three sixes on three fair dice, you know, which is one in 216, by the way. Okay. So what you're doing there is you're saying, how likely am I to see this result, given my hypothesis at the dice are fair? If we assume the dice aren't loaded, and they're fair, they'll expect to see it. That's fine. That's interesting, right?

It's useful if you're gambling. But for scientists and statisticians, it's the opposite of what you want. It's the opposite of what you want is, I've seen this result. I've gone and done a trial of a COVID vaccine, say, and in people I gave the placebo, 10 of them got COVID,

but the people I gave the actual vaccine, only one of them did. I could tell you how likely I would be to see that if there wasn't any effect, using the same maths as the dice thing. But what we actually want to know, isn't that? What we want to know is how likely is it that my vaccine works, which is the opposite, the exact opposite question, not how likely am I to see this data given a hypothesis,

but how likely is my hypothesis to be true given this new data? Sort of the history of probability up until the sort of history of the study of probability, I suppose, up until Bays, and that's sort of mid-8, 18th century, up until that had been very good at answering the how likely am I to see this data hypothesis, and hadn't really worked out that there was a difference between two questions. Bays was the first, I think, to say, this is how we answer the other question, the question that if you're a scientist, you really want answered. And really the fundamental insight is that you need to add that new information, the data, to the information you already had.

So what you might say, your existing beliefs, what he called your prior, well, what statisticians now call your prior probability, but it's basically what you already thought. prior probabilities, base rates, or stuff. It's a fancy term for, this is what you previously thought, right? And that transforms it. That allows you to turn it into this is how likely something is to be true, given new information. The trouble is, and the reason it's been really controversial for the last, like, 250 years, how long it's been, the reason it's been really been a deeply controversial thing and the stats wars between Bayesian's and the other guys, who we call frequentists,

that have been really tasty. They get really crossed with each other. And the reason is because when you use Bayesianism, you're admitting it's all subjective. Like, it's your best guess. Now, that's not the same as saying it's all made up and it's pulled out the air. You know, your best guess can be better and worse and that sort of stuff, better and worse at predicting the world. But nonetheless, that's the reason it gets really controversial. But fundamentally, that's what Bayesianism is.

It is using pre-existing information, your best existing guess, adding that to the new information you've got and using it to tell you how likely something is to be true, not just how likely you are to see something if you're hyperboiseise. is true. And you got a little bit ahead of me because I was going to ask you about the stats war. We can come back to that an enormous length. Yeah. And obviously, I'm sure that this is going to be another controversial thing to dive into. But what is it that makes Bayes arguably better than the Frequenist approach? Why would people choose that over the other option? Most scientists, right? Most of science is carried out using the sort of statistics we were talking about the pre-base, if you like, version of statistics. You might have heard people talk about statistically significant results or P values and all these things, right? A statistically

significant result just means you would be unlikely to see this result by chance. You'd only see three sixes on a fair dice one time in 216. You'd only see a statistically significant result. I'm doing big air quotes here, which your listeners won't be able to see, but you'd only see that if one time in 20 is the traditional method, you know, the yardstick they normally use. That doesn't mean there's only a one in 20 chance that it's false, that it is a coincidence. You know, it means, I mean, you could do, and people do, and this is a problem with scientific statistics, you could just do 20 tests, right? You'd expect to get one coincidental result because that's literally what it means, and then you don't publish the 19

that don't come in. That would be a very simple form of what we call P hacking, probability hacking and making it. So, and that is exactly what that means. What a basium would say is, that's not answering the question you want. What you want to answer is, How likely is my hypothesis to be true? Not just, is it unlikely, am I unlikely to see this result given hypothesis? And they would say, to do that, you need to use prior probabilities. So if you do a, I don't know, a study of are people psychic? Can people psychically predict the future or something?

Then if you see a one in 20, you know, well, this guy predicted, you know, saw what the next card in the pack was going to be, slightly better than chance. You only see results like that one time in 20. Instead of saying, well, in which case we should say that is probably true, you'd say, hmm, what's more likely that the guy's actually psychic or that there was a one and 20 coincidence? And that's what Bayesianism lets you do. It lets you sort of include prior probabilities, your actual estimates of how likely things are to be true. What frequentists would say,

and it's not a crazy objection, is you're bringing subjectivity into science, you're bringing dogma into science. You're saying, I don't believe this thing to be true. Therefore, I'm going to be less likely to publish the results about it. Whereas what the frequentists would say, well, no, you get your unlikely result, you publish it. If it's unlikely, fine. New studies will come out and they'll show that it was wrong. Now, both sides have some merit. But from my own, that is just purely my own point of view is like, and bear in mind, I'm just a journalist who writes, who's interested in these things. I'm not a statistician. I'm not a, you know, I'm not a scientist, but I'm just a guy who's written about it a lot and find it interesting. My own feeling is, firstly, the Bayesian system, it avoids a lot of the problems that do lead, I mean, I'm sure you've covered on this podcast. The replication crisis in science. Yeah, the idea. that a lot of what we thought was true, especially in social sciences, psychology, things like that, they haven't stood up to scrutiny because a lot of the results have been based on bad statistics, basically. Things like peahacking, which we're just talking about. And that's not that Bayes theorem completely avoids that, using Bayesian methods, completely avoids that, but it takes out

some of the incentives, the bad incentives to do it, and it avoids some of the methods for doing it. So it does have some advantages. And also, it's just sort of more aesthetically pleasing. It does everything in a sort of neater way with everything included in the original maths. If you want to work out how big something, you can just say, I think this is X percent likely to be true and you can draw these things. And I think the effect is this big. And it's all sort of included in the system. Rather than in the frequentist bit, frequentist version, you've just got the, this is how likely I would be to see this result. And then all the other stuff, you have to sort of make ugly bolt-ons to make it work. And it's a bit sort of ad hoc and fixed together and sort of jerry-built,

you know, doesn't mean it's bad, but it's just not quite as pleasing somehow. Those are the sort of, I hope both frequentists and Bayesian's would say I've given their side of the argument of fair hearing there. I'm sure you have. I trust they'd all agree on that. Obviously, we're talking about this from the perspective of studies and more serious research, but how can the theorem be applied by the average person going about their day? Well, there are a couple of angles to this.

Firstly, what we're talking about here, Bayes' theorem, is not just a useful, statistical tool in science and things. It is literally the maths of how you predict the future, of how you work out whether something is true. Obviously, we can't go around doing base theorem all the time on whether or not the shops are likely to have like orange squash. No one's going to do that. But when our brains incorporate new information to our existing information, the extent to which they're getting that right is the extent to which they are approximating base theorem. And the extent to which they're getting it wrong is the extent to which they're not. It's the maths of it. It's like thermodynamics explained.

car engines. You know, it's the same, it's the same relationship, right? It's not that cars are literally running on idealized carno perfect heat engines, but to the extent that they work, they are doing the thermodynamic work. And it's the same as this. Insofar as brains are working, as far as anything is doing, then we are approximating base theorem. So, yeah, we are constantly using it. Now, that's not to say you're going to be, like I say, you're not going to be going and running base theorem when you're going, trying to make, making everyday decisions about stuff. But I think it's useful to keep in your mind, a lot of the time, I think we spend our time saying something is true or it isn't true or something is going to happen or it isn't going to

happen. We don't have to say X will definitely happen, Y will definitely happen. We can say, and then when new information comes in, we can move our probability estimates up and down. We don't have to doggedly defend things and say, this is what I believe. So, you know, and that's true of sort of political or other sort of beliefs as well. I think that, I don't know, trying to find some relatively uncontroversial thing that won't get people angry, but, you know, I don't know. I think tax and spend is better than libertarianism or whatever, you know, instead of just saying, this is just what I believe. You can say, this sort of set of beliefs makes sense to me.

I'm 95% sure that it makes more sense than the alternative. But as new information comes in, I can shift my beliefs. I can change. I don't have to just sort of stay in one place and then either reject or accept new information. I can add it to my existing beliefs. I think that's really important. I think there's also another thing, which is that it's a reminder that our beliefs should be connected to the world in some way.

They should make predictions about the world. When I say, I don't know, The example I always use and it gets controversial is cancel culture, right? There's an awful lot of people, maybe it's probably a five years old, out of date now. This is people arguing about in 2018? But anyway, you know, people say, is cancel culture real, isn't it real? They get really, really angry about it, you know, and have these big rows online. I don't know what, you know, to some extent, what are they arguing about?

Like, you're not arguing that this particular guy did lose his job over something he said. Maybe you're arguing about debating whether it's a good thing or not. But if I say, yes, it is real or it isn't real, will I change anything I predict about the world? Does it make any difference to how I predict? And if it doesn't, then maybe we're just arguing about a label, right, rather than about any beliefs about the world. And I think that's the point of base there. My prior probability is a prediction about the world.

I think this will happen. When new information comes in, if it's very different from my prior probability, then it will shift my belief. And my prediction was, to some degree, wrong, and I have to update it. If my beliefs are just statements, they don't have any predictive value, then they're not doing any work. And maybe we can just sort of not argue about that and argue about something else.

So I find that quite a useful sort of way of looking at the world. And this might veer back into the controversial, so I'm sorry if I drag you through the mud here. No, don't worry, don't worry. Obviously, two of the big issues on the internet are misinformation and conspiracy theories that I think seem to pop up more now than ever. Can the theorem be used in a way to better understand that and separate likelihoodness and truth? I think so. So one thing I always get annoyed about when we're talking about conspiracy theories is that there's

a sort of tendency to assume that conspiracy theorists, people who believe these things that we don't believe, are somehow mad or alien or they're different brains to us. Whereas we keep showing them these scientific studies that show that vaccines don't cause autism or whatever, you know. But the thing is, out of base theorem, this falls out very naturally, right? If you have very different priors on something, then as in very different strong prior beliefs, then new evidence will be incorporated differently because you are not just choosing between either vaccines cause autism or the scientific evidence is true. There is always other hypotheses you could have. So, like, in that

case, I mean, an example I use in the book, imagine that someone showed you evidence. They said, do you believe in psychic powers? He said, well, no, I don't think psychic powers are real. But then they showed you evidence. Look, this guy is constantly outperforming chance. When he predicts what's the next card is in the pack, whatever. There you go. Checkmate, you know, I've proved you wrong. You're not just choosing between those two options.

The third option, which is this guy is a fraud. Right, he's cheating somehow. No amount of magic tricks you show me, you'd have to be incredibly good evidence to convince me that it was really magic or really psychic powers and not just a conjuring trick, you know. And so my probability mass, my sort of best guess moving from psychic powers aren't real to psychic powers are real, my best guest moves from psychic powers aren't real to psychic powers still aren't real, but this guy's a fraud. If they really strongly believe that and they get new evidence coming in that shows, look, look, actually this scientific study shows and this BBC news report off the back of it, says and all the sort of stuff, they could move their best guess, their probability mass, onto, and now I believe that vaccines don't cause autism, but it's actually given their prior beliefs, perfectly rational to move it onto vaccines cause autism, and the BBC and the scientific

establishments are lying to me. They're doing exactly what you're doing in the psychic thing, and it falls out of their pre-existing prior beliefs in totally the same mathematical way, and we don't have to go around saying, this guy's a mad, weirdo, he just say he's got a different set of beliefs to us to me, and therefore he ends up incorporating new information a different way. And we don't have to sort of say that these people are stupid and wrong, and we are the owners of the true information. And then again, we can say, but which of these sets of beliefs predicts the world better? And I would argue my set of beliefs in which vaccines don't cause autism and you don't see a big spike of autism cases after the MMR vaccine is introduced

or whatever, that predicts the world better. And I think my set of beliefs are better than theirs. It's not that their brain is going wrong in some catastrophic way. They've just started out the different set of prize. I find that, I mean, you can see how that could be a little disconcerting because it becomes rational for some people to believe what we would consider conspiracy theories. But it also means we don't have to just write off a decent chunk of humanity as defective or alien and just say that this is just brains doing their normal brain thing in a slightly misfiring way. I prefer that. Some people never stop being part of what's happening around them. They stay curious, engaged,

connected to ideas and to others. That's what defines life at Villa Gardens in Pasadena. It's a community shaped by conversation, culture, and a shared sense of curiosity. So when the conversation turns to what's next, it doesn't feel like stepping away. It feels like staying exactly where you belong. Explore your options at VillaGardons.org, a non-profit life plan senior community within the Front Porch family. This podcast is sponsored by Name, Audio and Focal. With over 100 years of combined expertise,

Name and Focal have been bringing music to listeners, just as the artist intended. Since day one, this mantra has shaped every innovation in high-fi design, technology and acoustic engineering, balancing craftsmanship and tradition with pioneering thinking. Name Audio pushes cutting-edge technology to ensure digital precision whilst sustaining Pratt, pace, rhythm and timing, the elusive quality that makes music feel alive and gives it emotional texture. Today, in partnership with French acoustic

specialist focal, name audio creates systems that deliver exceptional sound, and unforgettable listening experiences at home. Try it for yourself at a focal powered by name boutique. Visit focal powered by name.com for more information. Going quite a long way away from conspiracy theories and different points of view, for a lot of people, I guess the future in general is quite anxiety-inducing, it's scary, it's a lot to deal with.

Can the theorem be used in a way to quell those concerns about the future holds, even just in a sense of an event someone's going to, like a surgery or a job interview or something like that? I mean, I think it can, in that you know that the future is usually like the past. It depends on the individual thing, right? If you're using base theorem to predict the likelihood that you'll be burnt to death

if you jump into a volcanic crater or something, I don't think it'll be very reassuring. It will probably tell you quite confidently that you probably will burn to death. I'm not going to work out how exactly I'd do the maths, but, you know. But for example, if you're looking at like cancer tests and all these things, then quite often you might find that it'd be reassuring to know that if your cancer is rare, then even a positive result might well not mean that you've got it. So that could be very reassuring.

I mean, I suppose I would say when used correctly, it is reassuring when it is appropriate to be reassured. It is not reassuring when it is not appropriate to be reassured. And sometimes things are scary and it is appropriate to be a little bit unnerved. So I can't give you the full answer you want there, but a bit of both, you know. Sure. And in your book and please stop me if I am putting words in your mouth here, but you talk about the theory as, I guess, an almost catch-all solution to a lot of predictions. often theories, algorithms, anything like that, are given this sort of omnipotent ability to solve anything they relate to.

Do you think there's a, I guess, a bit of a risk there when you look at theories this way? Oh, yeah. I mean, obviously, if I can say this book is about the one theorem you need to know to understand the world, then that tells a lot more books than the, I hope it does, than me saying that this is one theorem which will help you understand this very small part of scientific statistics.

That said, I do think it is true, right? I think, like, you can use Bays to describe what AIs are doing, right? You can use it to describe how humans make decisions in the world. You can use it to describe what science is doing. I don't know if you've discussed super forecasting on this podcast before, but the people who do the best at making predictions of the future, they outperform CIA analysts and all this sort of stuff. They do it normally, and almost explicitly basing, sometimes literally explicitly running the maths, sometimes using it as a sort of mental framework for this is what I believe before, where do I find a prior probability before I add this new information in? And they're the ones who do best. right. So I do think it is an amazingly powerful tool. I genuinely think, well, and I don't think this is me having a breakdown. I genuinely do like, but the more I've learned about it, the more I see it everywhere I look, you know, that all right, you can describe that in a Bayesian way. There's a guy who I quote the beginning of the book who says, the general rule in psychiatry is if you think you found a theory that explains everything, diagnose yourself with mania and check yourself

into the hospital. And I sympathize with that, right? There's a lot of people going, you know, I've made a perpetual motion machine. I've worked out the start of the universe. My theory of everything does everything. I've combined quantum gravity and relativity theory and all the sort of stuff. And most of the time these people have basically had developed mental illness. So I should be nervous of the idea that I see base theorem everywhere. I do think it is an extraordinarily powerful and ubiquitous thing, which, you know,

the human brain can be described in this very Bayesian way. Like I said, AI, all these things. So I think, yes, it is a risk of overstating the case, but I think this is one of those very rare times when genuinely you can make the claim it applies to everything. And I hope I make that case in the book. So either Baysfram can be applied quite heavily or you had a breakdown in the book. It's just not quite clear which one yet. Well, yeah. I leave it to readers to decide.

Perfect. Okay. And it was something you just touched on a little bit there. And I think it's something that could be quite an interesting view is, you know, as you're well aware, AI and computer. They're growing rapidly right now. It's used more and more all over the world. Baysfirm was originally, you know, discussed in the 18th century. Is there a risk that as, you know, computers and AI take more control of everyday situations that it's harder to make human predictions based on these sort of situations?

So, I mean, there's a lot of ways I could answer that question, and I'm going to try a couple of them. Right. AI won't make humans worse at predicting in its own right. AI will make humans better at predicting probably because you can use AI to do the prediction for you. Yeah, it is amazing, right? The extent to which an 18th century equation describes what AI is doing. And it is very, I mean, there are explicitly Bayesian AIs, but all of them work on Bayes to some extent.

I mean, the example I sometimes give, imagine chat GPT or something, you ask it, how are you? It answers, I'm very well, thank you, how are you? It's not because it is very well or cares how you are. It's just that its training data has taught it that the sort of thing that comes after the string, How are you, is something like, I'm very well, thank you. Before you're given any training data, you asked how are you,

it would have presumably given you a string of complete gobbled to gook, because it's best guess of what follows after that is spread out over all the possible combinations of letters and words that it could be. But after you fed it, you know, trillions of bytes of data from all over the internet, and it's read the phrase, how are you, you know, 10 million times, whatever, through simple Bayesian methods, it pushes all its probability onto a few responses like, I'm very well, thanks, fine, cheers, that sort of stuff. It is purely, it's absolutely a

Bayesian system. They'll probably use different algorithms because Bayes is very computationally demanding, but it'll be, what it's doing is approximating base. So AIs are absolutely Bayesian. You asked whether it also would make the future harder to predict. This is more the topic of my first book, but I'll still, which was about AI and how it's going to change the world and whether it'll kill everyone and all that sort of stuff, but it's still relevant, right? Which is AI is just another bit of the world speeding up, you know, the economic growth was flat for tens of thousands of years before the Industrial Revolution and then sort of zoomed up and everything, we're getting faster and faster and AI may well be to something that makes the world change faster in a year

than it did in a decade a century ago and did in a century, you know, so part of the world speeding up and becoming harder to predict, you know, the old singularity thing, but it may be that because of AI, the world becomes harder to predict further out, that may well be true. But that's a subtly different question from whether AI itself will make things harder. And I think actually using an AI will make that will be, you know, you can see it now. It's doing amazing things about predicting how proteins would fold and all the sort of stuff which will help us in drug discovery. It's, it's just becoming an incredibly powerful tool. And obviously you've spent a long time researching this. As you said, it's become a bit of a hobby, a bit of a view for the whole of life for you.

While you were researching the book and since then, what has been? been, I guess, the thing that's surprised you most, the thing that's interested you most, the thing that's stuck out to you most about this theory? I think it's the seeing it everywhere. I mean, an example from late in the book, do you see it from the really quittidian, normal everyday things, spam filters, right? Spam filters are Bayesian. I mean, they're explicitly Bayesian.

They're the most common, or one very common type is the naive Bayesian spam filter, you know. I think it's something like 50% of all emails, a spam that are going around the world. I can't remember the actual number, but it's something like that. So that's your prior probability, right? 50%. Then you get some new infidant. information and you know that like if the word penis enlargement appears in it, that appears in 80% of spam emails and only 5% of non-spam or something, you know. And you use that data and you

update and the spam filter just uses that to push up its probability of how likely a given email is to be spam or if you know if it says, you know, buy now or whatever. Then when it gets above like 80% likely to be spam or 90% or whatever, it says this is probably spam, kick it to the spam folder. That is a very normal, quotidian everyday thing in which space. is absolutely crucial. A big version, evolution can be seen as a Bayesian process, right? The frequency of a gene in the population

is a prediction of the environment that that gene will find itself in, like a gene for sharp teeth is predicting that it'll be an environment full of animals to eat, right? Or, more specifically, it'll be in an organism that has the sort of legs and digestive system

that will allow it to catch and eat those animals. It would be no use if it was in a house plant, right? And then each, then, and that's your prior probability is the existing frequency of the gene in the population. And then when, whether or not that gene survives, there's a bit more information, which, you know, whether it is replicated into the new generation, is a bit of information which updates in a Bayesian way, whether or not that was the right decision. And it is absolutely, it's a very slow, noisy Bayesian process, but evolution, it can be described in a Bayesian

way in exactly the same way as a spam filter or Bayesian statistics and science. And it's just this one equation which does all these different things. And you see why I do worry sometimes that maybe I'm having a, developing, I've probably found a theory of everything. But it is, but it is, and it does. And I find that absolutely fascinating. The idea that you can use this one equation to describe the human brain, AI, spam filters, whether or not the psychological science, you know, all these different things. And it's just one equation.

I find that absolutely amazing. And obviously sometimes it doesn't help to try and do the maths on each of these things. It is just a sort of a useful model, a background thing, but it is fascinating, I find. Thank you for listening. to this episode of Instant Genius. That was Tom Shivers on Bayes Forum. The Instant Genius podcast is brought to you by the team behind BBC Science Focus magazine, which you can find on sale now in supermarkets and newsagents,

as well as on your preferred app store. Alternatively, you can come and find us online at sciencefocus.com. This podcast is sponsored by name, audio and focal. The texture and emotional depth of music can be lost through digital sources. or poor signal. Name Audio believes you can have digital precision with analog warmth. Alongside French acoustic specialist vocal, name creates high-end audio systems combining innovation with craftsmanship so you can listen to music, just as the artist intended. Discover more at name audio.com.

Ambition comes in all shapes and sizes. At First Citizens Bank, we roll with your goals because we're built for what you're building. Fit for your ambition for citizens back.