Instant Genius - How to see through misleading numbers and statistics

Episode Date: February 24, 2025

Whether it’s from our friends, workmates or via various media platforms, both social and traditional, these days we’re all faced with a never-ending bombardment of information expressed through numbers, percentages and statistics. But how exactly should we go about interpreting them? In this episode we catch up with Prof Sir David Spiegelhalter, a statistician and science communicator based at the University of Cambridge. He shares some simple advice to help us understand how the risks and benefits of various lifestyle factors such as exercise and diet are presented to us, explains how framing can be used to make any number look big or small, and tells us what everyone, including scientists, could be doing better when we speak about numbers. Watch the episode here. Learn more about your ad choices. Visit podcastchoices.com/adchoices

This podcast is sponsored by name, audio and focal. Streaming has made music more accessible than ever, but true listening is about more than ease. It's about quality. British audio experts name audio, alongside French acoustic specialist focal, combine handcrafted tradition with cutting-edge innovation and high-end materials,

delivering digital precision with analogue warmth. So you can experience exceptional sound at home. Music just as the artist intended. Visit name audio.com. to learn more. Hello and welcome to Instant Genius, a bite-sized masterclass in podcast form. Every Monday and Friday, you'll hear world-leading scientists and experts talking about the most fascinating ideas in science and technology today.

Whether it's from our friends, workmates, or via various media platforms, both social and traditional, these days we're all faced with a never-ending bombardment of information expressed through numbers, percentages and statistics. But exactly how should we go about interpreting them? In this episode, we catch up with Professor Sir David Spiegelhalter, a statistician and science communicator based at the University of Cambridge. He shares some simple advice to help us understand how the risks and benefits of various lifestyle factors,

such as exercise and diet, are presented to us, explains how framing can be used to make any number look big or small and tells us what everyone, including scientists, could be doing better when we speak about numbers. So, David, thank you so much for joining us. Oh, great pleasure. It's sort of nerve-wracking, but a pleasure. So today we're obviously going to be speaking about numbers and statistics.

So a topic a lot of people will think is maybe a touch dry. But I think in the next 30 minutes or so, you'll change their minds. Well, possibly. I mean, I'm a nerd and always have been. And I really love numbers and trying to explain what they mean and what they don't mean. And I actually think it is interesting. and should be interesting because, you know, if we look at just being a citizen today, we're bombarded by claims on social media and from friends and contacts and everything,

which are often based on numbers. And so I think everyone needs capacity for dealing with them, not to shy away and say, oh, I can't manage numbers. No, you should be just as embarrassed about saying that and saying you can't read. So let's look at one of the hats that you wear, which is this notion of risk. So you read all the time in the news, doing such and such, raises the risk of this, if I'm eating bacon or something, you know. So they're getting things wrong. You know, what are they getting wrong?

Well, unless they're getting wrong. It's the communication is there for purpose. And so whenever I read a news story about or X increases the risk of some, something bad Alzheimer's or cancer or something like that, I'm immediately skeptical because, and not even, even before I look at the evidence, I just think, You know, what are they trying to make me feel? They're just, you know, there's so many clickbait stories, and they're trying to get me anxious a bit to read the story and they say, whoa, do you know what? You know, if you have yoga, it'll give you asthma or something like that.

So these are great stories. They're obviously very popular because the media covers them all the time. But if you start looking beneath them, you know, there's two things you should look at. First of all, actually, is this a big number? Because if someone says increases the risk, that's almost meaningless. Because if the risk is incredibly low to start with increasing it,

is still very low. So you want to know, well, actually, does it increase it by an important amount? And two, an important risk is one of the first things to ask. And the second thing, of course, is do you believe the number anyway? So let's have a look at that then. What is an important risk? You know, is there a level that we can determine? Yeah, well, the point of you, the standard thing is that when someone does an epidemiological study

of what you eat and then what happens to you, what comes out of that study is a relative risk. It's how much that exposure to the bacon or the alcohol or something like that raises the risk proportionate from what it was before. So we hear that, yeah, eating a bacon sandwich every day or something will increase your risk of bowel cancer by 20% at one fifth increase. Now, it's a relative increase. And that can sound quite frightening. Why, God, you know, and actually I do believe those numbers. I actually do believe those. So I think it's based on some reasonably consistent science now.

But the crucial thing, well, 20% of what? And so in order to interpret it and to know whether we actually should be anxious about this or change our habits, we have to know how important it is. And so we have to know, well, you know, is this something that we should care about? And so we need to know the baseline risk, the absolute risk, rather than just the relative risk. And so in fact, you know, out of 100 people, right about six, we'll sadly get bowel cancer during their lifetime anyway, if they don't eat bacon. And so that's what we're talking about is a 20% increase. over six percentage points. Now, that's using percentage in two different ways. One is a relative increase and one is an absolute number of percentage points. I don't know any journalists who can do this. They tend to get very confused.

This one can. This one can. Exactly, yes. You're unusual, I think. I'm sure you can. People find this really challenging, to use percentages in two different ways. But it's one of the basic ideas

of proportional reasoning that people try to teach in schools about using numbers. Journalists tend not to be very good at it, But I think have learned increasingly that these should be asking about the absolute risks and then communicating it in the way that's been really explored deeply and shown to be effective as a communication device is just to say, what does it mean for 100 people? And so you could say for 100 people who don't eat bacon, six will get bowel cancer during their lifetime, unfortunately. And 100 people who do eat a bacon sandwich every day, we're talking about a 20% increase over that 6 percentage points, which takes it to just about 7 percentage. points. And you can illustrate this beautifully by showing 100 little dots or little people and

colour them in and six become seven when all those hundred eat a bacon sandwich every single day of their lives. So they're all stuffing this thing in their gob. There's a great big greasy sandwich in their go. And one more out of those whole hundred will get, you know, we'll get bowel cancer during the whole lifetime. At which point you might think, well, I don't know, pass the brain source. So it's not actually that huge. From an individual, perspective. Now, from a national perspective of the whole country, that is important. Process meat does produce a measurable number of bowel cancers in the country. And so from a public health perspective, you would like fewer people to eat a lot of processed meat, definitely. However,

from an individual perspective, it might be quite reasonable to reject that advice and say, well, I really like it. I'm willing to trade off the enjoyment of eating this stuff for this small increase risk of getting bowel cancer. And so, and I think, you know, well, I think personally, I think, well, when I, because I do believe these numbers, I've cut down on my processed meat, but I still love a bacon sandwich, occasionally. Me too. So how about these neat numbers? Like, so recently there was a study saying that smoking a single cigarette takes 20 minutes off your life. How have they come to that number? Yeah, I've done, I used to do a lot of calculations about this. I got to 15 minutes when I did it years ago. So this isn't a new way to communicate

stuff. You could say, you know, I always said the first drink puts half an hour on your life and then the second one takes it off again and the third takes off half an hour. So it goes medicine, poison, poison, poison, poison, poison. And so lots of other things. So, you know, the first, various other things, but put, you know, your first 20 minutes of exercise a day, basically puts an hour on your day. And then after that, after the first 20 minutes, you know, every half an hour of exercise puts about half an hour of your life. So you better enjoy it because that's the gain you're getting by doing it. Now, as you said, it's a bit nonsense because we've got no idea an impact of a single cigarette or a single half hour. Now, what we're talking about is averages

over habits. They're gone for years, average it over large numbers of people. What is the linked change in life expectancy? And then we're reframing it in terms of the effect of of an individual exposure. Now, I quite like it, but I've stopped doing it because I think that people might take it literally instead of really just another way of framing an overall average risk at a population level. So let's have a look at averages then. I think this is really interesting because there are several different ways of expressing

an average. So what are they and when are they best applied? Oh, averages are quite tricky because I don't like that. averages. People, you know, think the statisticians are obsessed with averages. No, they're not. They're obsessed with variability. So just talking about an average actually can be very misleading. And, you know, so if you think of, you know, how long, I don't know, let's take a simple thing, you know, Darren Brown on television flipping a coin 10 times and it came up heads every time. And then it turned out that he had been filmed for nine hours flipping it before he finally did the 10

heads in a row. And you can work out how long do you expect flip these coins until you get 10 heads in a row. And you get on average, that's going to be about a thousand attempts because there's a one and a thousand chance of getting 10 heads in any particular attempt. That's one average, which is this mean average, if you did it many time. But then you've got another sort of area, you know, which is the mode of the most likely time is the very first time you flip it. That's the most likely time to first flip 10 heads. It's the very first time. Because if you think of it to get it first on the second attempt, you're going to have to have not got it on the first attempt. And so the probability of getting it the first time on the second attempt is lower

than the probability of getting it at the first attempt. So you've got the mode. And then you've got median, which is, you know, 50% of the time out of a lot of people who's halfway along the spread of the distribution. So these, you know, the median and the mean, often, if you've got a nice sort of symmetric curve of a distribution, they're fine. They're about the same. But if you've things like income, they're massively different because the mean income, if Bill Gates walks into the room, is suddenly shut up. The median income stays pretty well exactly the same. So it depends on the shape of the distribution and a lot of distributions are not nice and symmetric. And so you have to be really careful about whether you're talking about the mean, the median or the mode.

So continuing on from that, you talked about Bill Gates there, like one of the richest men in the world. So let's have a look at outliers. And so something that I spoke to you earlier about, was what I call Uncle Norman syndrome, where people say, well, my uncle Norman ate bacon and eggs. He lived on bacon and eggs. He smoked 20 cigarettes a day. Live till he was 110. Yeah, yeah. There's always one.

And there is always one. Because someone's got to be in the tail of the distribution. That's the point. That's why averages can be so misleading and why statisticians are interested in variability. What is the spread? And some people may live absolutely blameless and enormously healthy lives and die young. And other people will, as you said, but will be Uncle Norman and live for a long time.

Good old Uncle Norman is what I say, how lucky he was. And I like the idea of luck. Richard Dole, one of the greatest epidemiologists who co-discovered the link between lung cancer and smoking. Said, whether you get cancer, which is matter of luck, matter of luck. Some people will or some people don't. However, the chances are changed by how you live.

So Uncle Norm was extremely lucky to do if he did that. because all the odds were against him. And so you'd expect someone like him to have died, you know, smoking 20 a day, takes about eight years off your life. If he's a slob and he eats his bacon sandwich, that's going to take a few more years off his life. You'd expect him to die younger. But it doesn't mean everybody.

You know, people, there's huge spread and variability. And that's great and that's fine. But you can't use that as a basis for policy or even, you know, people might use it, of course, oh, well, I'm not going to take any notice of health advice. Uncle Norman lived till he was 110. Yeah, fine, but you're very unlikely to be as lucky as Uncle Norman was. So let's have a look at luck then, because I know you've written about this. It's really interesting. There's actually several different types of luck. Yeah. Yeah. So what's that? I love it. Well, first of all,

I don't think of luck as being some objective force out there that makes things go well badly for us. I don't believe in that at all. I believe it's just a label we give afterwards to things that happen to us, which are outside our condition. control were rather unpredictable and that had an impact, either good or bad. And so it's really useful. Just, you know, is the operator's way that chance hits us. There's another way to view it. Reading about this and writing about it, it's changed my life. It's changed my perspective on life.

This idea of what they call constitutive luck, who you are born as. We appear in the world. We've got no control of who we are, who our parents are, what period of history we're born, what social class were what into, our genes, you know, where we are. nothing. We just appear. We didn't ask be born. Here we are. By this extraordinary, odd sequence of circumstances we're born at all. I mean, I could talk about my conception, but maybe this is not an appropriate forum to do this. But, you know, the point is the fact that we're here at all is just the result of a whole lot of micro contingencies. Like all these things that could so well not

have happened. But here we are. And who we are born as has an enormous influence on the rest of our life. You know, we have our privilege or lack of privilege is huge. I mean, I was a boomer. I born and a family. It had no money, but we had enormous access to post-war welfare, you know, provision. And I went to a really good, I went to a good, free grammar school. I got it's free education. I went to university. That was free as well. Then I was at a period when I could pretty well choose what job to have for life with a final salary pension. And when they, meanwhile, watching all house prices go up. I mean, it's staggering, over which I had no control whatsoever. And yet, it's been hugely influential in my life. So I think people underrate just how important it is

who you're born as. They like to think that their successes are due to, you know, all their personal characteristics and their qualities and their hard work. Well, yeah, a little bit. But that's not the only type of luck. The other type of luck is one that's called circumstantial luck, which is being at the right place at the right time or the wrong place at the wrong time. You know, like being on a plane that gets into trouble or, you know, just having to be in a car where someone else hits it. You know, just through no fault of your own, you just happened to be in the wrong place at the wrong time. Example I give in my book is my grandfather who ended up as brigade gas officer in 1918, just north of Passiondale, one of the most dangerous jobs in the most dangerous places you could ever be.

You know, that's where he ended up. And he lasted three weeks in the job, which I thought was pretty good going. before he got blown up. But he wasn't killed. Otherwise, of course, I wouldn't be here, by definition. What he had was very good outcome luck, which is the final bit of luck, which is just how, at that moment, just how things panned out for you,

which could be very unlucky or could be very lucky, while you are in those circumstances. So the constituted circumstantial and outcome luck, I find very helpful when you're actually trying to take apart when people talk about good or bad fortune that they've had. Lots of places can expose you to identity theft. Oh, no. That's why LifeLock monitors hundreds of millions of data points a second

for threats to your identity, which is way more than anyone can do on their own. If we find anything suspicious, like new loans or changes to your financial accounts, we alert you right away, all through text, phone, email, or the LifeLock app. Get the alerts that could make all the difference. Save up to 40% your first year at LifeLock.com slash special offer.

Terms apply. There's a moment when you start to wonder, what's the right next step? Not about changing who they are, just finding the right kind of support. At Kingsley Manor, life stays expressive, connected, and full of character, shaped by people who have lived interesting lives and aren't finished yet. So it doesn't feel like a change. It feels like a continuation. Explore your options at kingsley Manor.org,

a non-profit month-to-month senior community within the Front Porch family. This podcast is sponsored by Name, Audio and Focal. With over 100 years of combined expertise, Name and Focal have been bringing music to listeners just as the artist intended. Since day one, this mantra has shaped every innovation in high-fi design, technology and acoustic engineering, balancing craftsmanship and tradition with pioneering thinking. Name Audio pushes cutting-edge technology to ensure digital precision whilst sustained. in Pratt, pace, rhythm and timing, the elusive quality that makes music feel alive and gives

it emotional texture. Today, in partnership with French acoustic specialist focal, name audio creates systems that deliver exceptional sound and unforgettable listening experiences at home. Try it for yourself at a focal powered by name boutique. Visit focal powered by name.com for more information. So let's look again at data reporting. So I know the O&S talk about certain values, trustworthiness, quality and value. So can you break those down for us? Oh, I know you can really get me going here. And I should say conflict of interest. I'm a non-executive of the board of the UK Statistics Authority, in particular on regulation committee, which oversees the work of the Office for Statistics Regulation. That produces the code of practice for statistics for this country. I always

used to say, good, this is one of the dullest documents ever, you know, if you went for, no, it's not actually. It's being revised and it's turning into really quite a good read. And it is, it's the thing that all official statistics in this country and an increasing number of non-official statistics are signing up to the code should adhere to. And it's got three pillars. And the first one is trustworthiness. Then quality and value, really important. But it's trustworthiness that I feel is the most important one. And when I look for all the time in everyone who's communicating. Are they being trustworthy? Because too often, those in authority, particularly scientists, say, oh, how can we get people to trust us? It's ridiculous.

People aren't trusting us. We're scientists. We know best. How can we get people to trust us? And I follow the philosopher, Orna Neal, who's been so influential on the statistics system, who says, that's the wrong question to ask. You should be asking yourself, how can we be more trustworthy and demonstrate it. How can we deserve that trust? How can we open ourselves up to be trusted? Not to try to force people to trust us. That's something they should only offer that up to us if we are trustworthy.

So this is, she's a Kantian philosopher, this is duty ethics, even before any idea of the, you know, utilitarian point of view of how can we actually get people to trust us? You should be having the duty of being trustworthy. And it's so simple. So I've, you know, worked on quite a lot about this. colleagues in trying to define what we mean by being trustworthy. Office of stats regulation has also got this very powerful idea of intelligent transparency of being open, of being honest, of being clear about what the uncertainties are in your evidence,

how good the evidence is, when you're communicating, not trying to manipulate someone's emotions, but giving a balanced view. You know, you're trying to inform people rather and try and persuade them as something. And so much communication is persuasion. And then actually, you know, taking a balanced view and, you know, pointing out, you know, if you've got something, the winners and losers, the benefits and harms, not taking a one-sided view. All these seem basic ideas. You certainly don't see them within when numbers get politicised and weaponized in society,

as so often they are. So even before looking at the quality of the number, one should be asking about, well, is this being communicated in a trustworthy way. The final thing that I'm fascinated by, which is, and which is becoming increasingly noticed as an important thing, is what we might call preempting misunderstandings or pre-bunking misinformation, to know and kind of try to know in advance how people might misinterpret what you're saying and use it wrongly. And then hit that hard.

You try to preempt the misunderstandings by knowing your audiences, by testing the materials, because it can be really surprising sometimes how people can misunderstand these things. Sometimes, of course, it's deliberate. It's disinformation rather than just misinformation. But sometimes it's genuine misunderstanding. So you need to preempt this. You have to be able to say, this does not mean X. It would be wrong to interpret this in this way.

So you can't just leave that to the audience. If there's a clear wrong answer, you can't tell them exactly what to think, but you can't say this is not what it means. And I think that's very important and challenging because it means. means you have to know your audience. You have to listen to them. You can't satisfy everybody. There'll always be people who won't believe you anyway. But our research has shown that, you know, and we did experiments with, you know, randomized trials on thousands of people looking at things like vaccines and nuclear power and everything like that, showing that if you do communicate in this trustworthy way, preempting misunderstandings, balanced, trying to inform rather than

persuade, overall, it doesn't make much difference in terms of how people trust the source. but there's an interaction. The people it does make a difference to are the people who are skeptical at the start about vaccines or nuclear power. They recognize when they get a one-sided message. And that's partly why they're so skeptical. Oh, we're just being told vaccines are safe and effective.

But if you give them a message that doesn't say vaccines are safe and effective, it actually says, well, vaccines can have problems, but they're safe enough and effective enough to use in some people in some circumstances. People aren't daft. They recognize that you're actually being open with them and honest with them, and their trust goes up. So you can increase the trust in skeptical people

by being open and honest and trustworthy. The flip side of that, probably, you know, as a bit of scientific research, I think is really powerful, the sort of thing good experimental psychologists can do with randomized trials. But the flip side of that, it means that when government communicators or any communicators are giving a one-sided persuasive message, they are actively decreasing the trust in the group they are trying to reach the skeptics. They're making it worse. There'll always be some people that they can never reach. But for the people who actually are listening, then they're making it worse by doing that, which I think is a really powerful message that comes from a scientific investigation of communication.

How about trustworthiness in science and the way that the scientific method progresses and the way that people understand it. So say we're told one day, eggs are bad for you, next day, eggs are good for you, eggs are bad for you, eggs are good for you. And they say, well, I just can't believe anything these people say. I better not say what I think about most of nutritional epidemiology, frankly, because that's what people see in the news. All is so clickbaity as we've talked about and so newsworthy. And they say they see, oh, constantly this is bad for you, this is good for you. coffee's bad for your coffee or give you cancer, coffee's good for you.

And all this nonsense. And frankly, you know, it's based on observational data following it. And it's not very rigorous science, to be honest. The quality of the evidence is low. The confidence and the conclusions should be fairly low. And so it's a shame that that is what science people do. Some of the worst science, you would say, is what most people see. And what they don't see is the proper scientific method, which is very much, not completely,

but it's very experimental. It's to do with setting up, having ideas, testing hypotheses, of small incremental gains, of disagreement between scientists, then trying to resolve it, to watch that self-correction going on. There are some claims because every sort of some people would do, and then people check it and it doesn't work.

We can see that has happened in psychology in so many areas. So that sort of self-correcting mechanism, people, on the whole, don't see very much because it's being portrayed in the media is it science says and sound says one and then it suddenly it says another. No, it's not. It's that somebody's done some study and making some outrageous claim. That's not what science says at all. Maybe people are starting to learn. Certainly journalists are starting to learn the power of what I might call meta-analysis or evidence synthesis where you don't just take one study,

you put a lot of stuff together and they really are grasping that, that that really is a level of evidence which is higher than most. But even then, if you put a whole lot of not very good stuff together, you get something it is not very good. of tangentially related to that is this notion of surveys. So this is particularly amusing when you get, say, a moisturiser and it will say sort of nine out of ten women said it improved the complexion within X amount, you know, a month or something. Then you'll see a little star at the bottom and it'll say sample size 47. Yeah, it really was nine out of ten. No, I know. People aren't daft. They can spot the nine or ten cats. When I was young

is cats prefer whiskers or something when they give two bowls of this stuff and I think people can spot you know when things are being done spurious for that so I'm not so concerned about the sort of idiotic surveys and also people are not you know

the other thing because people do surveys in order to plant a story in the news that's what a lot of PR companies that will do they'll do some crummy survey to show how much you know people like nuts or something like that is obviously funded by the Nut Council or thing So, and I, yeah, it's annoying, but honestly, I don't take much notice of it.

No, one of the big problems is, you know, surveys are incredibly important. You know, during COVID, there was an infection survey in this country, which is the envy of the world, where you actually knew how many people were infected with COVID. Nobody else knew because you were testing vast numbers of people at great expense and recording the results. People aren't answering surveys anymore. Response rates have plummeted. and that is really causing a lot of difficulties. You know, for example, all the employment statistics in this country,

you might think were based on statistics, on labour, you know, counts and, you know, actual administrative data. No, they're based on a survey. Right. And people aren't answering the survey like they used to. They're trying to make it simpler online and so on. And so, and a lot of GDP, you know, measures for growth and expenditure and things like that. It's surveys of businesses.

So a lot of the economic statistics in this country are based on surveys rather than complete enumeration and response rates are going down. And it's not just in this country, it's everywhere. So that's causing a lot of soul searching among official statisticians, which I'm kind of involved in, of to what extent can surveys be replaced by administrative data? And for example, you know, the migration statistics in this country were hugely politicised and incredibly important. And up to recently, they were based on, frankly, not a very good survey where people just got stopped coming into airports.

And I don't know if you've been stopped coming into an airport or a port. Once or twice. Yeah, and you ask what you're here and how long are you going to stay for and things. It's not a great way to actually work at how many people, migrants, you know, the migration patterns of the country. So that, again, is now being supplemented by administrative data from a variety of different sources. So that's official statistics is in a period of transition of moving away from the classic survey towards using the data that are just available. So this is what we now call big data, that sort of thing? Yeah, yeah, it is.

I mean, it's all the data. For example, working out expenditure patterns and things that they're rather than surveying shoppers to get the transaction data from major supermarkets and so on. So how about this notion of false positives, which I've heard you talk about, say, what is that and what traps can it lay? It's incredibly important. And, you know, they're important during COVID. I mean, classically, in COVID is when you get a test and the test shows a positive result, but yet you didn't have COVID. Because there's maybe something else you had or something you've done to make the test show positive. And that's a real problem because if something's quite rare, you know, if not many people have got COVID, then suddenly false positive.

If there's loads of COVID around, a few false positives doesn't make a difference at all. But if it's very rare, the false positives can start being a major part of the people who are reporting positive. And eventually you get situations where most of the results are false positives. And this happens quite a lot. Breast screening, for example, you know, most of positive mammograms are false positives. Because it's quite very rare and the test isn't perfect. And so that's why you always have to have a follow-up visit, of course.

And that's saying for a lot of screening devices that most of the. natural things that go triggered, go ping, are false positives. Because if you're looking, oh, say, if you're looking for a needle in a haystack, there's a lot of bits of straw that look like needles. Right. How about randomness, though? So people often just throw this word around everywhere. Yeah, yeah.

What does it actually mean? Well, it's very difficult. I mean, it's like a real struggle. You know, what is randomness? I mean, loosely, it just means unpredictable. But if you're going to use it at all technically, it usually means that it obeys some sort probability distribution. We know, for example, I know, lottery balls, you know, going bouncing around and they drum and they fall out, you know, is that random? Well,

if it's random, then we'd expect all the numbers, they wouldn't all come out equally all the time. But eventually they'll settle down to be roughly equal chance. There's an equal chance of every number coming out. And we can check that. And I did the recent analysis, just checking all those numbers. And yeah, it obeys the rules of randomness. You know, and similarly, random number generators that computers use, you know, for example, premium bond draws, there's continual tests that they are in fact random because most random number generators don't have any, in a sense, intrinsic randomness in them at all. They're completely deterministic. They're just multiplying up two big numbers together and chopping off the last few numbers. And so, you know,

so most, if you use a random number generator in your computer, it's not random at all. It's just, it's completely, it's just an algorithm that produces a number that does have a uniform distribution, but you could run the algorithm again. You get exactly the same number. There's no intrinsic, you know, in a way, uncertainty about it. And similarly, lottery balls bouncing around. In fact, that's, you know, not random because it's all, they all just, they're just balls bouncing around.

They obey Eutonian mechanics. You know, you could know exactly what's going to happen. It's just so staggeringly unbelievably complex that it's completely unpredictable, according to this probability of distribution. So randomness is a sense, is a useful term, but it, it, we, really essentially means unpredictable. It doesn't necessarily mean what I'd call stochastic, that

there's a genuine, irreducible uncertainty about the situation. How about this idea of people misinterpreting data based on, well, I was going to say correlation and causation, but that's just one. That's just one. No, people are misinterpre. It's difficult. It's difficult.

You know, people ask me, you know, I've been working this whole era for 50 years, and people ask me, well, why does everyone find probability and statistics so unintuitive and difficult. And I just say, well, after decades of study, I've finally concluded it's because it really is unintuitive and difficult. It really is. Yeah, yeah. Our brains are not designed and evolved to handle probability and randomness in a, in a way sense, mathematically correct way. And I think there's a couple of reasons for that. People have pointed to the fact that we tend to overestimate rare events. We tend

patterns where they don't really exist because that makes a lot of sense. If you're in the, you know, jungle and you hear some rustling in the bushes, you don't wait and exactly, you try to work out, you know, think through all the possibilities. You run away quickly. So that sort of precautionary approach to potential threats could, I don't know I even really whether this is the case, could have led to this characteristic that people are apophonia, which is the tendency to see patterns where they don't really exist. Oh, yeah, okay. But I think it's. also the feeling that it's uninsured to just how clumpy randomness is. And the classic way is,

you know, I get a handful of rice and I throw it in the air and it drops on the carpet. It's not going to spread itself evenly. There's going to be clusters and lumps all over the place. And if I say, if I then drew a map around it and said, well, these are cancer cases in the UK, people start interpreting the clusters. And why are there so many there? There's a big gap there. No, that's just the way it goes. Randomness is not even. Plane crashes tend to come in threes, as they, as we've seen recently. You know, they really do tend to cluster. They don't evenly spread. Once you've got a plane crash, it doesn't say, oh yeah, well, we'll have to wait another two years before another. No, they tend to cluster. And the classic example of that is birthdays, you know, there are 365 birthdays, and yet, you know,

if you take 23 people, like all the people on a football game, 50% of all football games, there's two people on the picture with the same birthday. Birthdays tend to cluster. The one I like, which I think is sometimes it might not be that intuitive, is if you got 20 people and you ask them the last two digits of their phone number, okay, there's a hundred different possibilities and you only got 20 people, but there's an 87% chance that at least two of them have got the same last number. Very likely that there will be a match among those 20 people. It's something I do with audiences all the time. It surprises people at the time. And people think,

why is there, you know, that seems really odd. It's much higher than your intuition is. And that's because randomness does not mean

evenly spaced. It's very clumpy. Yeah, so sort of sticking with this, how about this, what I guess you'd call the framing of numbers? So if I say,

if you do such and such, you're 10% likely to die, people are just thinking, well, really, I won't do that. You're 90% likely to not die. I'll give it a go.

Yeah, yeah. No, it's a standard with framing is incredibly important to be studied a lot in psychology. I know that I can make any number look big or small, depending on how I tell the story, what context I put it in.

I can make any number at frightening or reassuring, just, you know, because I learned the tricks. And the standard one you mentioned is that, you know, in America, they report, you know, heart surgery

in terms of mortality rate, 2% mortality rate. In this country, we have 98% survival rate. Wow. Sounds much better, doesn't it? So in order to be trustworthy, then, whenever you're reporting a percentage, you should always do the percentage with and the percentage without. You report the whole hundred.

Two deaths and 98 survivors and show them both. That's why a little picture of 100 people with two colored in, it's a really good way of doing it. But there's other forms of framing. That's called a positive and a negative frame. So the one example I give all the time is that there was something called the 99% campaign, which had some posters on London bus stops. And it said 99% of young Londoners

do not commit serious youth violence. It's a lovely story. And so I took a picture of this. You stand there and think, okay, that's a good news story. So a positively framed message. There's two tricks to frighten you. The first is to turn it into a negative frame.

That means that 1% of young Londoners do commit serious youth violence. So that's positive to negative frame. But the next trick is to go from a percentage to what it means for the whole population, to the whole group. So there's about 9 million people in London. There's probably about a million between 15 and 25.

And 1% of a million is 10,000. Oh my God. That's 10,000 violent young maniacs in this city. This is really frightening. So that's, and I could, maybe it's getting a bit political, but I could do exactly the same for the Brexit campaign,

you know, 350 million pounds a week to the EU on the side of the bus. I can make that look like a small number. And I watch for these tricks all the time. And to be trustworthy, you shouldn't do it. You shouldn't be trying to frighten people or reassure them. And the framing and the way you tell the stories, numbers do not speak for themselves.

The context, the storytelling is absolutely vital in the impression they give. And anyone who communicates numbers has to be aware of that. And that's why I think, you know, statisticians, because I do feel they should be involved in the communication of the work they've done, need to be aware of this and it should be an integral part of the education for statisticians. Yeah, so we've covered an awful lot here.

Sort of by way of summary, do you have a sort of cheat sheet for people who are interpreting data or when they're just presented with it? Oh, yeah, yeah, yeah. Well, I got that in my book, Artists Statistics, I got that. And the first one, the first one,

actually I stole from Tim Halford, who also says this, and I thought that's right. Even before you even look at the number, you ask, how does this make me feel? Do you look at your own emotional response to the number? And that gives you insight into why you are being told that number. Is it to frighten you, is to reassure me? Then, of course, you look at the source and you look at the trustworthiness of that source, and you ask, why am I hearing this? Who has chosen to tell me this? And that's the very first

thing, even before you look at the number. So it sounds cynical, but it's just, I think we just need to be aware of how many people are trying to manipulate our emotions out there using numbers, which sound cold and hard and scientific. No, no, no. There's soft, cuddly things, numbers. So, and then I'd start asking, well, you know, so in other words, I'd ask whether the source is trustworthy. And then I'd be looking at the number a bit and ask whether the actual number is true.

Can I actually believe the number? Is it trustworthy? Often, does it actually represent what I think it represents? You know, what does it actually mean? And then I can start looking at, you know, where did it come from? Did they do a proper experiment? Or is this just asking 10 people? You know, where's the source of the number?

And then, you know, the next one is the claim trustworthy based on the... Because the number might be right. But then people make some exaggerated claim about, therefore, we should ban alcohol. You know, we should do this, that and the other. Some unsupported claim. So then you ask about whether the claim is trustworthy. And that hugely depends on the context. Is this really a big number?

you ask and so on. So in the last two, I'm not sure what ordered best to ask about whether you believe the number or whether you believe the claim. They both sort of can be integrated in together. But basically you ask about the source, the number and the claim, and you have to go through those. And the crucial question was, is this trustworthy? Professor Sir David Spiegelhalter,

thanks very much for that. Fascinating conversation. Oh, well, I could go on for hours, so thank you very much indeed. Thank you for listening to this episode of Instant Genius, brought to you from the team behind BBC Science Focus. That was Professor Sir David Spiegelhalter. If you liked what you just heard, then please do consider subscribing to Instant Genius on your preferred podcast platform. You can now also find video editions of Instant Genius on our YouTube channel at Science Focus. The current issue of BBC Science Focus magazine is out now.

Pick up a copy wherever you buy your favourite magazines. Check us out on Apple News. or find us online at sciencefocus.com. This podcast is sponsored by Name, Audio and Focal. The texture and emotional depth of music can be lost through digital sources or poor signal. Name Audio believes you can have digital precision with analogue warmth.

Alongside French acoustic specialist focal, name creates high-end audio systems combining innovation with craftsmanship so you can listen to music, just as the artist intended. Discover more. at name audio.com. Ambition comes in all shapes and sizes.

At First Citizens Bank, we roll with your goals because we're built for what you're building. Fit for your ambition for Citizens Bank. Thinking about training for a new career in healthcare, American Career College offers multiple healthcare training programs that can be completed in a year or less, including medical assistant, dental assisting, pharmacy technician, and optical technician.

And if you're interested in nursing, the vocational nursing program can help prepare you for a more advanced hands-on role in patient care. Financial aid and scholarships are available to those who qualify. Visit accfuture.com. That's acccfuture.com. ACC cannot guarantee employment.