Science Friday - Alzheimer’s Treatment Controversy, Science Mistakes, Chonky Fish. June 11, 2020, Part 2

Episode Date: June 11, 2021

FDA’s Approval Of Debated Alzheimer’s Treatment Raises Controversy This week, the FDA gave the green light to a drug for the treatment of Alzheimer’s disease. The drug, a monoclonal antibody called aducanumab, is the first Alzheimer’s treatment to receive approval in almost 20 years. It targets the amyloid protein that forms the tangled plaques found in the brains of people with Alzheimer’s. But while researchers agree that aducanumab leads to less amyloid plaque, no one really knows what that means in terms of real benefits for people with the disease. Researchers still don’t understand the role of amyloid in the progression of Alzheimer’s disease—and in two studies conducted by the company Biogen, only one showed taking aducanumab provided a slight cognitive benefit to people with early Alzheimer’s. The other study showed no effect compared to a placebo. However, the FDA elected to ignore the recommendations of an outside advisory panel, and approved the medication under an accelerated approval process. The drugmaker will be required to conduct additional testing on the treatment while it is on the market, and the FDA has the option to rescind approval if a Phase 4 trial fails to show efficacy. Biogen will sell the treatment under the trade name Aduhelm, at a list price of around $56,000 per year—not including the extensive office visits, tests, brain scans, and monitoring that will go along with the course of treatment. Pam Belluck, a writer covering science and medicine for the New York Times, joins host John Dankosky to explain the decision, and how the drug might fit into the larger picture of Alzheimer’s research. When Scientists Get It Wrong A couple of years ago, Julia Strand was trying and failing to replicate a study she’d published. At the time, she was an assistant professor without tenure, and the original study had presented her most exciting finding to date. But when she and her co-authors tried to replicate it, they got the opposite results. Then one night, Julia discovered why. In her original code, she’d made a tiny but critical error, and now, with her reputation and job on the line, she was going to have to tell the world about it. Science is often said to be “self-correcting”—through peer review, replication, and community dialogue, scientists collectively find mistakes in their work, and continually revise their understanding of the world. But what does self-correction look like in practice? And how likely are scientists to admit they’re wrong? Julia eventually submitted her story to the Loss of Confidence Project, which invited psychologists to publicly admit mistakes in their published research. Our guest, Julia Roher, a lecturer in psychology, organized the project, along with two others. In an anonymous survey of 316 researchers, almost half said they had lost confidence in one of their findings, but ultimately, only 13 researchers submitted public testimonials to the project. Brian Resnick, who co-created Vox’s Unexplainable podcast and has written about intellectual humility, explains why we often think we’re right when we’re wrong, how others perceive us when we fess up to mistakes, and what all this means for our trust in science.     Charismatic Creature Corner: Chonky Fish Edition In South Africa in 1938, a young museum curator named Marjorie Courtenay-Latimer was performing one of her regular duties when she saw something incredible. Courtenay-Latimer was tasked with inspecting fish brought in by local fishermen that were considered out of place in the region. That’s how she found what she later called the most beautiful fish she had ever seen: a coelacanth, thought to be long extinct. Courtenay-Latimer’s discovery did not immediately register as a coelacanth, because the creature was thought to have gone extinct at the end of the Cretaceous period, 66 millions years ago. The fish was seen as a modern Lazarus—a mysterious creature brought back from the dead, stumping scientists. At six feet long and 200 pounds, some consider the coelacanth to be a big, beautiful fish. According to Dr. Prosanta Chakrabarty, professor and curator of fishes at Louisiana State University, the coelacanth is the meathead of the sea. “They are chunky,” Chakrabarty said. “You can hold their fin and it feels like you’re shaking somebody’s hand.” Because they’re so old, coelacanths are closer to the human genealogical lineage than they are to any modern fish. But because this is the Charismatic Creature Corner, only one thing really matters: Is it charismatic enough to enter the Charismatic Creature Corner Hall of Fame? Joining guest-host John Dankosky to argue for the coelacanth entering the Charismatic Creature Corner Hall of Fame is SciFri producer Kathleen Davis and Dr. Chakrabarty. Subscribe to this podcast. Plus, to stay updated on all things science, sign up for Science Friday's newsletters.

This is Science Friday. I'm John Dankoski in for Ira Flato. Later this hour, I'll look at intellectual humility. Admitting your wrong can be difficult, even for scientists. And we're going to nominate another charismatic creature, this one known as a living fossil. But first, this week the FDA gave the green light to a drug for the treatment of Alzheimer's disease. It's a monoclonal antibody called aducanamab. It targets the beta amyloid that forms the amyloid plaques found in the brains of people with Alzheimer's. The drug will be sold by the company Biogen under the trade name Aduhlum, and the price? It's going to list for about $56,000 a year. But the data is less clear on the efficacy of the drug. While researchers agree that it leads to less amyloid plaque, no one's quite sure what that means

in terms of real benefits for people with the disease. An outside advisory panel had recommended against the treatment's approval. Joining me now to help sort through the approval, and what happens next is Pam Bellick. She's a science and health writer for The New York Times, and you'll find some links to some of her recent articles on our website at Science Friday.com. Pam, welcome to Science Friday.

Thanks so much for joining us. I'm happy to join you. So first of all, walk us through this drug. What does it actually do? Yeah, so adiativea or, Adjahelm, as it's going to be branded, as you said. It is an anti-ameloid drug. What it does is targets a key protein that is involved in Alzheimer's. Amyloid is the protein that clumps into plaques in the brains of people with Alzheimer's. Aducanamab does a pretty good job of clearing amyloid

out of the brain. And the question is, does it actually produce any benefit? And the evidence has really been pretty contradictory. There were two large phase three trials that were nearly identical that biogen conducted of this drug. And they were both stopped early by a data safety monitoring committee because the committee said, we looked at the data, this drug doesn't look like it's working. And so everybody thought that this was yet another failure. And then a few months later, Biogen was sifting through some data that had come in in the three months between when the data monitoring committee was looking at and the time where they sort of pulled the plug. And they said, you know what, in one of these trials, we see a slight benefit.

In people who are taking the high dose, they saw a slowing of cognitive decline of about 22% over 18 months. about four months of slowing over 18 months or a little over two and a half months of slowing a year. That was in one of the trials. The other nearly identical trials showed absolutely no benefit of the drug over placebo. And so this is why this drug was so soundly rejected by the FDA's advisory committee and by a number of Alzheimer's experts. And they just say, look, you've got one positive and one negative trial and why should we prioritize the results of the positive trial? And then there's also a question that even if we do prioritize the results of that trial, is the benefit so slight just a few months of slowing of decline that it outweighs the risks of

this drug and there are some risks. And before we talk about the risks, I mean, maybe you can just talk through a little bit more how unusual this is to have trials that are in some way so unsuccessful actually result in a drug being cleared for usage. Yeah, I think it's very unusual, certainly very unusual in the Alzheimer's space. This kind of pathway to approval has been used by the FDA for a lot of cancer immunotherapy drugs, where there is sort of a suggestion of benefit, but they don't have the kind of slam-dung evidence that the FDA likes to see. Typically, they're supposed to look for two significant, you know, convincing clinical trials to grant approval. And this pathway that the FDA used, it's a program called accelerated approval that's often

used for cancer drugs where there's sort of this sign of a benefit, but it isn't a slam dunk, and it's combined with some sort of additional action that we can see that the drug does, usually sort of acting on a biomarker or some kind of physiological underpinning of the disease. And that's what the FDA chose to do here. They said, we get that. We get that. the clinical evidence here is not a slam dunk. We think there's a suggestion of benefit, but we get that this is really not as convincing as we would like. But we're going to take that and add in the fact that the one thing everybody does agree on is that this drug reduces amyloid, and we're going to take that and give it this approval. And the approval is contingent on biogen

doing another trial, which kind of sounds like, okay, you know, get it out there and then have them come back and do another trial and see if it works. But in reality, there are some issues with that because the trial could take up to nine years. And so while that is going on, there are no restrictions on who can receive this drug that a significant number of people in the scientific community are not confident actually works. So what exactly then happens as this very long trial goes on? Let's say they find out it doesn't work over time. Can it be unapproved?

Yes. And the FDA has done that with some drugs in the past. If the what they call the confirmatory clinical trial does not confirm benefit, then the FDA can revoke approval. But it doesn't have to. revoke approval. And in fact, there have been some pretty strong fights waged over cancer drugs that have not shown benefit in their follow-up trials. And sometimes the FDA has said, well, we're just going to let it stay on the market anyway. And their stated reasoning for that

is that these are very serious diseases with not a whole lot of other treatment options. And so I think There's some element of we understand the desperation of patients with these diseases. And if there's any kind of suggestion that something might help, the FDA seems to be leaning in that direction. You mentioned side effects. What are the possible side effects of this drug? So the most serious ones are that the drug can cause brain swelling and brain bleeding. And in the phase three clinical trials, 40% of participants experienced those side effects. Now, that sounds really serious.

And in actuality, it can be very mild. And in many of those cases, it was very mild. People had the brain swelling, but they didn't have any symptoms or they had relatively manageable symptoms like headaches and dizziness. But in some cases, it can be very serious. And in those trials, 6% of participants had to draw. out because these effects were very serious. So anybody who gets this drug will need, and by the way, it's given as an intravenous infusion. It's not a pill. It's a monthly intravenous infusion.

So you have to go into a center. And then you will need to get periodic brain MRIs to monitor and see if you're getting any of this swelling or brain bleeding. When you're when you're doing this kind of regimen in a clinical trial, there's a whole lot of safeguards. It's very strict. It's very carefully conducted. It's going to be harder to make sure that the memory clinics and the providers who are going to be giving this out to, you know, potentially millions of patients are equipped to with the correct safety protocols and trained enough to know how to read those brain in or not. How does this fit in with some of the other treatments that have already been approved or are still in clinical trials.

Right. So this is the first new treatment for Alzheimer since 2003, so 18 years. And that's obviously why there's been such excitement. Prior to that, there were five drugs available. They're still available. And this is the first drug, the adjutantamab, that directly attacks some aspect of the biology of disease. So the amyloid plaques, as we said.

The other five drugs are designed to basically try to treat the symptoms of forgetfulness and cognitive functional issues. And they do help for some patients and they can help patients in all stages of the disease, but they tend to have a limited amount of time that they'll work. So depending on the person, depending on the stage, these drugs may help stave off to decline for three, six, maybe nine months, about the same, you know, even longer, I guess, than the promise of staving off decline that Adjakanema has. So everybody wants to have something new in the arsenal. That's what the excitement is about. What is coming down the pike could be

better. There are specifically two other monoclonal antibodies that also attack amyloid that are in phase three trials now, and they have had pretty promising phase two trials that have shown that they reduce amyloid by a greater degree than adjudicantamap does, and they do seem to show a somewhat stronger clinical benefit. And those are now in phase three trials, and if that continues, and you know, there have been many times when that hasn't happened in the Alzheimer's field, But if that continues, then those could be ready for approval in maybe about three years. So the interesting thing is that, you know, well before we have the results of the confirmatory trial that shows, you know, whether or not there's any benefit to aducanabab, we may actually have a couple of other treatments that could be a bit more promising.

There's a lot of concern in the field that in the effort to try to get something out there, it could hurt the ability to actually conduct trials on other things that might be more promising. And the reason is that it's very hard to get trial participants for Alzheimer's trials to begin with. And so if you were a patient with Alzheimer's and you are being told, hey, you could try something that's out there on the market now, or you could participate in a clinical trial where you maybe have a 50% chance of getting a placebo, you know, a lot of patients are going to say, well, let me take the bird in the hand, let me try what's out there. And so there's concern that this could actually slow the pace of more promising research.

Pam Bellick is a science and health writer for the New York Times. Thanks so much for joining me today. Oh, thank you. You can find some links to some of Pam's research. articles on aducanamab on our website at science friday.com. When we come back, we're going to be talking about self-correction in science, and what happens when scientists get it wrong? Stay with us. This is Science Friday. I'm John Dankowski. You'll often hear that science is self-correcting. Even though science doesn't always get it right the first time, researchers collectively catch their mistakes and correct them and are constantly updating what they believe to be true based on new evidence. But what does self-correction really look like?

And how willing are scientists to admit when they're wrong? A few years ago, a psychology researcher named Julia Strand discovered a big mistake in her own work. It was in a study that had gotten her lots of positive attention. There was even talk of a new app based on it. But when they tried to replicate it, it just didn't work. And then one night, Julia's sitting at her laptop, trying to figure out what went wrong, and she notices a tiny error in her original code. So when I realized what had happened, I mean, first, first, I saw that little thing and I thought,

oh, wait, is that what did it? No, no, don't panic yet, Juliet. That might not be it. And I tested a couple of other things. And then it was time to panic. Yeah, I had made this mistake. So the bottom dropped out of my stomach. I started crying. And I just started realizing all of the consequences that this was going to have. So I would, you know, if bringing this to light would mean telling my research students, telling my co-authors, telling the chair of my tenure committee and the dean of the college, because I was currently under review for tenure. Like, my committee was meeting that month to decide whether I should have a job for the rest of my life or get fired. Oh my. And for just a moment, she thought, what if she just didn't tell anyone? But obviously, in the end, Julia came forward. Now, you can hear more about her study and what happened when she told the world about her mistake on the Science Friday podcast. We heard about Julia's story because she submitted it to something called the Loss of Confidence Project. It's an initiative that invited psychology researchers to fess up and correct their own mistakes. So today we have two guests to talk about this. Another Julia, Julia Rohrer, who is a

a lecturer at the Department of Psychology at the University of Leipzig, who organized the loss of confidence project, and Brian Resnick, senior science reporter and co-creator of Vox's Unexplainable Podcast. I'd like to welcome you both to Science Friday. Hi. Hey. So, Brian, in your writing, you've made the case for intellectual humility. So first of all, what do you mean by intellectual humility? What got you thinking about this? Yeah, intellectual humility is just the, it's just the trait. It's the, it's the, way of thinking of thinking this, just the simple question, what if I'm wrong? It's a way of measuring your own confidence and your own thoughts. And what really fascinating me here, so I've been

reporting on psychology for a while and watching psychology go through what they call the replication crisis, kind of the clip we just heard is a little bit of an example of that of studies that, you know, upon re-inspection, aren't holding up. And so I was just fascinated by this whole field trying to become more intellectually humble, trying to be more willing to admit it when they were wrong. And I think the loss of confidence project, like when I first came across it, it's like, ah, this is so interesting that, you know, on paper it sounds so beautiful and pristine that in science, it's self-correcting. When you see an error, you report it. But also, like, these are people doing science. We have incentives as in people's careers.

They have assentives to be confident and to continue on the path they were on. So that's just good drama. It's good story. And it's also something I think a lot of us can relate to. Julia, tell us how did the loss of confidence project start? So the project started when actually somebody who wasn't involved in the project, Dana Kane posted essentially a loss of confidence statement. So she was one of the original authors of the paper on power posts that supposedly

demonstrated that taking a particular expansive posture really boosts people confidence, increases risk-taking and even affects their hormonal levels. And so she actually had moved on from that type of research, but she always got student requests because students really wanted to work on that. So at some point she was just like, well, whatever, I'm going to post a public statement that I'm going to refer students to. And in that statement, she simply explained how the study actually came into being. So how they analyzed the data. and re-analyzed it and cherry-picked a finding that sounded particularly good. And so she simply explained how she now understood that these practices, many false positive

findings, and that she no longer actually believes in that effect. And so when she posted that statement, it got a lot of attention by other psychologists who started to talk about it in a Facebook group. And a lot of the people were like, wow, this is amazing. This is how science should proceed. And I want to make such a statement as well, but how can we do that? So out of that idea, there was like a collective effort to start a project where people can submit similar statements and publish them together. So what kind of submissions are you asking people for?

So in particular, we asked people to describe a finding in which they had lost confidence. But we wanted a special type of loss of confidence. So first of all, the original finding needed to be in some way novel. And then the submitting author had lost confidence in the primary or central result of the article. because of a theoretical or methodological problem. So we really wanted people in a situation where they have to say, ah, I screwed up. I made a big mistake and it's on me.

And just to clarify, right, we're talking about honest mistakes here that have later come to light, not any deliberate manipulation of data. Well, I guess somebody could have submitted a deliberate fraud case, but of course the people that came forth were people who now knew that they did something wrong. So it's mostly people who didn't understand what they were doing at the time, but in hindsight, see how their practices were problematic. And Brian, I can see why you think that this is such an elegant idea. This could be really important for more scientists to engage in. Yeah, it's a cornerstone of science, you know, re-evaluating old claims, doing replication, seeing if things check out.

But it turns out in practice, this is really hard to do. It's really hard to, you know, do an about face. As we heard before, it's hard to admit it when we're wrong. And I think there's a lot of, like, interesting reasons why that is. And I actually, I commend psychology for, or a lot of, there's this reform movement in psychology that's really interested in this problem. And they're interested in it, I think, because they're psychologists, because they know they are people, studying people. And, you know, why not cast some light on themselves? Could you talk more about that idea of how humans tend to think that we're right an awful lot, even when we're really, really wrong?

Yeah. So this is what I think is the most interesting central problem here is that we often don't know when we're wrong. Like, we don't often have the perceptual abilities to identify errors. We don't remember what we've misremembered. There's also this concept called naive realism, which is that when we've come to our perception, you know, our brains are kind of these organs that make guesses based on our, you know, faulty sensory organs. And sometimes they make a wrong guess. But when they make a wrong guess about what we're seeing or what we're perceiving, like our brains don't tell us it's a wrong guess. It just seems like it's a right guess. So for me, this is such a fascinating problem, not just for scientists, but for everyone, really, because there's always going to be something like a little bit behind the veil of, you know, what we don't know. And I think

just starting with the curiosity of like what's behind there. And is it, it might be scary. It might be, it might implicate our own work. It might change the how we think about things. But I think just being curious about the things we're missing. And then once we discover what we're missing because we were curious about it, having the convictions to act on it, to send in a loss of confidence statement, to say I was wrong. There's, there's so much in our culture that rewards bluster, rewards just, you know, talking off the top of your head and confidence. And I really would hope that intellectual humility, you know, for people who are kind of sick of that arrogance they see in society to, you know, take into themselves. Well, and there's a lot there,

too. There's the fact that bluster is often perceived very favorably, certainly in in politics. And I'm sure that we can talk a little bit more about that. But there's just also this idea that, In some cases, Brian, humans, when faced with something that they don't know, they can almost be more confident about saying something that they're not sure about than something that they've studied, say, for years and years and years. This is a very human trait. Yeah, yeah. I think you're referring to the Dunning Kruger effect where people who perform poorly on a task tend to overrate their ability on that task. And then people who perform really well sometimes underrate their ability on that task. Julia, let's get back to this idea of self-correcting. When we talk about science being

self-correcting, we're talking about collectively, usually scientists correcting each other. Why did you decide to focus on individuals correcting their own work? So in principle, I actually think that the collective self-correction should be sufficient. So even if no scientist ever self-corrected their own mistakes, things could get cleaned up in the next generation and so on. But I believe that that process would be horribly slow and tedious and maybe involve a lot of hostile interactions between researchers. And so if we're talking about individual self-correction, there's that huge benefit that the individual knows all the details about the study, has the best overview over the literature, and will be just much more capable of spotting

mistakes. And so I think the individual is like in a privilege, position to point out errors, but at the same time, the individual has all the incentives to just hide them. Yeah, those incentives, as we heard in Julia's story, are very, very strong. People have all sorts of incentives. She was just talking about things like tenure, professional reputation. These are really big barriers toward people wanting to step forward. They are, indeed. But there's one particular thing in psychology, and that is a bit funny. And so people are very scared of retraction. So that is like, the worst case scenario that your published article gets retracted from the literature.

And it turns out there are actually empirical studies looking at the impact of retractions. And what those studies kind of consistently find is that if authors self-restract their work, it does not seem to harm their reputation. So it has only very small effects on like citation metrics of previous work of the same authors. And it seems to have very little impact on their reputation within the field or even sometimes a positive impact that they are respected more. because people see how much they care about science. When you asked people to share their mistakes publicly in this project, how many people came forward?

So that is an excellent question. So it started very slowly. So we got like one, two, three, four statements. And then it kind of just got stuck there. And so at some point when we had a handful, we were like, we really need to like motivate more people. So we actually published a preliminary version of the project with those statements included to encourage more people. So, like, you won't be alone if you submit here.

There's already a handful of people. And in the end, we also contacted some additional people and so on. And we ended up with 13 written statements. And now, depending on how you look at it, that is either like a lot because there is no similar project. So this is the largest collection of laws of confidence statements today. Or actually, you could also say there should be hundreds of people with similar statements to make that did not participate in the project.

Yeah. Brian, how do you perceive those numbers? Does that sound like a success or is there a lot more people out there who maybe could step forward? Yeah. Well, the disquieting thing behind this is that there are, you know, there have been these large studies looking at the replicability of psychological science and finding like, well, maybe 40, 50 percent of papers replicate, you know, these aren't definitive numbers. But, yeah, there's definitely more wrong papers out there than there are wrong papers

acknowledge. I would feel confident saying that. I'm John Dankosky, and this is Science Friday from WNYC Studios. But Brian, I'd like to talk more about this idea of how admitting mistakes affects how we're perceived. And I'm going to guess that that is a very different thing, depending on the field of study, depending on the, I don't know, the field of life that you're in. For instance, in a personal relationship, saying that you're wrong sometimes is a very positive thing. In politics, as we've found, it's often a very damaging thing. We often want people in politics to step forward and say that they've been wrong, but that almost never actually happens. And sometimes when they do, it hurts their political chances. Talk a little bit more

about how hard it is for people to admit when they're wrong. Yeah. So it would be impossible to put an overall rule on this. You know, I wouldn't be so arrogant to say that. You know, there is one way to admit mistakes. But I think at least, you know, when you look at these more small-slice examples like the Loss of Confidence Project, like I saw, you know, as a reporter outside observer here, I saw this project being commended and being really applauded. And, you know, people, especially when you're talking about scientists who, you know, these are a part of their ideals or their stated ideals at least. So like to see them being applauded for things. that their culture should be applauding. I think that makes sense. You know, politicians, I'm not a political science expert, so I don't really know the impact of, like, a politician admitting they're wrong. You know, there's often a news media cycle that maybe it reflects how people feel about the politician. Maybe it just reflects like a juicy story. It's hard to know sometimes, but I think in our personal lives at least, it's easy to overrate the negative

perceptions we might garner if we admit we're wrong. And I think at least also at the end of the day too, like the truth is really useful. You know, like this isn't just about being virtuous. It's like, you know, when we admit we're wrong about things, like truth, whether it's in science, whether it's in politics and society. Like we can use truth to move forward. It's like, you know, emitting the reality of climate change is useful. And if you're wrong about that,

you should admit it because it's not useful to deny it. So at the like, yeah, so there are like the kind of personal feelings about it. And yes, I think there are studies that people tend to overrate how negatively they'll be perceived when they admit they're wrong. But also just it's just good because truth good. I like that as a bumper sticker. Truth is very useful. Yeah, I hope so. It can be upsetting

sometime, but it's always useful. Julie, very quickly, before we have to take a break, psychology, as you said, is a bit on the forefront of this idea. How do you think that it's affected positively or negatively the reputation of the field of psychology? That is a very good question. I'm not sure I can answer that because I'm an insider, right? And so I got an insider view. And so in my opinion, it seems like, so people have strong opinions on psychology, of course, and those can go either way. And I'm not sure whether the replication crisis has left a huge impact on the public perception. Maybe in some circles, people who have closely followed the news, they will have gotten the impression that something is wrong. But I think the important thing is to note that similar problems exist in other fields, and they might just not get discussed publicly in those fields, not yet, at least.

And so I really think in the bigger picture, it doesn't really matter whether it will affect how people perceive psychology because their perceptions will just become more accurate if you talk more openly. And I think similar processes might start for other fields as well. Do you think that we can become better at self-correcting over time if we employ more of these principles? I think it is possible to foster a different culture where people feel more open about things and are more willing to admit their mistakes. So I very much believe that such a cultural transformation is possible in the long run. We're talking with Julia Rohr and Brian Resnick about self-correction and intellectual humility in science. We're going to take a short break when we come back. We're going to talk more about this.

This is Science Friday from WNYC Studios. This is Science Friday. I'm John Dankoski in for Ira Flato, and we're back with Julia Rohrer and Brian Resnick. We're talking about self-correction in science. we've heard a lot about psychology's replication crisis that some of the coolest findings turned out to be wrong with. This isn't just a psychology problem. I guess big picture, and I'll start with you, Julia. How do we know what to trust when people are changing their mind so much over time? That is a very hard question. The thing is right, so there is that huge discourse that people should just trust scientists and trust the science. But at the same time, we see that scientists keep making. mistakes and keep self-correcting and that has become particular salient during this pandemic,

where really the official recommendations have completely changed over time. And so I believe it's still possible to trust science, but I think people need to have an understanding that self-correction or mistakes are very much part of science, so they are very much part of the process and you can't get perfect science. So science can't always get it right on the first try. So while I believe that people should, of course, trust science. I think they should have realistic expectations for what it can do in situations with huge uncertainty. And, Brian, I think that this is, it's such an important question, as Julia said, it's big. One of the reasons it's so big right now is, if we take the example of Julia Strand that we played earlier, she had an interesting, but not

necessarily a life-altering finding that she had to go back on. But when we're talking about COVID drugs and millions of people dying and a rush to save both lives and economies from collapsing. You can imagine that this idea of intellectual humility is put under very different pressures when that's what you're dealing with across the globe. I think what we've seen during the pandemic is that we just haven't known a lot of the answers when people want to know them. And lately, the lab leak, I put that in square quotes. Hypothesis has been in the news. And, you know, people still don't know where COVID came from, and we might not ever know. For me, the big picture here is never about, oh, scientists don't know anything.

Scientists have learned things for sure. It's that knowledge is really hard one, and there are different levels of evidence for things, and it's more often the case that we're in the middle of a scientific story, like we're in the middle of a path to discovery than we are at the end. So I think I, at least in my work and what we're trying to do on our unexplainable podcast, is really reorient the stories around the questions and hopefully telling people about that scientific process, where science exists along the timeline can help people understand, like, oh, of course we don't have the perfect answer right now.

because the perfect dancer is really hard to get at, and we have imperfect tools to get at it. This idea of a podcast just exploring what scientists don't know, it's something different than usually what science journalists do. We're always interested in what people do know. That's the big exciting stuff. Yeah, I have to say, like Julia, the loss of confidence project really was a huge inspiration for me in helping to develop the show. And like when I was looking at the replication crisis in psychology, I was thinking, well, when the answers fail, like when scientists say, oh, I don't have confidence anymore, like, the questions remain. There's just so much more unknown than known. And, like, the world is still haunted with mystery. So it's about taking this, this problem of, like,

realizing that we're wrong or realizing that we don't know something and make it exciting, like realizing that the journey to getting, to just understanding what we don't know can be thrilling sometimes in itself. Brian Resnick is senior science reporter and co-creator of Vox's Unexplanable podcast. Brian, thanks so much. Thank you. And thanks also to Julia Rohr lecturer at the Department of Psychology, University of Leipzig, who organized the Loss of Confidence Project.

Thank you so much, Julia. Thank you, Joan. Earlier this segment, you heard a bit from Julia Strand, who discovered a tiny but critical error in her own published study a couple of years ago, and she had to decide was she going to admit this or pretend it never happened. So just for you podcast listeners, here's Julia's story. I mean, first I saw that little thing and I thought, oh, wait, is that what did it? No, no, don't panic yet, Juliet. That might not be it. And I tested a couple of other things. And then it was time to panic. I'm Julia Strand. I'm an associate professor of psychology at Carlton

college. So my research is on how people understand spoken language. So you know if you are in a crowded, noisy space and a lot of people are talking, it's harder to understand what the person is saying, not just because you can't understand the words that they're saying, but also you feel like you're kind of like squinting your ears in order to understand the message. So that feeling of squinting your ears in your brain is called listening effort. And so we're interested in ways to reduce listening effort for people. In 2018, I published a paper reporting this very cool finding that we could dramatically, substantially reduce listening effort by presenting a modulating circle on the screen that changed in size and color as speech gets louder and quieter. So when

someone is talking loudly, you see the shape get bigger. So the way that we measure listening effort in the lab is to ask people to do some kind of secondary task while they're listening to speech. So the task they were doing is they had to push a button as quickly as they could every time they heard a noun. The idea is if you have to expend more effort listening to the speech, there are fewer resources available to push that button, and so it slows people down. So in 2018, we published this paper showing that presenting this circle makes people expend less listening effort. And one of the things that was really remarkable about this study is that the effects were really big. In fact, every single participant who was in the study showed the effect.

And it also made us really excited about potential clinical applications, right? This is something where if you're on a phone call and you can't see the person's face, if you have a little visual stimulus, that that could actually make the task of listening easier. So we could make an app that lets you see the speech as well as hear it. So we were really excited about this paper. It got published in a great journal. We presented it at conferences. People were very excited about it.

I, in part based on that work, wrote and was awarded my first major grant, a grant from the National Institutes of Health. And so it was really, it was a very exciting finding. So about a year later, we were trying to replicate and extend the effect. And so we were very surprised when we found that under really similar circumstances, the finding did not hold up. In fact, the finding was reversed. And it wasn't just like the effect got a little bit smaller.

We went from 100% of the participants showing the effect to most of them not showing it, to most of them showing the opposite effect. we were, of course, really puzzled, right? And like a little worried, but at that point, I thought there must just be some kind of bug with the new study, because the old study, the results were so clear and so convincing. So my research students and I went through, tried to think of every possible thing that could have caused this difference, right? Like, we'd updated the operating system of the computer. Could that have done it? We were using slightly different words. Could that have done it? We were using slightly different words. Could that have done

But all of those things were really minor changes and shouldn't have broken the result. And then one night, I was working at home. I had put my kids to bed, sat down at my laptop in my dark living room and was just going over the program. And then I found it. I realized that in the original study, I had made just a tiny little programming error. but it was entirely responsible for the effect that we saw. So the nature of the error was that I had inadvertently set the timing clock to start measuring the participant's response time before the words were presented in the condition that didn't have the circle.

It's basically like I started the stopwatch before the runners even got to the starting. line. So what that means is that it looks like the condition with the circle was super fast, but it's actually just because I was giving it a head start. So when I realized what had happened, I mean, first, first I saw that little thing and I thought, oh wait, is that what did it? No, no, don't panic yet, Juliet. That might not be it. And I tested a couple of other things. And and then it was time to panic. Yeah, I had, I had made this. I had made this. mistake. So the bottom dropped out of my stomach. I started crying and I just started realizing all of the consequences that this was going to have. So I would, you know, if bringing this to

light would mean telling my research students, telling my co-authors, telling the chair of my tenure committee and the dean of the college, because I was currently under review for tenure. Like, my committee was meeting that month to decide whether I should have a job for the rest of my life or get fired. And so all of this is, like, spinning through my head. And I am just to remind you of what's happening in this situation, like, sitting in the dark in my living room alone, and nobody else knows.

Right? And so, and so to be totally honest, I also gave a couple minutes of thought to what if I didn't tell? Like, what if I, you know, to my research students, maybe I could say, boy, this is weird. I guess that modulating circle isn't very helpful. Let's just study something else. And, you know, not try to retract the original paper, not jeopardized tenure. I had also just gotten this grant.

I had my grant for like a month. And so I was like, I don't know. Are they going to make me give the grant money back? what's, I had no idea what was going to happen. And so, yeah, I had, I had some, I had some dark moments of maybe I don't tell. As you can see, that is not the choice that I made. I, um, you know, that night drafted a list of everything I was going to have to do, everybody I was going to have to tell. And then I laid in bed and didn't sleep. And then, you know, got up in the morning and went about just calling and emailing so many people to say, guess what? I screwed up in a major way.

And it was incredibly hard. Like making all those fun calls, writing all those emails was just everyone was like this gut punch of like, yeah, I screwed up. I made a mistake. Yep, I screwed up. I made a mistake over and over again. And what was really striking about that day was both that it was terrible. And also, I was really surprised at how kind and supportive everyone I talked to was. You know, my co-authors said, yeah, this is too bad, but at least we found it. At least we found it pretty soon after the paper was published. The editor of the journal, my department chair, my dean, everyone basically said,

this is really too bad, but you're doing the right thing, and good for you for doing the right thing. So because the mistake was systematic, right? It added the same amount of extra time to every single response. It was actually very easy to subtract that time off and arrive at what the answer should have been. And so we were able to go to the journal and basically say, the paper that you have published in your journal is wrong, but we know what the right answer is. And so we went back and forth with the journal for months trying to figure out what the right thing to do was. And I was very sure when I initiated the process that they would retract the paper because it was wrong. Like, it was so wrong that even the title was wrong. And after a lot of back and forth, the journal decided not to retract it, but to put an error notice on the original paper and then publish a new version with the results updated.

So in the end, I was awarded tenure. I told the National Institutes of Health, and they did not revoke the grant. You know, in the end, all of the things that I had feared would happen didn't end up happening. But what I realized is that in that moment where I was sitting in my living room making this decision, I didn't know what was going to happen because I had never heard a comparable story. Right? Like you hear about people retracting papers because they engaged in scientific misconduct. And I've heard about people catching problems in their work that happen that they catch before they're published.

And I've heard about people catching errors in other people's work. But I had never heard of a story of someone finding their own mistake and coming forward with it. And so because of that, I felt kind of compelled to share my story so that somebody else, when they're sitting in their living room alone in the dark, would know that at least in my case, it actually worked out much better than I would have expected. If we want people to be willing to do what is right for the science, even when it goes against all of the personal incentives, it seems like we need to change scientific culture in a way where making mistakes is not seen as a mark of shame. in which we understand that, like, mistakes happen, and we just need to find ways to reduce the likelihood that they happen,

but that it isn't a failure of someone as a person to have made a mistake. Finding these errors and not talking about them, although it may be better in the short term for a given individual, it's terrible for science. And so we need to be willing to talk about our mistakes so that we can figure out how to make fewer of them, but also so that we can better trust the scientific record. So it has been a hard process, but I wouldn't have done it any differently.

Today, Julia Strand is an associate professor of psychology at Carleton College. This segment was produced by Ella Fetter, and music is composed by Daniel Petersmith. It's the end of another week, so let's have some fun before we end for today. it's time for another charismatic creature corner. Joining me today, per usual, is our charismatic creature correspondent, sci-fry producer Kathleen Davis. Hey, Kathleen.

Hello, hello. So what creature have you brought us this week? Well, to remind our listeners, this is the segment where I bring you a creature that is, let's say, less than obviously cute and charming. And so an expert and I will try to convince you that this creature has got the chops to enter our charismatic creature corner hall of fame. Okay, okay, let's do it. And this creature, if you deem them charismatic,

will sit among slime molds, vampire bats, and hell ants in this prestigious position. It is quite the Hall of Fame you've put together. So what do you have for us this week? Yes. So I wanted to do something a little bit different. In the past, we have talked about some creatures that are alive, you know, today in 2021. And we've also talked about some that are extinct. but the creature that I have brought for you today was actually thought to be extinct for millions of years before it was actually rediscovered back in the 1930s.

Do you have any guesses for what this is? Let me see. I mean, I don't know, maybe some sort of bird dinosaur hybrid of some sort. Okay, you're about halfway there maybe we can say. It is actually a sealicant, which is a big old fish that is sometimes called a living fossil. A Cilacanth, could you spell that for me? Yes, it is a word that I have misspelled probably three dozen times as I have prepared for this. Okay, are you ready? It is C-O-E-L-A-C-A-C-A-N-T-H.

Okay, so exactly like it sounds. Yeah, right, right. Now, I am no expert on Cilacanth, so I am going to actually turn it over to someone who is. They are a mega celicantth fan. Dr. Prasanta Chakrabarty is a professor and curator of fishes at Louisiana State University in Baton Rouge, Louisiana. Welcome to Science Friday. Thanks for having me. John, do you want to start off the line of charismatic questioning?

Oh, sure. Well, let's see. The first thing I'm going to have to know to determine whether or not this is a charismatic creature is, what exactly does a celicant look like? Acelicant is, let's say, the football player looking thing of a fish. So when it was rediscovered by Marjorie Courtney Latimer, she said it was the most beautiful fish she ever saw. So these things are humongous, so six feet, 200-pound thing with blue colored scales,

big, bright things with whitish flex. So they're really gorgeous. Now, I'm looking at a picture of Acelicanth, and our listeners can, see them too on our website, ScienceFriiday.com slash big fish. We figured that Celicamp was too hard to spell, so we had to get creative. But looking

at this fish, I mean, it really does look like it belongs in dinosaur times. It's pretty meaty looking, I would say. They are chunky. If you hold a, and the cool thing is that their fins have limb bones similar to

our limb bones, so our arms and legs. And so you can holds, you know, they're fin, and it feels like you're shaking somebody's hand. It's really incredible. But they're, yeah, they're kind of chunky. Okay. So they're big. They're kind of chunky. But I don't know, these limb bones sound kind of cool. What else makes these fish cool aside from, you know, how old they are? Well, yeah, besides how old they are. I mean, their rediscovery sort of made us better understand just how closely related they are to us. And in fact, they're part of the same, on the same part of the tree of life as us. So they're lobed finned fishes, which we are also lobed finned fishes, the sarcopteridgee.

And almost every other fish you can think of is a rayfinned fish. So a celacanth is actually more closely related to you or to a sparrow than it is to almost any other fish you can think of, like a goldfish or a catfish. So for me, that alone, I mean, that's pretty charismatic if a fish that's more closely related to you than to other fishes. Now, I have to ask you about something that I came across in my research, that celicants have a butt fin, I have seen it described. A, is this true? And B, what the heck is a butt fin? It's so funny. I love that you asked me that because, yeah, everybody concentrates on their pectoral fins, which are like their, they're,

arms and their pelvic fins, which are like their legs. But they have a bony second dorsal fin, so it's like a backhand and an anal fin that's also full of bones, kind of like a butt leg. So I could use a backhand. It would make us probably better at a sport. Yeah. Or scratching your back, you know. Yeah, right, right. So you're making a pretty convincing case that this is not really like modern fish, right? This is something that has quite a few differences from, say, I don't know, a trout or a bass. Well, so trouts and basses are on this other part of the tree of life that are the rafe and fishes.

And these guys, they have a mix of old and new characters, which I think makes them charismatic. So they have, you know, things that were around for 400 million years. And then they have these other organs that no one else has. They have this rostal organ, which is not dissimilar to what sharks have, which it gives them an electromagnetic sense so they can sense their prey. And really no other fishes have that. You know, most fishes have like a gas bladder that's full of air. Instead, they fill theirs with fatty oils and lipids so they can float.

And so, you know, other fishes don't do that. And no other fish, I would dare you to find another fish that swims in a doggie paddle. And it's so cute. And they're so dumb, too. You know, they have this big brain case. And that's full of fat, too. I mean, they're just bodies just full of fat and oils. and even their brain is mostly fat and oil.

So they're not intellectual giants, but they're awesome. Well, hold it. But if they're filled with fat and oil, I mean, should we be eating them? I don't know. Do they taste good? Would they be good for us? No, they taste terrible. I have not personally eaten one, but their skin, also like Sharks, is quite full of urea.

So they have another organ that helps them do that or actually doesn't do it as well. as efficiently as some other animals. So they don't taste good. Yeah, you would think, yeah, oily, yummy, but no, it's the bad kind of oil. I'm John Dankoski, and this is Science Friday from WNYC Studios. And we're here in Charismatic Creature Corner, learning about the sealacanth.

I vaguely remember about the rediscovery of this fish. Were there, like, fossilized celacanths in museums before it was refound? Yeah, in fact, they were first described from hundreds of millions. year old fossils that were actually small, you know, a few inches long and in fresh water. And so out pops this gigantic, you know, like I said, six foot, 200 pound fish off the east coast of Africa.

And there's now two species known, another one from Indonesia, which also has sort of a dubious discovery process. But yeah, this 20-year-old South African curator, Marjorie Courtney Latimer, who was very curious about stuff and not knowledgeable about fishes, but she found this fish and knew it was something special. And that was back in 1938, and the last time that celacanths had been known were from fossils from 80 million years ago.

So their fossil record went from 360 or so million years ago to 80 million years ago, and then boom rediscovered in 1938. Now, I'm wondering, and you mentioned that they taste terrible, but I'm wondering what may have helped them? live through so many millions of years. I mean, are there theories for why they've lasted this long? Yeah. Well, they're pretty shy, it seems. So they hide out in these deep sea caves. So they live between 2,000 and 2,000 feet and 300 feet or so. And they find these sort of extinct volcanic caves. And they just hang out in there during the daytime or whenever there may be a shark around. They're only real

predators and they stay in little groups. So they're not coming out up to the surface where, you know, anyone would see them. So they're quite uncommon, even, you know, just a couple of them may be collected every so often, incidentally, just from trying to catch something else. So they are probably evaded detection for a long time just because they live in such deep water and stay down there. And, you know, not a lot of fish hooks go into caves. So it's when they when they come out of those caves to hunt themselves, that they're a little bit more vulnerable. If they live in caves, how's their eyesight?

They have big eyes, but they don't, it's not like they're blind. They are attuned to these blue wavelengths that like the color of their scales and for deep sea life so they can see bioluminescent creatures. But, yeah, they don't have particularly awesome or terrible vision. Now, I'm about to show my youth,

my youth here for a minute. But the first time that I came across the celacanth was actually in the game Animal Crossing. And one of the things that you can do in that game is fish. And the celacanth is notoriously one of the hardest fish to find in that game. I mean, do people fish them and are they hard to catch? So people don't fish them intentionally. But they do come up every once in a while when people are fishing for other deep-sea fishes. And unfortunately, you know, we don't know.

know what their population sizes are. They're pretty likely that their population sizes are pretty low. So when they do come up, it's something to be concerned about because they're even, you know, catching two or three of these every few months or every, you know, year or so we don't know what that does to their populations. So they live, you know, maybe 60 years. So that might be a long time. They can have 20 or so pups. They give birth to live, live babies, live pups. And we don't, you know, those are very poorly known, like how their life history works in the deep sea. So we don't want to find out about them from, you know, a fisherman catching them. We want to know more about them in situ in these deep sea caves.

And unfortunately, we still don't know that much about them. Well, I am convinced that this is a charismatic creature, but I'm not the one who matters in this situation. Because we are just about out of time, I have to ask you, John, do you think that celicants belong in the charismatic creature, Corner Hall of Fame. I got to tell you, I was not encouraged when I first heard about this. It just seems like a big fish to me, right? It just seems like a big, meaty fish. I think it was the thing that you said about how closely we're related to them compared to modern fish. I think that

that's the thing that really makes me feel, yeah, this is actually a pretty charismatic creature. Yes. So just because it's like you, that's why you like it. Just, well, that's it. The more it is like me, the more charismatic, I think it is. You're on to me, Keff. For Asanda, I don't think I need to ask you this question, but do you think that celiacans are charismatic? I think they're the absolute best. You can, you know, I'm even wearing a celicants shirt and my cup has a celicant sticker. I love them. I just think they're the coolest animals on the planet.

So if just being close to as charismatic as John, then I'll take it. They're as close to John as any animals you guys will talk about. Well, thank you so much, Dr. Prasanta Chakrabardi is a professor and curator of fishes at Louisiana State University in Baton Rouge, Louisiana. Thanks so much for joining us. Thank you. And thanks also to our charismatic creature correspondent, Kathleen Davis. Thanks so much, Kathleen. Anytime.

If you missed any part of our program or you'd like to hear it again, subscribe to our podcasts or ask your smart speaker to play Science Friday. You know, every day is now Science Friday. Oh, and one more thing. If you're a STEM educator, we have something just for you this summer. Join our researcher educator teams as they explore the phenomena behind contemporary STEM research and discuss how to incorporate it into the classroom. At our Science Friday Educator Phenomena Forum, you can learn more and register at ScienceFriety.com slash phenomena.

Say hi to us on social media, Facebook, Twitter, Instagram. You can email us. The address is SciFri at ScienceFri.com. Send feedback and tell us what you'd like us to come. cover to. I'm John Dankoski and have a great weekend. One last thing before you go, on vacation, that is, we know that many of you are vaccinated and about to embark on a long anticipated vacation. I've got one planned myself and I've got to say I'm looking forward to it. But as a quick reminder, our work here at Science Friday doesn't stop. We're going to be here all summer

reporting on important science news. But we need your support to continue this work. So please make a donation before you head out on summer holiday. Go to ScienceFriday.com slash support to make your gift today. Thanks so much and have a safe and happy summer.