Stuff You Should Know - Research tips from SYSK

Episode Date: July 5, 2016

People often ask us how we do our research. We're not going to disclose all of our secrets, but we'll give you some tips on how to root out the bad studies from the good ones. Learn all about shady studies and reporting right now! Learn more about your ad-choices at https://www.iheartpodcastnetwork.comSee omnystudio.com/listener for privacy information.

On the podcast, Hey Dude, the 90s called, David Lasher and Christine Taylor, stars of the cult classic show, Hey Dude, bring you back to the days of slip dresses and choker necklaces. We're gonna use Hey Dude as our jumping off point, but we are going to unpack and dive back into the decade of the 90s.

We lived it, and now we're calling on all of our friends to come back and relive it. Listen to Hey Dude, the 90s called on the iHeart radio app, Apple Podcasts, or wherever you get your podcasts. Hey, I'm Lance Bass, host of the new iHeart podcast, Frosted Tips with Lance Bass. Do you ever think to yourself, what advice would Lance Bass

and my favorite boy bands give me in this situation? If you do, you've come to the right place because I'm here to help. And a different hot, sexy teen crush boy bander each week to guide you through life. Tell everybody, ya everybody, about my new podcast and make sure to listen so we'll never, ever have to say. Bye, bye, bye.

Listen to Frosted Tips with Lance Bass on the iHeart radio app, Apple Podcasts, or wherever you listen to podcasts. Welcome to Stuff You Should Know from HowStuffWorks.com. Hey, and welcome to the podcast. I'm Josh Clark with Charles W. Chuck Bryant and Jerry. This is Stuff You Should Know.

Josh, we're gonna do something weird today. We're gonna do a listener mail at the head of the podcast. What? I know, right? What? All right, let's do it. Okay, this is from Bianca. Wait, wait, hold on.

Do we have the listener mail music going? Oh, I don't know. Jerry. Should we go the whole nine yards? So let's do it. People might freak out. I know. All right, this is from Bianca.

Boisich, is what I'm gonna say. I think that's great. Hey guys, Rocha Not Too Longo asking about how you research your own podcast. It just got back from a class where we talked about research misrepresentation and journal articles. Apparently, journals don't publish everything that is submitted.

A lot of researchers don't even publish their studies. They don't like the results. Some laws have been put into place to prevent misrepresentation, such as researchers having to register their studies before they get results, and journals only accepting pre-registered studies, but apparently this is not happening at all,

even though it is now technically law. This ends with the general public being misinformed about methods and drugs that work. For example, there are 25 studies proving a drug works, and 25 that don't. It's more likely that 20 of the positive results have been published in only one or two of the negative. And that is from Bianca.

And that led us to this article on our own website. 10 signs that that study is bogus. Yeah. And here it is. Nice, Chuck. Well, we get asked a lot about research from people, usually in college. They're like, you guys are professional researchers.

How do I know I'm doing a good job in getting good info? And it's getting harder and harder these days. It really is. One sign that I've learned is if you are searching about a study, and all of the hits that come back are from different news organizations, and they're all within like a two, three day period from a year ago.

Copy-paste. Nothing more recent than that, then somebody released a sensational study, and no one did any actual effort into investigating it, and there was no follow up. If you dig deep enough, somebody might have done follow up or something like that, but for the most part,

it was just something that splashed across the headlines, which more often than not is the case as far as science reporting goes. So that's a bonus. That's the 11th. Boom. How about that? Yeah.

So we just start banging these out? Let's do it. Or do you have some other clever segue? Well, apart and parcel with that. I don't know if it's clever. You do come across people who you know can be trusted and relied upon to do good science reporting. So like Ed Yong is one.

Another guy named Ben Goldacre has something called Bad Science. I don't remember what outlet he's with. And then there's a guy, I think Scientific American named John Horgan, who's awesome. Yeah, or some journals and organizations that have been around instead of the test of time

that you know are really doing it right, like nature. Yeah, Scientific Americans are like really science. Yeah, like I feel really good about using those sources. Yeah, but even they can, you know, there's something called scientism where there's a lot of like faith and dogma associated with the scientific process. And you know, you have to root through that as well.

All right, I'm done. The first one that they have here on the list is that it's unrepeatable and that's a big one. The Center for Open Science did a study, it was a project really where they took 270 researchers and they said, you know what, take these 100 studies that have been published already, psychological studies and just pour over them.

And in 2015, just last year, it took them a while, it took them several years. They said, you know what, more than half of these can't even be repeated using the same methods. They're not reproducible. Nope, not reproducible. That's a big one and that means that they, when they carried out, they followed the methodology.

Scientific Method Podcast, you should listen to that one. That was a good one. That they found that their results were just not what the people published, not anywhere near them. For example, they used one as an example where a study found that men were terrible at determining whether a woman was giving them

some sort of like a clues to attraction or just being friendly. Yeah, sexy, sexy stuff. Or just be friends. Or yeah, or good to meet you. Yeah, or buzz off, Jerk. Sure, yeah. And they did the study again as part of this

Open Science Center for Open Science study or survey and they found that that was not reproducible or that they came up with totally different results. And that was just one of many. Yeah, and in this case specifically, they looked into that study and they found that it was, one was in the United Kingdom, one was in the United States.

It may have something to do with it. But the point is, Chuck, is if you're talking about humanity, I don't think that the study was like the American male is terrible at it. It's men are terrible at it. So that means that whether it's in the UK, which is basically the US with an accent and a penchant for T, I'm just kidding, UKC, see you soon.

It should be universal. Yeah. You know? Agreed. Unless you're saying, no, it's just, this only applies to American men. Right. Or these 100 American men, then it's not even a study.

Yeah. The next one we have is, it's plausible, not necessarily provable. And this is a big one because, and I think, we're talking about observational studies here more than lab experiments, because with observational studies, you sit in a room and get asked 300 questions about something

and all these people get asked the same questions and then they pour over the data and they draw out their own observations. Right. And one of the, very famously an observational study that led to false results, found a correlation between having a type A personality and being prone to risk for heart attacks.

Yeah. And for a long time, you know that the news outlets were like, oh yes, of course, that makes total sense. Right. This study proved what we've all known all along. And then it came out that, no, actually, what was going on was a well-known anomaly

where you have a 5% risk that chance will produce something that looks like a statistically significant correlation. But it's not at all. When really it's just total chance. And science is aware of this, especially with observational studies, because the more questions you have,

the more opportunity you have for that 5% chance to create a seemingly statistically significant correlation when really it's not there. It was just random chance where if somebody else goes back and does the same study, they're not gonna come up with the same results. But if a researcher is, I would guess willfully blind to that 5% chance,

they will go ahead and produce the study and be like, no, it's true. Here's the results right here. Go ahead and report on it and make my career. Yeah, well, and they also might be looking for something. In fact, chances are they are. It's not just some random study. And they're like, let's just see what we get

if we ask a bunch of weird questions. Yeah. It's like, hey, we're looking to try and prove something most likely. So that Bader-Meinhof thing might come to play where you're kind of cherry picking data. Yeah, that's a big problem. That kind of comes up.

A lot of these are really kind of interrelated too. Oh, totally. The other big thing that's interrelated is how the media reports on science these days. Yeah, you know. It's a big deal. Yeah. Like John Oliver just recently went off on this

and NPR did a thing on it. That's great. Like they might even, like the researcher might say plausible, but it doesn't get portrayed that way in the media. Sure. I remember that poor kid who thought he found the ancient Mayan city.

The media just took it and ran with it. You know? Yeah, I think there was a lot of maybe or it's possible we need to go check kind of thing. And then he was like, no, he discovered an ancient Mayan city never known before. Yeah. And let's put it in the headline.

And that's, I mean, that's the, that's just kind of the way it is these days. Yeah. Do you have to be able to sort through it? And I guess that's what we're doing here. Aren't we, Chuck, we're telling everybody how to sort through it or at the very least take scientific reporting with a grain of salt.

Yes. Right. And that will, like you don't necessarily have the time to go through and double that research and then check on that research and, you know? Right. So take it with a grain of salt. Yeah.

Unsound samples. Here was a study that basically said how you lost your virginity is going to have a very large impact and play a role on how you feel about sex and experience sex for the rest of your life. Yeah. It's possible.

Sure. It seems logical. So we'll just go with it. But when you only interview college students and you don't, you only interview heterosexual people, then you can't really say you've done a robust study. Now, can you? Plus you also take out of the sample size

your sample population, anybody who reports having had a violent encounter. Yeah. Throw them out. Yeah. For that data out. Because that's not gonna inform how you feel about sex. Right, exactly.

You're just narrowing it down further and further. And again, cherry picking the data by throwing people out of your population sample that will throw off the data that you want. Yeah. And I'd never heard of this acronym weird. And a lot of these studies are conducted by professors and academics.

So a lot of times you got college students as your sample and there's something called weird, Western educated from industrialized rich and democratic countries. Right, those are the participants in the studies. Yes. The study subjects. But then they will say, men.

Right. Well, what about the gay man in Africa? Right. Like you didn't ask him. So that was, that's actually a really, really big deal. In 2010, the three researchers did a survey of a ton of social science and behavioral science studies.

Found that 80% of them used weird study participants. So basically it was college kids for 80% of these papers. And they surveyed a bunch of papers. And they took it a little further and they said that people who fit into the weird category only make up 12% of the world population. But they represent 80% of the population of these studies. And a college student Chuck in North America,

Europe, Israel or Australia is 4,000 times more likely to be in a scientific study than anyone else on the planet. And they're basing psychology and behavioral sciences are basing their findings onto everybody else based on this small tranche of humanity. And that's a big problem that's extremely misleading. Yeah, and it's also a little insulting because what they're essentially saying is like,

this is who matters. Well, yeah, but what's sad is, this is who I am going to go to the trouble of recruiting for my study. It's just sheer laziness. And I'm sure a lot of them are like, well, I don't have the funding to do that. I guess I see that, but at the same time,

I guarantee there's a tremendous amount of laziness involved. Yeah, or maybe if you don't have the money, maybe don't do that study. Is it that simple? I'm probably oversimplifying. I don't know, I'm sure we're gonna hear from some people in academia about this one. We'll stop using weird participants.

Or at the very least say like, this is only for weirdness. Heterosexual Dartmouth students, this applies to them. Right. Not everybody in the world. I mean, 80% of these studies use those people as study participants. And they're not even emblematic

of the rest of the human race. College students are shown to see the world differently than other people around the world. So it's not like you can be like, well, it still works. You can still extrapolate. It's like flawed in every way, shape, and form. We'll probably take a break, huh?

Yeah, let's take a break, because you're getting a little hot under the collar. I love it. Man. We'll be right back after this. Just like the number of stars in the sky, there is so much stuff you should know. On the podcast, Hey Dude, the 90s called,

David Lasher and Christine Taylor, stars of the cult classic show, Hey Dude, bring you back to the days of slip dresses and choker necklaces. We're gonna use Hey Dude as our jumping off point, but we are going to unpack and dive back into the decade of the 90s. We lived it, and now we're calling on all of our friends

to come back and relive it. It's a podcast packed with interviews, co-stars, friends, and non-stop references to the best decade ever. Do you remember going to Blockbuster? Do you remember Nintendo 64? Do you remember getting Frosted Tips? Was that a cereal?

No, it was hair. Do you remember AOL Instant Messenger and the dial-up sound like poltergeist? So leave a code on your best friend's beeper, because you'll wanna be there when the nostalgia starts flowing. Each episode will rival the feeling of taking out the cartridge from your Game Boy,

blowing on it and popping it back in, as we take you back to the 90s. Listen to Hey Dude, the 90s called on the iHeart radio app, Apple Podcasts, or wherever you get your podcasts. Hey, I'm Lance Bass, host of the new iHeart podcast, Frosted Tips with Lance Bass. The hardest thing can be knowing who to turn to

when questions arise or times get tough, or you're at the end of the road. Ah, okay, I see what you're doing. Do you ever think to yourself, what advice would Lance Bass and my favorite boy bands give me in this situation? If you do, you've come to the right place, because I'm here to help.

This, I promise you. Oh, God. Seriously, I swear. And you won't have to send an SOS, because I'll be there for you. Oh, man. And so, my husband, Michael. Um, hey, that's me.

Yep, we know that, Michael. And a different hot, sexy teen crush boy bander each week to guide you through life, step by step. Oh, not another one. Kids, relationships, life in general, can get messy. You may be thinking, this is the story of my life. Just stop now. If so, tell everybody, ya everybody,

about my new podcast, and make sure to listen, so we'll never, ever have to say bye, bye, bye. Listen to Frosted Tips with Lance Bass on the iHeart Radio app, Apple podcast, or wherever you listen to podcasts. Learning stuff with Joshua and Charles. Stuff you should know. All right, what's next, buddy?

Uh, very small sample sizes. Right. If you do a study with 20 mice, then you're not doing a good enough study. No. So, they use this in the article. They use the idea of 10,000 smokers and 10,000 non-smokers.

Yeah. And they said, okay, if you have a population sample that size, that's not bad. It's a pretty good start. And you find that 50% of the smokers develop lung cancer, but only 5% of non-smokers did. Then, your study has what's called a high power. Yeah.

It's, if you had something like 10 smokers and 10 non-smokers, and two of the smokers develop lung cancer, and one develop lung cancer as well, you have very little power, and you should have very little confidence in your findings. But regardless, it's still gonna get reported if it's a sexy idea. Yeah, for sure.

And because these are kind of overlapping in a lot of ways, it was, I want to mention this guy, a scientist named Ulrich Dernigl. He and his colleague Malcolm McLeod have been trying, I mean, and there are a lot of scientists that are trying to clean this up, because they know it's a problem, but he co-wrote an article in Nature

that's called Robust Research Colon. Institutions must do their part for reproducibility. So this kind of ties back into the reproducing things, like we said earlier. And his whole idea is, you know what, good funding, they should tie funding to good institutional practices. Like you shouldn't get the money if you can't show that you're doing it right.

Yeah. And he said that would just weed out a lot of stuff. Here's one staggering stat for reproducibility and small sample size. Biomedical researchers for drug companies reported that 25% of their, only 25% of the papers that they publish are even reproducible. And then there's like an insider stat.

And that didn't matter. They still, the drugs are still going to market. Yeah. Yeah. Which is, that's a really good example of why this does matter to the average person. You know, like if you hear something like, monkeys like to cuddle with one another because they are reminded of their mother's study shows.

Right. You could just be like, oh, that's great. I'm going to share that on the internet. It doesn't really affect you in any way. Yeah. But when there's studies being conducted that are creating drugs that could kill you or not treat you or that kind of thing

and it's attracting money and funding and that kind of stuff, that's like, that's harmful. Yeah. Absolutely. I found another survey. Did you like that terrible study idea that it came up with? No, I liked it. The monkeys like to cuddle. 140 trainees at the MD Anderson Cancer Center

in Houston, Texas. Thank you, Houston, for being so kind to us at our recent show. They found that nearly a third of these trainees felt pressured to support their mentors' work, like to get ahead or not get fired. So that's another issue is you've got these trainees or residents and you have these mentors

and even if you disagree or don't think it's a great study, you're pressured into just going along with it. I could see that. For sure. There seems to be a huge hierarchy in science. Yeah, for sure. Like in a lab, you got the person who runs a lab, it's their lab.

Yeah, you can go against them. Right. But there are people like Science and Nature, two great journals are updating their guidelines right now. They're introducing checklists. Science hired statisticians to their panel, a reviewing editors, not just other peer reviewed, like they actually hired numbers people specifically.

Oh, gotcha. Because that's a big... It helps the process. That's a huge part of studies. It's like this mind breaking statistical analysis that can be used for good or ill. And I mean, I don't think the average scientist necessarily is a whiz at that,

although it has to be part of training. Yeah, but not necessarily. And that's a different kind of beast altogether stats. We talked about it earlier. I took a stats class in college. Oh man, I had so much trouble. I was awful at it. It really just, it's a special kind of,

is it even math? Hell. Yeah, I didn't get it. I passed it though. I passed it because my professor took pity on me. Oh, that's nice. That Ulrich Dernigl, Ulrich Dernigl,

he's a big time crusader for his jam, making sure that science is good science. One of the things he crusades against is the idea of, you remember in that virginity study where they just threw out anybody who had a violent encounter for their first sexual experience. Apparently that's a big deal with animal studies as well. If you're studying the effects of a drug or something,

like there was this one in the article, if you're studying the effects of a stroke drug, and you've got a control group of mice that are taking the drug, or that aren't taking the drug, and then a test group that are getting the drug, and then like three mice from the test group die, even though they're on the stroke drug,

they die of a massive stroke, and you just literally and figuratively throw them out of the study and don't include them in the results, that changes the data. And he's been on a peer review on a paper before. He's like, no, this doesn't pass peer review. You can't just throw out, what happened to these three rodents?

You started with 10, there's only seven reported in the end, what happened to those three? And how many of them just don't report the 10? Yeah. They're like, oh, we only started with seven. We're going, you know? Well, I was about to say I get the urge. I don't get it because it's not right,

but I think what happens is you work so hard at something. Yeah, oh yeah. And you're like, how can I just walk away from two years of this because it didn't get a result? Okay, the point of real science though. Yeah, you have to walk away from it. Well, you have to publish that. Yeah.

And that's the other thing too. And I guarantee scientists will say, hey man, try getting a negative paper published in a good journal these days. They don't want that kind of stuff. But part of it also is, I don't think it's enough to just have to be published in like a journal. You want to make the news cycle as well.

That makes it even better, right? So I think there's a lot of factors involved. But ultimately, if you take all that stuff away, if you take the culture away from it, if you get negative results, you're supposed to publish that. So that some other scientists can come along and be like, oh, somebody else already did this using these methods that I was going to use.

I'm not going to waste two years of my career because somebody else already did. Thank you, buddy, for saving me this time and trouble and effort to know that this does not work. You've proven this doesn't work. When you sought to prove it does work, you actually proved it didn't work. That's part of science.

Yeah, and I wish there wasn't a negative connotation to a negative result because to me the value is the same. Sure. As proving something does work, as proving something doesn't work, right? Right. Again, it's just not as sexy. Yeah, but I'm not sexy either.

So maybe that's why I get it. Here's one that I didn't know was a thing, predatory publishing. I didn't know about it either. Never heard of this. So here's the scenario. You're a doctor or a scientist and you get an email from a journal

that says, hey, you got anything interesting for us? I've heard about your work. And you say, well, I actually do. I have this study right here. They said, cool, we'll publish it. And you go, great, my career is taking off. Then you get a bill that says, where's my three grand for publishing your article?

And you're like, I don't owe you three grand. All right, give us two. And you're like, I can't even give you two. And if you fight them long enough, maybe they'll drop it and never work with you again. Or maybe let's be like, we'll talk to you next quarter. Exactly. That's called predatory publishing.

And it's a, I'm not sure how new it is. Maybe I- It's pretty new. Is it pretty new? But it's a thing now where you can pay essentially to get something published. You, yes, you can. It's kind of like who's who

in behavioral sciences kind of thing. Yeah. You know? And apparently it's new because it's a result of open source academic journals, which a lot of people push for, including Aaron Schwartz, very famously, who like took a bunch of academic articles and published them online and was prosecuted heavily for it,

persecuted, you could even say. Yeah. But the idea that science is behind this paywall, which is another great article from Priceonomics, by the way, really just ticks a lot of people off. So they started to open source journals, right? And as a result, predatory publishers came about and said, oh, okay, yeah, let's make this free,

but we need to make our money anyway. So we're gonna charge the academic who wrote the study for publishing it. Well, yeah, and sometimes now, it's just a flat out scam operation, 100%. There's this guy named Jeffrey Beal, who is a research librarian. He is my new hero,

because he's truly like one of these dudes that has, he's trying to make a difference and he's not profiting from this, but he's spending a lot of time by creating a list of predatory publishers. Yeah, a significant list too. Yeah, how many, 4,000 of them right now? Yeah.

Some of these companies flat out lie, like they're literally based out of Pakistan or Nigeria, and they say, no, we're a New York publisher. So it's just a flat out scam, or they lie about their review practices. Mike, they might not have any review practices, and they straight up lie and say they do. There was one called Scientific Journals International

out of Minnesota that he found out was just one guy. Like literally working out of his home, just lobbying for articles, charging to get them published, not reviewing anything, and just saying, I'm a journal. Yeah. I'm a scientific journal.

Look at me go. He shut it down, apparently, or tried to sell it. I think he was found out, and this other one, the International Journal of Engineering Research and Applications, they created an award and then gave it to itself, and even modeled the award from an Australian TV Award, like the physical statue.

Wow, that's fascinating. I didn't know you could do that. I'm gonna give ourselves the best podcast and the universal award. I like that. It's gonna look like the Oscar. Yeah. Okay.

The Oscar crossed with the Emmy. This other one, Med No Publications, actually confused the meaning of STM, Science, Technology, Medicine. They thought it meant Sports, Technology, and Medicine. No. Well, a lot of science journalists, or scientists too, but watchdogs,

like to send gibberish articles into those things to see if they'll publish them, and sometimes they do. Frequently they do. They sniff them off the case. That's the big time. How about that callback? It's been a while.

It has been. It needs to be a T-shirt. Should we take a break? Yeah. All right, we'll be back and finish up right after this. Just like the number of stars in the sky, there is so much stuff you should know. On the podcast, Paydude the 90s called

David Lasher and Christine Taylor, stars of the cult classic show, Hey Dude, bring you back to the days of slip dresses and choker necklaces. We're going to use Hey Dude as our jumping off point, but we are going to unpack and dive back into the decade of the 90s. We lived it, and now we're calling on all of our friends

to come back and relive it. It's a podcast packed with interviews, co-stars, friends, and non-stop references to the best decade ever. Do you remember going to Blockbuster? Do you remember Nintendo 64? Do you remember getting Frosted Tips? Was that a cereal?

No, it was hair. Do you remember AOL Instant Messenger and the dial-up sound like poltergeist? So leave a code on your best friend's beeper, because you'll want to be there when the nostalgia starts flowing. Each episode will rival the feeling of taking out the cartridge from your Game Boy,

blowing on it and popping it back in as we take you back to the 90s. Listen to Hey Dude, the 90s called on the iHeart radio app, Apple Podcasts, or wherever you get your podcasts. Hey, I'm Lance Bass, host of the new iHeart podcast, Frosted Tips with Lance Bass. The hardest thing can be knowing who to turn to

when questions arise or times get tough, or you're at the end of the road. Ah, okay, I see what you're doing. Do you ever think to yourself, what advice would Lance Bass and my favorite boy bands give me in this situation? If you do, you've come to the right place because I'm here to help.

This, I promise you. Oh, God. Seriously, I swear. And you won't have to send an SOS, because I'll be there for you. Oh, man. And so, my husband, Michael. Um, hey, that's me.

Yep, we know that, Michael. And a different hot, sexy teen crush boy bander each week to guide you through life, step by step. Oh, not another one. Kids, relationships, life in general can get messy. You may be thinking, this is the story of my life. Just stop now. If so, tell everybody, ya everybody,

about my new podcast and make sure to listen so we'll never, ever have to say bye, bye, bye. Listen to Frosted Tips with Lance Bass on the iHeart radio app, Apple Podcasts, or wherever you listen to podcasts. Learning stuff with Joshua and Charles. Stuff you should know. So here's a big one.

You ever heard the term follow the money? Mm-hmm. That's applicable to a lot of realms of society. Yeah. And most certainly in journals, if something looks hinky, just do a little investigating and see who's sponsoring their work. Well, especially if that person is like, no,

everyone else is wrong. Right. Climate change is not man-made kind of thing. Sure. You know, if you look at where their funding's coming from, you might be unsurprised to find that it's coming from people who would benefit from the idea that anthropogenic climate change isn't real.

Yeah, well, we might as well talk about them. Okay. Willie Soon. Yeah. Mr. Soon. Is he a doctor? He's a physicist of some sort, yeah. All right.

I'm just gonna say Mr. or Dr. Soon. Okay. Because I'm not positive. He is one of a few people on the planet Earth, professionals that is. Right. Who deny human climate change. Human influence climate change, like you said.

Yeah. He said the fancier word for it, though. And anthropogenic. Yeah, it's a good word. Is that it? And he works at the Harvard Smithsonian Center for Astrophysics. So hey, he's with Harvard.

He's got the cred, right? Right. Turns out when you look into where he's getting his funding, he received $1.2 million over the past decade from ExxonMobil, the Southern company. The Kochs. And the Koch brothers, their foundation, the Charles G. Koch Foundation.

Exxon stopped in 2010, stopped funding him. But the bulk of his money and his funding came, and I'm sorry, I forgot the American Petroleum Institute, came from people who clearly had a dog in this fight. And it's just, how can you trust this, you know? Yeah, well, you trusted because there's a guy, and he has a PhD in aerospace engineering, by the way. All right, he's a doc.

He works with this organization, the Harvard Smithsonian Center for Astrophysics, which is a legitimate place. It doesn't get any funding from Harvard, but it gets a lot from NASA and from the Smithsonian. Well, and Harvard's very clear to point this out when people ask them about Willie Soon. Right.

They're kind of like, well, here's the quote. Willie Soon is a Smithsonian staff researcher at Harvard Smithsonian Center for Astrophysics, a collaboration of the Harvard College Observatory and the Smithsonian Astrophysical Observatory. Like, they just want to be real clear, even though he uses a Harvard email address. Right, he's not our employee.

No, but again, he's getting lots of funding from NASA and lots of funding from the Smithsonian. This guy, if his scientific beliefs are what they are, and he's a smart guy, then I don't know about getting fired for saying, you know, here's a paper on the idea that climate change is not human made. Yeah, he thinks it's the sun's fault.

But he doesn't reveal in any of his conflicts of interest that should go at the end of the paper. He didn't reveal where his funding was coming from. Yeah. And I get the impression that in academia, if you are totally cool with everybody thinking like you're a shill, you can get away with it. Right.

Well, this stuff, a lot of this stuff is not illegal. Right. Even predatory publishing is not illegal. Yeah. It's illegal. Right. And if you're counting on people to police themselves with ethics,

a lot of times they'll disappoint you. The Heartland Institute gave Willie Soon a courage award. And if you- We're not caring about what other scientists think of them. If you've heard the Heartland Institute, you might remember them. They're a conservative think tank. You might remember them in the 90s

when they worked alongside Philip Morris to deny the risks of secondhand smoke. Yeah, that's all chronicle in that book I've talked about, Merchants of Doubt. Oh, really? Heartland Institute. A bunch of scientists, legitimate, bona fide scientists who are up for being bought by groups like that.

It's sad. It is sad. And the whole thing is they're saying like, well, you can't say without beyond a shadow of a doubt with absolute certainty that that's the case. And science is like, no, science doesn't do that. Science doesn't do absolute certainty, but the average person reading a newspaper

sees that, oh, you can't say with absolute certainty, well, then maybe it isn't man-made. And then there's that doubt that the people just go and get the money for, for saying that, for writing papers about it. It's despicable. Yeah, it really is. Self-reviewed, you've heard of peer review.

We've talked about it quite a bit. Peer review is when you have a study. And then one or more, ideally more of your peers reviews your study and says, you know what? You had best practices, you did it right. It was reproducible. You followed the scientific method. I'm gonna give it my stamp of approval

and put my name on it, not literally, or is it? I think so. It says who reviewed it. Yeah, I believe so. Okay, put my name on it. And like it in the journal when it's published. But not my name as the author of the study, you know what I mean?

Right. As a peer reviewer. Yeah, as a peer reviewer. And that's a wonderful thing. But people have faked this and been their own peer reviewer, which is not how it works. No. Who is this guy?

Well, I'm terrible at pronouncing Korean names. So all apologies. But I'm gonna say Nung in Moon. Nice. Dr. Moon? I think, yeah, let's call him Dr. Moon. Okay. So Dr. Moon worked on natural medicine, I believe,

and was submitting all these papers that were getting reviewed very quickly. Because apparently part of the process of peer reviews is to say, this paper's great. Can you recommend some people in your field that can review your paper? And Dr. Moon said, I sure can. Yeah, he was on fire.

Let me go make up some people and make up some email addresses that actually come to my inbox. And just posed as all of his own peer reviewers. He was lazy though, is the thing. Like I don't know that he would have been found out if he hadn't been careless, I guess. Because he was returning the reviews

within like 24 hours sometimes. A peer review of like a real study should take, I would guess weeks if not months. Like the study, the publication schedule for the average study or paper, I don't think is a very quick thing. There's not a lot of quick turnaround. And this guy was like 24 hours.

Well, they're like, Dr. Moon, I see your paper was reviewed and accepted by Dr. Mooney. It's like, I just added a Y to the end. It seemed easy. If you Google peer review fraud, you will be shocked at how often this happens and how many legit science publishers are having to retract studies.

And it doesn't mean they're bad. They're getting duped as well. But there's one based in Berlin that 2015 had 64 retractions because of fraudulent reviews. Oh, wow. And they're just one publisher of many. Every publisher out there probably has been duped. Maybe not everyone.

I'm surmising that. But it's a big problem. We should do a study on it. I'll review it. It'll end up in the headlines now. Right. Every single publisher duped says Chuck. And speaking of the headlines, Chuck,

one of the problems with science reporting or reading science reporting is that what you usually are hearing, especially if it's making a big splash, is what's called the initial findings. Right. Somebody carried out a study and this is what they found. And it's amazing and mind-blowing. And it supports everything everyone's always known.

But now there's a scientific study that says, yes, that's the case. And then if you wait a year or two when people follow up and reproduce the study and find that it's actually not the case, it doesn't get reported on usually. Yeah, and sometimes the scientist or the publisher is they're doing it right. And they say initial findings, but the public,

and sometimes even the reporter will say initial findings. But we as people that ingest this stuff need to understand what that means. Right. And the fine print is always like more study is needed. But knowing if it's something that you want to be true, you'll just say, hey, look at the study. Right.

It's brand new and they need to study it for 20 more years, but hey, look what it says. Right. And the more you start paying attention to this kind of thing, the more kind of disdain you have for that kind of just offhand sensationalist science reporting. Yeah. But you'll still get caught up in it.

Like every once in a while, I'll catch myself like saying something to you and be like, oh, did you hear this? And then as I'm saying it out loud, I'm like, that's preposterous. Yeah. There's no way that's going to pan out to be true. I got click baited. I know.

I mean, we have to avoid this stuff. It's stuff because we have our name on this podcast, but luckily we've given ourselves the back door of saying, hey, we make mistakes a lot. It's true though. We're not experts. No. We're not scientists.

And then finally, we're going to finish up with the header on this one is it's a cool story. Yeah. And that's a big one because it's not enough these days. And this all ties in with media and how we read things as people. But it's not enough just to have a study that might prove something.

You have to wrap it up in a nice package to deliver people. Get it in the news cycle. In the cooler, the better. Yep. Yep. It almost doesn't matter about the science as far as the media is concerned. They just want a good headline.

And a scientist who will say, yeah, that's cool. Here's what I found. Yep. This is going to change the world. Loch Ness Monster is real. This kind of ended up being depressing somehow. Yeah. Not somehow.

Yeah. Like, yeah, it's kind of depressing. I know. We'll figure it out, Chuck. Well, we do our best. I'll say that. Science will prevail. I hope so.

If you want to know more about science and scientific studies and research fraud and all that kind of stuff, just type some random words into the search bar at howstoveworks.com. See what comes up. Yeah. And since I said random, it's time for listener mail. Oh, no.

Oh, yeah? You know what it's time for. What? Administrative details. All right, Josh, administrative details. If you're new to the show, you don't know what it is. That's a very clunky title. Yeah.

We're saying thank you to listeners who send us neat things. It is clunky and generic. And I've totally gotten used to it by now. Well, you're the one who made it up. To be clunky and generic, and it's stuck. Yeah. So people send us stuff from time to time. And it's just very kind of you to do so.

Yes. And we like to give shout-outs, whether or not it's just out of the goodness of your heart, or if you have a little small business that you're trying to plug. Either way, it's a sneaky way of getting it in there. Yeah, but I mean, I think we brought that on. Didn't we say, like, if you have a small business,

then you send us something, we'll be happy to say something? Exactly. Thank you. All right, so let's get it going here. We got some coffee from 1,000 Faces, right here in Athens, Georgia, from Kayla. Yeah. Delicious.

Yes, it was. We also got some other coffee, too, from Jonathan at Steamworks Coffee. He came up with a Josh and Chuck blend. Oh, yeah. It's pretty awesome. I believe it's available for sale, too. Yeah, the Josh and Chuck blend is dark and bitter.

Jim Simmons, he's a retired teacher who sent us some lovely handmade wooden bowls. Oh, yes. And a very nice handwritten letter, which is always great. Thanks a lot, Jim. Let's see, Chamberlain sent us homemade pasta, including a delicious savory pumpkin fettuccine. It was very nice.

Yum. Jay Graft, 2F, sent us a postcard from Great Wall of China. It's kind of neat. Sometimes we get those postcards from places we've talked about. No, he's like, look where I am. Thanks, Jay. You guys aren't here. Let's see, the HammerPress team, they

sent us a bunch of Mother's Day cards that are wonderful. Oh, those were really nice. Really great. You should check them out, the HammerPress team. Yeah. Yeah. Misty, Billy, and Jessica, they sent us a care package of a lot of things.

There were some cookies, including one of my favorite, white chocolate dipped ritz and peanut butter cracker. Oh, yeah. Man, I love those. Homemade, right? Oh, yeah. And then some 70s macrame for you, along with 70s macrame magazines, because you're

obsessed with macrame. We have a macrame plant holder hanging from my microphone arm holding a cup. A coffee mug sent to us by Joe and Linda Hecht. Oh, that's right. And it has some pens in it. And they also sent us, Misty, Billy, and Jessica, a lovely little hand-drawn picture of us

with their family, which was so sweet. That is very awesome. We've said it before, we'll say it again. Huge thank you to Jim Ruane. I believe that's how you say his name. And the Crown Royal people for sending us all the Crown Royal, we are running low. Mark Silberg of the Rocky Mountain Institute

sent us a book called Reinventing Fire. Oh, yeah. They're great out there. And they know what they're talking about. And I think it's Reinventing Fire colon, bold business solutions for the new energy era. Yeah, they're basically green energy observers. But I think they're experts in all sectors of energy,

but they have a focus on green energy, which is awesome. Yeah, they're pretty cool. John, whose wife makes delightfully delicious doggy treats. Delightfully delicious is the name of the company. There's no artificial colors or flavors. And they got Sweet Lil' Momo hooked on sweet potato dog treats. I thought you were going to say hooked on the junk.

The sweet potato junk. She's crazy cuckoo for sweet potatoes. Nice. Oh, man. That's good for a dog, too. It is, very. Strat Johnson sent us his band's LP. And if you're in a band, your name is Strat.

That's pretty cool. Sure. Diomea still. I think that was great. Yeah, I'm not sure if I pronounced it right. D-I-O-M-A-E-A. Frederick, this is long overdue, Frederick at the 1521 store, 1521store.com, sent us some awesome low profile cork iPhone

cases and passport holders. And I was telling them, Jerry walks around with her iPhone in the cork holder. And it looks pretty sweet. Oh, yeah. So he said, awesome. Glad to hear it. Joe and Holly Harper sent us some really cool 3D printed

stuff you should know things, like S-Y-S-K, like a little desk. Oh, like after Robert Indiana's love sculpture? Yeah, that's what I couldn't think of what that was from. Yeah, it's awesome. It's really neat and like a bracelet made out of stuff you should know, a 3D carved like plastic. It's really neat.

Yeah, they did some good stuff. So thanks, Joe and Holly Harper, for that. And then last for this one, we got a postcard from Yosemite National Park from Laura Jackson. So thanks a lot for that. Thanks to everybody who sends us stuff. It's nice to know we're thought of. And we appreciate it.

Yeah, we're going to finish up with another set on the next episode of administrative details. You got anything else? No, that's it. Oh, yeah. If you guys want to hang out with us on social media, you can go to S-Y-S-K podcast on Twitter or on Instagram. You can hang out with us at Facebook.com

slash stuff you should know. You can send us an email to stuffpodcast.howstuffworks.com. And as always, join us at our home on the web, stuffyoushouldknow.com. For more on this and thousands of other topics, visit howstuffworks.com. Hey, dude, the 90s called David Lasher and Christine Taylor, stars of the cult classic show, Hey, Dude,

bring you back to the days of slip dresses and choker necklaces. We're going to use Hey, Dude as our jumping off point, but we are going to unpack and dive back into the decade of the 90s. We lived it. And now we're calling on all of our friends to come back and relive it.

Listen to this. Listen to this. Listen to this. Listen to this. Listen to this. Listen to this. Listen to this. Listen to this.

Listen to Hey, Dude, the 90s called on the iHeart radio app, Apple Podcasts, or wherever you get your podcasts. Hey, I'm Lance Bass, host of the new iHeart podcast, Frosted Tips with Lance Bass. Do you ever think to yourself, what advice would Lance Bass and my favorite boy bands give me in this situation? If you do, you've come to the right place because I'm here to help.

And a different hot, sexy teen crush boy bander each week to guide you through life. Tell everybody, yeah, everybody, about my new podcast and make sure to listen so we'll never, ever have to say bye, bye, bye. Listen to Frosted Tips with Lance Bass on the iHeart radio app, Apple Podcasts, or wherever you listen to podcasts.