Muscle for Life with Mike Matthews - Welcome to the Hierarchy (of Evidence)

Episode Date: September 4, 2020

Do you want to know how to use science to optimize your health, fitness, and lifestyle? Do you want to know how to protect yourself against misguided, misleading, and even menacing advice supposedly supported by research? And do you want to get up to speed quickly, regardless of your educational background? If so, then my new book Fitness Science Explained is for you. It’s a crash course in reading, understanding, and applying scientific research, and it teaches you in simple terms what most people will never know about how to not suck at science. Fitness Science Explained covers all of the big moving parts, including . . . The basics of the scientific method The differences between randomized trials and observational studies The power of the placebo effect The importance of sample sizes The anatomy of statistical analysis And much more In this episode, I’ll be sharing the third chapter of the audiobook “Welcome to the Hierarchy (of Evidence)” So, whether you want to discover and use evidence-based methods for building muscle or losing fat faster, reducing your risk of disease or dysfunction, or maximizing some other aspect of your body, mind, or life, this book will show you the way. Click here to get your copy now: ⇒ https://legionathletics.com/products/books/fitness-science-explained/ And get ready to learn how to use science to get fitter, healthier, and happier. Go for it! P.S. Also, to celebrate this joyous occasion, I’m giving away $1,500 in Legion gift cards! All you have to do for a chance to win is… 1) Buy a copy of Fitness Science Explained (any format) 2) Forward the receipt email to [e-mail] . . . and voila, you’re entered in the giveaway. You have to act fast, though, because the winners will be chosen on Friday, September 4th. --- Mentioned on The Show: Fitness Science Explained: https://legionathletics.com/products/books/fitness-science-explained/ --- Want to get my best advice on how to gain muscle and strength and lose fat faster? Sign up for my free newsletter! Click here: https://www.legionathletics.com/signup/

Hello, hello, and welcome to another episode of Muscle for Life. I'm Mike Matthews. Thank you for joining me today. And this episode is special because it is one of the chapters of my newest book, well, newest audio book in this case, but it's also available in digital and hard copy formats as well. And it is called Fitness Science Explained. And you can get it right now at fitnesssciencebook.com. And this book is a crash course in reading, understanding, and applying scientific research. And it teaches you in very simple terms what most people will never know about how to use science to optimize your health, fitness, and lifestyle. Fitness Science Explained covers all of the big moving parts, including the basics of the

scientific method, the differences between randomized trials and observational studies, the power of the placebo effect, the importance of sample sizes, the anatomy of statistical analysis, and much more. You'll also learn in the book how to get access to full-text studies without spending a fortune, and the most popular journals out there for exercise, nutrition, and supplementation, and you will get a scientist-formulated cheat sheet that will help you quickly and accurately estimate the quality of research that you want to review. In my opinion, the cheat sheet alone is worth the cost of the book. It is very, very practical. So whether you want to discover and use evidence-based methods for building muscle or losing fat faster, or maybe reducing your risk of disease or

dysfunction, or just maximizing some other aspect of your body, mind, or life, this book will show you the way. Also, to celebrate this joyous occasion, I am giving away $1,500 in Legion gift cards. And all you have to do for a chance to win is head over to fitnesssciencebook.com, buy a copy of the book, any format, and then forward your receipt email to launch at legionsupplements.com. And that's it. You are entered in the giveaway. You got to act fast though, because winter is coming. No, because winners will be chosen this Friday, September 4th. Oh, and you can also increase your chances of winning by buying extra copies of the book. Again, any formats.

So specifically, if you buy three copies, you're going to get five giveaway entries. So that is a plus 400% chance to win. If you buy five copies, you're going to get eight giveaway entries. And that is a plus 700% chance to win. And if you buy 10 copies, you're going to get 15 giveaway entries, which is a plus 700% chance to win. And if you buy 10 copies, you're going to get 15 giveaway entries, which is a plus 1400% chance to win. And you are going to get an autographed copy of the book. So for instance, if you buy the paperback ebook and audio book, that is three copies, five entries to win. And if you were to buy three paperbacks, as well as the ebook and audiobook, that's five. So

you get eight entries to win and so forth. Anyway, to get your copy or copies of Fitness Science Explained, just head over to fitnesssciencebook.com now. You have to act fast though, because winners will be chosen at the end of the day today. This is your last chance to win a Legion gift card, and I'm giving $1,500 in gift cards away. All right, let's get to the episode. Chapter three. Welcome to the hierarchy of evidence. If I have seen further than others, it is by standing upon the shoulders of giants. Isaac Newton. You now know how science fundamentally works and the role that evidence plays in the scientific method. You also know that not all evidence is created equal. There's high and low quality evidence, and the ultimate goal of any line of research is the accumulation of high quality evidence for a given hypothesis.

What determines the quality of evidence though and why do some kinds of evidence carry more weight than others? Answering this question can seem all but impossible when you're poking around online. Every time you think you know what a study means, someone else comes along and points out why another better piece of evidence trumps that one. The good news is that scientists have a proven system for ranking evidence called the hierarchy of evidence. A hierarchy is a system or organization in which people or groups are ranked one above the other according to status or authority. Here's how the hierarchy of evidence

works, with the highest quality evidence at the top and the lowest quality at the bottom. Systematic reviews and meta-analyses of RCTs. Randomized controlled trials, RCTs. Observational research. Anecdote and tradition. Let's take a closer look at each, starting at the bottom with the weakest form of evidence. Anecdote and tradition. Anecdote and tradition represent the lowest quality of evidence. Anecdote refers to things like, it worked for me, or my friend got great gains from using this supplement. And tradition refers to things like, everyone does it this way, or bodybuilders have always trained like this.

These are valid forms of evidence. The scientific method is relatively new, especially as applied to the fields of health and fitness. So anecdote and tradition are what most people relied on for years. And it tends to get the basics right. Ask many experienced bodybuilders and athletes what you should do to get in shape, and most will probably tell you, lift weights, do some cardio, and eat lots of fruits, vegetables, and lean meats. The problem, though, is that you'll get plenty of conflicting opinions, too. If you only rely on anecdote and tradition to make decisions about what's true and what isn't,

then you can waste years second-guessing yourself, jumping from one diet, supplement, and exercise plan to the next. Sound familiar? The reason anecdote and tradition are considered low-quality evidence is because there are too many unknown variables and things that aren't controlled. For example, just because bodybuilders have traditionally trained a certain way doesn't mean it's the best way to train. Many aspects of bodybuilding training are based on ideas

that have been passed down from one generation to the next. People tend to engage in herd behavior, following what everyone else is doing, even if it's not necessarily optimal. So there may not be any truth to these things that everyone knows about bodybuilding. Another example is that just because a particular supplement has worked for someone

doesn't mean the supplement generally works. Perhaps the psychological expectation that the supplement would work. Perhaps the psychological expectation that the supplement would work resulted in the person training harder and paying more attention to diet, which would explain the better results. Perhaps the person was simultaneously doing other things that were responsible for the improved gains. Or perhaps the individual was already improving his body composition

and the supplement had nothing to do with it. As you can see, anecdote and tradition occupy the bottom of the hierarchy of evidence for good reason. Observational Research Next on the hierarchy is observational research, which you briefly learned about earlier in this course. With this type of research, scientists observe people in their free-living environments, collect some data on them, and then look for relationships between different variables.

In the world of disease, this type of research is known as epidemiological research. An example of this type of research would be where scientists take a very large group of people, think thousands, assess their dietary habits and body weight, and then review those numbers 10 years down the road. They may look at how many people had heart attacks during that 10 year period, and then use tools to see whether heart attacks could be related to diet. This type of research is higher quality than anecdote or tradition because the data has been systematically gathered from large numbers of people and formally analyzed. However, as mentioned earlier, it can only establish correlations, relationships, not cause and effect. For example, you might be able to show that higher fat intake is related to heart attack risk

regardless of body weight. That doesn't mean higher fat intake causes heart attacks, though, since there are other health variables like diet and exercise that can influence that relationship scientists can try to use statistics to reduce the likelihood of other variables skewing the results but they can never fully isolate one variable as the causative factor in an observational study in other words they can point you in the right direction but that's about it that's what these studies are typically used for, justifying more expensive detailed studies on a particular topic. And that leads us to the randomized controlled trial.

Randomized Controlled Trials, RCTs. Next, we have one of the highest quality forms of evidence, the Randomized Controlled Trial, or RCT. highest quality forms of evidence, the randomized controlled trial or RCT. In an RCT, scientists take a group of people and randomly divide them into two or more groups, hence the term randomized. The scientists try to keep everything the same between the groups except for one variable that they want to study, which is known as the independent variable. Scientists change the independent variable in one or more groups and then see how the people respond. One group may not receive any treatment or may get a fake treatment,

also known as a sham treatment or a placebo. And this group is often called the control group, hence the term randomized controlled trial. For example, let's say scientists want to see how creatine supplementation affects muscle strength. Here's how the study could go. The researchers recruit people for the study and then randomly assign them to a high-dose creatine group,

a low-dose creatine group, and a placebo or control group, a group that thinks they're getting creatine but aren't. Before supplementation begins, everyone's strength is measured on a simple exercise like the leg extension, and then all of the subjects do the same weight training program for 12 weeks while also taking the supplement or placebo every day. After 12 weeks, leg extension strength is measured again and the data is analyzed to see if some groups gain more strength than others. If the changes are significantly greater, more on significance soon,

as this has a specific definition. In the creatine groups, the scientists can state that the creatine caused greater increases in strength compared to not taking creatine. They can also compare strength gains between the high-dose and low-dose creatine groups to see if a higher dose is more effective. RCTs rank above observational research because their highly controlled nature allows them to establish cause and effect relationships. In the earlier example, assuming the research is conducted honestly and competently, it's

scientifically valid to say that the study showed that creatine causes an increase in strength gain. RCTs do have limitations though. One major factor is the number of subjects involved because, as you know, the smaller the sample size, the higher the risk of random findings. Another is how similar the research environment is to real life. In some cases, the research environment can be so highly controlled that the results might not be applicable to the general population. For example, many studies that look at appetite have subjects consume food in a lab environment. However, the number of foods and the amounts of food given in the lab can impact how much people eat. Thus, how people eat in the

lab may not reflect how they eat in the real world. By and large, though, a well-conducted randomized controlled trial is the most accurate tool we have for deciding what's probably true and what isn't. So, what could be better than an RCT? A bunch of RCTs. Systematic Reviews and Meta-Analyses of RCTs When a group of studies exists on a particular topic, a systematic review or meta-analysis can tell you where the weight of the evidence lies on that topic. This is why they occupy the top of the evidence pyramid. Let's unpack exactly what that means. I hope you are enjoying this episode which is one of the chapters of my newest book fitness science explained which is live right now at fitnesssciencebook.com now this book is a crash

course in reading understanding and applying scientific research and it teaches you in simple terms what most people will never know about how to use science to optimize your health, fitness, and lifestyle. So whether you want to discover and use evidence-based methods for building muscle or losing fat faster, or maybe reducing your risk of disease or dysfunction, or just maximizing some other aspect of your body, mind, or life, this book will show you the way. Get your copy now at fitnesssciencebook.com and forge your email receipt to launch at legionsupplements.com and you will be entered to win a Legion gift card. I'm giving away $1,500

in Legion gift cards to celebrate this joyous occasion. Again, that URL is fitnesssciencebook.com. A systematic review is a type of study that involves gathering all of the research on a particular topic and then evaluating it based on a set of predefined rules and criteria. In other words, the research is reviewed in a systematic fashion designed to minimize researcher biases. This is different from a narrative fashion designed to minimize researcher biases. This is different from a narrative review, where a researcher or researchers will informally gather evidence around a topic and share their opinions about what it means. A meta-analysis is similar to a systematic review, but takes things one step further and does a formal statistical analysis of RCTs. further and does a formal statistical analysis of RCTs. You can think of it as a study of studies because scientists gather a bunch of RCTs that fit predefined criteria and then run complicated

statistical analyses on them to give you an idea where the overall weight of the evidence lies. Systematic reviews and meta-analyses represent the highest level of evidence because they gather the best RCTs and research on a topic and show you what that research generally agrees on. For example, one systematic review and meta-analysis of weightlifting research looked at 21 studies comparing light loads, less than 60% of one rep max, to heavy loads, greater than 60% of one rep max, and how those training plans impacted strength and muscle gain. The analysis showed the gains in strength are greater with heavy loads compared to light loads, but gains in muscle size were similar as long as sets were taken to muscular failure. This type of analysis is superior to any individual study because it produces findings that are more likely to be true for most people.

While systematic reviews and meta-analyses represent the highest level of evidence, they still have flaws. Poor study selection or statistical analysis can lead to poor results because these types of studies are only as good as the data and methods used. For instance, if you take a bunch of studies with only as good as the data and methods used. For instance, if you take a bunch of studies with mediocre results, combine them into a meta-analysis, and then run them through the right statistical tools, you can make the results look more impressive at first glance than they really are. A good example of this is a 2012 meta-analysis on the effects of acupuncture on chronic pain that reported that acupuncture reduced pain 5% more than sham acupuncture on chronic pain that reported that acupuncture reduced pain

5% more than sham acupuncture, placebo. When you look at the details, though, you discover that none of the highest quality studies in the review showed significant effects, and the researchers included several studies with extreme results that were likely the result of bias or flawed research methods, as well as several smaller studies with a higher likelihood of false positive results. Therefore, the study's conclusion that acupuncture is effective for the treatment of chronic pain was misleading and oversold. You've probably asked yourself at one time or another,

with all of the conflicting studies out there, how are you supposed to know what's true and what's not? Now you know the answer by examining how robust evidence is according to the hierarchy. You can understand which claims are more likely to be true than others. You can know, for instance, that anecdotes online or even published case studies shouldn't be weighted more than the results of RCTs, that the results of an individual RCT aren't the final word on a matter, and that research reviews and meta-analyses can be scientific treasure troves.

Key Takeaways Scientists have a proven system for ranking evidence called the hierarchy of evidence. A hierarchy is a system of organization in which people or groups are ranked one above the other according to status or authority. Anecdote and tradition represent the lowest quality of evidence. Anecdote refers to things like, It worked for me, or my friend got great gains from using this supplement. And tradition refers to things like, Everyone does it this way, or bodybuilders have always trained like this. The reason anecdote and tradition are considered low-quality evidence is because there are too

many unknown variables and things that aren't controlled. Next on the hierarchy is observational research, where scientists observe people in their free-living environments, collect some data on them, and then look for relationships between different variables. This type of research is higher quality than anecdote or tradition because the data has been systematically gathered from large numbers of people and formally analyzed. However, as mentioned earlier, it can only establish correlations, relationships, not cause and effect. Next, we have one of the highest quality forms of evidence, the randomized controlled trial, or RCT.

In an RCT, scientists take a group of people and randomly divide them into two or more groups, hence the term randomized. The scientists try to keep everything the same between the groups, except for one variable that they want to study, which is known as the independent variable. RCTs rank above observational research because their highly controlled nature allows them to establish cause and effect relationships. When a group of studies exists on a particular topic, a systematic review or meta-analysis can tell you where the weight of the evidence lies on that topic. This is why they occupy the top of the evidence pyramid.

Systematic reviews and meta-analyses represent the highest level of evidence because they gather the best RCTs and research on a topic and show you what that research generally agrees on. Well, that is it for this episode of Muscle for Life. I hope you liked it. And in case you did not hear the intro or the mid-roll ad, this was one of the chapters of my newest book, Fitness Science Explained, which is live right now at fitnesssciencebook.com.