Invest Like the Best with Patrick O'Shaughnessy - David Epstein – Wide or Deep? - [Invest Like the Best, EP.133]

Episode Date: May 28, 2019

My guest this week is David Epstein. David is a writer and researcher extraordinaire and the author of two great books. His second, Range, is out today and I highly recommend it. We discuss the pros and cons of both the generalist and specialist mindsets in detail and go down many interesting trails along the way. Please enjoy our conversation. For more episodes go to InvestorFieldGuide.com/podcast. Sign up for the book club, where you’ll get a full investor curriculum and then 3-4 suggestions every month at InvestorFieldGuide.com/bookclub. Follow Patrick on Twitter at @patrick_oshag   Show Notes 1:12 - (First Question) – What he uncovered in “The Sports Gene: Inside the Science of Extraordinary Athletic Performance” that led him to his latest book             2:38 – Debate with Malcolm Gladwell (YouTube) 4:12 – What did the public pay most attention to and what did they gloss over 7:56 – How his views on nature vs nurture shifted during the process of writing The Sports Gene 10:05 – Blending practice with your nature 13:04 – His process of reading 10 journal articles a day as part of his research 19:06 – Exploring his new book “Range: Why Generalists Triumph in a Specialized World”, and his idea of Martian tennis 23:03 – Idea of the cult of the head start and how we set up our own feedback loops 28:58 – What does his research say about the nations education system 30:42 – The Flynn Effect chapter 33:54 – Hacks for learning 37:52 – The concept of struggle and harnessing the power of it 46:31 – Personality changes and how to drive those changes in a positive way 52:00 – Using the outside perspective in businesses for more productive outcomes and how it applied to Nintendo             52:59 – Josh Wolfe Podcast Episode 1:04:45 – Other examples of using withered technologies, 3M 1:09:00 – The arc of his work and how it has evolved 1:13:54 – Taking a different view on problems             1:17:52 – Ending Medical Reversal: Improving Outcomes, Saving Lives 1:18:04– Anyway to change these bad trends with new strategies   Learn More For more episodes go to InvestorFieldGuide.com/podcast.  Sign up for the book club, where you’ll get a full investor curriculum and then 3-4 suggestions every month at InvestorFieldGuide.com/bookclub Follow Patrick on twitter at @patrick_oshag

Hello and welcome, everyone. I'm Patrick O'Shaughnessy, and this is Invest like the Best. This show is an open-ended exploration of markets, ideas, methods, stories, and of strategies that will help you better invest both your time and your money. You can learn more and stay up to date at investorfield guide.com. Patrick O'Shaughnessy is the CEO of O'Shaughnessy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Shaughnessy asset management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of O'Shaughnessy asset management may maintain

positions and the securities discussed in this podcast. My guest this week is David Epstein. David is a writer and researcher extraordinaire and the author of two great books. His second, range, is out today and I highly recommend it. We discuss the pros and cons of both the generalist and specialist mindsets in detail and go down many interesting trails along the way. Please enjoy our conversation. So, David, thanks for being here with me. We've got a whole ton to discuss covering your two books, a lot of your other writing. I know you're interested in a million things. We're going to go all over the place. Since the antecedents for your latest book range really are buried in your first book, the sports gene, I thought it would be fun to start there and really focus on what you discovered

in the first book's research that led you to get so interested in this idea of specialization versus kind of broader set of interests. So maybe talk about that, what you uncovered, the kernel that you uncovered in the sports gene that's led to this later research. First of all, it was, I noticed that some countries that were sort of turning around their national sports programs, like at the Olympics had embraced what they called talent transfer programs,

which was taking adults who'd been practicing, who had a broad history of sports participation, a bunch of different sports, were trying to make national teams, weren't making it in whatever they were training in, and said, hey, why don't we let you try these other sports before we kick you out of the developmental pipeline? And all the countries that were doing that, like Australia, the UK, not only gold medalists, but world record holders out of adults who had never participated in their sport at the Olympics prior to when they won the gold medal.

And this sort of went so contrary to the 10,000 hour rule thinking, right, where you'll get behind if you switch, that I started thinking more deeply about that. And eventually, when the book came out, obviously I critiqued the 10,000 hour rule in the sports gene. and I got invited at the MIT Sloan Sports Analytics Conference to debate Malcolm Gladwell, 10,000 hours versus the sports team. It's on YouTube, even though we have some common ground for sure. And I'd never met him before. Very clever, and I didn't want to get embarrassed. And so fear-motivated researching.

I anticipated what I thought he would argue. One of his arguments I figured would be that a head start in deliberate practice or highly technical focused practice. And whatever you're going to do is like an insurmountable advantage. And so I said, okay, if that's the hypothesis, Let me go look in the research and see what we know about the development of athletes who go on to become elite from childhood. And it turned out to be very much the opposite. So the athletes who go on to become elite have early on what's called a sampling period.

They do a wide range of sports. They gain a breadth of general skills. They learn about their own abilities, their own interests, and they systematically delay specialization later than their peers. So I go to this debate with Malcolm and I frame this as the Roger versus Tiger problem. Tiger Woods is the most famous maybe ever developmental story. walking at six months, golfing at 10 months. And Roger Federer, where we don't really know that story so much, played like a dozen different

sports as a kid. Mother was a tennis coach, refused to coach him, forced him to continue playing badminton basketball soccer long after his peers had specialized. And he turned out okay. And so my question was, which one of these is the typical developmental model? And it turns out that in almost all sports, it's the Roger model, not the Tiger model. And so that became the afterward of the sports gene. And furthermore, golf is a really bad model of almost all things that humans want to learn.

And so it sort of grew out of the sports gene and then out of this debate that I prepped for with Malcolm. So in both books, you cover a ton of different ideas. We'll cover a lot of them today. I'm always interested in books like that, that it's not just one core idea. It's sort of a collection of things that often afterwards the public will glom on to a couple core ideas from the book. So before we leave the sports gene, I would just love to hear your. perspective on what the things are that the public really paid attention to, and then maybe what the things are that maybe they tended to gloss over, which you most enjoyed about the first

book. I was not good at predicting what people would glum on to necessarily. So I thought the chapters about race and about gender would be by far the topics of discussion. And in fact, it was the 10,000 hour rule. The sex and gender chapter is actually kind of coming back right now because of debates in sports about how to separate the female and male classifications in sports for people who have differences of sex development or transgender athletes. But it was by far the 10,000 hour rule stuff. So the 10,000 hour rule, as I explained in the sports gene, is based on a study of 30 violinists who were so highly pre-screened.

They were already in a world-famous music academy. This is a problem that statisticians call restriction of range. You've restricted your subject range based on something correlated to your dependent variable. And when that happens, you can have all kinds of weird results. So I liken it to, if I were going to do a study of what causes basketball skill, used only centers in the NBA in my subject sample and said, okay, they all practiced a lot, therefore only practiced got them where they are, not practiced plus being seven feet tall because I've squashed that range. And in fact, I've done the analytics on this. In some years, there's a really high positive correlation between height in the American population and points scored in the NBA. But if you restrict your range to only NBA players, it turns negative.

So if you were a scientist who did that study in NBA players, you'd give parents the advice to have short children for them to score points in the NBA if you don't recognize the restriction of range. And so the other thing that 10,000 hours was that so the top 10 violinists in this school had practiced on average 10,000 hours by age 20. But there was tremendous variability. Tremendous. This average kind of squashed all that variability. And so I just explained the background of this study, where it really came from, what you could really infer from it, looked at studies that tracked development. in a more systematic way and larger music studies and saw that that wasn't the case in larger studies. People still had sampling periods and things like that. And told this story in the book

called The Tale of Two High Jumpers, where one high jumper basically spends 20,000 hours practicing and one like randomly discovers that he has this talent and they end up competing against each other. And that became like one of the most popular stories, this story that I used to display the variability around the 10,000 hour rules. So that one and the last chapter about the Finnish cross-country skier who has this genetic mutation that basically causes his blood to be naturally like what Lance Armstrong's was with technology. Those two things people really, really caught fire in a way that I didn't expect.

Some of the things that were the closest to me, there was a chapter about sensitivity to pain and the extent to which that is modified by your nature and your environment. And in that chapter, there is a section about sudden cardiac death and explains the sudden death of my own training partner at the end of a race. And that is the reason why I got into journalism to write about sudden cardiac death and athletes. I was geology grad student living in a tent in the Arctic, you know, I was going down a different path. And I decided to merge my interests in sports and science and science and athletes. And in fact, that's sort of what led me to become the science writer at Sports Illustrated. But that was some of the stuff closest to my heart and that is most

traumatic to me and went totally under the radar in the book. So it shows my bias of what, like, I focused on. In the process from, and maybe you could touch on some of the what you thought were going to be the popular chapters in the race and gender thing in terms of kind of this nature versus nurture argument, from when you started doing the research for that book to when you ended, what was your move on that spectrum? I'd be curious what you kind of your priors were of the nature nurture debate in sports specifically, I guess, but we're going to talk about that more generally too in skills and how much the needle moved during the process of writing that first book. Yeah, it moved a lot from me. I mean, I mean, the answer ultimately becomes things that I thought

that were completely nature, like the reflexes it takes to a hundred mile per hour fastball turn out not to be that at all, to be totally learned. And other things that I thought were completely volitional, the will to train, turn out to have these incredible genetic components. And so it turned out most of the things I wrote about were contrary to what my intuition was. And I really want to go back and read my book proposal because the books that I likened my book to in the proposal were like outliers and the talent code and talent is overrated. And in the end, my book turned out being said exactly in opposition to those books. I think there's a lot more nuanced than that. The seed of that book was a 7,000-word article I wrote for Sports Illustrated about genetics

that basically interpreted a lack of certain types of evidence in genetics for evidence of absence that genetics matter. Right. So this is like a little secret that nobody's really called me on in the sports gene, which is that I cite one of my own Sports Illustrated articles as being wrong in my book, which is scary because you can spend a couple months. And it's not like there was a fact wrong, right? It passed through fact checking. The published papers were there. The people with the PhDs who did them were telling the fact checker. But the problem was they were making inferences from their data that their data did not support. And once I had a year, so my process for writing a book, in the first year, I don't write, try to read 10 journal articles a day every day for

the first year. And after I did that, I realized that they were making conclusions they could not make with their data and they were ignoring other data. And so the book ended up focusing, I think it was still very much balancing nature and nurture, but focusing a lot more on nature than any other book had because I started to realize that evidence had just not been discussed and not kind of plumbed with the same depth that a lot of the nurture evidence had been. We were talking before we hit a record about the interesting analogy between like health and sports and investing and this constant desire to intervene.

Everyone reads these books and wants to know, okay, like, what's the couple sentence summary of what this means I should do for myself, for my kids, for my friends, you know, whatever it might be. And obviously there's always more nuance than that, right? And people want the snack versus the full meal, which I get. One interesting idea in the sports team is maybe a simple practical outcome of your research is to practice which you're naturally good at. Maybe talk a little bit about that that blends the practice and nature. That's true. So I'd say right after the 10,000 hour chapter, one of like the next tier of the ones that were the most popular was one about trainability. that suggested this, like anyone who's ever been in a training group or gone on a certain, like, fad diet or whatever,

they noticed that, and I mean I noticed this in my college track training group, we're all doing the same things. And in many cases, getting more different, not more the same. And it turns out that that's one of the same lessons like we've learned about from medical genetics. You might need three Tylenol, well, I need one because of your genes involved in a xenaminifin metabolism or different from mine. Or maybe it just doesn't work for you at all. the same thing is basically true about the medicine of exercise or training. The American College of Sports Medicine slogan is exercise is medicine. I think that's very apt for that reason because no two people have the same exact response

to that given stimulus. So what you really want to do is find what fits best for you. Where do you get the best response? And not only that, but in some cases, we always judge talent based on what we see right now. And I think one of the revelations in the sports gene for me was that in some cases, like the more important talent is trainability. And in some cases, your ability to improve with training is uncorrelated from how good you are at baseline. So it's a really different – so in one of these studies where people were being trained for endurance, the researchers said, okay, on day one, here are 10 most talented people, put everyone on identical training and missed 100 percent of the people who ended the study looking the most talented because we have this differential response to training.

And so I think in the book, one, more people are screened out of elite performance now in sports by either their nature or their nurture than ever before because it's so competitive. I think one, you need a sampling period to figure out where you fit. You want to delay specialization because when we pick before people are, go through puberty, especially, the more likely you put the wrong person in the wrong slot. And also then you need to be able to experiment to find the best training environment for yourself. So I think when we are directing people to particular sports before they have a chance to learn about themselves and their own abilities, we are minimizing the chance that they find the right fit. And that's why these talent transfer programs that like Australia and the UK and to some

extent China started, have produced Olympic champions from people who have just started their sport is because there is so much room still to gain with fitting people into the right sport. But they won't do that for themselves because the feeling is you'll be behind if you switch. And so just giving them that liberty to go exploring, using what they've learned about themselves and see if they can fit into another sport, has produced Olympic champions, even given how competitive things are today. You mentioned something in your own research process, which is fascinating, which is the idea of reading 10 journal articles every day for a year.

era. I didn't make it every day, but I made it most days. Yeah. I think most great books on creativity start with this process. Like it's just a collection exercise early on without judgment or bias, just like get information. I would love to hear a little bit more about that process. I'm surprised that there are that many journal articles to read on a topic like this. So talk through that. Is there anything you're looking for? I think this could stand as an analogy for anyone trying to get into anything, right, that they want to be in this collection phase, a simple exploration phase early on. Any special insights from that period of your research? Yeah, it's interesting. You mentioned that because that you're surprised there are that many journal articles. The fact is sometimes

I end up down rabbit holes that turn out I surface like a week later and say, how the hell did I ever think that that was going to be useful. And I used to really chide myself for that because it's so inefficient. And I'm not sure if I've just come around to feeling self-send, to like a self-serving message or it's true. But my thinking now, and partly this thinking is influenced by the research that went into my new book range is that that is actually one of my competitive advantages, the time and the willingness to have this expansive personal search function where I find and connect things that other people don't and that I cannot eliminate the inefficiency there without eliminating some of like my competitive advantage, basically. The other important thing I do is there's a psychologist and statistician

who I essentially keep on retainer sort of. He would do it because like we just really like doing this. I think it's fun for him, but I don't want to like abuse his time. I mean, I do want to abuse his time. I mean, I do want to abuse his time, but I want him to feel good about me abusing his time. And basically so that he'll just, if I have a question about methodology of a paper and things like that, we'll talk any time. So in that process, we talk constantly, basically, while I'm trying to figure out, am I seeing this right? Just another brain, basically, to weigh in on that. And I started to notice, as I was thinking about this, quotes from people I respect like Eric Larson, Chris Nolan, where they would say, well, between projects, I really just need to read without any purpose whatsoever. And I sort of feel that way about it too, is that we want to set aside some time where I think

it's recommendations, algorithms and things are great sometimes, but also they can kind of, they're comparing you to a bunch of other people that they think you're just like, right? And sometimes they can really track you into a certain set of interests. And if I go off into like a library or rare books archive or I just follow one journal to another, what I would do for a while is I'd go to, I had an alumni reading card at Columbia University, and there were four computers that were logged simultaneously logged into every journal. that the university had access to. So I'd go there, USB drive, read the article, all the hyperlinks and the citations.

I could download 300 articles in a day or something like that and take it back. And there you can follow it right away because they're all hyperlinks. So I would just go sit there open until close. And again, I think the difficulty is it's seen as very inefficient. And I think I'm very fortunate and privileged to have gotten the space to be a little inefficient and find things other people's don't. And I think my last job was at ProPublica. and I left to finish this recent book,

but I think one of the sort of bits of genius about ProPublica, whatever you think about the individual stories, was they brought people in, say, get your arms around as big a project as you can, and they accepted that that means some of them will fall through. Like when I was there, there was someone who worked on one project for a year that fell through. They had nothing to produce after a year,

and they were okay with that. Of course, the writer was horrified. But I think you have to sort of tolerate that and that going off and being inefficient in your search, you land on things that other people don't because you're looking in different places, but it's hard to convince people that that's okay to do. One of the characters in range this world-renowned scientist says everyone brings like lunch back to their desk now and really like I want them crashing into each other,

even though that seems really inefficient, basically. Is there a single article or discovery as part of that process that was most surprising in your memory thinking back on either book or on any research project actually? For sure, when I started reading the studies that were the foundation for the $10, hour rule, school of thought. And I was like, all these very famous writers have written about this. I must be wrong. And I'm looking at this stuff. There are no measures of variance. The accounts of the practice hours are inconsistent on multiple attempts, which normal, but very inconsistent. What am I missing here? This doesn't look like rigorous research to me.

And then I started to say, okay, let's start at the most basic thing. If only hours of practice matters, then how come I broke the women's world record in the 800 meters after two years of practice? This is a genetic difference that matters. Let's just start at the most basic. And I started talking to a prominent scientist in the 10,000 hours school about that. And I remember she sent me a paper, and we're kind of arguing in a very civil way. She sent me a paper that said, look, this even holds for other organisms. If you look at racing dogs, racing whippets, they have equivalent to their lifespan 10 years, 10,000 hours or 10 years rule of practice before they become elite. And I'm going through the citations of this paper. And one of them notes that 40% of those

dogs also have this incredibly rare genetic mutation that causes them to be really fast and have more muscle. And so I'm like, these people have total blinders. I can't believe she would send me this paper as evidence of this. And so I started to have this turnaround where I said, gosh, I've written some things that are wrong. A bunch of other people are writing some things that are wrong. And there's a real opening here for someone to come in and kind of dissect this work. And so that was a big surprise to me. When I decided I wasn't, I wasn't just missing something. And in fact, you know, I was right about that. This idea of narrative violation is really interesting. And one of the lines in your writing, I think it was in the story on Stents in Hearts, there's a line that says

something like, our cultural rewards discovery, not replication. And it's a huge problem. Again, both in health, probably in sports and investing. You get these ideas that then 10,000 hour rule has this inertia. Like, it just won't go away. And I'm sure even after, after listening to this, people will probably still believe that. And I probably, you know, there's probably elements of myself that will still believe it. Well, there are good things from it, too, right? Like the emphasis on practice and the practice matter. No, but uncontroversial, the practice matters. Yeah. Let's bridge now to your new work in range. And I think this is an incredibly powerful set of ideas. There's a lot of things I want to go into. You already laid out the

Tiger versus Federer kind of framework. Maybe flesh that out a little bit more with this idea of Martian tennis and how this analogy is helpful, but it's not perfect, and that the real world is neither of these people and neither of these sports. Right. This was a tricky thing to get to, because again, when I first did the book proposal, the proposal was called Roger, for Range, was called Roger versus Tiger. So, and the book opens with that thing that I call the Roger versus Tiger, problem where I tell the Tiger Woods story, again, incredibly precocious. Father's media training him at age three because he realizes he's, he's got something special, whereas Roger's totally opposite. One of my favorite little anecdotes about Roger's early

life was his parents were, as a Sports Illustrated writer, called it Pulley, not pushy. They were trying to get him to take tennis less seriously, even once he wanted to focus. And once he got good enough to be interviewed by a local reporter, the reporter asked him what he would buy with his first check if he ever became a pro. And he says a Mercedes, his mother's aghast, and asked the reporter, can she listen to the recording? And she listens, and what Roger actually said was, Mer Cs Cs in Swiss German. You just wanted more CDs. And she's like, all right, it's okay. We've still got him under control. But so my plan was to contrast those developmental paths and say, let's look at different domains.

When is it best to be a Roger and when is it best to be a tiger? And what I realized was that in some ways those were insufficient because all of sports is a very limited analogy, which again is sort of a problem because I think of the ways that I've used sports as an analogy. Once the sports gene sort of took off and I got asked to speak about it in certain ways, having done range, I'm more tempered in the way I'm willing to use sports as an analogy now. So Robin Hogarth, this psychologist, you mentioned Martian tennis, characterized, he was trying to address this question of when do people just get better with experience. And when they just get better with experience is when they're in what's called kind learning environments. A kind learning environment is where the next steps are clear, all the information is available, the goal is right in front of you.

Every time you do something, feedback is immediate and perfectly accurate. And so if you just do it and are cognitively engaged, you get better. golf. People take discreet turns, those sorts of things. Tennis is somewhat less kind because you have to deal with teammates and things like that, and you have to use what's called anticipatory skills where you have to judge what other people are doing and things like that. But it's still more on the kind end of the spectrum. The information is available. You get quick feedback. On the other end of the spectrum, the far end are wicked learning environments, which are actually in the wider world where most of us are. We're not in a game that has lines drawn that we have to play between.

We can't see all the information. We don't wait for the next person to take their turn, et cetera. Information is hidden from us. Feedback is delayed, if given it all, sometimes inaccurate. And one of the examples Hogarth uses is this famous New York City physician who was renowned for his ability to predict that patients would get typhoid. And he could do this time and again. He would palpate their tongue, feel around their tongue, and predict weeks before they had a single symptom over and over. They get typhoid.

It became very famous for this. and as one of his colleagues, another physician later pointed out, he was a more productive carrier of typhoid than even typhoid Mary because he was giving these patients with his hands, typhoid. So in that case, the feedback he was getting was reinforcing exactly the wrong lesson. Okay, so that's like the extreme of a wicked environment where your feedback teaches exactly the wrong lesson. Most things aren't as wicked as that, but most of the things outside of sports are more toward the wicked end of the spectrum. And so Robin Hogarth said, usually we're not playing golf. We're not even playing tennis. We're usually playing Martian tennis.

You see people out there playing. Something's going on. You don't know the rules. It's up to you to deduce them. And they could change at any moment without notice. And that's the situation that we're actually in for most of the things, the complex things that most of us care about. There's a great term in the book which starts to bridge into this idea of maybe avoiding early specialization is a smart thing to do, which is the cult of the head start. and I'd love you to weave that together with this notion of how we want to set up our own feedback loops.

Because obviously everyone likes a tight feedback loop, and I think in many ways getting tighter feedback loops in life would improve our lives. But in some ways it's actually really bad. So talk about the cult of the head start and the notion of feedback loops. In the more general sense, the cult of the head start was this, I started to see in different domains that the easiest way to give someone an advantage is to teach things they're usually called closed skills. So things that are technique that you can teach very quickly and see an advantage. But those fade out or they appear to fade out because everybody actually learns them. And then the people that were learning these sort of broader frameworks for skills will surpass those people. But tell me not to if you don't want, but you asked before what the most surprising study was in the sports gene.

And this question just prompted what the most surprising study was for range. Okay. The study at the Air Force Academy in the fourth chapter where I apologies to Danny Kahneman titled Learning Fast and Slow. So this study at the Air Force, the Air Force has an incredible setup for a kind of natural experiment that you'd never be able to replicate in a lab, really. And they wanted to study the impact of teacher quality on math learning among their cadets. And every year a freshman class comes in, the Air Force, and they have to take like three levels of math, calculus one, calculus two, and then they go on to more advanced math. and every year it's like 20 kids are in each calculus one section they are randomized to professors and the test scores and all that end up everything the skills they came in with are randomized across

the classes after that class they all get the same exact test the tests are graded by groups that just do like a certain question so no teacher can like favor their students the test is like the whole grade and then those kids are re-randomized to calculus two same thing and then they are re-randomized randomized again to follow-on courses. And the same thing happens in the engineer courses. They're re-randomized every time. And so they do this with thousands of students over the years. And what these researchers wanted to see was, okay, what's the effect in a given class of a professor, and what's the effect on the student's trajectory? And what they found was that the professors who were the best at causing students to do well in their own class, to do well on the test,

and by well, they meant overperforming compared to the baseline characteristics they came in with, systematically undermined those students' performance in the follow-on courses, which is incredible. So the professors, out of like 100 professors they tracked, the one who was sixth in performance, students in his own class performing above what they'd be expected, and seventh in student evaluations, because the students evaluate them well usually when they do well or they feel like they're learning, was dead last in what they call deep learning, which was how well his students did going forward. And so what happened was the teachers that taught a more narrow curriculum were better at prepping their students for the test they would face. But because they didn't give them this broader, what's called making connections framework where they would connect ideas in math, they were systematically at a disadvantage for all the classes going forward.

And the bigger problem was that those students, because they see their progress so quickly, because they're learning this narrow set of skills, they rank those professors that undermine their long-term development. really highly. So there was a negative correlation between deep learning or how well the students would do going forward and how well they rank their professors. So that's really wicked feedback. Did they get into what was causing maybe, I don't know if it was the exact inverse, where some of the teachers that weren't having students do as well in their own class, but then setting the kids up for greater success long term, what those teachers were doing right? Yeah, basically, part of it was they were ignoring what was on the test, essentially. And by ignoring what was on the test, they ended up teaching a much broader conceptual curriculum.

So what they did was they used more what researchers call making connections knowledge and less using procedures knowledge. So using procedures is like there's a certain kind of problem, you know, why this formula or whatever. Exactly. More like that. And making connections is much more like, let's try to figure out if this formula always works or why does it work or can you find a place where it doesn't work. And this stuff is kind of frustrating. And you try to pick single problems that have multiple modes of attack. And the using procedures knowledge, you get really good at that.

And if you see the same problem, using procedures is great if it's golf, where you're going to see the same situation over and over again. But for these follow-on courses, what you want is someone to have knowledge that they can apply to situations they've never seen. And in that case, you want this broader conceptual knowledge, this making connections knowledge. What you really want to teach them is how to match a strategy to a type of problem. And that's what those teachers who were doing better were doing a better job of teaching in ways that produced so many different scenarios. The students got frustrated, but they started to learn, develop these more abstract models of how do I match a strategy to a problem? This showed up in other types of research in the book.

Like in that same chapter, there was training for people to respond to naval threats. And people were being trained were split up into two groups, so digital simulations. One group trained on a certain type of threat over and over and over and over and over and over again, and they make super progress. other groups same number of training rounds but mixed up all crazy they're never seeing the same thing twice group a that's doing the same thing makes tons of progress feels really good rates their learning experience highly all that stuff group two didn't perform well feels like crap rates their learning experience low when they bring them back like a month later and give them new scenarios that

none of them have ever seen group two destroys group one and that's what you want that's like kind of a theme of the book is when you want people to be able to apply knowledge, whether that's physical or cognitive, to situations they have not encountered before, you need to teach them early on in this much broader way, so they build these conceptual constructs. What do you think this batch of studies and other ones that you found say about our early education system and maybe our entire education system up through college? Does it suggest that we should have more of a liberal arts broad kind of education? Or my sense is that we're moving more towards managing to the test, managing to the SAT, managing to whatever, and away from no specific

goal in mind, don't teach to the test type education. So I very consciously stayed away from making the arguments for a liberal arts education of freed Zakaria has a book that does that. And I wanted to stay away from that because I don't necessarily view that as inherently more valuable than, you know, a lot of people get a liberal arts education and don't know anything about science. And I wouldn't consider that a well-rounded education. So I think more than about what they should study, it's about how we study the things we

study. And by the way, there have been other studies now showing that professor evaluations, this may be different for like middle school and elementary school. But at the college level, the professor evaluations are often like inversely related to in like STEM courses, especially how well the students end up doing later on. And I think it is a problem when we incentivize professors and students, when their full incentive basically looks like doing as well as they possibly can on a short-term test. Then what you get are strategies that produce short-term skills and learning that evaporates when they start facing these more novel situations. What you really want is information to stick and to be flexible. So ultimately, you want them to be able to apply

it in situations they've never seen. And teaching narrowly and using procedures knowledge is basically a way to undermine that. And so I think we need methods to evaluate, right? But I think if we are too focused on these narrow sets of tests that can be a big problem. Speaking of ways to evaluate, you talked a bit about this idea of the Flynn effect in the book, and I'd love for you to recount the idea there. I was surprised by you show that classic picture. I don't know what it's called. The test is called.

You ask people to identify whether one ball is bigger than the other. You could describe what that is. And I was surprised by the kinds of people that make certain choices. So talk through that idea. Yeah, that's called the Ebbinghaus illusion anyway. But that chapter of the Flynn effect is the finding that. IQ scores around the world are going up at a rate of about 0.3 points per year or three points per decade over starting in the 20th century steadily. And it's not just the bottom of the curve moving

up or it's not just the top of the curve. It's like the whole curve shifting to the right. And in fact, it's not only that, but it happens where it was least expected on the most abstract portions of tests. So the parts of tests, cognitive tests that are on things you might learn in school have not moved so much. But tests like the Ravens Progressive Matrices is this test. that's totally abstract. This was created to be the test that's what's called totally culturally reduced, meaning nothing you learned in life or school matters. You get these abstract designs, one's missing.

You have to deduce the rules from Martian tennis and figure out what pattern is missing. And this was the test that if Martians landed on Earth, this was the one that could tell how clever they are because you don't need to have learned anything. It's just pure reasoning skills. And what Flynn saw, James Flynn, who the Flynn effect is named after, is that this is the test where the biggest gains were occurring over the years, the one that was supposed to stay stable. And why is that?

And it turns out what he suggests is that we have moved to a world where we are used to classifying things, to grouping things, instead of being stuck with lots of concrete knowledge and factual knowledge, we face a world that constantly a work world, especially. He found most of the effect was from modern work, as opposed to modern school, which seems to have some effect, though. But we're in a world that forces us to constantly be playing Martian tennis and trying to deduce rules when we've been given on whether that's learning

how to play a video game or operate a computer without an instruction manual or how to coordinate work with people that are far away. We constantly have to do these abstract things that allow us to classify sets of knowledge and transfer it between domains. And when he looked at research done on what I call pre-modern people in some of the studies where they are no worse. Like one way of life is not better than the other. We're just adapted to different things. But people who are like subsistence farmers, classification doesn't really make like any sense to them. If you give them a hammer in axe, a knife, and a bird, and ask them which one doesn't belong in these studies, they'll say, like, well, there's no use for just like the hammer and the axe unless you have the bird because you

need something to cut up. And so you could go this way or could go that way. Like we would say, like, well, three are tools and one, you know, is an animal. And where this kind of classification is very native to us. And so, of course, we would suck in their domain. But for our world where we're constantly laterally translating knowledge to different areas that we've never seen, this ability to have knowledge that we don't have from hands-on exposure is really important. And that's shown up on these tests where we do much, much, much, much better at abstract thinking, which is a signal that we really are well equipped to move our knowledge between domains in a way that our forebears were not. I want to come back to this feedback idea. And you already mentioned the chapter

title, learning fast and slow, which I really liked. And the conclusion that we should want to learn probably slower than we do. So I'd love to hear that kind of translated to maybe advice is the wrong word, but how people should maybe change their behavior based on your findings in this part of the book. Because I've changed my behavior a little bit based on those findings. And here's like the concrete tip. We just went from a very abstract one. Here's like the more tips from learning fast and slow. There are, I get like on at least a monthly basis, probably more often press releases about learning hacks. There's basically like five learning hacks that are really supported rigorously by research and largely ignored in popular media. And three of them that I focus on in that

chapter are testing, spacing, and interleaving. Basically, testing is actually, which, you know, I just denigrated, is wonderful for learning. In fact, you want to test people before they've had a chance to study because it actually turns out it primes your brain for when you then hear an answer to retain it. Even if you get stuff, actually, especially if you get stuff wrong. So there's something called the hyper-correction effect where if you're quite confident about an answer and turns out you're wrong, you're more likely to remember the right one when you get it. So testing before you're ready, spacing, or what I call deliberate not practicing, is you want to leave space between bouts of practice of the same thing. So again, if you practice the same thing over and over and over in one

session, you'll see progress right away. But what you really care about is how long is it stick? So to give you an example of spacing, there was one study, one famous study where two groups of people were learning Spanish vocabulary. One group got like eight hours intensive on one day. The other group got four hours one day and then the other four hours a month later. They had the exact same studying. One was just separated by a month. Eight years later, they brought them back. And the group that had the spaced practice intervals remembered 250% more with no studying in the interim. And so you have a certain amount you want to study. It becomes more efficient if you space it out. You take time of deliberately not practicing. So testing, spacing, and interleaving means just, or mixed practice is like

mixing up the things you want to learn. And this, again, has to do with forming these conceptual models for knowledge. So if you're teaching kids how to learn a certain type of math problem, instead of giving them the same type over and over, let's say actually you have the same 20 math problems that has five of one kind, five another, five another, five another, five another. What we usually do is you give them five of type A, five of type B, five of type C, five type D. You'd be way better off mixing those all together. So again, they will struggle more. It'll take them longer, but they'll learn again more how to match a strategy to a type of problem. So this shows up in all sorts of learning studies. If we were going to go over to the museum, we're like not that

far from MoMA. If we wanted to recognize Basquiat and Picasso and Brock or something like that, I mean, those guys aren't so hard to tell apart. And we wanted to recognize paintings we'd never seen before. And we had a stack of flashcards. Instead of looking at all the Baskiats and all the Barack and all the Picasso, we'd want to shuffle those all together, then try to quiz ourselves on them. And we would be much better when we then go to the museum and identifying paintings, who painted them, even if we had never seen them before, because it forces you to build these abstract models that you can then apply going forward. It reminds me of this past guest had this formula called stress plus rest equals growth, some similar concepts baked in there that I like a lot.

Before talking about struggle a little bit more, because I think that's an important word for learning, What were the remaining two that have good empirical foundations in terms of hacks for learning? Well, one of the other ones that's also in the book, but not in as like tippy of a form, I guess, is using making connections questions. So that's using type of questions that force someone to bring together multiple concepts. So it sort of disables them from being able to do just using procedures knowledge. They have to draw in multiple concepts. And then one of the others had more to do with tailoring learning, basically,

which was like a little more like practical on the grounds for, classroom stuff. So expand on this word struggle because it seems like I think the quote in there somewhere was struggles better than repetition, which you've kind of just laid out. But for those thinking about how they might harness the power of struggle as a means of getting better at something, talk about that a little bit. Yeah. So Nate Cornell, this cognitive psychologist, I quote a bunch in range, had this phrase I like where he said basically difficulty isn't a sign that you're not learning, but ease is. So he studied these things like testing, spacing, and interleaving. And he says, if something's getting easy, you better mix it up, mix it with more things, or stop doing it, go do something

else, and wait until it's hard again. He said, you should wait. There's actually these interesting papers that calculate how long you should wait. If you want to, information to be maximally sticky, they're like known time horizons, how long you should wait. Basically, you want to wait until you've just forgotten it again, until you can't retrieve it. And the reason that seems to work is because it makes it hard again. And every time you practice it when it's hard, you have like very steep learning. And so you can sort of incorporate these things. You don't have to like not be doing anything. But if you have a certain amount of things you have to learn, you can mix together interleaving and spacing because by interleaving, like mixing the tasks

you have to do, shuffling them up as it were, you effectively space them anyway. So it's not like you have to just go to sleep and waste your time, but you should take different tasks and mix them together because it causes you to space them and interleave at the same time, basically. So I sometimes like, I give talk sometimes like that started. That was a thing. thing I didn't know was a thing until after the sports gene. And I memorize, I'll memorize hour long talks, like word for word. And I'll improv off of that, but I like to have it memorize so that I can then improv. And every time I give a talk, people come up to me and say, do you have a photographic memory? If I put my keys down and spin in a circle, I still lose them. I definitely don't have a

photographic memory. But I've totally incorporated that stuff into how I learn. So if I make my slides, I want to talk into slides like before they get there, I'll test myself on them right away before I've memorized them. And then I make sure I leave some space and interval, no cramming. and I'll mix the slides into different order, not even the order I'm going to have to practice them in case maybe I want to add lib a little. It's been amazing. So I look like I have a photographic memory when I definitely don't. What's your take on the role of passion and grit in all of this? So two very different things. I think you would probably argue that those are good things, generally speaking, but maybe sometimes misapplied or misunderstood. So talk about the role of passion and grit in terms of finding what you want to do, should do, are good at and how much when perseverance is good and when it's bad.

I'm curious if this will end up being one of the things people glom on to in the book, because I have this chapter titled The Trouble with Too Much Grit. And grit is, as a lot of people have probably heard, and what exactly you think about it, probably depends on where you heard about it. But this incredibly popular psychological construct most associated with the researcher Angela Duckworth. And it comes from a survey that basically asks a half dozen questions about your personal resilience and your perseverance. and then another half dozen questions about your consistency of interests, which is sticking to the same thing that you're doing. And the most famous study, and you can stop me if I'm digressing too much here, but the most famous study is the one that Duckworth and colleagues did at West Point at the U.S.

Military Academy on cadets going through what's called Beast Barracks, which is the six-week orientation where, like, high school kids are taken and this is time to convert you into officers in training. So physically and emotionally rigorous, you know, they get gassed and they don't get get to sleep a lot and all that stuff. And what she found was that the whole candidate score, which is the most important thing for admissions at West Point, it's like test scores and some athletic tests and leadership experiences, stuff like that, was not so good at predicting who would make it through Beast. In fact, it was very bad, although it is quite good at predicting how well

people do in the Academy overall. And the Grit survey was better at predicting who would make it through Beast. And so then she started applying this to other things like who in the finals of the National Spelling Bee would advance to later rounds. It was great. If people had high verbal IQ test scores, that was good. But if they were lower, they could make up with some of it for grit. And if they were lower on grit, they could make up with some of it with higher verbal IQ test scores. So what she was finding was that this construct that measures your passion and resilience and also your kind of stick-to-itiveness in terms of the same idea had some independent predictive value of how people perform separate from any other skills.

And I think that's a powerful notion, but I also think it's been wildly misinterpreted in some ways. So we talked about restriction of range a little earlier. So this famous study at Beast Barracks, that's an extreme restriction of range. You've already siphoned off this tiny sliver of humanity who are quite similar on whole candidate score from the rest of the world.

So all your other variables better look like they're more important because you've flattened one of them. And most every cadet gets through Beast Barracks. and the amount of variance that grit explains in any given paper that I've seen is usually between about 1 and 6%, which is probably not what like companies they're testing for it think. They probably think it's a lot more than that. But I think the bigger problem is that all of the studies that are done on grit like that with people in the finals of the National Spelling Bee or at Beast Barracks, they're looking at

people who have a very narrowly defined goal for a very short time. Those people are trying to get through six-week orientation. people who are already in the finals of the National Spelling Bee are just trying to get farther in the National Spelling Bee. The goal has been restricted to this incredibly narrow thing for a very short time. And what I thought was interesting was looking at, so grit has some predictive value for cadets to get through Beast Barracks. But since the explosion of the knowledge economy, since like the early and mid-90s especially, those gritty cadets that get through Beast and then get through West Point have been dropping out almost half of the, them have been dropping out the first day they can from the military. And those rates have gotten higher and higher and higher and higher. And so when we zoom out and look at a longer term goal, it's gotten

so high that this high-ranking officer suggested defunding West Point because it's an institution that, quote, teaches its cadets to get out of the military, which of course is not the case, right? But what's really happening is as we've entered, gotten less into this sort of corporate salary man, up or out kind of economy, these cadets learn transferable skills. They learn things about themselves in their early 20s. Age 18 to late 20s is the time of the most personality change. That's not just my intuition. That's in psychological research.

And they decide to go do something else. And is that a loss of grit? Or is that because they learned something about themselves in those intervening years? They learned valuable skills that they can transfer. And they change their goals. So here you have these cadets who would score lower on grit when they're 27 than they did when they're 18. Because grit's not a stable construct of people. there's no evidence that it is.

And what those cadets are doing is they're on the hunt for what economists call match quality, which is tried some things. They've learned about themselves. Match quality is a degree of fit between your interests, your abilities, and the work that you do. And I don't think it's a failure of grit for them to go hunting for match quality because it turns out that that's incredibly important to your overall performance, how well you fit with what you're doing. This goes back to something we were discussing in sports initially.

And so I think a better construct or a more holistic, one because I think grit has is very useful and feels to me intuitively a ton is to look at this more holistically in this aspect of personality study called if then signatures where it's like if David is at like a giant rave then he's an introvert but if he's with his small team at work then he's an extrovert and those things are true and it turns out that's how personality functions where we don't just have necessarily these stable traits we have traits that are consistent in a given situation and I think a better way to think of grit is when you help someone get the right fit it will look like grit and that should

shows up in some of like the music research that I cite in range where you see kids that practice a little but they don't like it that much and then they switch instruments and when they find an instrument that really fits with them then their practice explodes and so when you just look at the end results of the study you're like well these kids just love to practice but then if you look at the whole developmental trajectory it's actually that they cycled through some instruments when they found one that fit their interests and what they like to do then the practice explodes so it's getting to the right spot that causes that to happen so I think we need to be conscious that the studies of grit have been in these very contained, very

short-term environments. Like, if you're in the finals of the National Spelling Bee, not deciding, you know, you don't like the spelling bee anymore is an advantage. But would it be a bad thing if you decided that, like, memorizing root words, you don't have any idea what they mean, but you know how to spell them is maybe not the best match for the way to spend your time? I don't think that's a bad thing, necessarily. A lot of these concepts, and especially this range of personality change, I think he said 18 to the late 20s. I'm curious what the drivers are behind that and how that matches up with what, I guess, given all your research, you would suggest people do more of in that time period to take advantage of that, we'll call it,

like, malleability early in a life, early in a career, pre-kids, like this kind of this formative years of adulthood that people do to make sure that that personality change, I guess, is in the right direction. Yeah. I mean, the drivers, I'm not sure I'd answer that. I'm not sure anybody knows. There are things we suspect, but the correlation between personality traits between the team, years and like middle age is usually about 0.2.3, so like low to moderate, which means like between your teen years and middle age, there are traces of the teenage you that are distinct in the middle aged you, but mostly you're a very different person. And the biggest change there goes on from 18 to 28. What those drivers are, I think it's hard to say. Are they natural? Are they

just that we're having all these, it's the first time a lot of people are like truly autonomous? and so I can guess at that. I don't know the answer. I just know the empirical findings. But I think what it means is that if you're matching early, I think it adds more fuel to the Roger model because you may be like Tiger and you may find your thing right away. And that happens.

I think it's just a lot less likely because you're in the position of having to choose. If you're like if we thought about careers the way we think about dating, we would never tell people like settle down so quickly. You'd be like, go get more data before you like make a commitment here. And we may spend as much time working as we do with our partners or more, at least waking hours. And so I think when we're like rushing people to specialize, I think it's with good intent. But you're putting them in the position of choosing a match for a future of them who they do not yet know in a world that they cannot yet conceive, basically.

And so the approach that the people who find better match quality take, and there's sort of two sections on this in range, one called the Dark Horse Project and then this woman Herminia Ibarra that studies people. She has one of my favorite quotes in the book. She studies how people find good career fits in part and change careers was we learn who we are in practice, not in theory. And what she means is we like to think that we can just introspect and like the commencement speech advice. Actually, Paul Graham had a great, I guess, never delivered, but he wrote like a commencement speech telling people to ignore the commencement speech address. And he posted it online. And what he basically says is totally in line with Hermione Ibarra, which is most of the commencement speeches will tell you,

picture who you want to be in 20 years and like confidently marched every step toward that. And he says like what computer scientists call that is premature optimization, basically. You shouldn't do that. What you should do and what Hermini-Ibarra suggests is all this research shows what we know about ourselves and our skills and our potential is constrained by our roster of previous experiences, which are really limited when you're a teenager, especially. And so I think one of the reasons why personality changes is because we're just expanding our experiences and learning more about ourselves and adjusting to the world.

world. And we learn who we are in practice means we do things and then we reflect on them. And that turns out to be the way we learn about ourselves as opposed to just being able to introspect and have a theory about ourselves and be accurate. And so what she says, the way to go forward then is to do things that are essentially little experiments. Say, I'm going to try this thing. Here's who I am right now. Here are the skills I have right now. Here are the things I want to learn. Here are the opportunities in front of me. I'm going to try this one. And maybe a year from now I'll change because I will have learned something about myself and maybe I'll have different opportunities

and they keep zigzagging until they find better match quality than the next guy and that improves their performance. So in this Dark Horse project at Harvard that similarly to Minibara looked at how people find their best match quality which improves their fulfillment and their performance, short-term planning.

A focus on short-term planning was like the main trait that most of them had. But when they all came into the study, they would say it was called the Dark Horse Project because all these people would view, and not everyone again, but it was most of them, would view themselves as having come out of nowhere because they would come from some other field or have some weird miscell experiences. So they'd all come in and say,

well, don't tell people to do what I did because, like, I'm an oddball. But that turned out to be the norm, is that they would bounce between things learning about themselves. Really, it's an investment in learning who you are and how you can better fit. And so Robert Miller in the book, economists modeled this and statistician as what he called a two-arm bandit process. So one-arm banded is a slot machine, is a nickname for a slot machine, and the two-arm-banded process is, or multi-arm-banded process, because he looked at multi, not just two. Essentially career matching, he modeled as if someone's sitting in front of like a bank

of slot machines, and each one has a different reward and each one has a different probability of reward, and they're tasked with pulling different levers, trying to extract some information about where to focus, and then when they find the one that has the best probability of reward, they stick with that. And he found that to be similar to the way that people test careers. As they go around, you try some things and you start to figure out where you should focus. So he advocates, he thinks like the young and stupid, you know, people who dive into like high-risk careers is not stupid at all because you get high-information signal from those careers very quickly.

So he advocates, or what his model suggests, is that younger people should dive into these high-risk, high-reward careers where you get lots of feedback quickly early because that's the best time to try it out. out and sort of get these high information signals. You don't want to go into like a low information signal career early on. There's a tremendous amount of advice for the individual and everything that you just described and kind of everything we've talked about. As the book progresses, there's also some ideas about applying this idea of range, I guess like a more mature or static or business-like environment. And one of these is this notion of using the outsider. So I'd love you to expand on this idea of how, I guess businesses is kind of what I'm thinking, but maybe we use the Game Boy example was one fun one. But the ways in which an outside perspective in my world, the buzzword for this

is cognitive diversity in your investment process or something can be used is really harnessing the powers of range for more productive outcomes. Yeah, that gets, there's like several chapters that touch on that in one way or another. I'm guessing that your listeners are kind of pretty familiar with the outside view work. So maybe I won't go like too in depth on that part. But the Game Boy example, for example. So I think one of, you know, actually listen to one of your previous episodes because I'm always interested, like I said, we talked a little bit before, and I know Josh Wolf, and I think he's like a very interesting guy, and to put it mildly. And one of the things that he mentioned, I think, when you were talking to him was that he diverges.

I'm going off on a tangent again here. No, no, go for it. That's what's all about. Beauty of the format. That he disagrees with Robert Gordon, the economist, who sort of says, Robert Gordon has this very compelling point where he says, if you took someone from 1860 and put them in a house in 1940, they wouldn't even know how to use it because it would have indoor plumbing and like a refrigerator and electric lights and a telephone and a car, whereas they would have had like an

outhouse and a horse and like whale oil lamps or whatever. So that's 80 years before 1940. But if you go 80 years from 1940 forward, which is us, we would know how to use all that stuff. And his suggestion from that is like we haven't made that many leaps, like the big leaps are behind us. And I think that's interesting. But one of the things Josh mentioned is that he thinks communication technologies kind of diverge from that and allow this sort of dissemination of information more widely and collective action in a way that we haven't had before. And I tend to agree with that. And I think, and again, I'm seeing this through my own range lens, of course, right now, because, I mean, the book's about to come out and it's on my mind a lot. So that's my bias,

obviously, range colored glasses. But I think one of the ways that he was right about this really changing the game is that because specialized information can be disseminated so much more quickly. And because there's so much available, it's a lot easier to be broader than a hyper specialist now, because you can take this information very quickly from all these different domains, and there's so much out there. There's so many opportunities to recombine things in new ways that people who are in a more sort of narrow focus, or what one of the scientists in the book I loved called a system of parallel trenches, where everyone's in their own trench and not usually standing up to look over at the next trench, even though that might be where their answer is,

you can be broader now because there's so many chances to combine thing and information is disseminated so quickly. And one of the guys who you're alluding to, I think, is a great example of that is this guy, Gunpei Yokoi, who I sort of profile in the book. By the way, speaking of another research advantage to go back to our other topic was he had some great books that had not been translated from Japanese. So I hired someone to translate them from Japanese. I love that. Right. So that's a great advantage because he laid all this stuff out, but nobody had translated to English. Similarly, for the sports gene, I had some old.

like German sports journals translated. And it didn't end up being as big a part of the book, but it's still like, I kind of look for stuff like that. And so Gunpeyokoi was, he studied electrical engineering, but he wasn't so good at it, frankly. And this was at a time when, I think, as one of his, one of his colleagues said, the technology was changing faster than snow melts in sunlight. So he just couldn't keep up. And all of his peers who did really well in these tests went to big companies in Tokyo. And he didn't do well in the test, so he didn't get Tokyo interviews. And so he had to settle for going to this small company in Kyoto that had started in the 19th century as a wooden shop that was selling playing cards, so called Hanafuda cards.

And when he got to this company, there were about 100 employees at this point. They were making these playing cards. And he was hired just to maintain the playing card machines. That's it. But there wasn't much to do because it wasn't that big of a company. He'd always been a tinkerer. He just loved getting involved in tons of different stuff. And he started tinkering with the machines one day. And the company was having a lot of trouble. Playing card business wasn't going that well anymore. The one good thing they had going was they had machines that helped the process be cheaper. But one day he cut these like criss-crossing slats of wood using some of the company equipment

and made like an extendable arm that you see popping out of like a robot's stomach in the cartoons. And he started lazily grabbing stuff that was far away. And the company president who was in a tizzy about turning the company around, sees him doing this, come to my office and thinks he's going to get fired because he's supposed to be maintaining the machines, not playing with them. and the company president says, turn this into a toy, we're going to market. And this turns into what's called the ultra hand, like grab things. And this was, the company was Nintendo. And this was Nintendo's first toy.

And now he starts, they had been trying to make, at the time, they had converted some facilities into making, in like a desperate attempt to save the company while the playing card business was deteriorating into facilities that made like cartoon branded noodles, basically. And so they got rid of that. didn't work very well. They got rid of that and turned it into toy development. And Yo-Koi started trying to develop different toys. And he realized that he wasn't so good at cutting-edge stuff. That first year, he works on another toy that's called drive game, where it has an electronic motor and you have a real steering wheel and you guide a little plastic car over like a moving, like the motor moves the road underneath the car. And it turned out that the mechanism was complex and that was like he needed to work with some people who knew more cutting-edge stuff and it was constantly

breaking and defects and it was super expensive, total flop. He says never again, none of this cutting edge stuff. And he develops a theory he calls lateral, that translates to lateral thinking with withered technology, with withered technology, lateral thinking with withered technology. And what he means, withered technology, he means stuff that is cheap, well characterized, easily available, other people have overlooked it at this point. They've moved on. Lateral thinking mean taking it into some use or some domain where it hasn't been before. And he starts doing that with everything. Remote controlled cars were big at the time, you know, in 1970s, but they were expensive, so only adults could get them. And the biggest

expense was multiple radio frequencies to control things, power of the motor, turning, etc. And he says, and everyone's racing to make more and more features. They're getting more and more expensive. And he says, forget that. I want to democratize this. I think the barrier to this being more of a success is not more features. It's more people being allowed to do it, democratizing innovation. And so he strips it down to the bare minimum, one radio channel. So the car can only turn left. He calls it lefty RX. Markets it at a tenth of the cost of all the other ones. Now kids can buy it. And kids don't care. They'll left turn their way out of trouble. No problem. And so there, democratize it. So now he sticks with, so he starts getting momentum for this lateral

thinking with their technology, one thing after another, just repurposing things from different domains. One day he's riding the bullet train, sees a salary man trying to cure his boredom with a calculator, playing with his thumbs and thinks, gosh, what if I could make games that you could hold, like play with your thumbs? And he randomly has to play show for. for the company president for a day, pitches his idea. President's going to a meeting with Sharp, makes calculator screens. And the president goes there and says, oh, we've got this guy who's got this idea. Yokoai thought, like, it was going nowhere.

Next thing, you know, is they bring the sharp calculator. The sharp calculator people are engaged with, like, Cassio and this calculator wars, where they're both, like, just trying to make better and better screens and prices going down. And Yokoai says, all right, you guys make me a screen really small, like the size of a business card that I can display some graphics on. And I'll take these old processors that aren't like cutting edge at all anymore, but cheap, and your screens that are getting cheaper,

and we'll put those together, and we'll make small games. And they're like, nah, we don't think we can make screen that small, but we'll try. They sort of make him a prototype. And the problem is the stuff is crammed together so tightly that it causes this thing called Newton's rings where there's distortion on the screen because pieces inside the screen are touching.

And so he says, gosh, we need to separate those. Like, this is a problem. And the calculator, the sharp people are like, This is like a law of physics. You're screwed. And he thinks, well, what other industries? He starts searching other industries make small separations on something the size of a business card. He realizes in credit cards.

They're embossed. They like raise the lettering a little bit. So he goes, he gets someone to help him tweak the machines so that they can emboss like a credit card. And he just embosses little dots on the screen components. It separates it enough. Newton's rings are gone. He creates what's called the game and watch, which is, and this is Nintendo now getting into the game sphere. So it's like just little games that you can play.

play with your thumbs and they explode. I think their goal is to sell half a million units and they sell like 11 million or something like that right out of the gate. And so he gets emboldened in this philosophy of lateral thinking with their technology and just one hit after another. And then eventually he decides that they're going to go even farther back. You know, in the technology's progressed a lot. In the 80s, he says, we're going to take processors that are even older, cheaper. I want to step up our handheld games. Things like Nintendo Entertainment System have exploded now, so lots more people have had video games in their house. And there's this competition on graphics and those sorts of things. And Atari and Sega are developing these

color handhelds. And he says, we're going to go old processors that would have been cutting edge 10 years ago. We're going to have four gray scale shades in ours, right? It looks like screen color of like old spoiled alfalfa. Graphics smear across the screen when you move fast because it doesn't move fast enough. The processor's not that good. And some of the of his colleagues are sort of like, well, you're crazy. But at this point, yeah, he's earned it. And they've paired him with some more specialized electrical engineers who can sort of, it's sort of like the jobs Wozniak thing. Like Saturu Okada was the guy who sort of will politely say like, Yokoi was not good at electronics. But he sort of backed him up, you know, made up for some of his deficiencies.

And they put these things together. And I remember on one of the day that when the Game Boy is about to come out, so this turns into the Game Boy, of course, what I'm talking about. one of Yochoy's colleagues comes and says, we're in trouble. Sega and Atari just came out. And Yokoai says, are they color? And the colleague says, yes, they are. And he says, then we're fine. Don't worry about it. And it turns out the Game Boy absolutely blows them away.

It becomes the best-selling console in the 20th century. What it lacked in color and graphics, it made up for in durability, you can drop it. Like, in the reporting of this, I went to my basement, my childhood basement and found mine. The batteries it had had expired. It had two batteries that had expired in 2007, two that has. expired in 2013. I flipped it on and played Tetris for a couple minutes. One of the batteries had exploded and it was still fine. And it was covered in grime, right? So I've made sure to take video when I turned it on so if I can use it in presentations and stuff. So it was durable, like

even if it died because it got wet, you could dry it out. It would come again, it could fit in a big pocket. You could play for weeks on batteries. And because the tech was so simple, people started making games for it inside and outside of Nintendo like crazy, almost like app developers, right? It's like he drew people onto his team by using simpler technology that they understood. And it just totally blew away the competition. And I've seen in some articles I had translated more recently, Nintendo is still embracing this philosophy. The president, an article I translated, he said, we was very much out of gun pay. Gunpei Yokoai passed away in a traffic accident in the late 90s, unfortunately. And the president said, this is very much out of Yoko's

lateral thinking with withered technology philosophy, because the barrier to entry again was not graphics. It was complexity of gameplay. So we made it simple. And it democratized. this right like Queen Elizabeth started playing and all these things and what they just did the thing Nintendo Labo or Labo however you say it's like cardboard that people build around their switch or whatever and so I think they're still succeeding with that philosophy and Yoko's feeling was a he said if you draw two circles on a blackboard people will know it's a snowman they can feel the color of the snow they don't necessarily have to see it and you know he just felt that once there was a competition for graphics or screens like

that was for the specialists to compete. That was like a zero-sum thing where like one company was going to blow another one out of the water eventually, Sharper, Cassie, or whatever. Whereas there were many more opportunities to take this stuff that was already well known, that everyone was looking past and recombine them in new ways. And I think it's a good symbol of where we are and why some of these companies like Innocentive and Cagall that I talk about in the book have had such success with farming out problems that stump NASA or that stump Eli Lilly to just like,

people out there and they get some success because I think we're in an age because of communication technology specifically where it's easier to be broader than a specialist and still have an impact. That was like the really long story. It's an absolutely incredible story with so many interesting takeaways and lessons. We've seen it firsthand where you put people who are talented in their own ways and sick them on the problems that we've been trying to solve for a long time. And often very quickly, like alarmingly quickly, they come up with different takes that lead us down very different paths. So I think that that's very broadly applicable, this idea of the outsider and combining all of the old technologies piece. I'm curious in the reporting, did you find any other

favorite examples, not at Nintendo, but elsewhere of whether technologies being applied? Oh, yeah. In that chapter, actually, I talk about 3M basically quite a bit, and this research that shows that, and I should caveat this a little bit, so there are, when I looked at research that look at patents for technological innovators, in areas where sort of next steps were more clear, specialists were better, where you just had to drill down into a certain technology. In areas where like the next steps were less clear, people who had worked across a large range of different classifications according to the U.S. Patent and Trademark Office were the ones that made the bigger impacts. So I think you want an ecosystem with both the specialists and the generalists.

And it kind of depends how well characterized your problem is. But one of the things I liked about 3M was that they had a lot of examples of this. And they make it really easy for people to do that because they have this thing they call the periodic table of technologies and it sort of helps everyone see like what else is going on outside of their field and like all the technologies that they own as a company so that you can go back to old stuff and sort of recombine it. So even things as simple as like the Post-it note, which is one of their blockbusters, came from basically a reusable adhesive that they had found like no use for. And one of their inventors was to have trouble like marking pages in his hyminal when he was

like at church on Sundays and you wanted to be able to like unmark and remark it. And so we basically combine a bookmark with this old adhesive that they had just abandoned him with post-it notes. That's a pretty withered technology. Or I mean, one of my favorite inventions that's mentioned in the book is this multi-layer optical film, which is in some ways cutting edge in the sense that it's like in your computer, in your cell phone, recycling light that's bouncing around in there so that you need less battery power to get a bright image. But the technology that sort of convinced the guy who led the team that invented it that it was possible was the plastic of a water bottle. This wasn't in his area. He wasn't a materials scientist, but he knew that the plastic in a water bottle had certain

properties of light, that people just hadn't, it's like in front of you every day, but the specialists in the field just hadn't thought about using it for these applications of like modifying reflectivity. But probably my favorite example overall of lateral thinking with withered technology because of how withered it is was to Yu Yu, the first and only woman and Chinese National to win the Nobel Prize on Sciences. She's called the professor of three knows because she has no postgraduate degree. She has no foreign research experience and she has no membership in any like national academy or anything like that. Or at least she didn't. And she had interest in science and interest in history. And so she went back through these like archival old centuries old manuals of Chinese medicine to look

for hints at treatments for malaria. Just for hints. She wasn't going to do like fake dietary supplement because this thing has been used forever, it must work. She wanted to see hints of like just old notes and found in this like several centuries old manual a hint about something that someone thought might like help with certain treatment for malaria. And she starts testing all these, well long story short, it leads to this extract called artemisinin that she gets the clue again from this century's old text. And it turns out to be the world's most effective treatment for malaria. So a recent study credited artemisinin-based therapies with between like 2000 and 2015 preventing 146 million clinical cases of malaria in Africa. So she won the Nobel Prize for that.

And I just like that example because technology doesn't get much more wither than that. She found it in like an ancient Chinese text. Researchers before her had tried almost a quarter million different compounds looking for something like this. And she just went back to things that had been overlooked and looked for hints that could target her searching. And she won the Nobel Prize. Would it be fair to characterize kind of the arc of your work as starting with this interest in performance and like what affects performance? And it seems like so much of the examples you've given today are more about innovation and creativity and how range intersects with those two things. And it makes me think of the James Carson, you know, finite versus infinite games.

And it seems like you kind of started with the finite games and performance in those and it moved on to a much more open-ended playing field. Very astute observation. Yeah. Totally. Totally. And part of that was just my own interest. I mean, some of my, you know, I guess the not so secret secret of both of my books is I'm

just investigating things I'm interested in, right? Like the sports gene was like a bunch of questions I had in the back of my head of like, well, why can't baseball hitters hit a softball pitcher when the transit time of the ball is longer? And why are certain athletes coming from certain places, you know? And then, and similarly with range. And yeah, no, you're definitely right. You're definitely right. And that reminds me of something sort of semi-related, which is, in looking at some of this creativity and performance, I tried to pick studies sometimes that could account for certain types of survivor bias.

Because when I was reading some business books, I think there's some amazing ones, and there are also some that suffer in the extreme from forms of survivor bias. Most of them. Yeah, yeah, for sure. Not just business books. Books. So I particularly liked sort of studies like this one. on comic books that looks at what causes comic book creators, whether they are individuals or teams, to do well on average and to be more likely to have a blockbuster and innovate.

And the scientists liked this. Comic books have a very, like, defined area of the start of creative explosion when there was this doctor who's kind of convinced Congress that in like the 60s that comic books were like making kids deviance. He fabricated some of his work actually. And so in order to avoid regulation, the comics industry started self-censoring. But then in 1971, the government asked Stan Lee, they were worried about drug addiction and overdoses. And they asked him to make a Spider-Man narrative about that addressed drug addiction. And so he made one where, like Peter Parker's best friend, ODs.

And the Comics Code Authority, the self-censorship body, didn't approve it. And they published anyway. And it was a hit. And suddenly the censorship went away and floodgates are open. Now you start seeing complex characters with emotional problems, diverse characters, all these sorts of things. One in a series I love, Love and Rockets, where the characters age, diverse cast that ages in real time with the readers. And so they studied starting at that point of creative explosion and tracking the value of comic books, because you can track them easily, whether they went up or down, which I think is a very important thing. And the variance of creators, whether they made things way below their norm or way above.

And so I tried to pick studies like that that can track things if they go bad, too. And one of the interesting findings of that study was the scientists made predictions about what would cause someone to be better on average and more likely to produce a hit. And they predicted resources of the publisher would be more likely to produce a hit, years of experience in the field, number of reps, number of creations they had done, and they could track individual creators. That would be true if it was like an industrial task, but turned out none of those things were true. years of experience had no effect. Repetitions had a slight negative effect when it got really high, maybe a sign of overwork or something. Resources of the publisher, no effect. But what did was the number

of different genres that a creator had worked across. They characterized comic books into 22 different genres, you know, whatever, fantasy, adult, drama, comedy, adventure. And the number of different genres, a creator had worked across, improved their performance on average, the value of their comic books on average, and the likelihood that they would make like a massive blockbuster that was way. above their norm. And the really interesting thing to me, so I like that because it followed people down and up. But really interesting was they asked the question, well, can you recreate a broad individual with a team? Like if we get three people who have each worked in one different genre and put them together, is that as good as one person who's worked in three different genres?

And the answer was initially yes. So if you have two people that have worked in one genre each combined, they do better than one person who's only worked in one genre, on average and more likely to innovate. But after four genres, that crosses over. An individual who has worked in more than four genres becomes better than a team who by platoon has worked in the same number of genres. And so one of their conclusions is that the individual in some ways is the best unit for integrating information. And as that genre experience goes up, the individual becomes more and more, the broad individual becomes more and more valuable, even compared to the diversified team. So they titled the paper, Superman or the Fantastic Four. And they said, what do you want for creation?

you want the one Superman who's worked across many genres. If you can, if you can't get that, you want a fantastic team of broad individuals, basically. Yeah, I love that. I mean, it's a great closing story for the idea and the return on range as a concept and for people to encourage people to really collect things and try things. I'd love to close with something fairly unrelated, but just fascinating to me in reading your work, which was the story about, I think it begins about implanting stents in people to high blood pressure with heart problems. And just this broad literature in the medical world of intervention, which has really no basis in fact or empirical data that, let's say cardiologists in this case, nonetheless continue to do, spending huge amounts

of money every year and the reasons why. And you wrote this really thoughtful piece for ProPublica that I recommend people read. And this is an adjacent topic. I'm obviously different from what we were just talking about. But I'd love your thoughts thinking back on the research for that one. And again, the obvious analogies in the investing world of very often we're doing stuff that we think seems right to do or we need to do something because things are going wrong when very often these things are unsupported by the evidence and maybe the best action is something totally different. So in some ways my interest in that topic grew out of my interest in specialization because the first thing I saw was so I knew this certain type of repair of the meniscus,

you know, this piece of fiber in the knee was one of the most popular, if not the most popular orthopedic surgery in the world. And I started seeing these studies in Scandinavia, particularly, that would do placebo-controlled trials of surgery, which is crazy, right? So they take people in who have torn meniscus. Some of them get the surgery where it's repaired or shaved back into like its crescent shape. And some of them get sham surgery where they make an incision, bang around, make some surgical noises, sew them back up and send them to physical therapy. And it turned out that those people did just as well. In fact, a year later, the surgeons couldn't even them apart from the people who had actually had surgery. And so I started to see that and say,

like, that should be a sea change. And then I look at the number of times that procedures being done. I'm like, hmm, still going up. And then look at the specialist journals and the ways that they would write off that kind of research. And they would write off that research with their experience. I've done this on patients. I've seen them get better, which is not a question. The people in the study got better, right? That wasn't the question, but it seems like it's the PT or the time off or whatever. And so I became sort of alarmed at that where you would see, and then that led me to this book called Ending Medical Reversal, where these two doctors basically just cataloged a huge number of standard practices that were implemented based on either intuition or low levels of evidence. And when better studies that zoomed out looked at what we really care about, it showed that these things didn't work. And yet they continue to become more popular because specialists are so tied to them, even when their compensation isn't necessarily tied to them.

So removing compensation from being tied to procedures has been something like the Cleveland Clinic has tried, and they still end up overtreating with things like stents, which are these like metal tubes that open an artery. And one of the reasons for that, I think, one is partly like the when all you have is a hammer, everything looks like a nail problem. It's like when people come in with stable coronary artery disease or, because a stent can save someone's life if they have a heart attack. but when they have stable condition and they come in with chest pain, they're probably going to end up getting a stent if imaging shows that they have a narrowed artery. And over and over again, there's something like 12 randomized trials that show that it will open up their artery. But what you really care about is heart attack or stroke are dying. And they have heart attacks and strokes and die at the exact same rate.

The same thing shows up with one of the most popular blood pressure medications in the world that I mentioned in that article, where because we've gotten so specialized in medicine, everyone usually is looking at surrogate markers. They're not looking at the outcomes you care about. They're not zooming out. So they're looking at, did I lower blood pressure numbers? And that's a surrogate marker again for stroke, heart attack, and death. And so some of these drugs, they lower blood pressure, and people die at the exact same rates of the exact same stuff with lower blood pressure numbers. So we need those specialists because knowledge is exploding, but you also need people that are zoomed out looking at the bigger picture and looking at the outcomes we actually care about. And once I got to that,

book ending medical reversal and realized how common so many of these things that have been overturned that should not be in practice anymore are, I kind of felt I've really got to, really got to write about this. Is there anything that you think might change this trend? So this idea that we do stuff, we study the stuff, we find that it doesn't work, and then it keeps happening. And maybe we try to write about it to stop that, but that it keeps happening again. Are there new strategies because my interest in this is both health, but also the investing parallels are so obvious that we don't even need to go through them. This just relentless amount of information and strategies and ideas that are clearly

useless or wrong, yet perpetuate. What do you think we're doing wrong to trying to debunk this stuff? Why can we not beat the inertia? I mean, I think, and one of the reasons, like I used the last chapter and range to talk about doctors and scientists is partly because those people are viewed from the outside as like the epitome of specialists. And I wanted to get to say, okay, even within these things, things that compared to the world at large are quite specialized. How do we harness some of the

advantages of breadth? And I think Arturo Casa Deval, who's one of the characters in that last chapter, who's arguably the most renowned immunologist in the world, went to Johns Hopkins to try to start a new education program where instead of teaching all this specialized techniques to future doctors and scientists, he starts with how does scientific thinking work more broadly? How do we evaluate evidence? How do we know what is true? And that, I think that program is going to have a huge impact, partly because of his own renown. But I think, I confess in the book that I, when I was a grad student, I committed unknowingly statistical malpractice. Talk about a wicked learning environment on the project that got me a master's degree from Columbia. First, I was embarrassed about that. Now I'm

just mad about it. And it's because I was shuffled into this very narrow set of knowledge before I even knew what the statistical program I was hitting buttons on actually even did in a deeper sense. and I think recognition is important. I think it's going to be generational change. I think it's going to take some generational change. But I think changing the way we educate doctors and scientists to teach. As Arturo says, all the world's knowledge is available on your phone. We've got people walking around with all the world's knowledge and no idea how to integrate it or evaluate it.

And so he's despecializing education of doctors and scientists and saying, let's start with how information is evaluated and how scientific thinking works. And when I saw him on a panel saying that, the editor, head editor of the New England Journal of Medicine was on the panel. and said, no, you can't add all these classes about scientific thinking and stuff to doctors training it. There's already too much time. And New England Journal Medicine, by the way, one of the most retraction-prone journals in the world. And Arturo said, yeah, get rid of all that other stuff. They can learn that stuff like in practice. They need to learn how to think. And I think he's really hit on something. And he's noticed that we have no ability to correct because our specialists, not just our

specialists. In general, we don't have a good ability to evaluate new information. And so I think some despecializing of education is going to help, I think, but I think some of these problems are devilishly hard, too. The stent issue. So all these studies show that stents don't work for stable disease, but they do work for heart attacks. So there have been examples of hospitals where the doctors are so sure that stents still work, right? They're so committed to this idea because it's biopausible. This is a term that Mike Joyner, a physiologist at the Mayo Clinic, sort of uses that I really bio plausible, meaning there's a clogged artery. How could opening it up not work? It's got to work, except it turns out the body is much more complicated than like a kitchen sink, and we didn't design

it, and the disease is much more diffuse. And so some of these doctors started saying, show up making emergency room appointments. They'll say show up in the emergency room tomorrow at 3 p.m. We'll give you a stint because maybe insurance is starting to not cover it for stable anymore, so they find ways around it. I think that's tremendously difficult. And I think we need some of these, what Freeman Dyson calls birds as opposed to frogs, right? He says, we need birds and frogs. He's talking about math and specific. The frogs are down on the ground looking at like a very narrow area of the ground. The birds are up. They don't have as good definition on the ground, but they see the bigger picture. And I think we need to make the medical ecosystem more friendly to some of these birds

who are looking at the outcomes we actually care about, not just those surrogate markers of did I fix the meniscus, did I open the artery, basically. Well, this has been just incredibly interesting, and I'll use this as a conversation, especially to give to young people, as some sort of an ode to tinkering and range and curiosity and exploration, especially early in lives and careers. So thanks for all the fascinating examples and the great conversation. It's my pleasure. And I'm sure, I mean, you can gather from talking to you that you're like a big reader, right? So obviously you do this in your own life. And I'm sure it's, I'd be curious what your background is and like what you studied because you obviously consume a lot of information

to have range yourself. So I appreciate it. And we'll do that offline and hopefully do a round two sometime. Great. Hey everyone, Patrick here again. To find more episodes of Invest like the Best, go to investorfieldguide.com forward slash podcast. If you're a book lover, you can also sign up for my book club at investorfieldguide.com forward slash book club. After you sign up, you'll receive a full investor curriculum right away and then three to four suggestions of new books every month.

You can also follow me on Twitter at Patrick underscore Oshag, Oshag. If you enjoy the show, please leave a quick review for us on iTunes, which will help more people discover Invest Like the Best. Thanks so much for listening.