Modern Wisdom - #024 - Peter C Brown - The Science Of Successful Learning Habits

Episode Date: August 8, 2018

Peter C Brown is a writer, retired management consultant and the author of Make It Stick; The Science Of Successful Learning. No matter what your goals in life, your capacity to learn effectively is t...he foundation upon which everything is built. Whether you're learning Archery or Law, Economics or Knitting, your capacity to consume and recall information mediates your ability to progress. Today we learn what science tells us is the most effective method to learn. Florida International University Law School implemented the Make It Stick approach and went from typically 4th or 5th in The Bar Exam to placing 1st Place in 5 out of the last 6 exams. Teacher or student, you should listen to this. Extra Stuff: Make It Stick The Book: http://amzn.eu/2JY3yHB Check out everything I recommend from books to products and help support the podcast at no extra cost to you by shopping through this link - https://www.amazon.co.uk/shop/modernwisdom - Get in touch. Join the discussion with me and other like minded listeners in the episode comments on the MW YouTube Channel or message me... Instagram: https://www.instagram.com/chriswillx Twitter: https://www.twitter.com/chriswillx YouTube: https://www.youtube.com/ModernWisdomPodcast Email: https://www.chriswillx.com/contact Learn more about your ad choices. Visit megaphone.fm/adchoices

Transcript
Discussion (0)
Starting point is 00:00:00 Hello friends. This week I'm very happy to say that we are going to learn how to learn. Peter C. Brown is the co-author of Make It Stick. That book is about a seminal as you can get in the world of learning to learn. No matter what your area of pursuit in life, it's pretty likely that being able to expedite your capacity to intake information and then recall it at will is probably going to be pretty useful. And that doesn't matter if it's learning a new subject or learning a new physical skill, if it's knitting or archery or law, all of them require you to be able to remember and recall what it is that you're trying to learn. And Peter manages to lay out a really good framework
Starting point is 00:00:56 for doing that today. In other news, the Modern Wisdom YouTube channel has now crossed a thousand subscribers and has nearly hit two million watch minutes, which is pretty crazy in the first two months of it being up. So if you haven't already, please head to YouTube, search Modern Wisdom podcast and give it a subscribe. It would be a massive help. Also if you haven't already, whatever platform you're on, whether it's tune in FM, I'm not even sure if that's a podcast listening platform, we are everywhere. Spotify, iTunes, Stitcher, wherever you're listening, please try and give us a five star rating.
Starting point is 00:01:35 I don't really know if it helps, but it struck the ego a little bit, so do it if you can. But now we're going to learn how to learn Peter Brown. Bring it on Mr. Peter Brown, welcome to modern wisdom Thank you Chris. It's great to be here. How are you today? I'm doing just Dandy. It's a middle of summer here in Minnesota. It's a nice time to be here. Lovely. I've recently returned from the states
Starting point is 00:02:19 and the weather was fantastic, but apparently I've missed the one warm week that we get in the UK as well. Well, I'll do it. So I should have doubled down and just stayed home. I guess. I understand. So I want to get straight into it. Can you define to me what learning means?
Starting point is 00:02:40 Yes. Yes, for me, I define learning as picking up knowledge or skill that resides in your memory and is available to you when you need it later to solve a problem or take advantage of an opportunity. Okay, that sounds like a very curated definition. Was that something that you came upon easily? Or is that something that you had to develop through a lot of, a lot of thinking, a lot of conceptual deconstruction? Now, I just made it up, but it's because I made it up when I started working on writing the book, make it stick, because I felt that I owed the reader a definition of what we it stick, because I felt that I owed
Starting point is 00:03:25 the reader a definition of what we're talking about with learning, and I figured, I'll just try that and see if it holds up, and it held up fine. Was there a satisfactory definition in advance of that? No, I'm sorry, but I might have a problem with my headphones, because that kind of comes on and off. Oh, okay. I'm not too sure what's going on there. That might be on my side.
Starting point is 00:03:47 Okay, so, ask the question again. Was there an existing definition of learning that you were happy with, or was that one that you just created? Did that need to be there as far as you were concerned? I felt we needed it. We owed the reader a definition of writing a book about learning.
Starting point is 00:04:07 We owed the reader a definition of what learning is. So I thought about a little bit and I wrote that and I thought we'll start with that and see if it holds up as we go through the book and it held up fine. Some people in science, the science is referred to learning as three things, one being in coding, which is when you first encounter material and it's encoded as traces in the hippocampus. Consolidation, this is the process by which it moves, it migrates over hours or days into other parts of the brain where long-term memories are stored in retrieval, being able to recall it again later. So the scientist think of it is that's learning for them.
Starting point is 00:04:53 For me, you learn it and you remember it and you can recall it when you need it. I understand. So can you give us a little bit of background to the book make it stick and why you were compelled to write that? Yeah, sure. I've been retired for some eight years. I made my living as a marketing and planning consultant, corporations here in Minnesota. I had turned to writing books and I was between book projects. Now I was sitting down with my brother-in-law, Henry Rotiger, who goes by Roddy, is internationally preeminent in the field of memory and learning. He's a cognitive psychologist, and he was telling me that he was just coming to the end of a 10-year study of what strategies laid to better retention
Starting point is 00:05:50 of the new material. And he had a team of cognitive psychologists at 11 different universities doing this work with their doctoral students and postdocs. And over a decade of such studies, they were coming to some findings that were counterintuitive. And he said, we've been trying to figure out how to get this out to a broader general audience. And maybe we should collaborate on a book. So that's how I got into it. I'm not a scientist, but I was very taken by what they were finding. So, Roddy Henry Rodeger and one of his colleagues Mark McDaniel, who is also a cognitive psychologist. They're both at Washington University in St. Louis, Missouri. The two of them and I collaborated on this book. Okay, so I guess you've got the cognitive firepower in there on those guys' sides, you know, 10 years and
Starting point is 00:06:47 11 facilities that have been used for that particular study. It's not as if you're short of research on that side. No, and the book is based on decades of research and it reaches far beyond the work of that team. But you're right. It's firmly grounded in the empirical evidence, but I tried to write a book that was highly anecdotal and engaging to read so that people would stick with it. And we could tell stories that illustrated what the science shows about how learning works. So that gave it a different kind of twist than the typical scientific study publication.
Starting point is 00:07:32 Yeah, for sure. I think in order to get the wider population to be able to buy into books like these, you need to bring it back down to earth. There needs to be some tacit examples and some things that people can relate to everyday life talking about standard deviations away from the norm and you know statistical modelling. It doesn't it doesn't light it doesn't light anyone's eyes. Well it lights some people's eyes but perhaps your two co-authors would be would get would get turned on by that but I guess for the for the normal reader that's not going to be that's not exactly going to light the fire in their belly,
Starting point is 00:08:07 so to speak. When you keep me up at night, it will be right to sleep. Agreed. So you've got the existing research there. Did you think that writing a book on the science of learning was important to publish to the wider audience? Well, I thought so, and they did too. And I think the reason it struck home with me is because I have a bachelor's degree from college. I don't have a graduate degree, you know, higher training. I've always been someone who's taken an interest in things and then gone about learning about them, figuring them out. And what their research showed is that the way a lot of people, particularly students, go about learning is by review and rereading and trying to push stuff into their brain. But what is far more effective is trying to get it out of the brain. It's the tinkering, the trial and error, the experimentation, the learning from the turns and setbacks that you get. And I just felt very much personally
Starting point is 00:09:23 affirmed by what the research showed and I thought I would really enjoy Getting my hands on this and because I felt it was very important, but I also felt I was really interesting Well, it's superbly fundamental, isn't it that no matter what Discipline it is that you're studying in, that the science of learning to learn is foundationally ahead of all of the things, ahead of everything that you're trying to do, you need to be able to know how to learn because it's universal, right? It's ubiquitous. It is. And from the moment we leave the womb, children are experimenting, touching, tasting, trying things. And when we get older, we lose some of that. And I think the traditions of our schools are such that learning
Starting point is 00:10:16 involves an expert who imparts knowledge to students. And in fact, it doesn't really, it's not really very effective. I think athletes know this when they go out on the field, and they have a coach. And the athlete knows that she or he has to figure this out, and the coach can give feedback, and they could work at it. But most students, including athletes who walk into a classroom, expect the learning to be a parted to them. And learning is not imparted.
Starting point is 00:10:50 Learning is something that's really acquired, unless extraordinary circumstances where you have some tremendously significant, emotionally significant event happen. Then you will remember that. But the kinds of things that we're trying to build in the way of metal understanding, metal models and mastery, you have to acquire. That's interesting. So, did you have a look at, or can you give us some of the typical approaches of an unlearned learner, as I'm going to call someone, someone who's
Starting point is 00:11:23 uninwritened as to perhaps what the research suggests. Did you have a look at what's the typical approach? Well, in surveying college students, by far and away, the major study strategies are rereading material, underlining and highlighting note-taking, that kind of thing. But the science shows that what's far more effective after you've heard a lecture or read a passage just to turn aside and then give yourself a little quiz. What were the big ideas in this material? Try to recall it and try to say to yourself, this relates to what
Starting point is 00:12:06 I already know in the following ways. Or if I were to put this in my own words, here's how I would do it. It's engaging the mind in the material. So I summarize all of make it stick with three ideas. The first one is the one I've just described. Learning happens when you struggle to get the learning out and apply it, not when you continue to re-expose yourself to it. It's about getting it out. The second big idea for me is we try to make learning simple for students, but actually there are some kinds of difficulties that are desirable. An example of a couple one would be when you practice a motor scale like your golf putt or you practice something you're learning, like solving the mathematical problems. It's good to practice a little and then space it out and come back to it at a later time
Starting point is 00:13:14 when it takes more effort to recall what was the formula and how to successfully apply it. And that kind of spaced practice feels like you're not getting it, because your performance is rough. But the actual effort involved in recalling that new material from your long-term memory strengthens the connections to it, helps it reconciledate, helps bring forward the most important points, and it easier to recall again later. So spacing out your practice is better than practicing things in a massed fashion.
Starting point is 00:13:54 That's a desirable difficulty. It doesn't feel good. It definitely doesn't. No, another desirable difficulty is mixing the practice of similar problem types. So if you're, let's take the math example, if you're practicing learning how to find the volume of a cube and a wedge and some other kind of series of different solids, the typical way that you would do that with a math book was you'd, you'd look at eight different wedges and you'd apply the formula eight times and then then you move on and do these other different kinds of solids.
Starting point is 00:14:28 And during practice you do very well. You get in tests that you do about 90%. But if in fact, once you've learned each of the formulas, your practice problems were in random sequence, you would only do about 50% right and you wouldn't feel too good about that. But a week later, you will have remembered that same 50%. You would do extremely well on a test compared to the others who have gone from 90% down to 23% because they didn't mix it up. And when you get a test where it's mixed up, they can't remember which formula went with
Starting point is 00:15:04 which problem. Yes. So this notion of mixing your practice, which does not feel good, is very powerful for improving your ability to recognize the kind of problem you're facing and picking the kind of solution that's going to be correct. That's interesting. So there are other kinds of desirable difficulties. So the notion of if it's clear as a bell, I'm sure I'll remember it, is not really true. If you have to reconcile different ideas between the lecture and the book chapter or what have you, that mental effort is what's going to make it stick. That's interesting. That's a second big idea that some difficulties
Starting point is 00:15:48 are desirable. Okay. And the third. Number three, the number three is that our intuition leads us astray. So our intuition is, when I reread it many times, I get very fluent in it. I'm on top of it.
Starting point is 00:16:04 I can do that and take a test the next morning. I pull it all nighter and I can get a great grade on the test. It doesn't stick though. What happens if your test is again a week later, you've lost about half of it. Our intuition says if we practice our 20 foot putt over and over again, we see improvement and it's true you do but that improvement means a short-term memory. The skills have not been consolidated in long-term memory. That takes overnight or takes days. So you walk off the golf course thinking you've really done a service to yourself on your 20-foot putt.
Starting point is 00:16:39 But much better would be to do a few 20-foot putts and then do other strokes and then come back and mix it up and it doesn't feel, you don't see that same kind of improvement, but your brain is getting better at judging distances and the motor scales required to make a good stroke. So our intuition leads us astray and it causes us to spend time in strategies that are not paying us back.
Starting point is 00:17:05 Okay. So I mean, that's a, to me, conceptually makes sense, but it's quite a, probably quite a big departure away from anecdotally what I would have thought good learning should be done. You know, the, the kind of, um, the force it down your throat, so to speak. Sure. You know, the really drilling into you stick to one task for a long time. I know if you listen to the podcast that I did with Dr. Ewan Lawson, we discuss in that
Starting point is 00:17:42 about multitasking and the parmodoro technique, deep work and trying to focus wholly on one task. It's not too far of a jump to think that instead of just focusing on revision, you should drill that down again to just focusing on one topic within your revision, right? And I think that that could quite erroneously be one step too far in terms of the how specific you're being with your revision time. So let's go from the start. Can you explain how learning and memory you're learning and recall relate to each other? to each other. Sure. Memory has two components. Actually, my co-author, Roddy Rodiger, is his field is memory. He's really written a lot of the breakthrough material on memory, including discovering this
Starting point is 00:18:40 whole field of false memory. But in any case, long-term memory is in different parts of the brain than short-term memory. So when we're talking about short-term memory or working memory, it's the list you go to the grocery store, you may be remembered long enough to pick up the things and then maybe you might or might not remember, and you just stop by the dry cleaner on the way home. But that's gone. Is there a time, is there a approximate time of them usually on that 24 hours or?
Starting point is 00:19:10 Not particularly, but I don't think so. I think it depends on whether you're trying, if you're making an effort to remember it for longer. For example, you're you're you're around a bike and it's got a four digit combination lock and you're trying to remember those four digits when you can give yourself
Starting point is 00:19:28 a tool to do that. I was with a friend and in Adelaide Australia and she said I can't I can't even remember my pen number on my card. How am I gonna do this with my bike? Well, what are the what are the numbers? She said it's 5268. And I said, well, break it into 52 and 68. What can you do with that? Oh, 52 is easy. That's the cards in a deck of cards in 68. Although, that's the swirly girl, as she said.
Starting point is 00:19:54 So she still remembers this four digit number years later. So there are devices you can use, you know. Well, if you need some information to keep any money. Yeah. use, you know, a piece of information to keep to keep in your mind. Yeah, so but long-term memory is different from short-term memory and that is it needs to migrate to another part of the brain and be connected to the other things that you know. So the two aspects of long-term memory are one that it gets thoroughly embedded in the brain and connected to many different points of knowledge. If you can attach a visual image like a medical school student now is talking to who was trying to remember stuff, medical stuff, and he said, I realized after reading the research that
Starting point is 00:20:41 if I paused and made a mental picture of the organ I was reading about, and thought about its connections, that I could remember that whole thing better. Well, you get more different connections in the brain, which become roots to finding it again later and recalling it. So it's partly about having, having it thoroughly embedded in your brain, and it's partly about having the cues you need to find it later when you want it. So it's partly about having, having it thoroughly embedded in your brand and it's partly about having the cues you need to find it later when you want it. A lot of things that we've learned in life, we can't recall because we've reassigned the cues to other things in the meantime. In terms of what you mean by cues.
Starting point is 00:21:20 Well, what I mean, Chris, is when you come to LA and you rent a car, you have to create no cues to remember to stay on the right side of the road. The cues, if you are going to take your A levels, for example, in psychology, as was the case of a psychology professor at Oxford that we talked to, he took his students, his students were had mastered the content, but the pressure on the A levels, you don't know which materials you're going to be asked to write, and then in each of the ones you're asked to write, there's multiple paragraphs. I'm sure you know those better than I, but yeah, so you took his students to coffee shops and to say, okay.
Starting point is 00:22:09 Imagine that you're walking into the coffee shop and you're going around the perimeter of the shop and they will attach meaning to the different parts of the furnishings or the layout that give them cues, a name of something that cue that next paragraph in their write-up. So when they learn what the subject is, they have to write about, they say, okay, that was at such and such pastry shop.
Starting point is 00:22:32 And I was sitting there and I can see I came in, oh yeah, I called that firm, something or other. And on they can go, they have the cues. So under pressure, when we tend to kind of freeze up and being able to recall material, that's a sort of mnemonic device. Yeah. Those are cues by which you can bring it up. It's the beginning of the deck of cards that starts to tumble into the rest of the knowledge, right?
Starting point is 00:22:56 Exactly. So I can certainly draw a parallel between that and my learning style. So to give you a little bit of context for myself, my learning style from probably my GCSEs or 15 and 16 was to write summarized notes and to then completely verbatim record in my mind what the notes were on the page. So I would know that top left starting at the top, it was this particular bit then this this then I could see when I got into the exam I can actually feel in my mind where let's say that I'd written a word incorrectly and I'd crossed it out I'd be like okay, so it's three pages in it's that old is that there's the word that's not written correctly and
Starting point is 00:23:38 Those sort of cues. I'm aware that that you know had I have had I've been able to read your book before I went to university I'd probably had probably had a much easier time of it. But it was kind of, for me, I was using recall, but that was blunt force, like just complete force of the baton. Great. And that wasn't using the desirable difficulties, spacing, and that also wasn't using point number three either. So, I think I'd maybe focused on the recall, and I discovered myself that using recall was a good way for me to at least, it's pointless learning something. If you can't then pull it back out,
Starting point is 00:24:21 as you say. It's almost as if you haven't learned. It is pointless. And so we had an email from someone recently who read the book and said, I'm, you know, I've developed a new way of taking notes. And I'd like to know what you think of it. He said instead of trying to capture everything that the lecturer tells me, I've read the material and I hear the lecture, I start, I've been writing questions in each area so that when I go back to review my
Starting point is 00:24:51 notes, I'm actually requiring myself to recall from memory and look up if I can't recall it, what the material is. That's a clever way of doing it because that's what you're going to have to do at test time. Yeah. And then the other issue about memorizing notes or memorizing a text is a tentation to memorize terms and as opposed to concepts. So it's always important when you've read something or taken the notes to be able to elaborate on them to explain why is this important?
Starting point is 00:25:36 What would happen if this weren't true? How does it relate to what I already know? I guess that's the difference between pure recall and comprehension there, right? Yes, and which leads ultimately into conceptual learning, right? Okay, that's interesting. So, I had a question that I wanted to ask, and we've led onto it pretty perfectly here. Is it possible to learn something but not remember it?
Starting point is 00:26:01 Well, sure, there's all kinds of things we've learned that we don't, that we can't recall. It does functionally, does it exist? I guess there's two ways of thinking. We can't recall it, yeah. Yeah, there's a lot of stuff that you've learned that you can't recall until you get some kind of a clue. You see someone out of your past or you start talking to someone about past things and all these memories start flooding back or you catch a smell that's particular to
Starting point is 00:26:28 a place you were when something happened, and the memories start flooding back. So that's maybe a little different from what you're asking. No, I think that's right. I think you're correct. What we've fallen back to, here is the cues, right? Right. Exactly, right. I had this problem in 1998. My wife and I
Starting point is 00:26:46 spent a year in Italy and we'd go and take lessons on the language. And the only non-English language I had in me, and I didn't have much of it, was four years of French. And so my brain kept presenting the French whenever I was trying to say something, it ain't anything. But she's not going to go down very well with an Italian. Yeah, I didn't go down. But it startled me. Well, I understood it better after I wrote this book that I'm queuing, you know, a foreign word for whatever.
Starting point is 00:27:17 And then I discovered after a year in the country and getting mildly able to get by with a language that I could differentiate between the French and the Italian on demand. But that was because I got more diverse cues, I suppose. Yes. That's really interesting. So, okay, we've understood that recall, is it fair to say that recall forms the foundation for learning or that the ability to? Yes.
Starting point is 00:27:48 Okay. So without the capacity for recall, any learning that you build on top of that from comprehension to further, what was the terminology that you used? Conceptual knowledge. Conceptual knowledge, right? Okay. So when in there, here's an interesting thing that, um, let's take, uh, or you can take, uh, hitting a fast ball, you know, you don't play baseball over there, but take some kind of a sock or move or take,
Starting point is 00:28:20 uh, you know, driving a stick shift car. Yeah. You get into a car, you're a brand new driver. You gotta adjust the seat, adjust the mirror, you gotta look here and look there, you gotta push in the clutch and push in the shifter and let out the clutch while you're putting on the gas. I'm very complicated, so I'm moving, and look and steer out into the roadway and move along.
Starting point is 00:28:41 Well, you and I do that without any thought whatsoever. I mean, we're thinking about the luncheon, we're about to arrive at. Yeah. So there's a lot of learning, and particular procedural learning, that the recall is, that the procedural learning is kind of chunked.
Starting point is 00:29:00 Like the sinus, actually call it chunked in a part of the brain, and it's available to us just like that Without having to think now how what do I do with a shift lever? What do I do with my feet is there and Will you get to people who are at a very high level in the sciences for example or any field the musicians whatever they have I spent enough time with the fundamentals of physics or energy and transfer or whatever those things are that they just invoke this law or that law subconsciously as they're looking at a situation. So those of us who don't
Starting point is 00:29:39 have that knowledge went to think through every darn step. We had to think through every part of that shifting and driving sequence. But with practice, especially if it's spaced out, it becomes second nature to us. And then we possess it. And then that becomes a mental model that we can build on and connect it to other abilities or other bodies of knowledge. I understand. So time and attention there to a degree is going to be an important factor that trying, you know, cramming and rushing your learning together is not giving you enough time to
Starting point is 00:30:14 allow that to settle and then to come back to it, right? Well, that's exactly right. So it isn't just spacing out the practice of something with intervals between so that you're a little rusty, so it's a little more difficult to do and that difficulty actually strengthens the design difficulties again right yeah right but it's also true that in a course if you're let's say you're a professor in a course is better to introduce a lot of the material earlier in the course and come back to it and distribute it over the period of the course because when learning is distributed like that
Starting point is 00:30:52 It sticks better and it has more opportunities to connect with the other related material instead of doing it in Blocked fashion or in a silo. Okay. You know first we we're going to do this, and we're going to do that. Yeah, so you kind of look, you take an overview of the entire map, and then you begin to move through the map piece by piece. Right, and so if you're managing your time, I know I'm going to have a test next Thursday on such and such. You don't want to say Wednesday is my day,
Starting point is 00:31:22 I'm dedicating to that. You better off dedicating some, a little bit of time each day between now and then because that distributing that out over time is going to help your brain do what it does well. Our brains are such that if we go to bed pondering a challenge or a problem, the brain we've discovered is working at that through the night and it will throw out your relevant stuff and try to connect this to other stuff. It's a remarkable thing. So you are kind of empowering your brain to take some responsibility for learning this
Starting point is 00:32:01 material. If you start it early and come back to it from time to time, instead of trying to force it into your brain. Well, I'm afraid that 18 years of full-time education led me to do everything last minute. So I'm probably patient zero for how bad you can be at cramming. But, you know, I can certainly appreciate the times when I have spent a little bit more time over stuff and I have distributed it throughout my week or however it may be. I'm writing thinking I read a I read a study recently that talked about the capacity for problem solving before and after a night sleep versus if you had
Starting point is 00:32:40 one five-hour chunk or two two and a half hour chunks spread over two days. And the difference between the group that had the sleeping between and their brains were allowed to reset and look at the problem with fresh eyes, the capacity for them to complete the problem was so much, so much more impressive than the group that just had one, one go at it for five hours, which two degree almost sounds counter-intuitive, because you think, oh, I've got to get back into the same head space and I've got to get myself up to speed again when I start, yeah, I've got to recall all this stuff
Starting point is 00:33:14 that I did that I might have forgotten. But yeah, almost slightly counter-intuitively, it doesn't seem to be that way, it seems to be that like you say, distributing it is. Yeah, there's a really interesting study of medical residents who are learning to reattach tiny vessels with a surgical stitch in the way. In this country, they do that. They go away for a day. And there's four lessons.
Starting point is 00:33:40 The first, they get a video and then they're given a little bit of rubber tubing called a pen rose drain and they're Shown how to pass their stitches through an anti-surgical knots and then they see a video and they're given some synthetic tissue and they practice that Then they're seen a video and they get given a tricky thigh. So there's four videos four sessions one day the end of the day They are supposedly on top of this this skill. Okay, so in in the study, half the group did it that way. There were 38 of them. And the other half came in and did the first video in practice, and they went away for a week. And they came back the second week for the second video in practice. And I'm thinking those doctors sitting there thinking, well, see if I can remember. What was that? Yeah, exactly.
Starting point is 00:34:23 My life is busy. And they struggled to recall what it was and they had this next lesson and it went away for a week. So it went like this, four weeks. So they had the exact same training, but those who's learning was spaced out over four weeks, far out exceeded the others in all expert measures. And they had the surprise tests. They were given a Rat that had a severed aorta and their challenge was to save the rat to reattach the aorta and
Starting point is 00:34:52 all of those whose learning was spaced out succeeded in a Large percentage of those whose learning wasn't spaced out and we're not able to save the rat And lucky if you were right. I'm lucky if you're a rat unless you get with the right guy. I wanted to disperse the guy, I want them. But that's just a great example of the simple power of exposing your brain to something and working with a little bit and then going away and coming back
Starting point is 00:35:23 later and making that effort to recall and then adding in. Okay. So we've got recall, we understand the importance of that. Moving on to the second key concept that you had, which was desirable difficulties and the spacing out effort. Can you explain how someone can apply that? How can you, how can you force desirable difficulties into a studying practice? Well, let's take a subject. I mean, you can, you can study, let's say there was a great
Starting point is 00:35:59 study of, well, I don't want to use one, where people were identifying, learning to identify birds. And so they had, I don't know, four or five different families of birds. And they were given exemplars of each of the families. So they'd be given a bunch of thrashers to learn. Then they'd be given a bunch of, you know, I'm not a bird guy, maybe rapture. It's your work. Put everything in different groups. Okay. And then and then they were a challenge. They were showing a bird they hadn't studied and they were
Starting point is 00:36:44 challenged to identify which family belonged to. And the interesting thing about birds is within these families, there's no single characteristic that is true of all birds within a family. So you're looking at a a sort of a group of different kinds of characteristics that generally would come to define family. So those who learned it in this way, looking at many types of one and many examples of another's worth, didn't learn them nearly as well as those who had the different exemplars shown them in random order.
Starting point is 00:37:19 So this is a study. If you were studying, you study it in random order and they learned far better. They learned they were better able to identify the unifying characteristics as well as the differentiating characteristics of those birds. And in other kinds of, say, motor skills learning in the same way of mixing up the particular problems that you're trying to do or the motor skills movements, they become much better at transferring that skill
Starting point is 00:37:49 to a non-familiar situation. So if you're in a study situation where you've got a group of different types of problems that you're trying to master, you can take it upon yourself, use flashcards to mix up your exposure. In a randomized order. In a random way, right?
Starting point is 00:38:12 Okay, that's interesting. Neurologically, there must be something going on within the brain that this particular mix-up of recall requirements, that must help to embed the learning somehow. Yes, so I'll say that one of the differences between cognitive psychology and neuroscience is that the cognitive psychologists do these empirical studies. If this happens, what is the result? And they learn from that. The neuroscientists then are saying, well, how does that work in the brain?
Starting point is 00:38:51 What is going on in the brain? And so those two fields are coming together, but there's a long way from our really understanding exactly how this works. But I'll give you a very brief example. These were elementary school children who were tossing bean bags into baskets. Okay. And 12 weeks every day at gym for 12 weeks.
Starting point is 00:39:12 That is a four in 12 weeks. Okay. And you got to get shit. Well, they are pretty young kids. They like tossing the bean bags. They're easily placed. They're easily placed. Well, I grew tossed every time into a four foot basket. The other group tossed sometimes into three foot and sometimes into five foot, never into four foot. At the end of 12 weeks, they're all tested on the four foot basket, and those
Starting point is 00:39:38 who were best able to hit the four foot basket were the ones that had practiced on three feet and five feet, but never on the four feet. That's so interesting. And so the reason for that, the speculation, you know, what the scientists believe is the reason for that is that these three foot and five foot challenges required variation in judging distance and the motor response to it. And gave them more complex, the really interesting thing for me was that they believe that this skill of the three and five foot tossers was encoded in a different part of the brain, where more complex motor skills are encoded,
Starting point is 00:40:22 than those who just repeated the four foot toss over and over and over again. One was a little more robotic and didn't require the extra comprehension, right? The conceptualization. Yeah, right. That's really interesting. That's a fascinating study.
Starting point is 00:40:39 So I think what you've done there is you've touched on what again, it is a cognitive bias that we sometimes have to presume that because we're able to recall, you know, if you stick to one thing for a night and you're able to recall that one thing, you're like, yes, I've got some work done. It's planting a flag in the ground that, okay, I can remember this one thing. Whereas the desirable difficulty, which is potentially a better route to learning longer term, in the short term can actually make you feel like you're doing less work, right, or like you're making less progress.
Starting point is 00:41:13 It's like, oh, well, I can't really, in full, recall any of these individual parts, but across the whole, the amount that you've learned in quotation marks has increased. Is that fair to say? Yes, it is. I think you are more easily discouraged when you do that. I just must say, in any of these things that you're going to try to practice in a random order, you want to try, you want to try your 20-foot putt a few times, you kind of get a sense of what you're trying to do, you want to find the volume of the spheroid, a few different ones, a few times. So you get a sense of what that is. And then the practice is mixed up. So don't keep doing the single one until you've got it nailed so to speak
Starting point is 00:42:10 Interrupted interrupt it and move on to the others and come back to it later. It won't feel as good but Just going back to my own personal experience struggling with the Italian language We got over there the first of the year and by the fall of that year I was with some young Italians. We had befriended from the school and we were chatting in Italian. And I heard myself using idioms that I didn't know I knew. And they said, come to me through stopping. I rode my bike around a lot and I'd stop at the
Starting point is 00:42:45 town water pumps to get water and I'd try to chat with the people who are hunting birds or whatever, you know, and it was always a struggle and a frustration. But the brain is probably for both probably for both parties as well probably a frustration for them as well as for you. I'm sure of that. Down to Americans. So you can't give too much weight to your frustration. You just need to kind of trust your brain and your brain's working at making sense of it.
Starting point is 00:43:13 As long as you give it the challenge. Yeah. Okay. So I get that sort of touches on the one of the third main concepts, isn't it, as well, that to you don't really know, I guess using your own judgment for how your learning is going, sometimes you kind of just need to stick to the program, right?
Starting point is 00:43:33 Well, right, a really important thing that these cognitive psychologists talk about is what they call it, metacognition, which is thinking about your thinking, and that is we're easily duped to thinking we're on top of something that we've got it. And so what's important is to test yourself from time to time. Do you really have it? Can you really do it?
Starting point is 00:43:57 Can you really explain this? And so self-testing has a couple of benefits. One is you learn whether your judgment of what you know and can do actually is accurate. And it helps you focus where you need to bring it up a little bit. But the other is, the fact of recalling this stuff is a great way to strengthen your mastery and retention of it. So as a study strategy, self-quizzing is potent in several ways. There's a...
Starting point is 00:44:32 In fact, excuse me. I was just saying, when we're talking in schools with professors and teachers uh... one of the fundamental things that is highly effective is for uh... the instructor to incorporate frequent low stakes quizzing in the course low stakes so that people aren't freaked out by no no pressure yeah right you should really die really dialed out in the pressure
Starting point is 00:45:04 and have the the experience and ultimately the habit of recalling from memory what this stuff is. And if you can do that in a course where you're reaching back to earlier material as well as more recent stuff, that stuff gets brought forward and better connected. It's a very potent and not very difficult strategy for helping students get to the middle and the end of their course on top of the material. That's interesting. There's a program that I know a lot of my friends who are doctors, medical students use called Anki. I'm not sure if it's... Anki's? Yes.
Starting point is 00:45:39 So Anki's Q-Cards, right? Randomised Q-Cards, mostly with multiple choice questions, and would that fit into your model of consistent low-level testing? Yes, Anke is great, yeah. I wish I'd known about Anke while I was at university. I think it might have made my last-minute procrastination tactics a little bit lesser. A little bit less proliferant. How far does the nice thing about Anke and some other online stuff is that you can set it up to come to your phone periodically.
Starting point is 00:46:12 Yeah, they get reminders. We'll be sat at dinner and I'll look over to one of the guys and they'll have his Anki Q cards out because his reminders popped up. Well, if you were in our household, if you were doing anything other than practicing what he's supposed to learn, he'd get a steep scolding from my wife, having his phone with the... Oh, no. Oh, no.
Starting point is 00:46:32 That's a... But that's a... But that's a... That's an undesirable difficulty, right there. But that's a... Okay, so we've got the capacity for recall, we've got the desirable difficulties, and we've got kind of trying to ignore what your brain is telling you about how your learning is going.
Starting point is 00:46:53 Yeah, I think it helps if you can think about stuff you struggle with that you've had these kind of breakthrough moments. It might be, I think of people on BMX stunt bikes. I think of people doing video games, trying to find their way to the next level, trying this, trying that. Oh, yeah, this is what I did last time that we're kind of thing. All the different ways that we're not thinking of as academics, but in our lives, where we go through these kinds of space and episodic exposures to something we're
Starting point is 00:47:33 trying to master. And then at some point, you realize it's coming to you. You know, you've kind of got it. That's going to be the same with the semantic material in a class or with the motor scale stuff when you're trying to learn how to stitch up a rat or trying to, you know, do a fire of air. Yeah. Okay, I understand on that. Did you in the book, did you touch on anything to do with the focus and diffuse mode networks? Did you look at that much at all? I don't think I've heard of that. Well, okay. So there's a course from Coursera, which is a massive open online course, and it's called Learning How to Learn. I had a little bit of a look through that
Starting point is 00:48:25 before I knew we were gonna have our podcast today. And with that, they're focusing a lot on the focus and diffuse mode networks. What they were talking about there was, it seems to me the longer term, the ability to recall over time. And this, as you say, this more open mode of thinking, as opposed to a more procedural mode of thinking, that when you're learning something and you go away
Starting point is 00:48:52 and then you come back, you often have a very open mind to what the, what the solutions may be. We've discussed it previously. I've said, I did a podcast on meditation not very long ago with Cori Allen and one of the things that I wish wouldn't happen, but I also appreciate does, is when I sit down to meditate and I quiet in my thoughts, that's often when I have some of my best ideas of the day, which is terrible because I'm trapped in this, I'm supposed to be trapped, focusing on clearing my mind and I've got all of these really good, like awesome ideas coming to me. And I think, God, can you not happen when I'm not meditating, can the meditating just be
Starting point is 00:49:30 allowed to continue on its own? Alright, yeah, I've had that experience, but of course, this is the brain offering you some ideas. And if I understand this difference between the focus and the fused, I think one of the astonishing things in writing is when your mind presents a metaphor for something that helps make clear this other thing, you know, this notion of the similarities between things that your brain recognizes. And so, this is one of the things I believe from what I've read about sleep that begins to happen when you think about, struggle a little bit with a problem, or maybe a conceptual problem in the evening.
Starting point is 00:50:25 And in the night, the brain will kind of look around and see what do I know that's similar to this, are there models that are like this, is it like this other thing? And you can get some great breakthrough ideas that way. So maybe in the meditation, where you're just giving your brain, you're trying to focus on the mantra, but you're also focusing on your breathing and your relaxation.
Starting point is 00:50:52 And it's a very free thing. And if the brain is presenting you with images and ideas, I think that I can only think that that's constructive. You know, I agree. I can only think that that's constructive. You know, I agree. I couldn't agree more. I think one of the examples that was used was Newton and it was that he would sit in his chair with ball bearings in his hand and he would wait until he just fell asleep and the ball bearings would drop from his hand and crack on the floor and it would wake
Starting point is 00:51:21 him up. And that was the moment at which for him, he often had a lot of breakthroughs. And you know, if it's good enough for Newton, it's probably good enough for us. Well, that's, and happens for me when I get when I get frustrated with my writing and I get on the bike and I head up the hill and I just smell the smells and look at the horizon and a bike along and all of a sudden, you know, my brain starts giving me ideas. I'll get there's like, aha moment. This is what I could do about that problem. And then I can come back and work on it. Do you think it's difficult to, as someone who is learning or trying to be productive
Starting point is 00:51:58 or writing or whatever it might be? Do you think it's difficult to learn when enough focused time is enough and it's time to move on to, or it's time to take a, as you say, like a planned break to give yourself that room to breathe? Because you know, you need to spend some time on detention exposed to the material or exposed to the particular physical practice or whatever it might be. But then also you need this time away from it as well. Do you think it can be a difficult scheduling problem for people to know when enough is enough? Well, I'm sure it can be. I don't know. I don't know of research into that question.
Starting point is 00:52:37 My own human nature tells me you need to have, you need to mix it up and you need to get out of your chair or get out and get some exercise and then come back to it. But I don't, I'm not aware of research on it. One of the things that was a pleasant surprise for me when I was working on this book, I hadn't been aware of the search tool, Google Scholar, where you can search for published research on, you know, you could put in a term and see where research has been done and thumbed through the different published studies. Yeah, it's kind of useful. I suppose most everybody else knows about it, but I didn't, and I thought it very awful. But, you know, to me, intuitively, yeah, but of course I'm saying intuition leads us to stray with learning. So you got to be careful on both sides of the fence, right?
Starting point is 00:53:28 Yeah, well, I was one thing, you know, one of the things about being an on-site is writing this book with two scientists was I was, I tried to be, I tried to take liberties in relating the science to stories of real people and incidents in their lives and so forth and drawing parallels, but stay within the strict limits of what the empirical research shows us. And where we're drifting into speculation to say this is speculation, but it might be because of such and such. So we've gone through the three key concepts that you've got there. The capacity for recall, the desirable difficulties, and then ignoring the cognitive biases to one degree or another.
Starting point is 00:54:14 Or cutting yourself some slack if it feels difficult thinking you're not getting it. Yeah, cut yourself some slack. You think you're not getting it, but you probably are. Yeah. Okay. So, are there any other elements that you think that people need to be able to understand when it comes to learning? Is there anything that we haven't gone through yet?
Starting point is 00:54:31 Well, there is this theory of a growth mindset that has been introduced by cognitive psychologist at Stanford University in California named Carol Dweck. And she's done some work that she was studied with, she took an interest in why do some people become help us when faced with a difficult problem. And so she did some work with some low-performing New York middle school, junior high school students and gave them a little seminar on how the brain works. And then half the students, she took a side and talked about memory. The other half, she took a side and said,
Starting point is 00:55:18 a lot of people think that their abilities are set at birth by the gift of their genes. But in fact, when you work hard to learn something, you are building new connections in the brain and over time, you're actually increasing your intellectual abilities through these challenges, these mental challenges. She sent them all back into class. She didn't tell the teachers about these two different
Starting point is 00:55:40 subgroups and the students who had been taught that they have some influence over their intellectual abilities by picking tougher problems and persisting at them began outperforming the other students. And she is Dr. Duaktev's, I'll have this sort of duality of a fixed mindset versus a growth mindset.
Starting point is 00:56:00 And the people with a fixed mindset tend not to pick problems they won't do well at because it'll indict their sense of their native ability whereas those who have a growth mindset when they encounter a difficult problem try a little harder, try a different way, you know, carry on forward and do better. So there's an issue here I think when we're saying that some difficulties are desirable, it's not, it doesn't sound like good news to students. No, I mean, especially a lot of students want the easiest path possible, right?
Starting point is 00:56:35 Well, I mean, we all do. We're all, we're all, yeah, we all want to get as much as possible for us, little as possible, right? It's human nature. I think we all agree on that. Yeah. Yeah. So I think that there's more research that needs to be done about whether a belief in and an understanding of this ability to improve your brain is actually a motivator for people. But the fact remains nonetheless that when you learn new things, especially you rise up the complexity of the topic you're dealing with, you are increasing your intellectual ability.
Starting point is 00:57:15 And there's that many more things that you now have the ability to learn because you have places to attach them. You can't learn something new if you don't have something to attach it to that you already know. So I think there's the field of motivation is one where there's some interesting work being done and it's one that I think it helps for people to keep in mind that you need to be kind of a coach to your own brain saying, I know this doesn't feel too good, but guess what, it's really going to pay off. Yeah, I totally get that. Is it a weird, a weird analogy that I can draw between this and a recent podcast that I did with some
Starting point is 00:57:57 CrossFit coaches from the UK, and in it I was asking about mental toughness in sport. And I was talking to them about how they prepare for these competitions. And during these CrossFit competitions, a lot of the athletes very, very heavily mediated by their capacity to suffer discomfort. And again, with that, it's a lot to do with mental state. So I was asking the guys, you know, how do they prepare in an appropriate way? And what they said was, how many events is it over a weekend? Is it seven events over a weekend? Okay, well, during your prep for it, do 20, do 20 events over a weekend. And then when you get there, you realize,
Starting point is 00:58:34 I've prepared so much further above and beyond where I needed to, to bring it back to. But you're right. The difference between an athlete that's doing that and typically, how someone will be learning, even if they're being taught by a lecturer or a teacher or on a formal course at university or whatever it might be, the coach is mediating that impression of how the training is going. Right? It's very rare. I don't think I ever once spoke to a lecturer at university or spoke to my tutor and said, my learning is going slowly. I just perceived that as a byproduct of, oh, well, I spoke to my tutor and said, my learning is going slowly. I just perceived that as a byproduct of, oh, well, like learning isn't easy and I need to just,
Starting point is 00:59:10 you know, put my head in the book a little bit more until it sticks. And I think that the paradigm, the environment that students exist in is very different to different to that typically if someone perhaps who's doing sport, that you know, your coach can see when you are and when you're not getting it. Whereas there's a lot more, a lot more variables that can, well, does your lecturer think that your test scores during the build up to exams are low because you're not working hard enough? They don't really know. They don't see you practice. Do you know what I mean? And the practice is a lot less visual as well. It's a lot less easy for them to work out. So yeah, I think the concept of desirable difficulties is it's something that shy of definition
Starting point is 00:59:56 as we've done today is probably going to be really hard for people to actually work out where it's right. Yes, I don't think it has to be real hard. I have a couple thoughts. One is that we're encouraging teachers to help students construct their own understanding of the material. As opposed to lecturing,
Starting point is 01:00:22 here's what the material is in trying to impart the lecturers' understanding. So that means the classrooms become much more involved in exercises that engage the students in working through and figuring stuff out. And so that's one thought. I've lost the other one, so I might just have to go. You know, Did you end up with your comment? It was about CrossFit and it was about having a high level of difficulty than you need. Train hard, perform easy. Right. I don't know. It's gone. sorry. That's okay. It'll come back.
Starting point is 01:01:05 That's totally fine. We'll get it in a bit. But no, I think you're totally right. I can definitely say, I enjoyed my time at Newcastle University, but the course that I was doing quite often we'd have 200 to 250 people in Electro Theatre. And that was for a lot of the modules that I took. And in terms of comprehension and that two-way
Starting point is 01:01:27 communication with the person who is disseminating the course information, it was nonexistent. You can't ask questions from a room of 200 people. I mean, they could barely keep us awake because it's such a big room and it's so unengaging, so on and so forth. So yeah, the implications for teachers here are almost as wide as the implications are for students, right? Right, I think that's right, and I think the revelation for the teacher is, it's really about learning, it's not about teaching. This is about learning. This is about how you can help students become the learners, not you be the teacher. learners, not you be the teacher. And the, the, oh, God, I had this other thought came and now it's going back again. It's just making me, yeah, I don't know. It's all this, it's all this talk of, it's all this talk of recall, it's ruining
Starting point is 01:02:19 your capacity for recall. I think my short-term memory is, it's my working memory, it's just a little fraud right now. That's fine. That's totally fine. Maybe you should go away, maybe you need to go away and come back to it. I'm not sure. Probably need to do that. I think we've laid out some quite nice concepts there. We've gone through what does learning mean, how do learning and memory relate, what does recall mean the desirable difficulties and giving yourself a little bit of a break and not. I think I like the the third key concept that you've got there because I think that's the one which will mediate all of the rest that if you don't believe that you're making progress, you're going to fall
Starting point is 01:03:06 back into the typical brute force, one concept at a time approach. And I think that all of the others are relying on you having faith in the learning process working. Right. So I got my idea back. So I have two thoughts. One is that often an instructor, what instructor is teaching you about a topic,
Starting point is 01:03:34 the information is arbitrary. The student doesn't know why it's important or what it connects to, and so forth. So it's really important for instructors to, when they lay out something to explain. Here's how this relates to where we've been and where we're going. Here's how it connects to what we already know. So that the students have a way of connecting to it and grabbing hold of it and making sense. Now the other side of that is I think students
Starting point is 01:04:02 have an opportunity if they're not getting it to say to the instructor, I'm not getting it. I don't quite understand how this relates. Could you give me an example of how this is true elsewhere or another topic or something to help me understand it? And then I can practice at building that learning, a dialogue, I think, is, we just use to a kind of, I'm the expert, you're the student, I'm going to tell you how it is, you're going to take notes, then we're going to have a test. Yeah, that's definitely my functional experience of learning. That it's very much a one way street and that there will be, it almost felt sometimes like lecturers knew that they had to ask a particular number of questions and that that was just a hurdle they had to overcome. The same as saying the next slide, it wasn't
Starting point is 01:04:55 something which was being done to actually embed the learning. Right. Yeah, which I think that's really interesting interesting the implications for teachers and lecturers. I mean, you know in an ideal world Every university course would probably start with two weeks on learning how to learn right I think every incoming student at a university should have Of course in that how learning works and how to manage your time and you know I was in Florida International University in Miami where they'd make a stick and sat down at the law school
Starting point is 01:05:32 and they've completely restructured their orientation and their management of students through the two-year law school around these principles. They've gone from typically fourth or fifth place in the Bar exam in Florida as a school to placing first place five out of the last six exams, but it's by this conscious effort is when the students come in, let's talk about these principles and how they work and having experience with it. I just was having correspondence with someone in
Starting point is 01:06:03 Ecuador who was saying I would like to have a simple classroom experience. And my co-author, Mark McDaniel, said, well, they were learning vocabulary. He said, well, here's what you could do. You could give them half the vocabulary in a sheet to memorize. And you could give them the other half of the vocabulary and flashcards to learn. And then have a test and see for themselves which ones they learn better. Yeah, yeah. And then, you know, I think that getting students to buy into the process is super, super important that you're right, that there's more difficulty going to be through this particular style of learning. It's going to feel less immediately gratifying, right? Because you can't straight away recall big chunks of
Starting point is 01:06:49 concepts. And you need to have faith, you need to trust the process. It's what we say about when when we talk to powerlifting coaches and stuff like that and some of the athletes aren't happy with where their lifts are at and they don't feel right and they say, look, just trust the process. Have faith in the fact that you have outsourced the specific task of programming what is to be done and when to someone who knows what they're talking about. And it is your job to do what they say, what they tell you to do. And not vice versa, you shouldn't be questioning it. Obviously, there needs to be a back and forth, as you've said, and you need to at regular points reassess where you're learning is that and how your learning's progressing. But stick to the program, I think. Right. Not only stick to the program, but I think have two goals. One goal is to master the content or the skill. The other is to learn how to be a superior learner.
Starting point is 01:07:47 So that when you get out of whatever this program is and you find something else you want to do, you know you have the habits that will make you of a highly effective at mastering that. And that will make you a competitor in this crazy world, give you a great advantage. For sure, I mean, it's a principle that's completely, as we've said at the start,
Starting point is 01:08:07 it's ubiquitous, right? If you can learn well, then it's the first domino has fallen on every single, everything. And that goes from, as you've used a wide range from physical pursuits to intellectual pursuits, specialists, it's definitely something very crucial. I think, you know, I wish that I'd had a better comprehension of time management when I was at university and of how learning works and of the strategies that I could have
Starting point is 01:08:38 implemented. I think how they have had that, it would have made my life a lot easier. I by look, managed to fall on at least one of the three key concepts that you have put in here and I genuinely think that's probably the majority of what's carried me through my university degree. So, you know, it's it's fortunate that I stumbled upon that kind of just through trial and error. But if people can implement all of these, I think they'll have a much easier and much more comprehensive learning experience. Well, I hope that some of your listeners tumble to it and give it a shot. I'm sure that they will. So can make it stick.
Starting point is 01:09:14 Make it stick indeed. Can you tell the listeners where they can find you online? Well, there's a website makeitstick.com. All one word, make it stick. Yep. The book is make it stick. The science of successful learning is published by Harvard University Press.
Starting point is 01:09:34 It's available on Amazon. It's being translated into 14 different languages. So if you have listeners who prefer to read it in a language other than English, some of those are already out and others are all be out soon. So some of your Italian friends can I'm going to make them learn English. Yeah, that's it. Yeah, exactly. Of course they are. Yeah, but that's fantastic. Peter, I'll make sure that links to the book and to your website will be in the show notes below. So if anyone
Starting point is 01:10:07 does want to get hold of the book, I highly recommend it and they can grab that through the show notes in the links. I'm sure that we're going to be coming in and dated with some questions and stuff like that. So if we do have something, I'll fire them over to you and I'll be able to answer those on future episodes as well. But I really appreciate your time. I think I appreciate yours Chris. It's really been fun chatting with you. It's been fantastic. I really do think that will have helped some people in reframe their approach to learning. We keep coming back to it, but I do think that you can't emphasize it enough that the capacity, your understanding of how to learn is the fundamental case of what is going to restrict your ability to learn anything. And I think it's a really fundamental task.
Starting point is 01:10:51 And hopefully as well over the coming years, as you say, universities and institutions have started to latch on to the idea that this is important. And if you can continue singing that song, then maybe university won't be quite such a daunting and difficult task for some people in the future as well. I think the wind will be at your back if you do. That's the plan. Well, Peter, thank you very much again, and I appreciate it. Thank you very much. Cheers.
Starting point is 01:11:16 Thank you very much. Yeah, oh, yeah, oh, yeah

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.