Psychiatry & Psychotherapy Podcast - Using Deliberate Practice to Improve Psychotherapy Results with Dr. Scott Miller

Episode Date: July 7, 2023

Dr. Miller's first appearance on the Psychiatry and Psychotherapy Podcast was in episode 077, "Getting Better Results from your Patients as a Psychotherapist," during which we discussed his book, Better Results. We explored the methodology behind improving outcomes in therapy through targeted development of what Dr. Miller has dubbed the Common Factors, which include therapy structure, hope and expectancy, working alliance, client factors, and therapist factors. In this episode, Dr. Miller returns to expand upon our prior conversation with a focus on how therapists can use deliberate practice to improve their efficacy. We discuss Dr. Miller's new book, The Field Guide to Better Results, a companion to Better Results, which was recently published in May, 2023.   By listening to this episode, you can earn 1.5 Psychiatry CME Credits. Link to blog. Link to YouTube video.  

Welcome back to the podcast. I am joined today by Dr. Scott Miller. He is a PhD therapist. He is someone who is on the podcast for an earlier episode talking about his book, Better Results. May 23rd, he has a new book coming out, The Field Guide to Better Results. You can pre-order this on Amazon and on the APA website. And I'm excited to have you come back to further explore. your life work and things that you're finding out. It's been a couple years since I had you on. And so, yeah, just to kind of start it up, I was thinking we could talk about how you started. And I was listening to some prior interviews that you had and you were talking about how you were initially into brief psychotherapy. And there was this moment where you were, where you invited some researchers to come out and study. what you guys were doing. Do you want to tell that story?

It's kind of like a... Sure. And thanks for having me here. And also plugging my books. The field guide is actually a follow-up and a companion volume to the book Better Results. We dig into the elements and factors that have influence on the outcome of psychological care in much greater detail.

We're very lucky to score the biggest names in terms of research in the field in the various domains, things like the relationship, client factors, hope and expectancy factors, structure and focus in therapy. So I feel really very fortunate. And like many projects, that particular book, the new one that is, we knew we were going to write it as we were finishing up the other book because there was just so much that was missing that we didn't know. And the same thing has happened with this one. We feel like this is a real journey trying to understand how can we help therapists improve their results? And we find therapists very hungry for that. And as a beginning psychologist 35 years ago, having moved from sunny Southern California to Milwaukee,

to work with two very gifted and well-known clinicians in Sue Berg and Steve DeShazer, that was still at the forefront of my mind. I was trying to puzzle out, how could I be more effective? And I think my interest in the brief therapist was peaked because they were willing to show you their work, let you peek through the one-way mirror and actually watch what they did in the room with clients and also be quite specific about what they did. Now, it can be pretty hard in our current protocol-driven therapy times to imagine a time when you weren't able to see what therapist did and where they didn't give you specific recommendations. But when I was going to gradual school, it was really very much theory-based. And the concepts were hard to grasp,

and you were responsible for the translation of those theories into the actual work you did with clients. The only video that was available, it's hard to remember, were the Gloria films, where we saw Fritz Perl's and Carl Rogers and others work with this one client. So I was very lucky to meet Insouberg at a workshop, and I began to write to her. I was sending her some of the things that I was writing and thinking about. One day I said that I'd been and offered a job to work with Patricia Hudson and Bill O'Handlin up in Nebraska, and she said, well, don't take that job till you come to see what we're doing. And when I went to the Brie Family Therapy Center at 6815 West Capitol Drive in Milwaukee,

I knew there was really no choice to be made. It was a working clinic where they were really trying to understand how could we improve the outcome of psychological care. And, of course, they developed a very specific model. that model was solution focused at the time called brief therapy. And the idea was by focusing on strengths and resources, looking more at when problems didn't occur rather than trying to understand why they were occurring, that we would end up with more effective care, more single session cures, et cetera. And what was really great in part about being there was they told you exactly what to do. We watched each other work. We experienced. experimented with new ways, my confidence increased in major ways while I was there. And so probably at the third year, I suppose it was, during my tenure, we asked a couple of researchers from different universities to come in and follow up on some of our clients. I think there were 250 of them in their survey, and they asked, how did we do?

And I can see the day. It was, I believe, a kind of cloudy, overcast, cool day in the spring when these two researchers came back and said, we have good news and bad news. And we said, well, what's the good news? Well, what you do works. It works very well. I thought, well, what could the bad news be then? And they said, well, it doesn't work any better than anything else. And it's not any briefer than anything else.

And yes, you do have single sessions, but that's been the case from the 1940s. The modal number of times people seek out the help of mental health professional is one. And this was a big shock to the team. And we had a choice to make it that particular time, right? We could either continue and act as if what we were saying really mattered or we could start to question our assumptions. Yeah. And so you mentioned that there was kind of a breakup. There was a breakup.

It was painful because these are people that I admired and respected. And we had a phenomenal chemistry. People came from all over the world. We had a huge room behind the one-way mirror in one of the main rooms where we actually had stadium seating because many times there might be 30 or 40 people watching the sessions. Such rich discussion. and I'll admit a little secret here. I would say that oftentimes I enjoy the process of watching the work and discussing it with other clinicians more than the actual work.

There's something about the puzzle of working together with another human being, with a family of beings to see if we can't help them. That is so stimulating. And so that was painful part of the breakup. But we were also pretty dedicated to really trying to figure this out. So we had to split, we had to follow what the research was telling us. You know, I can both appreciate how wonderful that experience was, you know, but then also your intellectual and their intellectual honesty with like, okay, like this, you know, we can't just

narcissistically act now as if we are doing something so extraordinary, right, that like it needs to continue with the same fervor. Yeah. We need to go back to the drawing board and figure out, okay, what is, you know, what is the best way of moving forward? And I think that's what you did with your career, right? And you kind of, yeah. I think I've tried to stay in those crevices or at the end.

of what we know. A complete and total, all-encompassing theory of everything quickly loses interest for me. And it just so happens that what we were doing, while it was effective, it didn't really account for that effectiveness, and it didn't make us any more effective. And so the team did split, and part of the team went off, and I began reaching out to former professors that I had, rereading material, and we began to look at common factors, what are referred to as common factors. The basic logic of the argument was that if it wasn't the specifics of this particular treatment model or any other, well, then maybe it's what these approaches all shared. And we spent probably eight, nine, ten years looking at those

common factors. Before, once again, we came to the realization that, well, studying these common factors wasn't likely to make us more effective. It was just explaining what we already really knew. And here, the team split again. Part of the team stayed with common factors. The other part of the team decided that maybe we'd never figure out how psychotherapy worked so that in detail enough so that we could teach others to do it. What we could certainly do is begin measuring our results. And by doing that, we would instantly know who we were helping and who we weren't. I hope this logic makes sense.

So first we start with this specific model. Then we decide, well, maybe it's what all the models share. We're looking for an explanation for why therapy is effective. We figure out that teaching that is not likely to make us more effective. But what we can do is know from session to session, client to client, if what we did is actually making a difference in their life. And so we developed a couple of simple measurement tools. They are in wide use around the world. And as a matter of fact, soon we started to see that we were helpful sometimes as a field and at other times we weren't.

And that applied individually to us as well. Sometimes I helped certain clients and sometimes I didn't. The big breakthrough, I think for me, most recently came when we noticed that certain therapists rose to the top in terms of their effectiveness over and over again. It didn't matter who they saw, what the problem was. Get that client in their hands and their outcomes tended to be better than average therapists, of which I was one of those. Once again, interest was piqued. What was it about those people? And I will say that the pushback about investigating this was really curious.

Because, well, people wrote to me and said, you know, why are you studying what in the research had been called all the way back in the 70s, these super performers or super shrinks? Why are you studying them therapists? They have a hard job. All they should really be expected to do is just turn in an average performance. And I thought, oh, geez, you know, I hope that person is not flying my plane. You know, I would really like the person flying my plane to be slightly above average, if you know what I mean. So I said, I'm not interested in turning everybody into a super performer.

I'm interested in helping that one person from last week that I didn't help as much as I did others. And maybe these top performers can provide us with some, type of information, something to research. It turns out that there are people who've been studying top performers. I had no idea. No one had ever talked about this in any great detail in our field, even though the person who'd done all the seminal research in that area was a psychologist, a Swedish psychologist

by the name of Anders Erickson. And he'd been investigating the best chess players, the best computer programmers, the best teachers, the best surgeons, for 40 years. And he had an idea about why some were better than others. And as soon as I encountered his material, for me, it really opened a door and seemed to point a way that average performers like myself, and by the way, I don't call us average to denigrate our performance, our average outcomes are very good. On par with most medical treatments are actually better, an effect size of 0.8. That's a large effect size by doing talking therapies. But what always created doubt in my mind was that one client last week that I didn't help as much. Or you would notice

that some agencies were less effective than other agencies. Erickson, in studying these top performers, coined a term to describe what they did different, and that was deliberate practice. And it truly was like a light going off because deliberate practice meant reaching for performance objectives that lie just beyond your current ability. And so we decided to do some research. I was very fortunate to have a colleague and graduate student at the time, Dr. Daryl Chow, now Dr. Daryl Chow, who was willing to do the original research, the first study on deliberate practice in the field of psychotherapy. And as a matter of fact, that study showed that

top performers spent more time engaged in deliberate practice compared to their more average counterparts. Top performers in psychotherapy? Yes. Sorry. No, I just wanted to clarify. So yeah, what do those initial findings show? They showed that top performers, that is therapists with the best outcomes spent 14 times more time engaged in activities designed to extend their performance at their learning edge than the poorest performers in our sample. Here's what's curious. If you ask the poor performers, how effective are they? They rate themselves as good as the best performers. To have a more realistic comparison, top performers spend about two to two to two and a half times more effort aimed at deliberate practice than the average performers in the sample.

And so, so like how did you, like, give me some examples of what it looked like to practice at the leading edge? So what we did in that original study to assess deliberate practices, we created a list because we're in an exploratory mode. We have no idea what deliberate practice looks like in the field. But we followed, we got advice from Canders Erickson who had developed a research methodology, retrospective recall, where he had people recall or keep records about how they spent their time outside of the therapy hour. This is a critical distinction, by the way, because all of us are familiar with practice. Your parents told you to practice, whether it was the piano or throwing the baseball or trying to shoot hoops. What that meant is you'd go out there and repeat things over and

over and over again. That's not deliberate practice. That's purposeful practice where you're trying to reach a certain criterion level of performance. That's what our whole field is based on trying to reach that criterion level of minimal competence. Deliberate practice means getting right to your edge where your performance begins to break down and then pushing yourself beyond that. So we had people keep logs of how they spent their time. We used a survey called rapid practice, and we also asked people how much time they spent in a whole host of activities and how much cognitive effort was required in order to do that. And we found that the top performers simply spent more time in more activities at a greater cognitive expense. In other words, they were depleted by the activity. They were

pushing beyond what they could presently do with comfort. The way we're looking at this now, as we've become, I think, more focused in our research, we've just finished a research project called the Difficult Conversations and Therapy Project, where we expose clinicians to a difficult or challenging clinical scenario, and we measure what we know is one core ingredient of effective clinical work, empathy. Okay, yeah. So you're exposed to this difficult encounter as a therapist. You might be sitting in front of a keyboard and type in your response or just be asked to

record your response. We then rate the degree of empathy. Two independent raters rate that degree of empathy using a standardized scale. That becomes your baseline. That's how good you are in general. Okay. Then we start giving you very focused feedback. about how to improve your empathic response.

It's tied to the deficit that you exhibit. So therapists, by the time they've finished graduate school, they've seen a few hundred clients, they're pretty decent at providing an empathic response, but they don't get better over time. Let me say that again. They think they do, they don't. How do we know that? We've got research to show it.

So what we do is we measure that baseline, and we, we do. we identify the components of the empathic response where they fall slightly short, and we point to those and give them some ideas, actual principles that they might operationalize in their responses. We give them an opportunity to practice, get more feedback, including scores on the standardized measure. Over time, that research shows clinicians' empathic responsiveness improves. and here's the cool part. It generalizes. So not only in the specific scenario, maybe a client, for example,

who's pissed off at the therapist, angry at them for some kind of failure in the therapeutic relationship, to a sad client who's threatening to commit suicide at the end of the session. So it generalizes overtime. That's an example of deliberate practice, getting you right to your edge,

identifying where in your performance the deficits lie, giving you some coaching specific to that deficit. That's so good. Yeah. That's, and what did you find from that study? Or what was like, is that study ongoing or is, that study is, that study is under review in frontiers of clinical psychology, I think. It's the name of the journal that it was submitted to.

Okay. What we found is that therapist's empathic responsiveness improved and, you. generalized. As compared to controls who attempted to simply reflect and make themselves deliver a more empathic response. So the idea that I can sit at home in my office and think myself more empathic, it just didn't hold up in the research that we've done. We need some quite specific coaching to our deficits in order to improve. That's exciting. I can. I guess I say, David, I'm sorry to interrupt. The big test will be, does that training lead to better outcomes? And that's sort of where we're at now. We can improve the skill level so that outside judges that are measuring and scoring your performance can actually see that it's better than it was before.

Hopefully, we tie that to individual clinicians improving in their results. Yeah, that aligns with my, like, what? what I've been working with the residents. We do continuous case conference. We watch video of them, seeing clients. And pretty much my whole theme in the last five years has been noticing moments where they don't give empathy or they could give empathy a little bit better. That being said, sometimes I watch more of the video and I see what they're doing. And it's like, okay, I could see. You were trying to give empathy, but you were not quite giving it yet. But sometimes I'll stop it prematurely and I'll go around the room. and I'll have each resident say what they would say at that moment,

you know, like if they were trying to give some empathy. So I love your focus on this. I'm like very curious as you've been in this journey, have you changed your responses to people as you study empathy and as you study like how the scores are best scoring empathy versus not scoring. Like what for you and your personal journey in empathy, what have you noticed or what have you learned in this little journey of coaching empathy? So I would say that I haven't measured my empathic responsiveness in the same degree of detail as we did in this particular study.

And empathy is one of five different factors which have leverage on outcome. There is also the creation of hope and expectancy. There's structuring. there are other alliance variables. There is being able to utilize and take advantage of certain aspects of the client that they bring into the room events that happen outside of the room. All of these are potential areas that we could leverage. I can tell you that as you, as we have explored these various areas, including empathy, you can't help but reflect on this as a clinician, that's where it starts in some ways that you notice that what the research is telling us doesn't, it doesn't fit with exactly how I work. And then that causes me to reflect.

But in order for me to actually get better, our data say, you have to move beyond simple reflection, you would actually have to identify the specific deficit and get coached to that particular deficit. What am I saying? I think that cognitive appraisal, the impact of that is vastly overrated. And it probably leads to a false confidence on the part of clinicians. Because in our minds, what educated people do is they talk themselves into believing that they're better at it than they are despite their efforts. So deliberate practice, in my estimation, is not something you can do on your own. You need a coach who stands from the outside and does a systematic evaluation tied to your results. So I'm trying not to be falsely modest here when I say, I think it's affected me, but I have

haven't really participated in the protocol we developed just for this one area of empathic responsiveness. Okay. I'm, okay, so I was talking to Nancy McWilliams recently. Oh, I like her. Isn't, like, isn't she wonderful? She's wonderful, really. So one thing, I said something similar about like overconfident therapists and she said by and large that isn't her experience by and large her experiences therapists are often very underconfident and self-deprecating you know it's like every day I get emails from therapists like I don't feel like I'm doing this right and I'm studying this modality this this this the modality and I just did this trauma-unformed treatment you know seminar and like where do you think I should go next what book should I read next you know

And so I think it's rare in my mind that I get someone who's like, I know what I'm doing, I'm good, like, I'm doing the same thing over and over again. You know, that's like a rarity to me. It's like more common that you have a therapist that's like a little bit more depressogenic, a little bit more, you know, negatively self-reflective without that false confidence. But you're saying, I don't know, you're seeing something different or like, do you? Oh, I don't disagree with that observation in general. And I think it's really important as a field that we allow that to take place that conversation where people are able to admit it's interesting that people write Nancy letters. But do they say that in a forum together with their colleagues with her? So I do think it's really important that we allow that conversation because I think that people can be, can lack confidence. And here's an interesting piece from our study on empathy. Hang with me. If you ask people to rate their confidence level at the beginning, they rate it higher than it actually is.

Now, maybe this is a semantic issue. I would label that confidence. Once you start to expose them to feedback, hold on to your seat. once you start to expose them to feedback about their deficits, their confidence level declines, but their empathic performance improves. Yeah. Yeah. So to me, that's critical. And it's not just about being humble in some general sense. It's humility helps you recognize the deficits and pursue more information.

Now, let me contrast that with what happens in continuing education. evidence base is really thin here that I'm citing, but it's the evidence that we actually have. Right now in the United States, psychiatrists, psychologists, social workers, we all are mandated to achieve a certain number of hours per year every other year. The assumption in that is, is that you learn something. And the data say, yes, you do, and it makes you more confident, which is probably the antithesis of what you need to, the state you need to be in actually, and to actually learned. So you go to a two-hour workshop on some new trauma-informed therapy, and you leave feeling, okay, now I know more about how to work with this. Bad idea, because our study found that when

therapists actually, they experienced themselves as they were less and less confident about their empathic, empathic responsiveness, their actual performance improved. You know, the feedback that I get, because we provide continued medical education with this podcast, right? And a lot of the feedback I get is exactly what you're saying. Like, as I listen to Dr. Cummings, he's a regular psychiatrist who comes on who humbles anyone. Like anyone who's a psychiatrist is going to be humbled by him. As I listen to War Dr. Cummings, I realize how much I don't know.

You know, or like as you dived deeper into that topic that I thought I knew a lot about, you know, I realized I have so much more to learn. And so I think, like, I don't know, like, is it like, like, good knowledge makes you more humble that you don't know very much. Like as I hang out with people like yourself, I'm like, there's a lot I have to learn about common factors. I didn't even know,

which I think I know a lot, right? But as I like, so isn't it true that any good education will lead to more humility? And that you should, you should be, okay, here it is. You should be questioning the charismatic leader who makes you believe that you have now, the silver bullet, right?

Or is that? Yeah, this is sort of, it's very difficult, I think, to capture this. There's a well-researched area on something called the illusion of explanatory depth. And so in this research, really very interesting, you ask somebody, how does a toilet work? Or how does a toaster work, for example? And before you ask that question, you say, how confident are you in your knowledge, about how a toaster works or a toilet works. People say 90%. Then you say, well, how does it work? And they will say things like, well, you push this little lever in the water and the waste goes down.

Yeah, that's not how a toilet works. You've now explained how you get it to flush. But what are the principles and people's knowledge is fall apart very, very quickly? So we all suffer from the illusion of explanatory depths, I would say staff meetings are the best example of the illusion of explanatory depth. We talk at this level that it really doesn't relate to what might help this particular client. The other problem is that clients aren't rats in cages receiving a stimulus for us. So we have to account for all of the chance variables that auger in favor of a positive outcome but have nothing to do with us. And that reinforces. It's one reason I say somewhat jokingly that my field, in particular psychology, has 2,000 different treatment models. And that's because no matter what you do, it works some of the time.

So we're on like this intermittent schedule of reinforcement. So first I have to establish a reliable baseline performance. Then I have to identify where the deficits are in that particular performance and get feedback to that. When that happens, that illusion of explanatory depth starts to become apparent. And then I might be open to new learning. Okay.

Am I getting at it at all here, David? Yeah, no. I think we're having a good conversation. I think we're, you know, what I've realized is that people enjoy a conversation. They enjoy back and forth. There's something about the brain that is built to learn from conversations.

And I think that's why people enjoy podcasts rather than just the straight lecture, you know, because they're naturally having a conversation with the speaker. They just can't have that conversation. And so I'm imagining when I have these conversations, the mind of not just myself, but the mind of maybe the average listener.

If we come back to this issue about confidence, I want to separate it from boastfulness or pride. I don't find therapists prideful or boasting about their abilities. And I do think there, for example, for me, I have said for most of my career that I'm a very anxious clinician that lacks confidence. That's the way I've been from the very beginning. And I think that might be a precursor to learning. But I'm talking about confidence at a different level. How would you rate your understanding or knowledge in this area?

Once we get down to brass tax, numbers, et cetera, then we have therapists saying, oh, yeah, I do this empathy thing pretty good. Yeah, I can resonate with that. And I can resonate with the experience of teaching residents. There needs to be an openness to continue to learn. And it's like, how do you allow that space of openness while not squelching their, ego so much that they can't learn, right? Because it's like if you induce too much shame, I'm sure this is on some inverted kind of you curve, you know, where there's an optimal amount of lack of confidence. I will say this. We can't see in our research any evidence that experience leads to better outcomes. And so if you're working primarily with residents and students, The question for me is always, what exactly are we training them to do? Because if I put them in a room with a client, they are frequently as or more effective than five-year post-PHD postdoc trained professionals.

And the quality, I think, that they may possess is, in fact, the very naivete and curiosity and lack of confidence that we're trying to inspire. They don't know, and that has them be open to experimentation. Yeah. Yeah, I think we're kind of talking about the idea of fixed first growth mindset, right? This idea that it's best to stay a student, to stay in wonder and awe, to stay curious. And without that curiosity, that wonder, that awe, you can't be empathic because you're just looking at things from your own reference point. so perhaps perhaps that's what we're talking about

with this kind of like wisdom where like the illusion of wisdom the illusion of education can sometimes overly move us from a place of curiosity and awe and wonder and not knowing into kind of like a rigid state sure and I think that that curiosity

is the door I don't think it's the path I think the path, if we're going to be metaphorical about this, the path really is identifying clinician-specific deficits. Now, that may be in empathy. It could be in how the session is structured. It could be in my ability to inspire hope. It could be in my ability to achieve consensus with my client around the goals, meaning, or purpose of the work. That's going to require some expert assessing my performance in relationship to my outcomes in those areas. So we've developed a tool that we call the TDPA, which is the taxonomy of deliberate practice

activities. And that really was the backbone of the book better results. So first, you would measure your outcomes. You would then be able to use that to establish a baseline. We present two very simple, straightforward tools that can be used on an ongoing basis, every client, every session, to measure outcome and engagement levels. We then look for deficits. It could be the type of client. Maybe it's the gender. Maybe it's the age. Maybe it's the problem presentation. Where do your outcomes fall short? Once we find a deficit, we want to look for the factor which has leverage on that deficit. And again, we've identified five common therapeutic factors, client factors, relationship factors, hope and expectancy factors, model structure, and technique. All of those are on this

document, this tool called the TDPA. You identify your deficit, you then begin to work through the TDPA, which helps you find the factor that has leverage and identify the specific activity that you might be able to do to improve your performance in leveraging that factor. Oh my gosh, I can't believe how long I just talked. Wait, is that you being critical of how long you talk there? Okay. No, no, it's helpful to hear you talk. And I'm glad you kind of brought us back on track to where you are currently.

in kind of you're thinking about things here. You know, the thing that I'm like, the thing that I'm thinking about as I hear this is, I'm thinking how much growth comes from a place of authentic transformation of the individual. And is it a skill that can be learned? So like, let's say you discover that this person is struggling with empathy, right? Well, maybe this person is like going through cancer in the family, right?

And it's like, is that internal psychological struggle bleeding out into, it's like they themselves need the healing that then would lead to the transformation? Like, is it like, I guess I'm struggling with this idea that there's some skills that we could like hone like a basketball player in a very sort of relational heartfelt field where it seems to me that me experiencing. or going venturing on my own journey of healing has allowed me to help other people or stay empathic or to stay within the frame. So I'm sure that what's going on in a professional basketball player's life outside the court can also have the same influence on their game.

The way to know that is to monitor your performance and then identify what we call non-random errors. So you have errors of responsiveness or random errors. Stuff just happens and there's no good reason for it. You have to separate those errors from the non-random errors, the things that recur in your performance over a period of time, which is why you need to establish a baseline that is reliable and valid in terms of that performance. So I tell people, for example,

if you're going to start using our outcome tools, you're going to have to use them for at least 60 cases. If a player, and that has to do with the psychometric properties of our particular scale, some scales that are longer, you can actually get a sense of their outcomes. You can get, you can predict their outcomes with far fewer cases. I'll give you an example. Once you have 60 cases using our tools, I can predict the outcome of your next case with 90% accuracy. Okay.

That's pretty amazing if you think about it. So it's the same in basketball. Everybody has a lucky shot. Everybody misses this game, what would have been the game winning point had they sunk that basket, but they don't. They're distracted by somebody in the crowd. They're thinking about their bending divorce. They had a bad fight with their child before they get, whatever. Can't learn anything from those things because they're random. How you can notice them is that they are at variance or at odds with your usual performance. And one of the things I'm going to ask as a coach,

when I see a general deficit in performance. What's going on? Yes, of course. Especially if that change is abrupt. So you're achieving certain effect size with all of your clients, and then suddenly you end up with problems that don't have another explanation. But generally, using the TDPA and using the measurement scales, these deficits are deeply embedded in the individual's performance and you can see them over and over again.

Do you see, okay, so I go back to this article, Super Shrink, right? Yeah. Which the best therapist was like 10 times better than the average therapist. Yeah. This was called Waiting for Super Shrink 2003. Yes. And I wonder, like, do you ever come across, like, when you start, when someone starts recording their data and they're 60 cases in and you look at their outcomes and you're like, okay, this person is so much better and I have no clue why? Do you have those cases? Or is it? Go ahead. The, that's what, that's where we were 20 years ago. That's exactly where we were. So the journey was specific model.

doesn't seem to account for common factors, teaching people doesn't improve backgrounds, measure outcomes, suddenly we discover these top performers that are consistently better. My initial assumption, I'm a scientist, my initial assumption was these top performers are going to be like stocks, one year up, the next year down.

They're not going to be reliable high flyers over the course of their career. So we followed them. when we publish the study indicating that, in fact, top performing therapists are not only more effective, but consistently more effective, regardless of who they see. That was an unusual, new and novel finding. And we had no explanation for it. And in that very article, O'Kishy's article that you're citing, there is a line in like the third or fourth to the last paragraph that says, how these people do it remains a mystery. Yeah, right. That's what caused me to lean in. There's that crevice. Well, yeah, what the heck did they do? Because my question was, could we learn anything? And one part of the variability in their performance, and there may be others, I'll share where my head's at with that, is that they engage in more deliberate practice, more effective deliberate practice. I think there's probably also a fair bit of,

of life, lived life experience amongst these top performers that gives them an edge over everybody else very early on in their careers. See, and that's, okay, so it's interesting to say that because I recently hired a therapist, and I tell this guy, and I believe it, like, he emanates as like a natural. You know, like, his empathy, his attunement, it's, It's like, it's incredible, right? And I can feel it. And I've seen the opposite where, like, I was at this non-therapy organization and they were hiring someone to do, like, lay work.

And I'm like, no, you cannot hire this person. It's the wrong person. I was dead right. Right? They hired the wrong person. This person was like... Psychonoxious. This guy blew up the organization a little bit.

Oh. So I think that they're... there definitely are people who you can get that feeling for. But I'm curious, because you've actually looked at the data, do you have any predictive capacity? Do you meet someone before you test them? And you're like, I bet this person is awesome. Well, not me.

But I think what you're doing is asking the right question. Somebody yesterday in a training I was doing said that people who had advanced degrees were, they were implying they had more knowledge, et cetera. And I said, I think what my advanced degree proves is that I'm better at taking tests than everybody else, because that's how you get from a bachelor's to a master's to a PhD or an MD. You have to be good at taking tests. You're not good at that. You're not getting in, frankly, but does that prove you're any smarter or better or more capable? So we have to have predictive validity. And the person who's done the research in this area is Timothy Anderson. And his research has gotten a fair bit of attention,

but not what it deserves, because what still gets the headlines in our field, what still causes the lights to flashes, I've invented a new model. Moving my fingers, tapping this unusual technique, I found 50 million correlations that if you do X, this Y happens reliably with clients. Yeah. You hear all that stuff. Wait, are you, I wanted to ask you about this. I'm sorry. Okay, go on, and then I want to ask you about your thoughts on EMDR and the efficacy comparing other treatments for trauma. So Timothy Anderson, our study on empathy, was in many ways modeled on his pioneering research. He would put, the question was, is what predicted better outcomes from a therapist? So what he does is he puts them in a room, and he administers a test

that's called the Facilitative Interpersonal Skills Test. And you are presented with difficult interpersonal circumstances. I've read this article. The video, right? And then you have to reply to the video. And then that was the one thing that was linked to therapist effect compared to all the other things. The other things were not, yeah, I love that study. So like general social skills doesn't predict.

But your performance in this novel, difficult. Now, here's the interesting part. He gives this to beginning-level graduate students first year. And then they go through four years of training, where they have practicums and classes and supervision. Turns out the best predictor of therapist outcomes at the end of graduate school was how well they did on the facilitative interpersonal skills task before they started school. So clearly, these folks have an advantage in dealing interpersonal skills. with difficult challenging circumstances. And the question is, of course, how do they get that?

Is it genetic? Eh, maybe. What I probably think is they have been rehearsing and practicing this on their own. But I have no evidence of that. My thought on that, and I'd be curious to your thought, and I want to, is that they often, like, because I've done, because of my job right now,

I sort of attract therapists and psychiatrists to my practice, you know, because of the podcast and such. So it's like what I've found as a commonality is that they often play a role early on in their family as the peacekeeper as the person that is co-regulating their family dynamic. So they've been playing therapist for decades, just like the best leaders are the leaders on the playground, right? Somehow they found a way when they're four or five, six years old to like literally start games and lead they end up being the best leaders right these kids are often shoved into a dynamic with their parents you know there's contention between them maybe they have a depressive genetic parent and they're playing the therapist they they're regulating so i don't know if

you've seen that as well or if there's any research on that you could probably say it better than me yeah undoubtedly not. I don't know about that. And my interest has really been in training the current crop of clinicians regardless. What I will say is, in addition to the GRE or the exam to get into medical school, why aren't we administering, if they're going into psychiatry, psychology, or social work, why are administering the FIS? And selecting in part based on their interpersonal abilities. Right now, and by the way, when I interact with grad students, I have to tell you, unbelievably smart, scary, smart. Is it smart in the ways that you need to be a psychotherapist? Not always. Not always, yeah. No, I would say informally, our program, like, because I've interviewed a lot of

medical students, we look for people with high EQ on how they respond to the questions. And we also weed out people who are very, are testing very high, but don't have relational capacity. Like when they're, you know, we're watching them at dinner. We're watching them when they're not thinking that we're watching them, right? And it's the little, it's the little moments. How do they treat our, you know. are people serving the food at the dinner the our our coordinator who you know this is a you know 30 an hour job she has veto power we give her that right if she doesn't like someone if they've been rude to her forget it we're not we're not hiring them right so we are looking for EQ we're not but

but this is a it's interesting when you talk about this this you know the video interactions and they're looking for the response to these difficult interpersonal situations. It sounds like you've recreated that in your most recent study with empathy. Because essentially you are training how to respond to difficult interpersonal situations, which is really exciting, actually. Yeah, so Tim Anderson's really truly groundbreaking work. Everybody should read it and not just the original 2015 articles, but those that have followed. his was an assessment and tying it to outcome.

Could you say that these two things were related, outcome and performance on the FIS? The thing that Daryl added and our crew added to the difficult conversations and therapy project was a training component. So now, and because Tim wasn't talking about deliberate practice, he is now, but we were saying, okay, now that we've identified that you've got a weakness in this particular area on one of the, the FIS domains, what could we do to train you to improve in that particular? How could we apply deliberate practice principles to that? That's great. Yeah, I'm ready to do it. And it's such a small area. It's a really tiny small piece of this huge puzzle. It's a meaningful piece, though. And I think it's the first, it sounds really exciting. Now, is this something that you're offering, like, can anyone jump on a website and do this at this point? Or is this just research?

This was just research at the present time, but we do have a deliberate practice intensive and asynchronous course that my colleague, Daryl Chow and co-author Daryl Chow, have pulled together. You can find that on the Better Resultsbook.com website, a link to that asynchronous course. They can really start to fill in the details. The Better Results book and the field guide are neither of these are books that are designed to be read from cover to cover in one sitting or in a week. They really require reading a chapter, putting something into action. So, for example, I mentioned earlier on our conversation that 60 cases are what you need in order to get a reliable, using our tools, a reliable estimate and

valid estimate of your baseline. Well, we describe how to use the measures in the first two chapters. What are you supposed to do then? Put the book aside and wait. Gather your data. And so we're talking about a long-term commitment with deliberate practice. The only study we have of real-world impact of deliberate practice is one that's out of the Calgary Counseling Center in Calgary, Canada. And there, what's interesting about it is through a combination of measurement, feedback, and deliver practice, we could actually see the natural improvement that takes place at the individual clinician level. And it does not go like this. It does not, it's not perpendicular, it's not orthogonal, right, to the ground. It's instead slow, steady improvement over time. And I think this explains why deliberate

practice is popular as an idea, but not as a practice. It's why so few adopted over a long period of time because it requires a tremendous amount of effort for very slow return. Now, in the long run, you're at the Olympic level. But in the short run, all you are is tired. Now, do you have any data to show like how much of, like, how much people have shifted if they've done this for 10 years? I mean, I imagine not many people have done it for 10 years at this point. So that is exactly what the Calgary Counseling Center study looked at. And the article title is called Creating a Climate of Therapist Improvement, I think. Creating a Climate for Change is the title.

of the article, Simon Goldberg was the lead author, and you can actually see the graphic that shows it's very small improvements over time. We're talking about, you know, a tenth of a standard deviation from perhaps year to year. So we're talking very small improvements. And what's the last name of the first off? Goldberg, Simon Goldberg is the lead author on that particular study, and it's called creating a climate for, cleaning a climate for therapist change, I think is what the name of the article is.

Okay, let's see, here we go. It was the 2000, right? Yes, yeah, for therapist improvement. Yep, that's it. Okay, and so. And if you scroll down, there's going to be a scatter plot. So the D is 0.034.

Yeah. So that's, that's, small. And what does that mean? What is that over time or how much time? Yeah, that's the slope of change. Those who did and those who didn't. So, but scroll down a little further and you'll see the scatterplot.

Keep going. This will be on the YouTube if you guys want to check it out. Okay. And if you look at that dashed line. Yeah, that's a very, okay. You're seeing the average overall improvement. And it's a slow incline. And that's because, remember, most therapists are starting off average in terms of their effect, effectiveness.

And the effort it takes to move beyond that is much greater than the effort involved in achieving average results. So we're literally talking about a Olympic level performance, and that's just putting in the effort over a long period of time. And so in this, I'm just trying to wrap my head around like what actually was done. Like what actually, so you're measuring outcomes. Every session with every client. Every session with every client. And you're measuring like different common factor questions, right, are in some of those things. Like how connected did you feel?

Yes. Right. You're measuring improvement. in functioning, individual, relational, and social functioning. That's on the outcome side at the client level. And then you're also measuring engagement. So qualities of the relationship, empathic understanding, agreement on goals and tasks, and whether or not the therapist, what the therapist was doing fit with the client's preferences. And you're feeding that information back in real time to the therapist so that they can make adjustments and improve their responsiveness. Remember, there are two

types of errors, errors of responsiveness or random errors. We're human beings and we're meeting new clients every day, sometimes two and three times a day. We're not always going to create a perfect fit. That does not mean there's something wrong with you. It just means that sometimes we're off a little bit. So there's nothing to learn there, except in the moment I might be able to adjust. That's what the feedback does. However, over time, I'm able to start to see a pattern of recurring errors, maybe an empathic, less empathic responding with a certain type of client or a certain time of day that I meet my clients. Who knows what it will be? In order to find those, I'm going to have to take a deep dive into your data. So is it okay to give an example? Sure. I love an example.

Here's what a recent therapist, Michael Harloff, Canadian therapist, and you can see the interviews that I've done with him on my website at scottymiller.com. He calls me, he wants to engage in deliberate practice. He develops a spreadsheet. He monitors his outcomes. We get 60 cases. And he starts to notice that he has slightly lower effect sizes with men. And so now you think like a therapist, what do you think? Maybe I need to learn how to have better relationship with men. We didn't stop there. Instead, we did a big and thorough review of his case notes,

and we find out it's not all men. It just is men that are angry. The word anger shows up in his notes more frequently, in the treatment of men with lower effect sizes. So now, here's Michael and Mike's question. Should you go to a workshop on empathy and relationship building with angry men? How do you deal with an angry man? As we look more closely at the notes, we find out that the men are not just angry in general.

They're angry about something quite specific. And that is, they are asking for direct counsel and advice. And the response of the therapist is empathy. And from the client's perspective, evasiveness. This is the therapist who the client says, would you just tell me what you think I should do? And we say, I understand you really want me to tell you, but it's not my life.

Let's figure out I can help you figure. Blah, blah, blah. This select group of men wanted direct advice. Now the question is, where does that fall in terms of factors which have leverage. It's not a relationship issue. It is a structure issue and technique issue that requires that we begin to work on Michael being more directed, feeling comfortable with that, knowing when to give direct counsel and advice, and the type of direct counsel and advice to give. You know what, supervision I would say to that as well.

if just thinking out thinking off the top of my head I would be curious because the anger feels directed at him you know is it transference and is it is it anger

in the transference towards him that is attachment related from past people in the men's lives you know so is it beyond like that they want advice right

Maybe it's their way of trying to process anger towards other attachment figures in the past. I don't know. So this is a diagnostic framework that you're proposing, and it would aim mostly at relationship factors in terms of its leverage. The question is, would those interpretations improve the client's engagement? Would they feel more understood, more heard, et cetera, from a. from a deliberate practice perspective and thinking about the factors that have leverage, I take that theoretically driven concept of transference and I say, well, where does it map into these five factors?

What we did was we looked at what he was doing in the room. He did a lot of reflection. We reviewed all of his case notes. And what it turns out is that it didn't seem to be an empathic failure. Okay. So much as it was a structural failure, he did not feel comfortable, given we can direct advice. He didn't really even know how to do it. Everything in his training and said, you'd never do that.

Okay. And so when he made that change, did his outcomes begin to climb? Okay. You begin to improve, rather. So with men, specifically, his outcomes improved. Men who were looking for direct counsel and advice. Okay. Now, we might think that some expert from the outside would be able to determine that easily. But I, I don't think it's so simple. It first required measurement, establishing a baseline, identifying where the shortcoming was. And even once we did that, we're looking for what was responsible for it. Was it a relationship issue? Was it, in fact, Michael, not working well with men who had a poor attachment history. Was that the issue or was it something else? And the more the reflection and then filling out the TDPAs, the tool, the clear

it became that this really was a specific technique or skill deficit. Okay. Yeah, I think it's great that you're able to help him make that shift. That sounds exciting, you know, that he was able to benefit from that level of analysis and then connect with those men better, right? Yeah, and we're talking about, because Michael is a very talented clinician, we're talking about a small subset of his overall clinical sample that this particular recurring error occurred. And so if you looked at his overall effect sizes, you're going to see just a gentle upward move.

So one thought that I had as I'm listening to you is like, how do you choose the people to teach the empathy or to teach the ideal, right? Like, are these people that you that you are having teach it, are they themselves proven to be at much higher levels in their effect size or in their effectiveness? Yeah. I love the logic that's involved in that because a good coach doesn't necessarily need to be good at the activity they're coaching. Okay. It's an assumption, right? But many of the best coaches in sports aren't actually players of the game. Their particular skill is getting others to excel given their strengths and their particular weaknesses.

Right. But when we're talking about empathy, for example, how did you come to those two experts that reviewed the most expert level of empathy? What we did for empathy and what the entire book, the field guide, was all about, was fleshing out that particular picture. Because in the first book, Better Results, we introduced the TDPA, we gave some examples. But what we recognized is that we were going to have to be able to have people coach others or coach themselves in what to do in response to feedback that they had a deficit. in one of the areas that have leverage on outcome. So we reached out to, again, I'm really surprised, to the experts in the field. So, for example, the person that was the lead author on the Relationship chapter is John Norcross. He spent the last, well, 30, 35 years.

He's written multiple books, conducted multiple studies, led the task force on relationships for APA. and we said, first, would you please review the evidence? We know relationship is a factor, one of the big five that has leverage on outcome in psychological care. Would you please first review all the research? The second thing we said is, and this is now we're moving into where we're at in our own development, we'd like you then to distill from that research principles, not techniques, but principles. and I'll give you one example in a minute. And then with those principles,

we would like you to offer a series of exercises showing how people can apply those principles should the examination of their own performance indicate that they have a deficit in that particular factor. So John Norcross wrote that. Helene Neeson Lee wrote the chapter. So your answer is to look for world experts in a given topic by what they've what they have published not only what they published but most of the

people well all of the people that we asked to write chapters were not just publishing research of their own but they were expert at meta-analysis and systematic reviews of the literature okay so i guess i guess i'm still like like how do you like would you take someone like that and would you have them do 60 cases to, but you're saying it doesn't matter how good their effect size is in their own therapy clients, right? It probably matters at some level. Again, it's probably on one of these usually. I mean, if they have no ability whatsoever, it may be it affects, we don't know. I don't know the answer to that question, to be honest. But I think this idea that you have to be one to do one is, I don't think that that's supported by the evidence. Or like, even with your like self, like, have you seen a steady increase?

I imagine you've been watching yourself. Like, irrespective of where you started, have you been able to slowly incrementally improve from doing these things that you talk about? In my own performance as a clinician? Yeah. So, yes. Okay.

But I wouldn't use that as a good example. I would say that it's far better to find out. And the area where I think I've spent much more time in deliberate practice is actually as a presenter. So I engage in regular coaching on an ongoing basis with two different performance coaches when I'm up in front of an audience. Oh, that's. And they're constantly, and by the way, neither of these people are people who are on the stage. their production and direction kind of folks. Okay.

And perhaps maybe that's a good analogy. What you're needing is not another actor to teach you acting. What you need is a good director and producer who can get you to do your best behavior. And the way to do that is to have you identify the principles that are evidence-based, that are associated with better leverage of the factor that you're trying to improve upon. Right. So you have been studying yourself on your ability to communicate, which you're obviously a very good communicator, very effective communicator.

And it's been a good conversation to have you communicate what you've learned, because I'm sure a lot of us have not had your journey. and your analytical skills applied to your curiosity on how to make things better. So it's, yeah, it's obvious that you have done that work. And so, yeah, I appreciate that. But you're saying more than maybe therapy for you, you've approached the discipline practice for presenting. Yes. And you've seen a lot of improvement in that domain.

That's where I focused most of my effort in the last 15 years. That's my public persona. That's my public face, as opposed to doing therapy on an ongoing basis. What we can say is that if I can identify the area in your performance, the factor, then these researchers, they have. identified and distilled principles from that, which then we teach you, we provide those principles as feedback, and then these coaches coach you to enact that principle in better, more effective ways. Okay. That's good. So that EMDR question still lingers in my mind. Like someone is listening to

this and they're like they wanted you to to put a bow on that one so if someone says like okay i'm i'm struggling with trauma maybe i just need to learn EMDR what would you say to them how do you know you're struggling with trauma well i've been doing this this this thing and it shows that my trauma cases the effect size is not as large as you've been measuring your results yes they've let's say they've been actually measuring their results uh so The particular strategy or technique you use is one of five different factors which have leverage. Model structure and technique is one of those five factors. It also, in general, is the weakest contributor to outcome and psychological care.

So just because... What would you say to someone, though, who says, no, but EMDR is different, it's unique. Yeah, definitely that. like do you agree with that or not agree with that i mean from the data trauma is unique no or emDR specifically like the the type of modality EMDR i'm not i'd say absolutely not i don't see i don't see evidence that it's better more superior quicker i i just don't see that okay so you're saying modality is one small component and there's four others yeah and so you're going to often it's the first thing people reach for when they think they have a deficit

in performance. And it's what the field coughs up constantly. I have problems with clients who have generalizing dis-a-disciitis. We have a drug for that. We have a model for that. And that could be it, but I'm going to guess just based on averages that it's probably not that. Okay. And it's probably a relationship issue. It's probably creation of hope and expectancy, or it's a therapist factor that's getting in the way. which is again one of the five. Okay. So you would start to,

so rather than point them towards a new modality, just one more modality to learn, you would kind of like start to try to break down or try to assess like which of these four, or which of these things are impacting the most? I'd want to see which one has leverage on outcome. Therapists typically reach, and by the way,

if a therapist came to me and said, I've been monitoring my outcomes, And I found out that when I categorize my clients in terms of low effect sizes, it's the trauma cases. I'm going to go, wow, amazing. You're so far ahead of everybody. Because people who approach me typically are not measuring their results in any reliable or consistent fashion. They have no idea about how effective they are. But what they will say is I did get my first level training in X model and I think I'm more effective.

And I say, well, how do you know that? Well, you know, this client said they were, it's all end of one kind of work. So if they've managed to measure their outcomes, wow, bravo, that is no small task. That is a huge shift for most therapists to adopt a routine outcome measure and begin using it with their clients. The assumption that learning a model is going to make us more effective, you know, I'm going to listen to it, but I'm going to say, well, let's look actually. Let's do a more thorough analysis here and see which of the factors are having leverage. And it could be that there are two. maybe you do have structuring problems.

But then the question is, is it a structuring problem related to the rationale and how you describe your work with people? Is it in the execution of it? Is it how you start or end your session? There's so many aspects and flavors to this. And I guess, again, and by the way, I think that EMDR and other approaches

do a really good job at helping therapists get a sense that they are becoming more effective. Most of these workshops have small practice exercise where they say, well, think of a distressing moment and then let's do some of this stuff. And you feel better and wow, it's very convincing. People say EMDR has revolutionized my practice. And it doesn't matter if it's EMDR. It could be act.

Act is also really popular. It's revolutionized my practice. How do you know? Really. Did you measure? Did you see a difference in your outcome? come you know i had i did um i had this a super fan of act who you know we did this interview stephen hayes it was a great interview loved meeting him but then after the interview i went back

and i was trying to find any study that showed act was better than cbt or an act of control and um on my website i i put every study i could find and and there was CBT was better for depression, slightly better or of exactly the same. There were a couple things like maybe chemical dependency or anxiety, but there wasn't huge effect sizes. And it was like complicated anxiety, maybe a little bit better for act. But by and large, I was underwhelmed. And so I'm totally with you.

I'm aligning with this kind of idea of it may not be modality. Or recently I looked at for borderline per size disorder. I was looking at, you know, schema focus. therapy, mentalization folks, mentalization therapy, transfer focus therapy, dialectical behavioral therapy, and all of them across the studies are very equivalent. And so I'm like, why do we continue to fund these horse race studies? I don't get it, frankly, except that we are imprisoned by a certain way of thinking about our services, and that is the mental and emotional concerns people bring to us, can

be solved with the psychological equivalent of a pill. And I just, I just, I don't see it. And for me, at least, where I see most of the deficits in clinical performance, first off, I want to say, we're talking right at the edge of performance for this last hour and a half. Clinicians do good work, period. There's no doubt about that. But when it comes to professional development, we fool ourselves into thinking that we're

better than we actually are and that we're growing when in fact we're not. And if I had to guess where most people could use some work, it's not going to be in terms of the model and technique that they use. Yeah. You know, I've been measuring the OQ45 because of some of these early studies. And I think I need some further discussions to hear about or to figure out like how easy it would be to integrate your stuff. But I've been using the OQ45. And I do it at the very beginning. Good. And I usually only do it again when there's a point of like, are we moving forward, right?

There's a question of that. And the other thing that I've been measuring is the Big Five personality type because I did a series on that. And then I measure it again in certain clients. If I want to see how the personality is shifted and I've seen neuroticism, like one and a half standard deviations change with intensive work, which is really cool because of that sort of internalized self-critic, which neurotic system picks up so well in the measure changes. But I'd be curious, like your thought, OQ45 versus your tool, obviously yours is a lot easier to measure session to session, but the OQ45 was used in a lot of these studies that measured

session to session changes. Yeah. So thanks for asking this question. I wrote an article that's probably been eight or nine years ago called, I think it's called Beyond Measures and Monitoring. And in there I said at some point our field, if history is any guide, we'll be fighting about which is the right measure to use. And most of the outcome, and I say it's a pointless argument because almost all of these

tools differ only in length, but not in the, the factor, not in the factors that they assess. So most outcome tools are single factor measures, most. Ours is a single factor measure. Michael Lambert wrote letters for me to get into grad school. I worked as a research assistant for him for several years as an undergrad. So, and we started with the OQ45. The reason we transitioned to the ORS, and the ORS, the history of it is it measures the domains assessed supposedly by the OQ45. So three domains, individual relational and social. We just removed all the individual questions and just had people rate in those in those domains. And the reason was, is we were working in a clinic setting where many people had

difficulty reading. And so we had spent with maybe 20% of our clients time reading the measure to them. and getting the results, that just didn't work for us. So I think of the ORS as a stethoscope. Interesting. And, you know, as a physician, you know, listening to heart sounds and lung sounds, that is a skill. That is a big skill. And that means that we're using a very simple measurement tool that can give big results,

but it depends on the skill level of the clinician in interpreting it. You have the OQ45, it's much lengthier. We know from test construction, it's going to be less variable as a result. But the tool gives us, as a physician, just like physicians in a busy medical practice, you can put the stethoscope on the patient most times doesn't need more an MRI. They don't need it. This is sufficient. Yep.

So if you're using the OQ, more power to you. If you're running up against difficulty in terms of time, reading level, cultural differences, the ORS is a reasonable measure, and it is designed to be used every visit because modal number of sessions, as we said in the beginning of the conversation is one. So the likelihood of them dropping out is early on. Secondly, huge amounts of change if it's going to occur, tend to occur early. So I need to be sensitive to whether or not the client is experiencing progress earlier than session 5, 6, 7. Third, therapist's assessment of progress have a low correlation with the client's experience of progress. So I don't want the treater being the person judging whether or not I'm going to measure now. That's just the logic of it.

Great. Hey, we got to wrap it up. This has been an excellent conversation. I feel like what would be cool is as I dig in more to your new book and the chapters and the sort of subchapters, I think we should almost like revisit some of these like individual sections and really chew on them. I would love that. I really appreciate the opportunity to have your listeners hear about these ideas. I hope I have communicated things clearly. It's a very exciting area of research.

Yeah. And you're at the, it's like my curiosity is increased through this conversation. I hope my listeners have had the same experience. And I've pointed people towards your website and towards your book. And I will continue to do that because I think we are, you know, we do things slightly different. but I do believe that you are sort of approaching things with a perspective of someone who's looking for answers, willing to be wrong and then shift directions and willing to sort of continue to be curious and look for what the data actually says rather than just like what you're sort of, what maybe,

not yours, but someone else's internal narcissism would say, like, this is, this is the, thing, right? So I appreciate it. I hope you see that as a call. I do. I see it as a as a big compliment and as you were speaking I was thinking I have I think been wrong more often than right or I would say that our understanding was was very limited but thankfully we've had great colleagues, great clients that have pushed us to move beyond that. Awesome. All right, Dr. Miller, until we meet again, thank you so much for your time. I appreciate it.