Psychiatry & Psychotherapy Podcast - Getting Better Results from Your Patients as a Psychotherapist

Episode Date: March 19, 2020

On this week's episode of the Psychiatry and Psychotherapy podcast, I interview Scott D. Miller, Ph.D. and Daryl Chow, Ph.D., authors (along with Mark A. Hubble, Ph.D.) of Better Results. Better Results is a book that sums up thirty years of research to demonstrate what clinicians can reliably do to improve therapy results by personal and professional development.  Link to Blog.

Hello and welcome to the Psychiatry and Psychotherapy Podcast. I'm here to talk about getting rid of burnout, increasing job satisfaction, and feeling like an expert in what you do. One thing that created a lot of burnout and angst for me was trying to get continued medical education right at the last minute. So why not join the CME membership and do CMEE while listening to this podcast. Go to Psychiatrypodcast.com, sign up, sign in, take the test, and the certification is email to you in seconds. Hi, welcome back to the podcast. This is your host. I am here with Scott. Bob Miller and Darrell Chow, both PhDs. They have written a really interesting book that I became aware of. It's called Better Results.

It is a book about how to get better results. I would say in every domain of life, like if you were an expert musician, I think this book would help you be a better musician, but specifically in psychotherapy. And that's really what their life work has been. That's what they've been publishing on for years. And this is a book that brings it

down. I don't think it waters it out by any means any of the hardcore research, but it really does make it accessible to a larger audience. So this is really exciting. Thanks for coming on, guys. Thanks for having us. Yeah, my pleasure. So I wanted to start maybe by having you guys give the elevator pitch for this book. Like if you were to sum it up, some up your research, sum up what your passion is. Scott Miller, you want to start? An elevator view of better results is the summation of, I would say, almost three decades of research looking for what clinicians could do that would reliably lead to professional development. And our field, sadly, despite lots of frenetic activity and moving our feet, have not managed to

cross much distance in terms of better results. Our data indicate that the outcomes of psychological treatments have not improved in over 40 years. That's a pretty stunning finding. And it means we have to take a different way, a different path. And this book offers that path. Yeah, tell me a little bit more about that. Like, how do we really know that psychotherapy has not improved in more than 40 years? You know, going all the way back to the 1950s, there was some research that was published at that particular time that questioned whether or not psychotherapy actually worked. In fact, the data seemed to say that people in psychotherapy as compared to other helping approaches fared worse. It really spurred the field into action. The researchers picked up the task of trying to provide evidence that.

that psychotherapy was actually effective. By 1976, hundreds of studies had already been conducted and the claim that psychotherapy didn't work really was overturned. So we have very solid outcome data known as effect sizes. How big of an effect do we achieve that date all the way back to the 1970s, a meta-analysis by Mary Smith and Eugene Glass that estimated how effective we were. if you compare those effect sizes to the effect sizes, despite our, the explosion in the number of treatment approaches, the effect sizes are equivalent. And I'll just say that we're not the only one saying this Uber researchers, James Purchask and John Norcross in a article that was published in the American psychologist, the November issue,

finally have admitted this as well in public, that our outcomes simply aren't improving. Again, we have lots of movement, lots of activity, but we see. to have gone largely in circles and our outcomes just have remained flat. Yeah, but yet our therapies have increased exponentially. Like I think every week I'm hearing about some new psychotherapy that's being launched. And so one of the, one of my hopes from this episode is really to get to some of the core features and some of other episodes I talk about therapist effect and kind of like, you know, what are the sort of common features of,

features of good therapy, you know? And so I think it would be really interesting to get into that a little bit from your perspective, as you have in the book. Let me, let me ask one more thing. How do you define effect size? And are there certain types of therapies that have been shown to be more helpful for certain things like borderline per size order or like, you know, like I'm thinking, like is that sometimes when you go to a meta-analysis are you losing the nuance in some of those bigger studies so you know let me jump in on on this one and scott feel free to fill in as well but i think first and foremost when we talk about effects size really what we are looking for is the magnitude of change from beginning and all the way to the end of treatment so we would

have clients and therapists use measures that capture the outcomes from session by session hopefully, if not, some studies would do more pre-posts. And then we compare that magnitude of change to see what level of impact that has on the client's well-being. So you get studies that look at symptom-specific measures, and you get studies that look at more global well-being type of function. Now, then the next thing that leads to that is, what about studies that say that this particular treatment is better for, this type of psychiatry problems. So you have a classic situation like the DBT approach for borderline personality disorder. That's where you kind of need to get into the weeds or the methodology and comparison.

First, we have to ask, are we truly putting a bona fide comparison between A and B? So case in point, if we look at some studies, let's say in depression, when they pin one particular treatment model compared to another. So usually the comparison is what they would call supportive therapy, right? And in that mode of supportive therapy, you have to see what exactly they are not getting compared to the people in the clinical trials that get support, supervision, and sort of very tightly held guidance from their supervisors in their work. Another case important in trauma comparisons when they look at treatments for trauma-focused work,

the supportive therapy, they are sort of told not to touch or not to talk about stuff related to the trauma and to focus more on the here and now type of issues. So all of us know in the field of mental health that you, you know, the moment you try and clan down on what a client might like to go into, that's sort of impeding any kind of therapeutic end of defa, isn't it? Yeah. So I just jump in and add a bit that because I think this is the question that people have approached professional improvement through. What method do I need to learn in order to be more effective?

And that begs the question or assumes that it's the method that makes a difference. And while the method does contribute to outcome, it's, it's a very, very small contribution that it makes. And other factors are far more important. So to come back to your question, are there differences between models? Well, one study says a little bit. Another study says no. When you summarize all of these, any differences are very, very small.

Or they wash out completely. And then there's the next question, which is, let's put that method in the hands of clinician and there's no guarantee that any small difference in a tightly controlled randomized trial, as Daryl's pointing out for those variety of reasons, is going to make a difference at the individual therapist level because they may have skill deficits that get in the way of those differences. And that's why deliberate practice is essential. We have to find what it is about you and your performance that holds outcomes back and address that specifically. Yeah. Yeah, and that'll be kind of like the next thing we go into. So, but before we get there, like with effect size, I just want to point out again, and I've pointed out of this before, it's like how many standard deviations you're moving from the control, right? So it's like, an effect size of one is like you're moving one standard deviation from where the control is at the end of the study. So it's like how what your control is really changes the outcome as well.

Absolutely. And I think it's going to be important to, to, to, to, think about effect size as like, okay, because especially when we start talking about how individual providers vary on their effect size of how they're able to move their clients, right? Which I think is something that you guys talk about. And it's an important way of measuring how effective a provider is. Yeah. Yeah. And may I jump into add one thing about this point you're raising, David? I think it's key that if we, you know, if we look at this. with perspective, sort of zoom out for a second and see, methodologies and approaches account for about 0 to 1% of outcome.

And then if you compare that, as you know, David, about therapist's effects, who the treatment provider is accounts for something like 5 to 9% of the treatment. And then if you think about what I'll feel is doing, we're always pointing to the various types of approaches and metaphor, right? where the variance lie more in the person. Yep, 100%. I have this conversation. So, you know, I'm a psychiatrist.

I teach residents the psychotherapy portion at the University of Loma Linda University where I work. And I'm always trying to get them back to like, okay, yeah, you can do that modality because that's what you're passionate about, you know, like if they're into ACT or they're into CBT. But let's work on, you know, your therapeutic alliance, your empathy, you know, how you deal with interpersonal conflict in the relationship. And I keep coming back to those things.

Brilliant. Yeah, brilliant. And you're talking here about core fundamental skills. And I usually tell the people that I work with and supervise that if you're having a difficulty, it's probably not because of the absence of technical knowledge. It's probably because there's something amiss in the connection between you and the person you're working with and you're not managing that. We've got to find out what that is.

Yeah. I'll also go one step further to make a case that I think the traditional mode of our pedagogy and training people in the mental health field is if we kind of look at it, there are really three fundamental types of knowledge. I think the first one is content knowledge. You need to know the landscape that you're stepping into the types of problems that we face in the human condition, very necessary. The second piece is the process knowledge,

which is how to then translate that into conversational nature of reality formation with the people that you're working with. And that, as Scott is pointing out, it's also a different type of skill to step into the interpersonal domain. Then you have a third condition, which is conditional knowledge.

If you see a person, with, let's say, depression. The work that you do will be really different, even though the symptom presentation is similar, which is low mood, from a person who's experiencing grief, bereavement, compared to somebody who may be experiencing domestic violence at home, and compared with somebody with a very strong perfectionistic trait

that's leading to depressivegenic moods. So these three types of knowledge, I would argue that in our education, we focus a lot on the content knowledge and we fail to figure out a way of intersecting these three domains into the therapy room. Does that make sense? Yeah. Yeah. And I think it comes back to how does change really occur in therapy. And I'd be curious what you guys, how you would respond to like if someone said, let's say some beginning, you know, therapist said, okay, but how does change occur? Yeah.

Hell if I know. Come on. Tell me what you think. I think the question is an interesting one and it's an intriguing one. And what we end up with is loads and loads of theories. And the answer is people find many different ways. You can emote your way to better mental health. You can explain your way and think your way to better mental health.

You can act your way to better mental health. You can do nothing and life moves you to better mental health. So the key here is to not get locked. into one way of viewing that. And a great deal of the wisdom can actually come from the client. So following up on what Darrell said, in this emphasis on content knowledge, my experience is I've never met a therapist who cannot explain why a case isn't making progress. We're full of explanations.

But what ends up happening is they're locked in terms of their process knowledge. What do I do from moment to moment with this client? and what are the conditions under which I do something different? And we have to first be alerted that we need to do something different. And that means we're going to have to likely measure more closely what we're doing. We have to know under which circumstances we do something different than we've been doing and how we've been explaining the case to ourselves. That's tough.

What we did 20 years ago was develop a set of very simple measurement tools that clinicians could that would alert them to changing conditions. And now, 20 years later, some really intriguing work done by a fellow colleague named Jeb Brown found that increasing therapist responsiveness. In other words, being alerted to the conditions that I need to do something different can actually accounts, has an effect size in and of itself

between 0.2 and 0.4. it's a big deal to become more responsive when the work isn't working. The temptation, and I think only because of our training, not because we're motivated in some way to avoid it, is we can explain it. So we can map the client's prison in exquisite detail, but knowing how to help them escape is much more challenging. Yeah, and these questions really look at,

it's a rating scale, but you have the patient do it in front of you. It's asking interpersonal questions in nature. A lot of them are. And questions like, I felt heard and understood and respected. You know, we worked on and talked about what I wanted to work on and talk about. So, you know, goals. The third question is the therapist approach is a good fit for me. you know so does the patient at the end of the session think that your approach the way you're doing

thinks is a good fit for you and overall today's session was right for me so I like the first one's like empathy empathy and uh therapeutic alliance and the third question are really like looking at goals and fit you know like and so here's what I took away from your your lecture I heard from you years ago if they score just a little bit to the left it doesn't need to be very far at all just a little bit, then it's like, huh, help me understand what would move it just a little bit to the right, you know, like, because there's a little bit of a disconnect here. And because often with these high trait agreeableness patients, they're not going to put it all the way to the bad side, you know, they're going to put it just a little bit, but not perfect, right? And, and it, and the thing

I took away from you, and this was, gosh, this is, this was six years ago. So that was a good, that was a good point was it's like just a little shift. And that was really helpful to me. Tell me a little bit more about that. Well, we're looking for an increased attentiveness and responsiveness, especially when progress is not forthcoming. All relationships have variability, moments of boredom versus moments of excitement, moments of engagement versus less engagement.

But the key is to tie that engagement to progress. And that's the second tool. The second tool is the outcome rating tool. And whether you use our tools or someone else's tools, it makes no difference to us. What we're looking for is some standardized formal way to see is what we're doing together, making a difference in your life outside of therapy. And if not, how can we get closer to each other? How can I become more responsive? So the initial purpose of the scales, and this is just the first part of it, is to improve that responsiveness, as opposed to explain it to ourselves.

oh, my clients not getting better because they're unmotivated. And I tell therapists, it's not the client's job to be motivated. That's your job to be interesting to them, to be provocative. We're shifting the burden back of managing the alliance back to the therapist's shoulders. You know, I heard a, I just did an episode on the unconscious. And one of the things that came out was that people often attribute other people's action and behavior to their personality where we attribute our own action and behavior to the preceding events that led up to our action behavior. And that's kind of a good example of it

of like, no, this person is just unmotivated because they're an unmotivated person. You know? So I always find it interesting when like yesterday there was one of the residents wanted to move a patient to an every other week session from a weekly session to like a every 14 days. And my inquiry was what is motivating you to want to do this. There's some counter-transference reaction that's like pushing you, wanting to push, you want to push this patient away potentially. You know, is it, are you giving up that they're going to make the change?

Are you accepting that this is their certain level of functioning, you know? Are you co-opting this person's, just desire to stay stuck stuck um that you would probably jump on that right what do you think of there which part the last line something i said the uh the the the the the the phroidian slip you mean go ahead well instead of stuck you said suck you know and my my guess you know how can a psychologist not jump on that shit you know that's that's an amazing that's an amazing statement really oh man. Yeah.

Here's what, I think, for what Daryl and I would say is that I have no way of knowing what to say to a supervisee until I see their results. So increasingly when people come to me and they say, you know, I have client X with problem Y and I've been doing Z. What do you think? I'm saying, well, what do the data say? And I think, you know, in terms of the clinical supervision domain, which is like the main pedagogy in our field for psychotherapist, you know, we again fall into becoming esplanaholics. We explain things, you know, and we have interpretations, speculations, theorizing, hypothesizing, and all these are but a shot in the dark at best.

when you look at even the clinical supervision literature, you know, and there's been a replication of that recently as well, they found that the impact of clinical supervision, even though we feel the benefit, the impact of outcomes is close to 0 to 1%. Again, like methodology. It's just so small in terms of impact. And the real question is, how can this be?

How is it that we feel we reap this benefit? from the care of our guidance and our supervisor, but yet the translation is so minuscule. Yeah, so you have, and there's probably multiple reasons for it. Again, we have a strong evidence base about this. It's very clear, like Darrell said, that we value this. I think it's number two on the list of items that therapists say influences their professional development. And yet when you look at the data, it doesn't turn into anything. Well, why would that be? Why would that be? Number one, therapist view and reports about the alliance and outcome have a low correlation historically with the client's view. So if the client can't tell they're connected or getting better, who cares what the therapist says? Secondly, then that therapist goes and chooses cases to talk about, not based on their data, but based on their interpersonal experience, which is already flawed, as just indicated, and talks to a third party.

who has no idea about the case. So essentially you have two people talking to each other about a third party, and the two know nothing about the person. My grandmother would have called that gossip. We call it supervision. You know, I, so this is where I do some research. I specifically look at the supervision

and the connection in the supervision, which hasn't been done a ton. There's a little bit of research on it. But one of the things that I found was that if you have a supervisor who is experienced by the resident as very empathic, psychologically safe, they also have good feedback. You can't be rated in high empathy and low in feedback. And the therapeutic alliance will be high as well.

So all of them kind of flow together in the factor analysis. And what we found is that it's really that high level of connection in the supervision, that led to decreased burnout, increased personal accomplishment in the resident. And so I wonder when we're looking at some of the supervision research that you're talking about, we're seeing kind of like, we're seeing a large linear line when we talk about like, are these things correlated? But is there another way to model this? Because what I found is that there are some supervisors that because they're highly connected, that gives value to that person.

You know, like if you have a less burned out provider, that, I think, gives a large value to patient care. We know that, you know, burnout and patient care are linked in a number of ways. Depression, you know, if you're depressed as a provider, that's linked to patient care in a number of ways as well. So I'm wondering if that's part of the issue because I'm looking at your scale and I'm thinking, what if we, so I kind of created a scale for, or the interpersonal between the supervisor and the resident in a similar way that you've done it with the patient and the provider. You know, maybe that's what we should be looking at more is one of my thoughts that actually probably came from your lecture as well. So you influenced me, Scott Miller, and six years ago before I started this. Thanks.

So I have no doubt, and I think there's good evidence. It's not great evidence, but it's good evidence. And most of that criticism of the literature is due to the absence of studies of things that are daily occurrences in clinical practice and training. It's astonishing that there are fewer than three randomized trials on supervision. That's pretty astonishing, given that every single state in the United States requires two to three thousand hours of this activity. but it's based on air, really. So we know that, and I believe that the data is pretty good that therapists value the relationship, they consider it central to their professional development,

and it probably leads to less burnout. But as we all learn in our stats course, the best predictor of an eventual outcome is something directly related, not indirectly related to effectiveness. So while the alliance is necessary, we also need to be having discussions with clinicians as supervisors based on evidence that the case they're talking about isn't getting better or who has alliance problems. So that's essential. Otherwise, you know, we're always going indirectly.

So let me give you an example of a study. Intriguing study found that therapists who rated their supervisors as interpersonally attractive, had supervisors who in turn rated them as effective. But neither are related to outcome. You know, but think about that as like, this is that interpersonal piece. You know, if you experience empathy and connection with this person, you're going to rate them highly. You're going to rate their feedback highly.

Absolutely. And when you're praised, your supervisor in turn rates you as effective. but the key here, the twist, is neither of those things are related to the outcome. Now, that doesn't mean that the activity called supervision isn't valuable. Or I think I'd like to get away from this thing about supervision. It has, you know, it's got baggage that dates all the way back to Freud and probably call this consultation or coaching. At least that's what we call it in the better results book. There has to be a more direct focus on the effectiveness.

and discussion of cases that are provably ineffective, if we leave that up to the clinicians whose view of progress and the alliance is poorly correlated with those two variables, then we're in a hopeless situation. And then we start searching for indirect variables that might improve outcome, like decreasing burnout. And if I could jump in about that,

I think your point about having a sense of safety, connectedness in the supervision context. I think that's very important, David. I think Edward Watkins raised this question in one of his paper. He's been quite a leader in this feel of the literature and supervision. And he says, he asked the question, what is supervision for? You know, what is it for? I think, you know, to ask that fundamental question and he sort of answers it himself, isn't it really just on trying to figure a way to impact positive outcomes? for our clients. And I think that's the directiveness that Scott's speaking about.

And we are just trying to bridge that, trying to figure a way to connect this two, three steps removal, talking about somebody, and trying to bring that more closely into the room of the discussion. And I think there are two sterephonic views that, you know,

in some vision people need to take. One is to coach for performance, which is to deal with stuck cases where the therapist is having, trouble with. But that's been, that's actually more of the norm of what we do. We argue that actually there's another view that you need to look at, which is coaching for development, which is not just focusing on a case by case, but focusing on a therapist's growth edge, focusing on where they're at, or rather figuring out where they're at based on their reliable aggregated outcomes,

and then figure out what to work on before the how. Yeah. We may have gotten a, head of ourselves as well. I really want to touch on what is deliberate practice. And I think this is for me this dates back to when I read talent is overrated years ago. And they talked about how through practice, through having good coaching, through focused attention, you can improve your ability to do things. And it's not just genetic or, you know, miraculous. So you want to Tell me a little bit about what is deliberate practice and how that applies to psychotherapy. So first off, let me just, let me just give a plug for Jeff Colvin's book, Talent is Overrated, came out in 2009. And an article that he had written and that was published in Fortune magazine

that I stumbled across on a flight home from Europe really helped us pull our thinking together because we had this anomalous finding, and here's what it was. No treatment models didn't seem to make a huge difference in terms of outcome. And yet, when we began measuring individual therapist effectiveness, some consistently rose to the top. And like everyone else, figuring out how they did it was the grand prize. How did they manage to do this? We watched their videotapes. We couldn't see any difference in what these top performers and how much more effective were they? about a half to a full standard deviation more effective. So we're talking about them over their lifetime of services, helping many thousands more people than if they were seeing a more average therapist. So it was a big mystery. It didn't seem to be explained by models. It didn't seem to be explained by so-called mastery of the Rosarian core conditions.

Then I stumble across this article. And Jeff Colvin is describing why he's describing actually the work of Anders Erickson, a Swedish psychologist, who's been looking at why some performers across a wide variety of domains of human performance achieve better results. And he was also intrigued by the fact that most of us work in our particular jobs for a lifetime. We spend more time at the job than with our families. And we don't get any better over the course of our careers. In fact, we generally deteriorate in our performance. And deliberate practice, the work of Erickson was, the explanation. And he later on to went on to write in great detail in applied in a more business context, talent is overrated, where he explained this process. And you've really given a superb definition. And that is focused concentration and attention at the edge of your current level of performance. We're going to focus on where your individual performance breaks down. And we're going to train to that. Yeah, dear child, do you have anything to add to this?

No, I think, you know, the point about where all this came from and Scott stumbling into the book about this, I think that was like quite a key turning point that unfolded so much of where we were scoping our lens, where we were moving forward to and led us close to a decade later to do this now. It's been a long time coming. Wow. that's fascinating that all three of us read that book back then and how kind of an influence is because I read that when I was starting my residency and I was thinking through like, okay, how do I become a better therapist? How do I break these aspects into smaller pieces?

How do I, you know, break emotion into the smallest pieces? So I like studied microexpression. I studied like Paul Ekman's work and learned how to do that. And then, you know, different aspects of the interpersonal. How do I think about that or break that down? And for me, it was watching a lot of video with one of my key mentors, Dr. Tar. And I think actually video supervision in one study was better or the best type of supervision. Coming back to the supervision, because one of the aspects of deliberate practice is a good coach, right? but I think that you guys monitoring as well through these different inventories is so important

in that sort of giving giving like this is where I am at right now this is with this patient and giving you that like feedback you know because like in in the book they talk about you know getting that real feedback the honest feedback moment to moment which is so hard in therapy because it's not like golf where it's like you hit the ball and the ball doesn't go in the hole. It's like sometimes you're not even aware where the hole is. Yeah. And David, I think you're hitting on a really important point.

I think two things. I mean, the first is using the golf versus psychotherapy analogy. The golf is what we would call a kind environment. You hit, you put up. You could see the ball go in and you know the outcome of that. The feedback is almost instantaneous and you calibrate from there. Whereas the practice of psychotherapy, therapy is more of a wicked environment.

You know, you do something and then the feedback that you get, you see the impact on the client's outcome happens probably when you see the person the following week. Or if you don't see the person, you end up becoming an explainaholic and you have various ways to kind of talk about that. That's one. Second, I think the deliberate practice thing that you said about, you know, teasing it apart, I think the deconstruction is so critical on how to. break it to its component parts and to examine really what are the pieces that one can work on?

Because what David, you might need to work on, what I need to work on, what Scott needs to work on, will be different. And this leads us to then to think about this deliberate practice idea. We sort of use a framework. We have like a four-tenant framework about how we conceptualize deliberate practice. one is to be able to identify your individual's growth edge, the key thing that you need to work on that has leverage on impacting outcomes. And this is really interesting because if you left that to therapists alone to figure that out by themselves, most of the time they're picking up stuff that is very treatment model specific, that has very low yield on actually impacting outcomes.

Like, let's just say, working on how to do the empty chair work on or how to get to the core beliefs, which, you know, it depends on where the therapist growth is at, but these are very small yield on actually impacting outcomes. So individualized learning objective is one. The second is figuring out a way to get feedback one client at a time, which is a performance type of feedback, but also learning feedback about what you're doing. is it really impacting on your outcomes? Third piece is the successive refinement to be able to reiterate, if you've learned something, put that in and see if there are ways that you can tweak

and improve on what you're actually doing at that time. And then finally, of course, as you stated, the central piece is to have somebody to guide you, a coach, somebody to guide you to kind of push you gently to your growth edge in a safe learning environment. Okay. And can I add this last piece that Darrell mentions. To me, this is the chief distinguishing characteristic because when we think about, oh, geez, you know, I'd like to improve my effectiveness

with this type of client. What we do is reach for off-the-shelf solutions. And this also happens in golf. You may remember back in Tiger Woods heyday that there was a fist fight between his caddy, I think it was, and a person who was hiding in the bushes trying to film Tiger Woods' swing, his new swing. He's reinvented his swing several times. And what would they do with those videos? Well, then they sell them. And then all of the wannabe great golfers buy these videos and try to copy Tigerwood Swing, which is organic and specific to him and his developmental pathway. So what we're talking about is a coach that can facilitate your development, given your genes, your brain chemistry, your physical makeup, the way you experience the world, which is going to be

unique to you, bounded by the outcome. We'll know if your organic development is good if, in fact, we see an improvement on the measures. So the coaching here that Daryl's talking about, I think, is quite different. We're not going to coach you to learn short-term dynamic psychotherapy. We're not going to coach you to learn this is the way you do empathy. You know, that you end up with a bunch of people saying, what I hear you saying is, or so what you're telling me is, They learn stock phrases. If we teach them and coach them differently, then we can help them develop an organic style specific to them

that leads to the experience of empathy on their clients' part. Okay. Okay, I think I'm following you guys. So we're talking about how to improve. And there was this one nice graph that you put in here. on the effect size affecting outcomes in psychotherapy. And you talk about how powerful the different aspects are in promoting good outcomes. And alliance, empathy, goal consensus are on that list.

Did you know this before you started your research? Or did you, was this kind of like after you started the research? I think this graph that you're citing is actually from Bruce Wampel and Zach. Mill in this book that I highly recommend people to read, The Great Psychotherapy Debate Edition 2. And what's interesting in the graph, if the listeners get a chance to look at it, is that the thickness of the bar, right?

So the thickness of the bar represents the amount of studies is being done. So the thickness of the bar for things regarding treatment model specific stuff is really thick, right? Yep, yep, yep. But then the effect size for that, really short. And if you compare that to the other extreme about goal consensus, effective focus, developing a kind of directionality of treatment, that bar graph is tall and skinny.

And that is one area that in the context of helping somebody improve their work, especially when somebody's stuck, there's a high chance that that's one pit stop you want to make to kind of check. is there a consensus of where the client and therapists are going? Is it explicated? Are there, is it on the table? Is that a kind of shit vision, if you wish? And if I can add here, because I think what Daryl says is so important, you asked us,

did we know this beforehand? And the answer is, we had a gazillion facts and we couldn't figure out how the heck to organize them usefully. That's our field, a gazillion facts with no organizing thrust. We don't have a, so we have all these therapists who are doing great results. We can't explain it. We trip into the deliberate practice literature. We, at that time, had been measuring outcomes.

That's how we knew that there were some therapists that were better than another. Well, then it occurs to us we can use that data to identify particular therapist deficits. That's all ex post facto kind of reasoning. Oh, my God, look what we can do with this. Then once we find that there are therapists that have specific deficits in particular areas, the next question is, what should we train them on? Yes, we know that they have poor empathy skills, but we have to figure out what has leverage on the outcome.

And this is what we've been hinting about through this entire interview. Well, if you're going to learn something, well, I wouldn't probably reach. for a new model as the first thing because its leverage on outcome is very small. At the same time, if we find out that your therapy, based on your results and our careful analysis of the cases, lacks structure and focus, then maybe what you need to be trained on is a particular model. And hue to that model, not because it's the most effective model, but because that's what you need to improve your outcomes. And there are four or five other factors that have less leverage on outcome that the field has known about for decades, but has remained mired in this

confusing battle between the so-called common and specific factors. Yeah. And you know, one, one, somebody I was consulting with recently, he said something really wise. He said, I now realize that the models are less for the clients and actually more for me. There's a way to kind of have a sort of scaffold structure to help them think. through. Yeah. I think I think the models are to reduce our own internal judgments of our clients and to our sort of countertransference and to have the patients and the unconditional positive regard because I think when if if we have our own judgments, then that kind of leads to the patient

will experience that in some small way and that will be a thwarting in the therapeutic alliance. That's one of the thoughts that I've had, I don't know if you've thought any similar things, or maybe it's okay if you have any pushback on that. I don't have a comment. It's not language that I use is all I could say. I do think, interestingly enough, David, that the models are there to give structure and focus to the work. It's like learning scale. It's like learning scale, in music. It's not music, but it does give you an understanding and appreciation about musicality to understand the scales. And I think it's the same in therapy. It gives you an understand of the therapiness. But one is as good as another. And the real question is,

did it structure the work in a way that engaged your client and led to progress? Yeah. One of the things you talk about is the, it's, it's, So we talked about therapist effect, how many standard deviations you move from the mean from beginning to end of therapy. But you also talk about the relative effect size of the practicing clinicians and how one clinician can be like a 0.3 standard deviation worse overall in all their effectiveness, you know, before and after with their clients or they can be above 0.3 or even one standard deviation. Have you ever seen people who started low and then were able to improve their score? Yeah, and that's what causes hope in what we're doing, isn't it, Scott? And I think to see this on an individual level and on teams and the agency level, that's where it's edifying.

That's where the payoff is when you see people, you know, most of them are pretty good. But you could see that when they channel their passion and their interests and find a direct way to translate this, we see people actually improve in the outcomes from their reliable improvement, reducing the deterioration rates, engagement increases, and consequently impacting outcomes as well. That's what we are seeing. And if you start with what Darrell said, which is most of us are, most of the results are actually pretty good. average clinicians have an effect size, at least in the studies we have, that's roughly equivalent to the effect sizes seen in randomized clinical trial. So effect sizes of about 0.8 are what you get for the last 50 years in psychotherapy outcome studies. We now have over a million cases in our database, well over a million, and the average effect size of clinicians working in very diverse

settings with diverse clients and comorbidity as the rule rather than the exception is a about 0.76. So, and when you compare them head to head, you know, there's, there's not much of a difference between them. But the question is, what would take them forward? Because two interesting findings, studied by Goldberg and colleagues from 2015, I think it is. You can correct me, Darrell, if I'm wrong here. When we, when we followed therapists over time and measured their results, in contrast to what the therapist believed, their outcomes actually slowly deteriorated over a five to 17-year period. So therapists, you have them on the one hand, believing they're getting better and confident that they're doing good work, but their outcomes actually declined. The only study in the history of the field to show improvement at the individual therapist level was a study that included ongoing feedback and deliberate practice, came out again by Goldberg in 2016.

and but the amount of change and i think this is the this is the challenging part the amount of change was small and the effort it took to get it was greater than the effort it took originally to become average yeah i mean that's what any performer at a high level has to do right that's right i i guess i guess i'm coming back to like okay i want teach me like what are you guys doing that is changing an individual's effectiveness to do psychotherapy. Like, okay, so you want the, I want to move from being average to one standard deviation from the mean. You know, how do I do that?

So, okay, so I think, you know, Scott jump in, but I would say this. Here, the distinction is important to be made at the outset, which is here we are talking about better results, right? And, you know, I come, I live, I live in Australia now, but I come. I come from Singapore. In Singapore, we're all about performance. You know, we're all about being number one in every single damn thing. But here's the thing.

An over-emphasis in performance can impede deep learning. Hallelujah. Hallelujah. And, you know, that's the paradox we've got to try to manage because we're not trying to measure competency. We're trying to measure growth. We're trying to help people angle in a way, that orients them to this infinite game as opposed to just, am I doing good, am I doing bad?

Because if we look at it on a client-by-client level, it's up and down, right? So I think first and foremost, if we can distinct the learning framework and as well as a performing framework, they'll be great. And then when we start, you know, one of the biggest difficulty that you would see for the majority of therapists is figuring out where they're at, figuring out where they're at on a reliable basis, using, as what Scott was saying, using some systematic measure consistently in a routine basis, because measurement precedes professional development, trying to figure out using measurement as a language, not just get hopped on the stats, but using that as a way to tell the story

of where you're at. Because if you're trying to tell your friend how to come to your house pre-Go Maps age, you need to know where they're coming from, right? And I think that is such an underrated place to work through. And to be able to do that, I would say clinicians need to start to build enough cases where they have enough numbers to start to analyze about where they're at. And then the second piece after they figure where they're at is to figure out what to work on before the how. Because traditionally, our feel is really obsessed with the hows.

Here's how you do this. Here's how you solve this problem for this kind of psychiatric problems. figuring out what means that we need to look at it on an eidomatic level, on an individual level, about where this person is at, what the results are saying, what the data is saying, what intrusively they're experiencing as a clinician as well, marry this together, and then discuss about what are some micro areas that they can work on. So Scott and I created this taxonomy of deliberate practice activities that you can find in the book where it sort of takes you through the whole therapy hour

and stuff that you can work on. And then after that, again, rating that by yourself, but also getting a coach that knows your work to go through that and having this conversion discussion about what is the thing to put your eyes on right now to focus that has leverage on improving outcomes. And then third is to be able to develop some kind of learning system in their work so that they can deliberately hone in and refine

on stuff that they're working on. And let me just let me just say, let's just contrast that with our field's typical way of doing this. So here's a conversation I have three times a week. The supervisor will say to me, I need to learn some more about how to do better trauma treatment. And I say, really, why? And they say, well, because you know, I have lots of trauma cases recently. I say, well, how effective are you already? Blank face.

I say, well, how do you know you need to learn anything about trauma per se until you know how effective you are with the people who present with that anyway? And I would say most of the time when we start looking at their data and separating that trauma versus no trauma, they're already just as effective. So the juice doesn't lie there, so to speak. And I'm not just anti-trauma or anti-learning a technique. It's not about that. It's about investing where there's an opportunity to learn something and where what you learn has leverage on the outcomes. And as Darrell says, we've got to stay away from the performance area. I want to improve my outcomes because that's not going to get us anywhere. Mostly it leads to shame and hiding our results. What we need to get in is to into the learning zone. What can I learn here? Yeah, I have to be honest, that was my, one of my responses to reading this book was like, oh gosh, I don't know if I'd want you guys to see, you know, like, how would I measure up? You know, because I've never been like compared to other therapists other than not in a data-driven way, you know.

So it's like, my first response was like, oh my gosh, like, okay, what would it be like to sit down with you after you look at, you know, of my cases, you know, what would that look like? And yeah, shame was one of those sort of preemptive thoughts. I don't know if you have it. That's probably a normal occurrence for the work that you guys do, right? I think, David, that's one thing that your research about creating a safe holding environment is so critical because you can almost feel the visceral vulnerability involved in talking about that, not just on a client by client level, but you're looking at this on a real, aggregated but personalized level. And we spend a fair bit of time in the book

sort of separating the performance from the learning zone and supervisors and coaches, and I know this personally, because what each of us have done, each of the authors of the book have done while we were writing and doing research was pick up activities and try to apply these principles in areas outside of psychotherapy.

And a single word from a coach or a trainer or a supervisor can send you quickly into the performance performance zone where what you want to do is look good rather than learn good. Yes. Okay. How do you, it's like the therapist experience of like, okay, I'm revealing this thing to you that I haven't revealed. And then they don't experience shame and how healing that is. So I'm thinking maybe that that's a common experience for you guys as well when you do this supervision. It's like people are expecting like, oh, I'm point three standard deviations below where the average is, you know, I'm expecting critique or criticism, you know, what, what is that,

is that what people are expecting when they, when they see these scores? I do think that it's, it's not uncommon. And we tried to be very careful about this in the book, starting the book by saying, chances are you're doing work comparable to the outcomes in randomized trials. So we spend a lot of time saying you're probably already pretty good. The chances you're going to find that you're one of the stinkers in the sample of thousands of therapists is pretty small because the distribution is so tight in terms of therapist effects, which is kind of uninteresting. The fact that all of the top, the difference between the runners of the 100 meter at the Olympic level, the differences in time are really thin. But being able to cross the finish line first, that's the whole point to improve that running piece. And a lot of these runners don't just have a coach or supervisor teaching them a method for running. Instead, they have a strength coach. They have a person who attends to their equipment, the equipment coach.

They have all of these various people looking at aspects of the performance for where there might be an area that has leverage on their event. performance. Right. So it's like they're trying to get down to this is the one area if you were to make an improvement, you would see a bigger effect, right? Kind of like what you're talking about, like looking at that one, or looking at where in the deliberate practice literature, it's like you're looking at that's the edge of your ability. Right. So how are you assessing the edge? of your ability. Darrell, you know what I think would be a cool idea here is to talk about the list of activities that you looked at that therapist engaged in with their deliberate

practice time and what you found in the original study. Do you know what I'm referring to here? The rapid practice survey? Yeah, yeah. So I guess initially we were speculative and, you know, we, we were looking at the impact of deliberate practice in performance and therapies. So one thing that came out, of course, from that study was that deliberate practice, you know, the amount of time spent devoted to working at the growth edge contributed to

the predictive value of where a therapist is in terms of their performance. And mind you, this was based on five years' worth of outcome from a pool of 69 therapists in the UK that we were looking at their results. So that was not a surprise of sorts. Where we were trying to look at was we were trying to zoom in on really specific stuff. So we had like, I think, 25 items, like how much time they spent in reviewing the recordings, how much time they spent in doing other stuff and supervision and whatnot. Reading a magazine.

Reading a magazine, reading a journal, reading research papers. Turns out that all these things was not predictive on an average level. What was predictive? Anything? Well, it was the time that they spent, you know, the amount of time they spent working at the growth edge. You know, your question is, you know, the way you responded to that is reminded me when we first presented this result in a conference and answered them. Some guy in New Zealand stood up before we could give the punchline. He said, oh, come on there.

Okay, can just tell us what to work on? Yeah. Yeah. Yeah. You know, but that's the, that's the, you know, we're not trying to be embevolveillance about that, but that's the edge because everyone has got a unique piece to figure that out. And the combination here, two things. The vulnerable feelings that you said, David, you met with when you read the book and the feeling that Daryl just highlighted from the participant, and this is something we hear all the time, just tell us what to do. those two things make therapists ripe for the picking.

Anyone who comes up with a box or off-the-shelf product, here's what you need to do. And therapists will buy it. Why? Because they want to get better. They want to get better. And secondly, they know, they know that when they go in the room, there's a lot of shit that happens that they didn't figure out beforehand

and that they're not sure what to do about. The activity is fraught with ambiguity. It's fraught with ambiguity. and that stuff's not going away. And that's why, like, this is such an important conversation to have because there are people out there who will get swayed into fads and, like, you know, I met with this one guy who was really into light therapy and, you know, flashing these different lights or this tapping or, you know, this or that.

And I even have people come to my clinic after trying some EMDR and they'll say things like, you know, but the therapist just didn't listen to me. They weren't, they didn't hear my story. They didn't hear my lament. They, the alliance wasn't there, right? And so I'm thinking after, you know, every single patient I have, you know, as a psychiatrist, I asked them like, what were your past therapy experiences like? What were the positive aspects, the negative aspects? And they'll tell me, you know, the negative aspects were they didn't feel connected.

They didn't feel heard and understood. And so I'm just thinking like therapists and people, right, are often swayed into a charismatic leaders, you know, confident assertions that their particular way is the correct way, right? And I'm afraid that, you know, often it's not what's actually going to be helpful. You know, analogy I could draw about this is my other world that I'm interested in as well as in music. It's so much easier to get tools than to get good at what you're doing. So instead of working at my craft at songwriting, I look at gear. I try and see what I could get to.

Maybe if I get this one, this would improve my songwriting. Maybe this synth, maybe this Les Paul, maybe this, you know, but then that's a, I can, guess that's a thing that we get drawn to. The harder thing is to have this difficult conversation within yourself to go, wait a second, hang on. What do I need to work on if I want to be a better songwriter? And building on that, the whole deliberate practice movement is really about making you a better version of you. That's right. A more effective version of you. not a miniature version of your EMDR or tapping person. And not that any of those things can't be helpful to people.

Of course they can. But the problem is, so can talking to empty chairs and having people lay on a couch. I mean, all of these things can structure a helpful interaction. But for you to improve, it requires working in a different way than pulling a solution off the shelf. Yeah. Okay, so we're coming to the end of our time. Yeah. Was there anything that was not mentioned that you feel like you just want to put out there?

Deliber practice is a marathon, not a sprint. And it does take time. But I truly believe, and my experience, and one of the reasons I stay in our field, is because therapists have good hearts, and they really do want to help. And the promise of deliberate practice, in comparison to pulling the latest thing off the shelf is that your results will gradually and steadily improve over time as a result of your hard work. And over that lengthy period of time, many more clients will be helped by you. I think for me, I'm reminded of a writer and artist, Austin Cleon, he would say, most of us want to

be the noun and not do the verb. I think I worry that the idea of deliberate practice, come to capital D and capital P. And people will start to see this, this is like a new method or even working at improving specific methods using deliberate practice. But I think the challenge is how to translate this into real individualized action, getting themselves with the support that you need from the community or from individual coaches, because this, this, you know, should not be an individual sport. Okay.

Yeah, I want to thank you guys for coming on. I think this is going to be one of my favorite books to recommend. I really do value the time with you guys. I feel like I have a lot of questions still. I feel like I need you guys to coach me a little bit so I can get, so I could figure out, you know, where are my, where are the edges that I need to work on. But yeah, this has been valuable.

And I think the book is coming out, what, in May? Yeah. May. Okay. So I will write the, this up, link your tools on my website, link the book on my website. So that'll be all linked on the show notes. Great. And I'm looking forward to having you guys out again. So if anyone has any questions that they want me to ask, Scott Miller or Daryl Chow, maybe you can send me a message

and I'll keep a list of them. This is going to, David, this is going to sound like a shameless plug. But if you pre-order the book, you can pre-order the book on Amazon and APA's website right now and send the receipt to us, recognizing that ongoing support is critical to deliberate practice. Daryl and I are organizing a series of supportive resources and such. So all you have to do is pre-order and then send us the receipt and we'll connect you to this. We'll connect you to these resources. Okay, do you have a, you want to get, you want to get thousands of emails? You know, we'd love to, we'd love to do that because, and I wouldn't be surprised if that happens.

Again, it underscores my belief that therapists really do want to do good work and they want to do better work than they did yesterday. This isn't about being the best therapist compared to others. That's the performance mindset. It's about learning something today that will help your clients tomorrow. Well, and I think, I think I would put it as every, people want, to get better. What do you focus on to get better?

Yep. Right. And I think that's one of my big takeaways is we want to get better. We want to improve. We want to be masterful for various reasons. But I think it's like getting some data to show you where you're at, things that you can improve. And then focusing in on some areas. and then not sort of becoming victim to a lot of the placebos out there

or a lot of the sort of charismatic new approaches, but being mindful that there are common things that we know are very effective therapeutic alliance, empathy, do we have similar goals, those types of things. And we should focus on those things first and foremost, like how do we improve the, those abilities. Yeah.

Any other thoughts? Just a big thanks from us, David, for this opportunity to speak about deliberate practice. As you can tell, we're excited about the potential. And we think we've really just managed to describe the tip of the iceberg here. Really, there's, yeah, I totally agree. There's a lot, a lot more. And a lot of fun things. You guys talk a lot about, like, music.

and Mozart and all of the sort of basketball and, you know, all this aspect of deliberate practice for those things as well, which was, it's actually really fun to read. Cool. Thanks. So I enjoyed it. It's very readable. All right. So we'll leave it there. Thanks, David. Thank you.