Psychiatry & Psychotherapy Podcast - Connection Index

Episode Date: June 2, 2022

In this episode of the podcast, I will be discussing something near and dear to my heart—a tool I created to measure the connection between physician/student and teacher/medical learner in medical education. It is a tool called the Connection Index and its purpose is to improve the quality of the medical education experience. I wanted to answer the question of how we create better supervisors and mentors as students embark on their own "hero's journey" to becoming physicians. By listening to this episode, you can earn 0.5 Psychiatry CME Credits. Link to blog.  

Hello and welcome to the Psychiatry and Psychotherapy Podcast. I'm here to talk about getting rid of burnout, increasing job satisfaction, and feeling like an expert in what you do. One thing that created a lot of burnout and angst for me was trying to get continued medical education right at the last minute. So why not join the CME membership and do CME while listening to this podcast. Go to Psychiatrypodcast.com, sign up, sign in, take the test, and the certification is emailed to you in seconds. All right, welcome back to the podcast. Today, I will be presenting. I have no conflicts of interest. I am going to be discussing the Connection Index. It is an index that I made with a team of medical students and residents

and a statistician and a colleague. I was a first author on a paper that came out this last year. I'll link that in the show notes, if any of you want to check it out. And it looked at a new measure that we created called the Connection Index. This is something I, it's kind of like a, it's like, I think we all have that project, which is our highest close to our heart project that, you know, basically also carries the most resistance. This is the third time I'm recording this. It is not going to be perfect. I'm going to just lay perfection at the table right now. what I mean by resistance is that there's this book called The Art of War and then there's this book called The War of Art. And in the book of The War of Art, it talks about how when you are doing something that's close to your core, your true person, there's going to be resistance.

And I believe that because this is, you know, the most difficult paper. I think we were on somewhere above 100 versions of the paper. We had to submit it to a couple different journals. We had to rewrite it multiple times. And I'm indebted to my mentor, Dr. Cashner. I'm very thankful for the tireless effort of a group of medical students. So that being said, what is this about? this is about measuring the connection that goes on in medical education.

And why is that important? Because I believe that will improve medical education. You know, if you're not in medical education, you may not know what this is like. But if you are, this may feel like empathy when I say it. It's really hard because you have to play a role in medical education. You have to kind of put your head down. you have to show up whenever you're told to show up, leave whenever you're told to leave, you miss meals, you miss family connection times, you know, you're working 80 hours a week

often, and you just want to get through it. And, you know, the environment really does change your level of burnout. So I used to give these lectures. to medical students third years on the mental status examination podcast number one. And I would always have them raise their hand if their most stressful event was a patient encounter. And I would get maybe two people out of a group of 60 that would raise their hand. And then I would say, how many of you the most stressful thing was some interpersonal interaction with a supervisor, either a resident or an attending, and everyone would raise their hand. So I would raise my hand for that.

So it's really hard, though, to give feedback up the chain, right? So power structures, it's like, it's really hard to get accurate feedback. And this goes back to, like, our tribal nature. It's just hard to critique the chief. And, you know, you may be like, well, but Twitter, you know, people tweet all the time at the president or, you know, these people. And it's very different than giving someone feedback face to face, which is how medicine is usually operated. And especially someone who potentially could make your ability to graduate more difficult. So there was motivation from me to take the things that I had learned from psychotherapy, therapeutic alliance,

connection and then look at what has been actually studied in medical education. And there was no measure to look at empathy between a supervisor and a student. There were lots of measures to look at empathy between a patient and a doctor. But none in training. There were measures to look at psychological safety. You know, how safe do you feel giving interpersonal feedback to your supervisor, but they were often, there were all group questions. So it was like on this team, how safe do you feel? So it was very team-based. It wasn't individual-based, where a lot of medicine happens on an individual interpersonal basis. And if you want to get true data on a person and how they're influencing people one way or the other, then you have to go on the individual

to get individual data points on this particular attending has this level of psychological safety, right? And then there's other things like feedback, how feedback is given. Well, there's lots of surveys looking at it, but not really like in a cohesive way. It's hard to explain. It's like you get a lot of,

there are definitely a lot of survey-based feedback things, but I wanted to do something that was a little bit more that created data that was valid, right? And then Therapeutic Alliance, we all know Therapeutic Alliance helps, you know, the patient, doctor interaction. And I was wondering, what about the therapeutic alliance between a supervise and supervisor? So there's a term in the education literature. It's a pretty new term called Education Alliance. So, you know, As you're going on this journey, let's call it the hero's journey as a student, you will come across a threshold. It's like this journey into this is more than you can handle. And it's really that

a helper and mentors that help you overcome the challenges. And so I kind of saw this as like, how do we create better mentors and helpers? How do we therefore aid people who are on this? on this journey, this hero's journey. Okay. Well, we all know the villains of medicine. Time demands, pressures, busyness, EMR, prior authorizations, work going home, lack of control over time management, death of patients.

And, you know, for example, in EMR, there was one study, Zelensky, 2016, they found that for every hour doing patient care, two hours were spent on EMR. that was in different ambulatory settings. We also know that the hero faces their own internal psychological battles. You know, higher degrees of burnout, higher degrees of depression. I could put all the studies here that show that, I'm just not sure I need to because everyone kind of understands that.

depression increases as medical school progresses. And in one study, they found that it wasn't the patient encounters, you know, that near-death experiences that patients had that increased PTSD. It was actually bad role models that increased depression in medical students. I have looked at tons of things. studies showing the increase in suicidal thoughts that happen, especially the first three months of residency. Obviously, you're in a new place, often new friends or no friends, less community, the most demands, the least knowledge, right? So, you know, we know that there are risk factors for

depression, working more than 70 hours per week being one of them, and we know that that's common in our training to work more than 70 hours a week. So, you know, there's depression, there's suicidality, there's burnout, these are pretty common in residence. Actually, I found that in general, I think we're a pretty resilient group that go into medicine. There's less suicides than the general population in residence, but as at least, Later in life, physicians actually have more suicides than the general population. So, you know, why might that be?

Well, I think it's chronic stress. So it's not just the medical students and the residents that have it tough psychologically or mentally. It's everyone. I always tell medical students, if you're attending is having a hard time and being irritable, just consider that they might have a lot of stuff going on in their life. not that it justifies them being cruel by any means or. But I think it's helpful to kind of consider, you know, they might be going through divorce in a child custody battle,

just feeling financial tension, stress from all of life, right? So we know that suicide rate is higher and older physicians. And so burnout and depression is not just unique within trainees, but actually later in life as well. We also know that burnout interventions are not best if they target the person, the physician, the trainee. The effect size in one very nice study,

a meta-analysis, was 0.18 at reducing burnout. With mindfulness, CBT, personal coping, education on communication skills, It's very, very low, very low. Whereas if they made organization-directed interventions, the effect size was more than double. 0.45 at reducing burnout.

This was decreasing the busyness of the environment, increasing teamwork, enhancing job control. And so one of my thoughts for trainees was you don't target the trainees, you target the environment. You want to get information about the environment to know how to target it. One part of the environment is the attendings. And so the Connection Index looks at that part of the environment, although there may be multiple parts that play into the mental health.

And having looked at a couple, you know, through this work, a couple suicides that took place actually by residents, it's always more complicated, right? It's so complicated. It's never one thing. It's not like you can put your finger on like, oh, it was just this bullying by this attending. No, that's not what I was seeing when I was looking at these suicides. But we can have good attendings that decrease burnout, and that's what I'm going to get to

and what my paper has shown. So just to let you know, burnout has three domains, emotional exhaustion, depersonalization, and personal accomplishment. This is the mass law burnout inventory. And when they made the mass level in burnout inventory, they basically said the bottom or the people that scored in the top one-third, the people who scored the most burned out. These people were burned out. So to create the measure, they weren't looking at like clinically significant levels. They were just looking at like, well, one-third of these people are burned out.

Well, we know that that might not mean much clinically, right? And that's what I found. It's like someone could score just a little bit in the burnout range. And yet they'd be doing fine or they'd be doing pretty well. They're just tired. They've been just on some calls, you know. Whereas they scored really, really poor. It was actually like, wow, okay, this person is pretty significantly depressed and needs some help.

And probably, you know, has some level of dissociation at the workplace. So several other studies have shown that. supervisors, poor supervisors and burnout kind of go together. They're associated. So that study, those studies have been done. So where did we begin? We began, gosh, this was probably about like six or seven years ago. I put together a team of medical students, now they're residents, and Dr. Cashner, Dr. Pro, and we talked about what would be like the domains that we would study. we would look at. And so we picked seven domains, empathy, education alliance, psychological safety, effective feedback,

subjective emotional experience, bullying, harassment, prejudice, and bias. Those were the seven initial domains. And we then had residents in my department, the psychiatry department, over the course of two years, rate attendings that they were supervised with closely. We tested them every six months. We did this to all psychiatry residents. There was 100% participation. We did factor and content analysis, construct validation.

We reduced 61 questions to 12. We had good internal consistency with an alpha of greater than 0.9. We had a pretty amazing scalability coefficient. 0.836. Just to let you know, 0.4 is acceptable. And we did some qualitative analysis as well. In the test retest sample, found that there was a very, very close test retest, closer than the mass lab or not inventory even.

And we did what was called the nine criteria framework to look at this measure. This is well documented in the paper. Dr. Cashner, who was my statistician mentor, he's also a lawyer. who worked at the VA, he actually came up with the nine criteria framework and has published on that in other papers. So he was well qualified to help us walk through creating this survey. Okay, so the actual questions, let me go through the actual questions so you can kind of understand them and then we'll talk about what we found and how they're related. So the psychological safety questions, there were three of those. I would voice my concerns or

questions with this person. I felt free to express things that worry me. I felt free to ask for more information about his or her decisions and actions. And so they could score strongly disagree, moderately disagree, disagree, neutral, agree, moderately agree, or strongly agree. So it's like seven points. Strongly agree being seven, strongly disagree being one. So one being the worst score, seven being the best score. And then there were the education. Alliance questions, the way we communicated was clear and or helpful to our goals. This person seemed to respect me regardless of my mistakes. I felt grateful to have worked with this person. Once again, you can go from strongly disagree to strongly agree on those. Next to the three

empathy questions. I felt heard and understood. I felt understood and heard based on this person's body language, nonverbal cues, and facial expressions. That question embodied some of the affective empathy. And finally, this person was in touch with my perceptions and concerns. Next, there were the three feedback questions. When this person made decisions, they explained the thought process to me. This person gave feedback with specifics, not with generalizations, based on observations, not hearsay. And this person gave me a chance to work out answers for myself. Once again, one to seven, seven being the highest, seven being strongly agree. So when we looked at the Connection Index questions and subjective emotional experience,

this was a bunch of questions based on how stress they were in various encounters with this person, we found a linear relationship with the Connection Index. So as the connection between you and this person increased, the stress decreased. There was a linear relationship. When we looked at prejudice and bias, bullying, and harassment, there was actually a stepwise relationship with the connection index, meaning that if you scored anywhere from about four to seven on the connection index, so you scored between neutral and strongly agree,

you would not be scoring them for any prejudice and bias, but as soon as you scored below a four, there was a jump in the prejudice and bias bullying and harassment, meaning that until the connection was neutral or below, there would be no prejudice in bullying and harassment. We looked at the Mass-law burnout inventory, emotional exhaustion, and what we found was a stepwise decrease in emotional exhaustion as the connection score increased from 6.9 to 7.

So when you get to a near perfect connection index, there was a decrease on the burnout, emotional exhaustion, subdomain. What this means is that you would go literally from, for example, I feel emotionally drained from my work a few times a week to once a week with just one highly connected person. Okay. So that makes it seem, that kind of like made me think about this hero's journey. Like that really connected person is your guide. That is your mentor. That is the person that's helping you. And actually decreases your burnout, your emotional exhaustion.

Emotional exhaustion is linked to depression. So it's probably decreasing depression as well. So there was, and this is really an important thing that I learned from Dr. cashner. Not all data is represented by a linear relationship. Usually when you talk about a correlation, you're talking about a linear line that's going through

data. But some data is best modeled by steps. So we looked at different like was we looked at different steps and we found like the big step for precious and biased was with a really low

score, decrease emotional exhaustion, really high score. So we know from prior studies that empathy decreased diabetes complications. This was a very big study in Italy. It actually like the highest empathy physicians had about half the metabolic complications as the moderate and low empathy physicians. So we know that like empathy is a powerful impactor of not just mental health, but also physical health. Okay. And interestingly, all of the domains of the connection index flow together. And what I mean by that is if you look at a factor analysis, there's a slight differentiation between the domains, but all the domains flow together.

So it's almost impossible to score really high in one of the subdomains, like empathy and really low in like one of the other subdomains like feedback. And if you think about it, it makes sense. Like if the student is feeling empathy from you, then you're also empathizing with what they know and don't know and you're able to give them good feedback. When there's good connection, when there's good psychological safety,

when there's good empathy, when there's good feedback, when there's good therapeutic alliance or educational alliance. When all of those things are taking place, students thrive. It seems like a basic idea, but I think it's really important to put in people's minds how important this is. And so the next study I did, which is not published yet, but I'll talk about it, is we did a qualitative study on 16 medical students. We videotaped them. We looked at, we had them go through the Connection Index and talk about why they scored what they scored. And we asked them to talk about two attendings. Their most connected attending and their least connected

attending. And so what we then looked at were themes in the 16 transcripts with the domains. And so I think it's helpful to talk about what does an attending look like who's like the most empathic and the least empathic? Okay. So in the empathy domain, the most connected supervisor has these things in common. Was mentally present, paid attention, engaged, understood the concerns, or made an effort to do so, got to know the student on a personal level, greeted the student and acknowledged their presence and their work, whereas in the empathy domain, the least connected supervisor had these things in common, not mentally present, only concerned about their own matters, don't know the student's name,

no questioning about how they were doing, made the student feel non-existent, no respect of their time. The student feared making mistakes. In the psychological safety domain, the most connected supervisor felt safe to ask questions even if busy or if the student thought they were stupid.

The senior encouraged the student to learn from and to even question their decision-making, The senior cared about the student as an individual and allowed them to express their worries and their thoughts. In the psychological safety domain, the least-connected supervisor feared voicing concerns or questions because they were concerned they might look stupid or be seen as useless. When expressing questions, the senior was condescending, hostile, or saw them as an attention. on their judgment, and the student referred to as a medical student rather than by their name. Senior talked without breaks, so the senior was just blabbing on and on and on with no breaks. There was no expectation of the students, and the senior was very disengaged.

In contrast, in the feedback domain, the most connected supervisor had these things in common. taught at the level of the student, invested in teaching and making sure the student learned, voiced thought process out loud, gave the student's assignments, encouraged the student to re-teach other students, encourage the student to take ownership of their patients, gave them the autonomy and responsibility, gave specific feedback. In the least connected supervisor, this is what they said about feedback. In the feedback domain, the supervisor was concerned with showing how smart they were.

They exposed the student's lack of knowledge. They only gave criticism. They couldn't give feedback because they weren't paying attention. Mistakes weren't used as learning opportunities. They gave generic feedback, gave feedback based off others' observations and opinions. Finally, in the education domain, the most connected supervisor, the goal was to teach and that they took time to do so. Like I remember one supervisor would stay late and teach for like an extra hour.

Sometimes just being present or making that time to teach is so valuable. They taught and gave tasks at the student's level. They didn't make the student feel small or stupid. They used mistakes as learning opportunities. did not lose respect for the student after mistakes, gave specific goals, tasks, and responsibilities to the student, discussed their learning, goals, and objectives, made the student feel valued part of the team,

and was looked up to as a role model. In the Education Alliance domain, the least connected supervisor, so you'll see some commonalities here, was dictatorial and gave orders without explanations, shut down students' questions or input, look down at the student's, students' mistakes, yelled at the student for unspecified tasks and skills not taught, called

medical student, or the wrong name consistently, didn't give autonomy or responsibility over patients, concerned with finishing their tasks rather than teaching, ignored the medical student, blame the student for shortcomings outside the role on the team, expected the student to read the senior's mind, and sought to bring people down rather than teach them and bring them up. Okay. So in summary, connection includes empathy, psychological safety, feedback, education alliance. There are different ways to score the results. So it's not just a linear relationship with outcome measures, with other outcome measures. Tension and ability to understand the student was a theme of high connection attendings. and I think there was value, although we didn't show it in this data set, in coaching attendings

based off of good and bad scores. This is something I've done in departments. So I hope that this has been helpful for you. I hope that you check out the paper that I will list in the show notes. If you can't get access to it, let me know. And I hope that this can make some small improvement in the lives. of people going through training. Training is rigorous, it's tough.

This has brought me closer to the experience of people as they go through training, what it's like. I've had, I have a lot of gratitude for the people who have helped on this project, including Gretchen Asher, Christian Bishop, Adam Bereke, Daniela Bereke, Adriana, Martinez, Alice Ing, Kevin Inge, Olga Lane, Dr. Melissa Proe, Tyler Ridden, Sith Rietouan, Daniel Udrea, Joseph Wong, Dr. Michael Cashner,

and special thanks to Chloe Dominguez, who helped out quite a bit get the final draft completed. Once again, Dr. Michael Cashner was a great friend and advocate and support and understanding all of the the difficult statistical things that we did. If you read the paper, some of it will be completely non-coherent to you if you try to understand it from a statistical perspective because it is like graduate level and beyond. So we will hopefully be using this in different places. And if you want to use it, it's free.

You can reach out to me and I can give you the manual. I have a website dedicated to this. It's called Medical Education Research.com. And there's also a nonprofit that I've formed, connected with this. And I hope that this makes some small improvement in the lives of people in training and in the culture in medicine. So thank you for listening.