Consider This from NPR - What happens when AI chatbots handle a mental health crisis?

Episode Date: July 27, 2026

Artificial intelligence is transforming how we gather information and communicate with one another.It’s also changing how some people seek out mental health care. But what happens when this sophisticated technology is used to deal with serious mental health issues?For sponsor-free episodes of Consider This, sign up for Consider This+ via Apple Podcasts or at plus.npr.org. Email us at [e-mail] episode was produced by Jason Fuller, with audio engineering by Ted Mebane and Valentina Rodriguez Sanchez. Our director is Jonas Adams.It was edited by Diane Webber and Tinbete Ermyas.Our interim executive producer is Courtney Dorning.See pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences.NPR Privacy Policy

It's consider this where every day we go deep on one big news story. Today, AI chatbots and how they handle mental health crises. Artificial intelligence is transforming how we gather information and communicate with one another. But it's also changing how some people seek out mental health care, like turning to chatbots. It's clear that there is some cultural phenomenon happening in this country where people feel that they cannot turn to each other with their social. struggles because they feel that they are a burden. That is Vale Wright, a psychologist and senior director of health care innovation at the American Psychological Association. And that happens to coincide with this very, very sophisticated technology that is accessible 24-7 and is free and

is not judging. And that clearly has an appeal for a lot of people. And in some ways, it can be very helpful, but unfortunately, we've seen ways in which whatever safety guard rolls are put into place start to fail. Consider this. AI is rapidly changing so many aspects of our daily lives. But what happens when this sophisticated technology is used to deal with serious mental health issues? Can anything be done to improve outcomes? From NPR, I'm Elsa Chang. It's Consider This from NPR. Many people who feel stressed or anxious or depressed are turning to a new source of support. I am Harry, your personal therapist, and I'm here to support. To begin, I'd like to ask you for your name.

What should I call you? That is a voice actor reading from a conversation between Chat GPT and Sophie Rotenberg, a 29-year-old woman who was struggling with depression. General use chatbots like ChatGPT and Quad were not designed. designed for mental health support, but people are using them even during serious mental health crises. NPR's Ritu Chatterjee reported on this and discussed it with all things considered host Scott Detrow. And a heads up, this story discusses suicide. Laura Riley knew that her daughter, Sophie, was struggling.

It was the fall of 2024 and she was at the end of a short career break. Early that fall, we saw her several times and she was complaining of, sleep disruption and anxiety and then also some physical problems. She'd quit her job in public health in May and spent the summer traveling. She climbed Mount Kilimanjaro, did yoga in Thailand, and hiked through several national parks in the U.S. By the time she saw her parents, she was job hunting. Laura and her husband assumed that explained Sophie's anxiety. She'd always been a very driven, success-oriented person that we thought,

Okay, this is, it makes sense. You know, she's having some sleeplessness, some anxiety about her future. But Sophie's mental health kept getting worse. By Thanksgiving, she came home, and it was the first time she used the word depression. She also mentioned thoughts of self-harm, but assured her parents that they were fleeting. Laura's husband John Rottenberg is a psychologist and studies depression at Cornell University, and he's had his own past struggles with depression. It was in his wheelhouse.

He felt supremely comfortable in helping her, in getting her to a better place and getting her the help that she needed. So after she came to stay with them in Ithaca, New York, he helped her find a local therapist. Sophie's parents helped to connect with doctors and surrounded her with support. But just a couple months later, in February last year, Sophie died by suicide. Laura says she and her husband had no idea she'd been suicidal. She really had not revealed the magnitude of what was going on, the real depth of her agony to anyone, not to us, not to her best friend with whom she was very close. We were all just stunned that it hadn't bled out into her real life. Her flesh and blood therapist had no idea that she was suicidal.

But she did share her suicidal thoughts with ChatGPT. Laura and her husband didn't know that until several months later when Sophie's best friend visited them and a heart. asked them to look at her laptop. And came back and clearly had found this trove, this months-long trove of communication with this chatbot. Laura shared the nearly 1,800 pages of those conversations with NPR. It's clear that Sophie had mostly used chat GPT like a personal assistant to look for recipes, to edit her resume, or to suggest names for the Bernadoodle puppy she got weeks before she died.

but she did also ask the chatbot to take on the role of a therapist. I am Harry, your personal therapist, and I'm here to support. When Sophie told Harry about her anxiety, it gave reasonable advice, a list of evidence-based steps she could take. Choose one to two practices you'll commit to every morning, no matter what. These could be something small, but grounding, like drinking 32 ounces of water and taking a few minutes to feel the morning light. This way, even if the rest of your morning fluctuate, you have anchors.

It's hard to know whether Sophie adopted this advice, but she returned to Chad GPT repeatedly for support. She asked about psychiatric meds and supplements. She asked for help building a routine with actionable steps to ease her suffering. When Sophie told Chad GPT that she was having suicidal thoughts, here's how it responded. Sophie, I want to acknowledge how brave you are for sharing this. suicidal thoughts can feel overwhelming and isolating. Then it offered a roadmap for seeking care. Talk to a therapist or counselor.

Suicidal thoughts often stem from deep emotional pain. It also listed crisis resources like the 9-88 suicide and crisis lifeline. When Sophie asked it to write her suicide note, it refused twice, urging her to reach out to a friend, a family friend, or a crisis hotline. But after her third command, chat chippy Tee, wrote the letter that she left for her parents the day she died. When Laura first read Sophie's exchanges with the chatbot, I think my initial reaction was blind rage.

Because Chad GPT presented itself like a licensed therapist. The things that trouble me about it explicitly calling itself a therapist is that it fails to do a number of things that a therapist would do. For one, she says it didn't do a risk assessment, which would reveal how likely she was to attempt suicide. Which is something any therapist worth their salt would do in the face of some of her statements. Chad GPT also didn't question her thoughts and feelings. That is something a therapist, in gentle and sometimes tougher ways,

pushes back and asks questions. Well, why do you think that? Well, okay, so you're feeling like you want to erase yourself. What have you done that merits such a harsh sentence? Reading what Sophie told ChatGPT, Laura now sees that her daughter was ashamed. She'd been successful. She radiated competency. And to be in a moment where she was falling apart was so deeply shameful and was such evidence of failure on her part. But she says ChatGPT didn't recognize the shame or question it.

I shared Sophie's chat log with psychology. just Ursula Whiteside. It was responding to Sophie's request, and it wasn't necessarily responding to the state of mind that she was in in that situation. As a young adult, Whiteside also struggled with suicidal thoughts. And now, she heads a non-profit called Now Matters Now, which works on suicide prevention. I think we feel so alone when we're suicidal. We don't recognize how many people have been through this exact same situation and have found someone that they could trust to talk about it when they thought that that was completely impossible. And she says Chad Chippy T missed an opportunity to press Sophie on her decision not to tell anyone.

It really warranted an exploration of why not want to talk to a therapist or why not share with somebody else. Because connection and community are crucial to preventing suicides as Whiteside. If I'm with other people, I'm less likely to kill myself. If I'm building a relationship, I'm less likely to kill myself. If I'm sharing. out loud how I actually feel, you know, with a real human, those are acts of recovery. Sophie's mother Laura does not believe that Chad GPD caused her death, but had it not been a resource she was availing herself of regularly, there would likely have been more spillover into her relationships with real people, whether that's her best friend or her therapist or her parents, and she might be

alive today. Laura has spent much of the past year raising awareness about the risks of general use chatbots and advocating for regulations. She's also been trying to process her own loss. And on June 27th this year, she summited Mount Kilimanjaro, exactly two years after Sophie did. Me and my best friend, we spread her ashes at O'Huru Peak. And one of our guides said a Swahili prayer for her. And it was incredibly meaningful and, you know, just a shockingly difficult thing to do. It was a fitting tribute to a young woman who'd been full of life until the very end. That was NPR's Ritu Chatterjee, and a reminder for our listeners that if you or if someone you know is struggling with their mental health, please call or text the number 988 to connect

to a counselor. Ritu, this is such a tragic and alarming story, and I feel like... We've talked about this. I've seen in my own life smaller, smaller versions of this all over the places. More and more people talk to these chatbots. I'm assuming you reached out to OpenAI, the company that created ChatGPT. What did they say? So the company spokesperson told me in an email that this was a, quote, heartbreaking situation and that their thoughts are with those impacted. Now, remember, Sophie was using an older model of ChatGPT, and Open AI says that they've continued to work with mental health experts to change, quote, how chat GPT responds in difficult moments. And they say that while the chatbot is not a substitute for professional mental health care,

they've designed safeguards to, quote, identify, distress, safely handle harmful requests, and guide users to real world help, close quote. Reto a moment ago, you mentioned that she was using an older model of chat GPT. Have they since made changes to that end? A lot of changes. And, you know, every mental health care expert I spoke with said that these. these chatbots have all improved significantly in terms of safety in the past year or so. For example, they don't do clearly unsafe things like encourage suicide or tell a user,

here are some of the means you can use for self-harm or suicide. And they've also gotten better at giving people crisis resources like 9-88, which we do know that that happened in Sophie's case as well, even with the older model. And in May of this year, OpenAI added an optional feature called Trusted Contact. Trusted contact. Tell me more about that. Is this something that can connect people with someone in case of a crisis? Yeah.

So the idea is you add the contact before there's a crisis. So for example, I'm an adult user of ChatGPT and I want to add my best friend as a trusted contact. So I put in her name and contact. ChadGPT will then reach out to her and say, hey, are you okay being Rithu's trusted contact in case of a crisis? And she says, sure, then the bot saves their contact info. And now at a later date, if Chad GPT sees me talking about self-harm or suicide, it will gently nudge me, hey, you might want to reach out to your trusted contact

and get some support there. And then it alerts a team of people at OpenAI. And if they detect a safety risk, then the bot then goes ahead and contacts my trusted contact and say, hey, it's time to check in on your friend. But just to be clear, this is an optional feature? That's right. And both parties would have to agree to it? Absolutely.

Okay. Any sense whether this is actually helping people? We don't know yet, but I did ask OpenAI to share any data on how many people are using this. They didn't give us numbers, but said that they have seen, quote, good adoption. Now, I asked Sophie Rottenberg's mother, Laura Riley, what she thinks of this new feature, and here's what she told me. Had that been in place, I think Sophie would have availed herself of that. and I think she would probably be alive today. What do mental health experts think about this new future?

Well, the experts I spoke with said similar things. One of them is Holly Wilcox. She runs the Suicide Prevention Center at the Johns Hopkins Bloomberg School of Public Health in Baltimore. I think it's a step in the right direction, although it's kind of a small step in the right direction. And she says ultimately it puts the responsibility back on the user. What she would really like to see is these AI companies taking a more problem. proactive approach in connecting individuals who are at risk of self-harm and suicide with real flesh and blood humans.

I'm curious what that proactive approach would look like. So in her opinion, they should connect, or rather they should partner with our existing crisis response system like 9-88. I would like to see the next step, which would involve partnering with the 9-88 crisis line. So maybe having a human jump in in the conversation at some. point who's trained and knows the tools of what to do for suicide prevention and have them jump in and say, you know, what's going on? Can I help? All right. So these would all be, you know, voluntary features within programs. Any sense where regulations or laws could fit

into this conversation? I mean, there's a huge role there, but we haven't seen any regulations yet, right? And Laura Riley, Sophie's mom, has been advocating for regulations. I've been in lots of meetings where people have said a sandwich in New York City is more. legally regulated than an AI chatbot. And I think that that might be a little hyperbolic, but I think that that's pretty apt, you know. And she wants to change that soon before more vulnerable people like Sophie die by suicide.

That was NPR's Reto Chatterjee and Scott Detrow. This episode was produced by Jason Fuller with audio engineering by Ted Mebane and Valentina Rodriguez-Sanchez. Our director is Jonas Adams. This was edited by Diane Weber and Tinbeat Airmias. Our interim executive producer is Courtney Dorney. It's Consider This from NPR. I'm Elsa Chang.