Speaking of Psychology - How to Choose Effective, Science-based Mental Health Apps, with Stephen Schueller, PhD

Episode Date: September 23, 2020

Among the thousands of apps that aim to help people with everything from stress to anxiety to PTSD to sleep problems, how many are based on solid scientific research? How many live up to what they promise? And how can you choose from among all the options? Learn more about your ad choices. Visit megaphone.fm/adchoices

2020 has been a watershed year in digital mental health care. The coronavirus pandemic and a summer of racial reckoning have left many Americans stressed, anxious, and depressed, while at the same time the pandemic has made it more difficult to obtain in-person mental health care. Can technology help to bridge that gap? Many consumers seem to think so. One market research firm found that downloads of the top 20 mental wellness apps rose almost 30% from January to April this year.

But among the thousands of apps that aim to help people with everything from stress to anxiety to PTSD to sleep problems, how many are based on solid scientific research, how many live up to what they promise, and how can you as a consumer make informed choices based on your mental health needs? Welcome to Speaking of Psychology, the flagship podcast of the American Psychological Association that explores the connections between psychological science and everyday life. I'm Kim Mills. Our guest today is Dr. Stephen Schuller, executive director of One Mind Cyber Guide,

a nonprofit mental health app reviewing website that's been called the Consumer Reports of Digital Mental Health. He is also a professor in the Department of Social Ecology at the University of California Irvine, where he works on developing and evaluating digital mental health technologies with the goal of expanding access to mental health care. Welcome to speaking of psychology, Dr. Schuller. Thanks so much. I'm really excited to be on here.

As I just mentioned in the intro, One Mind Cyber Guide has been called the Consumer Reports of Digital Mental Health. Why did it get that moniker? What does that mean? And is this something that we need out there in the marketplace? Well, so I definitely think it's something we need out there in the marketplace. There are a host of different digital mental health products, most of these being apps.

So if you were to go to the App Store, and to put in the search box, depression, anxiety, stress, PTSD, you would get tens of thousands of responses across those different terms. In fact, some estimates suggest that there's about 20 to 25,000 mental health apps that are out there and trying to find a good one is like finding a needle in a haystack. And so definitely we find that a lot of consumers do need some help in terms of separating the good from the bad, the effective from the not effective, the science backed from the ones that aren't science back. And what we do at One Mind Cyber Guide is we identify and we review products on multiple dimensions.

We look at how credible is it? Does it have science backing that this product actually works? Is it based off of evidence-based techniques? We look at the user experience. Is it easy to learn, easy to use, easy to navigate? Is it free from technical glitches? And we look at issues around data security and privacy. Does the app live up to standards of regulation?

Does it keep your data safe? Does it not sell your data to third-party individuals? And I think all these things are things that should figure into a decision on whether an app might be a reasonable product for you to try to be able to promote your own mental health and wellness. How do you, at the Cyber Guide, decide which apps to review since there's so many out there covering such a range of conditions? Yeah, it's a good question when we get asked a lot. It's a little bit of an art and a little bit of a science.

And so, you know, we really want to focus on products that people are using. So we look at the number of downloads and the use of those products by consumers. We want to focus on products that are backed by science. And so we constantly are looking at the scientific literature in terms of, you know, what works, what is evidence behind it. We also want to focus on things that are useful for people. And so we focus on products that fit the needs of different consumers. We have partnerships with various organizations and they help us figure out what products they're constituents are interested in. And so I think the processes that we use to identify what products

that we look at kind of aligns with the ways that people decide to use these things, that they hear about them from trusted sources, from their doctors, their friends, from family members, they read articles about them on social media or in websites. Or they might see an advertisement for a product, you know, in a couple different places. And so I think we at One Mind CyberGide, were constantly looking all that information, and then we had prioritized things based off of, again, what people are using and what the science is saying. Who are your reviewers and what are their qualifications? How do they become reviewers for the Cyber Guide?

So One Mind Cyber Guide, we have a team of different reviewers in-house that review different products for our website. And those reviewers all undergo trading to be able to use the materials we used to be able to evaluate products. We also do some contracting out for some of our review material to outside experts, to clinical psychologists and other mental health professionals. But those are usually what we call professional reviews. And so on the One Mind Cyber Guide website, in addition to those criteria that I mentioned, the credibility, the user experience, and transparency around data security and privacy, we also have sort of a narrative. description of how you might use this product in your life?

And what are the pros and cons and what are some of the different clinical considerations? And those narrative reviews are all written by professionals. And so the individuals doing those credibility, user experience, and transparency reviews are all members of the One Mind Cyber Guide team that are trained to be able to use those rating scales with fidelity and reliably. Do you collect any information from the users and use that as a basis for how to you review various products? You know, that's a good question.

I think it's something that would be really useful. It's not something that we actually currently do at One Mind Cyber Guide, but definitely something that we have a lot of interest in. We've done a little bit of that information before, but it's hard to track that. I think that's actually one thing that really necessitates having a product, like One Mind Cyber Guide, is that you would think about where those consumer reviews often live, and those are in the app stores.

And so you go to the app store and you see, you know, this person thinks this is a five-star app, this person thinks this is a four-star app. And what we find is that those ratings and those user ratings from the app stores often don't relate to the credibility and the benefits of those products. Now, I think there's some problems with the way that those ratings appear in the app store. So I think that there's a lot of companies that are probably paying people to put positive ratings in the app store or, you know, they're biased. so only the people are having really good experiences

or going to the app stores to leave those ratings. So I don't mean that to say that I don't think consumer, the impact on consumers and the benefits from consumers and the views of consumers are important. I just think it's a tricky thing to figure out how to collect systematically in an unbiased fashion. Like we try to do what the work we do at One Mind Cyber Guide. And it's something that we're still trying to figure out

the best way to be able to incorporate. So I would really hope that that would be something that One Mind Cyber Guide would be using in the future, but I think it's still a really important open question about how to best collect that information. Yeah, kind of risky, like using Yelp to choose the restaurant where you want to eat tonight. Well, and I think that's a great point. And I think, you know, I sort of, I love restaurants, so I like the metaphor. I liked when we used to be able to go to them.

But, you know, I think what we think about sometimes at Cyberguide is that we sort of are the the Michelin Star equivalent. So we have experts going. We're trying to figure out, like, at the best, what is this product capable of? And I think the Yelps are important as well, you know, because there's a lot of information and value that gets added from consumer ratings. And, you know, Michelin Stars and Yelp ratings don't always line up. And I think that we would need both. But I think we need both in a way that sort of acknowledges what are the pros and cons of each approach?

If I have a four and a half star rated Taco Bell, I don't think about that as being a lot. on the same standard as a well-reviewed Mexican restaurant that has a Michelin Star, but maybe that tells me something about that particular location. And so I think bringing that information together, I think is really valuable. But again, it has to be done in a way that understands that each of those approaches, the Michelin approach and the Yelper approach, have strengths and limitations. And you want to leverage the strengths while downplaying the limitations. How much should consumers rely on apps for their mental health issues? Are there some conditions that are better suited to apps than others?

For example, I can't envision someone with a serious mental health issue, such as, say, schizophrenia, getting much help from an app. Yeah, well, first I'd like to say that apps are no replacement for a therapist. These are not meant to be replacements for traditional mental health services or for mental health professionals. And so I think that there's sometimes a false dichotomy made between apps versus traditional treatment. And I think that really the best case scenario should be technology plus traditional services, technology and human services. When we look at the research, however, though, pretty much anything you can treat through a psychosocial intervention through traditional treatments, you can treat using a digital technology. as either in some sort of combination with a human supporter or with some sort of relationship with some human support.

And so there are definitely, there are apps out there for psychosis. There are apps out there for bipolar disorder. There are apps out there for a host of serious mental illnesses. I think actually, you know, interesting, I think one of the products that I'm really interested in when it comes to psychosis and schizophrenia is looking at these cognitive remediation products. So these are technologies that essentially provide brain training to be able to improve the cognitive functioning. And those tools work.

There's a lot of robust scientific evidence behind them. But again, those aren't seen as a replacement for some of the other pieces that are part of the treatment than an individual with psychosis or schizophrenia might receive to be able to promote their recovery. So we find that these tools work for a variety of different people. Now, I think depending on the mental health challenge that someone's facing, there might be more or less appropriate uses of these technologies in sort of the continuum of care. So, for example, in the UK, the National Institute for Clinical Excellence actually recommends digital treatments as frontline treatments for the treatment of depression anxiety. They don't have that same recommendation for serious mental illness. And so I think for common mental health issues, these might be very nice sort of frontline treatments or might be ways to sort of really introduce the skills that get taught in traditional psychosocial

treatments. But for more serious mental illnesses, they might play a slightly different role. So I think research really supports that these things are useful across a variety of conditions, but the way that we use them might differ. How much is this being taught right now in graduate psychology programs? Are people being trained to use apps as part of the therapy that they may offer in the future? I don't think it's a major part of current clinical programs, at least not broadly trained. I think that there's still a developing expertise in this area.

I think there's some places where we see a lot of good work in training and implementation. So, for example, the Department of Defense and the Veterans Administration have been frontrunners, both in the development of these tools. They were leading experts in terms of developing some of the first mental health apps, including PTSD coach, and they continue to have a suite of different products. And because they have so many different tools, they're really training people in the DOD and VA to use these tools in their practice. But I think if you go to graduate programs in psychology, there's probably not a lot of work that's really looking at teaching people how to use technology thoughtfully in their clinical work. And so I think this is something that really would be an important area to address in sort of future training programs. And it's something that I care a lot about because I think that, you know, again, this is not technology instead of people.

And so I think we need to train a workforce that's educated to use these tools in their practice. What makes for an engaging user experience, why are some apps stickier than others and make users come back? And doesn't that make them more effective if people actually use them, systematically. Yeah, I think that that is the case, that people who use these products more tend to get more out of them.

I think that the science of engagement is a, it's a challenging area because I think that there are reasons where people may not use a tool as much and still benefit. So for example, there might be,

the term I use is happy abandonment, which is, you know, I got what I needed out of this app, and now I don't need to use it anymore. And that there's also some mechanism through which the people who need the most help tend to use the product the most or an app the most. And so there's a relationship between clinical severity and app use that sort of complicates understanding of the relationship between engagements and outcomes.

That being said, the only way these things could work is if someone uses it. And so I think to answer your question about what makes these. apps sticky or engaging or makes people want to come back, I think simplicity is a really sort of important piece, that it has to be a product that's easy to use, that integrates into your life well. I think a lot of these sort of digital health technologies make people's lives harder by giving them something more to do and none of us need something more to do in our lives. And so I think the best products are ones that sort of simply and reduce. burden and reduce complication by sort of seamlessly integrating to a person's life.

I think also being able to meet the needs of the user. So some people might really want videos and other forms of media to engage in. Some people might do well from having very brief sort of exercises to read. And so, you know, I think it's also the case that not one app is going to meet the needs of sort of all different users or really appeal to everyone. And so I think understanding who the app is built for and having the content and the interaction styles and the things that really kind of fit that different person. So I think an important implication of that, frame that thinking is that for a consumer, if you're interested in using one of these products, to try a couple different of them.

So if you want to use a mindfulness app to download maybe two or three mindfulness apps, try out a couple of the limitations, see which ones really sort of to fit your lifestyle, the way you like to engage content, the ones that make sense to you, that you like the color schemes, you know, all those things, I think, play a big role. And then to decide to kind of use that product. And I think that's one of the benefits. You know, I kind of introduce this as a challenge is like there's, you know, 20 to 25,000 things out there.

So it's really hard to find ones that are evidence-based. But I think the flip side of that is because there's so many, that means if you do a little bit of searching, you might be able to find one that really does fit what you need when you're looking for one of these products. Let's talk a little bit more about data privacy. You mentioned that as being one of the aspects that you look at when you're evaluating apps. And there's been a lot of chatter in the news lately about at least one of these big app companies possibly mining user data for marketing purposes. How much should consumers be worried about this and what can they do to protect themselves? Yeah, I think it's definitely something consumers should be concerned about. And we did a review

where we took a deep dive into the data security privacy policies of about 120 different apps for depression. And we found that about half of those didn't have a data security and privacy policy. And of the half that did, only half of those privacy and security policies were what we deemed acceptable. So there's a lot of variance and there's also a lot of, or I should say there's not a lot of information out there for a lot of these products. So I definitely think that a consumer should read the data security and privacy policy. And look, I know I've looked at a lot of terms and services and privacy policies for different technologies I do and it's a big wall of text and I just click accept and I move on. But I think, you know, one thing I think is important. One easy thing to sort of

look at is, does the product have a data security and privacy policy? Because I mentioned many don't. And so I think that's a red flag in itself. If they don't tell you what they're doing with your data, you probably don't like it. You know, as I mentioned those Department of Defense and VA apps, I think those are very good apps with very good data security and privacy policies because they are developed by the VA and the DOD. And so they don't collect the type of information that some of these other products to collect that are potentially using you as a revenue generating source when you're using their product. And so I think looking at where the app comes from is important. I think, you know, the other thing to consider is, you know, everyone is a little bit

different in terms of their sense of what data is being collected from them and how comfortable they are with that. So I fully appreciate that when I use Google Maps, Google knows where I'm going all the time. And that's a trade-off I'm willing to make because, you know, I'm not going anywhere that's all that exciting. And I really like to use Google Maps. And so I think that if we think about data and privacy as somewhat transactional, that a person can be, hopefully make an informed decision whether the transaction, the benefit they get from that product is worth whatever they're giving up or potentially giving up when they're using that product. And that's one reason why we actually call our data security and privacy scale, a transparency scale, because to us,

it's really the important piece is the transparency of the information, such that a consumer can make an informed decision of whether what information is being collected from them is worth the benefit they're receiving from using that product. So what are some of the best mental health apps out there for various conditions. I mean, I know there's no one-size-fits-all answers, but there must be some that rise above the rest. Yeah, I mean, I think that it's a challenging question. And definitely one thing that we do at One Mind Cyber Guide is we don't endorse any specific product. We have our scores, and we sort of, you know, emphasize, you know, what I've noted for you is that there's different ways to evaluate these things and there's different aspects that

people might care about. So someone might care a lot about credibility. I really want to know that this works and might be kind of technically savvy. And so they're a little bit less concerned about the user experience. On the flip side, maybe someone is more open to a tool that doesn't have as much science behind it if it's really engaging and they like it and it's, you know, really sticky for them. So, you know, I think, again, there's a lot of diversity in terms of what, what fits for, you know, specific people. I mean, I think that as I mentioned, Those VA DOD apps are, you know, they have a lot of evidence behind them. We really like them.

You know, so like PTSD coach. There's a COVID coach app that kind of focuses on stress and anxiety related to COVID. These apps do lean a little bit more towards veterans. And so there's a little bit more of that flavor in some of their content sometimes, which might be a barrier for some people. But I think what's great about those apps is that you know that they're based on evidence-based practices. they have a solid clinical and development team behind them and that you know that they work really well.

I think that for a variety of other apps, there are things that maybe kind of align with what you're kind of looking to get. So, for example, our group has been looking at mood tracking apps recently. I really like a mood tracking app called Dalio because it kind of lets me track my moods in a way that's really useful for me, but that might not be useful for everybody. So, you know, I really do sort of encourage, as I noted before, that one of the best ways to kind of find an app that might work for you, you know, go to the One Mind Cyberguide website, poke around a little bit, maybe find two or three apps in an area, you know, two or three meditation apps that seem like the best picks. So maybe you pick headspace, calm, and insight timer. You look at those different products and you see which one kind of fits best for you. And I think that's really an approach that we sort of advocate in terms of trying to figure out. which app a person might end up using to be able to help promote their mental health and wellness.

I know the health insurance company that I have at work has given us access to at least one mental health app for free, which you would normally pay for. Is that something that's on the increase where insurance companies are also promoting these apps? Yeah, that's definitely the case. We actually just did a review of that through One Mind Cyber Guide and looked at a couple different insurance companies and found that there were there were a lot of different products that they were providing, although nowhere near as vast as the 25,000 I know that are out there. So probably about like half a dozen or half a dozen to a dozen products that different insurance companies were providing.

My insurance company actually provides one as well. So I just got a notification that my strength, which is a cognitive behavioral therapy app, is available through my insurance. And so that is something worth looking at is that if you have insurance and are covered, there might be a product that your insurance is covering that there would usually be a payment associated with it, but you might be able to receive it for free. And then additionally, sometimes these are products that are not publicly available on the app store. So you may not be able to go download it yourself. They usually go through enterprise contracts to companies or insurance companies, insurance providers.

And so you might have some access to some products you might not be able to find on your own through your insurance company. In a recent paper that you published, you wrote about three misconceptions about digital mental health, and one of those misconceptions was that mental health technologies are a new way to deliver psychotherapy. Why is that a misconception if digital mental health is not a new way to deliver therapy? What is it? Yeah, I think that technology offers a lot of opportunities to do things in a new and different way. And so I think that if we are merely translating traditional practices, from therapy to technology,

we are doing a disservice to individuals who might be able to be better reached and better served by new technology. So for example, let me make sort of a metaphor here is that a lot of these early technologies came out. When they came out, they really had this sort of like session frame of thinking that was present in sort of psychotherapy practice. So it's like you log onto the website,

you do your 50-minute session, You read something for a couple of minutes and you do some exercises. And then you have to wait until next week to get your next session. And that's not the way people use technologies. We're like we're in Netflix. We're an on-demand world. You want things now and you want things in the moment. You know, if I wanted to learn how to cook an omelet and I signed up for a, you know,

went to the web and signed up for a class, it would be ridiculous to say that like, okay, well, Stephen, wait a week and we'll teach you out of crack eggs and wait another week and we'll teach you how to cook them. I want a YouTube video that's going to show me on make an omelet right here, and I want to be eating that omelet in 15 minutes. And so I think that technology has the potential to sort of better create these on-demand, real-world experiences. And I think that those lead to some open questions that we have to sort of further evaluate.

I think that, you know, so I think that we need to think about how clinical practices and behavior change techniques, how the science of what we know what works can be translated to technologies. I think we can't merely make a digital version of psychotherapy because that's kind of probably the most boring use of what technology is capable of. Beyond your work with mental health apps, you're also interested in the intersection of technology and mental health. I think that's what we've been talking about. You recently co-authored a report on teens, social media, and mental health that suggested that researchers look at the topic in nuanced ways and not just ask whether social media is good or bad for youth.

Can you talk a little bit more about that report and what you found? Yeah, I think that there is a lot of concern. I think reasonably so right now with as much time as everyone's spending in front of these screens about the impact that screens and social media might be having on people's mental health. And I think sort of more specifically some concerns that the increased rates of social media use are leading to increased rates of things like depression, anxiety, and suicide. I want to just say that I think what we've seen looking across the research is that the research

is not bearing out that relationship. It does not seem that social media use on average or at a whole is leading to the increased rates of these things. I think that there's a lot of nuance there that some people are helped by social media. Some people are hurt. And a lot of people it's probably not doing much of anything when it comes to their mental health and wellness right now. I think one thing that is really important that we talk about that report and I think is important in terms of reframing the debate is that screen time as a concept doesn't have a lot of meaningfulness in the current way we use technology. So like what does an hour of use of Instagram or Twitter look like? You're reading some content. You're creating some content.

you're contributing some content, you're direct messaging with some people. And so I think that one of the sort of things that we sort of note is that to think about screen time in the same way we thought about it with TV shows, radio shows, things where content is much more chunked and something that you absorb is really different in this new world

where you're sort of constantly interacting with these different, these different technologies. And so that we need, you know, better ways to sort of understand what is a person actually doing on these platforms when they are interacting with them. And we need to sort of think more about like what are the wellness facilitating activities and what are the wellness detrimental activities that might take place on these different technology platforms. What do you think the future holds for the use of technology and not just apps for the field of mental health, what do you think would be helpful going forward, both for the field of

psychology and also for consumers? Yeah, I really think that in 10 to 20 years from now, we're not going to be talking about digital health or digital mental health. We're going to talk about health and mental health and ways that uses technology. I think that technology is going to become deeply integrated into the way that we see services delivered and see people reached. And I think that the current pandemic is really accelerating that. I think we've seen a real uptake in the use of telehealth and virtual visits. And I think to me, the next stage is really thinking about how we create those kind of, as we were talking about a little bit earlier, those new sort of experiences using technology.

I think that there's a lot of exciting opportunities for things like virtual reality, augmented reality, different technologies that really make these things much more pervasive and persuasive in people's lives. And it wouldn't surprise me if in a lot of instances that they first door a lot of people walk through when they think about receiving mental health services or using a tool to sort of promote their wellness or well-being is really a digital tool first. I think that these are really potentially scalable, cost-effective, and effective resources, and I think that we'll see a lot more use of these and the sort of continuum of mental health care and wellness support. Well, thank you so much for joining us today,

Dr. Schuller. This has been really interesting, and I hope it's going to be edifying to our listeners. Thanks so much. It was great to be on the program. You can find previous episodes of Speaking of Psychology on our website at speakingofpsychology.org or wherever you get your podcasts. If you have comments or ideas for future podcasts, you can email us at Speaking of Psychology at APA.org. That's Speaking of Psychology, all one word at APA.org. Speaking of Psychology is produced by Lee Winerman. Our sound editor is Chris Kondyenne.

Thank you for listening. For the American Psychological Association, I'm Kim Mills.