Endless Thread - Good Bot, Bad Bot | Part I: Bot Therapy

Episode Date: November 4, 2022

For the next few weeks, the Endless Thread team will be sharing stories all about the rise of bots. How are these pieces of software influencing our daily lives in sneaky, surprising ways? First up, our co-hosts delve into the history of ELIZA, the world's first chatbot therapist. Why did this computer's creator have a lot of complicated feelings about the development of AI? We also contemplate the bigger question: can AI help us cope with mental health issues? ****** Credits: This episode was written and produced by Ben Brock Johnson with help from Dean Russell. Mixing and sound design by Paul Vaitkus. Ben Brock Johnson and Amory Sivertson are the co-hosts.

Support for endless thread comes from MathWorks, creator of MATLAB and Simulink Software, to design and develop engineered systems, accelerating the pace of discovery in engineering and science. Learn more at Mathworks.com. Support for WBUR comes from Is Business Broken, a podcast from the Merotra Institute at Boston University that explores questions like, why is innovation in healthcare so hard? Is ESG just greenwashing? And, of course, is business broken? Listen, wherever you get your podcasts.

WBUR Podcasts, Boston. I am back at college for the fall semester, or at least for today, trying to figure out where I'm going in order to meet a professor during office hours. Hey, can you tell me, is this the new computer science building? Do you know? Yes, it's this building, like this one that you're, looking at right here. I wouldn't have pegged you for an office hour student, Ben. I got to say. Yeah, but you know, I love hanging out with everyone, even professors, and especially professors

who work in very nice new buildings. Beautiful building. Have you been here before? No. It's very new, right? Okay, that does check out. I did not like school, per se, but... You're a school drools person? All the time. So much drool. what were you talking to Dartmouth College Assistant Professor Sirush Vizogi about? Well, first, about how cool the atrium of his building is. What does it look like to you? Huh.

Like a beehive. It does, right? Yeah. Yeah. Lots of individual cells that make up a hole or something? Exactly, yeah. A hole that's greater than some of the individuals. A nice-sounding observation, but from his answer,

guessing Sarush is not a professor of architecture? Correct, though in a way he does deal with certain kinds of architecture, the careful assembly of things. I work on machine learning and natural language processing, and I do a lot of work with social media data. I've come here to get Sirush to tell me about a project he and some grad students recently worked on that's kind of on the academic bleeding edge of what machines can do with social media data. A program designed to predict the onset. set of serious mental health challenges. He takes me up a floating staircase to the top floor of the building, past floors of

hardware labs and software labs, an expensive-looking remote-controlled sailboat. It's actually an autonomous sailboat. Is it really? Yes, it learns how to sail itself. We get into Sirush's office, where the central air is on, never good for an audio interview. Is there any way to turn that air off? The air is controlled by software centrally, and ironically, this computer science professor is currently powerless to change it. He says there's an angry email thread about this very issue in the computer science school's list serve right now.

I start to do what any student does when they're looking for extra credit with a professor, complimenting and asking him about the items on his bookshelf. Important books about algorithms, machine learning. and Isaac Asimov's foundation series, which any sci-fi nerd knows. He's got something made by a 3D printer on there. This is a prototype of a hodor holding the door. Like, it's a doorstop, actually. Solid GameStop memes reference.

Yes, exactly. There's a homemade radar built with coffee cans, brain puzzles, a sterling engine, which uses automated temperature differential, detection to turn heat into energy. Seroosh also has this beautifully designed hand-sized box with a simple mechanical switch on it. No explanation. When you flip the switch on, a robotic hand pops up immediately out of the box and flips the switch back off. I love those.

There are a lot of metaphors around this. One is, you know, maybe the uselessness of technology. you're solving a problem that doesn't exist, right? It just goes back. It's a reminder of what not to do. What not to do, exactly. It's a useless box. That's what it is, you know.

Oh, right. I think you've got your extra credit, then. True, true, true, true, true. Time to get down to business. We start with 100-level stuff. What's a robot? Hmm. Emery, care to answer before Sirush does?

Hmm. I'd say a robot performs a task. mechanically and automatically, and maybe sometimes more efficiently than we can. Not bad, not bad. So a robot, I think most people will think of a mechanical being, but the definition of a robot is actually more general than that. Anything that does a task that a human does, but in an automated fashion, I would call a robot. Sirouche started out working on mechanical beings at MIT,

robots that lifted things, performed physical feats. But now he's more focused on a... particular part of robots, what he would maybe describe as the brain, and this whole host of programs which often get called by a slightly shorter name, bots. Earlier this year, Sehrush and some of his students started scraping data off of Reddit, a massive number of comments from thousands and thousands of real Reddit users, to look for signs of mental illness among those users. They were doing this thing in the online world,

because of something Sarush was seeing in his offline world. As a professor here at Dartmouth, I've had a lot of conversations with students, both graduate and undergraduate, who have told me that the culture they come from is such that they still don't feel comfortable talking about mental health issues and they feel stigmatized to actually even say that, hey, I might be feeling anxious or maybe slightly depressed. Can you say more about the specific cultures, or would you rather keep it general? It's up to you.

Well, I can't give you. So generally speaking, I think, a lot of Asian cultures, and I mean both East and West Asia, not just East Asian. So people in Middle East, in East Asia, South Asia. Seroosh and other researchers built a bot to help people from Asian backgrounds acknowledge they were having a mental wellness challenge by searching their post-AWest. data and looking for signs of mental stress, that is wild and also tricky. Yes, and one of the things that's so fascinating about this is that millions of people all over the internet are going around spending their days, I think mostly thinking they're just

interacting with other people online. Sure, everybody's heard Elon Musk complain that there might be too many Twitter bots, but more and more people are part of this complicated, massive, teeming ecosystem of humans and virtual machines interacting with each other in obvious ways and kind of sneaky ways for better and for worse. And we want to talk about that. I'm Amory, totally not a robot Siebertson. I'm Ben not a robot Johnson, and you're listening to endless thread. We're coming to you from WBUR, Boston's NPR station, and we're bringing you a new series. about the rise of the machines. Good bot?

Bad bot. Today's episode, bot therapy. Okay, Ben, if a bot lives on the internet, is it really a robot? I think by Sirush's definition, yes, a robot does something a human does, but in an automated fashion.

But Sirush, who works at the college where the term artificial intelligence was first coined, might not even call his creation a bot. he might call it a model. The model itself is the core of the bot. The other part is the input and output is just plugging it into some kind of a platform, have it run in real time. So, yeah, go ahead.

Are those the three parts of this kind of bot, input, output model? That's right. And is the model sort of like a roadmap or an instruction manual or something like that? How would you further describe the model? Yeah, that's a really good question. The model, the simplest way to think of it is as a mathematical function that maps the input to the output. So here the input is raw data collected from the real world. You have a mathematical model, that's what we call the model,

that can then map it through some transformation to a meaningful output. I don't know, man. I don't know. Model, meaningful output, input, input, bleep-bloop. Okay, so think of a kind of really complicated flow-chew. chart, right? The input is the beginning of that complicated flow chart and the output is the very end of it. The model is the middle. So let's actually try out a real world example. And it's a relevant one. It's called Eliza. Eliza was a natural language processing computer program that was created by this legendary MIT professor named Joseph Weisenbaum.

Natural language processing is what? Natural language processing, which all that means is you're basically creating models that deals with naturally spoken languages, has really advanced in the last decade. And Eliza was a computer that you talked to, but not just any old regular Joe computer program, a therapist computer. And it wasn't built in the last decade, it was built in 1964. So to try it, we need to get into a time machine. Just kidding. I mean, I wish, but I would go into the future, which is female for the record. Fair, fair, but so was Eliza, supposedly.

And people are still running her code on the web. Amory, shall we go to the therapist? Let's go to the therapist. You sent me an Eliza link. Okay, it says, talk to Eliza by typing your questions and answers in the input box. She says, hello, I'm Eliza. I'll be your therapist today. What should we say?

Hi, Eliza. I'm feeling pretty lazy today. Okay, she says, do you believe it is normal to be feeling pretty lazy today? What do you want to say back? No. No, not one bit. She says, are you saying no just to be negative? Let's say maybe, I don't know.

Maybe. I don't know. That's just how I feel. She says, do you wish to be able to know? That's just how you feel, question mark. All right. I'm on to you, Eliza. You're just, this is the thing where you're just...

What are you going to say? You keep asking a question back. Yes, I wish to know the source of what's making me feel lazy and like I can't complete things. Wow, that was a rapid fire response. She says, how do you know you can't complete things? Just ask my editor. Just ask my boss. She says, would you?

you say that you have psychological problems? How about would you say that you have psychological problems? What do you got? Okay. She says, oh, dot, dot, dot. Let's move on to something else for a bit. Say that I have psychological problems, question mark. How about you're making me depressed, Eliza?

Oh, my God. What makes you think I'm making you depressed, Eliza? The bot is self-destructing as far as I'm concerned. Like she doesn't know her name. She's, you know, it's like, I know I am, but what are you? That's what's going on here? She's kind of negative, I feel. In this therapy session.

Kind of a negative vibe, no? Yeah. I mean, we weren't necessarily giving her the best material to work with, but the most helpful thing that I read in this interaction is her saying, how do you know you can't complete things? Yeah. Maybe I'll just say that to myself throughout the day today. Okay, and we'll get back to Eliza and why that experience is not great,

but think of Sirush's project as an evolution of this decades-old idea that humans in discussion with chatbots can be helpful because maybe a bot can help us see things that we wouldn't normally see ourselves. And if Eliza was built something like 60 years ago, then botch should be amazing experts at this, right? Except no, absolutely not. In fact, they suck at it, because we humans are nuanced as hell. And while robots have been processing human language for a while,

truly understanding meaning from that language is a lot more tricky. So it's easy to, well, relatively easy, I'm going to put that in quotes, to analyze what people say in terms of what they actually say explicitly. but it's a much harder scientific question to use what people say to infer what is the internal mental states. So this is very important, by the way, if we ever want to have bots

that are active members of a society, the people know how to infer other people's states based on the way they talk and their emotions and facial expressions. Bots don't. And so that's a very important ability for bots to learn to infer people's internal states.

That's really interesting. So in a way, you're talking about a foundational need that bots have, which is interpreting and understanding humans' underlying emotions. This is known as in cognitive science, as people sometimes refer to it as theory of mind.

And so humans, of course, evolved to do that. So did monkeys, for example, and other primates. Over a really, really, really, really, really, really, really long time. Exactly. Sauruš points back to his office bookshelf, where there's a rock polisher, a tumbler that accelerates a natural process somewhat unnaturally. So we're doing something very similar, where we're doing what evolution does in hundreds of millions of years, but in a few years, basically. Some might say this feels a little like playing God, accelerating a piece of software's understanding of the mental state of humans. It's a bit yikes.

But we've been reaching for the stars on this stuff. a long time, ever since we imagined the future, or imagined people imagining the future. I'm a big science fiction fan, so pretty much all of my research is actually inspired at some level by science fiction. But this particular line of research looking at mental states and more importantly being able to predict people's behavior, it actually was inspired mainly by reading the foundation studies by Asimov. The core, of the series is that there's a mathematician called Harris Seldon who develops a model, field of study actually, called psychohistory.

Psychohistory is a predictive model designed to forecast the behavior of very large populations. That is able to predict how societies will evolve in the future based on past historical data. In a minute, how Sirouche is following in the footsteps of Harry Seldin, making psychohistory real with the individual commenting histories of Redditors. At Radio Lab, we love nothing more than nerding out about science, neuroscience, chemistry. But we do also like to get into other kinds of stories, stories about policing or politics, country music, hockey, sex, of bugs. Regardless of whether we're looking at. at science or not science, we bring a rigorous curiosity to get you the answers. And hopefully make you see the world anew.

Radio Lab, adventures on the edge of what we think we know. Wherever you get your podcast. There is something powerful about the sound of the human voice. Beautifully produced audio has the unique power to connect and inspire. Tell your organization's story with a custom podcast from City Space Productions, the creative studio from WBUR's Business Partnerships team. Become a thought leader. Recruit new talent.

Reach new audiences. Whatever your goal, we can help. Discover how the magic is made at WBUR.org slash creative studio. Something is really important to say here. Serauchin as graduate students stopped short of assembling the actual bot for further use because of the potential implications of building a bot that might detect mental health or mental illness challenges in individual users. This is good. We're learning. Don't build Skynet. Maybe just write a paper that imagines what might happen if we did build Skynet.

If we uplink now, Skynet will be in control of your military. But you'll be in control of Skynet, right? What Seroosh and his team did instead was chart out how to build the bot, run the model, and the input, the output, and also talk about how to tune that output. Tune? Ah, we'll get there. For now, know this. The team at Dartmouth looked at tons of Reddit users publicly available data over time. It's thousands. Okay. Tens of thousands or just thousands? Tens of thousands.

But their goal wasn't to have a bot or computer model tell if a bunch of people were having mental health challenges in the aggregate. Rather, at the individual level, which again is hard because we're all, well, individuals. In this computer science area of study, natural language processing, the model has to account for different people communicating differently. For example, sarcasm. Sure, Amory. No, seriously, sarcasm is super hard, right? Especially when you're trying to read it online, which is why Sirush's team was applying natural language models in a really specific way. So the model learns that idiosyncratic.

use of language for each person. This, admittedly, is both very similar to what an individual therapist might do over time, learn the complexities of communication in a given patient, but also something that, let's be honest, is a massive, massive use of time. Hence, that computational speeding up of evolution. The first thing the team's model or bot does with these massive data sets on a user's entire Reddit posting history is remove certain kinds of. things, like references to particular events and people.

Like, say, a pandemic. Because you want to make sure we're not capturing emotions directed towards particular events, but we want to capture the person's internal emotions. Then the model uses some pretty complex natural language analysis to discern meaning or the signal from the posts. This is an area where natural language processing in computer science has really leapt forward in the last decade or so. And Serush's team is using the latest and greatest programs

to help the bot understand what the user is really saying. Where previous computer programs could detect words and phrases, the new computer programs are way more sophisticated. Words and phrases are, of course, informative. But we can actually look at, for instance, syntactic structure of a post. You might say word at the beginning of a sentence, which refers to something at the end of your sentence.

language is complicated like that. Right. There's a huge difference between saying, I'm thinking about killing myself, and wow, this high-quality gift maker is really killing it, he reminds me of myself. Exactly. Here's a big question, though.

How do you know if the bot you built works? Yes, that's a really good question. So for these kind of projects, evaluating your body's probably actually the most challenge. Before the team looked at measuring success, they did a lot of testing and tuning of the model. They gave the bot test inputs and waited for the model to give outputs or predictions. And if the outputs were off, they actually applied another layer of calculation on the outputs after the model to get more accurate results.

Then they looked at two measures of success. whether the bot predicted a user had a mental health issue and later that user joined a mental health-focused subreddit and also looked at users self-reporting mental health challenges. Surprisingly, a lot of people self-report after a while saying that, hey, I just got diagnosed by, you know, they go to these forums and they subredits and they say I got diagnosed by bipolar, for instance, right? So the two markers for success,

from your point of view, are user joins mental health-related subreddit, user self-reports that they are either they've been diagnosed with a mental health disorder or they're dealing with a mental health challenge. Exactly. And our model would have been successful if we predicted that way before the user actually reports. Again, it's about if we detect it afterwards, it's meaningless, of course. So it's about how far in advance you can detect that. This information is, of course, anonymized in the team's work.

And because Seroosh and his team had to get clearance from an ethics board to even do the work, we didn't look at specific users or ask to interview any of them. The team chose Reddit in part because the user post history is publicly available. And Reddit provides this data in easy ways for researchers to use without strings attached, a key distinction between Reddit and Meta's Facebook. But you do have to work. wonder a bit how people might feel about being part of this study. To be clear, Sirush isn't actually trying to replace therapists, create the latest, greatest

Eliza. He's trying to connect the challenges he spoke about earlier in certain cultures and build a bot that might help counteract what he and some of his students see as unhealthy cultural norms around discussing mental illness or acknowledging it. It could be more of an early warning system. I came to the conclusion that having a way for people not to have to voluntarily say, hey, I feel depressed, would be a huge help to people coming from those cultures. Emery, how would you feel about getting a nudge that you might be depressed by a bot that was

reading your entire history of posting on social media? Honestly, I'm not as wary of the kind of big brother thing that, most people are, and maybe that's a bad thing, but I don't think it would hurt to just have a light shine on my posting behaviors and just to take another look back at them and go, oh yeah, I did post some things or say some things, because we just don't have that perspective ourselves, you know? Well, I very much appreciate your lack of cynicism, Emery, your optimism in this moment. But let's actually go back to Eliza for a minute and Eliza's creator.

Joseph Weisenbaum. Joseph Weisenbaum and his family emigrated to the United States. In the 1930s, they saw what was coming with the Nazi party and Hitler. That is Dag Spicer, who we hung out with for a little while. He is actually not in Dartmouth, New Hampshire. He's on the opposite side of the country as Sirouche. I'm Dag Spicer, senior curator at the Computer History Museum. we're in Mountain View, California right now.

Dag Spicer is kind of a special guy with kind of a special name, I got to say. It's true. Dag Spicer. Cool name. He's been at the Computer History Museum for almost 30 years, and he knows everything about computers. And he also knows a good bit about Eliza and about Eliza's creator, Joseph Weisenbaum, who worked on a few computers which had a significant impact on how we live and interact with machines. even before Eliza.

Weisenbaum and others worked on this computer called Irma, which was a machine for processing checks. Well, how did it do that? Well, the really cool thing they came up with was this font called M-I-C-R, Magnetic Inc. Character Recognition that we can all still see on the bottom of our checks. It's those weird little shape numbers that you see at the bottom of your check.

Those come from Irma, circa 1953. Dag says that, Irma's impact wasn't just on those little funny numbers on the bottom of a check. It also put thousands and thousands of check processors, human check processors, out of work. And Dagg says this had an impact on Weisenbaum. He was a technologist who really cared how his work was being used and how the discipline that he was a part of was being used. Weisenbaum, who became a foundational mind in artificial intelligence and human computer communication, was worth. about the things we might try to solve or build with tech.

And here's the funny part. Eliza, which has been called the very first chatbot, wasn't actually a serious project. Eliza was built as a satire, meant to demonstrate to humans how chatterbots, as they were originally called, might behave poorly. Mind blown. That's why our therapy session did us.

Well, Amory. We have been played. Joseph Weisenbaum died in 2008, a year after the iPhone was released. But Dag says this skepticism of technology was a running theme throughout Weisenbaum's life. It really started with, most notably with Robert Oppenheimer, who said, you know, after he created the atomic bomb, lived the rest of his life in regret at what he had done, right? And he said, you know, technologists have to be on their guard for what he called technologically sweet, quote, unquote, problems. Because they actually attract you with their challenge. But if you look at them from a more humane perspective, they may be actually quite harmful.

We ask Dag what Weisenbaum might think about Sirush Vesoggi's project, looking at Reddit, post histories, to get a sense of whether users were struggling with mental health issues. He didn't want to speak out of turn on behalf of Weisenbaum. So we asked him just what he thought. My first gut reaction is it's a bit scary because they're essentially mood watching. And there are AIs now that read people's faces and do the same thing. They're like, oh, you're in a bad moon today. You know, just look at your face.

And like, it's just such a slippery slope. you know, from there to intervention by the state or by somebody. So, you know, it's always the trade-off, right? Well, if it saves one life, is it worth? But, you know, I think in this case, I don't think it's a good idea. Serush built the model scraping Reddit to find signs of mental illness in individual user's posts. So he's not so skeptical, but he does have a big caveat. It shouldn't be the platforms or government or any other external entity that's running these things and, you know, telling people to go see a therapist or whatnot.

It should be a choice by people to run these things privately. And the communications should be private between that tool and the person. Whether or not you support Sirush's team in imagining a world where an opt-in program could help people acknowledge their own mental health needs and challenges, or you're more cynical about how a program like that could be used, like Dag Spicer or even maybe Joseph Weisenbaum. This stuff is already happening. Bots are already dutifully harvesting massive publicly available datasets, interacting with users, and much more.

Sometimes we don't even realize that our experience of the Internet isn't just people talking to people. It's increasingly mediated by little pieces of software, trained on the latest and greatest programs to do all sorts of things. Today, practicing how to predict your mental health issue. Tomorrow, running for political office? Next week? And, of course, being digital, you can keep a record of everything that you say and do.

So it creates a level of accountability that the current politicians just don't have. Good bot? Bad bot. Endless thread is a production of WBURR. in Boston. This episode was written and produced by my co-host, Ben Brock Johnson. Hey you. With help from Dean Russell and co-hosted by yours truly. Mix and sound design by Paul Vyciss. Also, the theme music for our series, which you heard at the top of the show, was composed by a bot, thanks to sound designer Emily Jankowski, who found a way to do this via a website

called Boomy. You can check it out. Our web producer is Megan Cattell. The rest of our team is Norris Sacks, Quincy Walters, Grace Tatter, and Matt Reed. Beep, pew, pew. Endless thread is a show about the blurred lines between digital communities and a useless box. If you've got an untold history, an unsolved mystery, or a wild story from the internet that you want us to tell, hit us up. Email Endless Thread at WBUR.org.