Short Wave - Can you teach a computer common sense?

Episode Date: January 31, 2023

Over the past decade, AI has moved right into our houses - onto our phones and smart speakers - and grown in sophistication. But many AI systems lack something we humans take for granted: common sense. In this episode Emily talks to MacArthur Fellowship-winner Yejin Choi, one of the leading thinkers on natural language processing, about how she's teaching machines to make inferences about the real world.See pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences.NPR Privacy Policy

You're listening to Shortwave from NPR. Okay, so the first time I ever talked to a computer was sometime in the 90s. I was at a kids' museum, and I remember the computer screen had this inviting green font, which said, Hello, I'm Eliza. I'll be your therapist today. And I sat down at the keyboard and just started typing. Pretty soon I was spilling to Eliza all of my friends. my middle school friendship stress, and incredibly, she was responding in ways that felt

almost human. My encounter with Eliza was an encounter with one of the earliest natural language processing systems in the world. It was invented in the 60s. And nowadays, there are multiple systems like this. Spell check, spam filters, Siri, and Alexa. Sorry if your smart assistants are now paying attention to this episode. But my point is that natural language processing, or the ability of machines to understand and interpret human language is undergoing a revolution. Nowadays, instead of Eliza, chat GPT is talking up a storm. But back when Yejin Choi was in college at Seoul National University in South Korea, decades ago, AI seemed like science fiction.

I loved the intellectual appeal of it, but it looked like nothing really worked. In fact, this is during AI wintertime. This was when people used to think AI is doomed. See, when AI was brand new in the 50s, people were really optimistic about what it could do, reproduce human intelligence as a computer program. But as time went on and the challenges became clear, a period of doubt set in, an AI winter. And this was around the time that Gajun read about it for the first. time. They were thinking about solving common sense problems of intelligence, and they realized that

this is too hard. So for some time, people were not just as excited about AI research compared to other disciplines of computer science. So after graduation, Yejin chose a more established path. She went into operating systems and became a software engineer for Microsoft in Seattle. And it was fun, but something was missing. I felt that for my lifetime focus, I wanted to work on something that looks much more open-ended and requires deeper investigation and adventure. Are you like that as a person? I do have that side.

I mean, we all live once, right, and I might as well try to do things that excites me the most. She left her software engineering job and jumped on the AI train. It was a high-risk research direction where the chance of failure was so much higher than the chance for success. And today, Yejin is a professor of computer science at the University of Washington and researcher at the Allen Institute of AI. And last year, an unknown number from Chicago kept calling her and calling her. And eventually they reached her by email. So a University of Washington professor now officially a genius. Yejin had won the MacArthur Fellowship, unofficially known as the Genius Grant.

Every year, the MacArthur Foundation, quick disclosure, they're one of NPR's funders, chooses people on the cutting edge of different disciplines and gives them each a monetary gift. This time it was $800,000, over five years, to spend however they want. And in 2022, they chose her for her work on AI. Yejin was stunned. I just couldn't believe. This was not supposed to happen. in consideration of everything that I went through, that just, yeah, I had no clue.

When you say what you went through, what do you mean? Oh, so coming from a different culture where in only phase of my upbringing, things were very different. Let's just say it was not as equal as how things are today, which I think is still not perfect between men and women. is now one of the leading thinkers on natural language processing in the world. And as someone who has looked into the soul of AI more than most, she says the fear and speculation and excitement surrounding the science

doesn't take into account its current limitations. I think people overestimate what AI might do tomorrow based on what seems to work today, especially that for the first time, things are working way better than, ever before. Today on the show, we talk about artificial intelligence that's learning to talk to us. I'm Emily Kwong, and you're listening to Shortwave, the Daily Science Podcast from NPR. Over the past decade, it really feels like systems that use natural language processing have moved right into our houses, onto our phones and smart speakers, and grown in sophistication, in

their ability to seemingly understand us. And Yeager-Tur-Thing-e-chen-true. choice research focuses on one specific aspect of language, something we call common sense. You probably know what that is. It's knowing a wine glass will likely break if you drop it on the floor, but probably be fine if you drop it on the bed. We develop common sense reasoning as we experience the world around us and make inferences, and certain questions don't really need to be asked. Yejin gave me another example. How many eyes a horse has? Obviously, too. Don't talk about it. We don't usually spell these things out because, well, it feels obvious.

But not so much for AI. But if you ask GPT3, the OpenAI language model, it says a horse has three eyes, two in the front and one in the back. It makes things up. That doesn't make any sense. So if common sense is so common, why is it so hard for a computer, which is being programmed by humans? to mimic. Yeah, that's the mystery. It's an amazing machinery that can learn a lot of surface patterns in language, which to our big surprise, can get very, very far to the point of being able to write beautiful paragraphs. But it's able to do so without actually understanding

the meaning or concepts underlying this text. And that's because we hear. Humans are multimodal learners. In school, we might hear the teacher's words, but also see a drawing on the whiteboard to reinforce it. We watch someone wipe out on a skateboard and we don't make the same mistake on our own feet. This kind of knowledge that we take for granted is the very thing machines struggle with. They often follow one route of learning and exclude others. But that's really changed in the last decade. And Yejin has been a leader in that change. And I realized that all the past research was done in 70s and 80s, when there wasn't enough of compute power. And then the approaches that the researchers tried back then was very much based on logic because they assumed that common sense must be describable in logical forms. But I realized that common sense must not be attempted things. through logic. That was my working hypothesis. So when I tried very different approaches to common sense based on modern deep learning methods, combined with novel or different types of

symbolic integrations to these deep neural networks, we started seeing very exciting results. Okay, so here's what they did. It's really cool. Yejin and her colleagues built one of the first common sense knowledge graphs, a kind of textbook called Atomic, and they combined it with a neural network that could learn from that textbook. The result was called Comet. When researchers typed a real-world common-sense problem into Comet, a problem it had never encountered before, its guesses were 78% plausible. It was inferring things using artificial common sense. For example, when Comet was given the prompt, person X gives person Y some pills.

guess that person X wanted to help person Yajun and her colleagues then augmented comet with visual scenes like movie and TV show clips. This added context helped the machine make even better guesses and resembles how humans learn. Human intelligence is definitely very much multimodal. And for a long time, fields of AI, such as computer vision or natural language processing, each focused on only one modality at a time, but language is really used with intent and the background knowledge and interpretations, that's not in the sentence as literal content, but it's a lot about the subtext and implied meanings and everything that's outside

what's literally written there. Yeah. What would you say is the biggest shift in your thinking about language and common sense since you started these research adventures? The more I research into language and common sense, the weirder it gets. Common sense, in my view, is the dark matter of intelligence and language. And the normal matter is only 5% of the universe that we can see and touch and measure. And I think that's exactly what happens with language. and the meaning of language in the sense that what's written down or spoken out loud in the literal form

is only the surface of it, really beneath the surface, there's this huge unspoken assumptions about how the world works. And in investigating the mechanics of these unspoken assumptions, Yeagin's hope is to develop guardrails, to keep AI from doing us harm. Mistakes are real. So one example is that recently there was a home device that suggested a 10-year-old to touch a metal coin to an electric socket. Oh, gosh. And fortunately, the child did have a common sense that it's a very bad idea. But conversation systems don't have that level of common sense yet.

We are entering a new era where AI becomes... potentially really powerful part of our life, everyday life. And so we do need to think about the consequences or implications of using AI. Wow. Does it ever feel like a heavy burden, these questions, being on the cusp of this? Yeah, it's a big responsibility. Yeah. I am actually excited to bring more humanities, building more bridges between AI and

humanities. Instead of thinking of it as a burden, I want to think it as an exciting challenge to work on. I want to give a quick hello to our NPR Plus subscribers. Thank you for supporting our show. If you want to join this raging band of The Relentlessly Curious, sign up now at plus.mpr.org slash shortwave. As a subscriber, you'll get to listen to Shortwave without sponsor interruptions. It's totally worth it. That's plus.mpr.m.com. org slash shortwave. Today's episode was produced by Liz Metzger, edited by Gabriel Spitzer and fact-checked by Britt Henson.

Valentino Rodriguez Sanchez was the audio engineer. Rebecca Ramirez is our supervising producer. Brendan Crump is our podcast coordinator. Our senior director of programming is Beth Donovan, and the senior vice president of programming is Anya Grundman. I'm Emily Kwong. Thank you for listening to Shortwave, the Daily Science Podcast from NPR.