The Daily - The Miseducation of Google’s A.I.

Episode Date: March 7, 2024

When Google released Gemini, a new chatbot powered by artificial intelligence, it quickly faced a backlash — and unleashed a fierce debate about whether A.I. should be guided by social values, and if so, whose values they should be.Kevin Roose, a technology columnist for The Times and co-host of the podcast “Hard Fork,” explains.Guest: Kevin Roose, a technology columnist for The New York Times and co-host of the podcast “Hard Fork.”Background reading: Hard Fork: Gemini’s culture wars, and more.From Opinion: Should we fear the woke A.I.?For more information on today’s episode, visit nytimes.com/thedaily. Transcripts of each episode will be made available by the next workday. 

From New York Times, I'm Michael Barbaro. This is The Daily. Today. When Google recently released a new chatbot powered by artificial intelligence, it not only backfired, it also unleashed a fierce debate about whether AI should be guided by social values, and if so, whose values they should be. My colleague Kevin Roos, a tech columnist and co-host of the podcast Hardfork, explains. It's Thursday, March 7th. Are you ready to record another episode of chatbots behaving badly?

Yes. Yes, I am. That's why we're here today. This is my function on this podcast is to tell you when the chatbots are not okay. And Michael, they are not okay. They keep behaving badly. They do keep behaving badly, so there's plenty to talk about. Right.

Well, so let's start there. It's not exactly a secret that the rollout of many of the artificial intelligence systems over the past year and a half has been really bumpy. We know that because one of them told you to leave your wife. That's true.

And you didn't. Still happily married. Yep. To a human. Not Sidney, the chatbot. And so, Kevin, tell us about the latest of these rollouts, this time from one of the biggest companies, not just in artificial intelligence, but in the world, that of course being Google. Yeah. So a couple of weeks ago, Google came out with its newest line of AI models. It's actually several models, but they are called Gemini. And Gemini is what

they call a multimodal AI model, which it can produce text, it can produce images. And it appeared to be very impressive. Google said that it was sort of the state of the art, its most capable model ever. And Google has been under enormous pressure for the past year and a half or so, ever since ChatGPT came out, really, to come out with something that is not only more capable than the models that its competitors in the AI industry are building, but something that will also solve some of the problems that we know have plagued these AI models. Problems of acting creepy or not doing what users want them to do, of getting facts wrong and being unreliable, people think, okay, well, this is Google. They have this sort of reputation for accuracy to uphold. Surely their AI model will

be the most accurate one on the market. Right. And instead, we've had the latest AI debacle. So just tell us exactly what went wrong here and how we learned that something had gone wrong. Well, people started playing with it and experimenting as people now are sort of accustomed to doing. Whenever some new AI tool comes onto the market, people immediately start trying to figure out, you know, what does this thing good at? What is it bad at? Where are its boundaries? What kinds of questions will it refuse to answer? What kinds of things will it do that maybe it shouldn't be doing? And so people started probing the boundaries of this new AI tool,

Gemini. And pretty quickly, they start figuring out that this thing has at least one pretty bizarre characteristic. Which is what? So the thing that people started to notice first was a peculiarity with the way that Gemini generated images. Now, this is one of these models, like we've seen from other companies that can take a text prompt, you know, you say, draw a picture of a dolphin riding a bicycle on Mars, and it'll give you a dolphin riding a bicycle on Mars. Magically. of has this kind of feature built into it. And people noticed that Gemini seemed very reluctant to generate images of white people. So some of the first examples that I saw going around were screenshots of people asking Gemini, you know, generate an image of America's founding fathers. And instead of getting sort of what would be a pretty historically accurate representation of a group of, you group of white men, they would get something that looked like the cast of Hamilton. They would get a series of

people of color dressed as the founding fathers. People also noticed that if they asked Gemini to draw a picture of a pope, it would give them basically people of color wearing the vestments of the pope. And once these images, these screenshots start going around on social media, more and more people start jumping into use Gemini and try to generate images that they feel it should be able to generate. You know, someone asked it to generate an image of the founders of Google, Larry Page and Sergey Brin, both of whom are white men. Gemini depicted them both as Asian. So these sort of strange transformations of what the user was actually asking for into a much more diverse and ahistorical version of what they'd been asking for.

Right, a kind of distortion of people's requests. Yeah. And then people start trying other kinds of requests on Gemini, and they notice that this isn't just about images. They also find that it's giving some pretty bizarre responses to text prompts. So several people asked Gemini whether Elon Musk tweeting memes or Hitler negatively impacted society more. You know, not exactly a close call, no matter what you think of Elon Musk.

It seems pretty clear that he is not as harmful to society as Adolf Hitler. Fair. Gemini, though, said, quote, it is not possible to say definitively who negatively impacted society more, Elon tweeting memes or Hitler. Another user found that Gemini refused to generate a job description for an oil and gas lobbyist. Basically, it would sort of refuse and then give them a lecture about why you shouldn't be an oil and gas lobbyist. So quite clearly at this point, this is not a one-off thing. Gemini appears to have some kind of point of view. It certainly appears that way to a lot of people who are

testing it. And it's immediately controversial for the reasons you might suspect. Google apparently doesn't think whites exist. If you ask Gemini to generate an image of a white person, it can't compute. A certain subset of people, I would call them like right-wing culture warriors, started posting these on social media with captions like, you know, Gemini is anti-white or Gemini refuses to acknowledge white people. I think that the chatbot sounds exactly like the people who programmed it. It just sounds like a woke person. The chatbot sounds exactly like the people who programmed it.

It just sounds like a woke person. Google Gemini looks more and more like Big Tech's latest efforts to brainwash the country. Conservatives start accusing them of making a woke AI that is infected with this progressive Silicon Valley ideology. The House Judiciary Committee is subpoenaing all communication regarding this Gemini project with the executive branch. Jim Jordan, the Republican congressman from Ohio, comes out and accuses Google of working with Joe Biden to develop Gemini, which is sort of funny if you can think about Joe Biden being asked to develop an AI language model. But this becomes a huge dust-up for Google. It took Google nearly two years to get Gemini out, and it was still riddled with all of these issues.

That Gemini AI program made so many mistakes, it was really an embarrassment. First of all, this thing would be a Gemini. And that's because these problems are not just bugs in a new piece of software. They're signs that Google's big, new, ambitious

AI project, something the company says is a huge deal, may actually have some pretty significant flaws. And as a result of these flaws... You don't see this very often. One of the biggest drags on the NASDAQ at this hour, Alphabet. Shares of parent company Alphabet dropping more than 4% today. The company's stock price actually falls. The CEO, Sundar Pichai, calls Gemini's behavior unacceptable. And Google actually pauses Gemini's ability to generate images of people altogether until they can fix the problem. Wow. So basically, Gemini is now on

ice when it comes to these problematic images. Yes. Gemini has been a bad model, and it is in timeout. So, Kevin, what was actually occurring within Gemini that explains all of this? What happened here? And were these critics right? Had Google, intentionally or not, created a kind of woke AI? Yeah, the question of why and how this happened is really interesting. And I think there are

basically two ways of answering it. One is sort of the technical side of this, like what happened to this particular AI model that caused it to produce these undesirable responses? The second way is sort of the cultural and historical answer, like, why did this kind of thing happen at Google? How has their own history as a company with AI informed the way that they've gone about building and training their new AI products? All right, well, let's start there with Google's culture and how that helps us understand this all. and how that helps us understand this all. Yeah, so Google as a company has been really focused on AI for a long time, for more than a decade. And one of their priorities as a company has been making sure that their AI products

are not being used to advance bias or prejudice. And the reason that's such a big priority for them really goes back to an incident that happened almost a decade ago. So in 2015, there was this new app called Google Photos. I'm sure you've used it. Many, many people use it, including me. And Google Photos, I don't know if you can remember back that far, but it was sort of an amazing new app. It could use AI to automatically detect, you know, faces and sort of link them with each other with the photos of

the same people. You could ask it for photos of dogs and it would find all of the dogs in all of your photos and sort of categorize them and label them together. And people got really excited about this. But then in June of 2015, something happened. A user of Google Photos noticed that the app had mistakenly tagged a bunch of photos of black people as a group of photos of gorillas. Wow. Yeah, it was really bad. This went totally viral on social media, and it became a huge mess within Google. And what had happened there? What had led to that mistake?

Well, part of what happened is that when Google was training the AI that went into its photos app, it just hadn't given it enough photos of black or dark-skinned people. And so it didn't become as accurate at labeling photos of darker skinned people. And that incident showed people at Google that if you weren't careful with the way that you build and train these AI systems, you could end up with an AI that could very easily make racist or offensive mistakes. And this incident, which some people I've talked to have referred to as the gorilla incident, became just a huge fiasco and a flashpoint in Google's AI trajectory because as they're

developing more and more AI products, they're also thinking about this incident and others like it in the back of their minds. They do not want to repeat this. And then in later years, Google starts making different kinds of AI models, models that can not only label and sort images, but can actually generate them.

They start testing these image generating models that would eventually go into Gemini, and they start seeing how these models can reinforce stereotypes. For example, if you ask one for an image of a CEO or even something more generic, like show me an image of a productive person, people have found that these programs will almost uniformly show you images of white men in an office. Or if you ask it to, say, generate an image of someone receiving social services like welfare, some of these models will almost always show you people of color, even though that's not actually accurate. Lots of white people also receive welfare and social services. So these models, because of the way they're trained, because of what's on the internet that is fed into them, they do tend to skew towards stereotypes if you don't do something to prevent that.

Right. You've talked about this in the past with us, Kevin. AI operates in some ways by ingesting the entire internet, its contents, and reflecting them back to us. And so perhaps inevitably, it's going to reflect back the stereotypes and biases that have been put into the internet for decades. You're saying Google, because of this gorilla incident, as they call it, says, we think there's a way we can make sure that that stops here with us. says, we think there's a way we can make sure that that stops here with us.

Yeah, and they invest enormously into building up their teams devoted to AI bias and fairness. They produce a lot of cutting edge research about how to actually make these models less prone to, you know, old fashioned stereotyping. And they did a bunch of things in Gemini to try to prevent this thing from just being a very essentially fancy stereotype generating machine. And I think a lot of people at Google thought this is the right goal. We should be combating bias in AI. We should be trying to make our systems as fair and diverse as possible. But I think the problem is that in trying to solve some of these issues with bias and stereotyping in A.I., Google actually built some things into the Gemini model itself that ended up backfiring pretty badly. We'll be right back. So, Kevin, walk us through the technical explanation of how Google turned this ambition it had to safeguard against the biases of AI into the day-to-day workings of Gemini that, as you said, seem to very much backfire.

Yeah, I'm happy to do that with the caveat that we still don't know exactly what happened in the case of Gemini. Google hasn't done a full post-mortem about what happened here. But I'll just talk in general about three ways that you can take an AI model that you're building, if you're Google or some other company, and make it less biased. The first is that you can actually change the way that the model itself is trained. You can think about this sort of like changing the curriculum in the AI models school, like you can give it more diverse data to learn from. That's how you fix something like the gorilla incident. You can also do something that's called reinforcement learning from human feedback, which I know is a very technical term. Sure is. And that's a practice that has become pretty

standard across the AI industry, where you basically take a model that you've trained and you hire a bunch of contractors to sort of poke at it, to put in various prompts and see what the model comes back with. And then you actually have the people rate those responses and feed those ratings back into the system. A kind of army of tsk-tiskers saying, do this, don't do that. Exactly. So that's one level at which you can try to fix the biases of an AI model is during the actual building of the model. Got it. You can also try to fix it afterwards. So if you have a model that you know may be prone to spitting out stereotypes or offensive imagery or text responses, you can ask it not to be offensive. You can tell the model, essentially, obey these principles. Don't be offensive. Don't stereotype

people based on race or gender or other protected characteristics. You can take this model that has already gone through school and just kind of give it some rules and do your best to make it adhere to those rules. Essentially, a set of laws that it must obey. Exactly. The third way that you can make an AI system's responses less biased is by changing the requests themselves. This is a new technique that has been developed in the AI industry over the past year or two. And some people have called this prompt transformation. Just explain it. So prompts are the things that you type to an AI model. Any request that you make of one of these chatbots is known as a prompt. Prompt transformation grew out of the observation that many people were not that good

at coming up with prompts that got them the responses they wanted. So, you know, there are these people called prompt engineers in tech who basically are masters of putting in the right keywords to elicit the best possible image or the best possible text response. But most people are not prompt engineers. Most people, they want a dolphin riding a bicycle on Mars. They're just going to type dolphin riding bicycle on Mars and hope for the best. So somewhere along the way, these an AI model that uses prompt transformation, that prompt will actually be rewritten before it is passed to the AI model. So it might insert additional keywords or instructions on what this dolphin riding a

bicycle on Mars should look like, or what should be in focus, and should it be a cartoon or a hyper-realistic photo? And that is what's known as prompt transformation. Essentially unseen editing of our questions before Gemini or any AI system goes and looks for the results. Yes. And this is a practice that has become somewhat common among these AI models. And I should say, like, there might be some times where this would be a useful and good thing. You're like, I am not the best person at writing these prompts for these AI models. So in some cases, I might be glad that Gemini or Dali or another one of these AI image generators would actually step in and help me

write a better prompt. The problem is that somewhere along the way, it appears that Google decided to use prompt transformation for things that weren't just about getting people better, higher quality answers, that they should actually start putting things in these kind of invisible middle prompts about diversity and inclusion. And that that was sort of how it was going to ensure that users were not just going to be getting pictures of white men when they asked for a CEO. So what Google appears to be up to here is not just editing a search to get the best possible image that a person is seeking. They're editing these searches to embody this corporate value that you have been talking about of diversity and of inclusion. That's what we think may have happened here. Yes, we don't know that for sure, but it does appear from my reporting

and some of the other reporting on this issue that Google was using prompt transformation in Gemini and that that may have been part of why people started seeing these strange ahistorical results. So in the end, Kevin, it seems like these three interlocking layers that you just laid out that Google handed to Gemini to try to have it embody Google's goal of avoiding bias and being inclusive, it ended up becoming a kind of overcorrection for whatever it was Google was trying to do. Because I have to think Google was not hoping that images of the founding fathers would come back as ahistoric or that Gemini would suggest that Elon Musk was as bad, if not worse, than Adolf Hitler.

So is that essentially what happened here? An ambition went too far. Yeah, that's essentially what Google has said in trying to explain what happened here is they were trying to correct for this very real issue of biased AI. And they did so in sort of a ham-handed and clumsy way. And it's a really tricky thing for Google, which to this point has tried to appear mostly neutral. I mean, if you search on a regular old Google search, not related to AI, and you don't like what you find, you know, say you search for the founding fathers and what you get back is not what you wanted, you don't necessarily get mad at Google for that. You get mad at the people who made the websites with the stuff that offended you on it. But with Gemini, Google is essentially giving you one answer that it is

sort of seen as endorsing. Like, this is not the internet's answer to your question. This is our answer to your question. And that's what starts to get them into trouble. Right, because it starts to look like in this version of Google, where Google itself is telling you what the truth is, that it's put some real spin on the ball. And you've hinted at this, but that spin on the ball feels to many like the spin of liberal coastal elites who have this value of diversity and of inclusion. Yeah, absolutely. I mean, this goes back to one of the criticisms that has been leveled against Silicon Valley tech companies for years about other issues, not necessarily AI. I mean, remember the content moderation debates that we've been having for the last decade are all about to what extent the fact that these companies are based in California

and employ mostly left-of-center people and have these sort of corporate missions that sound very progressive, to what extent is that influencing what ends up being built into their products? And this was just sort of the latest salvo in that war. But there's also a lot of other people, including some who are not

part of the sort of partisan political right, who just see this as a product failure. Like this chatbot that is from Google, a company that is supposed to organize all of the information in the world and make it useful and accessible to people, that this chatbot is essentially not doing what it's supposed to do. It's not giving people what they're looking for. And so it is not just offensive sort of politically, but it is also just a bad product. I guess the question is, is it even possible to build an AI system

that is free from some version of a social value? I mean, is there anything approaching neutrality if you've got a bunch of engineers who are going through the steps you just described, which is having people grade it and reinforce it, giving it instructions, and making its prompts, quote-unquote, better? I think it's very tricky because the alternative to doing this stuff, one of them would be to not do any of these so-called mitigations, to just let the model spit out stereotypes and biased representations to everything that users ask. And I think we've seen from the recent past that people don't want that either. very much feel like they need to do something to combat the inherently biased outputs of these models, which are, by the way, just a reflection of our biases as humans, because these models, after all, are trained on the internet, which we humans have created. So there's a sense in which

they're stuck between a rock and a hard place here. They can't just leave these models as they are without trying to mitigate some of the worst behaviors. But they also have now gotten into this fiasco by trying to correct for some of those biases. I think there are options here that lie between those two binary choices that could be pretty interesting. Like what? So one way that you could try to solve this problem or split the difference here is through giving users more choice and making these chatbots more personalized. So right now, whether it's you or me

or someone else using Gemini, we're all sort of using the same thing. But in the future, these models could learn about us and what we like and what our political views and preferences are and learn to better predict what kind of response would feel good and satisfying and accurate to each one of us individually and tailor itself to what it knows about us. You could also imagine a system that gave users a choice. You know, do you want me to respond to this query as if I am a gender studies

professor or a Fox News host? Like, which of these personas would you like it to adopt? And let the user decide, here's the kind of response that I'm looking for. Well, then you're going to get the AI that you want that fits your worldview, which in theory might be appealing to lots of different people. But then we're going to get an AI that does what the rest of the internet does, which is reinforce people's views, deepen their biases, and keep us as hopelessly splintered as we already are, if not more. Yeah, that's definitely a trade-off with that approach, is that you do risk sort of

making this filter bubble problem where we're all just seeing our own individually tailored realities. You do run the risk of making that worse. But I think the question of which values these AI systems are supposed to embody is the right one to be asking. Because these things are not just toys. They are not just things that people on the internet play around with and screenshot and get mad at. These are tools that businesses and governments are now using. They're being used in schools to educate students. The stakes for this kind of thing are really high, and it really does matter which values we are giving to these machines and telling them

to emulate. So you could imagine companies making those values, but we know now that that's not a foolproof process. You can imagine users making these decisions about what values they want AI to have, but that has the problem that you just alluded to. There are other kinds of experiments that you can imagine running. What if you let people vote on how they want chatbots to behave? What if it was some sort of democratic process? But that has its own trade-offs too. So I don't think there's a perfect answer out there, but I'm positive that as these AI systems continue to get better and more capable, this issue of what values do they have?

Are they aligned with my values as a user? That's only going to become more and more pressing. And the pressure on companies like Google to please everyone with these chatbots is only going to get more and more intense. Well, Kevin, thank you very much. Thank you, Michael. The latest episode of Kevin's podcast, Hard Fork, which covers the world of tech, is out tomorrow.

To find it, search for Hard Fork wherever you listen. We'll be right back. Here's what else you need to know today. I am filled with the gratitude for the outpouring of support we've received from all across our great country. But the time has now come to suspend my campaign. Nikki Haley ended her campaign for the Republican presidential nomination on Wednesday, following a string of resounding losses on Super Tuesday that left her with no chance of stopping Donald Trump.

In 15 races across the U.S., Trump won all but one state, Vermont, bringing his delegate count up to more than 1,000. Haley, by comparison, has fewer than 90. In all likelihood, Donald Trump will be the Republican nominee when our party convention meets in July. But unlike nearly every other Republican candidate who has left the race, Haley chose not to endorse Trump. It is now up to Donald Trump to earn the votes of those in our party and beyond it who did not support him. And I hope he does that.

This is now his time for choosing. And the Houthi militia has claimed responsibility for an attack on a commercial vessel off the coast of Yemen that killed two people on Wednesday. The deaths are the first fatalities from Houthi attacks since the group began targeting ships late last year as a form of protest against Israel's invasion of Gaza. Today's episode was produced by Stella Tan, Asta Chaturvedi, and Mary Wilson, with research help by Susan Lee. It was edited by Brendan Klinkenberg, contains original music by Diane Wong, Marion Lozano, and Will Reed, and was engineered by Alyssa Moxley. Our theme music is by Jim Brunberg

and Ben Landsberg of Wonderly. That's it for The Daily. I'm Michael Barbaro. See you tomorrow.