The Vergecast - The ethics of AI with Google's AI lead Jeff Dean

Episode Date: June 4, 2019

What are tech giants like Google doing to tackle the ethical issues that surround artificial intelligence? Verge senior reporter James Vincent speaks with Google AI lead Jeff Dean and Verge editor-in-chief Nilay Patel about AI bias, facial recognition, and government regulation around AI. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Support for the show comes from Retool. Too many companies run critical operations on duct taped spreadsheets, Slack workflows, and whatever else they could cobble together. Not because they want to, but because building internal tools means weeks of waiting on someone else's backlog. That's where Retool comes in. Build custom internal tools just by describing what you need. Prompts something like,

Build Me a Revenue Dashboard on our Salesforce data. And Retool actually builds it on your company's data, in your cloud with enterprise security built in. Go to retool.com slash Verchcast. We all need to retool how we build software. What's up, y'all. I'm Skylar Diggins, seven-time WMBA All-Star, Olympic gold medalist, and mom. And I'm Cassidy Hubbard, host and reporter for nearly 20 years,

covering the biggest names and stories in sports and mom. And this is Am Mom, a community for athletes, game changers, and moms of all kinds. dropping May 14th. Tap in with us. Hey everybody, it's Neely from the Vergecast. This week's interview episode is a little bit different. We're trying to experiment.

I think it's going to go really well, but let me know what you think. I've got James Vincent with me. Hey, James. Hey, Nealai. How's it going? It's going great. James is our excellent AI reporter. And he interviewed Jeff Dean, who is head of Google AI, head of Google's AI research.

You might have seen Jeff on stage at Google AI. I was talking about AI. He actually ended the keynote. It was a big presentation at the end. So James interviewed Jeff a while ago, but AI moves so fast that there's been a bunch of things that have happened since that interview. So I've asked James to join us. We're going to run a bunch of clips from James' interview with Jeff Dean, head of Google AI. But because so many things have happened since they talked, I've asked James to sort of walk us in and out of the conversation and provide the newest information.

So James, let's start with the basics. Tell me a little bit more about Jeff Dean, who is, why it's important. Jeff Dean is a really big deal for numerous reasons. He's like an engineer's engineer. He has been with Google for a very long time, and he helped develop some of the technologies that help them scale in the very early days. And he's sort of essential to their growth, basically, and to their success. And recently, he was made head of AI.

So he oversees all of Google's AI products and services and their research as well. And, you know, as Sundara is always saying that Google is an AI first company now. So he's basically in charge of Google's entire future at this point. You know, he's the guy who is going to point Google in the direction that's going to continue, you know, securing its dominance in all these various fields for the next 20, 30, 40 years, however long they're planning. Yeah, and I did think it was interesting that at I.O. literally the entire keynote ended with Jeff Dean just explaining what he's doing with AI. So there are some big issues with AI, however. I think you report on them every day.

We talk about them all the time. They're quite obvious now. I think they've become sort of common knowledge. And the first big one is that when you train these models, they tend to reflect a lot of biases. And you have to counteract them in some way. Yeah, yeah. So AI bias has been like, you know, one of the big, worries and talking points about AI as it's getting integrated into all these real-life applications.

And I think it's like it's a really complicated topic. And it's even difficult to define what AI bias means neatly because there are lots of different ways to use the term bias. And you can use it in a statistical sense where it doesn't necessarily mean anything other than a sort of error in results that tend to skew towards a certain sort of error. But also bias has human meanings. It as sort of meanings in society. And I like to think of when we talk about AI bias, we're talking about an interception of these two things. We're talking about a technical error that reflects a societal prejudice or that embodies

a societal prejudice. So give me an example. Yeah. So think about the ways that algorithms are being used in the real world. So they're doing criminal sentencing. They're like judging creditworthiness. They're recommending medical treatment. So it's at these points where an algorithm, which has been trained.

on data is making a decision that affects someone's life. A fantastic example of this is within algorithms used to screen resumes or to judge applicants for a job. So there was this great story a while ago from, I think it was Royces who reported it first, from Amazon who had developed a tool, an algorithmic tool that they used to screen resumes. And this tool had been trained on data about the sort of people that Amazon likes to employ and who does well at the company and, you know, what their engineer is like already. It looked at all that data. It was like, great, we're going to use these things to learn what is a good fit for Amazon.

And then when it came to actually analyzing CVs and resumes, it turns out that it penalized women. If an applicant's CV said that they had attended an all-woman's college or if their resume even contains the word women's, as in, you know, it might appear in women's chess club or, you know, women's badminton team or something like that, their CV, their application would be downranked because of that. And that was because the algorithm had just looked at the data that had been presented with, which was a male-dominated world of computer science and engineering, and it reflected that data back. I should qualify that Amazon never launched this tool.

They saw these problems during the development process and they shut it down. However, that is exactly the sort of bias that does make its way. into live systems that are making decisions about everyone's lives. So you talk to Jeff Dean, who runs all of AI Google. What did he tell you? Jeff had a lot to say about this. It's a subject that Google are, you know, they're pretty forward-thinking on. They talk about it a lot.

And he told me that, as you would expect, bias is a big deal for Google. They do not want to have it in their systems. And they are developing all sorts of things to try and counteract it. They have their AI principles, which they kind of, you know, set out how. they want to implement these systems in the real world. And they also do some very good cutting edge research in terms of developing practical tools for engineers. The counterbalance to all this or the response to all this is, is that really enough? Just to give a sort of, I think this is a really telling of how these systems are working at the moment, is that yes, Google has its AI

principles, but Google published those AI principles after it came to light that the company was building tools for the Department of Defense using algorithms to help build systems that could analyze drone footage. And there was this huge outcry. This was the Project Mavin stuff. And there's a response to that Google was like, okay, we need to quote unquote do better and we can come out with these principles.

So I think there's this dynamic where these companies are doing stuff, but are they doing enough and are they only doing it when they get called out for it? Issues of bias and fairness in machine learning and AI are front and center because often what you do to solve a machine learning problem

is you take data from the world and you can collect some training data set for your problem and you train a model. And then you can maybe do that thing faster or more efficiently or hundreds of times per day instead of five. And the issue is that often the data you're training on reflects the world as it is, not the world as he would like it to be. Right? So for example, let's say you're training a model to predict who should get a home loan. And we all know that there's lots of kinds of biases in the whole loan process. And even if you don't include things like someone's race or gender in the input data,

machine learning models are very good at learning, you know, picking up on patterns. And so you can pick up on patterns of certain zip codes mean, I should say, no to a loan or whatever. Because that's what the data you're training. on actually showed. And so there's a whole sequence of research work. It's a whole line of research by the entire community really. And how do you actually take machine learning models and remove certain kinds of bias, but keep other kinds of bias.

Because you can't rob the models of all bias because that's sort of how a lot of their power. Is, you know, in a language model, you want to learn that the way that the way word surgeon is associated with scalpel and carpenter is associated with hammer, right? But you unfortunately learn that the word doctor is associated with he and the word nurse is associated with she in many cases because of the nature of the textual data that you're training on, doctors are more often referred to as he. I think Dean makes a really important point here. It's good bias and there's bad bias in a way, right?

So, like, the problem is, or a problem is that algorithms are reflecting disparities in the data and to a certain degree, or, you know, that's what an algorithm is supposed to do. It's supposed to look at the data, look for patterns in it, and then replicate those patterns. So that's just math doing what math does. However, the really tricky thing here and why AI bias is such a huge, messy, convoluted topic is like, are those biases, things you want or things you don't want? So what ends up being a sort of what starts as a technical solution, how do we make these algorithms work,

ends up being this huge question about how do we want society to work? You know, what judgments do we think these machines should be making? Every time this conversation comes up, it arrives at that point, right? Which is what you're really talking is about a reflection of society. Why is Facebook evil because it reflects society at large? Like so many conversations in tech and at this point of, well, we are just building a mirror to society, but there are some solutions to these problems. There are ways to move forward without having to wholly re-architect society, right?

I hope so. Yeah, yeah. We are not without recourse in this situation. As with Facebook, you know, there is stuff we can do and there's stuff that is being done. So, like, a big problem and an obvious problem to take on is data. But if you are training an algorithm for a facial recognition system, for example, and if you train it purely on white faces, If it has to identify or analyze a person of color, then it's not going to recognize that data.

You know, just on a pure training level, it just hasn't seen that before, so it won't know what to do. So that's a relatively easy problem because you just have to make sure that the data you're training the algorithm on is representative of the tasks you're taking on. But there are much more difficult problems that are to do with the sort of basic technical structure of algorithms. And one of those is that these black box problem, it's often called, which is that we can't get algorithms to explain why they make certain decisions. So if you have an algorithm that is supposed to be detecting lung cancer, how do you know what it is using to identify cancerous nodules in your lung scan? And so that's a really tricky thing. And that's something that engineers at places like Google and universities as well are trying to deal with. So this is what I asked. Jeff, I said, Jeff, you know, what are you doing about this at Google?

Here's what he said. Recognizing the bias can occur is a really important first step. There are algorithmic techniques you can do to say I would not, I would like everyone in these two different groups to have the same chance of achieving a certain outcome. All other things be equal. And then there's algorithmic ways you can adjust the output of the model to make that the case. There's some nice work by Moritz Hart and others in this space. So we build tools that help you visualize your data, your training data, understand what kinds of predictions your model is making.

But this is not a solved problem. This is an ongoing thing where when we use AI and machine learning today, we apply the best known techniques to remove bias. But we're also continually developing better techniques that enable us to do that. We definitely heard Sundar talk. at this at I.O. this year. This seems to be on everyone's mind. Yeah, 100%. Yeah, yeah. Sondar was talking about a specific tool that Google's been developing called TCAV or testing with concept activation vectors, which is lovely. Rolls right off the tongue. And this is all about approaching

this black box problem, which I mentioned. So it's like saying, when you have a neural network, which is analyzing a piece of data, which bits of the neural network are firing when it makes a decision. And then you can kind of dig into that like you would like a physical circuit board and you'd kind of trace where the activation juice was going through and sort of picking apart. So, you know, Google is building tools that help deal with this sort of problem. But the huge, the big question is like, is that really enough? You know, just because you have a tool that can work out how an algorithm made a certain decision, is that really going to change how companies behave, how governments behave?

Just explain this to me like I'm really dumb. Why can't the algorithm tell you why it made a decision? Why can't it say, here are the factors I weighed and I waited these ones highly, and this is the result? Oh, man. I mean, that's a tricky one to answer. And I would first off say that there are ways that algorithms can say that a little bit. There are tools like T-Calve and some other ones being developed in other places that do help with this problem. So the basics of machine learning is that you,

don't want to tell a computer explicitly want to do. You want it to work out what to do itself. So you feed it the data, you maybe put some labels on that data, and it will try and make connections between the data and the labels in order to make these decisions. So you might feed it. For example, you feed it a bunch of pictures of cats and dogs. It learns what a cat and a dog looks like. So in an old AI system or in that sort of traditional software, you might write those rules by hand. So you might say if it's got whiskers and a nice cute tail, oh, It's a little fluffy cat. Love it.

Great. You know, if it's got paws and it goes wuff, woof, woof, whiff, it's probably a dog. Fine. And what, obviously, scientists learned with this sort of thing, which is sometimes referred to as an expert system, is that writing down all these rules is exhausting and time-consuming and often doesn't work anyway. So instead of doing that, you have these systems, these architectures, neural networks that look at the data and make those connections themselves.

However, this is where the big problems start coming in. because they are teaching themselves, they are making these hugely complex, labyrinth connections, these mathematical connections deep within their structures, they're forming by themselves and digging into that is really, really difficult. There's no built-in mechanism for them to explain why they did that. That was never part of the brief when these systems were created. All the people wanted is for it to be good at making these decisions. You can kind of think of it like, it's A, because it's incredibly complex, and it's B,

because this was not part of the design brief. We just wanted decisions being made. We didn't need to know why they were being made. And now people are saying, well, actually, it would be really useful if we found out, you know. It would be great if you could tell us why you're going to jail. Yeah, 100%. Like, who doesn't want to know that? I mean, that does seem to be the end state, right?

The computer decides that you're a criminal and sends you to jail. And you maybe want to know why. Yeah. Can you re-architect the systems to make them more transparent? Yeah. And this is exactly what this stuff like TCAV and other systems like it are doing. They're sort of, you know, finding new ways to work out where the algorithm or the network is focusing its attention. So a lot of these approaches are like the testing with concept activation vectors.

They're saying, okay, so where is the machine looking in the case of visual data? So, you know, like, have you ever used one of those sorts of eye tracking softwares where you're showing a picture and it shows where your gaze went? Yeah. Yeah. So there are computer versions of that for machine vision systems where they look to see where the algorithm is looking. Again, if we go back to the cat, dog picture stuff, if your algorithm is looking at parts of the face, looking at whiskers, and it's looking at the tail, you'd go, okay, so it's

looking at the stuff that a human would look at. But if it's like looking at maybe you fed it pictures where all the cats were wearing bow ties, and your algorithm is looking around the neck of each animal to see whether it's wearing a bow tie or not, and that's how it decides what's a cat and dog, well, then you know you've made a mistake. And you really need to get a better data set, which isn't full of pictures of cats with bow ties, which is a shame. You're going to have to delete all those pictures.

Yeah, it does seem like the best possible data set. We're going to take a quick break for an ad. We'll be right back. Support for this show comes from Shopify. Every thriving, successful business has to start somewhere. A good place to start is a relatively simple question. What if, given the right tools, I've really put my all into this.

One tool that can help grow your sprouting business to new heights is Shopify. Millions of businesses around the world rely on Shopify for e-commerce. They offer a host of helpful tools you can take advantage of, from payment processing to analytics to website design. Their design studio includes hundreds of templates to help you create the exact website you've been envisioning for your business. If you're wondering, what if I need help? Then no worries, because you're never left to Fendphan.

yourself. Shopify's award-winning customer support is available 24-7. It's time to turn those what-ifs into a thriving business with Shopify today. Sign up for your $1 per month trial today at Shopify.com slash vergecast. Go to Shopify.com slash vergecast. That's Shopify.com slash vergecast. Support for the show comes from Grammarly. You don't need reminding that the world moves fast. But work today requires clear communication, and when every message counts, sounding rushed or generic can be getting lost in the shuffle. Gramerly gives you one place to think, write, and finish your work where you already write, while giving you access to agents that help you sound natural and engaging. No matter what kind of writing you're doing,

Gramerly helps you get ideas done faster and move from draft to done with less friction. You can use Grammally's AI chat to brainstorm ideas, outline a solid draft, then refine it with context-aware suggestions that fit what you're working on. See why 90% of professionals say Gramerly has saved them time writing and editing their work. In a world of generic AI, you don't have to sound like everyone else. With Gramerly, you never will. Download Grammarly for free at Grammarly.com.

That's Grammarly.com. All right, we're back with James Simpson. We talked a lot about recognizing pictures of cats and bow ties, but it seems like the scariest, most controversial use of AI right now is facial recognition. San Francisco is just bandit. You can get through an airport in China using just your face. The sort of scale of how it's being deployed is radically different depending on where you are. There's all kinds of different norms around it.

Is that coming up at the research level with the people that you're talking to? Yeah, I mean, so I think facial recognition is a really good example of all this stuff as well, because it's a something where there have been multiple studies showing how these systems have exactly these sorts of biases in them. And it's something where these companies are still pushing them forward all the same. So a lot of stuff has been done with MIT's Gender Shades Project, which is led by Joy Boulinni. And she and a lot of people who've been working with her have been doing really fantastic stuff, testing. these algorithms, for example, Amazon's recognition algorithm, which is one that is being sold to police officers, and showing how the systems that these companies are built just too before worse if you're not a white male. You know, it's really quite like, excuse the horrible

pun, black and white stuff, you know, like these are systems that do have biases in them and that yet are still being sold, that are still being used, that are still being incorrectly deployed. You know, Amazon's response to this is always well. You've tested our, our system's wrong and they shouldn't work like that in the wild. And we give police very specific instructions. But, you know, there was this big story out the other day by Claire Garvey from Georgetown showing that actually police have no rules with how they used this stuff. That was the, they had a suspect.

And the CCTV image of them was too blurry to feed into the facial recognition system. But one of the cops was like, you know what? Like that kind of looks like Woody Harrelson. So they just got a picture of Woody Harrelson from Google Images, drop that in instead and search for Woody Harrelson. Like, you know, it's real like wild, wild west stuff in terms of how these systems are being implemented. Did they capture Woody Harrelson? Yeah.

So the thing that really kind of undermines this story as an example of police abusing this technology was apparently they did capture it. Not Woody Harrelson. It did lead to an arrest, but he could not have done. And there's just a huge amount of the problem with taking that approach to this technology. Anyway. So this is one example. example where we know the technology has biases in it and it's still being used, it's still being deployed and there's no laws about it. You do have these odd occasions. You know, you brought up

San Francisco where the tech is being banned. And, you know, that's such a great example of, you know, the home of this technology where it's being developed and the people who know it best and they're like, uh-uh, we don't want to use it. It's all these tech CEOs saying, well, I don't let my kids on Facebook because we know what it's like. And the interesting thing is this back and forth is happening within tech companies. So Amazon sells facial recognition, but Google doesn't. And I asked Jeff about that. I said, well, what led to this decision not to sell facial recognition? Like a lot of technologies, facial recognition has a bunch of really good uses and some uses that are maybe a bit undesirable depending on how you use them. So for example, we do have

have facial recognition algorithms that we use in something like Google Photos. So we can identify that, you know, these seven pictures have the same person in them. Yeah. And then you as a user of the system can say, oh, yeah, that's my daughter. Please, you know, continue to find pictures of my daughter. And then when I search for my daughter photos of my daughter, I can actually see all the pictures of her. That seems like a really useful user benefiting use of face recognition. At the same time, we don't offer a general purpose facial recognition API because we think there are real some real downsides in terms of surveillance applications that could be built by third parties if we did offer that.

And that, to me, is the difference. If you offer a general API, you know, you really don't have very much control over how that API is used. Yes. And so for facial recognition in particular, because it's a sensitive area, you know, we've chosen not to offer that. And really, if you look at our AI principles, one of them that says we will not participate in is sort of surveillance-related efficacy. So Jeff is saying, hey, you don't need to worry about this potentially dangerous technology because we at Google know what's best and we won't sell it to anyone, even though a lot of other companies do. But there's just an explosion in self-governance inside of the companies, right? Microsoft has an AI ethics board.

Facebook has some grand vision of it. Google has one. Lay them out and then tell me if they're effective. Yeah. So the response to this, the current AI boom took off in 2012 website. In the years that followed, you know, obviously the experts and researchers knew about these biased problems much earlier. But as they became more mainstream knowledge and people like ourselves started talking about them, companies reacted by going, all right, we're going to do something about it. They set up ethics committees. They set up ethics boards.

They, you know, kind of published AI principles and pretty much any big tech company you want to name, you know, Google, whether it's Microsoft, Facebook, Amazon, IBM, whoever, they have done something like this. They've got an ethics board or they've got a set of principles. On one sense, this is good, obviously, that these companies are thinking about these problems and they are sort of, you know, hiring people to analyze them. But there's also been a little bit of a backlash within the AI ethics community saying that, you know, the term one academic I spoke to called Ben Wagner, he called it ethics washing, which is that the company set up these boards, they set up these committees, they publish these principles. So whenever someone says to them, it's like, why are you doing this? They go, oh, well, you know, we're really thinking about it. We are, we've got top men on

this, top men on this. They're looking into it. It allows. them to deflect this criticism and appear to be doing something while doing very little. The problem with these boards is they have no power. They can't veto decisions the company is making and they have no transparency. There was this Microsoft set up this board called AI Ethics Oversight Committee or Aether or something like that. It sounds like it came out of the Marvel Cinematic Universe. And they said in an interview that significant sales quote, had been cut off because of the group's recommendations. But they never said who the sales were to or what the applications were.

So all we have is Microsoft's word that it saw a bad thing and it stopped it happening. But we don't know anything more than that. And as we've seen with Facebook, do we really trust these companies to govern themselves? Right. Especially without any transparency into what they're doing or not doing. I think you see it now. Amazon employees and shareholders are. begging the company to not use recognition, which is their facial recognition system in certain

ways. And they don't seem to be getting anywhere with that protest. Yeah. I mean, so that's exactly the case. And this is where the problem or the challenge, the topic of AI bias ties into all these other trends we're seeing in Silicon Valley. It has gone right to the heart of what employees can protest and whether they can change their companies, their employers' actions on this.

You mentioned Amazon, but there's been similar agitation in different companies. And within Google, the Project Maven example as well. So this is something where employees are definitely agitating from within companies, but whether or not companies are actually going to listen to their employees is an open question. And I think, as we've seen so far, it's not really going the employee's way. So this brings up an obvious question. if we don't trust companies to regulate themselves, do governments need to do the regulation? You know, if we were, we were talking about facial recognition earlier.

This is a really interesting one. So Amazon sells it and it's been criticized for it. Microsoft has called for regulation and Google is refusing to sell it altogether because they think it has too many, you know, adverse uses. So Google is saying that here's a really dangerous technology that we could develop and we could sell and we're not going to. So they're saying you've got to trust us to regulate this stuff. So I asked, I asked, Beth Dean, I said, is it enough for us to trust companies or do we need governments to do them as well?

Because in their example, if they think that facial recognition is too dangerous for them to sell, why should they be happy that Amazon is selling it? I mean, we last year came out with a set of AI principles that we think is a pretty good list of things people should be thinking about as they think about applying AI and machine learning to different problems in the world. as well as a list of things we will not pursue because we don't think they're compatible with the values that we stand for.

And I think it was really important for us to come together as a company and have a crystallized list of these things rather than sort of vague notions of this because it really helped our thinking as we look at new applications of AI machine learning we can then look at how they are compatible with our principles or not.

So I think, you know, the reason we made those principles public is also because we think other companies and organizations who are starting to apply machine learning and AI to their own problems more can look at those principles and decide, you know, those look good or we like some of those, but our business is not necessarily such that we can adopt all of them or whatever. But I think it does start a good conversation. And I think the question around regulation of some of these things, there's already regulatory frameworks in place for many things, like medical devices and drugs have a fairly strong regulatory regimen in most countries already.

And, you know, the use of AI and healthcare and medicine, the current regulatory frameworks is sort of a good starting point. They might want to be adjusted a bit for things that are more algorithmic rather than a sort of a pharmaceutical pill and so on. But I think, you know, that probably makes sense. In other places, I think you want to interact with governments and policymakers to help them understand, you know, what is the technology, what are the risks of it, what can it do, what can't it do, what should they be thinking about.

That's super interesting, right? In the market, this company says this is too dangerous. We don't want to sell it. Another company says, we think it should be regulated by the government. And then a third company says, we're just selling it. And anything could happen. And there is no overarching regulation for this stuff outside of a few places like San Francisco where certain agencies can't deploy it.

But there is also the challenge of even the regulation that we have, which is this self-regulation of an AI ethics board, has been mired in controversy, particularly at Google. So this is the big thing that happened since you talked to Jeff Dean, was Google attempted to set up an AI advisory board, an ethics board. And it quickly fell apart. Just walk us through that. Yeah. I mean, so this was something I would have loved to ask Jeff about in person and didn't get the chance to. And it sparked a lot of this backlash that I kind of discussed earlier where academics are saying that companies just aren't doing enough to regulate themselves.

At the end of March, Google said, you know, we want to keep AI ethical. We want to do this all above board. So we're going to have a new advisory for the new advisory committee. They announced this group called the Advanced Technology External Advisory Council. So again, it sounds very grand. And, oh, you know, they're really taking this seriously. And it combusted in less than two weeks, I think. They announced it.

Big fanfare, nice little blog post. It's all going very well. and then they shut it down less than two weeks later. So the reason for this was they'd assembled a group of experts, and some of these were sort of academic, some of these were from private companies, and one of the individuals was the president of the Heritage Foundation, K. Coles James.

So the Heritage Foundation is a conservative think tank, and a lot of its policy positions are exactly the sort of stuff that Google is we're not happy with. This includes stuff like climate change denial and anti-trans rhetoric from Kay Coles James herself. So when Google announced this board, there was a huge outcry. And very soon there was a petition circulated inside Google for James to be removed. I think it got signed by just over 2,000 and a half thousand people in the end. And also one of the academics who had been pointed to the board, resigned as part of this because they said, you know, I don't want to be a part of this initiative.

Google kept very quiet about this. This backlash sort of bubbled over. I'm sure they were just sort of waiting to see where it would go, whether it would go away. It obviously wasn't going away, so they shut it down. And I would have liked to have asked Dean about it, but had no chance, but sent Google a question about it, and they sent me this statement. It become clear that in the current environment, A-T-E-A-C, which is the board, can't function as we wanted. So we're handing the council, going back to the drawing board, will be. will continue to be responsible in our work on the important issues that AI raises, and we

will find different ways of getting outside opinion on these topics. So it's a very little comment at all in that, basically. There's nothing more reassuring than a promise from one of the world's largest companies that they will continue to be responsible. Yeah, exactly. Because we know, we know these companies are incredibly responsible. I mean, I just can't fathom why they didn't see that this was going to be objected to. Oh, I can. Not to be overly charitable, but if you're putting the other board that it's supposed to generate some policy by which your work will be regulated that might turn into some actual government regulation down the line, it is not entirely surprising that you would go seek a conservative viewpoint for that. I think that they sought the specific viewpoint that they arrived at, they should have seen that coming. But the sort of broader idea that this needs to be across the spectrum of viewpoints before we build a law that governs. what seems to be just a title wave of technological progress, I see it. And I don't want to be too charitable, but you certainly see why they were like, well,

we need someone from the left and someone from the right. You know, like you see how you would build that to sort of insulate yourself from criticism and then the actual specific of choosing the people open them up to a wellspring of criticism that they clearly did not anticipate. Yeah, yeah. I just think it's such a good example of when these different interests within companies clash. because, you know, one of Google's principles is, like, to respect good science. You know, it's part of their AI ethics principles.

And yet they would be happy to have someone advising them who doesn't respect good science because they downplay the threat from climate change. And I just think, you know, obviously, sometimes you need to take a stand. You need to say that, you know, this is this. This is really happening. But see what you mean about trying to curry favor in certain sectors of society. Yeah, I mean, there's no doubt in my mind, at least, that, part of the reason you set up these boards

is say, oh, we've already done the work. Now, just pass the law that we've proposed. We've already done the work. It's a bipartisan group that we've assembled. Take our recommendations and turn them into the law. That'll be fine. You don't have to think about this too hard. Is that the right move? Certainly not.

And is there even consensus, I think, is something that you talked about with Jeff, where some of Google's own researchers were working on outside boards developing proposals that he doesn't agree with. Yeah, so one of the interesting and sort of troubling fallouts from this was a report that was originally wired that was connected to this, but also to the Google walkout, which ended in Google ending the policy of forced arbitration for sexual harassment claims. One of the leaders of that was Meredith Whitaker, and the other was Claest Stapleton,

and there was historian-wired about how they had been penalized for their work in this. It is not clear exactly what has happened since. Google obviously denied that there isn't any internal retaliation. But at the time, Whitaker, for example, was told that she would have to, quote, abandon and quote, her work on AI ethics, including her work with an institution called AI Now, which is attached to NYU. And there's absolutely fantastic research on all the problems we've been talking about. And this, for me, is very, very, very troubling that Google, these reports were entirely true. You know, Google's obviously denied that there was any in retaliation, so whatever. But that Google would try and retaliate against someone who is doing exactly the foundational work in mitigating these harms that they say they care so much about.

So, James, it seems very, very complicated to me. I have to be honest with you. The number of competing interests, problems, solutions, and people offering those solutions does not seem in any way simple at this point in time. No, it's not. It's hideously complicated. And I'm going to say something that's going to make me sound like a complete idiot, but like I've been working as a technology journalist for a while.

And it is all politics. It's just, it always comes back to politics. If you want this stuff to work correctly, it can never just be a technical solution. it has to be a wider conversation with the public about what they want to happen, and that has to be political involvement. How do you see that conversation taking place right now? Is it happening in a way that is productive, or does that need to change? It's happening, but it's happening slowly.

I think the thing that is often missing is that when the systems, say an algorithmic system is integrated into a certain decision-making process, you need to have experts within that original domain, so not people who are experts in AI, but people who are experts in, you know, I don't know, criminal sentencing, in whatever situation it is, and they need to be giving feedback about these algorithms. So for that to happen, you need broader education and you need to have money in place to make sure that these checks are being made, and we're not just handing this up over to the people who design the algorithm because they can't do it by themselves.

So if you're a Vergecast listener and you are interested in this stuff and you're very very interested in how technology and society and culture interplay with each other, and it seems AI is the center of that. What should you be watching? How do you pay closer attention to this? I mean, I would look at how the fallout to the San Francisco facial recognition plan plays out over the next probably years, to be honest, and look at how politicians are trying to formulate tactics to deal with this. Yeah, I don't know. You know, read good news sites. Is that my I like to say that? Yeah, I'm like to say that.

You know, there's so much interesting and fantastic conversation and discussion happening about these topics that, you know, if you're interested and you want to find out more, you can really dive in. All right. Well, James, where can they follow you? Because I know you are constantly reporting on these things. Yeah, sure. If you want to follow me, you can follow me at JJ Vincent on Twitter or just, you know, head to www.

That's good. That's a good plug. Thanks. I appreciate it. All right, James, thank you. No problem. Cheers, Natalie. Thank you so much.

We'll talk to you soon. Thanks to senior reporter James Vincent. You can check out all of his reporting on AI and more at the verge.com, also his Twitter, at J.J. Vincent. I want to let you know that why'd you push that button is off and running. They have a big series called Death Online starting next week. It's a three-part series about death and the internet. We're really excited about it. This week, I have an episode about whether quitting Instagram can make you happier.

Caitlin quit Instagram for a few weeks to see for herself. You just have to listen and find out. You'd also subscribe to the Virgin. for free in your favorite podcast app. You can just tap the link in the show notes to get new episodes. You can also subscribe to the virtual cast for free in your favorite podcast app. You can tap the link in the show notes to get new episodes. And please leave a rating interview on Apple Podcasts.

Again, the producers are asking me to say Apple Podcasts. That's where they want you to go. So please go there. Hit me up on Twitter. I'm at Reckless. Let me know who you want me to interview next. I love hearing your suggestions. We're pulling as many people as you can.

We'll be back on Friday with Dieter and Paul for the regular chat show.