There Are No Girls on the Internet - Biden’s executive order on AI protects privacy and boosts inclusion. Thank Black women like Dr. Joy Buolamwini.

Episode Date: November 2, 2023

Dr. Joy Buolamwini has been raising the alarm about the harm of AI for years, through her research and advocacy. People are finally starting to listen. Dr. Buolamwini is a 2020 Vital Voices Global Lea...dership Awards honoree and 2023 featured presenter.  Learn more about the Vital Voices Global Leadership Awards: https://www.vitalvoices.org/news-articles/news/honorees-22nd-annual-global-leadership-awards Check out Dr. Buolamwini’s new book Unmasking AI: My Mission to Protect What Is Human in a World of Machines: https://www.amazon.com/Unmasking-AI-Mission-Protect-Machines/dp/0593241835 Dr. Buolamwini runs the Algorithmic Justice League. Find out more about her work: https://www.ajl.org/See omnystudio.com/listener for privacy information.

Transcript
Discussion (0)
Starting point is 00:00:00 This is an I-Heart podcast. Guaranteed Human. Another podcast from some SNL, late-night comedy guy, not quite. Unhumor me with Robert Smygel and friends. Me and hilarious guests from Bob Odenkirk to David Letterman help make you funnier. This week, my guest, SNL's Mikey Day and head writer, Streeter Seidel, help an a cappella band with their between songs banter.
Starting point is 00:00:23 Where does your group perform? We do some retirement homes. Those people are starving for banter. Listen to humor me with Robert Smigel and friends on the IHeart radio app, Apple Podcasts, or wherever you get your podcasts. Run a business and not thinking about podcasting, think again. More Americans listen to podcasts than adds supported streaming music from Spotify and Pandora. And as the number one podcaster, IHeart's twice as large as the next two combined.
Starting point is 00:00:46 Learn how podcasting can help your business. Call 844-844-I-Hart. What's up, fam? It's Isaiah Thomas. And I'm C.J. Toledano. It's our favorite time of the year on our podcast point game, the playoffs. We're digging into the biggest surprises of the season. And I'm looking back on some of my greatest playoff moments. If we didn't talk ever again, I was harmed.
Starting point is 00:01:04 You just understood. That's how personal it got. Wow. Then after that game seven, Marquis come in to you, he's like, you know, I love you, dog. You know, it's all love. This was just playoffs. This was just basketball. So listen to Point Game on the Iheart Radio app, Apple Podcasts, or wherever you get your podcasts.
Starting point is 00:01:21 Justice looks like how do we use these tools in a joyful, uplifting manner, versus just being reactive to the next harms. There are No Girls on the Internet as a production of IHeart Radio and Unbossed Creative. I'm Bridget Todd, and this is There Are No Girls on the Internet. This week, President Biden signed an executive order to create some safeguards around the use of AI. This comes after black women, women like Dr. Joy Blomweeney, founder of the Algorithmic Justice League, and author of the new book, Amasking AI,
Starting point is 00:02:03 have been speaking up about the ways that technology like AI has already harmed marginalized communities and what needs to be done to stop it. Now, that last part is key, because even though her groundbreaking research has been critical to understanding technology harms, Dr. Blumweeney's vision of the future of technology is optimistic, blending poetry and technology. She asks, what is our collective just and joyful vision for the future? Hello, my name is Dr. Joy Blumweeney. I'm the founder of the algorithmic Justice League and the author of Unmasking AI. My pronouns are she and hers. So I've heard you call yourself a poet of code, which is awesome. What do you mean by that? So I am the daughter of an
Starting point is 00:02:52 artist and a scientist. So I do feel I've grown up with the arts and science literally together. And so when I use the term poet of code, it's really to reflect those two sensibilities which inform my work. So there's a major part of it, which is storytelling and humanizing what's going on with evocative audits, you know, and portrayals. And then there's another aspect of it that is getting into the analytical, technical pieces of what it means to evaluate machine learning system or other types of AI applications. So the poet of code is very much, indicating my origins as the daughter of an artist and a scientist. Do you think that that the way that you approach the work has really helped bring more folks into it?
Starting point is 00:03:47 Because I have been interested and invested in conversations about tech for a very long time. I did not care about slash maybe even fully understand the implications around bias and things like AI until you. And so you had this way of really making it visible, really. making it poetic, really making me understand what was at stake. Do you think that part of why that is why folks feel so drawn to your work is because you make it so poetic, so, you know, story-based, really help people understand, like, where they fit into it? I think there is that element. So as I was doing my research at MIT, that involves publishing research papers. And as fun as those are, you know, that's a very small community that will likely read those types of papers.
Starting point is 00:04:37 So I wanted to say, how do I go from the performance metrics of evaluating an AI system to something like performance art? Why does this even matter? How do we get to the heart of all of these numbers? So if we see bias in a system and we quantify it, that's only part of the story. The other part of the story is what does that mean for someone? who could experience algorithmic discrimination, algorithmic erasure or exploitation. And that's where the storytelling has to come in.
Starting point is 00:05:11 And it did for me when I was a student at MIT, I had an opportunity to do research that showed large skin type gaps and gender gaps with the accuracy of different gender classification systems. So these are AI systems that look at a photo of your face and try to guess your gender. Where could that go wrong? Well, so we decided to do a bit of an evaluation. And after we ran the numbers and we showed there were large gaps and biases documented, I wanted to show people why it mattered.
Starting point is 00:05:50 It does matter to all of us. Whether you spend a lot of time thinking about it or not, this kind of technology is becoming more and more commonplace, despite the fact that it doesn't work so well on women or people with darker complexions, setting us up to disproportionately experience harm from its use. In gender shades, Dr. Blomini's groundbreaking research, she was among the first to uncover the gender and racial biases that plague facial recognition technology.
Starting point is 00:06:16 But, ever the poet, Dr. Blomini's spoken word poem, AI, ain't I a woman, really brings the problem to life, where AI misgenders and misidentifies famous black women. in history, like Michelle Obama, who facial recognition recognizes as a young man wearing a toupee. And so from that Gender Shades Research Project came the art piece that is AI, ain't I a woman. Michelle Obama unabashed and unafraid to wear her crown of history, yet her crown seems a mystery to systems unsure of her hair, a wig, a buffon, a toupee, maybe not. Are there no words for our braids and our logs where I show tech companies you've probably heard of failing on the iconic women of
Starting point is 00:07:03 people like Oprah Winfrey, Serena Williams, Michelle Obama, historic figures like Surgeoner Truth, hence the title AI A&A, A Woman. And I saw that when I shared that poem, it's a video poem in all kinds of spaces, right? You defense ministers, you know, kids in middle school, it touched people's humanity in a way that the research couldn't. And that for me was really an important moment because for a long time, I felt that I couldn't bring my art into my research because it might not be taken as seriously or it might lessen its impact. And I found just the opposite. When you humanize what's going on, it extends the reach of the people who feel they have a place in the conversation about AI,
Starting point is 00:08:02 or even like, oh, this is how it could matter to me, not some abstract. Oh, there's discrimination or tech can be harmful. These harms aren't abstract or theoretical. They're very real and they're already happening. We talked about Portia Woodruff on the podcast before. She was heavily pregnant when she was falsely arrested and held for hours and needed to be hospitalized after being falsely arrested when police facial recognition misidentified her as a suspect that a carjacking she had nothing to do with. And she's not the only one. Back in 2020,
Starting point is 00:08:36 Robert Williams, a black man, became the first documented case of a person being falsely arrested thanks to the use of faulty facial recognition technology. Robert was arrested in front of his daughters after facial recognition mismatched his driver's license photo to someone who stole watches from a Shinola store in Detroit. But Robert had nothing to do with it. Tools like Turn It In that are used to detect students cheating by turning in AI-generated assignments routinely falsely accuse students of plagiarizing.
Starting point is 00:09:05 According to the markup, the technology is much more likely to generate a false positive for international students and students who are non-native English speakers. A group of Stanford computer scientists found that seven different AI detectors flagged writing by non-native speakers as AI-generated 61% of the time. On about 20% of the papers, that incorrect assessment was unanimous. Meanwhile, those same detectors almost never made such mistakes when assessing the writing of native English speakers.
Starting point is 00:09:34 Obviously, these kinds of accusations could throw vulnerable students' academic careers into turmoil. The people like Portia and the international students speak up when they've experienced harms because of faulty technology. So are the powers that be listening? Do their experiences matter as much as the companies trying to make money from rolling out this technology do? But an example like Portia Woodruff, falsely arrested due to AI-powered facial recognition, she was eight months pregnant, sitting in a jail cell having contractions for a crime. Now she was being held for crime she didn't commit. And so when you hear those stories, the stories of who I like to call the X-Coded,
Starting point is 00:10:16 you start to pay attention, right? Or maybe it's your kid and they got flagged for cheating. Turns out they didn't actually cheat. English is their second language, but some chat GPT detection system, right, is flagging them as cheating. And so I do think those stories are what helps people see that this is a conversation that requires their voice.
Starting point is 00:10:42 And it's so easy to think it's, like I have a PhD from MIT, but I was doing all this before, right? You don't have to have this type of indebt. technical background to have a voice and to have an important perspective. Because if you know you're being hard, that's enough. Yeah, a big part of what we aim to do here is to help people understand that you might not be an engineer, you might not have a doctorate, but you are the expert of your experience and you use this technology every day or it's being used
Starting point is 00:11:11 on you. And so you innately have a perspective that is valuable and worth sharing and worth hearing and worth centering about how that technology has impacted you. Let's take a quick break. Another podcast from some SNL late night comedy guy, not quite. Unhumor me with Robert Smygel and friends. Me and hilarious guests from Jim Gaffigan to Bob Odenkirk to David Letterman, help make you funnier.
Starting point is 00:11:45 This week, my guest, SNL's Mikey Day and head writer Streeter Seidel, help an a cappella band with their between songs banter. There's the worst singer in the group. The worst? Yeah. Me. Is there anything to the idea that because you're from Harvard, you only got in because your parents made a huge donation.
Starting point is 00:12:04 The yard birds, right? That's the name. The Harvard Yardt Yard's, but they're open. Do you have a name suggestion? We're open. Since you guys are middle-aged, one erection. Listen to humor me with Robert Smigel and Friends on the I-Heart Radio app, Apple Podcasts, or wherever you get your podcast. Humor me.
Starting point is 00:12:28 I need some jokes to make me seem funny. Run a business and not thinking about podcasting, think again. More Americans listen to podcasts than ads supported streaming music from Spotify and Pandora. And as the number one podcaster, IHearts twice as large as the next two combined. So whatever your customers listen to, they'll hear your message. Plus, only IHeart can extend your message to audiences across broadcast radio. Think podcasting can help your business. Think IHeart.
Starting point is 00:12:54 Streaming, radio, and podcast. Call 844-844-I-Hart to get started. That's 844-8-4-I-Hart. What's up, fam? It's Isaiah Thomas. And I'm C.J. Toledano, and our podcast, Point Game is about defying the odds. Like LeBron heading into the playoffs without Luca and Austin Reed. And finding ways to win no matter what. He's the smartest player to ever play the game.
Starting point is 00:13:15 His IQ is at a level that we've never seen before. And he knows. Without Luca and Austin Reeves, I got to manipulate the game. We get a player's perspective on the challenges of the play. playoffs. I think Joker's going to be exhausted this series because when they don't have Rudy in the lineup, he has to really guard guys like Nas Reid. He has to guard Julius Randall. And then he has to give us everything he gives us on the night-to-night basis on offense. And when IT's friends stop by, like Quentin Richardson, we dive into some playoff history too. Steve Nash will get that thing. That man, hell get the flying. He running up the court licking his fingers why he got the bar like,
Starting point is 00:13:52 after you go through a training camp with that, I said, you figure it. it out real quick. Oh, yeah. Get your ass up and down the court, and you're going to get the ball. So listen to Point Game on the IHeart Radio app, Apple Podcasts, or wherever you get your podcasts. And we're back. People who are traditionally marginalized, like women and black women, are made invisible
Starting point is 00:14:19 by technology every single day. We're faced with silencing, erasure, and hostility. So is that one of the reasons why the technology that these spaces build also can't really see us? Do you ever feel, I mean, this, stay with me here, I think that there is like a general hostility toward marginalized people, like toward black women. And I think in technology. And I sometimes feel that the technology that is being made in turn mirrors that same hostility, mirrors that same erasure. And so because these facial recognition is not being tested or trained on enough, you know, diverse data sets or whatever, in turn, erasure.
Starting point is 00:15:00 us. Do you feel that there's like a, that is kind of because of this underlying hostility toward people who are traditionally marginalized in the space? I actually don't think it's a intentional underlying hostility, which makes it even more dangerous. So well-intentioned people, right? Collecting data, doing what they think is good science, good machine learning can still create harmful systems. And this is what I learned after we did different audits and I would go talk to the teams behind some of these systems, right? They were nice people. You know, try to send the kids to college. And it was actually interesting to me because as much as I was wanting to humanize the people who are ex-coded, who are harmed by
Starting point is 00:15:55 AI systems. Part of what I try to do in the book as well is also humanize the people who are creating the systems and where things go wrong. But to your greater point, it's not an intentional hostility. Sometimes it is profound and harmful ignorance to not even think to ask certain questions or to test the system in particular ways. That's part of what the research was about. We asked, what happens when we put an intersectional lens on the way in which we analyze the performance of AI systems? And just doing that, right, opened up new areas of conversations where before people would just look at the overall score. And that gave us a false sense of progress within the space because we were testing these systems on benchmark data sets to see how well they do. And then you would look at the benchmarks.
Starting point is 00:16:51 Some of the benchmarks would be over 80% lighter-skinned individuals, over 70% people identified as men. And if that is your benchmark of success, you're already not going to see how you're failing. And so when we created a more inclusive data set, et cetera, it allowed us to see that the promises of potentially well-intentioned people weren't even panning out. But there's even more to that because I think with some of these conversations, it can seem like it's a very technical problem with a technical solution. Data didn't detect, you know, system didn't detect the face. Make it more inclusive, whatever else it is. But the problem is accurate systems can be abused. We've seen facial recognition systems deployed at protests, right, which we know.
Starting point is 00:17:48 can lead to chilling effects if you know you're under surveillance for daring to exercise your First Amendment rights to say this is not correct. Accurate systems create tools for state surveillance. So yes, you can say, well, my phone tracks me to them the other. You can leave your phone at home. Your face is a little bit harder. I mean, you know, some people do put on a face, but you know what I mean. You know what I mean, right? So, I think it's important to understand that even when we have conversations about the accuracy of certain systems and we should have those conversations, accuracy is not enough to assure accountability or equity.
Starting point is 00:18:35 Now, when we're talking about accountability, especially from tech companies, it is so easy to get caught in a cycle of name and shame, where you point out all the bad things that a specific company has done. And if I'm being honest, I might have done that a time. or two on this very podcast. But Dr. Blumweeney describes their method as less name and shame and change and change. They want to show companies what they're doing wrong so they can change for the better. But this hasn't always meant that those companies don't lash out when her teams point out the harm that they've caused. I'm curious, how have companies, I won't say any names, but companies
Starting point is 00:19:12 who you have called out in your research or said like, this is, hey, this is what's going on. How have they responded to your findings? So overall, I take a name and change approach. So the point of pointing out what's wrong isn't to shame a company is to say we can do better. Right. And sometimes we have companies that are reactive. We have companies that are proactive and we have companies that are combative. With the first set of research results that we released, we saw more of the reactive stance, which is, oh, now that there's a headline, right?
Starting point is 00:19:56 We're going to go, we are on the problem, or we were already working on the problem. You know, there are different ways. But now it's a priority because it's making headlines. So I saw that. And the reactive approach tended to be a technical approach. which is, okay, there were these disparities, so let's close them. We now have more accurate XYZ. Again, accurate systems can be of use.
Starting point is 00:20:23 Then we did experience some combative responses, right? So here we had a huge tech company coming out and saying, your research is misleading, attempting to discredit the research of a, at that time I was a graduate student. And I was so fortunate, you know, that I had senior scholars and people well respected in the AI industry who came to our defense, you know, cheering prize winner, somebody who was literally the chief AI scientist at that company saying what the research shows warrants our attention. And this is research we should be elevating, not dismissing because it makes the feel better as a whole. hold if we can acknowledge our limitations, understand what's going wrong so we can build more robust systems. Because this doesn't just deal with faces, right? If you want to use computer vision to help, let's say, with medical diagnoses, you want to make sure you understand where things can
Starting point is 00:21:30 go wrong so we can course correct for things to go right. So we had the combative approach, the reactive approach. But the approach I appreciate most is the proactive approach. Okay. We've heard there's some issues instead of waiting for someone to drop the paper or the headline, what can we be doing as a company now? And I've had the opportunity to work with Procter & Gamble with OLA on the Decode the Bias campaign. And when they came to me and they asked for an algorithmic audit, I said, given what you've described, your tool does, and this was a tool that would analyze your skin and give you product recommendations.
Starting point is 00:22:12 and how you train the tool on a set of data. I suspect if we dig in there, we're going to find some bias. They're like, that's okay. If you find bias, we'll do what we can to correct it. I was like, and if you can't correct it, we'll shut the system down. I'm like, can I get this in writing? I never hear this. I never hear this.
Starting point is 00:22:33 And then my other question was, if we did an audit, could we publish the audit results? because that adds another level of transparency. So it's not just, oh, we got checked, but no one knows what happened, right? They agreed to all of those things. We did, in fact, find bias as we thought would be there. And they actually, in the proactive, not only that they seek to be audited, they also agreed to a consented data promise. And this was inspired by their skin promise.
Starting point is 00:23:05 So when I first started working with LA, I was excited. And then they told me, you know, when we do the campaigns, there won't be any post-production airbrushing. What we capture is what we'll show, right? You know, truth in advertising. You want to think about people's body image and all of that. And I'll be honest, I was a little disappointed because, like, if you have, you just want to know you could be saved by the airbrush, right?
Starting point is 00:23:34 But because of that promise, which is a good promise, I get where they're coming from. As the person on the other side of the 4K camera in your face, you're asking, can we consider XYZ? But what I appreciated about that is it made me even more disciplined with my actual skin care regimen. And I also drank water. And I did all the right things for vanity reasons. I won't laugh. I did the right things for vanity reasons. But I think about that with, so that skincare, that promise, right, that was part of the inspiration for the consented data
Starting point is 00:24:19 promise. And just like when you make a promise and it's a public commitment, that's the important part. Now there's a little bit of accountability, right? So now you are going to bed early. Now you are drinking more water. Now you are exercising five days of the week, which might not have been the case before you made that public. So that's an example of more proactive. So from the reactive to the combative to the proactive, we've seen it all. I think what I learned most from the combative response was how much of a risk I was taking as a young researcher. to not only do the research, but to name the companies, and I'll name them now, right?
Starting point is 00:25:08 That I tested IBM, that I tested Microsoft, that I tested Amazon. And because of their power, that meant I was risking future opportunities. And also other researchers watching how I was treated, how my co-authors were treated, People like Dr. Gavru, they were also giving a sense of what is possible. When I look at research papers now where people openly talk about algorithmic bias and algorithmic harms and people openly name the AI models or the tech companies, that wasn't always the case, right? A price was paid for this more robust conversation to happen. Dr. Blomini is right.
Starting point is 00:26:00 This all comes out of cost. Dr. Temnakabrew, who she mentioned earlier, was a co-author on Dr. Blomini's gender shades research. Dr. Gibru was once the technical co-lead of the Ethical Artificial Intelligence Team at Google. While in that role, she worked on a paper about the risks of large language models from environmental impact to bias. Google demanded she withdraw the paper. It got contentious.
Starting point is 00:26:24 The conversation was hostile and the whole thing was highly gendered and racialized. Dr. Cabrew was belittled, discredited, and harassed online, and it ended with her termination. It cost us something. For Dr. Gabru, you know, it cost her her job to speak up when she saw some of the issues that we see in what they call large language models, the type of AI systems that will power chat GPT. Right.
Starting point is 00:26:53 And so I do think the timing of the different types of company responses also made a difference in my own trajectory. The first response I had when the Gender Shades paper was published was IBM invited me to their headquarters. You know, I spoke to their team members. They actually had released a new model by the time I was presenting that research. And I could share what their results had been. and then later we did our own study. So that was a very different reception gave me hope. I was like, okay, all right, let's work together.
Starting point is 00:27:31 Amazon situation, I don't know. I don't know about corporate anymore sort of thing. But I'm putting these as more extreme cases. But the point being, we can't really just wait on if a company is going to choose to be reactive, proactive, or combative. What we really need are laws and regulations that don't rely on the goodwill of companies. Yeah. I mean, I have to ask when you were this young researcher naming these companies in your findings,
Starting point is 00:28:06 did you know that you were taking on such a personal risk? Or were you like, oh, wait, glad it worked out, glad people had my back. Did you know that that was a risk that you were incurring and did it anyway? Or did you sort of, do you sort of look back and think like, wow, I'm really glad that worked out? I knew that once the research was published, it would be questioned. So before it was published, I actually sat with a law clinic, right? We went through what could be said, right? What might actually put you in legal jeopardy and so forth?
Starting point is 00:28:43 So I didn't go into the situation not thinking there might be blowback. I was actually surprised with the first round, we prepared. And they're like, oh, yeah, okay, these are issues. Come to the headquarters, XYZ, we've released new models, etc. The blowback that I got with the second paper is what I had thought I might experience, but just the magnitude of it I wasn't ready for. I remember with the film coded bias available on Netflix, it shares part of the story of,
Starting point is 00:29:18 Graduate students are an algorithmic Justice League and examples of people experiencing real world AI harms. The people to provide the insurance for that film were nervous because we critiqued Amazon. It wasn't that Amazon had said anything. It was just an acknowledgement of Amazon's power. Right. And so it didn't dawn on me just how powerful some of these things. tech companies are. I remember being at an international summit in Switzerland. And it was as if the heads of the tech companies were heads of state. You know, and so observing that closer made me real. I was like,
Starting point is 00:30:09 oh, I'm like, okay, I'm poking a dragon. It's like, okay, I'm poking a dragon. Oh, it's a fire breathing dragon. Oh, it's like a dragon dragon. And look when you go to kill a bug and you're like, oh, it's got wings, it flies. Right. So I knew, like, you know it's not going to be the best situation, but I don't think I was fully prepared, though I thought I had prepared. More after a quick break. Another podcast from some SNL late night comedy guide, not quite. Unhumor me with Robert Smygel and friends, me and hilarious guests from Jim Gaffigan to Bob Odenkirk to David Letterman, help make you funnier.
Starting point is 00:30:55 This week, my guest, SNL's Mikey Day and headwriters, Streeter Seidel, help an Acapella band with their between songs banter. There's that worst singer in the group? The worst? Yeah. Me. Is there anything to the idea that because you're from Harvard, you only got in because your parents made a huge donation.
Starting point is 00:31:14 The group. The yard birds, right? That's the name. The Harvard yard, but they're open. Do you have a name suggestion? We're open. Since you guys are middle aged. One erection
Starting point is 00:31:26 Listen to humor me with Robert Smygel and friends On the IHeart Radio app, Apple Podcasts, or wherever you get your podcast. Humor me I need some jokes to make me seem funny Run a business and not thinking about podcasting, think again. More Americans listen to podcasts than ads supported streaming music from Spotify and Pandora And as the number one podcaster, I hearts twice as large as the next two combined.
Starting point is 00:31:53 So whatever your customers listen to, they'll hear your message. Plus, only IHeart can extend your message to audiences across broadcast radio. Think podcasting can help your business. Think IHeart. Streaming, radio, and podcasting. Let us show you at iHeartadvertising.com. That's iHeartadvertising.com. What's up, fam?
Starting point is 00:32:12 It's Isaiah Thomas. And I'm C.J. Toledano, and our podcast Point Game is about defying the odds. Like LeBron heading into the playoffs without Luca and Austin Reed. And finding ways to win no matter what. He's the smartest player to ever play the game. His IQ is at a level that we've never seen before. And he knows without Luca and Austin Reeves, I got to manipulate the game.
Starting point is 00:32:33 We get a player's perspective on the challenges of the playoffs. I think Joker's going to be exhausted this series because when they don't have Rudy in the lineup, he has to really guard guys like Nas Reid. He has to guard Julius Randall. And then he has to give us everything he gives us on the night-to-night basis on offense. And when IT's friends stop by, like Quentin Richardson,
Starting point is 00:32:52 we dive into some playoff history too. Steve Nash will get that thing. That man, hell get the flying. He running up the court, licking his fingers why he got the ball. Like, you go through a training camp with that, Isaiah. You figure it out real quick. Oh, yeah. Get your ass up and down the court, and you're going to get the ball.
Starting point is 00:33:10 So listen to Point Game on the IHeart Radio app, Apple Podcasts, or wherever you get your podcasts. Let's get right back into it. Even though Amazon tried publicly discrediting Dr. Blomini's work, calling out the harms of their facial. recognition technology. In the end, they conceded that technology wasn't exactly safe. In 2020, they announced a pause on allowing police to use the technology and eventually extended that pause indefinitely. And correct me if I'm wrong, but your work ended up with Amazon rolling back some of the uses of their faulty facial recognition technology. So ultimately,
Starting point is 00:33:53 not only were you obviously vindicated, but that work went on to create a somewhat like safer landscape for everyone because Amazon had to step back and be like, okay, wait a minute, this technology maybe isn't really working that well. They would not put it in those terms, but they did make, they did take other steps. So I will say before Amazon, IBM actually said we are no longer going to sell to police departments. And this was in 2020, right? So we also had the murder of George Floyd happening at that time. And Microsoft said, we will
Starting point is 00:34:32 not sell this until regulations are in place. And then Amazon came third, and they said, we'll halt it for a year. And then they extended that halt. Right. But this is to say there was an acknowledgement
Starting point is 00:34:47 of the risk and the harms. Was it just risk and harms to people or risk and harms to the company's reputations? It could be a combination of both. I'm excited to share that this research led to numerous cities, you know, incorporating some of the findings in their analysis and in their statements for why they chose to enact certain laws that restrict police use of facial recognition. It also changed the conversation at the national and international level. The EU AI Act actually has a provision that would prevent the use of live facial recognition in public spaces. When I spoke with
Starting point is 00:35:38 President Biden at the AI round table some time ago, this was top of mind. I shared the story of Robert Williams being wrongfully arrested in front of his two daughters. We talked about racial bias in AI systems and other types of harms that can impact many people because no one, trust me, no one is immune. This isn't just other people's problems. And so to see the reach of that work certainly made all of the combative, you know, responses and things like that somehow worth it. And it really goes back to what you were saying earlier about how systems don't have to be biased to be misused. And like, I don't know, I want to believe in a tech landscape where companies with so much power
Starting point is 00:36:31 don't have to wait until something goes wrong. Don't have to wait until somebody is wrongfully arrested. Don't have to wait until they're called out to make things a little bit safer and more equitable. Like, do you believe that is possible where companies aren't just reacting? They're actually, you know, being proactive at wanting the technology that they deploy on all of us to be, more equitable. I think that, again, you do see companies taking on the mantle of responsible AI. You'll have other companies like credo.aI that will have services that are meant to help companies adopting AI systems do it within a responsible way. You'll see companies hiring responsible AI leads,
Starting point is 00:37:14 right? So I definitely think there is an intention there where I still push back. a bit is self-regulation is always self-interested. Not surprising. So I do think real accountability requires external accountability. And the other part that I don't really see companies focused on so much is redress. So there's a lot of conversation about being responsible in terms of preventing future harms. But what about those who have been harmed already? And I do think algorithmic redress is oftentimes missing from this conversation of responsible AI.
Starting point is 00:37:58 So when I see the company stepping out to say, and we're doing redress, I might be convinced. I haven't seen it yet, though. Prove me wrong. Prove me wrong. I want to be wrong. Well, as somebody at the helm of an organization fighting for algorithmic justice, what does justice look like? Justice looks like you live in a world where data is not destiny. where your hue is an cue to dismiss your humanity,
Starting point is 00:38:24 where you actually have data rights and you can consent to how your information is used. Justice looks like, how do we use these tools in a joyful, uplifting manner, versus just being reactive to the next harm? So when I think of social justice, you can't have social justice without algorithmic justice, because if you're saying we're pushing for,
Starting point is 00:38:50 gender equality. Yet you have an AI system that cuts out women's resumes. We don't quite make it, right? You can't necessarily say, oh, we have racial equality. And then you're adopting bias, facial recognition that's putting so far the folks I've seen have all been dark skin like us, you know, into prison due to misidentification. And so me, for me, right, algorithmic justice is truly being, in that place where we can be our full selves and not be targeted, right, or algorithmically placed as other, algorithmically erased, algorithmically exploited. And so that's the world we fight for, right? We say free the X-coded. And so this is algorithmic justice. The book is unmasking AI. Your namesake, Joy, I have to tell you, you are such a joyful person. Speaking to
Starting point is 00:39:50 you about this work is, it just comes through how much you care and how, I don't know, I have a lot of conversations about tech that are hard and dark and grim and your work is just so the opposite. It asks, what if, what's possible? What can we do? How can things be better? Like, it's just really nice to see somebody leading the way with such a joyful but justice rooted perspective. It's so refreshing. Thank you. And it's so. wonderful to be in conversation with you. I love supporting epic, badass. He said I could cuss women. So this is great. So how can folks learn more about the algorithmic justice league? I'm so glad you asked. We do have www.ajl.org. And so we invite people to be agents of change
Starting point is 00:40:44 and join the algorithmic justice league. We have a library there. as well. So if you're new to this area and you are curious, you know, what is even AI? We've created resources for you so you can be part of the conversation. And we also have an ex-coded experiences platform. So like you were saying, you are the expert of your own lived experience and we value that expertise. So we do campaigns where people can tell their stories of being ex-coded. For some people, we just launched the campaign about facial recognition in airports. So people are sharing if they saw signage, if they knew they could opt out. And all of that actually builds a database of stories that shows if the TSA or others are actually doing what they say. They said it was optional. I didn't even know
Starting point is 00:41:38 I could opt out. We have a disconnect, but we also have the data, right? So I do think people as they are encountering various AI systems and they have questions or stories to share, AJL is that place they can go to. So please check out AJL. It is, you're doing such incredible work. Thank you so much for being here. And just thank you for being you. Thank you for being in the space.
Starting point is 00:42:04 We need more people like you. Thank you for having me. Black women like Dr. Blanweeney have been speaking truth to power when it comes to AI. And it's critical that the people who hold that power are listening. Her new book, On Masking AI, is poised to be one of the most important books about technology of the year. And it could not have come at a better time. It's available now, so I'll hope that you'll join me in reading it. If you're looking for ways to support the show, check out our merch store at tangoody.com slash store.
Starting point is 00:42:38 Got a story about an interesting thing in tech or just want to say hi? You can reach us at hello at tangooty.com. You can also find transcripts for today's episode at tangoody.com. There are no girls on the internet was created by me, Bridget Todd. It's a production of IHeart Radio and Unbossed Creative. Edited by Joey Pat. Jonathan Strickland is our executive producer. Tari Harrison is our producer and sound engineer.
Starting point is 00:43:00 Michael Amato is our contributing producer. I'm your host, Bridget Todd. If you want to help us grow, rate and review us on Apple Podcasts. For more podcasts from IHartRadio, check out the IHartRadio app, Apple Podcasts, or wherever you get your podcasts. Another podcast from some SNL, late-night comedy guy, not quite. Unhumor me with Robert Smygel and friends. me and hilarious guests from Bob Odenkirk to David Letterman help make you funnier. This week, my guest, SNL's Mikey Day and head writer Streeter Seidel,
Starting point is 00:43:33 help an a cappella band with their between songs banter. Where does your group perform? We do some retirement homes. Those people are starving for banter. Listen to humor me with Robert Smigel and friends on the IHeart Radio app, Apple Podcasts, or wherever you get your podcasts. What's up, fam, it's Isaiah Thomas. And I'm C.J. Toledano.
Starting point is 00:43:51 It's our favorite time of the year on our podcast, point game. the playoffs. We're digging into the biggest surprises of the season, and I'm looking back on some of my greatest playoff moments. If we didn't talk ever again, I was hungry. You just understood. That's how personal it got. Wow. Then after that game seven, Marquis come until he's like, you know I love you, dog. You know, it's all love. This was just playoffs. This was just basketball. So listen to Point Game on the IHeart Radio app, Apple Podcasts, or wherever you get your podcasts. Hey, it's Ashanti Plummer from Fudderound and Find out. This week, AZ Fudd and I sat down with
Starting point is 00:44:23 Step and Curry. Step talks pressure, confidence, and what it really takes to stay great. There's different categories, I guess, so I'm like conditioning, shooting drills where you try to simulate kind of games. Look at her face. We have a love-hate relationship with those because you know you're getting something out of it.
Starting point is 00:44:41 You don't look forward to those days. Listen to Fud around and find out on the IHeart Radio app, Apple Podcast, or wherever you get your podcast. This is an IHeart podcast. guaranteed human.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.