TED Radio Hour - Listen Again: Warped Reality (2020)

Episode Date: March 18, 2022

Original broadcast date: October 30, 2020. False information on the internet makes it harder and harder to know what's true, and the consequences have been devastating. This hour, TED speakers explore ideas around technology and deception. Guests include law professor Danielle Citron, journalist Andrew Marantz, and computer scientist Joy Buolamwini. See pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences.NPR Privacy Policy

Hey, it's Manus here. It has been shocking to watch the Russian invasion of Ukraine happen live on TV and social media. It's also kind of shocking that Russians are seeing a completely different version of the war play out, full of fabricated images, videos, and supposed facts. Disinformation and propaganda have always been part of any geopolitical conflict, but now, of course, it's all over the Internet and any of us can fall victim to it. That's why we want to revisit our episode called Warped Reality this week. It's about how some high-tech deceptions get produced and why some people believe them. It's a show that's perfect to listen to or listen to again, as once more we are glued to the headlines.

I'll be back next week with a brand new episode. Meanwhile, thanks so much for being here. Oh, and before we get started, quick note, this episode makes a couple references to sexual violence, which might be hard for some listeners to hear. This is the TED Radio Hour. Each week, groundbreaking TED Talks. Our job now is to dream big. Delivered at TED conferences.

To bring about the future we want to see. Around the world. To understand who we are. From those talks, we bring you speakers and ideas that will surprise you. You just don't know what you're going to find. Challenge you. We truly have to ask ourselves, like, why is it noteworthy? And even change you.

I literally feel like I'm a different person. Yes. Do you feel that way? Ideas worth spreading. From TED and NPR. I'm Anoush Zamoroti, and on the show today, technology and deception. The word deception has particular meaning. Deception is the intentional falsehood.

That is, there is something done to manipulate how people feel in the world. It's designed to change people's behavior and mislead. This is Danielle Citron. She's a professor of law at the University of Virginia, where she teaches privacy and free speech, and also researches tech and cyber harassment. And one of the best examples of this, she says, is the story of a woman named Rana Ayyub.

It was the 22nd of April 2018. You know, Rana Ayyub is an investigative journalist in India who has exposed human rights abuses and government corruption. You know, I am somebody who sent one of the most important ministers in the Modi government behind bars in 2010. And that man now happens to be the second most powerful man in India. And in April of 2018, she received an email from a source inside the Modi government. And the person said, heads up. a video is going around about you. It was like a two minute, 30 second bond video with my image morphed on it.

It was a fake sex video. And I mean, she's got big brown eyes. It looked like her. That was Rana. No questions about it. When I got that video, I felt like I was humiliated. I was shamed by the people who wanted to discredit me. And it went viral.

Screen shots of that video were all. over my social media on Instagram, WhatsApp messages and forwards all over India. Within 48 hours, it's been reported that it was on like half of the phones in India. Before I knew it was on my father's phone, my brother's phone. Within a day after that, her home address, her cell phone number all over the internet, there were fake ads on adults like finder sites saying that she was available for sex and this is where she lives. She was inundated with death and rape threats. I think I was as good as dead for the next five days since I received the video.

And she pretty much didn't leave her house for like six months. And I kept asking my friend, I said, what have I done to deserve this? She became like a shell of herself. And so how was that video made possible? If it wasn't her, what was it? It was a deep fake. Her face was inserted into a porn clip. So, you know, when I first worked on it,

You know, what we knew about it was that you could insert faces into videos and use sophisticated neural networks to do that. You know, it's called generative adversarial networks that sort of insert a video and then find mistakes and then keep iterating so that it becomes pretty perfected. But even then, so two years ago, you could sort of tell, though, you know, like if you stared at it enough, it wasn't as good as Pixar. It wasn't as good as, you know, the Lucas films. And over time, what we've seen is that now we can create from whole, digital whole cloth, videos showing you doing and saying things that you never did or said. And they're really hard to tell with the human eye that it's just not manufactured, right? And so Rana was a perfect example of, and the first one I had heard of,

of a deep fake sex video being used to basically drive someone out of the marketplace of ideas. So if I go on one of these platforms right now, like what's the likelihood that I will come across a deep fake? Or are we talking about a future that we're careening towards? I'm largely imagining a terrible future, but it's pretty bad in the hair and out. Let me explain. A group called Sensity, they found that a year ago, there were 15,000 deep fake videos. online. And of those 15,000, 96% were deep fake sex videos and 99% of those 96% were of women's faces inserted into porn. Fast forward to just a year later, 50,000 deep fake videos, again, same

lineup, right? Mostly over 90% deep fake sex videos. And again, same lineup, mostly all of women whose faces are being insurgent into porn without their permission. And it's not just U.S. women. You know, they found that it was women from all over the world. Hmm. And, like, I guess women's images have been altered and airbrushed for so long. And in some sense, we're already surrounded by fake images everywhere. But this is clearly taking it to a whole other disturbing level.

Yep, yep. Lies are absolutely nothing new to the human condition. But what makes this phenomenon different is sort of two things coming together. And the first is that we have this human frailty where audio and video have this power over us, especially, you know, what we see. So that we see something we're going to believe it. What's new is that we're in an online environment in which online platforms, their business model, their incentives, is to accelerate, share and ensure that we make things go viral.

because then we're liking, clicking, and sharing, and they're making money off of online advertising. And so their business model is aligned with our worst instincts. Information travels faster and farther than ever. And it does much more than just spark titillation or outrage. It changes what we believe. Conspiracy theories, new kinds of fake audio and video, and algorithms working behind the scenes make knowing what's true or false harder and harder. Our sense of reality is warping.

And we can see the consequences, a deep distrust in each other and our fundamental institutions like democracy. And so today on the show, technology, deception, and ideas about what we can do to bring ourselves back to reality. Because, as a lot of the show,

Danielle Citrin says, it takes just a trick of the human eye to upend someone's deeply held beliefs. Deepfakes appear authentic and realistic, but they're not. They're total falsehoods. Danielle continues from the TED stage. Now, it's the interaction of some of our most basic human frailties and network tools that can turn deep fakes into weapons. So let me explain. As human beings, we have a visceral reality. reaction to audio and video. We believe they're true on the notion that, of course, you can believe what your eyes and ears are telling you. And it's that mechanism that might undermine our shared sense of reality, although we believe deep fakes to be true than not. And we're attracted to the

salacious, the provocative. We tend to believe and to share information that's negative and novel. And researchers have found that online hoaxes spread 10 times faster than accurate stories. We're also drawn to information that aligns with our viewpoints. Psychologists call that tendency confirmation bias. And social media platforms supercharged that tendency by allowing us to instantly and widely share information that accords with our viewpoints. Okay, so all that information leads us to believe things, whether they are indeed facts or lies. But what about the people who say that they have the right to produce deep fakes or spread other misinformation?

Because of the First Amendment, free speech. It's an misunderstanding both of First Amendment doctrine and free speech theory, right? Because not all ones and zeros and words are protected speech as a matter of First Amendment law and a matter of free speech values, right? Why do we protect free speech? Because it helps us figure out how to govern ourselves because the truth sort themselves out in the marketplace of ideas, because it helps us engage in self-expression, because it's a safety value. You know, there's so many reasons why we protect free speech, all those reasons. And we can add more. You know, we got a few more. But when it comes to defamatory falsehoods, let's take the deep fake

video of showing someone doing and saying something they never did. As a matter of First Amendment doctrine, we chill that kind of speech. If you, with actual malice, spread a fake video of a public official doing and saying something they never did, but you know it's false, you can be sued for that. Right? So much, and I've been writing about this stuff for a long time and in my book, hate crimes and cyberspace kind of explore how, you know, these online tools, it isn't all just the public square. You know, the Supreme Court has a series of silly kind of, you know, this understanding of the internet as like as if it's still 1996, right? Like, it's all the town square. And we're all town criers. That's foolish. What we're doing online is we're working. We're hustling for

clients. We're spreading ideas. We are finding loved ones, right? We are exploring ideas. We are doing everything that we do offline, we do it online because phones are wherever we are. And so the idea that everything that happens online is protected free speech is wrong. And it's not good for free speech values. So the deep fake sex video of Ron, guess what? It ended up with her offline and silenced. You know, your nude photo appears in a search of your name. You are offline. You take down, this is just my experience working with victims, you literally take down all of your presence online. You're basically canceled for something that you didn't do. That's right. Your private persona becomes your public persona in an unwilling way that destroys your public persona.

And so it's so easy for people to say it's free speech and, you know, I will often get the pushback often from people who are privileged. white men love you, but they will say to me, like, you know, Danielle, like you're a prude, why you make it such a big deal about nude photos, we should all just put our nudes online. And I just then take a beat. I'm calm, right? I don't get mad. But I say, I'm so glad you're going to make that choice.

But I'm not going to make that choice, right? Because it's going to cost me and other. women, women of color, trans women, gay men, trans men, you know, by folks, like queer folks, it's just going to cost them more. In a minute, Daniel Citrin on why deep fakes have the potential to undermine our democracy. On the show today, technology, deception, and our changing sense of reality. I'm Manusia Zamoroti, and you're listening to the TED Radio Hour from NPR. Stay with us.

It's the TED Radio Hour from NPR. I'm Anoush Zamoroti. And we were just hearing lawyer and privacy expert, Daniel Citrin, describe a recent internet phenomenon, videos called deep fakes. Walk with me into the future where we are in a place where we don't know whether to believe anything we see. What is that like? So that, we call that the liar's dividend. In a world in which we can't tell.

the difference between what's fake and what's real, that's a real boon to the mischief makers and the liars because they get to point to real evidence of their wrongdoing and say it's not true and get to walk away from responsibility and accountability for bad things that they've done. And we've seen illustrations of this. You know, I'm automatically attracted to beautiful. I just start kissing them. It's like a magnet. After the Access Hollywood tape came out, President Trump said, you know, hey, I said that, I'm sorry.

This was locker room talk. I'm not proud of it. I apologize to my family. I apologize. A year later, he shared with a reporter, I actually wasn't me on the Access Hollywood tape. He was sort of throwing out the liar's dividend. Maybe it'll work, right? Now, for the most part, that didn't really have great traction. And it's kind of part of his brand, all of it. So that, you know, the liar's dividend, like he tried it, didn't work, and maybe that didn't hurt him, it didn't matter. But in an environment in which we are sort of post-truths so that we're going to believe falsehoods if they accord with what we believe, and we're going to disbelieve truths if they don't accord with what we think, you know, confirmation bias, then we're in this kind of post-truth environment. And I never thought I would say this about our own country, but political discourse feels fragile in a way that makes me feel like we feel like we're in a way.

we're much more like a Myanmar than we are, you know, Canada. We don't feel like on solid ground in terms of discourse. And so in this environment, it feels so fragile, our democracy. As we see what happens in terms of the platforms taking responsibility or laws passed or whatever sort of systemic change may or may not happen, I mean, how much of us, each of us, as individuals who go online, who's a lot. Do we have, what's our responsibility? Maybe, I don't know, sometimes I say to people like, you know, you are up against massive corporations when you like and share and all that stuff. You are being manipulated. But maybe you see it differently. Maybe you think that we each have to do a better job as well. We do. You know, in the here and the hour, there aren't laws, right? There are very few state laws around deep, you know, digital forgeries. We are the guardian at the gate.

platforms aren't going to do it for us. Do you know what I'm saying? Like we can't we can't expect platforms whose incentives are to share because that's where their money is. It's on us. Each and every one of us, we need to protect ourselves and our democracy. It's ours. It's ours to lose. So I do think we have a huge role. What I'm asking is so modest. Think before you click and share. Ask yourself. Is this likely? And if it's really crazy, don't you think that it's fake? You know, that's why it's there, right? It's there because it's negative and novel. It's there to feed on our salacious curiosity. Don't do it.

That's Danielle Citrin. She's a professor of law at the University of Virginia, where she teaches and writes about privacy and free speech. You can see her full talk at ted.com. On the show today, ideas about technology, deception, and are changing sense of reality. And deep fakes make up one disturbing side of misinformation. Another conspiracy theories. Sure, they've been around a long time. Classics like the Earth is Flat.

Or another one that just won't go away. There's a secret cabal of shadowy, wealthy elites who either experience. explicitly are named as Jews or just kind of fit into type of mold that sounds like they're probably Jews. In the past few years, these conspiracy theories, along with a whole set of new ones, have moved from the extreme fringes into the American mainstream. If ever there were a question, whether a deep state existed, you got your answer this week. I believe George Soros is behind all of this, paying these people to give you. I was talking about all of this with the global warming and a lot of it's a hoax. It's a hoax.

I mean, it's a money-making industry. Well, they will be advancing their new conspiracy theory and their newest hoax. One recent example. An online conspiracy theory. It's unclear if it was right. QAnon. QAnon's baseless conspiracy theories have been repeatedly debunk, despite this, the far-right group continues to mad. A network of conspiracy theories, all leading back to a mythic anonymous leader named Q.

allegedly a high-level government official who has access to top secret information. Well, I don't know much about the movement other than I understand. They like me very much, which I appreciate. But I don't know much about the movement. Yeah, so Q&N starts with the assumption that Donald Trump is secretly saving the world and that, you know, he doesn't want credit for it. want to boast about it, but he's actually the only person who is ferreting out this massive deep state conspiracy that involves hundreds of people engaging in child sex trafficking

and satanic cannibalistic rituals. It's like, sorry, what? And, you know, that is just straight up misinformation slash disinformation. So in a way, that's almost a less ambiguous case because it's so completely bonkers, honestly. And it's almost like if I were writing the script, I would be like, nah, guys, that's too on the nose. Like, that's too much of a completely bizarre. Yes, no one will believe it. But, you know, millions of people do.

This is journalist Andrew Morantz. I am a staff writer at The New Yorker magazine. and I wrote a book called Anti-Social. I have to make sure I remember the subtitle, online extremists, techno-utopians, and the hijacking of the American conversation. Even before Q&N entered mainstream conversation in 2017, Andrew noticed a huge rise in far-right extremism and conspiracy theories online. Starting 2014 or 2015 around there,

I started seeing this informational crisis on the horizon. certainly was not the only person. But I was kind of behind the curve at the time because I wasn't back then thinking of this as a particularly political story. I was thinking of it as, you know, a business story or a tech story or. Yeah. But then the summer of 2016, I really started saying, okay, this is going to have a massive impact on the presidential election. I mean, in a way, how could it not? Right. Racist memes, misogynist propaganda, viral misinterpret. So I wanted to know who was making this stuff. I wanted to understand how they were spreading it.

Ultimately, I wanted to know what kind of impact it might be having on our society. Here's Andrew on the TED stage. So that's how I ended up in the living room of a social media propagandist in Southern California. He was a married white guy in his late 30s. He had a table in front of him with a mug of coffee, a laptop for tweeting, a phone for texting, and an iPad for live streaming to Periscope and YouTube. And yet, with those tools, he was able to propel his fringe,

noxious talking points into the heart of the American conversation. For example, one of the days I was there, a bomb had just exploded in New York. And the guy accused of planting the bomb had a Muslim-sounding name. Now, to the propagandist in California, this seemed like an opportunity. Because one of the things he wanted was for the U.S. to cut off almost all immigration, especially from Muslim-majority countries. So he started live-streaming. getting his followers worked up into a frenzy about how the open borders agenda was going to kill us all

and asking them to tweet about this and use specific hashtags trying to get those hashtags trending and tweet they did, hundreds and hundreds of tweets. It must have been kind of weird for you, like sitting there and watching how public manipulation works from a couch. Yeah, what you see when you sort of sit at someone's elbow and watch them do this is it's like getting good at poker or something. You kind of learn the basic mechanics of the thing, and then you play a lot of rounds. And if you're good enough, you know, you don't win every round, but you win a lot of them. So what that means in terms of social media is you kind of get a sense of what the algorithms want. And the really simplistic way of putting it is that they want whatever has the sharpest emotional impact on the viewer.

And specific kinds of emotions, too. emotions that make people do something with their either click or share. But for the most part, he wasn't breaking the rules of Twitter or whatever platform he was using. He was just really good at getting his message out there. And so this guy in California and all the other folks you spent time with, like, what was their mission? Was it just to create chaos, to tear down democracy? Because that guy doesn't seem like he really believes in this stuff, but there are people who do. Yeah.

it's a spectrum, right? It's sometimes we talk about, you know, people who are doing this for profit cynically who don't believe what they're peddling and then people, on the other hand, who are true believers. And I think that is a true and worthwhile distinction. And I guess all I would add is that there are many shades of gray in between. Yeah. Right. So it's not purely immediate monetary motivation in most cases. And then you'll get some cases where it is just people who have just been radicalized or red-pilled, as they call it. And they just think the world will not be safe until we have a white ethno state. And, you know, obviously those people are hard to deal with because they're pretty far gone. I talked a lot with one young woman who grew up in New Jersey.

And then after high school, she moved to a new place and suddenly she just felt alienated and cut off and started retreating into her phone. She found some of these spaces on the internet where people would post the most shocking, heinous things. And she found this stuff really off-putting, but also kind of engrossing, she started interacting with people in these online spaces, and they made her feel smart, they made her feel validated. She started feeling a sense of community, started wondering

if maybe some of these shocking memes might actually contain a kernel of truth. A few months later, she was in a car with some of her new internet friends headed to Charlottesville, Virginia, to march with torches in the name of the white race. She'd gone in a few months from Obama supporter

to fully radicalized white supremacist. Okay, so for the first, those of us who just can't wrap our heads around how someone's ideas about the world can change so rapidly. Right. Like, how does that happen, especially with something like QAnon, which is really an entire mindset? Yeah. So it's sort of like any cult, you know, you start with the stuff that sounds less controversial and then the more and more people get initiated, the more they're prepared to believe more and more outlandish things. So you kind of start with parts of it that are closer.

to the truth. Like, there was this guy, Jeffrey Epstein, who was really was doing all kinds of outrageously terrible things. And he really was friends with Bill Clinton and Prince Andrew. And, you know, there really was a conspiracy to cover up those crimes that is still ongoing. And then you kind of go from there to, you know, I bet Hillary was involved. And I also bet Tom Hanks was involved. And I bet Oprah's probably involved. And they all have a dungeon somewhere where they're locking up children and trying to harvest their adrenal glands. And, okay, you lost me with the adrenal glands. Well, I think when you start going down that rabbit hole, part of it is engaging in a kind of collective fan fiction. And part of it is, wait, wait, but what if this is real? And it's

flirting with that line. And, you know, I can get that to some extent. I get the thrill of being like, what if there really was an Illuminati? And it's just to a certain kind of person at a certain desperate moment in their lives or just spend too much time on the internet. They can't. keep those blurry lines straight and it becomes their entire reality. I can understand how in a time of financial problems, insecurity, this idea of being part of a movement and having meaning in your life and being, you know, part of like the revolution, right? Absolutely. That's, that's very exciting. Yeah. Part of the underground, part of the resistance. And having secret knowledge that no one else can see except for your compadres. Yeah, it's just bizarre how far it can go.

And some of these tropes are, you know, centuries old in some ways. They are not new. Yeah, I did not expect the fact that I am Jewish to matter in any way. I didn't expect to be talking about things like protocols of the elders of Zion and like Mienkamp and stuff. I thought, frankly, that they would be a little more original than that. But as a lot of the end of the end of As it turns out, there are certain tropes that just refuse to go away. And it does a lot of work for people. It helps explain things that are otherwise unexplainable. You know, why is the economy so obviously seem to be rigged against me?

Why can't I find meaningful work? Or why do I have work, but I still don't feel like my life has any purpose? You know, on and on and on with these kind of sometimes unanswerable questions. And if the answer is, because there are 10 people in a room somewhere, saying, I don't want people to have meaning in their lives, then in a way that's kind of a comforting explanation, because it means that there's at least a nameable reason or an identifiable enemy. And often that's Jews. Often that's women. Often it's just whoever's a visible other in terms of being a person of color or what have you. But I think one of the notable things for me

is not that these tropes still exist in the world, but the fact that they can be revived in terms of popularity and in terms of salience to the national discourse. That I did find surprising. I mean, it feels as though this idea that a single person switching on a story that changes people's perception of what reality even is has become so commonplace that we are in the midst of an era of very little trust. Yes. It is commonplace now. And the companies have had a lot of time to try to figure this out. In some ways they have, you know, it is no longer okay on Facebook to buy an ad in an American election using rubles as the crazy. That was a loophole that was open in 2016. That should not have been open, but it was. They did close that loophole. But the larger

loophole, which is the entire incentive structure, the entire thing that social media is built to do, that hasn't changed. And yeah, as a result, we are living in a pretty, confused and confusing time. So what do we do in the meantime? Like how do we fix this at least a little bit? Yeah. So a lot of the bigger solutions to this are going to have to be systemic and the companies are going to have to step up and it might involve government regulation and all kinds of bigger things. But until they rebuild and dismantle their business model, there are things that individuals can do. And one of them, I call it being a smart skeptic. So there are things that pass for skepticism online. that I think are actually just knee-jerk contrarian trollery.

So you often see people saying, well, I'm just asking for more evidence and I'm just asking the question, but that is not real skepticism. Real skepticism is being open-minded but not being so open-minded that your brain falls out, demanding evidence, but not demanding evidence past the time when a question has been settled. If you just sort of say, well, I don't know, you know, everybody says racism is bad, but like I'm skeptical. of that claim. I don't think skepticism is the best word for what you're doing there. And sometimes there just is consensus on something and there's a certain cast of mind of a person who just doesn't want to hear that. It's a kind of addiction to feeling like you have secret access to knowledge that, you know, the boundaries of polite society doesn't want you to have. If only we all had that secret access, right? Yeah. I mean, it's a thrilling idea. It's just that sometimes the real answer is the

answer that most people already believe. And I'm sorry if that's boring, but it just sometimes is the case. That's Andrew Morantz. He's a journalist and staff writer for the New Yorker. You can see his full talk at TED.com. On the show today, technology and deception. I'm Minnush Zamoroti, and you're listening to the TED Radio Hour from NPR. It's the TED Radio Hour from NPR.

I'm Manush Zomerode. On the show today, technology, deception, and our changing sense of reality. And so far, we've been talking about deep fakes, conspiracy theories, and other kinds of misinformation. But data and algorithms, they can warp our reality too. We can deceive ourselves into thinking they're not doing harm, or we can fool ourselves into thinking because it's based on numbers, that it is somehow neutral. AI is creeping into our lives. And even though the promise is that it's going to be more efficient, it's going to be better.

If what's happening is we're automating inequality through weapons of math destruction and we have algorithms of oppression, this promise is not actually true and certainly not true for everybody. Weapons of math destruction, algorithms of oppression, which basically means bias and human beings. error can be encoded into algorithms leading to inequality. To keep them in check, the algorithmic Justice League to the rescue. My name is Joy Blumwini. I'm the founder of the algorithmic justice league, where we use research and art to create a world with more equitable and accountable AI. You might have heard of the male gaze or the white gays or the post-colonial gaze to that lexicon. the coded gaze. And we want to make sure people are even aware of it because you can't fight the power you don't see, you don't know about.

Joy hunts down the flaws in the technology that's running every part of our lives, from deciding what we see on Instagram to how we might be sentenced for a crime. What happens when somebody is harmed by a system you created? You know, what happens if you're harmed? Where do you go? We want that kind of place to be the algorithmic justice. So you can seek redress for algorithmic harms. You are a lot of things. You're a poet, you're a computer scientist, you are a superhero. Like, kind of hard to put into a box.

Can you just explain why you created the algorithmic justice league? Yes. So the algorithmic justice league is a bit of an accident. When I was in graduate school, I was working on an art project that you some computer vision technology to track my face. Hi, camera, I've got a face. Can you see my face? At least that was the idea.

You can see her face. What about my face? And when I tried to get it to work on my face, I found that putting a white mask on my dark skin is what I needed in order to have the system pick me up. And so that led to questions about, wait, are machines neutral? Why do I need to change myself to be seen by a machine? And if this is using AI techniques that are being used in other areas of our lives,

whether it's health or education, transportation, the criminal justice system, what does it mean if different kinds of mistakes are being made? And also, even if these systems do work well, let's say you are able to track a face, perfectly. What does that mean for surveillance? What does it mean for democracy, First Amendment rights, you know? Joy continues from the TED stage. Across the U.S., police departments are starting to use facial recognition software in their crime-fighting arsenal. Georgetown law published a report showing that one and two adults in the U.S., that's 117 million people have their faces in facial recognition networks.

Police departments can currently look at these networks unregulated using algorithms that have not been audited for accuracy. Machine learning is being used for facial recognition, but it's also extending beyond the realm of computer vision. So who gets hired or fired? Do you get that loan? Do you get insurance? Are you admitted into the college that you wanted to get into? Do you and I pay the same price for the same product purchased on the same platform? Law enforcement is also starting to use machine learning for predictive policing. Some judges use machine-generated risk scores to determine how long an individual is going to spend in prison. So we really have to think about these decisions. Are they fair? And we've seen that algorithmic bias doesn't necessarily.

necessarily always lead to fair outcomes. When I think about algorithmic bias and people ask me, well, what do you mean machines are biased? It's just numbers. It's just data. I talk about machine learning and it's a question of, well, what is the machine learning from? Well, what is the machine learning from? Like, what's the information that it's taking in? So an example of this, what I found was that for face detection, the ways in which systems were being trained, involved collecting large data sets of images of human faces. And when you look at those data sets, I found that many of them were pale and male, right? You might have a data set that 75% male faces, over 80% lighter skin faces. And so what it

means is the machine is learning a representation of the world that is skewed. And so what you might of thought should be a neutral process is actually reflecting the biases that it has been trained on. And sometimes what you're seeing is a skewed representation, but other times what machines are picking up on are our own societal biases that are actually true to the data. For example, Amazon was building a hiring tool. You need a job. Somebody in your life needs a job, right? you want to get hired.

And to get hired, you upload your resume and your cover letter. That's the goal. It starts off well. But before a human looks at your resume, it gets vetted by algorithms, written by software engineers. So we start off with an intent for efficiency. We have many more applications than any human could go through. Let's create a system that can do it more efficiently than we can. And how to build that better system? Well, we're going to gather data of resumes, and we're going to sort those resumes by the ones that represented candidates we hired or did well.

Your target is who you think will be a good long-term employee. And now the system gets trained on the data. And the system is learning from prior data. So I like to say the past dwells within our algorithms. You don't have to have the sexist hiring manager in front of you. Now you have a black box that's serving as the gatekeeper. But what it's learning are the patterns of what success has looked like in the past. So if we're defining success by how it's looked like in the past,

and the past has been one where men were given opportunity, white people were given opportunity, and you don't necessarily fit that profile. Even though you might think you're creating this objective system, it's going through resumes, right? This is where we run into problems. So here's what happened with Amazon's hiring tool. What happened was as the model was being built and it was being tested, what they found was a gender bias where resumes that contained the word women or women's or even all women's colleges, right? So indication of being a woman were categorically being rank lower than those that didn't.

And try as they might, they were not able to remove that gender bias. So they ended up scratching the system. They scratched the system. And that's a big win. But one win compared to thousands of platforms that use skewed algorithms that could warp reality. It has not been the case that we've had used. universal equality or absolute equality in the words of Frederick Douglass. And I especially worry about this when we think about techno benevolence in the space of health care, right? We're looking at,

let's say a breakthrough that comes in talking about skin cancer. Oh, we now have an AI system, right, that can classify skin cancer as well as the top dermatologist's study might say a headline might read. And then when you look at it, it's like, oh, well, actually, when you look at the data set, it was for lighter skin individuals. Then you might argue, well, you know, lighter skin people are more likely to get skin cancer. And when I was looking into this, it actually, darker skin people who get skin cancer, usually it's detected in stage four because there are all of these assumptions you're not even going to get it in the first place. So these assumptions can have meaningful consequences.

You know, we were just talking before about the 2016 presidential election. Have you seen any examples of artificial intelligence being used in voting or politics? Yeah. So Channel 4 News just did this massive investigation showing that the 2016 Trump campaign targeted 3.5 million African Americans in the United States labeled them as deterrents in an attack. to actually keep people from showing up to the polls. They used targeted ads. Yes. And we know from Facebook's own research, right, that you can influence voter turnout

based on the kinds of posts that are put on their platform. And they did this in battleground states. And so in this way, we're seeing predictive models. modeling and ad targeting, right, being used as a tool of voter suppression, which has always been the case to disenfranchise, right? You might say black lives don't matter, but it's clear black votes matter because of so much effort used to rob people of what blood was spilled for, you know, for generations. So it should be the case, right, that any sorts of algorithmic tools that are intended to be used, again, have to be verified for non-discrimination before it's

even adopted. So as a black woman technologist, you know, they're not that many of you, frankly. Why not, you know, go work at Google or Amazon and make these changes to the algorithms directly? Why act as sort of a watchdog? Well, I think there are multiple ways to be involved in the ecosystem. But I do think this question you pose is really important because it can be an assumption that by changing who's in the room, which is important and needs to happen, we're going to then change the outcome and the outputs of these systems. So I like to remind people that most software developers, engineers, computer scientists, you don't build everything from scratch, right? You get reusable parts. And so if there's bias within those reusable parts

or large-scale bias in the data sets that have become standard practice or the status quo, right, changing the people who are involved in the system without changing the system itself is still going to reproduce algorithmic bias and algorithmic harms. So how do we build systems that are more fair? Like if there's no data for the artificial intelligence to sort of, you know, process to start to pump out recommendations, then how do we even change that? Yeah, well, it's a question of what tools do you use towards what objectives? So the first thing is seeing if this is the appropriate tool. Not every tool, not every decision needs to be run through AI. And oftentimes you also need to make sure

you're being intentional. And so the kinds of changes you would need to make systematically for even who gets into the job pool in general, it means you do have to change society to change what AI is learning. What do you say, Joy, to people who might be listening and thinking like, you know, let's take a step back and look at the bigger picture. We, in many ways, things are way better than they were. thanks to technology because, you know, here we are in a pandemic and anyone can work from anywhere because we have the internet and we have Zoom and all of these platforms.

Equality and access is on the whole improved. Why let's not like be Debbie Downers about it? Yeah, I mean, I always ask who can afford to say that because I can tell you the kids who are sitting in McDonald's parking lot so they can access the internet to be able to, attend school remotely, that has never been their reality. And so oftentimes, if you are able to say technology on the whole has done well, it probably means you're in a fairly privileged position. There's still a huge digital divide. Even there are billions of people who don't have access to the internet. I mean, I was born in Canada, moved to God and then grew up in the U.S.

I had very Western assumptions, you know, about what tech could do. very much excited to use the tech skills I'd gained as an undergrad at Georgia's tech, you know, to use tech for good, tech for the benefit of humanity. And so when I critique tech, it's really coming from a place of having been enamored it with it and wanting it to live up to its promises. I don't think it's being a Debbie Downer to show ways in which we can improve so the promise of something we've created can actually be realized. I think that's even a, a more optimistic approach than to believe in a wishful thinking that is not true. You know, one thing that you've said that I find so, I love this idea, that you say there's a

difference between potential and reality and that we must separate those two ideas. Yes, so it's so easy to fixiate on our aspirations of what tech could be. and I think in some ways is this hope that we can transcend our own humanity, right? Our own failures. And so, yes, even if we haven't gotten society quite right, ideally we can build technology that's better than we are. But we then have to look at that fact that technology reflects who we are. It doesn't transcend who we are.

And so I think it's important that when we think about technology, we ask what's the promise, what's the reality? And not only what's that gap, but who does it work for? Who does it benefit? Who does it harm? And why? And also, how do we then step up and stand up to those harms? That's Joy Wallamweeney, founder of the Algorithmic Justice League.

You can watch her full talk at ted.com. Thank you so much for listening to our show. this week about technology, deception, and our warped reality. To learn more about the people who were on it, go to ted.npr.org. And to see hundreds more TED talks, check out TED.com or the TED app. Our TED Radio production staff at NPR includes Jeff Rogers, Sanaz Meshkenspur, Rachel Faulkner, Diba Motisham, James Delahousie, J.C. Howard, Katie Montalione, Maria Paz Gutierrez, Christina Kala, and Matthew Cloutier, with help from Dan.

Daniel Shukin. Our intern is Farah Safari. Our theme music was written by Romteen Arablewee. Our partners at TED are Chris Anderson, Colin Helms, Anna Feelein, and Michelle Quint. I'm Manus Zamoroti, and you've been listening to The TED Radio Hour from NPR.