TED Radio Hour - Listen Again: Warped Reality

Episode Date: May 7, 2021

Original broadcast date: October 30, 2020. False information on the internet makes it harder and harder to know what's true, and the consequences have been devastating. This hour, TED speakers explore ideas around technology and deception. Guests include law professor Danielle Citron, journalist Andrew Marantz, and computer scientist Joy Buolamwini. See pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences.NPR Privacy Policy

Hey, it's Mnuch here. I've got a question for you. How do we sort misinformation from good information online? And whose responsibility is it to help us figure that out? Yeah, those are not easy questions. And they came up yet again recently with a ruling from Facebook's oversight board. This is a sort of independent body, which decided that Facebook was right to suspend Donald Trump from posting information that incited violence on the platform. but that Facebook was wrong to make that suspension indefinite. It's really easy to get bogged down in the details of these disputes. But I think we need to remember there are big issues at stake over how technology influences our actions

and what is real online. And so today we want to bring back an episode from October called Warped Reality. And I have to say, as a former tech reporter, I love this episode. It features some of the smartest people out there with ideas about how we build the future based on facts. A quick note, this episode makes a couple references to sexual violence, which may be hard to hear. Just want to give you a heads up before we get started. I hope you enjoy the episode again or for the first time. And thanks, as always, for listening.

This is the TED Radio Hour. Each week, groundbreaking TED Talks. Our job now is to dream big. Delivered at TED conferences. To bring about the future we want to see around the world. To understand who we are. From those talks, we bring you speakers and ideas that will surprise you. You just don't know what you're going to find.

Challenge you. We truly have to ask ourselves, like, why is it noteworthy? And even change you. I literally feel like I'm a different person. Yes. Do you feel that way? Ideas worth spreading. From TED and NPR.

I'm Anoush Zamorodi, and on the show today, technology and deception. The word deception has particular meaning. Deception is the intentional falsehood. That is, there is something done to manipulate how people feel in the world. It's designed to change people's behavior and mislead. This is Danielle Citrin. I'm a law professor at Boston University School of Law where I teach privacy and free speech. Danielle also researches technology and cyber harassment.

And one of the best examples of this, she says, is the story of a woman named Rana Ayyub. It was the 22nd of April 2018. Rana Ayub is an investigative journalist in India who has exposed human rights abuses and government corruption. You know, I am somebody who sent one of the most important ministers in the Modi government behind bars in 2010, and that man now happens to be the second most powerful man in India. And in April of 2018, she received an email from a source inside the Modi government. And the person said, heads up, a video is going around about you.

It was like a two-minute, 30-second bond video with my image morphed on it. It was a fake sex video. And I mean, she's got big brown eyes. It looked like her. That was Rana. No questions about it. When I got that video, I felt like I was humiliated. I was shamed by the people who wanted to discredit me.

And it went viral. Screen shots of that video were all over my social media, on Instagram, Twitter, WhatsApp, messages and forwards all over India. Within 48 hours, it's been reported that it was on like half of the phones in India. Before I knew it was on my father's phone, my brother's phone. Within a day after that, her home address, her cell phone number, all over the internet, there were fake ads on adults like finder sites saying that she was available for sex and this is where she lives. She was inundated with death and rape threats. I think I was as good as dead for the next five days since I received the video.

And she pretty much didn't leave her house for like six months. And I get asking my friend, I said, What have I done to deserve this? She became like a shell of herself. And so how was that video made possible? If it wasn't her, what was it? It was a deep fake. Her face was inserted into a porn clip.

So, you know, when I first worked on it, you know, what we knew about it was that you could insert faces into videos and use sophisticated neural networks to do that. know, it's called generative adversarial networks, that sort of insert a video and then find mistakes and then keep iterating so that it becomes pretty perfected. But even then, so two years ago, you could sort of tell, though, you know, like if you stared at it enough, it wasn't as good as Pixar. It wasn't as good as, you know, the Lucas films. And over time, what we've seen is that now we can create from whole, digital whole cloth, video showing you doing and saying things that you

never did or said. And they're really hard to tell with the human eye that it's just not manufactured, right? And so Rana was a perfect example of, and the first one I had heard of, of a deep fake sex video being used to basically drive someone out of the marketplace of ideas. So if I go on one of these platforms right now, like what's the likelihood that I will come across a deep fake? Or are we talking about a future that we're careening towards. I'm largely imagining a terrible future, but it's pretty bad in the hair and now. Let me explain. A group called Sensity, they found that a year ago, there were 15,000 deep fake videos online. And of those 15,000, 96% were deep fake sex videos, and 99% of those 96% were of women's faces inserted into porn. Fast forward to just a year

later. 50,000 deep fake videos, again, same lineup, right? Mostly over 90% deep fake sex videos. And again, same lineup, mostly all of women whose faces are being insurgent into porn without their permission. And it's not just U.S. women. You know, they found that it was women from all over the world. And like, I guess women's images have been altered and airbrushed for so long. And in some sense, we're already surrounded by fake images everywhere. But this is clearly taking it to a whole other disturbing level. Yep, yep. Lies are absolutely nothing new to the human condition.

But what makes this phenomenon different is sort of two things coming together. And the first is that we have this human frailty where audio and video have this power over us, especially, you know, what we see. so that we see something we're going to believe it. What's new is that we're in an online environment in which online platforms, their business model, their incentives, is to accelerate, share, and ensure that we make things go viral

because then we're liking, clicking, and sharing, and they're making money off of online advertising. And so their business model is aligned with our worst instincts. Information travels faster and farther than ever. and it does much more than just spark titillation or outrage, it changes what we believe. Conspiracy theories, new kinds of fake audio and video, and algorithms working behind the scenes make knowing what's true or false harder and harder. Our sense of reality is warping, and we can see the consequences, a deep distrust in each other

and our fundamental institutions. like democracy. And so today on the show, technology, deception, and ideas about what we can do to bring ourselves back to reality. Because, as Danielle Citrin says, it takes just a trick of the human eye to upend someone's deeply held beliefs. Deepfakes appear authentic and realistic, but they're not. They're total falsehoods.

Danielle continues from the TED stage. Now, it's the interaction of some of our most basic human frailties and network tools that can turn deep fakes into weapons. So let me explain. As human beings, we have a visceral reaction to audio and video. We believe they're true on the notion that, of course, you can believe what your eyes and ears are telling you. And it's that mechanism that might undermine our shared sense of reality,

although we believe deep fakes to be true than not. And we're attracted to the salacious, the provocative. We tend to believe and to share information that's negative and novel. And researchers have found that online hoaxes spread 10 times faster than accurate stories. We're also drawn to information that aligns with our viewpoints.

Psychologists call that tendency comfort bias. And social media platforms supercharged that tendency by allowing us to instantly and widely share information that accords with our viewpoints. Okay, so all that information leads us to believe things, whether they are indeed facts or lies. But what about the people who say that they have the right to produce deep fakes or spread other misinformation? Because of the First Amendment, free speech. It's an misunderstanding both of First Amendment doctrine and free speech theory, right? Because not all ones and zeros and words are protected speech as a matter of First Amendment law and a matter of free speech values, right? Why do we protect free speech? Because it helps us figure out how to govern ourselves because the truth sort themselves out in the marketplace of ideas, because it helps us engage in self-expression, because it's a safety value. You know, there's a safety value. You know, there's a so many reasons why we protect free speech. All those reasons. And we can add more. You know, we got a few more. But when it comes to defamatory falsehoods, let's take the deep fake

video of showing someone doing and saying something they never did. As a matter of First Amendment doctrine, we chill that kind of speech. If you, with actual malice, spread a fake video of a public official doing and saying something they never did, but you know it's false. You can be sued for that, right? So much. And I've been writing about this stuff for a long time. And in my book, Hey, Crimes in Cyberspace, kind of explore how, you know, these online tools, it isn't all just the public square. You know, the Supreme Court has a series of silly kind of, you know, this understanding of the internet is like as if it's still 1996, right? Like, it's all the town square. And we're all town criers. That's foolish. What we're doing online is we're working. We're hustling for a We're spreading ideas. We are finding loved ones, right? We are exploring ideas. We are doing everything

that we do offline. We do it online because phones are wherever we are. And so the idea that everything that happens online is protected free speech is wrong. And it's not good for free speech values. So the deep fake sex video of Rana, guess what? It ended up with her offline and sign up. You know, your nude photo appears in a search of your name. You are offline. You take down, this is just my experience working with victims. You literally take down all of your presence online. You're basically canceled for something that you didn't do. That's right. Your private persona becomes your public persona in an unwilling way that destroys your public persona. And so it's so easy for people to say it's free speech.

And, you know, I will often get the pushback, often from people who are privileged. So white men love you. But they will say to me, like, you know, Danielle, like you're a prude, why you make it such a big deal about nude photos. We should all just put our nudes online. And I just then take a beat. I'm calm, right? I don't get mad. But I'd say, I'm so glad you're going to make that choice.

But I'm not going to make that choice, right? Because it's going to cost me and other women, women of color, trans women, gay men, trans men, you know, by folks, like queer folks. It's just going to cost them more. In a minute, Daniel Citrin on why deep fakes have the potential to undermine our democracy. On the show today, technology, deception, and our changing sense of reality. I'm Manushe Zamoroti, and you're listening to The TED Radio Hour from NPR. Stay with us. It's the TED Radio Hour from NPR.

I'm Anoush Zamoroti. And we were just hearing lawyer and privacy expert Daniel Citrin describe a recent internet phenomenon, videos called deep fakes. Walk with me into the future where we are in a. a place where we don't know whether to believe anything we see. What is that like? So that, we call that the liar's dividend. In a world in which we can't tell the difference between what's fake and what's real, that's a real boon to the mischief makers and the liars. Because they get to point to real evidence of their wrongdoing and say it's not true

and get to walk away from responsibility and accountability for bad things that they've done. And we've seen illustrations of this. You know, I'm automatically attracted to beautiful. I just start kissing them. It's like a magnet. After the Access Hollywood tape came out, President Trump said, you know, hey, I said that, I'm sorry.

This was locker room talk. I'm not proud of it. I apologize to my family. A year later, he shared with a reporter, I actually wasn't me on the Access Hollywood tape. He was sort of throwing out the liar's dividend. Maybe it'll work, right? Now, for the most part, that didn't really have great traction.

And it's kind of part of his brand, all of it, so that, you know, the liar's dividend, like he tried it, didn't work. And maybe that didn't hurt him. It didn't matter. But in an environment in which we are sort of post-truths so that we're going to believe falsehoods, if they accord with what we believe, and we're going to. disbelieve truths if they don't accord with what we think, you know, confirmation bias, then we're in this kind of post-truth environment. And I never thought I would say this about our own country, but political discourse feels

fragile in a way that makes me feel like we're much more like a Myanmar than we are, you know, Canada. We don't feel like on solid ground in terms of discourse. And so in this environment, it feels so fragile, our democracy. As we see what happens in terms of the platforms taking responsibility or laws passed or whatever sort of systemic change may or may not happen. I mean, how much of us, each of us as individuals who go online, a lot, do we have, what's our responsibility? Maybe, I don't know. Sometimes I say to people like, you know, you are up against massive corporations when you like and share and all that stuff.

You are being manipulated. But maybe you see it differently. Maybe you think that we each have to do a better job as well. We do. You know, in the here and the hour, there aren't laws, right? There are very few state laws around deep, you know, digital forgeries. We are the guardian at the gate. Platforms aren't going to do it for us.

Do you know what I'm saying? Like, we can't expect platforms whose incentives are to share because that's where their money is. It's on us, each and every one of us. We need to protect ourselves. and our democracy, it's ours. It's ours to lose. So I do think we have a huge role. What I'm asking is so modest. Think before you click and share. Ask yourself, is this likely? And if it's really crazy, don't you think that it's fake? You know, that's why it's there, right?

It's there because it's negative and novel. It's there to feed on our salacious curiosity. Don't do it. That's Danielle Citrin. She's a professor of law at Boston University Law School, where she teaches and writes about privacy and free speech. You can see her full talk at TED.com. On the show today, ideas about technology, deception, and our changing sense of reality. And deepfakes make up one disturbing side of misinformation. Another conspiracy theories. Sure, they've been around a long time.

Classics like the Earth is flat, or another one that just won't go away. There's a secret cabal of shadowy, wealthy elites who either explicitly are named as Jews or just kind of fit into typele mold that sounds like they're probably Jews. In the past few years, these conspiracy theories, along with a whole set of new ones, have moved from the extreme fringes, into the American mainstream. Ever there were a question, whether a deep state existed? You got your answer this week.

You believe George Soros is behind all of this, paying these people to give. All of this with the global warming and a lot of it's a hoax. It's a hoax. I mean, it's a money-making industry. Well, they will be advancing their new conspiracy theory and their newest hopes. The most recent and obvious example. An online conspiracy theory. It's unclear if it's right.

Q and on. Conspiracy theories have been repeatedly debunk. Despite this, the far-right group continues to manage. A network of conspiracy theories all leading back to a mythic anonymous leader named Q, allegedly a high-level government official who has access to top-secret information. Well, I don't know much about the movement other than I understand. They like me very much, which I appreciate. But I don't know much about the movement.

Yeah, so QAnon starts with the assumption. that Donald Trump is secretly saving the world and that, you know, he doesn't want credit for it. He doesn't want to boast about it. But he's actually the only person who is ferreting out this massive deep state conspiracy that involves hundreds of people engaging in child sex trafficking and satanic cannibalistic rituals. It's like, sorry, what? And, you know, that is just straight up misinformation slash disinformation. So in a way, that's almost a less ambiguous case because it's so completely bonkers, honestly. And it's almost like if I were writing the script, I would be like, nah, guys, that's too on the nose.

Like, that's too much of a completely bizarre. Yes, no one will believe it. But, you know, millions of people do. This is journalist Andrew Morantz. I am a staff writer at The New Yorker magazine, and I wrote a book called Anti-Social. I have to make sure I remember the subtitle. Online extremists, techno-utopians, and the hijacking of the American conversation. Even before Q&N entered mainstream conversation in 2017, Andrew noticed a huge rise in far-right extremism and conspiracy theories online.

starting 2014 or 2015 around there, I started seeing this informational crisis on the horizon. I certainly was not the only person. But I was kind of behind the curve at the time because I wasn't back then thinking of this as a particularly political story. I was thinking of it as, you know, a business story or a tech story or. Yeah. But then the summer of 2016, I really started saying, okay, this is going to have a massive impact on the presidential election. I mean, in a way, how could it not, right?

Racist memes, misogynist propaganda, viral misinformation. So I wanted to know who was making this stuff. I wanted to understand how they were spreading it. Ultimately, I wanted to know what kind of impact it might be having on our society. Here's Andrew on the TED stage. So that's how I ended up in the living room of a social media propagandist in Southern California. He was a married white guy in his late 30s. he had a table in front of him with a mug of coffee,

a laptop for tweeting, a phone for texting, and an iPad for live streaming to Periscope and YouTube. And yet, with those tools, he was able to propel his fringe, noxious talking points into the heart of the American conversation. For example, one of the days I was there, a bomb had just exploded in New York, and the guy accused of planting the bomb

had a Muslim-sounding name. Now, to the propagandist in California, this seemed like an opportunity. Because one of the things he wanted was for the U.S. to cut off almost all immigration, especially from Muslim majority countries. So he started live streaming, getting his followers worked up into a frenzy

about how the open borders agenda was going to kill us all and asking them to tweet about this and use specific hashtags, trying to get those hashtags trending, and tweet they did. Hundreds and hundreds of tweets. It must have been kind of weird for you, like sitting there and watching

how public manipulation works from a couch. Yeah, what you see when you sort of sit at someone's elbow and watch them do this is it's like getting good at poker or something. You kind of learn the basic mechanics of the thing and then you play a lot of rounds. And if you're good enough, you know, you don't win every round, but you win a lot of them. So what that means in terms of social media is you kind of get a sense of what the algorithms want. and the really simplistic way of putting it is that they want whatever has the sharpest emotional impact on the viewer. And specific kinds of emotions, too. Emotions that make people do something with their either click or share.

But for the most part, he wasn't breaking the rules of Twitter or whatever platform he was using. He was just really good at getting his message out there. And so this guy in California and all the other folks you spent time with, like what was their mission? Was it just to create chaos, to tear down democracy? Because that guy doesn't seem like he really believes in this stuff, but there are people who do. Yeah. It's a spectrum, right? It's sometimes we talk about, you know, people who are doing this for profit cynically who don't believe what they're peddling. And then people, on the other hand, who are true believers. And I think that is a true and worthwhile distinction.

And I guess all I would add is that there are many shades of gray in between. Right? So it's not purely immediate monetary motivation in most cases. And then you'll get some cases where it is just people who have just been radicalized or red-pilled, as they call it. And they just think the world will not be safe until we have a white ethno state. And obviously those people are hard to deal with because they're pretty far gone. I talked a lot with one young woman who grew up in New Jersey. And then after high school, she moved to a new place and suddenly she just felt alienated. and cut off and started retreating into her phone. She found some of these spaces on the internet where people would post the most shocking, heinous things. And she found this stuff really off-putting, but also kind of engrossing. She started interacting with people in these online spaces,

and they made her feel smart. They made her feel validated. She started feeling a sense of community. Started wondering if maybe some of these shocking memes might actually contain a kernel of truth. A few months later, she was in a car with some of her new internet friends, headed to Charlottesville, Virginia,

to march with torches in the name of the white race. She'd gone in a few months from Obama supporter to fully radicalized white supremacist. Okay, so for those of us who just can't wrap our heads around how someone's ideas about the world can change so rapidly, how does that happen, especially with something like QAnon,

which is really an entire mindset? Yeah, so it's sort of like any cult, know, you start with the stuff that sounds less controversial, and then the more and more people get initiated, the more they're prepared to believe more and more outlandish things. So you kind of start with parts of it that are closer to the truth. Like, there was this guy, Jeffrey Epstein, who was really was doing all kinds of outrageously terrible things, and he really was friends with Bill Clinton and Prince Andrew. And there really was a conspiracy to cover up those crimes that is still ongoing.

And then you kind of go from there to, you know, I bet Hillary was involved and I also bet Tom Hanks was involved and I bet Oprah's probably involved. And they all have a dungeon somewhere where they're locking up children and trying to harvest their adrenal glands. Okay, you lost me with the adrenal glands. Well, I think when you start going down that rabbit hole,

part of it is engaging in a kind of collective fan fiction and part of it is, wait, wait, but what if this is real? And it's flirting with that line. And, you know, I can get that. to some extent. I get the thrill of being like, what if there really was an Illuminati? And it's just to a certain kind of person at a certain desperate moment in their lives or just spend too much time on the internet. They can't keep those blurry lines straight and it becomes their entire reality. I can understand how in a time of financial problems, insecurity, this idea of being part of a movement

and having meaning in your life and being, you know, part of like the revolution, right? Absolutely. That's very exciting. Yeah. Part of the underground. Part of the resistance. And having secret knowledge that no one else can see except for your compadres. Yeah.

It's just bizarre how far it can go. And some of these tropes are, you know, centuries old in some ways. They are not new. Yeah. I did not expect the fact that I am Jewish to do. matter in any way. I didn't expect to be talking about things like protocols of the elders of Zion and like mine comp and stuff. I thought frankly that they would be a little more original than that. But as it turns out, there are certain tropes that just refuse to go away. And it does a lot of work for

people. It helps explain things that are otherwise unexplainable. You know, why is the economy so obviously seem to be rigged against me. Why can't I find meaningful work? Or why do I have work, but I still don't feel like my life has any purpose? Why, you know, on and on and on with these kind of sometimes unanswerable questions. And if the answer is, because there are 10 people in a room somewhere saying, I don't want people to have meaning in their lives, then in a way, that's kind of a comforting explanation, because it means that there's at least a nameable reason or an identifiable enemy. And often that's Jews, often that's women. Often it's just whoever's a visible other in terms of being a person of color or what have you. But I think one of the notable things for me

is not that these tropes still exist in the world, but the fact that they can be revived in terms of popularity and in terms of salience to the national discourse. That I did find surprising. I mean, it feels as though this idea that a single person switching on a story, that changes people's perception of what reality even is has become so commonplace that we are in the midst of an era of very little trust. Yes, it is commonplace now. And the companies have had a lot of time to try to figure this out. In some ways they have, you know, it is no longer okay on Facebook to buy an ad in an American election using rubles as the crazy. And that was a loophole that was open in 2016.

That should not have been open, but it was. They did close that loophole. But the larger loophole, which is the entire incentive structure, the entire thing that social media is built to do, that hasn't changed. And yeah, as a result, we are living in a pretty confused and confusing time. So what do we do in the meantime? Like, how do we fix this at least a little bit? Yeah. So a lot of the bigger solutions to this are going to have to be systemic.

and the companies are going to have to step up, and it might involve government regulation and all kinds of bigger things. But until they rebuild and dismantle their business model, there are things that individuals can do. And one of them, I call it being a smart skeptic. So there are things that pass for skepticism online that I think are actually just knee-jerk, contrarian trollery.

So you often see people saying, well, I'm just asking for more evidence, and I'm just asking the question. but that is not real skepticism. Real skepticism is being open-minded, but not being so open-minded that your brain falls out. Demanding evidence, but not demanding evidence past the time when a question has been settled. If you just sort of say, well, I don't know, you know, everybody says racism is bad, but like, I'm skeptical of that claim. I don't think skepticism is the best word for what you're doing there.

And sometimes there just is consensus on something, and there's a certain catholicism. of mind of a person who just doesn't want to hear that. It's a kind of addiction to feeling like you have secret access to knowledge that, you know, the boundaries of polite society doesn't want you to have. If only we all had that secret access, right? Yeah. I mean, it's a thrilling idea. It's just that sometimes the real answer is the answer that most people already believe. And I'm sorry if that's boring, but it just sometimes is the case.

That's Andrew Morance. He's a journalist and staff writer for the New Yorker. You can see his full talk at TED.com. On the show today, technology, and deception. I'm Manus Sh Zamoroti, and you're listening to The TED Radio Hour from NPR. It's the TED Radio Hour from NPR. I'm Manus Shumeroati. On the show today, technology, deception, and our changing sense of reality.

And so far, we've been talking about deep fakes, conspiracy theories, and other kinds of misinformation. But data and algorithms, they can warp our reality too. We can deceive ourselves into thinking they're not doing harm, or we can fool ourselves into thinking because it's based on numbers, that it is somehow neutral. AI is creeping into our lives, and even though the promise is that it's going to be more efficient,

it's going to be better, if what's happening is we're automating inequality through weapons of math destruction and we have algorithms of oppression, this promise is not actually true and certainly not true for everybody. Weapons of math destruction, algorithms of oppression, which basically means bias and human error can be encoded into algorithms leading to inequality. To keep them in check, the Algorithmic Justice League to the rescue. My name is Joy Blumwini. I'm the founder of the Algorithmic Justice League,

where we use research and art to create a world with more equitable and accountable AI. You might have heard of the male gays or the white gays or the post-colonial gaze to that lexicon. I add the coded gays, and we want to make sure

people are even aware of it because you can't fight the power. you don't see, you don't know about. Joy hunts down the flaws in the technology that's running every part of our lives, from deciding what we see on Instagram to how we might be sentenced for a crime. What happens when somebody is harmed by a system you created? You know, what happens if you're harmed? Where do you go? We want that kind of place to be the algorithmic justice league so you can seek redress for

algorithmic harms. You are a lot of things. You're a poet. you're a computer scientist. You are a superhero. Like, kind of hard to put into a box. Can you just explain why you created the algorithmic Justice League? Yes, so the algorithmic Justice League is a bit of an accident. When I was in graduate school, I was working on an art project that used some computer vision technology to track my face.

Hi, camera. I've got a face. Can you see my face? At least that was the idea. You can see her face. What about my face? And when I tried to get it to work on my face, I found that putting a white mask on my dark skin

is what I needed in order to have the system pick me up. And so that led to questions about, wait, are machines neutral? Why do I need to change myself to be seen by, a machine. And if this is using AI techniques that are being used in other areas of our lives, whether it's health or education, transportation, the criminal justice system, what does it mean if different kinds of mistakes are being made? And also, even if these systems do work well,

let's say you are able to track a face perfectly, what does that mean for surveillance? What does it mean for democracy, First Amendment rights, you know? Joy continues from the TED stage. Across the U.S., police departments are starting to use facial recognition software in their crime-fighting arsenal. Georgetown law published a report showing that one and two adults in the U.S., that's 117 million people, have their faces in facial recognition networks. Police departments can currently look at these networks, unregulated, using algorithms, that have not been audited for accuracy.

Machine learning is being used for facial recognition, but it's also extending beyond the realm of computer vision. So who gets hired or fired? Do you get that loan? Do you get insurance? Are you admitted into the college that you wanted to get into? Do you and I pay the same price for the same product purchased on the same platform? Law enforcement is also starting to use.

machine learning for predictive policing. Some judges use machine-generated risk scores to determine how long an individual is going to spend in prison. So we really have to think about these decisions. Are they fair? And we've seen that algorithmic bias doesn't necessarily always lead to fair outcomes. When I think about algorithmic bias and people ask me, well, what do you mean machine? are biased. It's just numbers. It's just data. I talk about machine learning and it's a question of,

well, what is the machine learning from? Well, what is the machine learning from? Like, what's the information that it's taking in? So an example of this, what I found was that for face detection, the ways in which systems were being trained involved collecting large data sets of images of human faces. And when you look at those data sets, I found that many of the were pale and male, right? You might have a data set that 75% male faces, over 80% lighter skin faces. And so what it means is the machine is learning a representation of the world that is skewed. And so what you might have thought should be a neutral process is actually reflecting the biases that it has been trained on. And sometimes what you're seeing is a skewed representation.

But other times what machines are picking up on are our own societal biases that are actually true to the data. For example, Amazon was building a hiring tool. You need a job. Somebody in your life needs a job, right? You want to get hired. And to get hired, you upload your resume and your cover letter. That's the goal. It starts off well. But before a human looks at your resume, it gets vetted by algorithms, written by software engineers. So we start off with an intent for efficiency. We have many more applications than any human could go through. Let's create a system that can do it more efficiently than we can.

And how to build that better system? Well, we're going to gather data of resumes, and we're going to sort those resumes by the ones that represented candidates we hired or did well. Your target is who you think will be a good long-term employee. And now the system gets trained on the data. And the system is learning from prior data. So I like to say the past dwells within our algorithms. You don't have to have the sexist hiring manager in front of you. Now you have a black box that's serving as the gatekeeper.

But what it's learning are the patterns of what success has looked like in the past. So if we're defining success by how it's looked like in the past, and the past has been one where men were given opportunity, white people were given opportunity, and you don't necessarily fit that profile, even though you might think you're creating this objective system, it's going through resumes, right? This is where we run into problems.

So here's what happened with Amazon's hiring tool. What happened was as the model was being built, and it was being tested, what they found was a gender bias where resumes that contained the word women or women's or even all women's colleges, right? So indication of being a woman were categorically being ranked lower than those that didn't. And try as they might, they were not able to remove that gender bias, so they ended up scratching the system. They scratched the system, and that's a big win. But one win compared to thousands of platforms that use skewed algorithms that could warp reality. It has not been the case that we've had universal equality or absolute equality in the words of Frederick Douglass.

And I especially worry about this when we think about techno benevolence in the space of health care. Right. We're looking at, let's say a breakthrough that comes in talking about skincare, skin care. cancer. Oh, we now have an AI system, right, that can classify skin cancer as well as the top dermatologist's study might say a headline might read. And then when you look at it, it's like, oh, well, actually when you look at the data set, it was for lighter skin individuals. Then you might argue, well, you know, lighter skin people are more likely to get skin cancer. And when I was looking into this, it actually darker skin people who get skin cancer, usually

it's detected in stage four because there are all of these assumptions you're not even going to get it in the first place. So these assumptions can have meaningful consequences. You know, we are talking just before the U.S. presidential election. Have you seen any examples of artificial intelligence being used in voting or politics? Yeah. So Channel 4 News just did this massive investigation showing that the 2016 Trump campaign targeted 3.5 million African Americans in the United States labeled them as deterrents in an attempt to actually keep people from showing up to the polls. They used targeted ads. Yes.

And we know from Facebook's own research, right, that you can influence voter, turnout based on the kinds of posts that are put on their platform. And they did this in battleground states. And so in this way, we're seeing predictive modeling and ad targeting, right, being used as a tool of voter suppression, which has always been the case to disenfranchise, right? You might say black lives don't matter, but it's clear black votes matter because of so much. effort used to rob people of what blood was spilled for, you know, for generations. So it should be the case right that any sorts of algorithmic tools that are intended to be used, again, have to be verified for non-discrimination before it's even adopted.

So as a black woman technologists, you know, they're not that many of you, frankly. why not, you know, go work at Google or Amazon and make these changes to the algorithms directly? Why act as sort of a watchdog? Well, I think there are multiple ways to be involved in the ecosystem. But I do think this question you pose is really important because it can be an assumption that by changing who's in the room, which is important and needs to happen, we're going to then change the outcome and the outputs of these systems. So I like to remind people that most software developers, engineers, computer scientists, you don't build everything from scratch, right?

You get reusable parts. And so if there's bias within those reusable parts or large-scale bias in the data sets that have become standard practice or the status quo, right, changing the people who are involved in the system, without changing the system itself is still going to reproduce algorithmic bias and algorithmic harms. So how do we build systems that are more fair? Like if there's no data for the artificial intelligence to sort of process to start to pump out recommendations, then how do we even change that?

Yeah, well, it's a question of what tools do you use towards what objectives? So the first thing is seeing if this is the appropriate tool. Not every tool, not every decision needs to be run through AI. And oftentimes you also need to make sure you're being intentional. And so the kinds of changes you would need to make systematically for even who gets into the job pool in general, it means you do have to change society to change what AI is learning. What do you say, Joy, to people who might be listening and thinking like, you know, let's let's, Let's take a step back and look at the bigger picture.

We, in many ways, things are way better than they were, thanks to technology. Because, you know, here we are in a pandemic and anyone can work from anywhere because we have the internet and we have Zoom and all of these platforms. Equality and access is on the whole improved. Why let's not like be Debbie Downers about it? Yeah, I mean, I always ask who. can afford to say that because I can tell you the kids who are sitting in McDonald's parking lot so they can access the internet to be able to attend school remotely. That has never been their reality. And so oftentimes if you are able to say technology on the whole has done well, it probably means you're in a

fairly privileged position. There's still a huge digital divide. Even there are billions of people who don't have access to the internet. I mean, I was born in Canada. I was born in Canada. I was born in Canada. I was moved to God and then grew up in the U.S. I had very Western assumptions, you know, about what tech could do and very much excited to use the tech skills I'd gained as an undergrad at Georgia's tech, you know, to use tech for good, tech for the benefit of humanity. And so when I critique tech, it's really coming from a place of having been enamored it with it and wanting it to live up to its promises.

I don't think it's being a Debbie Downer to show ways in which we can improve so the promise of something we've created can actually be realized. I think that's even a more optimistic approach than to believe in a wishful thinking that is not true. You know, one thing that you've said that I find so, I love this idea, that you say there's a difference between potential and reality and that we must separate those two ideas. Yes, so it's so easy to fixiate on our aspirations of what tech could be. And I think in some ways is this hope that we can transcend our own humanity, right? Our own failures.

And so, yes, even if we haven't gotten society quite right, ideally we can build technology that's better than we are. But we then have to look at that fact that technology reflects who we are. It doesn't transcend who we are. And so I think it's important that when we think about technology, we ask what's the promise, what's the reality? And not only what's that gap, but who does it work for? Who does it benefit?

Who does it harm and why? And also, how do we then step up and stand up to those harms? That's Joy Bualamweeney, founder of the algorithm. rhythmic Justice League. You can watch her full talk at ted.com. Thank you so much for listening to our show this week about technology, deception, and our warped reality. To learn more about the people who were on it, go to ted.n.npr.org and to see hundreds more TED talks, check out TED.com or the TED app. Our TED Radio production staff at NPR includes Jeff Rogers, Sanaz-Meshkenpour, Rachel Faulkner, Diba Motisham, James Delahousie,

J.C. Howard, Katie Montalione, Maria Paz Gutierrez, Christina Kala, and Matthew Cloutier, with help from Daniel Schuchin. Our intern is Farah Safari. Our theme music was written by Ramteen Arablewe. Our partners at TED are Chris Anderson, Colin Helms, Anna Feelein, and Michelle Quint. I'm Manus Samarodi, and you've been listening to The TED Radio Hour from NPR.