Tech Won't Save Us - Don’t Fall for the AI Hype w/ Timnit Gebru

Episode Date: January 19, 2023

Paris Marx is joined by Timnit Gebru to discuss the misleading framings of artificial intelligence, her experience of getting fired by Google in a very public way, and why we need to avoid getting distracted by all the hype around ChatGPT and AI image tools.Timnit Gebru is the founder and executive director of the Distributed AI Research Institute and former co-lead of the Ethical AI research team at Google. You can follow her on Twitter at @timnitGebru.Tech Won’t Save Us offers a critical perspective on tech, its worldview, and wider society with the goal of inspiring people to demand better tech and a better world. Follow the podcast (@techwontsaveus) and host Paris Marx (@parismarx) on Twitter, and support the show on Patreon.The podcast is produced by Eric Wickham and part of the Harbinger Media Network.Also mentioned in this episode:Please participate in our listener survey this month to give us a better idea of what you think of the show: https://forms.gle/xayiT7DQJn56p62x7Timnit wrote about the exploited labor behind AI tools and how effective altruism is pushing a harmful idea of AI ethics.Karen Hao broke down the details of the paper that got Timnit fired from Google.Emily Tucker wrote an article called “Artifice and Intelligence.”In 2016, ProPublica published an article about technology being used to “predict” future criminals that was biased against black people.In 2015, Google Photos classified black women as “gorillas.” In 2018, it still hadn’t really been fixed.Artists have been protesting AI-generated images that train themselves on their work and threaten their livelihoods.OpenAI used Kenyan workers paid less than $2 an hour to try to make ChatGPT less toxic.Zachary Loeb described ELIZA in his article about Joseph Weizenbaum’s work and legacy.Support the show

I think at the other end of it, it's kind of the same issues we have with social media companies, the spreading of disinformation, trying to centralize power, moderation, and exploiting people, like a labor force that's sort of second-class citizens around the world. So I think that's kind of the path we're on. Hello and welcome to Tech Won't Save Us. I'm your host, Paris Marks. Just a quick reminder that we are running a listener survey this month, so if you have about five minutes, you can find the link to that in the show notes if you want to fill it out for us. Definitely appreciate if you can do that. Now this week, my guest is Timnit Gebru. Timnit is the founder and executive director of the Distributed AI

Research Institute, or DARE. And she's also the former co-lead of the ethical AI research team at Google before being fired in 2020. And that might be a story that you will remember quite vividly. It won't be news to you that we are in the middle of a hype cycle. You know, there's a lot of excitement right now around particular AI tools like chat GPT or stable diffusion, churning out writing and images based on things that they have scraped from the internet. As a result of that, there are a lot of big promises being made right now around what potential future these AI tools might bring, what our society might look like because of

all of these things that these tools can churn out based on all the data that they have taken from the web. But are these things exaggerated? Are we likely to actually realize a lot of these things that people at OpenAI and other boosters are saying we might? I think that there's reason for us to be skeptical of these things. First of all, because a lot of these promises have been made in the past and not followed through on. But second of all, because it's very beneficial to a lot of these people for us to buy into it and for us to believe that these AI tools are the next big thing after the failure of cryptocurrencies, after the failure of the metaverse. Tim Neat is also throwing some cold

water on these visions of the future being put forward by these AI hype masters. And I think it's a perspective that we should listen to. You know, Tim Neat is obviously much more familiar with all this than I am. She previously worked on large language models, and it was her work on that that ultimately got her fired from Google because the things that she was saying and trying to publish about these models, the reasons that we should be critical of them and thinking more critically about them, are things that Google didn't want us thinking about and didn't want to be brought to the public's attention as these tools could make up a significant piece of the future business model of companies like Google itself. So in this conversation, we talk about all this hype,

we talk about what these AI tools might mean. But of course, we also talk about Tim Neat herself and her own history, because I did want to know a bit more about what happened at Google and how it came to the point where, you know, she was ultimately fired by this company that many people see as, or at least have traditionally seen as, you know, a positive force in the valley or in the tech industry. And obviously that has started to shift in recent years, especially as, you know, the do not be evil slogan was officially kind of taken off of the company. So I hope you enjoy this conversation. I certainly did.

If you like it, make sure to leave a five-star review on Apple Podcasts or Spotify. You can also share it on social media or with any friends or colleagues who you think would learn from it. And if you wanna make sure that we can have more of these critical conversations on AI, which is a topic that I think we'll be returning to

quite a lot this year, as it seems to be the next big thing that this heck industry is trying to sell us, your support is definitely appreciated so that we can do that. And so you can join supporters like Randy Weinstein from New York, Patrick from Chicago, and Ben Finch from Boston by going to patreon.com slash techwontsaveus if you want to do that. And with that said, enjoy this week's conversation. Tim Neat, welcome to Tech Won't Save Us. Thank you for having me. I'm a huge fan of your work, and I am very much looking forward to reading your book.

But given all that's on our plate, it's like really hard to figure out how to do things that we need to be doing, like read books. I completely understand that. I have like a million books on my reading list. You know, I get sent books all the time. And I'm like, where am I going to find the time for all these? So yeah, no pressure. But I have like a million books on my reading list. You know, I get sent books all the time. And I'm like, where am I going to find the time for all these? So yeah, no pressure. But I have bought it.

Yeah, I appreciate that. When you get around to it, though, you know, you can certainly let me know what you thought of it. Definitely looking forward to it. I have obviously been following you and your work for a while as well. You know, I think a lot of people will know you and when you came to their attention. Admittedly, I didn't know your name before you were fired from Google, but at least we got to know you. Many more people did at that time. One upside to that whole situation, I guess. And I'm really excited to talk to you today about, you know, artificial intelligence,

all this hype that has been around it lately, how we should actually be thinking about these technologies and what they might actually mean for us, you know, going forward and into the future, especially as, you know, with the failure or loss of interest, I guess, in kind of Web3 and crypto and metaverse, seems like AI is going to be one of these technologies that the industry is refocusing on and trying to, you know, build this hype around into the future, right? And so, as I said, you know, we haven't talked very much about AI and machine learning and these kind of topics on the show previously. Certainly, they've come up. But I was hoping that you could give us a general understanding of what it

actually means. What is AI or artificial intelligence? What does that refer to? Is machine learning the same thing? Is that different? How should we understand these terms and what they actually mean? So I want to start with a comment that, you know, I got my PhD from a lab that's called the AI lab at Stanford. I have my PhD in electrical engineering, but the lab is in computer science department. And I find myself asking the questions that you're asking right now. So I was to start with that, right?

And so as a researcher, actually, I had never described myself as a quote unquote AI researcher until that term became like the mainstream term. And it was like super hyped up. I remember about, I would say about 10 years ago is when the hype started. And so everything has been rebranded as quote unquote AI. And so I first and foremost think of it as a brand, like as a rebranding, more of a marketing term. And some people, I think there was an article by Emily Tucker from Georgetown Law Center for Privacy and Security. The title is Artifice and Intelligence and kind of talking about how it's a misnomer. We should not call it artificial intelligence.

So I'm just that's just an aside. But so the way I look at it, I look at artificial intelligence like a big tent, a big field with subsets of things inside that field. So I would say my understanding of the field is that you try to create machines or things that can do more than, you know, what's been programmed into them, right? And so this could mean natural language processing, where people are more interested in analyzing text and speech. And so then they can create things like speech to text transcriptions in an automated way or automated machine translation from A to B, from one language to another language, et cetera.

It could be computer vision techniques where, you know, you want to look at what's in an image and kind of analyze that image more so than just taking it. Maybe you can do some statistics. Maybe you can say, you know, there's a chair analyze that image more so than just taking it. Maybe you can do some statistics. Maybe you can say, you know, there's a chair in that image. There's a hat, a person, a house, whatever. And that would be computer vision. And these are all under subsets of, at least at Stanford, they're under subsets of the AI lab.

There's robotics where people are working on robotics. And then machine learning is more of a technique. Machine learning would be a technique that many times people try to use for many of these various things, right? So you could use machine learning techniques in natural language processing, you can use machine learning techniques in, you know, computer vision, etc. So nowadays, because a specific type of a machine learning technique is one that is almost everywhere, people use many of these things interchangeably, right? They use AI, machine learning, deep learning interchangeably.

But it wasn't always like that. Actually, the deep learning people, I remember Jan LeCun, I think it was in 2012 or something, literally wrote an open letter to the academic community, the computer vision community, being like, you are not accepting my papers and you guys don't like us, the deep learning people and whatever. We're going to persist, you know. And now it's like they're the dominant paradigm. It's as if there was no other paradigm. So they're not all the same things, but they're often used interchangeably. Yeah. A really big tent, I guess, of a lot of things that are going on there in a term that we hear more and more often, I guess, in the way that you're describing it there

as more of a marketing term. Do you also feel that it's kind of misleading us and making us believe that these technologies can do things that they can't? Like, I think a bit as well, when we talk about like, smartwatches and smart gadgets and all this kind of stuff, it's a particular term that's created by PR people at these companies, I'm sure, that make us believe that, you know, this is a technology that's really smart, right? It has this term that we obviously think is very important that's associated with it, instead of calling it something else that wouldn't give it these particular connotations. Is the term artificial intelligence making us believe that these

technologies operate in ways that maybe aren't actually reflective of what the term would have us think? Yeah, absolutely. So many times there is a bunch of things going on. I don't only blame the corporations or other entities that might have a vested interest in marketing terms. I also blame researchers who definitely feed into the hype. So first is the naming of areas of study, aspirational naming, right? It doesn't mean that things are the way they are, but it's aspirational. So neural networks is one, right? So maybe for some people, the brain might be an inspiration, but it doesn't mean that it works like the brain. And so some neuroscientists are like, why are you coming to our conference and saying this thing is like the brain? Come on, you know, like, but that's aspirational, right? Computer vision can be full of hype then there is also the conflation of a bunch of things so whereas i look at ai as you know a field of studying with a bunch of stuff set like i told you there are a bunch of people whose interest and whose goal is to build what's called artificial general intelligence, AGI, which I personally don't even know what it is. It seems like a god to me. It's going to be here in a few years, though, don't you know?

Are we already there? I'm not exactly sure. I mean, it's this all-purpose, all-knowing god, where if you build it the right way, you build a god. If you build it the wrong way, you build the devil. So it's also the single existential risk for humanity, but we're building in. I just, I'm lost, you know? So there's also that. So because there's that segment of the population that is the loudest and has the most money right now, they also influenced the discourse around any type of AI discourse, because what they try to do is make it look like everything they're building is kind of that, AGI, kind of has AGI characteristics, right?

That's another kind of hype angle. But then when we go to the corporations and other vested entities, another reason where this hype helps is that if you make people think that you're building something that has its own agency, who knows what it's going to do. And it's so capable of doing this and that people are more focused on the thing because they think it has more agency than you as the builder of the system, the deployer of the system, the entity, the regulation, whatever. Because now responsibility is sort of not your responsibility, but it's like, who knows

how the machine will behave? So everybody sort of starts thinking like that rather than holding relevant entities accountable because it's an artifact that people are building and deploying. So at both ends of the spectrum, you have people building it and deploying it and people, you know, being harmed or benefiting. So when you move the conversation away from that, it makes it easy to evade responsibility and accountability. And then also here we have Hollywood, which really just has not been helping, right? So anytime people hear the term AI, the number one thing that comes to mind is either Terminator or some other, you know, I don't know, whole Hollywood-y thing. And so again,

that's also what's in the public consciousness. So none of these things are helping us in kind of really having a real understanding of what's real and what's hype. Yeah, they're really good points. And, you know, especially the makers of the technologies being able to use this to kind of get the blame or get accountability pushed to some other place, right? Like, oh, we couldn't have known it was going to do this when we developed it and trained it on these particular data sets and what have you, right? I want to keep exploring this thread, but I want to detour a little bit to give us another path into it, because I want to talk a bit about your career as well. You know, I'm interested, obviously, there are many different routes you could have taken into technology, into the tech industry, different things that you could have explored. Why was this area of technology of interest to you when you were doing your studies, your PhD, and then working in the industry? So, you know, it's really all I've ever wanted to be is an engineer, a scientist. I'm interested

in building things. I'm interested in learning about things. I'm just, you know, and that was really my goal for a long time. Since I was little, I was interested in certain subjects, etc. So that's really all I tried to do, right? But then during that kind of just trying to live my life, you have various experiences. You know, I had to leave because of war. Then I came to the States and just, you know, the racism was just like in my face since day one. I went with teachers telling me that I can't take classes, that I would fail if I took the exams these people take and all of that. That experience persisted for a long time. So I was, you know, I had a very, I would say,

you know, a kind of visceral understanding of some of these issues. However, what's really interesting is that I still hadn't really connected the tech, you know, so I would understand how I'm facing sexism in the workplace, right, while building a product or whatever. But I never really had an understanding of how the tech itself was perpetuating these things or how it was being used by powerful entities or the militarization of like the you know these universities like MIT and you know Stanford and Silicon Valley whatever that all came honestly way way way way way way later and so I was chugging along doing my engineering stuff and I was like oh I'm gonna start doing what's called analog circuit design then I did that

then I veered off into something else then I veered off into something else. Then I veered off into something else. And I somehow arrived at computer vision, which is a subset of AI, right? And so I was like, oh, this is super cool. Let's do this. And there was just three things that happened simultaneously. One is that I was, you know, shocked by the lack of Black people in the field. So, you know, graduate school was

even much, much, much more shocking than an undergrad. So in graduate school at Stanford, I realized that they had literally only graduated one Black person ever with a PhD in computer science ever. Now it's two, by the way. Yeah, since the inception of the department ever, right? 50-something. Yeah, exactly. And then you go to these conferences. And I remember before I started Black Nei, I counted like five Black people out of 5,500 people attending these international conferences, right? From all over the world. So that was one, just like dire lack of Black people. Then again, a fluke. Secondly, I started watching Joy Boulamwini's talk. She told me I don't say her name right. And we keep on saying each other's name,

Boulamwini. And she told me she doesn't say my name right. It was hilarious. We were like, just repeating each other's name to each other. But anyways, I saw her talk. And she was talking about how these open source face detection tools wouldn't detect her face unless she puts on a white mask. And they would detect, you know, her friends and other people's faces, but not her face. Right. And so I saw that around again, like around 2015, a very similar time. Like I said, when I wrote about this, just counting five black people out of 5,500 in the AI conferences, that was 2016. And then again, in 2016, I read the ProPublica article on crime recidivism. And so

there was this article that talked about a particular startup that purported to have a model that can tell you the likelihood of someone committing a crime again. And, you know, judges and courts were using this data with other kind of input to deter it, to set bail, to determine, you know, how many years you should go in prison for or whatever. And like, I had no idea that this kind of stuff existed because that's when I put all of these things together. I'm like, I know the people who are building these systems. I go to school with them. I work with them. I go to conferences with them. I know how they think. I know what they say when I talk to them about police brutality. You know, I know. And so these are the people building this. Are you kidding me? And then it's going to be used like just I already could tell,

you know, could imagine the issues, right? And also could see how other people were not really caring about these potential issues. Around the same time, the Google gorillas fiasco happened, where they were classifying people as gorillas, right? Black people as gorillas. So all of these things sort of happened at the same time. And then the final piece was the hype. Basically, you know, I had been going in and out of my PhD for a long time. I tried this thing, dropped out, this other thing, dropped out. And then so when I went back, finally, working on my PhD in computer vision, it was literally at the inflection point before and after of the hype in AI so before it was there was no hype I started my PhD when there was no

hype just in like two years it started exploding and so then around the whole Google guerrillas fiasco thing open AI was announced and oh my god I wrote this piece just for myself because I was just so angry. I was going to submit it as an open letter to somebody. My friend was like, everybody's going to know it's you. So it's not anonymous. Like, so it's just, I kept it to myself. And I was so angry because they were billed. So at that time they were a nonprofit. They changed into a for-profit, but they were a nonprofit and they basically were saying that they were going nonprofit. They changed into a for-profit, but they were a nonprofit and they basically were saying that they were going to save humanity from the dangers of AI. It was going to be an AI safety first company. They were worried that these large corporations who are profit

driven, we don't know, they're not going to give us the share their data and stuff are like controlling AI, which is very important and it's going to give us the share their data and stuff are like controlling AI, which is very important, and it's going to change the world. And it's like these eight white guys, and one white woman and one Asian woman in like Silicon Valley, all with the same expertise in deep learning, plus Peter Thiel and Elon Musk. I was just like, you gotta be kidding me, right? So these are the situations that led to me kind of starting to focus, pivot a little bit and try to like, you know, learn more about the potential harms and things like that. I appreciate you outlining that because I would imagine that a lot of people who are in the tech industry who listen to this podcast or don't have

kind of, I don't know, had an awakening about the politics of AI, you might say, you know, have a similar story. Sure, there might be some different experiences in there that they have experienced personally in seeing these things. But I imagine they went into it not thinking so much about the impacts of these technologies in the world. But then after working on it, after learning a bit more about, you know, what they are working on, they start to realize more about the impacts of these technologies and how they are kind of part of this system that is creating that, right? I want to fast forward a bit, you know, obviously, I can't not ask about it, but your time at Google, when you were the co-lead of the ethical AI research team, as far as I'm aware, you know,

there were a lot of people who recognized that this was a really diverse team that had been put together to really dig into these issues, to provide a critical look at what was going on in the AI space and at Google in particular. And then, of course, we all know the story of your firing in 2020, I believe it was, and, you know, just everything that went down there in the press, you know, what was that whole experience like? And what was it that Google found so objectionable about the work that you were doing to lead it to take such extreme action against you? So those are two years, but it was like so much in those two years. So by the time there was no question who I was at this point, right?

Joanna had published this paper showing the disparities in error rates and face recognition systems among people of different skin tones and genders. And I had, you know, co-founded Black Nei. We had our first workshop. I was very vocal about the issues of racism and sexism in the industry, etc., etc. So I joined Google September 2018. And I was already very, very scared. When I was about to join, I was just nervous. I'm like, what am I, where am I going? Like, I talked to a bunch of women who had a lot of issues of harassment who sat me down and telling me to think twice about it.

And so then Meg Mitchell, who was there at the time, and, you know, Meredith Whitaker was also there at the time I knew both of them. And that was during the whole Maven, you know, kind of protest. And so actually, to me, it was like the fact that there were people like that was a big deal. Because it means, okay, I can, you know, you know, there's dissent anywhere. But like, if you don't look like you have it, it means, you know, it's, it's a, it's bad. It's a bad place. So at least I can have that. And then I was thinking, you know what, Meg, you know, is someone I can really work with. She had started the ethical AI team at the time. And it was just like a couple of people. She asked me if I could co-lead the team with her. And I was like, well, at least I have this one person I could work with. And I joined the team.

And so right off the bat, November, there was the Google walkout. And right off the bat, I was seeing just the systemic issues, sexism, racism. Now, this is kind of not exactly about the work itself, right? But just kind of it impacts the work, the organizational dynamics of who gets to make decisions, who's valued, who's not obviously directly impacts what the output is. Totally. It's the wider context of the company that you were entering into, right? Absolutely. So I already I started making noise immediately. And after a few months, I was certainly not the friend of HR.

You know, like clearly I was not the friend of like the higher ups. Like, you know, it's just, you know, so that that is how it started. Then I really started fighting for our team, our team to have a voice, to be at different tables, decision making tables, to grow our team, to hire more people. And we had to fight for literally like every single person we were trying to hire. Right. We hired social scientists as research scientists for the first time. Dr. Alex Hanna was, she's now director of research out there. She was the first person who we hired as a social scientist to be a research scientist because we were like, you know, you just, you need different perspectives.

Is that what, if that's what we were really trying to do. So our work was, you know, we were primarily a research team, but we also, we had lots of teams coming to us for questions. If they were gathering data, you know, how are they going to annotate it? What are some of the issues? We worked on things like, for example, we had a paper called model cards and this was Meg spearheading it saying that, you know, any model that you put out there has to be accompanied by a set of tests, you know, and kind of think about it with tolerance. Like I did this, you know, as an engineer, like there's no way you would put out any products out there without extensive documentation and testing what is supposed to be used for. What are the standard operating characteristics?

What are the ethical downstream considerations, what was your task in mind when you did it, that kind of stuff, you know? So we were doing all this stuff. A bit of the opposite of the move fast and break things ethos, right? Absolutely. And you know what's really interesting is in hindsight, it's very clear that even something as simple as that. So I published a few papers like that where I didn't really have any kind of opposition from people in corporations or engineers, because I'm saying these are like engineering principles. You know, I'm not saying, you know, politically, you have to do this and that.

But that's super political, because what you're saying is that instead of making $100 in like one week by doing this, whatever thing you're doing, first of all, spend one year instead of one week because do these additional tests, etc. Secondly, like hire additional people to do these things. So I'm saying make less money per thing, put more resources per thing. And sometimes you may not even want to like release this thing. So we had to figure out in our team, we want to get people rewarded, like promoted and all of that, not punished. Right. So how do we say, Hey, we stopped you from releasing five products, you know, because they were harmful. You know, how do we like, how do we talk about it in these

terms? So that was sort of what I was doing. That was my job. And, you know, I was extremely tired, exhausted from all the fighting of, you know, sexism, racism, whatever, trying to, you know, get our team at various tables. But we grew our team, and it was definitely a very diverse team in many, in many different ways. So then came 2020. So at this point, there was the Black Lives Matter protests. And then there was OpenAI's GPT-3 that was released. And at that time, you know, it's not like this was the first language model that was released. But they were the first people who were basically talking about this as like this all-knowing, all-encompassing, just hyping it up, like ridiculous amounts of hype. And that seeped into the entire industry.

So all these researchers everywhere, like chats are like on the chat be like, oh my God, this is so cool, it's so cool. And then the, you know, higher-ups be like, why are we not the biggest? Why don't we have the biggest model? Why don't we? And I'm just like, oh my God, why?

What's the purpose of having the biggest, what kind of pissing contest is this, right? That's how it felt. And then at the same time, a bunch of people at Google were asking our team, when we're thinking about implementing infrastructure for large language models and things like that, what are the ethical considerations we should have? So at some point I was like, look, we got to write a paper or something. So I contacted Emily Bender, who is a linguist. She's written about curating data sets, documenting them, etc. I'm like, voices like hers need to be heard prominent. So I contacted her and said, hey, all these people at Google are talking about large language models. They're asking me questions. Some of them are just want to do bigger and bigger. Like, is there a paper I can send them? And she said, no, but why don't we write one together?

So that's what happened. We wrote one together. I was like, oh, excellent. Oh, we have a little emoji as a title of the paper. That's going to be cool. That was it. It was not a controversial paper, right? So we outlined the risks and harms of large language models. And so she coined the term stochastic parrots. You know, there was like other members of our team, obviously Meg Mitchell, who got fired three months after me. You know, we go through some of the risks that we see. One was environmental racism, because you can use lots of compute power and training and, you know and using these models. And the people who benefit and the people who pay the cost are different, right?

So that's environmental racism. We talk about all sorts of risks of bias and fairness and things like that because these models are trained with vast amounts of texts from the internet. And we know what's on the internet. So we spend a lot of time on that. Unfortunately on the internet. So we spent a lot of time on that. Yeah, we spent a lot of time on that. We talk about how just because you have so much data doesn't mean that you have quote unquote diverse viewpoints. We talk about how there are risks of people interpreting outputs from these models as if it's coming from a person. Of course,

that was on the, you know, like that happened. It was on the news. The other risk that we talked about was the risk of putting all of your resources and research direction in this thing and not and not other things. And so that was it, you know, was under review, whatever. And then randomly, I was told that after it went through all these internal processes that we had to retract the paper or, you know, remove our names. And first it was about retraction. And I was like, well, we have external collaborators.

And I also don't trust what you're going to do if we just retract the paper. It's not like you're discussing improvements or specific issues. So then they said, OK, you can just retract the names of the Google authors. And so even that, I was like, you know, I'm not comfortable with that, doing that without a discussion. I want to know what kind of process was used because I can't be doing research. I'm not in the marketing department, right? I'm in the scientific research department. If it was like PR slash marketing, cool.

Yeah, do whatever you want. If you're having me as a scientist to write peer review papers, now this is different. And so of course, then, you know, long story short, I was basically fired in the middle of my vacation. And then it was a whole, you know, public outcry. And then I had a lot of like harassment campaigns and threats and all of this. And so that's what happened at Google. I appreciate you outlining it in such detail for us, because I think it gives us a really important insight into the work that you were doing, right? And I do want to pick up on some of those pieces of the paper that you were talking about, right? Because these are really important

things that I feel like don't get the attention that they actually deserve, right? And that we should be talking about more when we do talk about these AI tools, these large language models, even beyond that, the environmental costs of this. It can be easy to ignore how these models are based on a lot of compute power, right? Having these massive data centers that require a ton of energy to train these models, to keep them going. You know, there are rumors now that chat GPT, even though it's free for you to go and use it online, that it's costing millions of dollars a day just to kind of keep the servers running, essentially, so that you can do that. But that doesn't get talked about as much.

And then, of course, how it's using a ton of data that's scraped from the internet. We know the severe problems that exist in the data that is actually coming off of the internet. As you were talking about, we know how these data sets can be very biased. This is not like a controversial or new thing that we're just hearing about. This is something that's been around for a long time and has not been effectively addressed. Of course, the opportunity costs in the research there, how there's all this hype around this particular type of AI, this particular type of large language model. So all of the energy, all of the research, all of the money is going to go into building on these

types of things. But what if there's a different type of an AI tool, a different type of a language model that is actually better suited toward the public, right? Toward having public benefits that might come of this, toward the types of things that we would be more concerned with as a people instead of just what's going to excite, you know, these people at the top of the tech industry make money for these large corporations. Those things might not be aligned, but when the hype goes in one direction, it gets distracted from anywhere else. So these are all really important points that come of this paper and the work that you were doing with your co-authors and the other people you were working with. Honestly, even though we were worried about the risks then,

I still did not expect it to explode in the way that it has right now, just in such a short period of time, right? All of the things that you outlined are about, to me, obfuscating what exactly is going on, right? The way we talk about things in the tech world. Oh, you're running it in the clouds. It's not a cloud. It's like there are data centers taking resources like water, people working in these data centers who are being exploited, you know, it's not cloud. So in AI, I just wrote an article recently about this, the exploited labor behind what so-called AI systems.

When people in the hype, when people talk about it, it's like, oh, my God, look at this machine. It's so cool what it can do. Well, look at the armies of exploited laborers that first whose data has been taken. Everybody's data has been taken. I think with the art, it's become much more clear because artists are galvanizing. But all of these systems have data sets that they scrape from the internet and data laborers that do some type of tasks. Some of them supply data sets or data. Some of them label them in crowdsource and micro tasks and all this stuff. And so that's what this whole field is predicated on. And that's when it started to become more popular with the advent of things like Amazon

Mechanical Turk, where you could like really crowdsource things and pay people pennies per little task and things like that. Now there are lots and lots of companies who are raising billions just based on that outsourcing kind of task, similar to content moderation and social media and things like that, right? It's again, the hiding of all of the different things that are going on. And so there's the theft, there's the data, the hidden cost, the hidden data labor, the hiding of all of the different things that are going on. And so there's the theft, there's the data, the hidden cost, the hidden data labor, the, you know, hidden costs of, like you said, you know, environmental costs. So that's actually the one thing I feel like we didn't really cover in our paper is the level of theft. So we talked about data curation, right?

And how you can't really just scrape everything on the internet and assume that something good is going to come out. You have to curate your data sets and document them and really understand them. And if your answer is, oh, it's too big to do that, then that means you just, you shouldn't do it. What other, you know, where else can you do that? Like, can you sell me a food item at a restaurant and be like, eat this food? I don't know what it's made of. Like, you know, there's some sugar. I know that there's some flour, whatever, whatever else is, you know, you're on your

own. Can't do that. Not allowed, right? We haven't understand. But in this industry is the most unregulated industry, right? And, you know, they can proliferate things because of that, right? And the other thing is, you know, look at proliferate things because of that, right? And the other thing is, you know, look at all of the different, I just learned today, you know, I was talking to some

people who know the AI Act in the European Union. There is a proposed legislation, the AI Act that, you know, there is still debating. And so people are trying to have, are lobbying for exceptions for what they call general purpose models. So chat GPT type thing, large language models would be general purpose models, right? Now think about it. OpenAI has been going around billing this thing as an all-knowing godlike thing, right? They're not saying, hey, you can, they haven't gone out of their way to tell you, hey, you can only use this thing in these limited ways, whatever. That's not how they're talking, right? When you see the press, when you see Sam Altman's tweets and all these people, they're like,

oh, this is AGI. However, they would be exempt from any sort of harm if they say, hey, they just have a one-liner saying, don't use this in any high-risk things. But they can go around saying, like, this is a god. But then you have these people who are talking about how they want to use these chatbots in Supreme Court. I don't know if you've seen this. You have these people who are using them in mental health applications, right? Unethically, very clearly. So now you have open AI who can make tons of money because it's not going to be free.

Obviously, it's not going to be free. They're going to start making tons of money from these kinds of people, but they won't be liable, right? But they created the hype. They created the race. They created what's in the public consciousness, you know, hiding all of the costs and hiding what's really happening, telling us they've created some sort of godlike thing. And now the people who are going to try to use it as such are going to be, they're the ones who are going to be liable, right? Even if we have any such are going to be there. There's the ones who are going to be liable. Right. Even if we have any sort of regulation, that's assuming that's the best case scenario. So, you know, I just even with the things we first saw in 2020, I just I still did not foresee it to be this prevalent just in the two years. And it's very worrying to see how much it has exploded.

Right. in the two years. And it's very worrying to see how much it has exploded, right? And how seemingly unprepared, you know, the discourse has been to reckon with what it actually might mean for us, right? You know, as I've been watching, say, chat GPT rollout or GPT-3 or, you know, the kind of art, quote unquote, art AI tools that have gained popularity over the past year or so, I guess, I've been a bit unsure how to think about it, right? Because on one hand, you have this kind of public narrative that is being promoted by these companies, you know, like OpenAI, where these tools are going to absolutely upend everything, right? All artists are going to lose their jobs, all writers are going to lose their jobs because these tools can make writing or images that look like a human has

done it. But that feels to me like something that we've heard about a lot of tech products in the past that have really been unable to live up to these really lofty kind of hype inflated claims that we've heard before. And then on the other hand, I wonder, you know, what might the real impact of this be when we get past the hype fueled hysteria that we're in now? And I'm unsure what to make of that. And I wonder what you think of it. That's a really interesting point because I'm like stuck in a cycle. So I, you know, I'm stuck in a cycle where I'm constantly being like, no, no, don't, no.

Yeah, where I have a hard time thinking past that, right? And I'm actually trying to force myself and my institute to carve out time to also work on our vision for that, like our affirmative vision for a tech future that is not dependent on what they're doing. But if we were to think about, you know, the actual impact of this, I do think centralization of power is a big, so let me give you one example, one example of an actual impact. OpenAI has some, you know, a speech to text model, transcription model, and a data set called like Whisper, right? And then Facebook Meta came up with a model called Galactica saying that, you know, it can write all scientific papers, whatever. But then they came up with another paper, which they called No Language Left Behind, saying that, oh, you know, we have this data set that

contains so many different languages, it's so accurate, whatever, and this and that. And there is one of our fellows. His name is Asma Lashteka, who created a startup called Lassan. It's a machine translation startup specifically for some Ethiopian languages, Ethiopian and Eritrean languages, a few of them. I mean, there's so many, first of all, Ethiopian and Eritrean languages, but just even a few of them. I mean, there's so many, first of all, Ethiopian and Eritrean language, but just even a few of them. So somebody who was thinking about investing in them, I believe, was like, hey, have you seen this paper from Facebook? Your startup is going to die. Basically, it's solved.

It's a solved problem. They say they solved it. Let's say it was true. The people who would make money are not people like him, right? The people who would make money are not people like him right the people who would make money are not people who actually speakers of that language and the people who get the money the workers of the startups and stuff are not those right it'll just be like this one company located in one location but it's not true right it's not true because he was looking at the the data set and it's so embarrassing. Some of the languages that are spoken by literally more than 100 million people, their data set was complete gibberish.

So I think really, whatever the path is, centralization of power is where we're headed. Yeah. The example that you give there seems really similar to what we're seeing. If you say that Facebook is saying that it had this model that was going to work for all the languages, but then if you look into look great. Like, you know, it looks like the AI is creating this incredible image or is writing this thing that makes a ton of sense. But as soon as you start to kind of tinker with it or try some different things that might be a bit less conventional, you know, you can see that hands don't generate properly in the images. And there are other kind of issues which show that the AI, so to speak, doesn't actually know what it's doing. It's just trying to combine these different images and styles in a way that you are looking for and the way that it's been taught happy to spit out complete lies and things like that, as long as you frame your question appropriately.

And that it's really unclear that it will truly be able to replace human writing, as we've been told. And one of the things that really occurred to me, as I've been seeing this over the past number of months, is that we had a whole similar narrative right around 2015 and 16, as you say, like in that moment, there was a narrative that, oh my God, all drivers were going to lose their jobs because self-driving cars were right around the corner. And we can see how that worked out. But at the same time, there were also narratives about how writers were going to lose their jobs because all these things could be automated. And it didn't happen in that moment. And I'm sure that these promises have been made before in the past as well. And so I'm always very skeptical when this comes up, like as we're seeing right now, as we're in the midst of this kind of hype-fueled hysteria, like what is actually

going to come out on the other side of this? And I'm very unconvinced that it's going to be what, you know, Sam Altman wants us to believe. Yeah, he said that the future is going to be unimaginably great because of their chatbots, I guess. I don't know how you create a text-to-image generator and you think you're going to create utopia, but this is the language of the long-termists and effective altruists and stuff, which everybody, some people, I am vindicated because some people used to think I was just obsessed with this niche group of people for some reason. But, you know, I've been around them for a long time and I just sort of used to roll my eyes, you know. But now it's just very much, you know, something that you can't ignore, right?

You know what it really reminds me of? I realized that when you were talking about stable diffusion, it reminds me of social media and the issues we're talking about in terms of social media, right? Because they're having issues of moderation too. So then you'll have issues of the moderators and what they're going to have to see and what they're going to be exposed to and have to moderate and the exploitation of these moderators because their entire business model is predicated on theft and not compensating people for stuff, right? And so if they had to, if they were forced to do these things, they just would decide not to do it because it's not worth it. It's not going to

make them money. So I think at the other end of it, that's what I'm seeing. It's kind of the same issues we have with social media companies, the spreading of disinformation, trying to centralize power, moderation and exploiting people like a labor force that's sort of second class citizens around the world. So I think that's kind of the path we're on. And while promising utopia, you know, which just like, I don't understand how you go from A to B. For me, we can never have a utopia that is, you know, based on the vision of some homogeneous group that, you know, has absolutely no ground in history or they don't understand art even. You know what I'm saying? Like, I'm, you know,

I, you know, used to play piano for a long time and I, you know, I'm saying? Like, I'm, you know, I, you know, used to play piano for a long time. And I, you know, I'm a dancer. And I don't, you know, I'm not a visual artist. My goal is not to watch a robot dance, even if they end up doing technically things that are, you know, whatever technique is just one aspect of it's like having words it's a communication mechanism between humans right it's a human expression mechanism people use art for social movements like people use i just i just you know what i'm saying so these are the people who want to like give us techno utopia and i don't know what it even means like a world where you just sit at your computer all the time. You never interact with other human beings. Everything you do is like with a chatbot and with a robot and with, I mean, like, why do

I want to live in that world? You know, honestly, so even their vision, which won't happen is dystopian. You know, I just don't see how this is utopia. Yeah, it seems fundamentally inhuman, right? Like, I'm going to connect a few threads here based on what you're saying. But, you know, when we think about the work aspect of this and what it might mean for work, one of the things that comes to my mind and that, you know, I reflect on a lot as I think about tech's impact on work and on the work that we do on the jobs that people can expect is that, you know, going

back to the gig economy and even before, you know, we often get promises that tech is going to automate things, right? Or is going to take away the need for human labor in various instances. And what we actually see is that it doesn't take away the need for the labor. It just kind of degrades the job, the expectations that the workers can expect when it comes to performing those sorts of tasks, right? So whether it's the gay economy or the people who are working, you know, to label AI tools and images and things like that, that you're talking about these people who are really poorly paid, who are treated very poorly. You know, we can see the rollout of technologies in like Amazon factories and things like that, how that's used against workers.

They haven't automated the whole factory, but they have instituted algorithmic management of these people. And then, you know, as you're saying, when we think about the future that is being positioned that these sort of AI tools like chat GPT or stable diffusion are supposed to create for us, it seems to be taking away like one of the most what I think many people would imagine would be one of the most human things is our ability to create art, right? To create culture and to try to replace that with tools, with computers. And we would just consume what the computers create rather than what humans themselves make. And if there's ever going to

be an argument for something that probably shouldn't be automated, it's those particular things, right? And then the final piece of this, because you mentioned effective altruism, and of course, its connection to long-termism, is that there seems to be a fundamentally kind of inhuman future being presented there too, right? Where we need to be concerned about the far future and humanity's future and all these sorts of things, but not like the issues that are really affecting people today, because the people who are pushing it, who are leading it, whether it's thinking up the AI models and the futures that they predict for us or promoting long-termism, they are incredibly disconnected from the actual

problems that people face in day-to-day life. And the kind of futures and the kind of solutions that they are imagining just do not deal with that or reflect that at all. Yeah, exactly. Absolutely. And it's kind of, you know, iterations of the same story each decade or each century or whatever it is, you know, even when you look at the rebranding of terms, you know, like big data is basically AI now. Like I think everybody says AI, they don't say big data anymore, right? Like it's just kind of, you know,

every so often you have a rebranding of things that sound cool. And I am someone who got my PhD in the field because I thought, you know, interesting things about it. I would like to focus most of my energy on just imagining like, what are, you know, useful tools, interesting tools you can build. But, you know, we end up, and that's another thing I resent these people for, right? Nobody said, go build us a god or try to build us a god. They decided that this is a thing that not only they should do, but it's like a priority for everyone to do. And so then they do that.

And then the rest of us have to like be stuck cleaning up rather than sort of trying to implement our vision of what useful things we can build without our skills and our little amounts of funding we have. We don't have billions of dollars that they do, right? So then we end up in this constant cleaning up mode, which means that we can never catch up. That's really the worry that I have, right? When I talk to legislators or like, you know, when I talk, for instance, we're still talking about, you know, regulating face recognition, right? We're still talking about the fact that we don't have any regulation for social media platforms, but the EU has some, California just came out with their privacy law. They have moved on to like the

metaverse and synthetic, you know, like stable diffusion type stuff, right? They don't even care about this stuff anymore, right? So I often talk about it as regulating like horse carriages when we're talking when actually it's like cars that they've created, you know, they've already paid off all the legislators like ruined public transportation, created roads and cities and all the infrastructure for cars. And we're still sitting and talking about, well, the horse carriage that you've created is not safe. And how can we regulate it?

Guess what? They've moved on, right? They moved on. And we're already going to be too late when we are. So I think it's really important for us to do both the uncovering, critiquing of the harms, but also investing in the people who can do something different, like alternative visions, because otherwise we're just going to end up trying to clean up and never catch up. Absolutely. It's essential, right? If we ever want to think of a different type of future or a different type of technology than what's on offer by OpenAI or by Elon Musk or by whoever else. I'm wondering, you know, as we're talking about this, of course, you are not at Google any longer. You have your own institute that you have founded and that, you know, some people who you worked with at Google have come to join you there. And

I'm sure some people who weren't at Google as well. You know, what is your focus in trying to do research on artificial intelligence now at this research institute? Like, what's the goal that you're trying to achieve without having to worry about what Google and the higher ups there are going to do? You know, what's the kind of lens that you're approaching this research through? So I think the first lens is interdisciplinarity, not just, you know,

in the research world, like sociologists and computer scientists and engineers, etc., but also like labor organizers, refugee advocates, activists, like people who haven't generally been involved in the shaping of this research, right? So I think that if you're talking about quote-unquote benefiting humanity, which, you know, that's not what I'm talking about. You obviously should start with the people who are already doing work to do that and incorporate their perspective. So our research questions and methodology are very much informed by that. We co-create, you know, together. Like, for example, Adrienne is, you know, working at our institute, she used to be a delivery driver at Amazon. And she just gave a talk at our one year anniversary about like wage theft. And so she was calculating, she was showing like her estimate of wage theft that was by Amazon and they, you know, for each worker, right? And I would have never thought to do that if we didn't have her, right? Or we have Mehron, who's a refugee advocate and also very much a victim of transnational repression. And so

we're doing this kind of project on analyzing the harms of social media platforms. And we have to work on natural language processing tools to do that and all that. And so it's really important to have her perspective. But it's really interesting, too, because it challenges your points of view, because Mehron, for instance, is very much into Bitcoin. And the rest of us are like, very, very skeptical, right? Like, I'm like, you're in the room full of skeptics. But I have to listen. This is someone who single handedly rescued 16,000 refugees from human trafficking, right? And has fought like, you know, against governments and all sorts of people. So I have to listen to her when she's telling me the ways in which she's

been using it to help refugees, the ways in which refugees have been using it, you know, so I have to then listen to her, right? So it's interesting. I think like, even to reimagine a different sort of future, it's very different to have these kinds of perspectives, rather than just like this techno utopic, whatever, you know, from some tech bro in Silicon Valley. So that really is the lens we're taking. And I think that what's really hard for us is basically what I just said to you is that because there's so much so many dumpster fires, we have to fight all the time, it's very difficult to just carve out space to be like, what is our positive vision, imagination for the future? And so we're trying to force ourselves to do that.

Like we have, we're starting this possible future series, which is like a few pieces to show like tech, it's not this deterministic march towards some predestined future, right? So the first piece is kind of talking about that. Second piece is about, like, what would an internet for grandmothers would have looked like? My grandmother did not read or write, did not speak English, was paralyzed for a number of years. So when we're talking about technology, you know, you never think of her as an early adopter or customer or whatever, right?

Yeah. So, you know, I think then speculating and then like then also bringing it back down to earth, it's like, okay, what are the components of such, you know? So I think we're just trying to create space for ourselves to think about these things and executing on them to the extent that we can given our expertise. Yeah, that's fascinating. And it reminds me of a conversation I had about, you know, people in the favelas in Brazil and how they were kind of utilizing technologies to serve their needs and rethinking about how they could be used in ways that

companies that were developing them probably never thought about, right? Yeah, it's so fascinating to think about those futures and those other potential uses and how other people might think about or approach technologies. I love that. And, you know, I'm sure I'll be paying attention to the work that you'll be doing at Dare, you and the team, of course. I'm wondering, as we close off our conversation here, where do you think this is all going in the near future? Certainly, there's been some critical coverage of chat GPT, stable diffusion, these types of technologies. But there's also been a lot of coverage. And I would say the majority of the coverage is really kind of uncritical, buying into the hype, you know, framing these things as though they are creating

text that you would never be able to tell the difference between a computer and a human and all these sorts of things. And it makes me worried to see that. And you talked about the company that was using ChatGPT to turn out responses for a mental health service that they were running. And it immediately made me think of Joseph Weisenbaum's Eliza chatbot that he made all the way back in the 1960s and the worries that he had around people thinking that this computer system, that this chatbot could recognize what they were saying actually had some degree of intelligence when it didn't at all, right? So, you know, that's a long way of asking, where do you see this going in the next few years? And what should we be paying attention to to try to have a more critical take and a more critical view on these tools as they get all this hype and attention. You know, similar to you, I don't know if it's just my imagination, but I have started to see a lot more critical approach to tech in general, much more so than like, let's say 10

years ago. I don't know if Steve Jobs died today, if he would have had the type of, I'm not exactly sure actually, the kind of reverence that he did have had the type of, I'm not exactly sure, actually, the kind of reverence that he did 10 years ago, right? Which was like a god. And when I was at Apple, I felt like I had joined a cult, actually. Like, whoa, my God, you know, the way they were talking about him. Did you wear a black turtleneck? Absolutely not. But it's just, I would say, but on the other hand, it's really depressing because it's the same story over and over again. Like if you look at Theranos and how the media just hyped it up, built it up all over and then like, oh my God, how did it happen? You know, or Elon Musk's like time person of the year, whatever. It's not like we didn't know. The rest of us being like, oh, come on, you know? And then the whole Sam Bankman-free thing, it's just kind of the same cycle over and over again. And with this hype, I mean, I am shocked that ChatGP,

I mean, the level of attention and hype it's getting is truly shocking. You do have to give it to them on the hype machine. They are so good at it. And so whatever kind of critique we have, it is not even comparable to the kind of hype that it is generating. And so to me, that's where I see it. I see the same cycle. If at some point a Theranos type situation happens, you know, it's not the media that's going to help us, right? Like, so it's legacy media. I think it's really not helping us because what they're doing is they're giving megaphones to the super powerful and making it even more difficult to raise awareness about

some of these issues and talk about reality. And so that's kind of what I'm seeing. That's where to me, it's going, you know, you have an explosion of startups. It's not even the big, just the big tech companies right now. Actually, Google is the one who didn't release a chatbot and they have to tell their researchers why not. I mean, they could have done that before they fired us. But like, you know, now and they have an explosion of startups, because this is the next thing that all the VCs in Silicon Valley are talking about and like pouring their money into. And so I just I think we're super behind in kind of really raising awareness about the potential harms.

And I just see like, I honestly see the hype continuing. Brando's talking about having chat, you know, I mean, this guy was saying that having a chat bot as a lawyer would level the playing field because poor people would have access you know to and it's really interesting because this goes into kathy o'neill's point in weapons of mass destruction where poor people are often much more interacting with automated systems than rich people. So now the fact that this guy is saying that automated systems are going to level the playing field for them, it's just obviously the idea did not come from someone who didn't have a good lawyer and therefore obviously it didn't come from a poor person who didn't have access to a lawyer.

So I don't see it slowing down anytime soon. Yeah. No, I share your concerns, right? Especially when we're in this moment when the tech stocks are down, when interest rates are higher, and the industry is clearly looking for the next big thing after Web3 and the blockchain stuff and the metaverse clearly haven't worked out. But what does give me hope is seeing how, you know, at least I feel that the criticism that was made of Web3, of cryptocurrency, of the metaverse, I feel like made a difference in trying to make sure that there were more critical perspectives of these technologies in the media, in, you know, the discourse that people were engaging with when it came to these technologies in the media, in the discourse that people were engaging with when it came to these technologies.

And so my hope is that if we're able to kind of hone a critique of these technologies early on, that we might be able to influence some of this coverage and hopefully not have the repetition of some of these kind of 2010s hype cycles around some of these technologies that we saw in the past. So that's what I'm crossing my fingers for. I hope so too. Yeah. I think it does make a difference.

When I talk to Joy and I'm like, look at face recognition. I thought we made progress and it's everywhere now. It's like exploding. And she's like, what I think about is what would have happened if we didn't do what we did. And so I do. You're right. I do think it's making a difference.

I'd like to see more of it, you know, I'd like to see more. Totally. Yeah. I completely agree. And I think that's a great place to end off our conversation. Timnit, thank you so much for taking the time. It's been fantastic to chat with you. Thank you so much for having me. Timnit Gebru is the founder and executive director of the Distributed AI Research Institute. You can follow her on Twitter at Timnit Gebru. You can follow me at Paris Marks, and you can follow the show at Tech Won't Save Us. Tech Won't Save Us is produced by Eric Wickham and is part of the Harbinger Media Network.

And if you want to support the work that goes into making it every week, you can go to patreon.com slash tech won't save us and become a supporter. Thanks for listening. Thank you.