There Are No Girls on the Internet - AI is built by people. We need to listen to their stories

Episode Date: October 17, 2023

On the just launched brand new season of Mozilla’s podcast IRL: Online Life is Real Life, Bridget explores the power of putting people over profit in AI. In this first episode, we look at the risks and rewards of open sourcing the tech that makes ChatGPT talk.  If you liked this episode, let us know! And be sure to subscribe to IRL: https://irlpodcast.org/See omnystudio.com/listener for privacy information.

This is an I-Heart podcast. Guaranteed Human. Another podcast from some SNL, late-night comedy guy, not quite. Unhumor me with Robert Smygel and friends. Me and hilarious guests from Bob Odenkirk to David Letterman help make you funnier. This week, my guest, SNL's Mikey Day and head writer, Streeter Seidel, help an a cappella band with their between songs banter.

Where does your group perform? We do some retirement homes. Those people are starving for banter. Listen to humor me with Robert Smigel and friends on the IHeart radio app, Apple Podcasts, or wherever you get your podcasts. Run a business and not thinking about podcasting. Think again. More Americans listen to podcasts than adds supported streaming music from Spotify and Pandora. And as the number one podcaster, IHeart's twice as large as the next two combined. Learn how podcasting can help your

business. Call 844-844-I-Hart. Your husband is not who you think he is. Your body is not what you thought it was. Your identity is formed by a secret history. I'm Danny Shapiro. And these are just a few of the stunning stories I'll be exploring on the 14th season of Family Secrets. He kind of shoved me out of the way and said, move. And he went out the front door and he jumped in a car and drove off. And that was the last time I saw him. Listen to Season 14 of Family Secrets on the IHeart Radio app, Apple Podcasts, or wherever you get your podcasts. There Are No Girls on the Internet is a production of IHeart Radio and UnBossed Creative.

I'm Bridget Todd. And this is There Are No Girls on the Internet. I'm hosting a new season of Mozilla's podcast, IRL. Online life is real life. You might actually know Mozilla. They make the web browser Firefox. This season of IRL is all about AI, specifically the people who make AI. And how important it is to put people above profit when it comes to AI.

Now, it's really easy to think of AI as just computer brains and robots, but it's built and trained by people. And as much as we talk about making sure AI is ethical and equitable after it's been built and is out in the world, we should also remember the people who build it from the very beginning too. It's something really important that I think gets overlooked in conversations about AI, turning the people responsible for making it into a kind of invisible human workforce. But they shouldn't be invisible. We should listen to them when they speak up about this technology and how it's going to shape all of our lives.

So I wanted to share the very first episode of this new season of IRL with you all here. This one is all about the risks and reward of AI technology like ChatGPT being open source. That is, built in a way that allows anyone to inspect, modify, and enhance its code on their own. So let me know what you think. And if you enjoy it, please subscribe to IRL. Online Life is Real Life. So the first thing I ever asked ChatGPT, wasn't work-related at all. It was actually for help drafting kind of a tough personal email I had to send.

I was having trouble finding the right words, the right tone. So I asked ChatchipT, and I was amazed it actually produced something that I might say. That was about a year ago. Fast forward to today, an OpenAI is said to be on track to earn $1 billion of revenue in the next year. Even though large language models aren't new, suddenly, more people can see the potential. through that simple interface, for good, for bad, and for making money. This is IRL, an original podcast for Mozilla, the nonprofit behind Firefox. This season, we meet people who are building artificial intelligence that puts people over profit. I'm Bridget Todd.

In this episode, we get into the risks and rewards of the tech that makes chat GPT talk. We're talking about large language models, LLMs for short, and the controversy over suddenly giving the whole world access to build with them. But chatbots are only one example of what powerful LLMs can do. Imagine, video games where characters can chat with you more, or virtual assistants that can draft emails for you at work. Banks, insurance companies, travel agencies, everyone is thinking about how to use this technology to increase productivity and more.

But there's also a lot of talk about the risks. I think a lot of people don't understand the detailed capabilities of large language models. You could use them to really tear apart the civic fabric of a country. That's David Evan Harris. Over five years, he managed teams that kept harmful content off Facebook, and later, also researched responsible AI for meta. Today, he's worried that LLMs can be used to generate disinformation and hate speech on a greater scale than ever. Like other big tech companies,

META develops its own LOMs, and now they're urging people to use them and tweak them with few strings attached. Meta's LLMs are called LAMA. They might have a cute name, but David says there's a potentially ugly side to META's open LLM. I have a long history with open source

and a big passion for it, but thinking about large language models and Lama and whether or not these things are safe to be open source has been a real turning point for me. I remember more than a decade ago having some conversations with a friend at MIT about the possibility of open source licenses that don't allow for military use. We love making open source software, but what if our open source software is being used to make bombs and kill people? We don't want to do that. Now, that connects to this question of what's the threshold?

for something that we're not comfortable having open source. I just think the bigger danger that I keep coming back to, and maybe not bigger, but the very important danger is misinformation and is the idea that a system like Lama 2 could be really effectively abused in a large influence operation campaign by what we call in the industry a sophisticated threat actor. And that basically means like an intelligence agency that probably has great. hardware and big budgets and well-trained engineers. David's argument, echoed by many in the industry, is that we don't really know how LLMs of today or tomorrow could be harmful in the long term. But he's also focused on the

harms of the here and now and how these disproportionately affect people who are already at risk of exclusion and discrimination. So here's how I think about LLMs. Put on your chef's hat for a moment and imagine your baking making a delicious cake, a layer cake. The foundation, or bottom layer of that cake, is a large language model. It's made out of lots of internet data. Now, some of these ingredients aren't the best quality, but with additional layers, coloring, icing, and sprinkles, you can fine-tune your system.

To make a chatbot, you fine-tune an LLM with data of people chatting. To make a safer chatbot, you train it with data that shows what prompts should trigger safety replies. Whenever you're building software with LLMs like LAMA, GPD4, or Falcon, that's just part of what goes into the cake. So there are a lot of options that go into creating an AI system, even when the so-called foundational models are the same. When you're using AI in a hiring system or in a applicant tracking system that's sorting through thousands and thousands of resumes, you don't need an LLM for that. But you could use LLMs for that kind of thing. you could use LLMs to give you analysis of different candidates.

And there may be situations where LLMs demonstrate bias. I say this because, you know, banks are using LLMs too. If a bank is using an LLM as part of their processes to evaluate loans and nobody has noticed yet because that LLM has never been systematically tested for bias, maybe that's introducing bias into that bank system. So I think there's some danger there. And a lot of people think, oh, danger, that's not danger. And, you know, if you're getting denied a mortgage because of your race, that's danger to me.

David feels the industry as a whole is rushing development. At the same time, responsible AI teams have been downsized at several companies. David himself was laid off from META's responsible AI team in 2022. As a company that's using AI or even, even as a government that's using AI or a nonprofit organization that's using AI, you need to create robust processes to figure out how and when it's appropriate to use AI systems. And you need to have people who are not interested parties. And in the case of a company, an interested party might be just the engineer who wants to ship

the damn thing and get the feature running with the AI. And you need to have someone who does not have an incentive to ship products in the there who can say, hold on. We might need another month of testing of this. Hold on. We might need to find a way to get someone out from outside the company to really give us an opinion about if this is a fair AI system or if this is safe. The reason so many LLMs are at our fingertips now is that investors with deep pockets, Google, Microsoft, Meta, Elon Musk, and others have been pouring money into AI research and powerful supercomputers. Some companies will bake LLMs into their own products.

Others will make money by licensing access to them. Everyone is competing for influence and for engineering talent that can help them go faster. Openness can be a strategic move to get ahead by attracting more developers. But often, companies also exaggerate how open they are, since it's not always possible to see their data or methods. So I've followed these models very closely,

and I know every time they're released, I know there is some element of deception. That's Abeba Burhani. Time magazine just named her one of the 100 most influential people in AI. She's a Mozilla advisor and a cognitive scientist from Ethiopia, working at Trinity College in Dublin, Ireland. I mean, Lama, for example, was introduced as, oh, an open-source large language model.

And I went into the paper hoping to find information, detailed information. Because I work with data sets, I went immediately into the data set section, and it was just one tiny small paragraph in that giant paper. Abeba wants to know what's inside the datasets for AI, because systems trained on them mimic their biases. Just a handful of datasets get used repeatedly across most LLMs.

And these usually include massive amounts of internet content from an open dataset called Common Crawl. The internet can be a really toxic place. It holds everything from the world's beauty to its ugliness and everything in between. For example, during our audits, we found content such as child abuse or genocide or a lot of explicit pornographic images. You also have to make sure that personal sensitive information that could be used to identify individuals, You have to make sure things like this are not included in data sets. That's one of the reasons why we need to audit the data sites we are using to train models.

Another podcast from some SNL late-night comedy guide, not quite. Unhumor me with Robert Smygel and friends. Me and hilarious guests from Jim Gaffigan to Bob Odenkirk to David Letterman, help make you funnier. This week, my guest, SNL's Mikey Day and head writer Streeter Seidel, help an a cappella band with their between songs banter. There's that worst singer in the group? The worst?

Yeah. Me. Is there anything to the idea that because you're from Harvard, you only got in because your parents made a huge donation. The group. The yard birds, right? That's the name. The Harvard yard, but they're open to change.

Do you have a name suggestion? We're open. Since you guys are middle aged. One erection. Listen to here. Humor Me with Robert Smigel and Friends on the IHeart Radio app, Apple Podcasts, or wherever you get your podcast. Humor me. I need some jokes to make me seem funny.

Run a business and not thinking about podcasting, think again. More Americans listen to podcasts than ads supported streaming music from Spotify and Pandora. And as the number one podcaster, IHeart's twice as large as the next two combined. So whatever your customers listen to, they'll hear your message. Plus, only IHeart can extend your message to audiences across broadcast radio. Think podcasting can help your business. Think IHeart. Streaming, radio, and podcasting.

Call 844-844-I-Hart to get started. That's 844-8-4-I-Hart. If you're watching the latest season of the Real Housewives of Atlanta, you already know there's a lot to break down. Norsha accusing Kelly of sleeping with a merry man. They holding Kay Michelle back from fighting Drew. Pinky has financial issues. I like the bougie style of Housewives show.

I think it looks like it's going to be interesting. On the podcast, Reality with the King, I, Carlos King, recap the biggest moments from your favorite reality shows, including the Real Housewives franchise, the drama, the alliances, and the team everybody's talking about. As an executive producer in reality television, I'm not just watching it. I understand the game.

As somebody who creates shows, I'll even say this. At the end of the day, when people are at, at home, they want entertainment. To hear this and more, listen to Reality with the King on the IHard Radio app, Apple Podcast, or wherever you get your podcast. Decades of research, so the internet has never

been representative of all the world's people or languages. But in generative AI, it becomes the ground truth. Abeba and her colleagues have coined a term to highlight the problem they see. Abeba, I noticed in one of your papers that y'all actually used the term data swamps,

not datasets, where did that term come from? Like, why data swamps? Data swamp is an attempt to kind of express how such a huge dump like the common crawl or even large-scale data sets now, how they represent not only the good and the healthy of humanity, but also the nasty and ugly of humanity because you find all kinds of horrible, hateful, degrading texts, especially towards minoritized communities, and you find all kinds of images

that is really disturbing to the human eye. Even when these enormous data sets are open, it can be too difficult and costly for independent researchers to audit because they're too big. But even using smaller samples of datasets, Abeba and her colleagues have uncovered a ton of problems. In the past, their audits of a leading image dataset for AI

documented so much racism and sexism that it was decommissioned after decades of use. So, Abeba, is it personal for you, the motivation to keep going? Yeah, it is a bit personal. When I go into datasets, for example, the first thing I query is around, you know, how black women are represented,

how Africa as a continent is represented and so on. So when I see all the negative images or extreme negative stereotypical caricatures or, you know, completely inaccurate false misleading information, you feel like if you don't say anything, if you don't do anything about it, nobody else is going to. Abeba says we need regulation to make companies more transparent about the data they use and where it came from.

She says if companies can hide this information, they can include data they don't actually have permission to use. These artifacts are not something that just remain in the labs of big corporations. These are tools that infiltrates into every social spheres. What information goes into them, what kind of data set is used to train them, where the data set is sourced and the quality of the data set itself and how the models were built. And any other important information should be open for auditing and for scrutiny, given that they are almost treated as social good

that are supposed to serve everybody. So some level of openness is really important. In terms of making them entirely open, some people have raised the issue of if they can be accessed by everybody, bad actors can download them and use them for problematic applications. There is always a balance that we have to keep working around. We have to always try and find that is between

opening closed. It's because LLMs and their data sets can be problematic that we need independent scrutiny of them. Could regulation empower people to work together to improve these systems? Currently, there's been a lot of kind of like polarizing discourse about open versus closed source, as if those were the only two choices, but they aren't the only two choices. It's kind of like more productive, more forward thinking to acknowledge the fact that it's a gradient, it's a spectrum. That's Sasha Lucione, a leading researcher at a startup called Hugging Face.

They run an online platform for testing and developing AI. It's so popular that they've been valued at $4.5 billion. Sasha and her colleagues have a fresh take on the open source debate. What point in the spectrum can I pick for this and this model? And I think it's important, especially for policymakers to understand that, that it's not an us versus them. It's not like a two-camp situation. It's really like, let's pick what,

works for each model, and also there's no one-size-fits-all solution, depending on the model, depending on the data, depending on the usage. Some point in that gradient is more or less fitting. The spectrum of openness Sasha talks about, it's not just for a model's code or the data sets. It can be for a lot more, like the documentation and the so-called weights that determine how it works. These are all decision points on openness, along with the usage terms. Sasha's research at Hugging Face depends on openness. That's because it's all about how to measure and lower the environmental impact of language models. She says, training the LLM GPT3 emitted as much carbon as 500 transatlantic flights.

And she says, open source technology helps with sustainability in other ways too. Definitely one of the reasons I joined Hugging Face was because I truly believe that by helping open source AI research, we can help the sustainability, the energy side of things, but also in terms of democratization, like giving more people access to models that they can both use out of the box or they can fine tune them in order to fit their context better. I think that's like a net positive for everyone. And for me, it's kind of like recycling or thrifting or, you know, buying something used and then, you know, patching it up or you're changing it a little bit to work with what you need it for. And I mean, I thrift like 95% of my clothes.

So that's definitely a philosophy I'm really on board with. And for me, open source is definitely much more sustainable in the long run because you're not constantly starting from scratch. And also, people can work together. And so you have less wasted effort. Sasha says a community initiative called Big Science is an example of this. About two years ago, Hucking Face backed a thousand people from 60 countries in a collaboration to develop an open LLM called Bloom.

was literally a thousand researchers and volunteers from all over the world who were like, hey, let's train a large language model together because we don't have the resources to do it, like each one of us separately. And it was great because we had people who were lawyers, we had people who were like specialists in archival studies to help get data from different places. Like, I mean, we had all sorts of people from all over the world and people who don't necessarily have like a supercomputer on premise who don't work in a big tech company that can give them access to some kind of computes

to train these models. Open communities like this one could be directly affected by policies that either limit or encourage important research for alternatives. During the Big Science Project, I joined Hunging Phase because I was like, yeah, this is the kind of work I want to do. I don't want to have to be secretive about what I'm doing. I want to do it in an open source way. And I want to help other people who don't necessarily have the means to train these kinds of models. I want to help them also benefit from this technology. The fact that we had all these people involved in Big Science made the whole project and the,

ensuing model much more representative of society, I feel. And that's important because when these models get used in downstream models or downstream tools or systems, then any kind of information that's implicitly encoded in the model will bubble up to the surface. So with all these gradients of openness, it's not only the biggest AI companies developing LLMs, and that can be a good thing. Another podcast from some SNL late-night comedy guide, not quite on humor me, with Robert Smygel and friends, me and hilarious guests from Jim Gaffigan to Bob Odenkirk, to David Letterman, help make you funnier.

This week, my guest, SNL's Mikey Day and head writer Streeter Seidel, help an Acapella band with their between songs banter. The worst singer in the group? The worst? Yeah. Me. Is there anything to the idea that because you're from Harvard, you only got in because your parents made a huge donation.

The group. side of the group. The yard birds, right? That's the name. The Harvard yard, but they're open. Do you have a name suggestion? We're open. Since you guys are middle aged, one erection.

Listen to humor me with Robert Smigel and Friends on the IHeart Radio app, Apple Podcasts, or wherever you get your podcast. Huber me. I need some jokes to make me seem funny. Run a business and not thinking about podcasting, think again. More Americans. listen to podcasts, then ads supported streaming music from Spotify and Pandora. And as the number one podcaster, IHearts twice as large as the next two combined. So whatever your customers listen to, they'll hear your message.

Plus, only IHeart can extend your message to audiences across broadcast radio. Think podcasting can help your business. Think IHeart. Streaming, radio, and podcasting. Let us show you at iHeartadvertising.com. That's iHeartadvertising.com. If you're watching the latest season of the Real Housewives of Atlanta, You already know, there's a lot to break down.

Georgia accusing Kelly of sleeping with a merry man. They holding Kay Michelle back from fighting Drew. Pinky has financial issues. I like the bougie style of Housewives show. I think it looks like it's going to be interesting. On the podcast, Reality with the King, I, Carlos King, recap the biggest moments from your favorite reality shows, including the Real Housewives franchise,

the drama, the alliances, and the team everybody's talking about. As an executive producer in reality television, I'm not just watching it. I understand the game. As somebody who creates shows, I'll even say this. At the end of the day, when people are at home, they want entertainment. To hear this and more, listen to Reality with the King on the IHard Radio app, Apple Podcasts, or wherever you get your podcast. There's an open source alternative to chat GPT called GPT for All. Amazingly, it works without an internet connection.

and the LLMs are compressed so much that you can download them to any regular personal computer. GPT for All was launched by a New York startup called NOMIC earlier this year as a privacy-preserving alternative to chat GPT. Tens of thousands of people flock to it. Here's NOMIC's co-founder, Andre Moliar. One of the biggest focuses that we have around GPD for All is making sure that privacy is the first thing we think about.

In some sense, one of the core reasons behind why we even built JPP for All and the ecosystem of models that came in with it was because of all these large sort of like issues and concerns about privacy with people using open AIs models. You may not know this, but when you type prompts into chat GPT, OpenAI can use whatever you type to further train their models. There have even been numerous privacy leaks because of it, both corporate and personal. The privacy angle that we focus on specifically is making sure that the application in its open source form,

you can see all of the code. So we start out from that. That makes it safe. We make sure that everything's audited by the community. And the next thing is we make sure we align by all laws and regulations across Europe and across the U.S. We don't gather user-specific data whenever they use, for instance, the models. And we make sure that the models can run without access to any internet.

So you can go in once you download the models to your computer. You can turn off your internet. If you're stuck in the jungle and you don't have access to internet, you can ask it for help. No Mix mission is to improve the explainability and accessibility of the technology. AI. Their main software product is a data exploration tool for massive data sets called Atlas. But Andre believes GPT for All is important for them to devote resources to as a company. When you run a business, there are certain things you get the opportunity to do that you wouldn't be able to do if you weren't running a business. One of those is you have access to

capital to be able to work on risky projects like GPT for All purely because you want to, not because, you know, there's some direct revenue driving source of it. Mainly, Andre says he's motivated by a wish-to-see AI developed by more than just a handful of companies. But he also raises a question of values. And who decides how LLMs behave? So biases aren't always bad. So an example of a bias could be the model always prefers to greet you with a salutation before giving you a response. That's a bias that might not be bad.

But obviously there's biases that could be bad, right? And one of these sort of important things with large language models is the fact that you can actually actually go in and customize this. So if you have your own examples of data that you would like your model to be able to output, you can actually change that by training the model. Andre offers the example of OpenAI training ChatGPT not to output hateful statements. Today, GPT for All gives access to models fine-tuned not to offend, as well as some that aren't. Andre says they've had some backlash from people criticizing them for giving more people access to LLMs that could be used for harm.

The reality is like this technology isn't going away. The biggest thing is we need to learn how to live with it and how to be able to cope with the side effects that emerge from it. A lot of them will be positive. Some of them are going to be negative. Like one of the things that I guess I think about quite a bit is like what happens in the 2024 election in the United States. You can go in and pick 10,000 people and get their Facebook profile and customize a chatbot that pretends to be a human to convince them to think one way or the other. And you can do that for like no cost at all. I guess the thing that keeps me awake at night is if we're going to live in this inevitable world

where we're surrounded by machines that can generate synthesized versions of information and all that information is being piped from one or two company servers. If there's a world where someone like OpenAI owns all the pipes for the information flow, and then they get the chance to manipulate that however they want. This is like why we do what we do. We want to make sure that these generative AI models that exist, and persist through the world, are built with everyone's view into how the models are being created, not just a couple of organizations behind closed doors with unlimited resources.

LMs are here. Open source communities that do put people ahead of profits are crucial to unlocking the positive potential of generative AI. The challenge for builders and regulators is to find that balance. On the one hand, so generative AI isn't developed or deployed in harmful ways, and on the other to empower independent researchers to contribute to how systems work. I'm Bridget Todd. Thanks for listening to IRL.

Online Life is Real Life, an original podcast from Mozilla, the non-profit behind Firefox. For more about our guests, check out our show notes or visit irlpodcast.org. This season, we're talking about people over profit in AI. Mozilla, reclaim the internet. Another podcast from some SNL, late-night comedy guide,

Not quite. Unhumor me with Robert Smygel and Friends. Me and hilarious guests from Bob Odenkirk to David Letterman help make you funnier. This week, my guest, S&L's Mikey Day and head writer, Streeter Seidel, help an a cappella band with their Between Songs banter. Where does your group perform? We do some retirement homes. Those people are starving for banter. Listen to humor me with Robert Smigel and Friends on the IHeart Radio app,

Apple Podcasts, or wherever you get your podcasts. Your husband is not who you think he is. Your body is not what you thought it was. Your identity is formed by a secret history. I'm Danny Shapiro. And these are just a few of the stunning stories I'll be exploring on the 14th season of Family Secrets. He kind of shoved me out of the way and said, move.

And he went out the front door and he jumped in a car and drove off. And that was the last time I saw him. Listen to Season 14 of Family Secrets on the IHeart Radio app, Apple Podcasts, or wherever you get your podcasts. Your 20s can be so exciting. but they can also be really overwhelming, confusing, and honestly, just kind of lonely. May is Mental Health Awareness Month, and the psychology of your 20s is breaking down the science behind the biggest roadblocks we face.

I was six years into my career, the 80-hour weeks, and just the first one in, the last one out, and I ended up burning out. There was a large chunk of my 20s that I, like, was just so wanting to, like, be out of that phase, out of my skin, and I just, like, really regret not living in the present more. You don't need to have everything figured out right now. You just need to understand yourself a little bit better. Listen to the psychology of your 20s on the IHeart radio app, Apple Podcasts, or wherever you get your podcasts.

This is an IHeart podcast. Guaranteed human.