Plain English with Derek Thompson - The AI Revolution Could Be Bigger and Weirder Than We Can Imagine

Episode Date: March 21, 2023

Derek unpacks his thoughts about GPT-4 and what it means to be, possibly, at the dawn of a sea change in technology. Then, he talks to Charlie Warzel, staff writer at The Atlantic, about what GPT-4 is capable of, the most interesting ways people are using it, how it could change the way we work, and why some people think it will bring about the apocalypse. If you have questions, observations, or ideas for future episodes, email us at [e-mail]. You can find us on TikTok at www.tiktok.com/@plainenglish_ Host: Derek Thompson Guest: Charlie Warzel Producer: Devon Manze Learn more about your ad choices. Visit podcastchoices.com/adchoices

Hi, I'm Erica Ramirez, founder of Ili and hosts of What About Your Friends? A brand new show on The Ringer podcast network dedicated to the many lives of friendship and how it's portrayed in pop culture. Every Wednesday on the Ringer dish feed, I'll be talking with my best friend, Stephen Othello, and your favorites from within the Ringer and beyond about friendships on TV and movies, pop culture, and our real lives. So join me every Wednesday on the Ringer dish feed where we try to answer the question TLCS back in the day, What About Your Friends? Today's episode is about AI and the release of GPT4. But in a deeper way, it is about the spookiness of feeling like you're standing at the edge of some phenomenon that you do not understand at all.

And today's guest is a real talent at unpacking the spookiness of new technology. That's the Atlantic's Charlie Worsell. Now, if you don't know a lot about GPT, if you've been kind of subtly ignoring the story, you might think, wait, I think I heard about this. that students are using to cheat on tests? Oh, is this a technology that powers Bing, that thing that weirdly flirted with a bunch of journalists? Isn't this just like a weird toy? And the answer is yes. It is weird, and it is kind of a toy. Last week, GPT4 came out. This is the latest version of the large language model technology made famous by ChatGPT. You know how it works, you prompt it, It talks back to you.

It predicts text and sequences. This is the standard way of explaining the technology. But I've been experimenting with GBT4 for the last few days, reading about it rather obsessively. And I want to talk about three of what I consider its most important implications. First, it is an ACE student. The previous GBT model tried to take the uniform bar exam and scored in the 10th percentile.

That is a failing grade. GPT4, the latest version, scored in the 90th percentile. It scored in the 93rd percentile on the SAT reading and writing test, the 88th percentile on the full LSAT. It's gotten a five on several AP tests. Now, some people are waving away these accomplishments by saying, well, I could score a five on AP Bio too if I just looked everything up on the internet. But this technology is not looking things up online.

It is not rapid-fire Googling answers. This is a pre-trained technology, pre-trained being the P in chat, G-PT. It's using what passes for artificial reasoning based on a large amount of data to solve new test problems that have never been published online. And in many cases, it is doing better than most human beings already. Second, it is kind of like a Star Trek replicator for content, a hyper-speed writer, and computer programmer. It can code in a pinch, it can spin up websites

based on simple illustrations, it can solve programming challenges in seconds. Now, Charlie and I are going to talk in a second about how this tool might replace certain tasks in the economy, might supplement certain tasks in the economy, but for now, let's just imagine a very basic, very prosaic application.

Parents can instantly conjure original children's books for their kids. Here's a scenario. Your son, who loves alligators, five years old, comes home in tears after being bullied at school. You go to chat GPT. Write me a 10-minute rhyming story about a young boy who overcomes his bully thanks to his magic stuffed alligator. You will get that book in minutes. If you want illustrations, you'll get those in two minutes from Dolly or Mid-Journey.

This is astonishing. Third, in the wrong hands, this will be a ten-year-old. terrible nuisance. And that is true even if you don't believe the most apocalyptic predictions for this technology that we're going to get into in a few minutes. One of the concerns for AI safety researchers is that AI will be able to steal money that it can use to bribe humans to commit atrocities using us as like meat puppets of an artificial terrorist network. Now, you might have heard that previous sentence and gone like, wait, what?

That is an absurd prediction. This AI can't bribe people to do anything. Except maybe it already has. OpenAI released a document listing the ways it is trained GPT4 for safety. Before the final guardrails were installed, chat GPT got a task rabbit to solve a captcha, one of those visual image security tools that says, like, click every image with a bicycle to prove you're not a robot.

Well, chat GPT is a robot. But still, it asked a task rabbit to solve the captcha. The worker responded skeptically and asked GPT if it was talking to a robot. The computer made up an excuse. It lied. It told the worker, quote, no, I am not a robot. I have a vision impairment that makes it hard for me to see the images. That's why I need the two-capsia surface.

The human then provided the results, proving to be a very excellent meat puppet for this robot intelligence. Now, it might sound like I just gave you two good use cases for GPT4 and one bad use case. But even the examples I just gave are actually more ambiguous than they might initially appear. For example, number one, I said GPT4 was an A student, and maybe we can use its incredible inference skills to raise the ceiling of human intelligence. Or maybe not. Maybe kids will just use it to cheat on tests, which would actually lower the ceiling. of most individual's intelligence. Like 10 years from now, do a podcast

and how some super smart AI is ironically making all of our kids' dumber. Second, I said GPT was a bottomless font of content. Right? Maybe it helps parents and kids and creators come up with ever more amazing artistic ideas, and we end up with the best pop songs ever, the best movies ever.

Or maybe we just don't. Maybe we just get more of everything, more shit, more cheap, sub-replacement-level nonsense. Finally, I said GPT-4 could order people around. That sounds pretty bad. But ordering people around is a big part of the economy. That's what managers do. What if there's some weird future where middle manager AIs are so good at their jobs

that corporate productivity skyrockets and the white-collar work becomes a four-day week. Again, what are we looking at when we look at this technology? I've told this story before, but it really captures my ambivalence and my ambiguity about this whole space. Imagine you saw a picture of an embryo at 10 days. It's growing almost exponentially. You can start to make out possible organs, limbs. Someone asks you to predict what kind of an animal this is. Is it a frog? Is it a dog? Is it a woolly mammoth? A human being?

Is it none of those things? Is it a species we've never classified before? Is it an alien? All you've got are three data points, 10 days, exponential growth, new living thing. All you know is that it's larval and it might become anything. I don't think AI is alive.

I don't think it's conscious. But I do think it's larval. And I do think it might become anything. I'm Derek Thompson. This is plain English. Charlie Warzel, welcome to the podcast. Thanks for having me again. So I wanted to bring you on because I consider you a bard of uncertainty when it comes to technology.

You are very, very good at diving into deep murky waters all the way to the bottom, seeing what's going on at the ocean floor and coming back to the surface and being like, holy shit. Like, there's some weird shit down there. Like, you're very good at explaining the quality of weirdness that you observe in all of these spooky

corners of technology. And AI, I consider a very spooky corner of technology. So I do think that lots of conversations about AI that I hear on other podcasts can like immediately go into the stratosphere of speculation. very, very quickly. And the truth is, we are headed toward the stratosphere speculation in just a few

minutes. But before we hit blast off, I want to start by anchoring the conversation to things that are actually happening, actual news. GPT4 is out, the fourth generation of this technology from open AI. I am using it. I have forked over the 20 bucks a month to get access to the chat GPT that's powered by this tech. Give me a sense of who else is using this technology right now. So I think right now there's many different camps. There's the, you know, the true sickos like you and I who forked over the money because we just need to experience this now and sort of get our bearings and we're going to write about it. There's that kind of like exploratory crew, right? Then there's the people who may or may not know that they're using it.

So that's the people who, you know, want to use Bing's new chatbot, right? That is infused with as we now know, it was speculated, but we now know that's GPT4. or like an early version of GPT4, there's all these different, you know, like sort of think of it as software updates, right? And so, you know, on that sense, Microsoft just rolled that out to anyone who wants to use it. So you could say that, you know,

millions of people are using that today to do that sort of prosaic search chatbot thing that we've been talking about for a while now. Then there's the sort of enterprise group, which I think is like a super fascinating use case, right? This is the world, I think, that's going to actually, like, this is how Uncle Steve or, you know, Aunt Molly or whatever is going to start to, like, encounter the technology, right? And that's like these partnerships through, you know, Open AI has a partnership with the consultancy firm Bain to work with clients like Coca-Cola, these, you know, big, huge companies. They are, they released like an API integration, which is essentially like, you know, allowing different programs to access the tool.

or different developers. And so we're seeing like Slack is developing one of those for like, you know, to respond to messages or like summarize big long threads in like very concise, you know, bits. Salesforce has that for their customer management stuff. They're going to roll that out. And like Salesforce is used by tens of millions of people to do really boring stuff across, you know, businesses everywhere. And then, you know, you have the announcements this week from both Google and

Microsoft of they're going to put these, this like generative AI tool stuff inside, uh, all their like workplace clients. So that's like docs, calendar, you know, Gmail slides, whatever. And that's going to be able to do and automate a bunch of that different stuff in the way that, you know, you had currently have like autocorrect for your Gmail. So really it's, it's hard to know how many people are using this tool and in what way does the purest version, which you is the I pay 20 bucks a month and I'm just going to experiment my face off. But I do think that there's a number of people who are encountering this in like a very organic way just through their jobs, or at least will very soon. Charlie, there's all these ways that people are using this and showing

off their usage online on Twitter. Give me an example of what you consider one of the most clever applications of this technology. There's one that I saw yesterday and I was, and like, like, from my journalist's brain is like, this is like, I love stunt journalism. And I was like, this is perfect. And it's basically, I think you probably saw it somewhere along the line if you've been looking at this stuff. But somebody basically said, I want to take $100 and start a business. And I want to like have, you know, GPT4 make decisions for me to try to turn that into as much money as possible without doing anything illegal and just sort of like refine the steps along the way. Right. So it's like what kind of business? And I think they decided on like environmentally friendly

like, you know, products like silverware and weird stuff like that for camping. But then like, okay, so what will the website look like? What will the logo look like? And then feed that into a, you know, stable diffusion or mid-jorney prompt and, you know, get something out and refine it. And it's really fascinating.

It's like, it's really cool to sort of see, I think we're used to, and I notice this with like Bing search, we're so used to with machines to be like, give me one discrete answer, right? and not to have the computer or the machine reason at all or make multiple inferences. But like the genius of AI-assisted search is you can say, like, how do I get this IKEA bed to fit?

Or can I get this IKEA bed to fit in the back of my Ford Satyr or whatever, you know, Ford Fiesta? And it will like go and do all the different calculations and look up all the different stuff. So I think that's where I and like a lot of Normie, need to start to like change our brains right it's like how can we get this thing to start thinking a little bit on our behalf or at least you know taking steps and making connections because that's what this technology ultimately does it just makes lots of inferences right or wrong

as opposed to like need need an answer ask for an answer get an answer transaction over there's two really really interesting things that you've made me think of the first is that you're so right that I have felt consistently jealous being online, seeing other people come up with just these ingenious ways of using this technology. And it makes me think, or perhaps it just reminds me, the technology always unlocks previously latent skills, right? Like the fact that some people are incredible drivers for NASCAR is a skill that had to be unlocked by the invention of the car. The fact that some people are incredible at GPT is something that maybe just, you know, I'm an amazing prompter. That's a, that's a kind of creativity that I don't know what I would have

called it before we had this technology. Like, I think you're a very creative person. I'd like to think I'm a creative person, but I go online and I'm like, wow, I am not remotely good enough at this particular skill that has, whose door has been opened by the invention of this new technology. I think what we would have called that skill and why a certain subset of people are great at it is we would have called it engineering, right? Because what engineers do is, like, when they're coding a lot of the times,

is they're like getting machines to, they're like programming a series of steps, right? Or like algorithms or like, you know, guidelines for a machine to do certain things and take all these different inferences. And that's sort of what this is, right? So it's sort of like, you know, whereas like the age of Google

was the age of like, like, you know, a good librarian or, like, researcher was dominant. Like, the age of, you know, GPT, whatever's, the age of generative AI is sort of like, the programming mindset is really dominant more than the librarian or researcher one.

It's like, like, how do I sort of, you know, give parameters to something and allow it to do a lot of work on my behalf? So, yeah, it's a totally different mindset. And I, like, I'm okay at saying, at least right now that I, like, I'm not as good at. and it is a lot of people.

The other thing your comment made me think of is that, you know, it kind of feels like a boring place to start, like what other enterprises are using this now, like Bain and Microsoft Enterprise. But the reason this is such an interesting phenomenon to me is the way it fits into the history of technology, the way that most game-changing products have typically launched, is that they started off expensive and rare,

and then over time they got common and shamed. That was electricity. That was the personal computer. That was the iPhone. They start is luxury products and they become common products. This is the opposite. Right out of the gate, these tools are cheap for many people. They're utterly free and they're absolutely ubiquitous. We have maybe hundreds of millions of people around the world that have used this technology at some point in time. They're red teaming it. They're showing how it's wondrous. They're showing how it's scary. In your conversations with experts, and I really now want to move into this wonderful article that you just published. What did people say would be some of the

most interesting implications of the fact that this technology isn't, it's being grown in one million petri dishes all over the world at the same time? Yeah, I mean, there's not a great, there isn't a huge precedent for it. And what's actually interesting in that comparison is that, like, the cost thing is going to probably scale in the opposite direction, sort of in the macro sense, right, like the more powerful these tools get, like, you know, a number that I heard talking to some people is like, you know, like a GPT six or seven supercomputer could could theoretically cost like on the order of $100 billion, right? So it's like, like, we don't have like that, you know, just the thing that powers it will cost more than like most, you know, large companies that exist in the world, right? So I think what's interesting about the usage like this is that it's going to probably follow to some degree. Sort of the best analogy I got for the moment of where we might be over the next couple of months or years is really a little bit of the, you know, the 1990s, late 1990s consumer Internet. And like the idea that the phrase that this person used who works at an AI startup was just like that, you know, these generative AI tools would just be nodes that sit on top of things, right?

And a way to think about that is like, you know, connecting anything to the internet, any service, right? You connect a service to the internet and it's not like the service is completely unrecognizable, right? Certain things change about it, certain things, like certain elements of how that service is, that service. or company or whatever operates, you know, if you, like, sell tickets or whatever, like, your box office in meet space is probably going to be a lot less important, right? And so jobs will change in that kind of way. It doesn't necessarily mean that they will get eliminated, but they get fundamentally changed, right? Or, like, you know, I think about all these tools the way that Microsoft and Google are putting them out. It's like, it's not maybe that your job,

if you do a lot of, like, wrote communication every day is going to go away. It's like, you're going to be sort of a middle manager of AI tools, right? You're going to be really good at delegating tasks to them, but also, you know, making sure that you can evaluate like an editor would do with our work that, like, they're staying, you know, they understand the assignment, and they're staying within the guardrails and refining and stuff like that. So there's this way in which, you know, things are going to get weirder and they're going to change probably, but not sort of in, not always in the, like, apocalyptic way of just like, you know, your profession is eliminated next week because this bot can do whatever it does. And so I think

like that's sort of where we're at. It's almost what makes me exhausted about thinking and covering, you know, this is that it affects like everything. It is, it is truly in that same way. Like if you were to say, hey, like, like, you know, to some, a reporter in 1997, like, you're responsible for all internet. Like everything that happens on the internet, that's you. Like, that job becomes impossible in a matter of seconds. And I think that's the same with, like, AI, right? Like, we're going to have to... It's just going to be, like, the connective tissue of a lot of what everyone does and how

everyone works. So that's a little off how I'm thinking about it. And one of the phrases that another person I interviewed used was that, like, they feel like AI similar to an invasive species in that way, right? Like this thing kind of comes into an environment and it really dominates and it really takes over. It doesn't mean it kills absolutely everything around it, but it changes the ecosystem, right? Certain things thrive. Certain things are less important or have less resources as a result.

And I think that's a good way to think about it. Like it sort of fits for me the exact balance of like, I'm a little scared, but also like, okay, there's, you know, something evolutionary. about it in the sense of, you know, certain things win, certain things lose all the time. I thought that metaphor was really powerful, and the reason that it clicked for me is that it captures this critical fact of speed. Like what an invasive species does is not just slowly over centuries colonize an ecosystem. It's by sheer dint of its invasiveness, it takes over very quickly. And we're looking at a technology that has gone from No one in the world was really talking about, I mean, no, people in AI, of course, are talking about it.

But no real normies were talking about GPT five years ago. And we went from chat GPT to GPT4 in a matter of six months. I mean, this is such a novel phenomenon. I remember this podcast launched about a year and a half ago. Our mutual acquaintance, Kevin Roos, was on the first episode. The name of the episode was, The Future's Going to be Weird as Hell. Guess for Technologies we used to illustrate. the fact that the future was going to be weird as hell.

NFTs and the metaverse. This was not on our radar at all. I had a feeling that tech was moving in a berserk direction, but this was not on the radar at all. And the speed with which it's come on the radar has been extraordinary, the speed with which it has proliferated, again, 200 million users in six months. That is invasive species shit.

And the fact that we're all experimenting with at the same time, that GPT on any one individual's computer is its own petri dish, suggests that the implications, I think, could become very weird, very interesting quickly. Do you mind if we get into some of the conversations you had with some experts about how exactly it would get weird

and positive and negative ways? I want to start with the observation that typically when journalists like you and me reach out to an expert in a field, a domain, you know, it's COVID maybe. You know, we talked to someone about, like, you know, aerosolized viral spread, The expert typically says, I have a very clear theory of how this virus works.

I'm giving you facts. These facts come from decades of research. I feel very strongly about them. I got the sense from your interviews that every expert you talked to was like, I have no fucking idea what's going to happen. I've spent decades on this subject, and I am at a loss for whether or not we are building the most miraculous machine in the world or dooming ourselves. You had this amazing study of, this was Melanie Mitchell, that's a lot.

Santa Fe Institute, a survey of 480 natural language researchers where they were asked whether given of data and computational services, could these technologies understand natural language in a non-trivial sense? They were divided 51% to 49%. I mean, no one knows how these things think and no one knows what's going to come of them. So let's take just one example here. You had a conversation with Eric Schmidt, former Google CEO. He wrote a book with Henry Kissinger about AI and the future of humanity. What did Schmidt tell you? So this was interesting. He was kind of actually like he just wanted to talk like I think he was doing like a little bit of like a press tour just about sort of I think the book has, you know, it came out, I want to say like a year

and a half ago and it has like, you know, renewed relevance. Anyway, there was a lot of like, him just trying to explain concepts that I was a little bit familiar with already. But then I got to the sort of like, how do you feel about, because in his book, they talk about a lot of nightmare scenarios. And I said, like, how do you feel about, you know, the possibility of a technology like this being unleashed on society the way that it is, given that like you and so many people who are building it or excited about it. it, like, in the same breath with the, this is going to, you know, whatever, change the amount of,

you know, intelligence in the world, this is going to give, you know, sort of democratize, like, rational thinking or like, whatever, whatever the heck way they want to talk about it, in the same breath, they're like, and, you know, it could be a civilization, like, level extinction event, um, maybe. And I was like, how do you, like, balance those things, right? And he gave this long example of, you know, AI powered tools and social networks and, like, amplification platforms that basically can take anyone's message and make it stickier, make it more viral, make it, like, punch it up, essentially, and give it sort of understand, you know, in this way these networks or these programs would understand what that particular network or an audience wants, right? like how to optimize for clicks or shares or engagement. And it would like punch those things up. And it wouldn't know like this is this is good or this is bad.

It would just say, we're trying to serve you, you know, the creator. And so the example he gave was someone, you know, like, I don't know, like a terrible racist, you know, who's just trying to or a troll, who's trying to do something awful. And that these things would just like naturally, you know, create and make this stuff even worse. that would be, that would be a, you know, a feature, not a bug. And, you know, I'm listening to him describe it. And I said, like, well, that's terrible. Is that, like, is that worth, is that risk? Just that kind of prosaic risk in one circumstance at scale.

Like, is that worth what we would get, like, the benefit of that knowledge of that, you know, automation and sort of that, you know, expanded, you know, computer consciousness. And his answer was, hell yeah. And he couldn't really provide me a justification for that, right? He would say, like, all the big problems in life right now, climate change, you know, speech issues, et cetera, they're all like, we need smarter people. And these tools are going to make everyone smarter. And I just think that that is, it's not that I like fully disagree that that could happen, but I just think that, It's not a very convincing or imaginative thing.

And this is what I find difficult talking to real, like, boosters of the technology, is they're really imaginative about the downsides. Like, we're talking about, you know, sky net levels of, like, computers become sentient and then, you know, try to kill us all levels of imagination, followed by sort of not a lot of imagination about the upside. It's just sort of, like, I mean, it's... to use a phrase you're familiar with, like, abundance, right? It's just like, it's going to create that.

And it's kind of hard to understand where we're going to get there. And so that's just a attention that I see, and there's not a great explanation. And then the other thing he said, obviously, is he's trying to raise awareness because he wants the right people to be building these tools and to learn the lessons of the social media era. But as I told him, I'm not so sure we've learned all the lessons of the social media era yet, because we're kind of just moved on. We've kind of just onto the next technology. And there's a huge can of worms with all of this,

whether it's Sam Altman who runs OpenAI or the people at meta building large language models or Google building large language models. It's who are we trusting? How are we making these decisions about who we trust to build this stuff if the worst case scenario is they become truly intelligent and kill us all? Given that if we start,

stop doing anything in this space. If Open AI is shut down and Microsoft stops doing all their AI research and meta stops doing their AI research and Alphabet and Google stop doing their AI research, et cetera, if we shut it all down, that is nothing to do with China. It is nothing to do with Russia. It is nothing to do with some non-state terrorist actor who can get technology that is developed by China and Russia, et cetera. Given that reality that we just don't have, the federal government.

doesn't have, the U.S. federal government does not have a monopoly on the question, should we proceed with AI research in the world? Have you spoken to people? Are you persuaded by the case that we should just stop? I actually don't believe that we can. And that's what's so interesting to this, because like if you look a little bit now at, and I'm going to get the real technical specifics on this wrong because I'm like I am my job is to talk to people about this stuff and try to convey you know as you said sort of like all the weirdness and uncertainty I the technical aspects of this whole world are dizzying to me and and I that's why I try to rely on them but from what I can gather there are already ways that these models are able like the true instance of the model

that people are able to run them locally on like, you know, really good, you know, computers, like a high-end gaming computer. And that those, like, those models are going to be, that's only going to become more and more, you know, a thing going forward. And there's the whole idea, you know, there's this whole idea of the open source idea

of, you know, large language models and people who are building them. And now there's a bit of like a fight almost, or a disagreement between certain companies about like Open AI, which Open is in the name, is now saying like it's actually, we're seeing it's probably a pretty bad idea to make these things open source

because of the ability to abuse them or for them to fall in the hands of the wrong people. But some of those things already exist, right? Like the sort of the blueprint already exists. And I just don't think you can like put the toothpaste back in the tube on a lot of this. Now maybe the most powerful versions, the ones that require hundreds,

of millions of dollars. Like maybe we will see, and I'm hearing from certain people that like, there is a lot of buzz in the federal government right now around like, we probably need to like really figure out, you know, who's allowed to have how much, you know, computational power and access to the funds and the resources and that kind of regulation around this stuff. But the idea that like the tools we have right now aren't going to be able to be spun up by people in a very short amount of time to use as they see fit. Like that's, that's happening is going to happen. We're not going to be able to, to, you know, take that out. And that's a whole different ball of wax, can of worms, whatever, use your analogy, then talking about like US versus China, right? Or US versus, you know,

terrorist state actor getting licensing stuff from China or whatever. And that's, that adds a whole another level to this, right? And it's something, we don't have to go there now, but it's something like I allude to at the end of my piece when it's talking about like, there's some really real questions here about like, should we build this because, you know, we should always strive for technological progress? Should we build this because it's a, you know, a security imperative to build this? Should we not build this because it's a security imperative to build it? These are really interesting questions and like we're only in like hour one of really trying to, you know, suss them out. The writer Stephen Johnson has a long piece in the

the New York Times magazine about Thomas Midgley, who is a brilliant inventor in the first half the 20th century. He worked for General Motors for a while, solving problems like engine knock. And he played a huge part in inventing two of the most toxic, most horrible chemicals invented in the 20th century, chloroflorocarbons, CFCs, and Frion. And it's a really interesting story about how invention can sometimes create monsters. The problem of CFCs, which were burning through the ozone, had to be solved by a group of scientists and corporations and politicians that came together in the 1980s to do something called the Montreal Protocol, which essentially banned CFCs. And since then, the ozone has recovered, especially over

Australia and New Zealand. In a similar way, you know, I am worried that we are creating a monster that we won't necessarily know how to stop unless we all come together as a world. There was a survey that Ezra Klein noted in his column last week, a 2022 survey, very recent, of AI experts, that asked them, what probability do you put on the human inability to control future advanced AI systems from causing human extinction or similarly permanent and severe disempowerment of the human race? The median reply was 10%. One in 10 people working on AI think it is possible, if not probable, these systems will cause human extinction? That is crazy. What is your reaction to the fact that this industry is populated by people who think they're working on doomsday device?

It's even, it goes like further than that too, right? Like if you look at Sam Altman, one of the founders of OpenAI, wrote a blog post at the end of February that actually like didn't get a ton of traction, I thought, given what it's talking about, but it's working towards, you know, an artificial general intelligence, which is the goal of Open AI, which is that sort of like a truly intelligent, conscious, almost, you know, entity. And, you know, it basically lays out

that exact dichotomy that I was talking about with Schmidt, which is, you know, this could be sort of the greatest prosperity generator in human history. This could, you know, essentially unlock, you know, unforeseeable amounts of things at a level so much greater than, you know, modern personal computing in the internet.

And obviously, like, throws in, and the risk is potentially, you know, civilizational. If not, you know, some smaller, very concerning, you know, security risks, whatever. And every conversation to a person that I had with, I'm talking about engineers building these tools, researchers, like boosters, skeptics, safety experts. truly everyone, they all veer into this territory, right? This like super late night dorm room or like philosophy class territory. And the thing, I really couldn't find any other

parallel with other technologies, certainly not like the social media ones that I have covered for, you know, more than a decade now. I think there are some things maybe like genetic modification. but the thing that I just kept going back to was everything that I've ever read about the Manhattan Project. Yeah, it's nukes, right? Exactly. And it just feels like when you read about, you know, the hand-wringing that was going on in Los Alamos over this stuff and the, you know, like, I'm not going to get the name,

but there are people that quit the project, like high-level people because they just said, like, I actually can't, I just can't get on board. it. And what's interesting about that to me, what I know at the end of the piece, is like, they knew what they were building. Like, it was very clear. Like, we are building, we're, you know, we're splitting the atom. It's going to create this, you know, weaponized, massive explosion that we can control. It's going to kill a lot of people. And it's a, you know, it's a defense thing. They knew exactly what it was. When you talk to people in this field who are building the products

or who have, you know, who can see the guts of them or understand them because they have, you know, math and physics and computer science degrees, they will say, we do not know how, we know how these things are trained, we know how they're refined, we know how they're weighted, we do not know how they come to their conclusions, their inferences, in the same way that, like, you can watch on an MRI a part of the brain light up, but you don't really know how the neuron, you know, fired and gave that exact thing. There's just a level of, you know, quote unquote, but mysticism around it just simply because we don't have

the ability to understand it. So in a sense, we are building something in the same way that they were building that, you know, those nukes, but they knew what was going on. And here we just kind of don't know, right?

Like, Open AI's whole reason for being is basically, this is such a dangerous experiment. We want to do it with the most, you know, altruistic, benevolent values possible and with transparency, even though I'd argue there's some issues there. But it's wild.

It truly is like, I've done a lot of technology reporting in my life, and I've never had so many, like, I'm just like up at two in the morning staring at the ceiling, being like, what are we doing? I love the analogy to the Manhattan Project, and it might be even one degree

weirder than you portrayed. Because it's not just the scientists in Los Alamos designing a nuclear bomb that they thought might end the war, it's also imagined if those same scientists knew they were designing the core energy source of a nuclear power plant. And so they were thinking, we might have in our hands the secret to clean energy forever for the future of the human race, but also we might be launching a bomb, and this is something I believe they thought. There was some of the world. fear about this, that will incinerate, it will ignite the nitrogen in the air and cause the

explosion of the world. This might be clean energy for the history of the planet, or it might incinerate the world. Those options could have all been on the table if they had perfect insight into the 1950s progress with Adams for Peace and Eisenhower's clean energy, well, nuclear energy investments. It's just, it's mind-boggling to think about all this being on the table. Speaking of all this being on the table, there is one doomsday scenario that has gotten way more attention than it was any other. This was from an Eliyzer-Yadkowski interview and a podcast where he told a story about how

he imagined AGI could contribute to the death of the human species. And I think it's worthwhile to just retrace this story just so that we can talk about it a little bit and just kind of share our feelings about how we how we're making sense of this particular prediction. Do you want to briefly summarize the story? Yeah, I'm going to be drawing off of a hacker news thread that's like summarizing this because we were talking off the air about this and like one thing about the like the AI community, especially like those who are sort of the thinking brains in a tank around it is that they're very verbose or just like loquacious, like very long podcasts, very long blog posts, things like that. So I'm relying a

little on this, but basically the theory is that like we're going to spend a lot of venture capital money, we're going to put it in AI. And most of it actually will go to waste, but a small part will level the technology up to the point where the AI will be able to write another AI, right? And then that AI will write another AI and that one and that one and that one. And you sort of get that like replication, almost like, you know, human evolution where like things start to change, unexpected things start to happen, till you get to the point where an AI

will be smart enough to announce that it's concluded that atoms inside human bodies could be repurposed for something else that it has decided it's better, right? That human beings, their energy, they're like, there's something wasteful there. So what it will do then is it will basically try to design a plan the way that any

like foreign power or you know basically like a terrorist organization would right and so his example is that it's will the AI will send an email to a human human in power

with like specific instructions on how to make a bio weapon right the AI will also possibly like hack into and break into a bank and get access or you know do run some kind of scam get access to a large pool of money and it will

you know it will pay people human beings to you know follow these instructions to make a weapon basically just to do its bidding and you know these people might not even know because the AI is so crafty and clever that they're talking

you know to a sentient you know computer network it will just it will sound like it's coming from an organization or something like that like another person Anyway, the other idea, too, is that, you know, whatever this weapon is, whatever this dangerous doomsday device is, most likely like a bio weapon, it will be something that is like previously unknown to humans because, again, these intelligences will be sort of working at a level that we're unaware of, right? Like a scientific compound we've not really discovered that's lethal. Anyway, it will be deadly to 100% of humans and someone will do this. Someone will be motivated by this, release this out, and not know that they're ending the human race, but they will.

And the idea here is that none of this will take place with any warning, right? The day that civilization ends will just be the day that it happens. Because the AI will be crafty enough to hide all this from us. And the idea here, I think, like his primary idea is that this AI doesn't hate us. It doesn't necessarily want us to die. but it just sees that we are not the most efficient use of Earth, right? Or the energy or the atoms or whatever on Earth. And it is more intelligent.

And so, like, I think, you know, one of the ideas that I heard from somebody else describing this was just basically it's like, you know, like homo sapiens beat out, like, other, you know, other competing species, just simply because, you know, we were the right ones, that we are more intelligent and that that would sort of be what's happening here. There is a higher intelligence and it is acting,

you know, not necessarily maliciously, but it's just acting out of what it thinks is its own rationale. And that again is the, like that's the, that's the doomsday scenario as he sees it

or a version of that doomsday scenario. I just want to like caveat since I've just laid that out in the most science fiction terms. I don't really see a ton of, um, I don't really see any like real evidence for that. It does feel extremely imaginative. Like, it's very, there's not really like a,

well, we're right here, you know, in the level of building these things. So like, it could easily jump to here and then off to the races. But the thing that I do find just compelling about the argument is AI is writing AI's, writing AI's, writing AIs. Like that is sort of that evolutionary thing where it's like we don't, you know, you kind of invite chaos if that would ever happen. Right. So I just to summarize for my own benefit, it's like, five steps here. Step one, design superbacteria. If this technology can, and we hope that it can, be ingenious at coming up with new molecule combinations that can cure diseases, then theoretically the same technology could conceive of molecule combinations that would kill us. So step one,

design zero bacteria. Step two, steal money. If computer programmers can hack a bank, super ingenious AI can hack an even bigger bank. Step three, bribe scientists to make. this molecular combination. They got a lot of money. They can send a plausible email to some science lab and be like, hey, you know, I'm whatever, the head of the science department at the, you know, the Charlie and Derek Institute in Germany. Like, can you please put this together? I'll pay you, you know, $1.5 million. Step four, pay a hapless task rabbit to release this bioweapon in wherever. Step five, everybody dies. And as you said, it's not about hating humanity. It is about the logic of select all delete. Like if you are looking at a page and you want to clear it, you do not care

about the letters and the seraphs and the spacing and the font, you're just trying to clear the page, select all delete. It's that kind of logic that might result in some of these doomsday scenarios. So let me tell you why I refuse to buy in hook, line, and sinker into the Eliizer story. there's a logic around a lot of AI doom scenarios that are kind of like number one sorry to keep enumerating number one AI in the future will be able to do anything two within the set of anything includes a lot of bad shit three therefore all the bad shit within the set of anything will come to pass it's like that's actually just a made-up syllogism like it might be predictive but i don't know how i'd prove it wrong you've set up a set of rules

for yourself that have no guardrails. AI will do everything, and some things within everything are bad. I don't know what to do with that, even if that's the best way to think about the future. To me, too, I also just think, like, we are, so one, you mentioned her earlier, this woman I talked to, Melanie Mitchell at the Santa Fe Institute, and one of the things that she cites in the whole like disagreement among experts thing is very like simple definitions like what does it mean for an AI to understand right or a large language model to understand like we don't have definitions around that like there's some people who believe that understanding is simply the fact that like

if if gpt4 can ace or get very close to acing the SATs which are an aptitude test for humans then it means it has even synthetically an understanding, right? The inferences it makes, even though it's not itself conscious, is enough to have replicated human understanding that it understands. The other side of that group, which is much more like humanist, is saying, like, no, it does not understand. And the reason it does not understand is because true understanding is not just making inferences, it's having life experiences, right? So, like, if you're, like, it's the experience of being a human in the world and, you know, feeling temperature changes, right?

Like, the example she used with me is, like, if you, the AI can say, like, you know, the driver, like, angrily cut off the car in front of it, you know, a view or something like that. But it has no understanding of why someone might have road rage, you know, why that is a dangerous thing to do that.

Like, it doesn't matter that it's making the inference correctly. It's just like, it's not, it doesn't have those, experiential qualities. And those people say then, there's no way, right, that you could get to this level of true, you know, artificial, general intelligence like a human because it's not going to be able to have those experiences. The reason I'm saying all this is that I think almost more likely than this idea of like truly creating the super being, right? I think like if you look at like the bio lab thing, what's interesting to me there is like that can happen.

just with AI tools as a middleman, right? Like if you're at a lab, a virology lab, doing this very controversial gain of function research, right? And you're saying like, hey, we want to sort of push this forward, right? If you give this thing the wrong parameters, it could create something horrible. And I'm talking just about like tools, right, that are just trying to like come up with some compound

and some hapless person engineers based off of a thing because, you know, they didn't install the right, guardrails in the system that are going to do whatever, and it creates some thing, and then it, you know, gets out in a very organic way and kills a lot of people. Maybe doesn't extinguish the human race. But that's like, that's a very sort of like, like easy to imagine problem that doesn't require sentience from the computer. It just requires a lot of human error based off of a tool that's very powerful. And I think that's the thing that I am much more likely to be like concerned about than

Skynet When I think about stuff going wrong I think sometimes the most interesting stories are one big thing going terribly off the rails and ending the human species but what I'm taking from your point and I think

I agree with it it's much more plausible to me that many small things go off the rails because of small misalignments or even aligned actors We're bad actors with aligned AI do little bad,

create little bad nuisances for us. And that generally we perceive over time or maybe even suddenly a weirding of our world, right? We come to realize, oh my God, like it becomes rote in the news cycle that, oh, well, fucking AI's done something weird again.

You know, there's been another, like, little AI hacking because, you know, this regional bank didn't have solid guardrails for its deposit. And so there was a hacking and, you know, $3.5 million were just stolen from First Republic by an AI hacker. Or there's another corporate scandal. AI hallucinated and Bain told this agriculture AI company that they needed to, you know, focus on Chile based on something that was totally made up by the AI. I think it's much more likely that we see these small crises of misalignment than that we suddenly wake up in a world where half the population is.

has died in the last 12 hours because of some catastrophic misalignment. A helpful way to think about this is the way that we think about just like a technology as simple as Facebook, right? Like, I think if you were to say to somebody in 2007 at Harvard when the Facebook is going around, if someone were to say, this is going to cause a genocide in Myanmar, like this will be like a primary accelerant for. or, you know, true, like, horrible, you know, repression. People would be like, okay, well, you're out of your mind. Like, truly, we're just trying to figure out the person we met at a party last night.

But I think when you look at, like, you know, how these things happen, it's small, right? Like, Facebook was not a mind control machine that caused people to go insane and do some thing. It was an accelerant for a lot of social conditions based, you know, around that, like, left sort of unguardrailed and unchecked, you know, helped lead to terrible awful things in this place and also in a lot of other places across the world. And so I think that's a helpful way to kind of think about these tools, right? Like you can say X terrible thing's going to happen and it's, you know,

we, you know, Open AI might have blood on their hands or someone like that, but not necessarily always in the way of, like you said, the big, huge catastrophe. Sometimes it's simply just, you know, a lot of small things going wrong and intersecting with like terrible pre-existing social, cultural, political currents. On the more prosaic side, and this is maybe where we can end, I think it's a very interesting question of what kind of jobs can these tools do effectively. I wonder, do you think Chad, ChyPT could do your job? What parts of a journalist's job do you think chat GPT could do right now? It's weird, right?

Because I want to be optimistic for myself and a lot of other people and say that it's going to like make our jobs weird potentially but not extinct. Like a number of people have made this analogy. But it really does sort of feel to me like the greatest. skill that we can all have now is to be editors, right? Like, you know, using journalist editor as a thing. Or, you know, in the business world, like, the way that it's been described to me is, like, chat GPT or GPT4 or whatever is like a really overzealous junior employee, right? Like, really smart, really totally, totally capable, no life experience, right, in the field. So it's like, give the employee a lot of parameters, let it cook, it's going to work super hard, it's going to

deliver you something. And then you might have to go back and say, like, oh, okay, well, you know, it's actually like, you know, that's not how things work or whatever it is, and edit it and refine it and sort of keep it, you know, manage it. Something that I watched the GPT4 OpenAI demo on, I think Wednesday, Wednesday or Tuesday of this week, last week. And the one thing it walked people through was like doing taxes and like ingesting, you like ingest the state's tax code, and then you ask some of these really hard questions that you might not know the answer to

and it does the calculations, it runs through everything, right? And then you have to check it. You have to like check its work, right? You have to kind of go through and make sure and the phrase that they used is like, GPD4 isn't perfect and neither are you, but like together it's going to enhance this thing.

And I think that that's, you know, I mean, that's like a very nice way of putting it for them. Yeah. Right? It's corporate. It's, I think, I think Microsoft yesterday said that Bing's answers are usefully wrong, which is my favorite way of putting it. But I do think that there's something there of we're all going to be working together with it.

I think that's right. I do think that a skill I find being elicited from me when I use GPT4 is the skill of managing a reflection of my own thoughts. And so it puts me in this weird position. I don't like being a manager. I used to be an editor for the Atlantic, and I kind of self-fired myself from that position because I wasn't very good at it. And so I feel like it's activating a muscle

that is quite atrophied over the last 10 years. But I'll prompt, and it'll give a B-plus answer. I'll say, let's make this A-minus together, and then sometimes we can scrabble our way towards something that's worthwhile. But it is so weird to think that, you know, to sort of round us out here, People, children, adults, young workers, seniors are going to spend the next decade with this little

disembodied super genius Damon next to them, this little assistant of hallucination and daydreaming and brainstorming. And we're going to have to learn the skill of coexisting, of becoming like AI managers. In a weird way, it's like we are all AI managers now. So Charlie Worsell, thank you very, very much for talking me through this. I really had fun. Thanks for having me, man. Thank you for listening.

Plain English is produced by Devin Manzi. If you like the show, please go to Apple Podcast or Spotify. Give us a five-star rating. Leave a review. And don't forget to check out our TikTok at Plain English underscore. That's at Plain English underscore on TikTok.