Pivot - How worried—or excited—should we be about AI? Recode Media with Peter Kafka

Episode Date: May 30, 2023

AI is amazing… or terrifying, depending on who you ask. This is a technology that elicits strong, almost existential reactions. So, as a Memorial Day special, we're running an episode of Recode Media with Peter Kafka that digs into the giant ambitions and enormous concerns people have about the very same tech. First up: Joshua Browder (@jbrowder1), a Stanford computer science dropout who tried to get an AI lawyer into court. Then: Microsoft's CTO Kevin Scott (@kevin_scott) pitches a bright AI future. Plus: hype-deflator, cognitive scientist and author Gary Marcus (@GaryMarcus) believes in AI, but he thinks the giants of Silicon Valley are scaling flawed technology now—with potentially dangerous consequences. Subscribe for free to Recode Media to make sure you get the whole series: https://bit.ly/3IOpWuB Pivot will return on Friday! Learn more about your ad choices. Visit podcastchoices.com/adchoices

Support for Pivot comes from Virgin Atlantic. Too many of us are so focused on getting to our destination that we forgot to embrace the journey. Well, when you fly Virgin Atlantic, that memorable trip begins right from the moment you check in. On board, you'll find everything you need to relax, recharge, or carry on working. Buy flat, private suites, fast Wi-Fi, hours of entertainment, delicious dining, and warm, welcoming service that's designed around you. delicious dining and warm, welcoming service that's designed around you. Check out virginatlantic.com for your next trip to London data, and a matching engine that helps you find quality candidates fast. Listeners of this show can get a $75 sponsored job credit to get your jobs more visibility at Indeed.com slash podcast.

Just go to Indeed.com slash podcast right now and say you heard about Indeed on this podcast. Indeed.com slash podcast. Terms and conditions apply. Need to hire? You need Indeed. Hello, I'm Kara Swisher. And I'm Scott Galloway. And today we're featuring an episode of Recode Media from our friend Peter Kafka. In this episode, Peter tries to answer the question, is AI going to change everything we know or is it just another tech hype cycle? He speaks with folks from Google, the New York Times

and others to get the answer. Scott, what's your biggest question around AI? You're still messing around with it, right? I'm on that thing every night. Which one? Chat, GPT or board? I'm on that thing every night. Which one? Chat, GPT, or Bari? I'm on all. I'm on Notion. You got to do Notion AI for writing. I'm on Dali. I'm on, I can't stop. And every time I watch TikTok, it tells me about a new one. Oh, really? Is there any voices you're listening to that like pro and con?

Anybody? Well, one of my colleagues, Gary Marcus, is pretty interesting on it. I hate to say it. I'm part of the, most of my AI I'm learning from TikTok and Reels. Oh, all right. There you go. But it's fascinating. What I tell everybody is,

AI is not going to take your job if someone who knows AI is going to take your job. That's right. You need to learn about this stuff. Yeah, yeah. Do you have a worst case scenario for AI? Yeah, I'm living it. Yeah.

Because everything is so catastrophizing or catastrophist, I'm actually, which is unusual for me, I'm an AI optimist. I actually think it's going to be, I think it's going to unleash incredible economic productivity, huge benefits around healthcare. I'm really excited about it. Yeah. Yeah. You be an optimist. That's good, you sunny optimist. There I am. Hello, sunshine. Hello, rainbow's coming out of my ass. Hello.

I think of you. There is a worry. I mean, I know the writers, people are worried about it. They call the fun jobs and, you know, but no matter what, Scott, you still have to take out the trash. Yeah.

Hey, I have a question. Do you or your wife take out the trash? No, Carminia takes out the trash. Hello, douchebag. Hello, douchebag. Who do you think between a man and I take out? We do not have Carminia. Who do you think takes out the trash. Hello, douchebag. Hello, douchebag. Who do you think between a man and I take out? We do not have Carminia. Who do you think takes out the trash?

Not the AI. Well, this comes down to who's the man. So who handles a really tough parking spot? Is that wrong? Is that wrong? I want you to guess. Who takes out the trash? The fact you asked means you.

It is. It's me. I take out because I ain't over-tensioned. That's what you call critical thinking, you. It is. It's me. I take it. Because I ain't over-tentive. That's what you call critical thinking, people. When someone asks who's a bit of a narcissist, it means they're setting themselves up to look good. That is correct. I love taking out the trash.

It's one of my favorite things. Anyway, an AI cannot replace me, and it shall not. Perhaps a robot, killer robot will. But we'll wait on that. More on that. Here's Peter Kafka and Recode Media. Enjoy. Joshua Browder is bad at driving.

This isn't a judgment. It is a fact, and it's verified by the police. I like to make the excuse that they give out tickets a lot, but in reality, I was a terrible driver and probably got about 30 tickets. But Browder is good at getting out of tickets. The trick, he figured out, was writing letters to appeal the tickets. He did this a lot, and it worked. And a rumor spread that I was the guy who could get my friends out of tickets, and people would just send me photos of their tickets. And after about the tenth ticket, I thought,

there must be an automated way, and so I created it just for fun. Browder researched the top 12 reasons that tickets were dismissed. Things like the signage or there being a mistake on the ticket. And then he wrote a piece of software that matched your ticket with one of those excuses. Then takes down all the details, inserts those details into a letter, and then faxes or mails it to the right place. Browder turned this talent into a company, Do Not Pay, so you can pay him to automate you out of tickets or bank fees, free trials, stuff like that. This is cool, but not mind-blowing, not for a tech startup.

But then Browder, like a lot of people across Silicon Valley, started to get excited about artificial intelligence. He wanted Do Not Pay to use it because he wanted to do more than write letters. He wanted to send AI software to court to replace a human and act as someone's lawyer. Over 80% of people who need legal help can't afford it. And a lawyer can't read 10,000 documents in one second, but AI can. And so it can actually craft pretty good defenses in some of these areas. So I put out an offer in December saying, we want to actually have AI whisper in someone's ear for a speeding ticket case. And over 300 people took me up on our offer because I said, even if you lose, we'll pay the ticket for you.

So how did you imagine this was going to work? So the person will wear an earpiece or these glasses where they kind of have the audio through the glasses. You'd walk into the courtroom with some device that had a microphone. Exactly. That they wouldn't talk to something on the internet. It would listen to what's being said in the courtroom, process it with the AI, and then whisper back to the person what to say. So it literally would be like having software tell you, poke you on the shoulder and say, say, I object. Exactly. It's like having a really good lawyer whispering to you.

Real lawyers did not agree. Browder's offer caused quite a stir with human professionals. So these lawyers started phoning up state bar associations. And there are these laws sometimes from the 1800s and 1900s saying that you need a license to practice law and it's actually a criminal offense to practice law without a license. And they were making these new arguments to suggest that by having a robot lawyer in the courtroom, it's illegal and I should go to jail for helping inspire that. And you couldn't just say this is the same thing

as bringing a calculator into the courtroom? That's what we were trying to do, but some of the state bar associations were saying, well, we'll arrest you and find out. Browder backed down. He did not like the idea of going to jail. And now his company is facing a lawsuit for the unauthorized practice of law. I think you can call this a mixed result.

Browder isn't happy about getting sued, but he also knows that guy who brings robot lawyer into court is a good way to get attention. What kind of attention we're paying to him right now. So you did this thing in part because you wanted to generate attention. You generated attention and that got you sued. Is that a fair summary? Yeah, it's we put a target on our backs. And if you have powerful, crazy lawyers with a target on your back, it's a scary situation.

Browder thinks that eventually this stuff will stop being a novelty and that we're going to use AI in court just like we're going to use it in all kinds of situations, so often that we won't think it's remarkable. I think it's inevitable, and all these lawsuits, it's just the dinosaurs suing to stop the ISH. AI is being used in the US right now to generate sentencing reports for judges.

Other judges, like a Colombian judge, are already using ChatGPT to write their briefs. And so why can't this same technology be given to consumers? There are a flood of people like Browder. People who say this tech is not just coming around

the corner, it's here now, and it's going to have massive effects on everything. Business, culture, science, the way we live, the way we do or don't make a living. This seems like it all came out of nowhere. You didn't hear people talking about AI like this a year ago. Now it's all anyone is talking about. It's hard to tell what's bullshit, what's a stunt, and what's real. But it feels all-consuming. People at big companies are spending a lot of money on this tech. Entrepreneurs are

creating new businesses. And there are people watching all of this happen who are getting really, really worried. I want to talk to all of them. So for the next three episodes of Recode Media, that's what I'm doing. I'm going on an AI tour while I try to figure this out. And you're going to come along with me. Our first stop, Microsoft. Yep, Microsoft. The people who bring you Excel spreadsheets and Word documents have bet more than $10 billion that AI is the future.

They put that money into OpenAI. That's the company behind ChatGPT, which is the chatbot software you've been hearing a lot about the last few months. And now Microsoft is bringing ChatGPT into their own products like Bing, Microsoft's search engine. ChatGPT can do amazing things. It can write software. It can make you a menu plan. It can also spout out alarming ideas and flat out untruths. Microsoft CTO Kevin Scott thinks this is the future. going needs to approach it with a little bit of humility because things are changing so fast.

But the sense that I have, having lived through three of the big technology platform shifts over the past 50 years, so I was a 12-year-old, 10-year-old kid when the PC revolution was happening, when I learned programming. I was in grad school when the internet revolution happened. And then I was, you know, pre-IPO employee at Google. And then, you know, I was running a mobile startup right at the very beginning of the mobile revolution, which sort of coincided with this, you know, massive shift to cloud computing. This feels to me very much like those three things. So that's very big picture i want to narrow it down for a minute or at least explain some terms to the listeners and to me as well people throw around ai as a term we hear a lot about generative ai it seems to be the particular

flavor that we're very interested in the moment there's also discussion about agi can you sort of help us set the table here and and and emphasize what we're talking about right now when we're talking about AI? Yeah, so artificial intelligence is the, or AI is the name of this field that we're all practitioners in. The term was coined in the 1950s. in the 1950s. And in a certain sense, like it's a terrible name for what we're doing because it invites

all of this comparison to biological intelligence, human intelligence. And like the way that AI software works is very, very different from how a human brain works and like what capabilities

AI has are very different from what a human brain. Like there are a bunch of things that are easy for AI that are hard for humans. And there are a bunch of things that are just trivially easy, even for, like, an infant human being that are nearly impossible for AI to do. And so, you just have to be really careful about drawing those, you know, connections between what human intelligence is and what artificial intelligence is. So So those other two terms that you mentioned,

so generative AI is the thing that is getting everyone excited right now. So these are very large-scale AI models, so big pieces of software that were trained on lots of data with huge amounts of compute that can synthesize things. If you embed them in a chatbot, for instance, they can have a conversation with you. So they're generating language. If you are, for instance, like wanting to do something creative, they can generate visual

images. And so like we'll have more and more types of generative AI over time. Right. I mean, you use the word, you hear compute a lot, and I just translate that as powerful computers. Correct. Yeah. Very, very, very powerful computers. And then AGI is the thing that everyone says, maybe we'll get there one day, maybe we won't, and that sounds the most science fiction-y? What is that? So, AGI is an acronym for Artificial General Intelligence, and this is, you know, sort of the thing that the founders of the field of AI were going for

from the very beginning. So, it's a thing that in a, you know, by some definition, is close to human-like intelligence. So where is this going in the near term? We've sketched out there's this idea that maybe one day the computer is actually going to replicate human thinking. That seems to be a far way off. If ever we have this stuff now that's really novel and interesting, it seems to be moving very quickly. If you're talking to a layperson, and again, me, how should we think about this tech impacting our lives in the next couple of years? Well, the vision that we have and that our partner OpenAI has is that we believe that you can think about this AI as a platform.

Like a platform allows people to build things on top of it. And so people will be able to pick these platform components up and build software with it. And so what I think everybody should expect to see over the coming months is, you know, you've seen this progression of what Microsoft and OpenAI are doing with the software. So, you know, it's like GitHub Copilot, which is a coding assistant, Bing Chat, which is a assistant to help you do a search. We announced the Microsoft 365 Copilot, which is a set of AI assistants to help you get your work done. So, you know, we're going to have more of those ourselves, but other people will be able to build that sort of software for whatever it is the important problem is to them. And this is why you hear people talking about this being sort of

akin to the iPhone slash app store moment. Like you have this technology and then you allow everyone else to build stuff on it and you get get fart apps, and you also get Uber. A hundred percent. Pick your range. It's great you mentioned the fart apps because I think one of the interesting things that happens with every one of these new technologies as they emerge is it takes a while for people to figure out what the interesting things are to build. So it's often the case that the easy things to build at the beginning are not the interesting things, which is why you get fart apps. But eventually you do get something like

Uber, which is a very complicated business that couldn't exist without that smartphone platform. And then right now people are also intentionally testing this stuff out, both the OpenAI version, the stuff that you have, and saying, what can we do with this? What can we do with this that the makers didn't intend? Can we get it? Where is it going to fall flat on its face? Sometimes they're accidentally stumbling into some exchange where the chatbot seems to be saying rude or even worse things to them. And you guys are going back and modifying that as you get the feedback. When you use Bing right now, there's a banner on it that says, let's learn together. Bing is powered by AI, so surprises and mistakes are possible. Make sure to check the facts, share feedback so we can learn and improve.

So I think I understand the intent behind that disclaimer. On the other hand, you guys are a giant, giant company, both enterprise and consumers. How are you thinking about rolling out a product that is by definition not finished at all? Yeah. And so if you think about the product as a platform, like it will never be finished, like it sort of has infinite possibility for what people can do with it. The way that we have been thinking about things is we've spent, you know, from 2017 until today, rigorously building a responsible AI practice. You just can't release an AI to the public without a rigorous set of rules that define sensitive uses and where you have a harms framework

and where you like even are transparent with the public about what your approach to responsible AI is. And so like we've publicly disclosed two versions of the responsible AI standard that we use inside of the company. All of these things that we've done run through this responsible AI process. That's people imagining what could we do with this that would be bad and let's prevent it from doing that. But even in the best case scenario, they're not going to think of everything. And even in the best case scenario, someone's going to foil those efforts. And I'm not even talking about that yet. I'm just talking about the basic fact that this stuff is still pretty raw.

It seems, I understand if you were a home hacker when you were building your own computer, you'd say, no, it doesn't work. You got to get in there with a soldering thing and get into the circuit boards. But this is going out into the public. Yeah. So, look, the point of actually launching the software is because it actually, for the first time, I think, really has given a very broad swath of the population an opportunity to see the software, to see what it's useful for and they're seeing all sorts of things that the software is

useful for that we hadn't imagined, but also giving us feedback. One of the things that has been really useful for us is, there are obviously this clear set of things that we vetted beforehand that we haven't seen anybody demonstrate a behavior or one of these AI systems that is... And people are talking about like, oh, how do we get it to not... We want to make sure it doesn't tell people how to make chemical weapons is the thing

people often talk about. You know, and among like many, many, many, many other things. I mean, there's so much stuff that we've done to try to make sure that the harms that we all, like any reasonable human being could sort of look at a thing and say, yep, this is bad. Like we don't want the system to do it. And then you're going to have this category of things where, you know, you're going to randomly choose two reasonable human beings from, you know, 8 billion folks on the planet. And like, you're going to randomly choose two reasonable human beings from, you know, 8 billion folks on the planet. And like, you're going to have disagreements about like what's acceptable

and what's not. And like, this is, you know, just sort of how you build products in general. Like, people have different preferences depending on what part of the country they're from. They have different preferences depending on what part of the world, you know some places one thing's acceptable and another it isn't and uh part of you know the the thing that we're learning right now by launching all of this stuff is you know where those lines of preference are and like how we can quickly conform the product to give people the thing that they want if you do any amount of even you don't have to read, you can just browse some of this stuff. And about AI, you inevitably get people talking about doomsday scenarios. And again, part science fiction and part a thing we ought to be thinking about. I'm

sure you've seen the survey of AI experts from last year. And depending on how you phrase the question, the question says something like, what percentage chance do you think AI has of wiping out humanity? And depending on how you phrase the question, it's five to 10 to 14%. Those seem like unacceptable odds to me. What am I missing? And how do you think about playing with tech that by definition, you don't really know where it's going to go and you don't know what its full capacity is going to be. It is absolutely useful to be thinking about these scenarios. It's more useful to think about them grounded in where the technology actually is and like what, you know, the next step is and the step beyond that. I think we're still many steps away from like the thing that people are worried about. And you would have to deliberately, as the developer of these platforms, go seek out some of this stuff.

And there are people who disagree with me even on that assertion. They think that there's going to be some uncontrollable emergent behavior that happens. And we're careful enough about that where we have research teams thinking about the possibility of these emergent scenarios. But the thing that you would really have to have in order for some of the weird things to happen that people are concerned about is real autonomy. So, a system that could participate in its own development and have that, and like have that feedback loop where you could get some superhuman fast rate of improvement. And that's not the way the systems work right now, not the ones that we're building, at least. So, you're assuring me that you have things under control. I'm going to take your word for it. We do. And look, you shouldn't, I mean,

this is one of the things, like we're going to figure out over the next handful of years, like, where governments and society want to, like, be able to look over the shoulders of the people developing this technology to make sure that everything is being done responsibly. And I think that's a reasonable thing for societies and governments to ask for. I don't feel particularly confident that government's going to be particularly good about controlling tech. There's the sort of lock it down. If we look at how governments around the world, including the U.S., are dealing with just social media tech, which is pretty simple, all things considered, they seem kind of baffled by it and they can shut it down or they throw their hands up. And this is much more complicated. Well, look, so I think it is and it isn't.

Like the interesting thing about social networks is like they, you know, sort of get right to this very complicated thing about like what happens when very large numbers of people are interacting with one another in a technology facilitated way. And like, you know, maybe you could argue that that technology is not as complicated as these AI things, but like that dynamic is maybe even more complicated than what we're talking about here. Let me, let me ground things in the here and now. What's the business model for Microsoft with AI? Who's paying for this? Because as we all learned over the years, even when you're using a free service, someone's paying for it. And that shapes the way the product is made. Yeah, I think just like all of these other technology shifts that we've seen over the past 50 years, we will figure the business model out as the technology matures. So the obvious ways that we're paying for it right now are sort of three-way.

So for Bing, it gets paid for with advertisements. And so the ad model itself is likely going to change over time. But right now, it's a search engine, it's free, and it's ad-supported. For things like ChatGPT, it's sort of a freemium model. So you have a free tier that you can sign up for, and that gets you one level of service. And you can pay for ChatGPT Pro, which is a subscription product that gets you another level of service. subscription product that gets you another level of service. And then there is the, you know, can we integrate it with existing products like Microsoft 365 that has like a pricing model and like there, you know, maybe some different way to price it than, you know, like an E5 subscription

or whatnot, which is what we do right now for big businesses. You've been very responsible in this conversation and you have not been promising me the world. On the other hand, you've been saying this is as big a deal as major technological revolutions you've lived through in your life. What are you most excited about when you're playing with this stuff? Yeah, so like I'll give you a couple of examples.

So my brother who lives in rural Central Virginia with my 74-year-old mother, is immune compromised and got COVID last fall for the first time. And, you know, in rural Central Virginia, he doesn't have access to the same quality of healthcare that I have access to here in Silicon Valley. And the responses that he got to his doctor about what he should do after having tested COVID positive just weren't great. And I just sort of think about the possibility for these AI systems to help people get access to high quality of medical care, like high quality of medical care, like high quality of educational resources, high quality of like a bunch of things

that you think of as skilled guidance that you get from people. Like I think these systems can make those sorts of things accessible to folks for whom it's not accessible right now. And like, I think that's a really exciting thing to me. So that's a good place to maybe wrap this up because it's one of the big questions and debates that I've found here is that scenario you've described. You hear

people who are excited about AI saying we could extend knowledge and resources to people who wouldn't have it normally, whether it's medical or legal, all kinds of stuff that you had to be rich or live in a certain place or know someone to get to, and we can extend that out. And then other people say, you're describing a dystopian scenario where the rich people get real human doctors and everyone else has to work through a bot. That sounds terrible. So you've sort of described the upside of your vision of it, but help convince me and others that this is a better solution than going to my GP and asking him for Paxlovid. Well, I think it just sort of depends on what the actual delivery mechanism is.

What I would say is you absolutely don't want a world where all you have is some substandard piece of software delivery mechanism. So, like, what I would say is, like, you absolutely don't want a world where all you have is some substandard piece of software and no access to a real doctor. But so, I have a concierge doctor, for instance, and, like, I interact with my concierge doctor mostly by email. And so, like, that's actually a great user experience. Like, it's phenomenal. like that's actually a great user experience. Like it's phenomenal. It saves me so much time and I'm able to get access to a whole bunch of things that just my busy schedule

wouldn't let me have access to otherwise. So I have thought for years, wouldn't it be fantastic for everyone to have the same thing? Like a expert medical guru that you can go to that can help you navigate a very complicated system of insurance companies and medical providers and whatnot. And yeah, sure, you want to be able to get access to human beings at the right point, but having something that can help you deal with the complexity of a very complex system, like, I think is a good thing. When the first iPhone came out, and this was my last question, there were people lined up to buy it.

And then I would see these people in New York. They'd walk around showing the iPhone off at parties. And people go, oh, my God, let me check out that iPhone. And pinching and zooming. And it was really great. X number of years now, no one ever looks at you and wonder when you pull a supercomputer out of your pants pocket. Is that the track we're on here for AI, where it becomes embedded and we don't think about it and it's fully part of our lives and we take it for granted? Well, look, I think it's either going to be that or it's going to be useless.

Well, look, I think it's either going to be that or it's going to be useless. So, like, every major transformative technology thing in human history gets past this sort of hump of adoption where, like, there's lots of skepticism and, like, people don't really know what it's good for and, like, enters the domain of ubiquitous. So either this is like one of those major platform things, in which case at some point it will be absolutely ubiquitous and we will completely take it for granted, or it just isn't. And, you know, like folks who are excited about it are, you know, like in some kind of crazy hype cycle and it's going to flame out and we will see which one it is over the coming years. In a moment, we're going to hear from someone who says Kevin Scott and most of the other AI optimists are wrong. Fox Creative. This is advertiser content from Zelle. When you picture an online scammer, what do you see?

For the longest time, we have these images of somebody sitting crouched over their computer with a hoodie on, just kind of typing away in the middle of the night. And honestly, that's not what it is anymore. That's Ian Mitchell, a banker turned fraud fighter. These days, online scams look more like crime syndicates than individual con artists, and they're making bank. Last year, scammers made off with more than $10 billion. It's mind-blowing to see the kind of infrastructure that's been built to facilitate scamming at scale. There are hundreds, if not thousands, of scam centers all around the world.

These are very savvy business people. These are organized criminal rings. And so once we understand the magnitude of this problem, we can protect people better. One challenge that fraud fighters like Ian face is that scam victims sometimes feel too ashamed to discuss what happened to them. But Ian says one of our best defenses is simple. We need to talk to each other.

We need to have those awkward conversations around what do you do if you have text messages you don't recognize? What do you do if you start getting asked to send information that's more sensitive? Even my own father fell victim to a, thank goodness, a smaller dollar scam, but he fell victim and we have these conversations all the time. So we are all at risk and we all need to work together to protect each other. Learn more about how to protect yourself at vox.com slash zelle. And when using digital payment platforms, remember to only send money to people you know and trust. Thumbtack presents the ins and outs of caring for your home. Remember to only send when to do it,

and who to hire. Start caring for your home with confidence. Download Thumbtack today. And now, an anti-hyper. My next guest believes in AI, but not the way it's going right now. His name is Gary Marcus, and he's a cognitive scientist. He's the author of Rebooting AI, and he's got a new podcast coming out called Humans vs. Machines. Marcus believes that things that companies like Microsoft and OpenAI are building aren't real artificial intelligence, but what they are is deeply flawed tech. You know, a lot of us dream of artificial general intelligence, a system that you could throw any problem at, kind of like the Star Trek computer. You say, solve this for me. The Star Trek computer could play chess, but it could also answer questions about dilithium crystals or whatever it is they needed at that time.

GPT-4 gives the illusion of doing that, but it's not actually that great at it. It can talk about anything, but it's not reliable. Some people say it lies. Literally, it doesn't lie because it has no intent, but it makes stuff up fairly regularly, and so you can't really trust its output. For GPT-4, you have to use just the right prompt. If your prompt is slightly off, then you get a weird answer. This is not something that happened with the Star Trek

computer. I sort of started having this fantasy, like the Enterprise is about to crash, and Captain Kirk says to Scotty, you know, we need to fix this problem. And Scotty is like, well, I just can't get the right prompt to work. And then, you know, maybe he gets the right prompt or maybe he gets the wrong prompt and the enterprise crashes. We don't have reliability here with these systems. They're a hint at what artificial general intelligence would look like. They don't work that well. Even if this is not the supercomputer that we're promised in science fiction, there are a lot of pretty rational people, some of whom have a vested interest in this, but others who don't, who say this seems like a really big moment. We just heard from Kevin Scott, the CTO of Microsoft. He does have a vested interest in this, and he described this

moment as being something akin to the development of the personal computer, the development of the internet, the development of the iPhone, maybe all three of those bundled together. And he is not promising that this is a transporter beam or it's going to manufacture food for us, but he does think that it's a really big deal. Is there a sliding scale where we can say, look, this is not science fiction fantasy, but it is a big deal. It is interesting and it is going to help us in some way. Is that more reasonable for you? I mean, it's absolutely interesting. I would not want to argue against that for a moment. I think of it as a dress rehearsal for artificial general intelligence, which we will get to someday.

But I think right now we have a trade-off. There's some positives about these systems. You can use them to write things for you. And there's some negatives. So this technology can be used, for example, to spread misinformation. And to do that at a scale that we've never seen before, which may be dangerous, might undermine democracy. And I would say that these systems aren't very well controllable. I think of them as like bulls in a china shop. They're powerful. they're reckless,

but they don't necessarily do what we want. And ultimately, there's going to be a question kind of akin to the driverless cars, which is, okay, we can build a demo here. Can we build a product that we can actually use? And what is that product? And I think in some places, people will adopt this stuff. They'll be perfectly happy with the output.

In other places, there's a real problem. So in search engines, for example, people are trying to replace traditional search engines with what you might call chat search, like Bing. And, you know, it kind of works and it kind of doesn't. The most recent version that I saw is BARD. Somebody put in my name and had it write a biography. BARD is Google's version of Bing plus AI. And there's some differences between the two. But what Bard did, and we've seen this with Bing,

is it confabulated. So it got the subtitle of my book wrong. It fabricated two quotes. It said that I made arguments there that I didn't because the book was published in 2019 before large language models were popular. And so it kind of like gave a flavor of a Gary Marcus argument, but it got all the details wrong. I don't really want my search engine to do that.

When I use a search engine, like when I'm writing a book, I want the facts, not like a flavor of the facts. So if, you know, we've gone, we've had this discussion about driverless cars in the past, this discussion about driverless cars in the past, and you've got people like Elon Musk saying, sure, the driverless Tesla is going to hit something occasionally, and maybe someone might even die. But people die in cars all the time. Human drivers are terrible. The computer is going to be better than X percent of them, and that's a worthwhile trade-off. Most people aren't comfortable with that analogy. but I'm also thinking of earlier versions of technology being directionally correct, but not right at the beginning, and people are comfortable with that, and it gets better. I remember the early Wikipedia,

when that was brought up as an idea that it could replace an encyclopedia, and that seemed ridiculous, and the initial Wikipedia results were often very bad because they're biased, et cetera. And now Wikipedia is still flawed in lots of ways, but is generally a pretty good resource. Are we on that kind of trajectory with this kind of tech? I think we might be, but I think there's a critical question of time and cost. So costs come in different dimensions. So with driverless cars, I warned in 2016 that there was an outlier problem

and that it might not be practical to actually use them anytime soon. And not long after that, we were told, hey, we're going to have a driverless car go from Los Angeles to New York. Elon Musk said that in 2016, promising it in 17, if I recall correctly. And we've gotten these promises pretty much every year since, that they're imminent. And the reality is they're still demos. They're not really products. So yes, you can take some rides in San Francisco, for example,

but it has to be at certain hours and certain roads. You can't take it wherever you want. And so it's been many years of promise not really realized. We could wind up in the same place with chat search. I mean, people are certainly running demos right now, but whether we can count on it, how long it's going to take for us to count on it, I don't know. I think driverless cars will eventually replace humans. I think we will reach a point where driverless cars are in fact better than people, but I don't think we're particularly close to that. It might be another decade or two. And so a question is whether chat search is more like driverless cars, where we see a taste of it, but takes a long time to deliver, or are they more like Wikipedia, which I think is actually pretty practical right now.

want to push a little bit on that. If we say, look, we don't want the AI to drive us from Los Angeles to New York or really anywhere because we're just not comfortable with it and we're not comfortable with it being wrong. But if it gives you search answers that are usually correct or mostly correct and you have to do a little bit of extra work to make sure it's entirely correct, you're not going to kill anyone if this gets wrong. Well, if it's medical misinformation, you might actually kill someone. And that's actually the domain where I'm most worried about erroneous information from search engines. People do search for medical stuff all the time,

and these systems are not going to understand drug interactions. They're probably not going to understand particular people's circumstances. And I suspect that there will actually be some pretty bad advice. Now, they may try to lock these systems down so they don't give medical advice, but then people may go back to Google. And so it might not be Microsoft's great desire to do that. So I mean, there's a lot of commercial questions and user questions we don't really understand.

So we understand from a technical perspective why these systems hallucinate. And I can tell you that they will hallucinate in the medical domain. Then the question is like, what becomes of that? What's the cost of error? How widespread is that? How do users respond? We don't know all those answers yet. I want to ask you to define some stuff. Let's start with hallucinate. It means something kind of akin to what we think of when it comes to AI, but something a little bit different. What does that mean? So what we mean by that is that these systems, large language models is what we're talking about, have a tendency to make stuff up. They don't have a built-in system for validating things,

although people are trying to put extra machinery on top. But inherently, they don't actually track facts about the world. They don't build mental models of the world. They don't build mental models of medicine, of physics, et cetera. They just really are autocomplete on steroids. They predict text, what might come next in this circumstance. And they do that over clusters of words. So they're like, this cluster of words predicts this other cluster of words. It's not even conceptual. It's really relationships between words. And so you get really weird things coming out of it. So if you actually look as recently as November of 2022, Facebook had a system or Meta had a system called Galactica, and it made up some really wild stuff. It, for example, said on March 18th of 2018, Elon Musk died in a car crash. And he didn't actually die in a car crash. It's not actually in the data that the system was

trained on. There's lots of data that contradicts that. So that's what we call a hallucination. It makes stuff that is plausible, but it doesn't know the difference between things that are true and not. And it takes months to train these things. GPT-3 was trained only on 2021. It didn't know what happened in 2022. Didn't know that Elon Musk was the CEO of Twitter, for example. told me that Twitter was a public company, it wasn't private, Elon Musk wasn't the CEO, basically said that I made all this stuff up, which I hadn't, right? We sometimes call that gaslighting, right? So if you use a pure language model, it simply won't be up to date. It sort of strikes me as, and I've been in this position in the past at various grades, where you're presenting a talk or writing a paper paper and you use lots of words that have come up in class and because they you think they sound right and you're hoping that no one's going to

sort of pay too close of attention and it might get you through the quiz yeah I mean gpt3 is a lot like that it doesn't have the intention it's not trying to bluff you it's not trying to please you but it is at some level bluffing because it doesn't really understand what it's talking about. I mean, that's kind of being anthropomorphic about it. I mean, really, it's just predicting words. You wrote an essay in the spring of 2022, about a year ago, called Deep Learning is Hitting a Wall. Can you tell us what deep learning is and why you thought it was hitting a wall? Deep learning is the major technique in machine learning, which is part of artificial intelligence. It's getting used all the time. It underlies large language models. We use these things for image recognition. They're in image synthesis. They're

in these chatbots now. They're very widespread. And what I said in this essay, which was in March of 2022, was that they were hitting a wall. I didn't say there's not any progress anymore, but I said they have particular problems. I said that they're not reliable, they're not truthful, they confabulate things, and they have a tendency to produce misinformation. And people got really, really upset that I should dare to say that their baby was hitting a wall. There was a whole satirical campaign almost of memes to try to attack me. And what's amazing to me about it is that it didn't take that long before a lot of people changed their views. Sam Altman, he's the CEO of OpenAI, he said, give me the confidence

of the mediocre deep learning skeptic. And then, you know, in December, he put out his own tweet that said, these systems still have a lot of work to do. We can't really trust them. They have problems with truth, with reliability, exactly the same things that I had said in that article. So I think in a way, the field has grown up. And in a way, like there's something deeply problematic about a field that so tries to

silence its critics that there's like this, you know, relentless campaign against them. And it's also a sign of like, if you're going to do science, you have to actually listen to people who have opposing views. If you try to silence them, you're not going to get to doing good science faster. In the fall of that year, there's this just wave of excitement around AI, largely from what I can tell, because it was now sort of something that regular people could now see in the case of images like Dolly or image generators like Dolly and then chat GPT that you could play with without having to know how to use anything other than how to turn a browser on. Was there any technological change that also spurred that on? Or is that, or sort of, if we look in the guts of what's happening there, is that the same conditions that you were talking about in March? And it's just the fact that it's now accessible to people that has generated that excitement. There's a few things. So the technology in it is not that different from its predecessors, but they added these guardrails. You could still so-called jail break it and make it say really terrible things without that much trouble. You could use it to

generate misinformation. But they had these guardrails that were good enough, even though they weren't great, that people could just try it out. And that made a big difference. And Even OpenAI didn't expect it to go viral the way that it did, but it did in part because it was so much fun to play with. You're saying, if I'm getting this right, that the folks at OpenAI took tech that was basically preexisting and that they already had, made it more accessible to humans, added software to make it less prone to error, and people got very excited about it.

This seems like the good version of tech progress, where you take a thing and you modify it and it becomes better and useful. They made it a better product. I mean, so Jan LeCun has pointed out, and other people, also Yashua Bengio, these are two of the pioneers in deep learning, have pointed out, I think rightly, that it wasn't big technical progress. It was good product progress. So it wasn't some new form of AI that we didn't have before. But yes, it was a better product. It was easier to use. And some of it was just like making it cute. Like it types out words

one by one instead of giving you the whole thing, somehow that makes it feel more human. So like from a product side, it was definitely a better product than we had before. From a technical side, those of us who are interested in so-called artificial general intelligence and more generally reliable and trustworthy artificial intelligence are like, yeah, we still have the same problems that we did before. It's just more fun to play with. So, I mean, there's a tension between like, what do people actually want to use? And it's better at that. And on the other hand, like, how do we make this reliable so that if you type in your medical questions, you can actually trust the answer? I have a broad and shallow view of technological progress, but it essentially goes like things get better over time. You have unexpected results. You don't always know where it's going to go, but generally things get better, faster, cheaper,

smarter. We know about Moore's law and chips used to get faster all the time and that sort of slowed down. And what you hear from AI proponents is these neural nets and other things that are the engines for this are getting better all the time. We're adding more data, we're computing more, we're learning more. Why won't that work here? They are and they aren't, is the first thing I'll say. And then I like to give you a historical parallel. So on misinformation, for example, they're not really getting better. On truth, they're not really getting better. A couple of days ago, NewsGuard released something showing that GPT-4 was actually worse than GPT-3.5 on discriminating truth. So they are getting better at being more plausible. They're not getting better at being

more truthful. And that really matters a lot. So that's the first thing I would say. The second thing is there are a lot of interesting historical cases where people got obsessed about a particular technology and chose wrongly. And it took a little while to back down from the popular thing to the thing we needed to do. So dirigibles were really popular in the 1920s and 1930s. Until we had the Hindenburg, everybody thought that all these people doing heavier than air flight were just wasting their time. They were like, look at our dirigibles. They scale a lot faster. You know, we built a small one, then we built a bigger one, then we built a much bigger one. It's all working great. And those poor guys with the heavier than airplanes are still stuck, you know, barely bigger than the Wright brothers. So, you know, sometimes you scale the wrong thing. In my view, we're scaling the wrong thing right now. We're scaling a technology that is inherently unstable, unreliable, and untruthful. We're making it faster, has more coverage, but is still unreliable, still not truthful. And for many applications, that's a problem. There are some for which it's

not, right? GPT's sweet spot has always been making surrealist pros. It is now better at making surrealist pros than it was before. If that's your use case, it's fine. I have no problem with it. But if your use case is something where there's a cost of error, where you do need to be truthful and trustworthy, then that is a problem. Because I think the way a lot of people look at it is, yes, they're doing surrealist pros today. And tomorrow or six months or a year from now, we can't project when, it's going to get much better at much more sophisticated stuff. Well, I mean, this is where I'd say people have been telling me that for 30 years. I first pointed out the hallucination problem in 2001. And so far, nobody has solved it.

Even though you can scale some things, you're not going to scale truth. And in fact, the NewsGuard results shows that they're not scaling truth. So to me, it feels like appeal to hopeful monsters. And let me pause here. It's really important to distinguish between what we can do with large language models and what AI might someday be able to do. So I am not making an argument

that we will never be able to build an artificial intelligence system that we can trust, that is reliable, that can keep track of truth. I'm saying this is the wrong technology for that problem. And until we broaden our search through the space of hypotheses, that we're a little bit stuck. So this is not artificial general intelligence. That's something you remain interested in and think could arrive one day. But we have this

technology. It is flawed in lots of ways. Let's stipulate that. It still seems like it ought to be useful for lots of present-day applications. Are there things that you are interested in using the current version of this technology for? I mean, me personally, no. Other people have to write much more boilerplate stuff than I do, and for them, it might make some sense. I would warn them about accuracy and things slipping through because our human tendency is to take things that are grammatical and well-formed and assume they're okay. And you may have seen how CNET got themselves in trouble.

They published 70 articles written by machine. It turned out 40 of them had mistakes in them. It's very hard for people to pay enough attention. We have a similar problem with driverless cars, which is a human sits in a driverless car for like an hour and thinks, all right, this is fantastic, and doesn't realize that the machine doesn't think like they do. And the machine, for example, could run into a jet airplane because it's not in the training set. And so these systems are not as reliable as they might seem in the first few minutes that we play with them. Fundamentally, the outputs just are not 100% reliable.

And so you have to decide, is that okay for my task? Do you think this technology ends up replacing people, replacing workers, because it can do it better, faster, cheaper, more efficiently, or it is a tool that people who know how to use this work use in their day-to-day? I suspect it'll be some of both. We don't really know for sure yet. We may find that where you have 10 people on a job, now you can do it with two, and that the two kind of oversee the systems and make sure there aren't mistakes.

We may see a bunch of that. It's a little bit too early to say for sure. There are other things that we haven't talked about as much, the generative systems that create images. And there are limits there. So it may not always give you exactly what you want. It may be hard to edit to get the thing that you want. But if you just need a rough thing, let's say for an illustration for a talk, they work pretty well. So I think they're going to cut there into the jobs for commercial artists by a lot. So I do expect this stuff will have impact on employment. I think it's as yet a little bit hazy about what that

impact is going to be. It may be small, it may be dramatic. I think it's going to matter domain by domain. So another place where we've kind of seen this movie before, but might have a different ending was with radiology In 2016, Jeff Hinton said that you might as well stop training radiologists. That's almost a verbatim quote. And here we are in 2023, we don't have enough radiologists because it turns out that the job of radiology is not just to look at an image, which deep learning is pretty good at, but also to look at a patient's file and understand the relation between that image and a person as a whole and diseases.

And you need to know about anatomy and physiology. And it's going to depend in every profession. Like, can you do a certain piece of it? If you can do a certain piece of it, does that mean maybe we can reduce the staff by 10%? There are a whole lot of questions that really depend on the details of what the job is and the cost of error and all kinds of different things. I don't know if there's

like a blanket answer there. I think there's going to be a lot of discovery process in the next few months. Everybody is interested in this question. Is there a rubric that you can share with us that's useful for an interested layperson to sort of say, all right, this person is telling you about this exciting new AI, whatever, inflected thing they're rolling out, how could a regular person who has no capacity to look into the code, et cetera, understand whether the thing they're getting is actual AI, whether it's fake AI, or it's just hype entirely? So, I mean, look, how you define AI is actually up for grabs. There's no fact of the matter kind of in the universe, independent people. We use the terms as we do. There is no

artificial general intelligence right now that just doesn't exist yet. We have AI for certain problems that works great. So, route planning, use your GPS to go from A to B, works great. That's AI. It's not large language models, not very sexy anymore your GPS to go from A to B works great. That's AI. It's not large language models, not very sexy anymore because we've habituated to it. But it's actually kind of amazing that that can be done. And, you know, I'll stick on a phone. Chatbots are real.

They're just not reliable. They can talk about anything, but they're often out of their depth. So you give them a math problem and sometimes they get it right and sometimes they get it wrong. You give them almost anything in any domain and sometimes they get it right, sometimes they make up stuff. Most of the things we're talking about right now are based on that. And if you want to ask, like, am I in trouble because of that? You have to ask questions like, how well can it do the thing that you do in your job? How much creativity do you need to do in your job? How systematic is it? How much does it cost if you make a mistake? Who's going to catch the mistakes if the AI does it?

Is that even feasible? For Rubik, there are a whole bunch of different questions that you might ask around that. How often do you come up with some unusual case that a system that only kind of knows the usual cases might fail on? And it's going to be different for every profession. Gary, this has been great. Thank you for your time thanks a lot thanks again to Joshua Browder Kevin Scott and Gary Marcus next week AI has a lot of critics but there are a lot more people in Silicon Valley happy to ignore them because they want to cash in. So next up on our AI tour, we're going inside the gold rush.

This is a special series, so we got extra help. None of them are robots. They're all humans. They're excellent humans. Thanks to Matt Frasica for producing this episode. Megan Cunane is our editor. Jelani Carter is our engineer, and Brandon McFarland scored and mixed this episode. We got brand new human-made music for this. We'll see you next week. Thanks for listening. You can subscribe to Recode Media wherever you listen to podcasts.

We'll be back on Friday with more Pivot. Unless the AI takeover starts early. I did not write that.