Plain English with Derek Thompson - What Happens When AI Learns to Do Our Jobs

Episode Date: October 24, 2025

Today’s guest is Ethan Mollick. Ethan is a professor of management at Wharton, where he specializes in entrepreneurship and innovation. He is the author of the book 'Co-Intelligence: Living and Working With AI,' and his Substack, One Useful Thing, is the single most useful guide I have ever found to make sense of these tools and use them productively. But he’s also a deep thinker of the Alfred Chandler school of big ideas who wants to not only help individuals use the technology more efficiently but also understand what happens as tens of millions and billions of people use the technology to make themselves more productive or even, at times, obsolete. Host: Derek ThompsonGuest: Ethan MollickProducers: Devon Baroldi and Devon Renaldo Learn more about your ad choices. Visit podcastchoices.com/adchoices

As the 21st century was getting underway, Hollywood released a series of films that were daring, entertaining, and absolutely unmissable. Films like, 25th Hour, Bring It On, Zodiac, and No Country for Old Men. They arrived during the George W. Bush era, a chaotic time in America. Think 9-11, Katrina, the mortgage crisis. After the Bush years, the country would never be the same, and neither would Hollywood. I'm Brian Raftery, and in my new limited series, Mission Accomplished, we're going to dive into some of the biggest movies of the bush years. And look at what they said about the state of the nation. We'll go behind the scenes with filmmakers and experts and relive some of your favorite movies from the early 2000s.

From Donnie Darko to Michael Clayton, from Anchorman to Iron Man. So slip on your sketchers, dig out your old Nokia, and join me from Mission Accomplished, starting August 12 on the Big Picture Feet. Today, AI. and work. In the year 1800, it took about six weeks to travel from New York City to Chicago. You were probably going by horse and carriage and the roads were correct. So if you wanted one summary of what the canals and railroads did to America, you could say this. They shrank the country. One implication of a smaller country is that it was easier for companies based in New York to do business in the Midwest.

Just as the train made it possible to move goods and people across the country at faster speeds, The Telegraph accelerated the speed of information, and together the Telegraph and the train collapsed space and time for American commerce. But trains did something more. They didn't just make it faster for companies to do business. They changed the sort of business that companies could do. In 1977, the business historian Alfred Chandler

made this point in a controversial but I think mostly wonderful book called The Visible Hand, the Managerial Revolution in American Business. Chandler pointed out that before the train, the telegraph, most businesses in America and throughout the world were small, local, and personal. A single owner might oversee every step of production and sale. But when goods and messages could suddenly travel hundreds of miles in a day, scale exploded. And with scale came complexity.

The owner couldn't do everything. He needed help. He needed dispatchers, accountants, foremen, supervisors. He needed managers. So the railroad and the telegraph did not just make old work faster. They created an entirely new kind of work that didn't previously exist. The work of management. And these managers shaped business and capitalism just as much as the forces of supply and demand. They were the visible hand to go along with Adam Smith's invisible hand. If you're a white-collar worker in an office and you have a regional or department manager, or if you are a regional or department manager, or if you are both a manager and have a manager,

you are ensconced in a system made possible by, invented by the railroad and the telegraph. I've been thinking a lot about Alfred Chandler's thesis recently. because it contains an important lesson for our own time. We tend to assume that inventions mostly automate existing tasks. The typewriter makes writing easier. The spreadsheet makes accounting easier. Or chat GPT just makes writing faster. But Chandler reminds us that general purpose technologies often do something more profound.

They reorganize the architecture of work itself. And that brings us to AI. We're at a moment right now that I think rhymes with Chandler's 19th century revolution. AI can make existing tasks faster, right? It can write memos, summarize meetings, answer emails. Fine, good, wonderful. What's less clear to me, though, is how to complete the following analogy. As railroads led to managerial capitalism, AI will lead to blank.

What exactly? When the Alfred Chandler of the 2030s looks back to explain how artificial intelligence changed work, what book will he or she write? What is AI's visible hand? Now, let me interrupt this open by appealing directly to a part of my audience. I know that some of you folks do not like AI.

Don't trust AI. Don't know what it can do for you. Don't believe perhaps that it can do much of anything. This is one area where you the analogy between the railroads and AI completely breaks down. Everybody looking at a train knows what the train does. You put the stuff on it, and it moves the stuff at a quantifiable speed across land. That's trains.

Generative AI has no fixed observable quality like this, because it responds personally and privately to prompts. And that means that its potential as a general purpose technology is often silo. and secret. Some people use AI constantly to code and to write and to research, and others just don't know how to use it at all. Today's guest is Ethan Mollick. Ethan is a professor of management at Wharton, where he specializes in entrepreneurship and innovation. He's the author of the book Co-intelligence, living and working with AI, and his substack, one useful thing, is properly named the single most useful guide I have ever found to make

sense of AI tools and how to use them productively. But Ethan's also a deep thinker of the Alfred Chandler School of Big Ideas. He wants to not just help individuals use this technology more efficiently. He wants to understand, I think, at a bigger picture, what's going to happen to us? What's going to happen to our economy? As tens of millions, even billions of people use technology to make themselves more productive and at times even obsolete. I'm Derek Thompson.

This is Plain Hitch. Ethan Mollock, welcome to the show. Thank you for having me. You say the intelligence of AI is best understood as a jagged frontier. What does that mean? So what makes AI kind of weird is it does a lot of things, like many, many things.

And some of the things it does well, some of the things it does badly, and it's very hard to predict what those things are in advance until you start using it. So we have this mental image in our mind of this sort of jagged wall, things are just beyond the wall of the AI can't do. Some things are inside the wall of the AI

could do it. And it's really hard to know what those things are going to be or what they're not going to be. And that can be very frustrating as a user. I think the concept of jaggedness is a really great one because it resolves, as you indicated, a really common question that I get from people who are less familiar with AI, which is often something like, Derek, if AI is so damn smart, why is it so dumb, right? A lot of people hear Open AI or Anthropic or others talking about, you know, this is a tech that could eliminate tens of millions of white-collar jobs in the next few years. And meanwhile, you go online and you tell one of these advanced tools to make a friggin' clock striking 9.30 p.m., and it cannot do it. You can ask over and over again, it can't make the clock strike 930.

And so I like this idea of saying, we don't need to declare AI a full genius or a full idiot. It is more sophisticatedly true to say that its capacities are jagged. It is very smart at some things and very stupid in others. How has the frontier, Ethan, gotten less jagged over the last few years and months? So there's really two ways that this has happened. One is AI's got it broadly more capable due to a couple different breakthroughs. So it's a lot stronger than it used to be in lots of different ways. And that has enabled a lot of use cases that didn't exist before.

So hallucinations, the kind of errors that we see, they've dropped, quite a bit, which lets AI do longer tasks than it did before. So that's one area. It's like it just gotten broadly better, which enables a lot of different things. So the frontiers moved out. And then the second big thing that's happened is some of the most jagged parts of the frontier. Like if we talked a year ago right now, we would talk about how AI is terrible at math. And that was the big joke.

You couldn't count the number of ours in Strawberry. There have been breakthroughs in the areas where the frontiers were at its worst. And now it's winning at the International Math Olympiad, for example. So both things have happened. The researchers in the AI lives have focused on the areas where AI was weakest and made it stronger. And AI has broadly gotten better. That doesn't mean there's still not jaggedness there, but it's far beyond where it was, say, a year ago. What is AI bad at right now?

Like, what is the canonical example, the same way that maybe 18 months ago, it was, well, it can't name the number of ours and strawberry or, you know, it can't show someone slicing a pizza. Is there a similarly broad or familiar way to describe the deficits of the frontier models right now? So that's kind of interesting. It's a part of a broader question about how to even measure success. I don't think there is one canonical example anymore. The strawberry example, you know, a couple of brain teasers, people used to use that. There are problems with some of parts of AI lag other parts. So, for example, image generation models have these notorious weak spots, right?

of, for example, clocks, right? Every clock in every advertisement is turned to, I think, 10-10 as the time, if I remember correctly. And so it's really hard to get AI clocks to show something else other than that. So image models are full glass of wine that's not spilling over. So there are weaknesses in some pieces, but in terms of the core text models that do that kind of thinking as we consider it that produce answers for us, it starts to be much weirder things. For example, if you ask your favorite LLM to show you a seahorse emoji, they will start to freak out for very narrow, weird reasons. But there's not a canonical big example anymore

the way there used to be, like count the number of ours in strawberry. Do you have any kind of big picture theory for why specifically seahorses? I mean, it is so interesting to me where it's jagged, right? Like, why not understand clocks if you can also make beautiful powerpoints? Why not understand seahorses if you can make clownfish? Like, maybe let's just narrow this to seahorses specifically. What do you think is going on there? I mean, we don't know for sure. There's some people doing some exploration of this.

But part of your intuition starts to be what are humans wrong with? If I told you there was a seahorse emoji, you'd probably believe there would be one if I surveyed you. They, like, surveyed your listeners, and there isn't. So that sort of leads the LLM astray here. And it keeps thinking you can produce one and producing logical combinations of, you know, Unicode information that should produce it at Seor's emoji, and that keeps getting frustrated when it doesn't work.

So LLM's weird. They have pre-existing beliefs that can get in the way of conversation. When GBT5, the latest offering from OpenAI, came out, the reaction was incredibly interesting and quite split. Like some people were saying this was a profound disappointment. And others were saying, no, this is one of the most amazing thing that's ever been built for white-collar work. You said in a recent piece that you believe that with GBT5,

AI has crossed an important threshold. What did you mean by that? So part of what makes this difficult, we were talking about what makes AI hard to use, right? It's a Seahorse emoji, the jaggedness. The other part is the labs don't help, right? So GPD-5 is actually a couple of different things in one bucket. And they both sort of went weird, right?

So one of those is it is a family of AI models that range from a pretty dumb model that's fun to talk to, to a model that can actually do PhD problems. not all of them, but a subset at sort of the doctoral, professorial level. And it's also a router, a system that decides, based on your question, which model should get to answer that question. And as a result, talk about jaggedness, if I ask the same question, if the router and its infinite wisdom decided to send me to the dumb model, I could get math errors. But if it's definitely the really smart model, I might have to wait 10 minutes and they get a really smart answer to a question that wasn't quite phrased the right way. So you result through all this really weird sets of interaction.

So everything was true. You've got very bad results and very good results. But if we look at the high-end version of the models, right? The GP5 thinking models are called the pro model. They are substantially better at being able to plan and execute large numbers of steps to solve a problem. Sometimes we call this agendic work. And as a result, you could just ask the idea to do something that does it for you. You don't have to do all the fiddly bits that we used to of phrasing your,

request exactly correctly, and going back and forth the AI constantly, you could just kind of ask for something and it happens. But again, that requires you to either be paying and knowing to use the higher-end models or that the free router in its wisdom decides to send you to the smart models that can do this. You mentioned agentic, and I want to make sure that we spend a good deal of time here talking about agents, because if white-collar AGI arrives, if a superintelligence that can do tens of millions of people's white-collar work, you know, coding and writing and Excel and PowerPoint. My sense is that the people at these frontier labs believe that that future will arrive in the form of an autonomous agent. So just from the beginning, what are AI agents

and why are people so excited about them? Okay. So everything in AI devolves to a marketing term as soon as it's invented. So we should separate out that there's no clear definition of agent and everybody in the planet will sell you an agent. But we have sort of a concept of what it is. So that's an AI system that would give it a goal, will follow that goal on its own and use tools as necessary to accomplish it.

And so what you need to have an agent to be successful is it needs to be able to go a large number of steps, right? If I ask for, to be prepped for this interview, for example, the AI would go to by Gmail and figure out what this interview is going to be. It would research you in your background. It would, you know, prep me on materials. It would go through a whole bunch of different searches. Maybe it would write an infographic for me.

So it might do like a thousand independent steps and give me the result. And I think the fear and concept from a lot of us was that this was very difficult because AI makes mistakes. There are errors that creep into the process. And if you have a long horizon on a task, a thousand steps, there's a really good chance that that error appears somewhere in those thousand steps that breaks the AI. We were wrong about that, it turns out. It turns out that AI is self-correcting enough, that when it makes an error, it doesn't just keep running with that error.

It starts to realize something's wrong and fixes it, not all the time, but enough of the time that we've gone from models being able to do maybe 20 or 30 steps independently to GPD5 could do over a thousand steps independently. And that changes what an AI can do. And so we talk about agents, an agent is really just, I want to assign an AI a task. It does it on its own without me having to intervene, which is a little bit different than the co-intelligence model that talking about my book, which is work back and forth with the AI. And so that is sort of what the labs have all been aiming for. And it went from AI can't really do this to suddenly, this is a real possibility. I have to imagine that agents have a jagged intelligence as well. Is there something that agents are really good at doing, say taking a corporate memo and turning it into a PowerPoint,

versus something that's a very common white-collar task that for whatever reason may be based on the number of steps or how long it typically takes someone to do, the agents are not quite as good at doing that. So I would say we're still over the very early days of agents, general purpose agents that go out and do anything, right? And you could use these right now. If you open AI has an agent mode and you could say, go buy me snow tires for my, you know, for my 2024 Corolla. And it will go out and research websites and read them and do it. But they have problems. They have errors and mistakes. So general purpose agents, it's not even like one jagged thing. It's not strawberries and ours. There's many, many things they can't do. The goal of these agents, though, is to be able to use them anything that a

human can do with the computer, the AI can do. And that they started, and their strongest area still, is easily verifiable output. So computer coding is by far their strongest case, because if the code doesn't run, the AI can fix the problem. Increasingly now they've got a good at PowerPoint Excel, which have sort of similar characteristics. I can verify as an agent that the output, looks right or that the Excel formula works. As things get fuzzier, the agents start to have more issues because we can't, as easily rate, is this a really good memo or not? And so it's easier to have errors and mistakes sort of compound. Doesn't mean they're not useful, but it's less understood about where they're good or bad. One thing that I've said before is that I think AI is very good at answering questions,

but not so good at telling me when the questions I'm asking are wrong. And a really important part of, I think, a good worker, maybe especially a knowledge worker, is having good taste and questions. Like, if, you know, I'm a journalist and I go to my editor and I say, here's my piece, can you copy edit it. Well, that's something that Chat, GBT, you know, GBT5 can do quite well, right? Finding grammar errors, finding typos, and editing them in line and bold. That's easy. But going back to me and saying, Derek, you didn't do enough reporting on part three of this essay.

or Derek, I think this entire essay is actually wrong based on this other piece of research that I went and found. Those are the kind of replies you're likely to get from a human editor who sees their task, their job, as being not just answering the question that they're getting from their journalist, but also posing new questions back to that journalist. Do you have such a lovely and clear way of seeing this. Do you have a good way of seeing the ways that agents aren't yet? yet sophisticated enough to work with their humans in order to sort of ask questions back? So I'm going to agree with you strongly on the general principle and talk about it and disagree with you in your instance. So on the writing side, I mean, what they were talking is proactivity, right? The AIs are not proactive, right? And they're also sycophantic. They tend to want to make

you feel good for a variety of complicated reasons, some nefarious, some much less so. And as a result, they're often bad at like pushing you, right? They'll agree. with you, they'll redirect you, that's improving a little. But I would argue that some of this is about prompting. I mean, some of this is about the questions you ask. So you started with the good questions. For example, I do a lot of writing, much like you. I always write material on my own first because I think my own writing matters and my own thoughts matter. But I always use AI to get feedback. And often some of the really good feedback I get comes from AI. I have other editors in my life. I find AI to be an incredibly good editor in terms of suggesting what's missing in a story.

So I would wonder if you're asking it, you know, going to a flawed model, you know, which tend to be the best writers nowadays, and, you know, saying, what are I missing here? What would a naive audience member miss? Is the story, like, you have to ask it to be critical. But it's quite good at being critical when you ask. Like, is there a through line here that makes sense? Is there anything unnecessary? And interestingly, by the way, you said grammar corrections are good. I thought the AI actually misses grammar corrections because it's not a detail-oriented piece of work necessarily. And sometimes it'll catch a grammar issue. Sometimes it won't. Sometimes it'll over sort of issues or not. I am such a sloppy writer that any improvement over mine looks like a dramatic improvement. So just seeing that it's found, you know, 1575 errors makes me feel good, like at least it's cleaning up my copy. I love this idea that a part of working well with these models, part of working well with these agents,

is us, the humans, developing a sophistication for, a taste for asking the right questions. I mean, let's just make it really personal. You write essays, you write essays, you write, books. When you're working with Claude or GBT5 to edit your essays, to find holes in your logic, or to make the flow of ideas cleaner for a reader, what are the kind of questions that you ask to make your writing better? Okay. So we talk about taste, and there's really two kinds of tastes that matter here, or three kinds, right? There is a taste in the good questions. There is a

taste in knowing my own style that matters. And then there's a taste of curation. So a typical way in my handle this would be, you know, first initial kind of thing is, hey, anything spot you is weird here that a naive reader might be confused by. But I, then it go very specific. I can't nail the transition of paragraph one to paragraph two. Give me 37 versions of how I might do that transition. Okay, more like three. Okay, though, that you're being corny. Stop being corny. Go in a different direction. This doesn't sound like me at all. Come on. You could do better than that. So like there's a my taste matters, my taste for the question matters, but also my taste in curation and selection matter in those cases. So I often will do that. I need 75 different ways of closing out this

paragraph. And I will often not use any of the ways it suggests, but it will spark my own writing. So there is this difference of like I'm not interacting with a human editor who has limited patience and I have limited time and get infinite kinds of results. So I do a lot more variation and selection than I would with human work. And less dialogue with the AI about, let's talk about this paragraph more. No, I think more like this, more like that. I do some of that.

But a lot of it is let's expand out the number of options, select them down, expand them out again, until we end up with something we really like. There's two different things we're talking about here, and I want to make sure that they're disentangled. What we were talking about just now, about writing with an AI, is an example of what your book calls co-intelligence.

but the long vision of these autonomous agents is that one day they will become the full Ethan Malik, or more specifically, they will become the full paralegal, they will become the full marketing assistant. If the amount of time and the amount of tasks that they master grows and grows, will if jobs are just a bundle of tasks, they'll eventually click every box in the bundle and do the whole paralegal job, for example. What are the jobs that you think are most? susceptible, our most vulnerable, given the trajectory of agent technology? So just to tie that not all we talked about before, like, I can ask an AI to write an essay for me and do the research for me. And it now does a pretty good fake Ethan essay on, you know,

research and something the AIs are good. That's an autonomous agent task. Does searches the entire web, generates themes, rewrites them, turns them into my style, right? So we can, right now, we were talking about the co-intelligence approach, as you said, agents are quite different. And it's hard to know, right? Because there are some tasks that AI is really good for right now, but we don't have a full range of evaluations. We've just started to get our first evaluations about how good these agents are.

The answer is pretty good. I think single focused tasks that involve collating information and writing about it are probably in the most kind of risk, right? So you're talking about jobs or bundles of tasks. A job is not just one thing. As a professor, I teach and I write, and I do research. I provide emotional support to my students. I do a lot of different things.

If AI does one of those things, like grading, we have some early results showing AI is a better, greater than most humans. I don't use it for grading because my students would get mad, right? So another barrier to sort of adoption in that case. It doesn't mean that my job disappears. So I think if your job is something AI is good at, and you don't have a bundle of tasks, you have one or two tasks that you do, I write press releases.

I think, you know, it's why freelance jobs are under threat to some degree, because a freelance job is an assignable job, that I give somebody that they give me an output for, as opposed to being a journalist where there's actually a ton of different things you do. And if AI freed up some of your time, you'd switch to other things you do as a journalist and do those really well. As you were describing how these agents work, I thought about one tool that I use on Chatibati, which is called Deep Research.

Yesterday, I asked Deep Research to write me a 10,000-word history of the rise and fall of the dot-com bubble, with a particular emphasis in the ways it's like the AI boom, or not like the AI boom. So I use this for my own research, essentially. And because I'm very nervous about hallucinations, I asked for links to every falsifiable claim. And it went off for 10 minutes and came back with a really elegant summary,

exactly what I would hope to get from a day's work from a research assistant or even several days from a research assistant. And it occurs to me that that is also agentic. Like the AI is taking my request, it's making out a kind of outline and adhering to that outline to construct a 10,000 word essay for my benefit.

Is the line between models like chat GPT and agents like crystal clear? Or is it getting blurrier in cases like this? It was clear until recently, and I think it isn't clear now, right? So when I worked with an AI before, I'd ask a question and I'd get an answer. That wasn't an agent, right?

That was an interaction. And there was lots of things that people were speculating about about what agents would look like. We kind of know now. So there are now agents available to you right now. GPT5 thinking is an agent. Claude 4.5 Sonnet is an agent. If you ask them a question,

they go through a planning process where they think about how they want to solve it. They may decide rather than answer your question, they need to do some research work first, or they need to write some computer code, or they create a PowerPoint. They'll do all of that work before returning an answer to you.

So in many ways, that is the beginning of what an agent does. So it's blurred the line between what a chatbot does and what an agent does. If you told me 10 years ago in 2015 that a decade from now, we're going to have a technology that's called an AI agent. And it's essentially going to be able to do an incredibly broad range of white-collar tasks at B-plus A-minus level, whether it's working with Excel or writing a memo or turning a memo into a PowerPoint. I mean, this is your classic meat and potatoes, knowledge economy stuff. I would think, if you told me that, that the companies that adopted this technology

would be suddenly, like, way more productive than the companies that didn't adopt this technology. It's not entirely clear to me, just a guy sitting in a room right now, that there's this incredibly dramatic schism happening in the economy where, like, the handful of companies that understand how agents work are just completely racing ahead of the rest of the economy. Maybe that is happening, and I don't know, but it just doesn't seem like AI is already like supercharging productivity at some places and not supercharging productivity at places that don't know how to use the tech. How do you see this landscape? Like if agents, I guess one way, a simple way to ask the question is, if agents are as good as some people describe them,

why aren't we seeing some companies absolutely race away in terms of productivity? So it's a really interesting question. And anyone who's worked in an organization probably can come with 10 answers off the top of their head that are all correct. Right. So like I think the view of technological sort of optimist, pessimist, the Silicon Valley folks is we have this new technology, the world gets transformed. That is not how technology actually works, right? I think AI adoption is far faster than we expect. And by the way, I do think you're seeing 90 percent. We have evidence that people are seeing very large productivity gates. If you survey people, they say large productivity gates. Those aren't being captured by companies. Well, why not? First of all,

probably some of this is illusionary, right? Like, we haven't figured out how to work with the AI yet. But a lot of people are just hiding their AI use from their companies. So, like, if I have a 10-time performance improvement, I know if I show the company how to do this, first of all, they'll think I'm less smart because the AI is doing the work. They ask me to do more work. They might fire me or my friends because now we have cost savings. And companies love cost savings from IT. So people are hiding it.

But let's say I even am in a company where I'm sharing this stuff. Let's say, which is already happening, I'm now a 10 times more productive coder. What do I do with that? I'm part of an Agile Development Sprint with a 15-person team that has a two-week meeting cadence and stand-up meetings. I get 10 times more work done. What do I do with that? What work should be doing? And so part of the problem is about process, right?

There's this danger that otherwise 10 times more PowerPoints is not going to make a company more successful, I guess, unless your job is making PowerPoints for some reason. And maybe that's what a lot of companies actually do. But I think that part of this is about there's a lot more pieces to just, like, similarly, if I gave any organization a thousand PhD interns, I don't think I would make them more productive overnight. And so I think there's a sort of fallacy of like AI is either good enough and it transforms everything right away or it doesn't work. And it's clearly untrue. I mean, I talk to companies all the time. There's no longer sort of this question does AI provide benefits?

Now it's this hard leadership and process problem of what do I do now that I have 10 times more production in this area. Where do I want to build agents that make a difference? How do they integrate with humans. Those are the really hard problems. In that spirit, I definitely want to move to really practical advice. So, you know, take somebody with a typical white-collar job. They work with Excel. They make PowerPoints. They write memos. And once in a while, they're expected to maybe come up with some new idea, some new way to solve a problem, a new line of business, get a new client. Have you found in your speaking that there's a piece of advice that you can give to somebody in this classic knowledge economy job whose attitude is right now something like,

quote, I just don't see how AI can make me better at work, end quote. Like someone who's at that first step of just, like, forget autonomous agents taking over the economy in the 2030s. I don't know what to do with the suite of technologies that exists right now to make me just a more productive knowledge economy worker. Where would you suggest someone in that position start? So I have really easy advice, which is like the same advice from my book back in the day, which I still think is the right thing.

There is no easy instruction manual. There is no manual for your job. The right thing to do is to use AI for everything you possibly can for about 10 hours in actual work tasks. And you will start to see the shape of the jagged frontier that we started talking about earlier. You'll start to understand what the models are good or bad at. You'll probably be freaked out at some point

because it does things you wouldn't expect. You need to do this with a sort of pro-ish-level model. So that means either a company gives it to you or you're spending 20 bucks a month for usually one of the big three, which is either Gemini or Claude or Open AIs models, chat TPT. And you just need to try it. And you will find tasks that it does for you.

And I think you have to keep pushing for more advanced tasks because otherwise people use AI for things that are time-saving, but not transformational, summarizing meetings or events, helping me write an email. Like, that's an important starting point, but then it starts to get even more interesting. We're like, here's all the context for the media I'm going to have. What are some angles I should approach this with? and how should I think through this. Give me some ideas about how to do this.

Create some mock-ups of this concept I had so we can kind of playtest them back and forth. Some people have different kinds of ways of reacting and interacting with these things. I talked to a Harvard quantum physicist at one point who told me his best ideas come from AI, and this was back well before the models were good at math, and I was like, is it good at quantum physics?

He said, no, no, it's terrible quantum physics, but it's really going to ask me good questions. So maybe you're someone who needs a partner to bat things back and forth with. You have a general purpose tool. You'll only figure out how to use it by using it. One of the thing that you're very good at is finding papers that describe this jagged frontier, the little things that AI is breaking through at.

Just in the last week, you've shared papers showing that retailers can predict purchase intent by asking an AI, a large language model, to pretend to be a specific kind of person and giving it a bunch of product pictures and asking what it wants, which is an amazing use case that I never would have thought of. You've showed that AI can, in a matter of minutes, reproduce to take. statistical results from science papers, which is work that can take professors or journal editors hours to do if they're trying to evaluate whether a new scientific result is valid or not. I wonder, in this sort of landscape of papers being written on discovering what AI can do

that we didn't initially anticipate, what are some of your favorites? There's so many interesting ones. There is a really sketchy paper, and sketchy, not because the research was sketchy, well, but because the methodology was controversial, where a bunch of European researchers put AI agents onto Reddit's Change My Mind Forum, which is their debate forum, without telling anyone. That was the sketchy part. They just put them on there.

They got IRB approval, but the Reddit community hated it. And they found there was the 99th level of persuasion in terms of persuading people on their viewpoint. So, I mean, I think that that's both important finding and an interesting one, right? In fact, the only thing that robustly lowers conspiracy theory belief that we've found and like social scientists have been working this for a long time, is it a three-round discussion with GPD-4? Just like three rounds back and forth, there's a replicated finding.

People believe conspiracy theories less two months later. Pause there. What does that tell us? What does it tell us that we've built an artificial mind that is better at persuading us, surely sometimes of falsehoods, but also it seems sometimes of truths that counteract conspiracy theories?

What is it about the alien intelligence that we've designed that makes it so good at persuading? Well, we actually have some evidence on this, and it turns out it's logic and personalization. The exact same stuff that makes it good at translating stuff from one frame to another, like explain this to me.

It does a good job of referring who you are. And it's not, when you look at this in detail, it's not, by the way, could be, right? So I think we should worry about this a lot. But it's not that the AI is using some nefarious persuasion technique or, you know, social or manipulation. It's that it's explained to you in a way you understand, and patiently listening to you and responding,

which is, I guess, the most hopeful of the persuasion answers. Now, that doesn't mean that it can't be used for much more nefarious purposes and will not be. But in those studies, it really is just like, I can explain to you at a level you understand, and we can go back and forth, and I can answer your queries, you know, well. I want to close talking a little bit about costs and worries.

I'm worried about several things with regard to AI that I've covered on a lot of these shows. I'm worried about worker atrophy, right? There's one question that is, how do I use these tools? There's another question that is, if I use these tools to do the same task over and over and over again, will my skill in that task decline? And I think there was a study recently that showed that doctors that use large language models to diagnose disease become on their own less talented at diagnosing disease, which is kind of interesting.

Do you worry about skill atrophy in the face of artificial intelligence? And if not, I'd be interested just to know because you're such a clear thinker about these things if you do have worries in the part of the workforce. Oh, I mean, skill after three is a huge concern. I mean, massive, right? But it's not an unknown concern. So, you know, I'm a professor and students have always cheated, right? There's actually these great studies that show, I think it was, there's one of Rutgers that showed, and I'm going to get the number is not exactly right, but something like 20% of, 80% of students who did their homework in 2008 did better on their tests. And now it's after 20%, right?

Why? Because people were cheating on their homework. And so they stopped getting the benefit of doing homework, which is miserable and annoying, but teaches you things, right? Learning is hard. And of course, we atrophy skills all the time. I've atrophied a lot of skills because I use computers to do my math for me, and I don't do it by hand.

And so we're going to worry about this a lot. Writing, suddenly essay writing can be done by AI. It didn't mean people cheated before, but now everybody can cheat at scale. I think any skill you don't use, you atrophy. And skills, and we have to decide what skills you want to keep. There are ways of doing that, of course, right? So we're going to have much more Blue Book testing and things like that in schools, and we can force you to learn because we have you in a learning environment and we can grade you on the outcomes. And then we can use the good part of AI, which is actually some early evidence.

It can be an incredibly good tutor would use as a tutor to support an educational experience as opposed to a thing that gives you answers. The workforce is an even bigger deal because I teach people to be generalists at Wharton at Penn. And then they become a specialist by apprenticeship there, right? they worked for you as an intern, and you get the benefit of an intern who does work for you, it might not be great, but it's, it's, you didn't want to do it yourself, and they get the benefit of feedback and learn to be good at their job. If you just turn to AI for everything, the intern learns nothing, and the intern will be dumb not to give you AI content because it's better than them. So we're going to, may have to think formally about how we teach skills in the workforce.

We may have to have assessments of people's ability, but that there is a crisis coming in that set of stuff, right? But it's not an unsolvable one because we always give up skills to technology. But it is one that if we don't think about, we are kind of in trouble, right, in a lot of different areas. I want to make a slightly philosophical turn here. You and I are talking about using AI to do human work. But AI doesn't have human intelligence. It doesn't learn like we learn. It doesn't process like we process. It's a fundamentally alien intelligence that's being used for human ends. And that's a bit of a strange and spiritual. In our conversations offline, you mentioned that you wanted to get at this point by introducing

a concept from AI research that's called The Bitter Lesson. What is the Bitter Lesson? Okay. So the Bitter Lesson is an idea that came from computer science. It's actually a fairly technical lesson, but I think it's been used more broadly. And it's important to understand, because I think it tells us a lot about the future of AI. So the idea of the bitter lesson is best illustrated with chess computers. So we're going to do a little, you know, a little toward the chess. And when you wanted to build a chess computer, what you would do is you'd hire a grandmaster to help you build that chess computer. And you put every move of opening gambits and every move of how to close the game and end game results and lessons from everything we knew

about chess. And even through like deep blue, the machine that would be Kasparov, it was trained that way. There was a grandmaster who helped do the training. It did lots, searched lots of possible moves, but it still had a human chess computer chess master at the center of the training process. In 2018, Google released something called Alpha Zero, which was a chess computer that didn't know how to play chess. What it did was play on a chess board, basically a mechanical chess, an automated chess board against itself, to learn the rules of chess and eventually can beat Grandmasters. And that was the bitter lesson. The bitter lesson is your beautiful handcrafted attempt to instill all of your amazing human knowledge into a piece of software gets lost if you're just thrown enough computing at the problem,

and the machine learns how to do it itself. And this was very relevant to me because I don't take any money from AI labs, but I talk to them all the time along my wife who co-directs the AI lab at Wharton with me. And we talked to the Open AI team that had just released their version of agents over the summer. And we've been building agents for education for a while, and we had a conversation with them where we said to them, you know, one of the problems with the agents that you have is they don't give you step-by-step directions. They don't tell you, I'm going to do this, and then do this, and then do that.

It just does stuff. and that makes it really hard for us to diagnose the problems it has or intervene. And like, well, of course it just does stuff because we just train the AI on PowerPoint and Excel directly. We have no idea what it does. It just does good PowerPoint and Excel. And I haven't stopped thinking about that because that's in some ways the bitter lesson coming for office work. If your goal is output, if the only thing you want to do is produce a thing, then you can be bitter lesson, right? You can learn the bitter lesson because the AI might just be able to make the thing better.

If the human process matters, the friction and interaction, right, and the learning process matters, then maybe there's more of a moat there against what AIs can do. So I spent a lot of time thinking about that with agents. Agents are best where you can have an output you can train against. Where the process matters, there's more room for humans. I want to think a little bit about, and this is a question that we're going to use, I think, to wrap up the agent section because I liked your note. I want to think a little bit about how strange this could get if agents really do take over the workplace.

So I wrote a piece about job applications coming out of college and talked to a lot of college career counselors. And they said, Derek, you have no idea how strange the job application process has become. You know, when I graduated from Northwestern in 2000-8, I sent out what I considered a ton of applications. I applied to 30 magazines before I got a callback for the Atlantic. Today, they said it's not uncommon for people to use AI to apply to 300, 500, even 1,000 jobs. and because that is way more applications and a human being can process in the HR departments of Amazon

and meta and all these companies, they use AI in order to filter the AI-enabled job applications. So that in a way is like a little diorama of what an economy might look like if rather than have human-to-human interactions, the white-collar economy is mediated by agents, right?

That Ethan's agent talks to Derek's agent to figure out the perfect line of questions to ask Ethan, and then we sign off on it, and then you and I both get a memo that's sent by our agents to each of us. I mean, things could get really weird if this technology succeeds at the level at which and is implemented at the scale at which a lot of these labs are attempting, right?

I mean, this is what I was talking about before. Process breaks. All the processes of the world are built for humans, right? The college essay, the letter of recommendation, right? I write a letter recommendation as a signal. I set my time on fire as an indicator that I care enough about of a student. The nature of the letter matters less than the fact that I wrote it, right? Now, and I actually ask my students, would you rather have the okay letter that takes me 50 minutes to write?

Would you rather have the AI letter in a minute that's much better and is in my style? No one-minute letter, right? Because it was a better version. I actually had a student send me the prompt they wanted to use to write their essay, which I would have a letter recommendation. which I thought was interesting. Like, what you read you want is the sentence now that says, yeah, they're good, hire them, right? But like, we have all these rituals and stuff built around this, right?

I was online. I just did a thing earlier where I was asking Claude for PowerPoints. I just kept saying more PowerPoints, more PowerPoints from a memo. And I ended up with like 17 really good PowerPoints. Like, no one wants to decompress that PowerPoint back to the original problem. So this is part of what's going to break, right? When you ask you about the productivity problem, the productivity problem is everything we do, things like writing, showing effort.

Like, think about an essay. An essay showed me competence by how well was written. It showed me effort by how long it was written. It showed me attention to detail by lack of errors. None of those things are signals anymore. So what do I do when I create an essay? All my heuristics are gone. Now it takes forever to grade an essay where it used to take a couple seconds to do

because I would know whether it was good or bad. This is happening everywhere. So I think a lot of things are going to break before they get reconstructed. And I wouldn't be surprised if we see lots of processes that we have to rebuild from the ground up. up to think about what it means to be in a world of AI. It's happening everywhere. Scientific papers, more of them coming in. The solution can't be AI grades AI content, and it's just a mess of a world where things just get filtered from AI to people.

So we're going to have to think about this. Maybe there's more in-person conferences and meetings, right? Maybe that there's more, you know, that we have new selection mechanisms, but it can't go on like this. Can we just talk very specifically about the way that it can't go on in science? because it's interesting that in the same way that in the job application process, AI can both amplify job applications and evaluate job applications. In the realm of science papers and scientific journals, AI can write science papers, but judges, reviewers with AI can also evaluate science papers. So how is that going to break and how is that break going to be resolved?

Because we can't possibly just live in a future where, there's just no more peer review and also no more research because it's just the AIs writing the paper for the AIs and publishing, you know, science, the journal by artificial intelligence. So there's actually a ton of things. First of all, almost all the systems we're talking about are already broken, but they kind of work well enough, right? So like, well before AI came out, there was a great article in science explaining they were under a flood of research and it's no longer, people aren't reading it anymore, right? So people are just referring to the same articles over and over again and not really reading. Like that was pre-AI. We had this problem,

between paper mills and just having too many scientists versus too little review, the way journals worked. I mean, there's a lot of broken systems out there that were just chugging along. We're talking about homework, right? Which people were cheating at before, but it was kind of okay. We kind of resolved. So a lot of this is systems already under strain or that were already broken that we're evaluating. But in science, it's especially interesting because right now, I think we're in a place where

the current AI models can produce trivial but true information. They can do, they can prove a new gesture or theorem in math. that isn't a big deal one, but a small one, but something that will probably get published before. So now there's that. And then I think very soon, the main goal of the AI labs is to produce actually good output. They want AI to do real science.

So we might actually start getting a flood of not trivial, but true and real research. And as you said, leaving aside AI content that's created, you know, for various reasons, we don't have a way of dealing with that. AI does a pretty good job peer reviewing, right? But we still want humans involved.

So is it some sort of, flagging process, we have a series of AI reading each paper that comes in and flagging the most interesting for human to skim. I don't know, but like these things don't get invented automatically. And I don't see the systems, the guardians of our systems thinking hard enough about how to change this. Like going back to your example of, you know, meta trying to review too many applications, unless there's some effort to rebuild what the job application process looks like, it's going to be an arms race of AI against AI. So one of the things that the breaking up pace of AI development doesn't let us do as much

is sit and think a little bit more about what structures we want to do. Even think about journalism, right? Let's say, extrapolate where they are, AI can do a pretty good journal, like probably become a pretty good journalist, not as good as the best journalist, but probably a pretty good local interest journalist, searching for information, you know, interviewing people even, and then producing pretty well, compelling and tight articles. What do we do with that? Like, what does that look like?

I don't think we have a lot of ideas. So this is going to be a series of kind of rippling changes everywhere. We will adjust, right? I mean, assuming that we don't reach some sort of superintelligence that solves all our problems, we'll figure this out. But we're in for a fairly disruptive period. It's interesting because listening to you, I feel like there are some people that consider themselves AI optimists that believe that once we pass some certain threshold,

there's going to be a fume that AI is just going to sort of like take over everything. And it seems like you believe that what's coming is going to be more of a muddle. There's going to be a muddle to figure out what the Jagget Frontier looks like for the models that exist. There's going to be an organizational model as companies figure out how to use this powerful technology. And there's also, I like what you said at the end, there's going to be almost like a civic or like interpersonal or a very, very personal muddle to decide, well, now that we have AI that can essentially do the job of the local reporter in Atlanta, Georgia. Do I want to read a newsletter by an AI? Do I want to be the kind of news organization that rather than hire human journalists instead hires AI?

If there's music that can be made by AI artists that is technically, according to some blind test, as good as Kendrick Lamar or Taylor Swift, do people actually want to listen to AI music? Like, there's going to be this really interesting, I think, very long period of trying to reconcile with the fact that we're building a technology that is as good as us at many tasks without understanding how to incorporate those tasks into organizations

or how to deal with the fact that we've just been sort of equaled or even vested in certain fields where we're much more familiar with dealing with humans rather than robots. Yeah, I mean, like, let's, people always flip this as like a disruption that is about, you know, robots taking our jobs. But the other question is

like if the AI is better diagnosing you than a doctor, right, and is accessible to you via your phone, what do you do about that, right? Like, we might want doctors to do that. Well, people individually do this. I think revealed preferences is that a billion people around the world are using AI. They might rail against it, but they might also use it. I mean, it's a general purpose technology.

It is going to influence every part of our culture and society all at once. And if you study other general purpose technologies, steam power, electrification, they're messes when they come out, right? Like, all of society reorient itself around this. Like, you know, we think of Steve Power, we think about like steam engines, but you don't think about like urbanization happened as a result of that, right? Like the nature of warfare changed overnight as a result of this. Nation states got born as a result of this. Like many things are downstream from technology plus society. And we're going to have a lot of weird stuff. Some good, some bad,

all happen at once in the very near future. I'm glad you see it as a mess because you followed this stuff closer than I do. But someone recently asked me, you know, what do I think will happen if in 2027, in 2028, Claude, Anthropic, Open AI, announces that they essentially have built what they're calling AGI. Like, what will that look like? And I said, look, if you think this is the next industrial revolution, just know that every other industrial revolution

went through a period that was objectively terrible, right? As you said, steam power in the early 1800s, urbanization was such a mess that I remember reporting this. In the 1820s, the average age of a male labor in Manchester fell to 29 years old, because the conditions were just so decrepit and diseased and full of literal crap. In the late 19th century with the second industrial revolution, I mean, you had unbelievable bubbles and bus and union conflict

and derrenching changes to American life despite the fact that electrification and the railroad and telegrapher changing everything, it would be crazy to think that this is going to be a similar industrial revolution, but also no mess. That sounds like the Eschaton. That sounds like a Christian rapture. It does not sound like the introduction of a general purpose technology to a human economy. When that happens, we tend to get a mess. So, Ethan Malik, this was absolutely fascinating.

Thank you so much. Thanks for everyone. Thank you for listening. Plain English is reported and hosted by me, Derek Thompson. This episode was produced by Devin Boraldi and Devin Rinaldo. If you like what we're doing here, please rate and subscribe. new episodes drop every Tuesday and Friday.