Plain English with Derek Thompson - Will AI Usher In the End of Deep Thinking?

Episode Date: August 6, 2025

Last week, the Bureau of Economic Analysis published the latest GDP report. It contained a startling detail. Spending on artificial intelligence added more to the U.S. economy than consumer spending last quarter. This is very quickly becoming an AI economy. I’m interested in how AI will change our jobs. But I’m just as curious about how it will change our minds. We’re already seeing that students in high school and college are using AI to write most of their essays. What do we lose in a world where students sacrifice the ability to do deep writing? Today’s guest is Cal Newport, the author of several bestsellers on the way we work, including 'Deep Work.' He is also a professor of computer science at Georgetown. One of the questions I get the most by email, in talks, in conversations with people about the news is: If these tools can read faster than us, synthesize better than us, remember better than us, and write faster than us, what’s our place in the loop? What skills should we value in the age of AI? Or, more pointedly: What should we teach our children in the age of AI? How do we ride this train without getting run over by it? If you have questions, observations, or ideas for future episodes, email us at [e-mail]. Host: Derek Thompson Guest: Calvin Newport Producer: Devon Baroldi Learn more about your ad choices. Visit podcastchoices.com/adchoices

Hey, it's Craig Horlbeck here to tell you that the NFL is back, whether you like it or not, and we are covering all the latest news, trades, rankings, and more on the Ringer Fantasy Football Show with my two co-hosts who are both named Danny. Check the Ringer Fantasy Football Show out on Spotify or on our new YouTube channel. Today, AI and us. Last week, the Bureau of Economic Analysis published its latest GDP report, and it contained a startling detail. Spending on artificial intelligence added more to the U.S.

economy last quarter than consumer spending. That means the only reason this economy is growing right now is because of the boom in artificial intelligence spending on advanced ships, data centers, and energy. I think this is one of those stats that's worth sitting inside for a few seconds. We talk a lot about what will happen if at some distant date AI eats the economy of the future, dominates the economy of the future. But we don't need to wait for the future. AI is driving this economy right now, today. This is already becoming an AI economy. If, for example, you want to know why is GDP growing rather than stalling, the answer is

AI. If you want to know why the stock market is soaring rather than stalling, here again, the answer is AI. According to Michael Sembalist at J.P. Morgan, nearly two-thirds of stock market returns in the last two years have come from a small, number of AI-related stocks, like Microsoft and Amazon and Meta. I'm working on several shows right now on what an AI economy will do to work and productivity and our jobs. The effect of AI on the economy is a really interesting and important question to me, but I'm not just interested

in how AI is changing the economy. I'm interested in how AI is changing us, our habits, our behaviors, are patterns of thinking. Some commentators have analogized artificial intelligence to the railroad buildout of the mid to late 19th century. Both of these are capital-intensive technologies that could transform the way we do work. But railroads, in addition to just transforming the economy, also transformed a way of thinking about the world.

The science writer James Glick has suggested that it was railroads more than any other tech that introduced the idea of time travel to our culture. Railroads made the introduction of time zones necessary, and they scrambled people's sense that there existed a universal time for all human beings. Glick writes, quote, travelers riding in steam-driven railroad trains,

looked out their windows onto a landscape where oxen plowed the fields as they had done in medieval times, horses still hauled and harrowed, yet telegraph wires split the sky, This caused a new kind of confusion or dissociation. Call it temporal dissonance. End quote. Technology profoundly changes the way we think about the world around us.

So how will AI change us? Several months ago, we did a show here called The End of Reading. I had on the Atlantic staff writer Rose Horowitz and Nat Malkis of AEI, where we looked at surveys, finding that leisure reading has declined significantly in the 21st century. And according to Rose, many professors at elite colleges now say that their students have never in their life read a full book. All they've read in high school is just little snippets of books to prepare for the PSAT and SAT. We are losing now, even at the highest levels of education, the ability to sit with a piece of text, read it, and make connections within it.

Soon after we did this episode, New York Magazine published a viral cover story on the epidemic of cheating in college. This was a long piece with a very short TLDR. Practically every student in college, and I would assume high school, is using CHAPT to write all of their essays. And also practically nobody,

the teachers, professors, know quite what to do about this. The ability to use AI to summon into existence, any essay has exploded colleges in high school's ability to assess intelligence with take-home essays. The students are just going to cheat. In other words, we aren't just losing deep reading in the age of AI. Students are sacrificing deep writing as well. Today's guest is Cal Newport.

He's the author of several bestsellers on the way we work, including Deep Work. He's also a professor of computer science at Georgetown. I wanted to have Cal on the show because one of the questions I get the most by email or in talks and conversations with people about the news these days is if these tools, if AI reads faster than us, synthesizes better than us, remembers more than us, writes faster than us. What's our place in the loop here? What skills should we value in the age of AI? And even more pointedly, what skills should we teach our children to value? How do we ride this train without getting run over by it? I'm Derek Thompson. This is plain English. Cal Newport, welcome to the show.

I'm happy to be here. I want to start with observations before we move into analysis. In your reporting, in your experience, have you seen major changes in the behavior of students in the age of AI? It's shifting rapidly. Like in my own case, for example, last year, I did a story for the news, Yorker about how were students using AI in the context of writing? And the premise of the story was I looked over the shoulder, virtually speaking, of a couple students to actually watch how they're

interacting with AI. And what I found in that story, like the thesis of that story, is that a lot of what students were doing when they were writing with AI, they were not outsourcing writing to the AI. They weren't saving time either, so it wasn't an efficiency play. My argument was they were having these almost parisocial back and forth interactions that were aimed at reducing the peak cognitive strain of the process of writing a paper. So it was this interesting sort of interactive back and forth they would have. Well, what about this? What do you think about that? Okay, I'm going to write this. What do you think about it? And my thesis was it was trying to make the peak difficulty of writing better. When you look today, so like with fast forward a year later, there's been

multiple things that have reported on this more recently. And it seems like this parissocial relationship up this sort of, I'm in this interaction with you to sort of take difficulty, the smooth over hard peaks has really maybe metastasized. So there's a big new article out recently from the Guardian where a reporter convinced, I think it was three students to basically let him have full access to their chat GPT plus accounts so that he could see every single thing that they were saying over a period of whatever it was a couple weeks. There's a huge transcript. And he says their interactions with it was constant. It was just all day long, not just about homework assignments. not just about a paper they were writing. It is, what does it mean to be a human being? What do you think

it means that this girl said this to me when we were walking by in the cafeteria, that it had become a sort of interactive back and forth partner? So there's this interesting thing, I think, happening among young people with these tools where it's expanding beyond its role in pragmatically speaking, this is helping me do certain work, which we can get into because it is. And I think it's making a big difference on how college actually operates. But it is immediately seeming to move over into the space where social media and other tools were as well in this sort of diversion, distraction, tickle your brain type of context. It's interesting. It gives you a release in the moment. It prevents you from being bored. It prevents you from having to feel some sort of negative strain.

So it really expanded its footprint. It looks like it's starting to expand this footprint in student's life more than I thought it would. So a lot of things are going on here. One thing I hear you saying is that while a lot of people think about AI as an economic technology, you're watching the way that it's already become a social technology. You talked about the parisocial relationship that people have with the large language models they're interacting with. They're talking to it like it's a professor, like it's a friend, like it's a research assistant. That's how I use, say, deep research. Let's ground this at the level of college. how has this changed the way students and professors operate?

So if we look at it just from a functional perspective, it's changed a lot of things. Like, for example, during the pandemic, I teach mainly math-type courses or theory-type courses. During the pandemic, obviously, you had to administer these exams remotely because there's no one in person at all. There was a sort of brief window after the pandemic where it was thought, I was like, oh, this is convenient. Like, why not keep doing it this way, right? during an exam period, this way we don't all have to stick around for five days instead of going home to take the exams. Let's keep doing this online. I was doing it synchronously online, essentially. Post-chatGBT, you can't do that, not for an introductory, discrete mathematics course,

because every single problem on that exam can probably be answered for you by chat GPT, because it's not only is it not that it's not very complicated math, but as I learned when I've experimented with using chat GPT to help write problem set problems. For a lot of these problems, it turns out there's only so many good examples that are out there. There's only so many actual ways to test like a sophomore level undergraduate on doing strong induction proofs or something like this. There's really like four good examples and you can obfuscate them, but that's that. So you could put almost every problem into chat GPT, right? So okay, we can't do remote exams anymore. I think the same thing is happening. I picked this up in my reporting. Whenever you have

lower stakes writing. There used to be a big thing where you would say, come on, have a response essay at the beginning of each class. This way, you know, I'm not going to look at it too carefully, but you have to do the reading because you had to write 500 words, like your thoughts or whatever. As one of the students told me on my reporting, oh, those type of essays are imminently chat GPTable. That was the phrase, because that exactly that type of writing is very chat GPTable. When it comes to like large papers, what seems to be happening is we thought when this technology first came out, we being professors, that maybe students would be able to essentially recursively generate the entire paper from scratch. You can't ask ChatGPT, give me 10,000 words on, you know,

Jung and the collective unconscious. But what you can do is say, give me an outline for an essay like that. Okay, let's look at section two of this outline. Break that down into like the three subsections. Okay, let's look at this subsection here. Can you give me some draft of text? So there was some fear that doing that you could basically have a whole essay produced, it would be like a custom version of those cheating websites that were around when we were growing up where you could pay for papers on the internet. That's not working too well. Those papers aren't coherent and the tone isn't that great.

So on long papers, it's more that students are using it for ideas. They're using it for, give me a structure for this argument. It's less about the craft of creating the actual words than it seems to be the craft of actually critically thinking about what I want to say. So that is creating changes, I think, for the human, humanities. There's been a move towards a lot more in-class assessment. If it's a quantitative class, there's a move towards, like, probably what we need to do is have a quiz in the first part of class once a month and weigh that more than we're going to weigh problem sets. There's more

of a move for intro mathematics classes that the problem sets are basically for practice, right? Like, you should do these problems as how you're going to practice, but it's going to be less of the graded assessment. That temptation is too hard that you can basically solve any one of these any one of these problems. So there is definitely a big shift that we have to deal with. It's maybe comparable to the, I think the consumer internet caused a comparable size shift in higher academia that that introduced a huge amount of changes. I mean, I was a student and grad student just as that got really big. That introduced a lot of changes we had to adapt to. These are kind of similar, I think, what's happening, but they're numerous as well.

Cal, there are so many committees right now at colleges and universities that are trying to help professors and deans and students figure out, how do we incorporate AI into a college education? And I was having a conversation with someone at a Washington, D.C. college about this question of what should schools do in the age of AI? And I had, we should have cobbled together an idea, and I'd love to just throw it at you and have you tell me if you think this is remotely plausible. So my wife just finished her PhD in clinical psychology. And I love the fact that the verb of the last verb of PhD is you defend your dissertation, right? You don't write your dissertation, you defend it. And it made me think, should we turn more of college into a defense, right?

What's chat GPTable is writing an essay about the Habsburg Empire. What's not chat GPTable is defending your essay about the Habsburg Empire in front of your History 105 class on ancient European warfare. And I wondered if turning college into something that was more stand-up, more presentational, more not only are you graded for defending your paper, I'm also going to hand out grades or incorporate the variable of how good are your questions of the person presenting at the front of the class. Is it possible, do you think, or just totally fanciful, to try to transform more classes into the equivalent of dissertation defenses? Well, I mean, it's not only possible, some universities do this.

I mean, this is largely the Oxford model where what you do is have tutorial with your tutorial head. And it's you, however many times a week, basically explaining and arguing to them about what you read and they ask you questions about it. And that's where most of your evaluation, you know, comes from. Tyler Cowen made a similar argument recently as well. They said, okay, colleges need the shift to become much more participatory and also much more of, yeah, I don't know if you use word defending, but this sort of like, yeah, oral and interactive in that way. There are a lot of tools. I mean, one way to think about it is traditionally a lot of college education was in some sense like this, right?

I mean, the blue book exam is this. It's not out loud, but you're filling in those composition books there in the exam room from scratch with nothing there. You're making a written argument where you have nothing to support you. I mean, that's how I took exams when I was coming up. in things like mathematics and science. I mean, this is what written exams and quizzes are. It's like you have to basically demonstrate your knowledge of this. How you got the knowledge?

Like, we gave you help. We gave you lectures. We pointed you towards things. But like essentially, you have to sit down now and answer a lot of questions. So I think we do have some of those tools. The big questions, here's the two things that the big questions are about is problem sets and papers. I think that's really the crux of it, right?

So if you're in a technical class, problem sets, which means here is, a set of problems you do at home and then you bring back in the class, right? So if you're not a quantitative major in college, this is like one of the core things you're evaluated on. These are tricky. These are really tricky in the age of chat chabit. And it's a problem because problem sets can give you hard problems. So they serve a purpose, right? Like, I'm a theoretical computer scientist. And when I'm taking theory classes, right, like I took theoretical computer science at MIT from the head of the math department there at the time, Mike Sipser had like written the definitive textbook on this.

The problem sets is where you really got your chops. Because in an exam, you only have so much time, the questions can only be so hard, but a problem set, they could give you one problem that could take you four days. And so it's hard. That is hard. How do we replace that? Papers you take home is, that's the other hard thing as well, right? Like, how do you

replace that studious, systematic effort of gathering information and organizing your thoughts? That's lost when it's just in, I'm going to defend something in front of a tutor, right? Right, because I have to understand it, and I have to know rhetoric, but there's not that I'm in day five of trying to organize my thoughts on this and now I'm finally seeing how it comes together. So I think that's, I don't know what to do about that. I don't know what to do about problems.

But I think in general what you're saying is yes, is what's happening. I mean, maybe it's not out loud, but I just more assessment is moving towards, let's think of it as like real-time pedagogy demonstrations. Real-time pedagogy demonstrations are becoming once again much more important. The question that I get most in this space is people looking at the rapid advancements that AI is making at writing, at math, at biology, at every single discipline, and saying what's going to be left in 10 years? And the rubber hits the road with a question as specific as what should we teach our children in the age of AI. As someone who's really close to this space and who I believe has kids, how are you thinking about this? Well, I would be wary at continuing to extrapolate up the curve, right? So I would be wary at extrapolating from, you know, GPT-40 produces more fluent text than, you know, 3-5 produced, and then continue to extrapolate and say, okay, so writing is not something that we'll be able to do in the future.

Or say, GPT-40, they did well on the International Math Olympiad. So math is something that, like, we're not able to do in the future. I like to be a little bit more grounded in this. Like what I'm interested in less right now from a journalistic perspective is what could be replaced by AI. I'm much more interested in what has been replaced because that will tell us a lot about what might be replaced in the future. Now here's the issue is the answer to that right now is it's pretty narrow and fractured. Like we are still very much in a moment right now of potential that we are seen is going to be played out. soon. We're still in an era of benchmarks. We're still in an era of in these measures,

this model is now doing better on this graph of the frontier benchmark. We have this other benchmark on reasoning. But we don't yet have a lot of examples of here is a small economic sector that here's the skill you could build for. And it is gone now, not it could go away. So we kind of need to see those to understand what it looks like when AI moves into an economic sector and sort of a more fundamental way. So I don't know how to answer that question yet. I mean, outside of like some pretty niche things, I said, let's wait two years. I think I'm going to have a much clearer answer to give my kids. Right now, we don't have enough. It's like early in the Industrial Revolution, like, we haven't replaced any of the looms yet. We just like know there's

these technologies coming, but we don't quite know like what it's going to look like yet when a factory automates, right? That's where I think we are in AI. And it's also unclear. exactly how far that's going to go or how many of these fields are actually going to end up being taken off of the plate of possible economic pursuit. So I don't know how to answer that yet because there's not enough case studies to look at.

Can I offer an answer that I've been thinking about? This is a general answer to the question of what should we teach our children, what should our children value might be even closer to what I'm trying to get at here. I was talking about this recently at a talk. In exercise, in weightlifting, there's this concept called time under tension.

So you can do a bench press in three seconds, or you can do a bench press in 10 seconds, or you can do the same bench press in 20 seconds and slow, slow, slow, slow, slow, slow, down. It's the same rep, but it's much harder. It's time under tension. I feel like we're in an age right now where young people are reading less.

Book reading rates have really declined significantly, even at elite colleges. And now with AI, as you've been explaining, students can write less because JSTGBT will always be game to do your homework. And I feel like if students aren't reading as much, and they're not writing as much, where's the thinking coming from?

The best ideas that I've come up with tend to come from me being able to sit with a group of thoughts that are far-flung, in far-flung departments of my brain, and having the patients to sit with them for a long period of time, and they cohere into something that's combinatorially new. And I think of that as kind of the cognitive equivalent of time under tension. Right. Without the capacity for long-form reading or writing, I worry that we're just going to lose that. It'll just be gone. And so my answer to this question of like, what should we teach young people? What should they value academically? I want to like, I would want my children to be masters of cognitive time under tension, right? This kind of like, this kind of academic patience that I think will pay dividends whether they want to be a theoretical computer scientist or a novelist. How does how does the idea of like time under tension sit? with you as you think about some of the awkward conveniences of AI for students. Well, I think this is a key shift you're making here, because I agree with you. And I think it's a

key shift in the way that people are thinking about AI, right? Because what I was responding to is what I think is a key way that people are thinking about AI right now, which is, what are the things AI is good at that now I have to find different locations for me to go? Like, what is it becoming so good at that I have to go to another lane, where really the issue that matters right now is what is AI making me bad at? And I think that shift in perspective is a really important one. Now, I've been talking about this for about a decade because basically AI as I see it is the latest of multiple heavyweight entrance into this prize fight against our ability to actually think. So it is the latest thing taking away. I think it was ubiquitous access to highly salient

distraction. So what was delivered through the smartphone revolution? This was the final nail in the coffin, I think plus also streaming and highly available high quality content, but really the ubiquitous pocket accessible, optimized entertainment. That was the final nail in the coffin of reading. And now we see AI is coming in and taking out this other thing we had left to put mind under tension, which was writing, and it's taking, either taking that away or like we talked about taking the time under tension required from writing and reducing it, because it can make it this more interactive thing where you don't have to hold ideas in your head in a very difficult way. It gives you this sort of cognitive relief. So yeah, it's a problem and it's a

problem that's in a bigger constellation of other things that are causing the same problem. Because I'm a big believer, like our current world and economy, it's a system symbolic economy. You have to be very good with systems thinking and symbolic ideas and thinking. Like that is since basically the rise of the Neolithic Revolution. That has been the evolution we've demanded of our brain, right? We have brains that were evolved in the Paleolithic. They're pragmatic brains for a very functional purpose, which is surviving in bands in a sort of hunter-gatherer, small band type of setting. We culturally changed our brains. This was a technology we figured out how to do is how to, it rewires our brains. How do we rewire our brains to be able to do system symbolic thinking? Reading was the real

driver of this. This is a cultural evolution. This is not something our brain is wired for. Reading forces us to make new connections in our brain that did not exist, that you would not find these connections in the average person from the Paleolithic. And it's what enabled all of this sort of system symbolic thought that, like, Yuval Harari would talk about as being sort of like key to like the development of all of human history. Right. I mean, this is grandiose types of stuff. And then writing helps you take these circuits you build with reading and apply them. And so figure out like, how do I like take these circuits and aim them at things that I actually care about? So we have this one-two punch, reading, writing literally changes our brain, right? It's like this,

this serum we have to take in a superhero comic book to game the superpower. And so I have been bringing, you know, this alarm bell probably all the way back to my book deep work, that it's something that we have to keep taking the serum for. We have to work on our brains to have this sort of arbitrary, constructed, culturally constructed type of brain that we need to thrive in today's world and that we should be super wary when stuff takes that proverbial serum away. So the death of reading because of smartphones, I'll say the file nail in the coffin. I mean, that was on a downward trend for a while. And now, as you're pointing out,

I'm 100% on board with this. Writing, which was the other part of this, AI is taking a lot of the hardness out of that. Yes, there is, I think, a sort of like social impact here, a sort of personal flourishing impact here. This is where I think the economic impact we should care about first is, is what happens when we have a knowledge economy that depends on our brains. We make the brains worse.

Like, I care more about that than what is going to be automated by these tools. That's like at least just as important. So I'm with you on here. This sort of time under tension, as you say, is critical, and it's been under a sort of unprecedented attack over the last 13 to 15 years. In thinking about this cavalcade of heavyweight fighters that you just described, right?

There was television, which is a distraction, computers, a distraction, smartphones even better distraction. These screens were distinct from one another. It's not like the smartphone is just more TV. It's also interactive. It's a conversation in a way that watching television is not a conversation. How do you see AI not just being more smartphone,

if you get my drift? How is this new distraction distinct if what we care about here is the preservation of our mental acuity? The way I often think about it is reading builds these smarter circuits and production of content is what helps you actually use to circuits

and get good at actually applying them, right? So Marianne Wolfe uses this term deep reading processes to talk about what actually happens in your brain when you do hard reading. But to learn how to apply those circuits to make an original idea, to have original argument,

you have to actually practice the production of complex content from scratch, which is like what writing has you do. What is AI really good at is automating production, right? So phones distracted us from consumption of things that could make us have better circuits. AI, the fear would be that it reduces the amount of time you have to spend producing original

content X Nilo. You take that out of the equation too and things could even worse. So yeah, I think these are, these are focusing on different things. And you can look at television. You're right, that was something else. I mean, that really hit sociality among other things. And so you're right. Every one of these technologies is different. And so it's the reduction of production. That is where I think AI is a problem from a cognitive perspective. There's been a couple studies here that I want to get your, brain on. One published by the MIT Media Lab that got a lot of attention in the media called Your Brain on Chatchabit. And I'm just going to tell people a little bit about this study

before you and I talk about it. MIT Media Lab assigned a bunch of participants to several groups. Each group wrote an essay. Some wrote the essay on their own. Some used Google search. Some used chat Shabit. And the researchers hooked up an EEG to follow the writer's brain activity and then maybe interview them after they wrote these essays. And basically they found that of the three groups, chat GPT users had the lowest brain engagement, quote, from the paper is they, quote, consistently underperformed at neural linguistic and behavioral levels, end quote. And maybe most significantly, chat GPT users got lazier as the experiment went on, so that by the end of the experiment, they were basically just copy-pasting essays with no possibility of recollecting what they

had posted because they didn't even look at it before they posted it into the box. I saw a range of opinions about this. Some folks said it was kind of cheap. Some folks said it was quite profound. What, if anything, do you take from this study? I think there's critiques about the specific construction, sure, right? In particular, one of the issues that these type of studies have is that when you're writing essays for a study, for which you're getting like a $20 gift certificate, you don't care. So you're like, you just start, this happens a lot with these sort of distraction studies.

You just start copying and pasting and stuff because also you kind of want to get out of there and is boring. And who cares? You're not getting a grade. It doesn't matter. If they put like $500 on the line, if you have the best essay, that would be sort of more interesting. So yeah, I think there are those critiques. But I think the reason why this study got so much play, and I wrote about it, I talked about on my podcast, is that directionally it's correct. It just seems self-evident that directionally this is correct.

Like, okay, who really cares about exactly what brain wave they measured? Of course, this is true. It's exactly what I observed when I did the more anthropological study for the New York. Yorker where I just looked over people's shoulders. This was exactly the same thing I observed is writing is very hard. I got into the brain science in that piece because you have to connect together these various parts of your brains using the deep reading circuits. And that feels bad. It's like difficult. It's time under tension. And when you're going back and forth with chat CBT, you can just reduce that tension. And you don't, and what's hard, what creates the tension is

trying to connect together these different parts of your brains and hold that connection and stasis. It's like, of course brain activity is going to go down if you're just measuring with an EEG. because that's exactly what people are, of course, doing when they're trying to write with this, is make writing less hard. And that means less brain activity. So I think directionally, this is true. It's just like self-evident that this is true. The details of the study are just, yeah, who cares? Like, it's really not that, it's really not that interesting. Just talk to a student, right? Hey, what's it like when you write with chat GPT? They're like, I don't know, it takes forever and I kind of glaze over, but it's not as hard. Like, that's what they're doing. So I'm not surprised by this study.

There was another study that I think really sits in interesting juxtaposition with your brain and GBT study. And this was one of software developers. The AI research nonprofit METR conducted this study where they had a bunch of seasoned developers use an AI coding assistant. And then after they completed their coding task, the developers were surveyed. You know, how much time do you think you saved? And they said, on average, we think we saved about 20% of our time.

Right? We got a five-hour project done in four hours. But the study itself found that using AI did exactly the opposite. It increased task completion time by 20%. You thought it was a five-hour task. It's actually a six-hour task. What, if anything, again, does this study tell us? Because whereas your brain on Chachby-T says we're using these devices,

we're using these technologies in order to cheat our assignments, This second study says, no, we think we're cheating, but actually the outcome is worse. It's making us less productive, not accelerating the degree to which we reached the end of the assignment. So how did you feel about this second study? Well, this one created much more of a stir, I think, within the AI industry, because if you look at the MIT Media Lab, like, look, these are academics writing about paper writing. Like, they don't want AI to do well here. Meeter actually has been, you know, it's a nonprofit research institute that most of the stuff they've produced has been touted by sort of AI companies.

They do all these benchmark measurements and they have these graphs that people show of like look at the rapid improvements and skills. So this was sort of someone from inside the henhouse saying like, uh-oh, I don't think we have as many eggs here as we think. So I think this landed more. It's also at the core of the current economic sort of engine of these companies. I mean, we know this is one of the use cases in the business, in the business setting. This is like one of the use cases, early use cases that seems to have traction. So they really care about is this making programmers more efficient? I think the message of that study is that it's very difficult to measure what we mean by productive, right?

So all the critiques of the study, they're fine critiques. It's like, well, I don't know. Like, what were these tasks? How do you measure how long, you know, are these? the right type of task, like what's going on. And the message I get out of it is, yeah, it is hard to measure productivity and productivity gains, but that that swings both ways. And that if you look at a lot of the chatter that's happening about, are we even going to have computer science majors three years from now, right? Like all program is going to go away and be automated. It's suffering

from those same flaws. It's suffering from the flaws of like, I don't know, this made something really easy for me. I guess you're right. Like maybe I'm, I don't need to hire someone or I'm way more productive than I used to be. Like it's all sort of like a vibe and anecdote, but it's really hard to put your finger on, did this make you faster or not or what does faster mean? Or was this code going to require you to debug it five times longer than it would have been if you spent more time right in the first place? I think that's what that emphasizes is that that type of work is complicated, and we should be wary, I'm wary about both the hyper-hypers and the hyper-sceptics. If you talk to programmers, and this is my world, what do you hear from programmers?

They say, first of all, you have to separate vibe coding from serious software development. Yeah, it's really cool. And just slow down, and just slow down here to define vibe coding from serious software development. Right. So vibe coding is where I might not know much about computer programming. I want a quick prototype of something. I'm basically having an AI model produce all the code for me or with minimal interaction from me, right?

So I want to put together a quick web-based scheduling app. This is a real example from a friend of mine for like an auction I'm having at my kids' school. And I don't really know how to program JavaScript or whatever. I could just sort of make this thing without having to code. And that's really cool. And that's vibe coding. Then you have serious software developers where what they're doing is not that. What they're doing is preventing themselves from having to go to Google and Google stack overflow.

So it's a giant bulletin board where people, publish answers to questions about programming to figure out, hey, how do I call this library or what's the right format for a sort of virtual constructor for a C++ class? There's a lot of looking up information about how to write certain types of code. And instead of having to go over to Google and look it up, these AI tools can just find it for you and put it right there in your code. So it's like really useful. It's integrating, searching for stuff you need to look up to keep writing your code, integrating that into. to the place where you write your code.

And developers really like that. And then there's vibe coding, which is people who don't really program much can build prototypes. That's kind of what's going on right now. And those are both two really cool things. But we don't have a lot of evidence. It's, okay, we're one step away from these programmers are gone,

and the next year these programmers are gone. That's not really the way people seem to talk about it. If they're not managers being quoted in like Wall Street Journal articles about AI, where you want to seem very, like, cool and with it and very high-tech. If you're just talking to actual programmers off the record, that seems to be what's going on. When I think about these two studies and I put them together, one takeaway that I have is that AI is really, really good at answering questions, but it's bad at telling you that you're asking the wrong questions, right? The first study and so much of college AI use is, I know what question I want to ask, right? how long did the Habsburg Empire last?

Like, who was Genghis Khan's grandson? Like, these kind of questions, AI's really, really good at, especially if the answers are in the distant past, because it's not as good at looking at more recent web browsing. But AI's never going to tell you, hey, you're totally on the wrong trajectory for writing this essay about Genghis Khan.

Don't write about his grandchild. You have to write about his nephew. Like, AI's never going to tell you that. And the reason that asking good questions is really, really important for productivity is that I think in a way, productivity in many fields is about having good taste in questions, right? It's being a software developer and knowing exactly how to tackle the problem of whatever, this particular part of the website broke. And it's true for writing as well. I mean, how many, there's so many essays where I've spent hours and hours and hours trying to write the essay and failing and failing and failing and realizing, oh my God, God, I have the wrong lead anecdote.

If I had this other anecdote that was just in this other note, the story would have flowed perfectly. And so I do think that one thing that we're beginning to recognize is this discrepancy between the facility with which AI will always answer our questions. And sometimes we conflate that feeling with productivity, right? And on the other hand, the fact that actual productivity is not just about answers. It's about being really, really careful about good taste and questions to ask. and AI just is not as good at doing that slash we maybe aren't as good at prompting AI to do that, which might be one of the reasons why we haven't seen,

you know, sudden massive productivity gains in the creation of a technology that is, in fact, really quite smart. I mean, I think I agree with that. You know, a source was telling me the other day that one of the reasons why you see so much focus in the announcements of AI capabilities in the last year or so, they're all focused on benchmarks, right? So why, for example, are we announcing, hey, our model can do really good at this high-level high-school math exam, as opposed to saying our model can do really good at solving these technical problems that are insanely lucrative for this company, right?

Like, why is it focusing on abstract benchmarks? And one of the things the source was saying is because, well, the actual, exactly what you're saying, the actual problems you're tackling and trying to solve in real companies are, they're complicated and they're really bespoke to those companies because of tape. like how do I have taste about like what is the right lead for an Atlantic article that's going to be sort of like a 4,000 word idea article? That is super bespoke to like exactly that context, right? And it's not profitable that try to say let's spend a lot of time building synthetic data sets and RL training them and tuning them to be like really good at this job that like 17 people have. And so this might be one of the things that is happening is that the taste required to do almost any non-trivial job is pretty bespoke to the position. and it would be very difficult to get a model to be good at exactly that, unless like you really put a lot of resources into,

hey, we've got traces, you know, from Derek from the last five years of everything he's ever written and we've really trained him, you know, but why would that be profitable? That makes no sense,

right? That's not worth the effort. So I think you're absolutely right there. It's not that like you couldn't maybe build a system to do specific things you do in a way that's like very helpful. It's just that it makes no financial sense for a company to build a system that does the things in your particular niche does where it.

So that might be one of the things that's going on. So I'm with you on that. There's this cliche that you write to figure out what you're thinking. And I think that's more or less true. But as I was listening to talk, I thought, I also write to figure out what I'm not thinking. And that's just as important, right? I think I have a thesis that makes a lot of sense.

And it makes sense in my head, as I'm thinking about it really, really quickly and taking a walk. And then I sit down at my computer and I write out what I think I think, And I'm like, oh, my God, this is gobbledy gook. It makes absolutely no sense when you reckon with it in the form of a written sentence. And I think that this is a good example of or another recapitulation of the idea that AI can be really helpful at being benchmark smart, at providing excellent answers to well-defined questions. But in most companies, it's the questions or the challenges themselves that aren't well-defined. And we don't quite have an artificial intelligence technology to be facile with that.

It looks like you're champing at the bit. So I'll let you jump in. Well, yeah. I mean, I've said, what's the Turing test we should care about in, you know, 2020, whatever? I think it is an AI that could empty an email inbox. So if an AI could empty your email inbox, think about what you would have to master. You have to master like a relatively bespoke sort of economic activity landscape. you would have to master interpersonal relationships that are subtle, like trying to understand,

like, who is this person and who are they in this hierarchy? And if they're asking me to do this, like, what's the right tone to respond to them? But also it's going to require future prediction in a way that a feed-forcedatic model can't really do. I wrote an article about this a couple years ago because you have to say, if I agree to do this, what is that going to generate in the future? How is that going to fit into my current workload and things I'm coming up? Okay, I think I need to defer from saying yes to this.

but this person is in this position of authority, so I got to be pretty, you know, I have to demur in the way I do it so that it's socially appropriate. You can answer an email, clear out my email inbox. Like, now we have AGI. Like, to me, that's AGI.

And that's not at all what you're getting, I think, by, yeah, instead having a training, a model on the entire internet for text production. But the other, I wanted to also just quickly, you know, point out,

go back to the thing you were saying before about what happens when you write and how that's where like real thinking happens, right? Because I want to just touch briefly again on the art of thinking and how this is, it's supposed to be hard. Just like if you want to get a good shape, it's your muscles are supposed to feel uncomfortable. You're supposed to be lifting heavy things. One of the things I found in my research on that writing paper is that when you're actually putting words onto the page, I thought this was really cool. One of the parts of the brains that actually gets activated is the part of the brain

that's used for spatial reasoning. So it's the same part of the brain that gets activated. So it's the same part of the brain that gets activated if you're trying to keep track of in your head where things are. So you can do this in MRI studies, but they have you trace patterns with your hand on a particular type of board. And you can see, oh, there's just part of your brain where it tries to keep track of things in physical space. Writing hijacks that part of your brain. And it hijacks that part of your brain to make a physically intuitive structure of the argument that you're writing. It uses something that is supposed to be used in your brain for keeping track of items in space. And part of what helps you figure out when you're writing if an idea makes sense is that

that part of your brain is basically building an abstract structure. And this piece doesn't fit. It doesn't work. These are the pieces that fit together to make this argument make sense. This is hanging out over here. And then this doesn't make sense that you're putting this piece here. Oh, I didn't really think it through. So it's almost like a physical metaphor is happening in your mind.

I mentioned that example because this is the type of. stuff you're working out when it's a blank screen and you're putting sentences down. It's exactly the type of training you begin to lose when instead of going through that exercise, you have chat GPT give you like a bad draft. And then you kind of look at it and are like, hey, can you tighten this up? Or are you sure that makes sense? You're not training that thing at all. And there's a dozen other things like that.

It does another mechanisms like that going on. But man, it's such an amazing art what happens when the human brain does thinking. From the very earliest days that we had people doing, you know, philosophizing as an actual activity, go back to ancient Greece, what were they rapidsizing about? Man, humans can do this type of complicated symbolic thought. I mean, this is the whole like Nicomachean ethics, Aristotle, ends up with, this is kind of what we're supposed to do at humans. The main thing that separates us is like deep contemplation, structured thought, no other animal can do this. We are so good at it. And it takes a lot of practice and it's so cool to get good at it. So it just made me think about that.

that why do we want to, we should be wary at least, about like taking those practice reps off the table. I love that. I also love the inbox zero tests, replacing the touring test. I think that's a brilliant idea. Just to pause, strangely, on Aristotle, are you sometimes, like, just aghast this guy with,

how many books could he have possibly had? Like, how many books were there in, like, 300 BCE Greece? There's no way that he had access to a fratist of the knowledge that we have today. And I remember reading him an introduction to philosophy, and it's like, you know, he's not right about everything. Like, he doesn't understand gravity. He's not Isaac Newton.

But, like, the ability of some of those ancient philosophers to be so voluminous and daring in their thinking and be such polymats, I think it's, I think it's, like, astonishing. And it sometimes makes me think, like, was there a way of thinking that has been lost to antiquity? I am not one of these like bronze age pervert kind of guys who's like, you know, let's all go back to the Bronze Age or the Hellenistic Age. But I do sometimes wonder, like, is there a modern mind that could write Plano's Republic?

Is there a modern mind that could write the complete works of Shakespeare or Aristotle? Because like those do seem to be singular achievements that I'm not sure I see on any given day these days. Yeah, I think a lot of what's going on. guess is, I'll be really reductive. Comfort with cognitive discomfort, right? Like, this is, this is such a key part of trying to extract the best value out of like whatever circuitry you happen to be bestowed about, right? I wrote an article last winter about my experience in the theory group, the theory of computation group at MIT, where I did my doctoral training. And it was an article that was titled like learning how to think, like what I learned being in this environment

And it was one of these places. There's not very many of these left, but it's one of these environments were like a tier one skill. The tier one skill that mattered was what everyone talked about, what everyone cared about was like your ability to think. That's how you made your living. That's how you got ahead?

This was the only thing that mattered is how good of a thinker are you and what does it mean to be a good thinker? And they thought a lot about thinking and they cared a lot about thinking. And one of the things that came out of that, that environment was comfort with cognitive discomfort. So it feels uncomfortable when you're trying to hold things

in your working memory. you're yoking together different parts of your brain, and then you're holding that in stasis while you try to rotate something mentally in your mind or move yourself around an argument. It uses a lot of energy, it's strain. It's not what our brains were evolved to do because we're not involved to be system symbolic thinker,

so we feel like a real feeling of resistance in it, and the great thinkers can sit in that, and they can hold that, and they can move with that. I think it was easier to be comfortable with cognitive discomfort 2,500 years ago. Because there wasn't that much else to do with your mind. I mean, like, life was, like, pretty rough and boring. It was equal.

I mean, what could Aristotle do? You had the grove outside of Athens, and then, you know, you would, it was very peripatetic. You would walk around with other thinkers and you would think and you talk, and it was like your very best entertainment. So, like, why not? Like, let's go and do it. There was, like, very little else to do.

The Pythagrians would, like, sit around and try to think so hard that they had mystical experiences. And that was kind of it. Like, that was their cult, you know? And so, right, I think partially there's probably a lot of genius being left undiscovered because it has to be the combination of, like, your brain formed in some way that's capable of doing aristotelian thought. Plus, you develop the ability to take advantage of that, which meant you got really, really good at just sitting in difficulty, sitting in strained, sitting in what it takes to pull together new thoughts or to master something you didn't understand before.

So I don't know that we're less capable from like a brain wiring perspective. I think it's 100x harder, however, to find those needles in a haystack. We should have a way, you're right. We should have way more Aristotle's because the population is like a thousand times larger

than it was back in that time. And we have a knowledge economy and it's much easier to have time to think. But it's way more rare to see a thinker of that caliber. So maybe that's what's going on. We're missing. Our hit rate at geniuses figuring out their geniuses is way worse than it probably was back then.

Well, it's fun to think that people are familiar with, obviously, the concept of physical fitness. I'd never thought of a concept like cognitive fitness that would become more flabby as we get worse over time, especially throughout modernity, at working out our ability to engage in deep thought and deep work, right? I mean, you've written about this a lot, but this idea that a particular kind of cognitive fitness, a facility with, I forgot if you put it this way, but disfluency, not fluent thinking, thoughts that come easily, but disfluent thinking, thoughts that come hard. A facility with that is something that you're right. It might have just been more common before it became so easy to just pull up the screen and dump our dopamine at it. Before we go, we've done a lot of criticizing.

I want to make sure that we end on a recommendation note. We've been talking about the bad. AI is also amazing, and it does something spectacularly well. If I need a 10,000 word essay about Hootsmalley tariffs of the early 1930s to understand Great Depression tariff policy from several different angles, I can whip it up in five minutes. And that's pretty extraordinary compared to how long certain kinds of research would have taken for me to go to the library and track down the book and everything else. I'm wondering, not at any high philosophical level,

because I've really enjoyed the high philosophy, but let's get down into the straight practicality of it. How are you using AI as a writer or worker, if at all? Yeah, I would put, yeah, it's a good question, because I don't use it much professionally. I mean, I use it in the sense of for articles, messing around with all the different article, different models of this or that.

I've tried to use it in a lot of different ways and have been come away very wary. So I use it in my personal life as a better Google. If there's specific information I need that is out there, but it's kind of annoying to synthesize. That's how I use it, right? I do some, like, electronics projects with my kids. It's really good.

Here's like a perfect chat GPT question. Hey, I'm trying to wire up like this type of string of lights, you know, what gauge, it'll be about this much voltage, like, what gauge wire should I use to be safe there? That information is out there, but you're going to have to go to a couple discussion boards and be like, okay, I think it's this. And it'll just answer that right away. I've been more wary about using it for research because I think on the other hand, it's this interesting, we're in this interesting self-reinforcing cycle where, yes, there's certain research that it can get you faster. But it's sort of more of a modern thing, reality that we need this research faster. that like I was thinking the other day

I was rereading Lewis Mumford's Technics and Civilization, the sort of like great early book and technology theory and studies that came out in the 1930s. And he has a great author's note where he talks about writing this. It took a long time.

He was like, oh, there's not a lot of sources on the history of technology in Europe. So I went to Europe and it took two years. There's these small museums that they have in these small towns where they have these old mill stones

and stuff they're proud about. And I pieced together like the whole history of, you know, technology, and probably doing that over two years helped him form his theories, right? Or this is like, you know, David Grant loves nothing better than just being in the archive and just turning those pages and then just getting in. I want to learn all the nautical terminology, and I'm going to spend six months doing that. But that also is probably the time there is not that bad.

So it does help you find things faster. It's burned me before. There was an occasion, and I will own this, but it's burned me before. There's an occasion where I was working on an article, and I don't use, I really don't use it for research. But we were crashing a deadline, this thing, we're trying to get it out. And my editor was like, hey, can we add in a quote from this story you were talking about? And I was like, yeah, but I left the book at my house, and I was at my office down the street. And I was like, oh, well, this story is all over the internet.

So I'll just grab it off there. In fact, chat GPT can grab it for me. So, hey, can you go find this story and give me the quote about this? And it was like, yeah, here you go. Then the fact checker calls like two hours later. I can't find this quote, right? Like, what version of the story are going on? And I go and I get the book and I'm like, oh, my God, it just paraphrased.

You know, so it kind of burned to me on that. I was like, ah, lesson learned. Like, you know, even when you think like this information is out there. So I don't know. I've tried deep research, but I like going a little bit slower on the research. I think turning the pages sort of. sort of helps me process it and imbibe it. So I use it for informational questions.

I use it for grammatical stuff all the time. I think it's very useful for that. It's very good at sanity checking descriptions of how AI works. So it's very good for that. I feel like, is this an accurate description of a transformer architecture? Because there's a lot of text on that and it's technical and it's good at technical text. So no, I don't use it a ton professionally. But I agree with you that, like, this is.

is a very cool technology. I think Google search is a $175 billion a year business, right? That was the 2023 numbers. And this does things what's better than Google for certain types of questions. So, like, that's nothing to sneeze at. I think that is a huge use case. Computer programmers, I don't do a lot of programming anymore, but they really love it. It really is great to be able to integrate this into the places where you type code and not have to go search for stuff in other places. I know a lot of people who love vibe code and that, you know, whatever, for, for small businesses they're in or for their own hobbies. It's just fun to be able to like make a program from scratch. It's just like a Steve Jobs sort of late 70s early personal computer type of

energy in it. I think that is really great. I find some of the like interactive brainstorming stuff to get a little parisocial. To me that creeps me out a little bit. It's like you're so great and what a great idea. And that kind of creeps me out a little bit. But as like a great Google as like a programming aid, I mean, those things right there are nothing to sneeze at. Those are huge, markets and it's hugely useful. And I use it for that. And I'm excited for, I'm excited for breakthroughs as well. Cal Newport. Thank you very much. Thanks, Derek.