The Vergecast - Lessons from the very first chatbot

Episode Date: August 11, 2026

Before Alexa and Siri, there was ELIZA. The first chatbot ever was the creation of a man named Joseph Weizenbaum, who built the bot almost as a magic trick — and then couldn't believe how many people fell for it. David Berry and Mark Marino, two members of a team dedicated to restoring and understanding ELIZA, explain how Weizenbaum created the virtual conversant, why he came to regret it, and what ELIZA can teach us about life in the AI age. Further reading: Why your Amazon order confirmation emails have become so unhelpful David Ellison’s ready to pull Paramount out of California What to expect from Google’s 2026 Pixel hardware launch event The ELIZA Archaeology Project Inventing ELIZA: How the First Chatbot Shaped the Future of AI From Eliza to ChatGPT: the 60-year history of chatbots Subscribe to The Verge for unlimited access to theverge.com, subscriber-exclusive newsletters, and our ad-free podcast feed. We love hearing from you! Email your questions and thoughts to [e-mail] or call us at 866-VERGE11. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Hello and welcome to the first cast, the flagship podcast of if-then statements. I'm your friend David Pierce, and today on the show, we're talking about chatbots. Specifically, the very first chat bot from the 1960s. It was called Eliza. And the way to think about it is basically as a very simple back-and-forth experience that you might have with a computer. One of the first instantiations of Eliza was this thing called doctor, which would play therapist for you. And it would ask you how you were doing, and you would respond and you'd say, I'm having a bad day. and it would go through and look for specific keywords and ways that you responded and respond back to you.

And it was able to carry on a pretty functional conversation. It remembered things about you. It could ask questions. It could move things along. It could make connections. It wasn't very sophisticated, just a few hundred lines of code, but it worked. It was so effective that it convinced a lot of people that there had to be a human on the other end. It was a fascinating product made by some fascinating people and a group of

of academics and researchers and writers recently has excavated all of the source code of Eliza, and they've written a book called Inventing Eliza about where it came from, who created it, and the lessons it might have for us in the current world of AI. Because, spoiler alert, there are a lot of them. And the way that we think about computers really hasn't changed that much since the 1960s. It's going to be very fun. I'm really excited to dig into it. But first, here's everything else happening on The Verge today. This is 90 Seconds on the Verge for Tuesday, August 11th, 2026.

Have you ever wondered why your Amazon emails are so unhelpful? They just say, like, your wireless accessory is confirmed, instead of giving you anything useful or specific? Well, the Virges Miyasato wondered too, and she found out that it's all about AI data. Basically, if Amazon puts your orders in the email, suddenly Gmail knows what you're buying, which means Gemini can do a better job of steering you to stuff you like on Google shopping.

Amazon says it's also about privacy, and convenience, but this is a company that is also fighting perplexity and others to keep AI agents from taking over any of your shopping experience, because it's very important to Amazon that you go to Amazon.com and accidentally click on a bunch of barely labeled ads. That's the business. Meanwhile, David Ellison, the CEO of Paramount, has reportedly decided to move his company out of California if the state's attorney general refuses to settle the antitrust case challenging Paramount's acquisition of Warner Brothers Discovery. This is all obviously a threat. The case isn't set to go to court until next March,

and Paramount will start owing shareholders about $7 million a day this October. Ellison would very much like to not spend that money. But if you're wondering how serious this plan actually is, Paramount reportedly hasn't even decided where its new headquarters would be, and it would be scheduled to move there in like six weeks. I'm not sure this is a real plan, at least not yet. Finally, we're only a day away from Google's next big launch event where we're expecting new pixel phones, including maybe new foldable,

a new pixel watch or two, and maybe even an air tag style pixel tracker. There's a big event tomorrow night hosted by Trevor Noah, but if it's anything like Jimmy Fallon's weird informer show last year, you can safely skip that one. You can read more about all this at Theverge.com. That is 90 seconds on The Verge for Tuesday, August 11th. Are you a pet owner?

Every six seconds, a pet owner in the U.S. gets hit with a vet bill over $1,000. Always an unwelcome surprise. That's where Fetch comes in. Fetch is the most complete pet-and-few. for dogs and cats, according to consumer advocate.org. You get paid back up to 90% of vet bills at any vet in the U.S. and Canada, with claims paid back in as a little as two days. Go to fetchpet.com slash save right now for your free quote. That's fetchpet.com slash save.

Support for this show comes from Odu. Running a business is hard enough. So why make it harder with a dozen different apps that don't talk to each other? Introducing O-DU. It's the only business software you'll ever need. It's an all-in-one fully integrated platform that makes your work easier, CRM, accounting, inventory, e-commerce, and more. And the best part, O-DU replaces multiple expensive platforms for a fraction of the cost. That's why over thousands of businesses have made the switch.

So why not you? Try O-D-O-4-3 at O-D-O-D-com. That's O-D-O-O-O-O-O-com. All right, let's talk chat. I am joined now by two of the co-authors of inventing Eliza, David Barry, Professor at the University of Sussex. Welcome to the show. Thank you. And Mark Marino, professor at the University of Southern California. Welcome to the show. Hello, thanks for having us. You're the American one. David's the British one. If you're listening, that's a useful way to remember who's who on this one.

Yeah, otherwise we're indistinguishable. That's right. You would never be able to tell the two of you apart, except for the accents. So it's good that we have those. I want to go back to the beginning here. And I want to talk about Joseph Weisenbaum a little bit because I think a lot of the study that you guys have done is of Joseph Weisenbaum and his own feelings about the thing that he created. So give me just a flavor, David, of who Joseph Weisenbaum was. So Joseph Weisenbaum is quite an incredible character. I think that's fair to say. I mean, he was born in 1923 in Nazi Germany. He was Jewish, born to a kind of upper middle class. family, actually. And of course, they start to suffer under the rise of the Nazis. And so

Heath family emigrate to the US in 1936. He was 13 years old. So that early on, he was already living quite an incredible life. He moves to Detroit, becomes Americanized, though he never fully loses his German accent, goes to Wayne University, now Wayne State University, I believe. And there he studied his mathematics because he doesn't feel his English language is strong enough for any other subject. And he was a natural at mathematics. And of course, mathematics in the 1940s coming into the 1950s is really seething with the excitement over digital computation. We've got a lot of interesting experiments taking place. And Wayne State is actually where one of the first digital computers is being made.

Being made by hand. And Weisenbaum is actually part of the team who are actually. actually soldering together and trying to figure out how to make this object. And so he's very much right there at the very beginning of the kind of turn to the digital. He's living the things that we can only read about now and the excitement of that time. But also he's a very fascinating character. He's not a usual technical person. He's got a very well-rounded personality, which I think probably comes out of that German-Berlin sort of sensibility,

where culture is so important to one's life. And so consequently, he has an interest in everything, and he's particularly fascinated by the American notion of the con. A wonderful book called The Con documents. For a Brit, actually quite strange part of American culture, which is that there were entire societies set up just to con people who would build semi-film sets. This was in the early part of the 20 centuries.

So 1910s, 1920s, where they would literally pull people in just to fleece them of their cash. And it was really quite a remarkable thing. And this book, the Khan documents it. And Fies and Ban was absolutely obsessed with this book. So this was the idea that one could build an artifice, and the artifice behind it would hide all of the hiding, the work that was being done to make this artifice work. It's very kind of dramatic, like a theatre, really. theater made to steal money from people.

And so when he's working at Wayne State, he starts thinking about these questions and he goes to work. He works for General Electric, who at the time had been tasked with producing a new computer system, really to save the banking system in America because of the use of paper checks. I don't know if you'll read us and watch us, whatever, remember those. But essentially, that's how we used to pay money between organizations and people. the quantity of checks was so huge that they couldn't be managed by human tellers anymore. So they needed to computerize the system. And Vizumat was very much part of that computerization process.

And he almost certainly would have witnessed the very famous situation where General Electric were asked to give a demo to the Bank of America. The system didn't work. And so they had to do a con, which is that they set up a facade of the computer system. And in the back room, where the system wasn't finished yet, or had too many bugs. People would literally fill in the bits to make it kind of work. The Wizard of Oz demo, right? This is the man behind the curtain in a very real way. Essentially, Bank of America didn't get wind of it at all, and they gave the contract to General Electric

and the rest of its history. Eventually, luckily, I suppose, that system worked and was a great demonstrator of the power of digital systems. And from that, really, that I think inspired Weisenbaum to think about the difference between what we would now call the interface or the surface of computation and what's going on behind. They don't have to be the same thing. And so he starts to already early on start thinking about, you know, what does it mean to present something to a user? And he writes a very famous paper called How to Make a Computer Appear Intelligent in 1962. And he says, you know, it doesn't matter if the system behind isn't intelligent. What matters is that people think it is.

And you can see already the bits of Eliza are falling into place for him to go on to do it. Mark, this is an interesting moment in time that you guys reckon with a bunch in the book is kind of this question of what was Joseph Weisenbaum building when he built Eliza. And there is a sense that he made a therapist, which you prove pretty conclusively is. not true, right? That's the demo everybody remembers where you're talking to a therapist. That was not the original or only conception of Eliza, that it's actually a collection of chatbots. But it also doesn't seem like he was building what he perceived to be some kind of breakthrough user product that someday somebody would sell and make lots of money on, that there was something more philosophically questioning about this thing that he was building. What do you think he was building

when he was building Eliza in those early days? That's a great question, David. I, I think it's complex. So Jeff Schrager, one of our collaborators, he's fond of just pointing to the title of that initial paper in 1966, Eliza, a computer program for the study of natural language communication between man and machine. So there's one version of this where he's doing an experiment to see what happens when people interact with a computer.

I don't think it's as simple as that, I guess. And this is maybe this is why it's wonderful to have eight contributors to a book all in dialogue over the course of four years. Yeah. Because on the one hand, you know, again, Peggy Weil, another one of our contributors, loves to quote the first line of that article, which is, it is said that to explain is to explain a way. So in other words, here's my clever little chatbot. Now I'm going to explain it to you. It's like one of those pen and tellers, you know, magic tricks, right? They do the magic trick.

Then they show you how they do the magic trick, right? And then that's also a bit of a performance. I think where things get really complicated for me is, and then he includes this dialogue called men are all alike. Or that's how we refer to it, right? And it seems to be between a young woman and this doctor script. It's so convincing. It's so interesting.

And it ends in this mic drop moment, right? Where the therapist seems to draw back around the conversation about the father and the conversation about the boyfriend. And it's imagine, you know, at this time where, you know, psychotherapy is popular in the consciousness of America, right? And so also some of our efforts were in trying to figure out how he was the only person who could have, Weisenbaum himself, could have known that that conversation, if you said it in a certain way, could have led to that particular moment. But I guess the reason why I'm calling our attention to that dialogue is that that dialogue to me

says that he was also invested in Eliza and Dr. seeming like it really did what, you know, it could converse, right? And he continues to tell the story about his secretary closing the door, you know, and having a private conversation with it. And so it's like these dueling impulses. On the one hand, I'm going to explain everything to you. On the other hand, I'm going to do the magic trick. And it is a good magic trick. Actually, this might be a useful thought exercise. I think a lot of people watching and listening to this will have had lots of experiences with chatbots. In the 1960s, just put me in the frame of mind of somebody sitting down in front of Eliza. So to give you an idea of what computation was like, because already the computers were

becoming commoditized, right? So Deck had produced the PDP1. And in fact, MIT received one of the first PDP ones as a gift from Deck. And so already people were moving, away from the idea that computers were hand soldered together, they would purchase them sort of pre-made, as it were. But the PDP1, when it was delivered, had no operating system, right? The first thing you would have to do is write your own operating system to use this computer, and then you'd have to write all your tools, and then you might be able to have enough time to write some actual software that did something that was quite interesting. So it was quite an uphill battle to get these machines working. Secondly, I would say,

that they were fairly primitive. So the idea of a screen in of itself was a really new idea. And in fact, most of the interface experience that most people would have would be with a teletywriter. So it was on paper with this clattering, incredibly loud device that you would type your request into the computer. It would clutter away and then give you an answer on paper. So it was very clunky, very slow, very loud.

very large, actually, and quite hot because these machines were obviously burning a lot of electricity. But what I would say to you is there was also a kind of imaginary there as well. I mean, Vannevar Bush's Memex, Linklider, and lots of other people were already trying to think beyond the existing restrictions of the hardware of the time and dreaming of new ways of seeing what human machine interaction could be. And I think that Eliza certainly was part of that because it was part of the project Mac project at MIT, which was explicitly about, you know, how can we use computers in a different way?

I always like to muse on this question. Like, how could people have thought that this chatbot was a person? It's so rudimentary, right? Although, of course, as we've already mentioned, so clever. And I think what David just described of not having experience of anything happening between you and a computer, perhaps for a lot of these people. And then least of all, instead of having a screen interfacing something just by, you know, we're a lot closer to the Turing test imitation game, passing notes with someone who's behind a curtain, then we are anything like we're experiencing right now. So if you get, if text goes in and text comes out, you know, I mean, I hate to harken us back to like a telegram, but I mean, communication is happening pretty much the same way it could happen with another human. And you had no experience of communicating with another human through a computer at all anyway. Right. Well, okay, this is actually the reason I bring this up is because one of the things I have been trying to reckon with for a long time and was thinking about this a lot reading your book is overwhelming. people who sit down and experience this very simple, very basic chatbot, Eliza, fall for it.

They believe in the thing. There are lots of people who cannot be convinced that there is not a person on the other end. The experience works. Like you said, Mark, the magic trick is successful. And I've been trying to decide, is there something about that that is fundamentally about inexperience with computers? And if you just gave that to a bunch of people who had used this kind of technology before, or if it would have been less new and less surprising and less magical,

or if there is something deeply human inside of us that just desperately wants that magic trick to work. And my sense is it's probably a little bit of both, but I know you've both spent a lot of time thinking about this. David, where do you land on this? So, you know, this is a fundamental question. I would take you back to the early part of the 20th century. When you read the cons that were taking place,

you read them with a sense of kind of disbelief that people were taken in. It seems utterly, to me anyway, that rich Americans would visit this fake bank and would be wowed by the fake technology that they were showing and the fake tellers and the fake stock exchange and then would hand over hundreds of thousands of dollars, millions today. and then when the con people ran away with the money, which is what they did, they ran, got to the train station and they'd be gone, the people would like pull back the screens

and it would be paper mashay, balsa wood, and, you know, why would they believe it? I think, as you said, there seems to be a deep kind of desire for us to want to believe. And maybe we believe because it seems like such a good offer. and Eliza seemed like such a good offer, right? You had a therapy time, which was free. I mean, that's a good starting point.

And it listened without, you know, putting you down or attacking you. It was a very nice experience. And it's also novel, right? Kind of interesting to chat to a computer and so on and so forth. So I think many of things play together in the same way that today you hear about people having AI boyfriends and girlfriends and you kind of probably can't believe. And there's a wonderful interview

in the New York Times actually with somebody who had an AI boyfriend and they said you do realize it's a machine. It's like, yeah, yeah. But that's what makes it so good. I can turn it off when I have enough. So we're able to do and this is a theatre,

we walk into a theatre. We know the theatre is a theatre. We know they're actors and yet we can allow ourselves to just become and listen to and experience the play. I mean, I threw two more things into that bucket. One is, it's asking you how you're doing, right, which is a very disarming question and puts us maybe in a bit of a vulnerable state,

especially if we need to express that as probably most humans need to do, right? So that's step number one. And I think we see that in a lot of chatbots. The second, and of course, the design of it is brilliant that it keeps asking questions. Of course, the, you know, chat GPT and all those other frontier models have figured out that trick. They end every sentence in a question. Again, drawing from the expertise of our members, Jeff Schrager is quick to talk about the kind of repair we do conversationally, even to have a conversation with another human. And then maybe a good example might be when you receive text messages from someone who's using speech to text, and they're kind of terrible, right? But then you do all this repair work to put it together. The reckoning with all of this strikes me is the thing that then Joe Weisenbaum goes,

and does, right? Like, he has this real, I think, I get the sense he was surprised by how well Eliza worked, that he's like, I built this thing that seems like an interesting way to communicate with computers. And then he's like, oh, my God, people are pouring their heart and soul into this thing thinking that the computer is a person. And he seems to, rather than do what, what it seems like everyone else did at the time and what certainly everybody's doing right now, which is say, you know, oh my God, I just got rich. This is going to be incredible and technology's They're Scourge McDucking rubbing their hands together. Yeah, yeah, yeah. He has a real crisis of faith in all of this. And before we skip to now, I just, I want to get to kind of where Joseph Weisenbaum got to at the end of all of this process, at the end of all of this thinking, he seems to have a real kind of kind of kind of moment at the end of all of this.

David, where does he, where does he net out after seeing what Eliza accomplishes and does to people? So I think that's quite a complicated question actually, because it's not a single moment of conversion. Remembering he's kind of experienced with the Khan and the reveal of the Khan, he loved this idea of doing the reveal. And he used to play with his children. He would write little programs for them, and then he would reveal that it was actually him that was the Wizard of Oz that was making the system work.

And I think what, firstly, what worried him was when he did the reveal, you know, people are like, yes, Joe, could you leave the room? Stop, stop ruining the experience. So that's the first thing that's kind of interesting. The second thing is that people don't want to have the magic revealed to them. I think this becomes a theme that I want to stay on here. But I think that's right. Even if he says, look, it's a trick, I did it.

Everybody says, no, no, no, go away. It's more fun to live in the trick. Right. Yeah, the worst kind of person is you're in the theater watching a play. And they're like, you know, this is a play. This isn't real. I'm like, thanks. But so that was the first thing, that question.

But I think the second thing was, was that he began to notice that people in MIT were talking about humans as if humans were sort of robots or disposable. And this really concerned him because, of course, it reminded him of the language of Nazi Germany and the language of how the Jews were described by the Nazis, right, as, you know, non-human, not important. And, you know, Minsky talked about humans as meat, machines. And he thought this was utterly unacceptable, actually. And this is a time, of course, with the rising political crises in America. You've got the civil rights movement.

You've also got the Vietnam War. You've got the use of particular kinds of rations of rancers. rational management in government to run the Vietnam War with little understanding of human context and human nuance. And all of this surrounds him together with Hannah Arendt's work that he was a big fan of and deeply read her work. And also, of course, Mumford, Lewis Mumford, another very important influence on him. And the context of the time is so important. was this concern about what was happening in America. What is happening to us? Where are we going wrong? And so I think all this adds up to a kind of a tipping point really, where he says to himself,

I'm actually not equipped to deal with where I am, this place at MIT. I'd wanted to get here the pinnacle of my career. And yet now I'm here, I'm seriously doubting my place. And so very strangely, particularly for a compute scientist, he applies to do a kind of humanities fellowship, actually, where he goes off and studies humanities texts,

a whole range of texts, philosophical, sociological, and political, and eventually tries to put this all together into a book called Computer Power and Human Reason, which is, of course, that really is the turning point in 1976,

which says, computing and particularly artificial intelligence is going wrong. There is something very wrong with the way we are approaching this topic. And I will just add that made him a lot of enemies, both at MIT and across the field. He became a heretic. Support for the show comes from Superhuman. It's summer, which means a lot of us have vacations planned. But there's one part of going on vacation that's not all that fun.

Coming back to work. Your inbox is full. tasks have piled up, and it feels impossible to stay on top of everything. Superhuman Go is an AI chat that's always there when you need it, already aware of what you're doing and doesn't ask you to start from zero. Go, from the makers of Grammarly, works inside the tools and sites you already use, meaning you can stay focused on the most important parts of your job. No new tabs, no switching apps, no losing your place. Say you've just gotten back from that vacation, and you're trying to figure out where to start. You can use Go's AI chat

to summarize a long email thread that started while you were out. Or maybe you have a day of back-to-back meetings. Superhuman Go can help you prep in seconds. This is AI that works for you, not on top of you. Check it out. Find out more at superhuman.com. Support for the show comes from Upwork. Business owners know that sometimes you encounter problems

that can easily be solved with an extra set of hands, but they also know how complicated it could be to find that extra set of hands. Upwork exists to streamline the process. Upwork is a one-stop platform to find, hire, and pay expert freelancers. You get fast access to specialized talent across more than 125 categories. On Upwork, you can browse profiles, review pass work, and get help scoping the role so you can hire with confidence and get started quickly. You also don't need to let the operational team get in your way,

since Upwork handles things like contracts and payments. They also have Business Plus, which gives you, access to the top 1% of talent on the platform. With AI-powered shortlisting, you'll get matched to the right freelancer in under six hours. No endless searching required. Visit upwork.com right now and post your job for free. That is upwork.com to connect with top talent ready to help your business grow. That's upw-o-rk.com. Upwork.com. Are you a pet owner? Every six seconds, a pet owner in the U.S. gets hit with a vet bill over $1,000. Always an unwelcome surprise.

That's where Fetch comes in. Fetch is the most complete pet insurance for dogs and cats, according to consumer advocate.org. You get paid back up to 90% of vet bills at any vet in the U.S. in Canada, with claims paid back in as a little as two days. Go to fetchpet.com slash save right now for your free quote. That's fetchpet.com slash save. Support for this show comes from Odu.

Running a business is hard enough, so why make it harder with a dozen different apps that don't talk to each other? Introducing Odu. It's the only business software you'll ever need. It's an all-in-one fully integrated platform that makes your work easier, CRM, accounting, inventory, e-commerce, and more. And the best part, O-DU replaces multiple expensive platforms for a fraction of the cost. That's why over thousands of businesses have made the switch. So why not you? Try Odo for free at Odu.com.

That's ODOO.com. I wrote down a line frameer book that he drew a distinction between calculating and understanding. And there's a real thread in all of this that I had not really thought of in this way, but is the sort of turning idea that humans are complicated and irrational and messiah. and impossible to understand. And by comparison, computers and math are perfect and rational and thus better. And I think we are in a very real way living in that moment right now, where there is a belief that computers are better than humans, that they are more predictable, that they are more

understandable, and that those are good things and we should thus trust them more than we trust humans. I wonder if you can just pull apart the way that Weisenbaum in particular was thinking about this, that we have computers that are very good. I don't think it wasembaum ever turned against technology. He was never like a, you know, we have to unplug everything and run back into the woods. But he seemed to understand that computers were very good for certain things, but were not to be trusted or even asked to do some of the things that humans are supposed to be doing. How would you say he drew those lines?

Yeah, I mean, again, the context is really important, right? So we have Robert McNamara in the US government who essentially thought you could just import corporate. management around making various kinds of objects into the Vietnam War. He saw the Vietnam War, the problem being bad management. And so they try to implement these very instrumental, these very rational systems into government, and they make it worse. They make the Vietnam War a lot worse. And so the line about calculation and judgment is actually inspired by Hannah Arendt,

who wrote a book that was highly critical of this turn in the US government. And she said they calculate, they do not judge. And of course, this was obviously very inspirational for Weisenbaum. And, you know, that distinction, that really important distinction between calculate and judgment, it's really a humanistic one as well, that there is a specialness to humans in that the way in which we approach something is with a lived experience that we actually care. about the world and we care for each other, we care for the environment. Computers don't by their programming just care for other computers. They don't care for the users that interact with them and they have no care for anything else either. So I think that's a very important

insight he found and a very convincing one as to why we should be very careful about using computers in all aspects of our life. I mean, I think this jumps ahead a little bit, but I think it's interesting to think about the way the Silicon Valley oligarchs cite their humanities background as part of their resumes. At the same time that that's the stuff that's used for the trick and the deception. I mean, the one thing just to mention that Weisenbaum wasn't doing that a contemporary chapwit does do is that he wasn't extracting data from his users and of course building mental dependency on them the way they are today. So there's, again, Weismam may have created one trick, but compared to the number of tricks that are being played on people, again, by, what would you say, weaponizing the Eliza effect? Well, yeah, let's talk about where we are right now because I think, I mean, obviously a lot of the stuff that you guys have been describing is very familiar right now. We are in this moment right now where the novelty effect of LLMs has been so intense that people have had,

genuine good faith discussions about whether these models are alive, which just to be clear, they're not. They aren't. But I think that the fact that we are back in this 60 years later, and again, with a huge amount of computer fluency and history with this stuff that suggests that we should be better at this, we're not. And I think a lot of the stuff that you guys have been describing still applies, this sort of deeply human idea that we want to be talking to other humans.

we want to be heard, we want to talk about all this stuff. Are we just doing this again? Does it really feel to you guys looking at the current moment that we're in that we are just reliving a full circle version of what we did in the late 60s and early 70s? Or is there something different about the LLM moment as opposed to the Eliza moment? Well, if I could pick that up, I think there is something different. I mean, I don't think history repeats itself or certainly doesn't repeat itself exactly. there's no doubt about it that the technologies in LLMs and other kinds of diffusion architectures are novel and remarkable.

And to me, astonishing that the scaling effect seems to be paying off in terms of new models and new systems. I think that's not debatable at all, really. But where we are in a similar situation is these kind of bubbles of excitement around the technology. And AI has been through, this is now the third kind of AI summer, if you like, before the usually inevitable AI winter follows. And the problem today, of course, is that the AI systems and the AI companies are linked heavily to huge debt exposure to... Wait, are you saying winter is coming, David?

Is that what you're saying? Are you saying winter is coming? And it's certainly the case that the finance is out of control. And, of course, there is an incentive on the company. to continue to push it because both OpenAI and Anthropic won to IPO, hopefully this year, and to do so, they need the excitement behind the AI bubble. But nonetheless, we do have to take our hat off to them and say that the technologies they've developed are really rather remarkable.

Anybody who's used either of the models of Chachipit or Claude to, for example, program systems, it's quite remarkable. And there's no doubt that computing and computer programming, particularly software development, is going to be under a paradigm shift, I think. I mean, it's such a fascinating question because, you know, on the one hand, we've had so much, so everything has accelerated and all the vectors coming off of this. So we talked about the beginning, about people having no experience interacting with a computer or even other humans through digital means, or computer.

or computational means. Well, now we all have that 24-7, right? Yeah. Also, we're, I think there's an increase of isolation and loneliness that's going on, right? There's more awareness of mental illness. There's a great, these companies are so much more wealthy than probably even the U.S. government was at the time when it was funding these operations. And so their incentives are so high.

You know, you read a book like Karen Howell's book. about OpenAI and you see terrifying things. A couple of things just to mention. Weisenbaum was hand programming

each, all of these effects. ChatGPT's ability to speak is like from where I said, again, I'm not an expert in this, but an epiphenomenon. Like, oh, look what happens when we increase the parameters.

All of a sudden, it can do this thing. And then there are a few safety people who say, and by the way, let's not do that because terrible things are going to happen, right? Yes. And so thinking about some of the people who started Anthropic and things like this and the security people and all the concerns they've said. And then everything that Weisenbaum predicted, we see playing out in people being diluted, people taking their own lives, people, you know, out of consultation with these chatbots. So I don't, I mean, in some ways, again, And that's, I feel like you read computer power and human reasoning, and you've got, you've got your marching orders, which is, you know, maybe a little regulation at this point would be sane if a society wants to stay healthy.

One of the real prescriptions for how to solve a lot of this is, is better understanding of how the trick works, right? And this goes all the way back to the cons. Even a big part of your project has been, we want to get into. the actual source code of understanding how Eliza work, because if we can understand how it worked, we can make sense of it better, that it actually becomes very important to see the thing. And Mark, I know this is a thing you've been working on a lot, is like, how do we make code explainable? I'm curious how you point that thinking at something like the LLM phenomenon, which is kind of by design, an unknowable black box. A lot of the people who work with and build

and operate this technology profess to not know exactly how it works. And I think I agree with the principle that we should all understand how our computers and how our systems and how our platforms work much more clearly. But I do wonder if that's even possible at this moment. Yeah, I think so. And I'm going to point us back to David in a second. But I'll mention, so we've been building some tools lately to help visualize this, to help break the spell, I guess you could say. One tool I've been working. working on. And admittedly, I did vibe-coded, which maybe invalidates it, but is after someone very close to me was telling me they were consulting their computer on personal matters, I built a little

web app that's called the Oracle that where you can ask this app questions about your future, what you should do, making certain binary choices between, you know, should I keep my love for the theater, get a practical job, and it'll answer you. But then you change the temperature, or you change some of the other weights and you get a very different answer. So just sort of help people think about, okay, this is a mathematical, this is a stochastic process that's getting us to some sort of answer. My first kind of reflection on this is that Vico, the theologian, who was writing in the I think it was the 1600s, he wrote very famously that if God made it, then it's impossible

for us to understand it. because obviously God is an omnipotent being. But something that humans make, we can understand, right? And I think that principle has held pretty well for 400 years, and it still holds today. We humans have made these machines. We can work out how to understand them. Do you think Joseph Weisenbaum, who didn't live to see the Transformers era of LLMs

and all the stuff we're going through right now, do you think he would have used chat GPT and been sort of thrilled by how successfully it pulled off the game that he was trying to play 60 years ago? Or would he be running around screaming to everybody about how the apocalypse has come and all of the things he's been warning about losing our humanity has all come to pass? What do you think he would make of the LLM moment in technology right now? It depends how old he is, I think, when he encounters the technology, right? because, like, again, I hate to put him totally in the sort of chicken little camp, or not the chicken little camp, but the, you know, person who's trying to tell us that there's a gremlin on the wing or, you know, like, I hate to put him totally in that camp because

he, he, I think he would, there would have been a moment where he would have been fascinated by the possibility and would have explored and would have pushed boundaries and would have, and would have maybe even extended the technology. Like so many people who are working for these companies right now, but I think the core would set it pretty relatively quickly. David, what do you think? I think that's a brilliant answer, Mark, to be honest. I think you're absolutely right.

The very early Weisman would have been, would have loved this technology. I mean, it's the ultimate con. I mean, it really is. And there is the potential to reveal the con, to reveal the trick, to reveal the illusion. The mid to late Weisenbaum would be very concerned, I think, at the commercialization of these technologies. Remember, he was working on university research projects to see it used in the ways in which it is being done for AI boyfriends and girlfriends, commercializing mental health, education, so forth, so quickly, with very little regard to thinking about that distinction between calculation and judgment, I think he would be very, very worried.

But I think his book, Computer Power and Human Reason, is still very valid today. the discussions, although the technologies may have shifted, the problems that he identifies, and the things that we as humans have to take account of, are still very, very live. True. All right. Thank you both so much for being here. This was great. I really appreciate it.

Thanks for having us. Thank you. All right. That's it for the show. Thank you to David and Mark for being here. And thank you, as always, for watching and listening. If you have thoughts, feedback, questions, any of that, I want to hear all of it. I think 1960s technology speaks to us today.

really interesting ways and I want to know how it made you feel. Send us an email, Virgcast to theverge.com. Call the hotline 866, Vorge11. And by the way, we'll put a link in the show notes to all of the Eliza archaeology project that these guys have been talking about, including the recreated, interactive Eliza. It's fascinating, and I think your mind

might be as blown as my was by how well the thing does its job. As always, if you want to support everything that we're up to, the best thing you can do is subscribe to the verge, theverge.com slash subscribe. it gets you all of our podcasts, including this one, ad-free, gets you all of our exclusive newsletters, gets you all of our coverage of AI and everything else. Theverge.com slash subscribe.

Thank you in advance. The Vergecast is a Verge production and part of the Vox Media Podcast Network. The show is produced by Josh Kajas, Eric Gomez, Brandon Kiefer, and Travis Thorchuk. We'll see you tomorrow. Rock and roll. Remitly knows if you live a long-distance lifestyle, you probably developed a few unexpected strengths,

like instantly converting time zones. So when a text says, let's talk after dinner in Mumbai, You know they really mean during breakfast in Toronto, or becoming the unofficial family tech support, explaining why Tiarosa in Mexico City can't hear you when she's the one on mute. Can you hear me?

Download the Remitly app today and keep your long-distance game strong. Remitly, trusted by millions to send billions worldwide. Ready to take your investing knowledge to pro-level? This is Fidelity Connect, your daily edge in the markets. Get deep insights on real-time market topics that may impact your investment portfolio. Listen to Fidelity Connects on Spotify today and power your next move tomorrow.