CoRecursive: Coding Stories - Story: The History and Mystery Of Eliza

Episode Date: July 5, 2022

I recently got an email from Jeff Shrager, who said he'd been working hard to solve a mystery about some famous code. Eliza, the chatbot, was built in 1964, and she didn't answer questions like Alexa or Siri. She asked questions. She was a therapist chatbot and quickly became famous after being described in a 1964 paper.  But here is the mystery. We're not sure how the original version worked. Joseph Weizenbaum never released the code. But Jeff tracked it down, and some of the things we thought we knew about Eliza turned out to be wrong.   Episode Page Support The Show Subscribe To The Podcast Join The Newsletter

Hello, this is Co-Recursive, and I'm Adam Gordon-Bell. Each episode is the story of a piece of software being built. Recently, I got an email from somebody who said he'd been working hard to solve a mystery about some famous code. Why don't you tell me your name and what you do? My name is Jeff Schrager. What do I do? I can never answer that question.

It depends which party I'm in. So if I'm in a party full of computer scientists, I'm a psychologist. If I'm in a party of psychologists, I'm a computer scientist. And if I'm in a party of, I don't know, philosophers, then I'm a linguist or something. The mystery Jeff was working on was about Eliza, the chatbot.

She was built in 1964, and she didn't answer questions like Alexa or Siri. She asked questions. She was a therapist. If I said I was worried about my family, she'd say, tell me more about your family. She was based on the work of the psychologist Carl Rogers, whose method of therapy was just to reflect back to people what they said to him. It's kind of a neat hack, right? Just say, tell me more about that repeatedly until the person works the problem out themselves. It's a little bit like rubber duck debugging, where you describe your problem to an inanimate object.

But anyways, after the paper about ELIZA was published, what people thought they could expect from computers changed forever. And partially it was because of a trick of human perception, like that we impute consciousness into things whether they deserve it or not. But also, this was a time when microwaves seemed impossibly futuristic, so people were right to be amazed. ELIZA was created by Joseph Weisenbaum, who is a professor at MIT, and published a paper about ELIZA. The paper had transcripts of conversations with ELIZA in it. And from there, ELIZA spread. There are knockoff versions of it everywhere, even in Emacs right now. But here's the wild thing. Here's the mystery. We're not totally sure how the original version worked. They didn't publish the code. They published the algorithm. And they prided

themselves, the computer scientists at the time, of describing the algorithm, not GitHubping the code. These days, we don't. We GitHub the code. You want the algorithm? Here's the code. Sometimes we write an algorithm that's highly mathematical, okay? But there was no code. Possibly, Weizenbaum and his students were the only people who ever actually saw the code. Maybe it did more than we thought it did. Maybe we've been judging it wrong. So that's today's episode.

We're going to find out. Because guess what? Jeff found the code. But first, to understand the history of the true ELISA code, we have to understand Jeff's connection, which starts when he was 11 and he got access to a computer at his school and built something called an inference engine. Let me be clear. The logic inference engine was not a complete deductive inference engine. It was a 11-year-old's version of an inference engine when he had no idea what he was doing. Being a genius is something I definitely do not claim. You know, bad basic software engineer at 13, that I'll buy. But this was the time when people weren't learning to program, you know,

JavaScript in the womb. So it was a little bit unusual. This was the early 70s. Computers were rare, but Jeff got hooked and eventually even his dad started programming. His dad was a doctor. He wrote a program that he didn't understand variables, but wrote a program to actually diagnose heart disease. And so all of this is actually in the AI vein. It would say, so how old were you? Right. It's sort of like a decision tree, right? You know, how old are you? Blah, blah, blah. But he didn't actually understand how variables worked. And so later on, and this is brilliant actually, later on when it needed to use your age again, it would say, oh, you know, I've forgotten. How old did you say you

were again? He couldn't, he didn't save it in like an A variable and then reuse it. I never could get that quite through his head. I still can't quite get variables through his head, but it was brilliant because it was conversational, right? It's the kind of thing that you would normally ask, right, if you actually forgot something someone had told you. So it was actually much more realistic than a boring decision tree. So the timeline's a little fuzzy here, but in this environment in the early 70s, Jeff is obsessed with computers and AI. 2001, A Space Odyssey, that movie with Hal, the AI, is a big hit from a couple years earlier. And everyone is talking about this ELISA program. There was quite a bit of stir about AI, of course, all through the 60s. And ELISA was a known part of

that stir. And then Jeff reads some ELISA transcripts that were republished somewhere. And he thinks that maybe he could build something like that. So I built it in 73. So I would have been 13 or 14, plus or minus, depending on what, you know, dates. I remember engineering the parsing algorithm, which to an adult would be trivial, trivial string parsing. But to me to me was really cool. So what

it does is, you know, one of the, one of the cool things about ELISA is that it talks to you. And so sorry, it repeats back to you. This is sort of one of the deep brilliances of ELISA, which sort of harks back to my father's thing, which is that it takes what you said and, and says it back to you. We still haven't got to discourse computing, right, in almost 60 years now. And part of the reason is that you need a very context-aware version of an LSTM, right? You have to basically be able to remember what was said and respond to it and bring it back up again and things like that. I forgot what age you said you were, right?

You could say something like, I forgot what age you said you were, but you were probably in the, were you in the 50s? Something like that. Anyway, so Eliza had to do that. And so in order to do that, you actually have to turn around the sentences. You have to do a little bit of grammar, right? You have to say, if you say, you know, I'm afraid of cats, it has to say, how long have you been afraid of cats? Building this reflect the statement back as a question part was hard. And that was the part that I was like so proud of, that it would do that little piece of grammar. It's kind of like the little piece of logic from the logics program.

Right. I got it to do this little cool piece of quasi intelligence. Right. And I remember that very clearly. I actually kind of have a flashbulb memory of sitting there and writing the STR dollars that would pull that out and would find it, find the spaces and get the numbers and, you know, all of that. And today it'd be a regex, right? It turns out to be a very complicated problem.

And my, of course, basic solution was trivial. Like when you were making it, if you recall, okay, you're going to, I'm going to show this to my dad. It's going to blow his mind. He'll think I'm talking to a person. Like what, what did you envision? No, no, definitely not. I just wanted to do it for fun. I mean, you know, I was a 13-year-old hacker. Why do hackers do things? For their own amusement, right? I don't really remember. I definitely didn't want to trick anybody or

I'd probably show it off to my friends maybe. I honestly don't remember why I did it other than I was vaguely interested in hacking symbolic AI and still am. And, you know, there was no GitHub, right? There was no place to put it, right? So Jeff just put the code away and moved on with his teenage life. High school finished and he goes off to Penn State to the electrical engineering department. And it turns out it's an important place in the history of computing. We had the EN engineering department. And it turns out it's an important place in the history of computing. We had the ENIAC.

They built the ENIAC. And in fact, the wall, the opposite wall of my office, it was like, you know, old buildings and whatever the opposite of the concrete wall was, was pieces of ENIAC. The ENIAC was basically the first electric computer. It was a military project, but it was built at the University of

Pennsylvania and came online in 1945. It was used to calculate missile trajectories, and eventually it ran the calculations behind the first hydrogen bomb. When it was first turned on, the press called it a giant brain and bragged about how it could calculate as fast as 2,400 human computers, human computers being the women who Penn had previously employed to perform the trajectory calculations. The ENIAC, this big brain that impressed the world and its history,

it should be a future episode of the podcast. But anyways, it was hit by lightning in 1955 and it went out of operation, but there was still a big room for it at Penn. And in fact, we used to make it a tradition to go into the locked room with the pieces of the ENIAC and steal tubes. Now, my guess is none of those tubes were original, but, you know, by 1975. But anyway, we would steal them anyway. Don't tell anybody. I think it was probably well understood. So I have one somewhere in some

box. I have an old ENIAC tube, but I'm sure it's, again, not original. Who knows? So at Penn, the world of computing was changing, going from punch cards to network timeshare computers. Penn had just gotten on the ARPANET. In fact, one of my summer jobs was tinning the ends of RS-232 wires for the ARPANET. And in this new world, BASIC was not the cool language. FORTRAN and something called LISP were where it was at. John McCarthy was at Stanford and he was pushing the concept of LISP and AI pretty hard. And

someone else on ARPANET, Bernie Cosell, seemed to have ELIZA running on a machine in the original LISP, although maybe we'll find out that wasn't the true ELIZA at all. But anyways, he had it running on some machine at Raytheon where he worked, had it running on ARPANET, and Eliza spread. At MIT, the main computer science system, ITS, they hooked Eliza right up to the default ARPANET connection. So if you tried to connect to ITS, you'd get, hello, I'm Eliza. I'll be your therapist today. And so this idea of talking with a computer and this Lisp version of Eliza knocking around, it really got a lot of people in computer science schools

excited about AI and excited about symbolic computing using Lisp. Pen even had its own copy of Eliza, and all of this reignited Jeff's interest. So I probably saw the Pen one and said, oh, this actually is very famous. I should send Creative Computing my old one, and I just basically packed it off and sent it. Steve North rewrote it and they published it. Creative Computing was a magazine and they published programs in BASIC that you could type into your computer. I think it started in 74 or 5, I'm not

sure. But after I would have written it, I said, oh, well, maybe I'll like send them my ELISA just for fun because they were publishing BASIC programs because now they're microcomputers and people were, you know, writing BASIC programs and publishing them in Creative Computing. I said, okay, well, what the hell? So the details of the publication are lost to Jeff's memory, but it seems like Jeff exchanged some letters with Steve North at Creative Computing. Who was the author of record on the byline on the article, even though in the comments it says it's my ELISA. I'm not trying to claim that he stole credit. He rewrote it into MS Basic. So I had written it in whatever the hell

timeshare basic it was and, you know, just sent it in like that. And he rewrote it in MS Basic and then published it. And then the funny story is it started because it was in basic, as I said before, like everybody started using that one because now everybody had personal computers and it ran MSBASIC and they could, you know, type it in.

That's the way, that's called downloading, which we type it in, right? And I still didn't, I still don't think I had the sense of, you know, that this was an important thing to do in a scientific archival, God knows, sense. I was just a, you know, slightly older hacker at that point, right? But it did show up in, whenever it showed up, 77.

And after its publication, it seemed like everyone with MS Basic who had that magazine, they started typing in Jeff's version of Eliza. And that's why Jeff is actually integral to this story. If you played with an Eliza bot, there's a chance it was actually based

off Jeff's code. But yeah, chatbots are fascinating. We are set up to communicate, to learn via talking, via discourse. And when people feel understood by a machine, that can be a powerful thing. But that feeling might be more about humans than about computers. In Jeff's undergrad days, all this talk of big brains and the power of 2,600 humans and one giant machine of vacuum tubes and the talk of how it made people think that something amazing was happening. This power could even be used for deception. One of my best friends, continued best friend from University of Pennsylvania was a fellow named Eric. I won't reveal his last name, but he was a brilliant musician and a theoretical chemist,

composer and pianist, really, really good. And a very good friend of mine, we lived in the same like sub dorm, whatever you call it. And I got him one day, I was talking about Eliza for some reason that I don't remember. And he said, you know, can I see it? And so I brought him down to the computer room. My Eliza was long gone as far as I was concerned. We did have an Eliza, but I didn't know how to run it. So what I did is I hooked up to chat terminals, right? But he didn't know it. And so he would talk to it and I would respond, okay? And he thought he was talking to a computer.

And he started teaching it about music because music is really the thing that was most dear to him. Well beyond anything, actually, I would say. So he starts telling it about music and I'm responding very flat, like, you know, Eliza-like, beyond anything, actually, I would say. So he starts telling it about music and I'm responding very flat, like, you know, Eliza-like, you know, blah, blah, blah. And he starts teaching it do, re, mi, fa, sol, la, ti, do. And I, it's called something in music that I don't remember. They're not notes, but whatever they are. And he would say, then you put them together in a, in a series like this. And he went do, re, whatever he did. And I said, I went like, you know, fa, fa, fa, fa, fa, or whatever. Okay. And I said, I went like, you know, fa-fa-fa-fa-fa or whatever, OK?

And he goes, yes, yes, like that, but now vary it. And then I'd go like, fa-re-mi, fa-fa-re-mi. He said, yes, exactly. And I'd say, is that what you call composing? And he said, yes, this is what you call composing. Yeah, and he was totally excited. I'm sitting right behind him on another terminal typing this stuff, right? He thought I was doing something else random.

He thought he was teaching an AI music, like first person history. You know what? In cons, they have the thing called cracking out of turn. He thought I was doing something else random. He thought he was teaching an AI music, like first person history. You know what? In cons, they have the thing called cracking out of turn. So then I made a mistake. I cracked out of turn, right? I said, fa out. F-A-H, you know, like do, re, mi, fa.

I took the fa. I said, fa out. And he turned around to me and said, this isn't really. It couldn't have made a joke. He was really pissed off. No puns. Yeah, right. Yeah, puns would be, actually anything that went on that night would be well beyond anything we have even now.

This is what's wild about chatbots. So much has moved forward, but not discourse, not discussion. Talking with a computer program is still in this uncanny valley. Anyways, after Weisenbaum built ELISA, there was a colleague of his who was working on a model of paranoid schizophrenics, Ken Colby. So Weisenbaum was credited with writing ELISA. Colby was actually an MD interested in AI, but Colby had written this thing. He had tried to simulate a paranoid mind in a famous 19, well, nobody knows it except me. And then they like claim each other stole the other one's work or whatever it was. Colby was a psychiatrist. Colby was trying to, you know, work on paranoia, but they were like modeling the psychiatrist.

And one of the things Colby says at the end of his 1964 paper is, you know, we want to build software that's going to use our model of paranoia and try and teach it, you know, to improve it, right? In order to fix it. Like we're going to model psychiatry so that it can fix paranoids. First, we have to model paranoids, and then we'll build something to try and fix them, right? Now, this isn't so crazy. Like we talk about this today, except not with modeling paranoids because we're kind of beyond that, beyond that in the

sense of we've given up. But, you know, we model all kinds of complex systems. We model the physics of cars in order to train self-driving cars, right? Yeah, yeah. But it does sound crazy though, doesn't it? Well, it sounds paranoid. No, not really. Here's how we'll fix schizophrenia. I'll make a bot. That is schizophrenic, right? And then I'll build a psychiatrist, right? Exactly. That's exactly what they were doing. And so Weisselbaum built a psychiatrist. Colby built the paranoia. And by the way, there's a famous conversation between Perry and Eliza. This is kind of the early version of, you know, they make the Alexa, the Google and the Siri bot talk to each other, right? Which is always fun. But these were way better. It's true. This is a real thing that happened, although it sounds so unbelievable. Eliza running at MIT on what seemed to the world like an

impossibly powerful machine and Perry at Stanford on another giant computer. And they hooked them up and they talked to each other. Good evening. Tell me your problems. People get on my nerves sometimes. I am not sure I understand you fully. You should pay more attention. Suppose you should pay more attention. What are you getting at? What do you think? I don't understand your motives. Why don't you understand my motives? And they just went on like this. No breakthroughs happened. And honestly, I feel that like Perry, the schizophrenic, he comes off better in the interactions than Eliza. But here's the thing. They talked to each other using early ARPANET.

What would become the internet? Also, Colby actually did this neat thing with Perry. He had real therapists talk to Perry, telling them it was a patient who, for some unexplainable reason, was only available over a teletype computer. Maybe a paranoid schizophrenic with access to one of the few ARPANET terminals that existed at that time. I don't know. And then he had real therapists talk to real paranoids. And then he took those two transcripts to a third party,

to a therapist who was a judge and had them try to pick which one was the fake. This was the paranoid schizophrenia Turing test. And guess what? Perry passed. Turing test defeated. Or sort of. Can you count this? I don't know. I don't think so. Anyways, back to the ELISA code. You see, the thing is, personal computers were starting to be a thing. And at Penn, where they have the ENIAC, of course, everyone's using Lisp and Fortran. But out in the world were Apple IIs and other early personal computers, and they all ran BASIC. Certainly, my BASIC version was replete on Apple IIs.

And by the way, infinite numbers of knockoffs is my BASIC version. At some point, it ends up on the early IBM PCs? Yeah, same route. David Aul, who was the editor, founder and editor of Creative Computing. So Steve North and David Aul did it together. And I don't know the history of that in detail, but they were the two people associated with it. So David Aul later went and collected the best stuff, Creative Computing, okay, into a book of the best computer games or whatever, best basic computer games. I forget what it's called. I think that came with a floppy disk. And so it, you know, you could run everything

in the best of computer games. And so everybody got it from that point on. You didn't have to subscribe to Creative Computing. You just had to be interested in computer games. And because there was the basic computer games from then on, it was everywhere essentially, right? Along with Hunt the Wumpus. I'm not familiar with that. Well, it's, I'm kidding. It's an early version of a Top View dungeon crawler. And from then on, Jeff's code was everywhere. Everyone was copying his version.

Because they couldn't read Lisp anyway, and they didn't have access to Lisp, but everybody had access to BASIC. So they, you know, they copied mine. And you can always tell because all of my knockoff Elizas, first of all, they suck compared to the original. And they always have 36 responses because mine had 36 responses. Okay, and they copied it exactly. Jeff's code was even ported to Lisp, which is funny because there was already at least one Lisp version in circulation. But for Jeff, what was most interesting about ELISA wasn't the AI part.

It was something else. So language was always the big mystery and remains. It was the big mystery. It remains the big mystery. We could make computers behave. We could make robots balance sticks. Back in the 80s, we could do that. Actually, almost in the 70s.

I don't care if a computer could play chess. Obviously, a computer could play chess. Obviously, a computer can play chess. Obviously a computer can play chess. Obviously a computer can play Go. Obviously a computer can drive a car. Okay. It's just a matter of getting it to do it. What's cool is that humans can play chess, play Go and drive cars. Right. That's really interesting. Big enough computer, sure, it could do that. Okay. And there's lots of interesting sub-problems in that. Okay. So I was never really an AI person. I was really a cognitive scientist, right? And discourse, discussion, the interactivity is what science is all about, right? It's what any kind of daily life is all about, but especially science. You're interacting with the domain.

You're like having, in a sense, a discourse with the domain, with the thing that you're working on. If you had models that you could build simulators or enough labeled data, you wouldn't need science, right? You could just, you know, statistics would give you the answers to your questions. So taking data, doing state-of-the-art science is a, it's the learning version of a discourse problem, right? You're having an interaction with the world. So Jeff decided to leave computer science and go to Carnegie Mellon University and study psychology. Now, going to CMU psychology isn't exactly blowing off computer science, considering it's a very small school, most of which is a computer science department. And like everything there is infused by computer science. And Simon Newell, who was the,

like Simon and Newell are the godfathers of cognitive science and AI. They basically invented those fields. And we're at CMU across the, you know, like literally you could throw rocks from one to the other's windows. So Alan Newell and Herbert Simon, they were Jeff's advisors. And along with Alan Turing and Marvin Minsky and John McCarthy, they were considered the fathers of AI. Simon and Newell, among other breakthroughs, created IPL, the direct predecessor of Lisp. IPL introduced new concepts like recursion and the heavy use of lists.

And they used this language to prove math theorems. This led to them winning the Turing Award and Simon got something even more renowned. I don't know if this is a story worth telling, but Raj Reddy, who was the department head of computer science, in computer science every year, there's this day called Black Friday. Black Friday is the day that the computer scientists sit around in a room, which we call the fleecing room. It's like a really nice room, fleecing, because that's where you took the government to get money to do demos. And all the computer scientist faculty would sit around and they would decide who they're going to kick out of the program, right? That's why it's called Black Friday, right? It's hard to get kicked out of CMU, but people got kicked out. And this was the one meeting that professors were not allowed

to miss. Now, I wasn't a professor. I was like a student, but I've heard on good authority and I believe this story. So Simon could not make the meeting, okay? And so Reddy said at the meeting, he said, thank you all for being here. I'm sorry, Herb couldn't be with us, but here's an excuse note from the King of Sweden. Right. Herbert Simon was off receiving his Nobel prize in economics for his work on decision-making. And this was at a time when AI and computer cognition was just having a Cambrian explosion. Every field was going to be revolutionized by computers. There was just so many different ideas

that everybody could go off in a different direction. And Simon and Newell at CMU, they were hugely competitive. Simon took the cognitive science part of it, the human modeling part of it, into the psych department, and Newell took the practical, you know, making AI do stuff. But they were very close to each other. But McCarthy was like off on another planet. That's John McCarthy, creator of Lisp, who was at Stanford.

And, you know, Weisenbaum was off on another planet. Colby was off on another planet, you know. And Minsky, from Simon and Newell's point of view, was off on another planet. That's Marvin Minsky, who was at MIT with Weisenbaum, Eliza's creator. And so there were lots of stories about that kind of, you know, things that I won't repeat that some of them told me about other of them in passing. But it was very interesting because there were all these different views of AI, right? Including Hinton, who was yet another one.

That's Jeffrey Hinton, whose work on neural networks and image recognition led to the current deep learning breakthroughs. This group, they were the leaders of symbolic AI, what we now call GoFi, good old-fashioned AI. In a sense, everybody had a love-hate relationship with everybody else. And probably because of all this excitement and all this competition and because of the PC revolution, Eliza was out there spreading, all through the 80s and 90s. And then Jeff made another career change. In about 2000, I dumped AI completely and decided to become a molecular biologist.

Like, I'm a remilking biologist, that's what I want to do. Screw the computers. Now, it turns out you can run but can't hide from computers, but I learned that later. And, like, went into a lab, was pipetting literally and growing algae. And, you know, when I went into biology, I learned about phylogeny and computational phylogeny, which at that point was just kind of coming up because it was just a few years after the genome. And, you know, we were sequencing not quite by hand. We had machines. But in bacteriology or microbiology, phylogeny is actually incredibly important because they change really fast and it really matters. OK, so the phylogeny

of COVID matters a lot. Phylogeny is building a phylogenic tree. It's like a family tree, but of how species evolve, which gets Jeff thinking. If he could measure the changes between all these different versions of ELISA out there, he could build a tree of that. And then wouldn't that be something? This is the really interesting problem here. Looking at the history of how programs come from other programs, the genealogy, right? The phylogenetic tree of programs, right? Which goes through human beings sometimes, most of the time, certainly historically. And it's actually critically important to law and patent law and all this kind of stuff, right? It's like who copied whose code. And there was a big brouhaha between Oracle and Google. And I think Eric Raymond actually wrote a program that tried to code match. And of course, code matching is critical in Git pushes

and things like that. And so code matching in general is an interesting problem. But to develop phylogenies is actually even a more interesting problem to develop phylogenologies of computer programs. And so now Jeff starts collecting versions of Eliza. He's now like Darwin in the Galapagos, collecting and cataloging specimens in hopes that he'll be able to find some relationship and build a tree. Eliza is his beak of the finch. But there's one program out there he doesn't have for his collection. A program that everyone's copying, but that no one's ever seen. I mean, who knows how this thing works? I'm talking about the original. And then Weizenbaum passed away. And it turns out that

MIT's librarians have an archival process. They archive and preserve the works of faculty members. So maybe if Jeff flies out to Boston, he can get access to it. But then COVID happened, all right? And the only good thing about COVID is it made remote stuff much more practical, okay? Not only practical, but required. And so I said, oh, well, I'm still not going to fly to MIT, but maybe I'll ping them and see whether they'll look it up for me. Okay. And they did. So this fellow named Miles Crowley from the MIT Archive Library, like they pulled the box that said computer conference. I looked it up in their like online list. It didn't say Eliza code. It said computer conversations, box eight. So I

said, can you like pull box eight? And we got on a Zoom and I don't know, we're looking for some code. Maybe there's some code here. Okay. We opened the first folder one box eight and there it is. Are they like holding it up to the? Yeah. Yeah. Well, he's got a flat, like an overhead camera, right?

Oh, he's got an overhead camera. Yeah. So he opens it up and there's Eliza. There's the original code or, you know, well, some version printed out on those old printers, like I used to use, right? And all people of a certain age used to love. And that big paper format was the original ELIZA with the exact, almost exact doctor script. The doctor script wasn't a transcript.

It was code. The code ELIZA ran to act like the therapist. But that's really only part of the code base. Then they found more code. Holy shit. That's it. That's what we're after. We're done.

I could die happy, right? And we took pictures of it and then I got a good copy. And it's fascinating. It's fascinating. And we're still discovering things about it. We found many conversations with Eliza that nobody had ever seen before. Okay. Many of them hand edited by Weisenbaum.

All right. Some of these were obviously classed like, you know, overheads, right? Or even better than overheads, something that I'm sure is before your time. Do you know what a ditto is? No idea. Ditto was early hallucinogenic drugs. It was the most wonderful smell. It was a copy machine, but it did it by some complicated printing VOC system, volorganic chemical, and they smelled great.

But anyway, there were tons of these in the files because he was teaching this stuff. So if you were to look at this original ELISA code, there's one thing you might notice pretty quickly. Everyone associated ELISA with LISP, right? And it's not true, right? It was originally written in a symbolic, a LISP processing language, but not LISP. It was originally this thing called SLIP. So everybody, let's say everybody after 1970 thought that ELISA was written in LISP, okay? The reason for that is that the only available version of ELISA was Bernie Cosell. He wrote it at BBN in MacLISP, I guess would be, well, it was BBN Lisp at the time. Okay.

And it ran on probably early PDP-10s or something. So it was the version which got around academically. This was the version Jeff saw at Penn. But that was the earliest code anyone had. Nobody had seen, at least after the publication of the CACM article was, nobody, as far as we know, had the original version. Because Cosell, the source of the quote-unquote, you know, original Lisp version, he had never seen the real ELISA code either. Because how could he have? Because, you know, we didn't publish code in papers at the time. In scientific papers, we published the algorithm, right? So it seems like the Lisp version wasn't the original either.

It turns out that Eliza was written in something called Madslip. And if you were a scholar of Weissenbaum's works, you might have already known this, because he had a previous paper published all about Slip. This is so fascinating from a just engineering standpoint. So Madslip, as I said, was not Lisp. Okay. It was actually explicitly, he, interestingly, he doesn't mention Lisp. This is, remember I was talking about the sort of infighting. Weisbaum doesn't mention Lisp even once, as far as I recall in the slip paper. It's like he made something almost exactly the same

as McCarthy's Lisp, but never mentioned that maybe he had seen. Exactly. And I'm sure he had, right? Absolutely had, right? And in fact, there's a line in the paper that says some people love their homebrew programming languages too much, right? Whereas what we really want to do is put it in Fortran, which everybody uses. Okay. I'm not an academic, but this seems like a huge burn. I won't mention that thing that that unnamed person is so obsessed with, but you know, by the way, I think he's a little bit too focused on his homebrew nonsense. Coincidentally, this is also why everybody thought Eliza was written in Lisp. Snippets of it were published in the original paper, and they looked like Lisp S expressions.

And really, considering the motivations here, it's wild that Weizenbaum's chatbot would actually be something used to show off the power of Lisp, when really, in this competitive little circle of AI pioneers, he was actually trying to kick some dirt at a rival. Anyways, the next thing Jeff found in the code was mind-blowing. But to get to that discovery, Jeff needed to understand the code. And to do that, he needed help. It turned out he wasn't the only one interested in the ELISA code. Unbeknownst to me, by the way, David Barry and the people who really care about computer history had been wanting to find the original ELISA. Anyway, they had looked at my ELISA.

I had no idea. I found this out later. They had actually done kind of a group, what do you call it? A literary criticism. Okay. They're a group of people. It's fascinating. They do software literary criticism. So Jeff brings them to the archives of Weissenbaum and they try to make sense of them. I mean, obviously dittos or overhead slides of conversations are easy to understand, but that's not the implementation. That's just the output. In a system this complex with this amount of antiquated languages, there's really

only one way to understand it. You want to get the thing running. You want to translate it to a modern medium and run it. This developer named Anthony Hay gets to work on that. There are really three, or depending on how you count, four completely different pieces of code. There's the ELISA, which was written in MadSlip. And MadSlip is a crazy program. I mean,

it's like an ALGOL, MAD itself is like ALGOL, except bizarrely, it had these words like whenever. So when we would write these days, when or if, it had whenever. Okay. Now who the hell is going to type out whenever, especially on a card punch? All right. And so you can abbreviate it as W apostrophe R. All right. And so the code is all, if you look at it, it's like unreadable because instead of, they even did it with like end. Instead of end, it's like E apostrophe D. It's like, what the fuck?

Why are you abbreviating end? So the code looks like a whole bunch of unreadable scribble. It's almost APL. Okay. So we translated it into reasonable modern language or at least modern readable stuff. Anthony Hay did a lot of that, but I did some work on it and some others. But in any case, the other thing is, so you remember I said that it was written in Mad Slip. So he built Slip as a set of packages that could be used originally in Fortran and then in Mad. But, but, Eliza used two special functions, which were Slip functions that did not appear in the Slip code. They were nowhere. More missing code.

That's going to make translating this hard. So far, we have MAD, which is Michigan Algorithm Decoder. And in it was written SLIP, which is not LISP, but sure looks a lot like it. And then we have the ELISA script, which contains the specific responses. And it's written in SLIP. But that's not enough to get it working. We still have these missing pieces. So they were called match and assemble. Okay. And really the heart of ELISA was these two slip functions.

One did the matching, the grep version of the regexp. Okay. And one did the rebuilding, the assemble version of the regexp. And the rest of ELISA is just framework for that. You know, sort of it's the workflow. These functions were nowhere. Right. So we found them. So recently we went back. So the, sort of, it's the workflow. These functions were nowhere, right? So we found them.

So recently we went back. So the original work we did was two years ago, but we went back again just a month ago to the archives, to the slip code, and we found these two functions, okay? We also found a hash function that Anthony Hay had been dying for. He was like bothering me constantly

about finding this hash function. And we found the hash function. And so now Anthony Hay is, you know, he could die happy, right? Basically we found his hash function and it is now a yet more perfect re-implementation in C. I think it might be C++. But Slip actually internally used this thing called, unfortunately named FAP. I didn't name it at the time. It probably didn't mean what we're all laughing about, but anyway. And what it was, was you could, you could basically, it was an API to put to foreign function calls from Fortran to assembly. So it had, you were like MAD, SLIP, FAP, and, you know, at the top, the doctor script. And at the bottom, of course,

you know, the MAD system, the operating script, and at the bottom, of course, the MAD system, the operating system is running on. And all of this is documented, but no one had the code. That is to say, it's mentioned. I say document in the sense of mentioned, right? But they didn't have GitHub. So like I said before, right, they never published the code. So we had to go find it. So we went and found it. We found it all the way down, right? So we're now down, like, we've got it down to the 7090 machine code. And Anthony Hay has been analyzing that. Really recently, like over the past, like a week ago, I got his note that he had analyzed it and made it run. They got it.

They got it running at last. And now with the true original version in front of them, or at least a port of it into C++, there are things that start to make sense. One of them is that, now you remember Eliza was named after the Pygmalion character, Eliza Doolittle, right? Who is not related to Dr. Doolittle as far as I know, who spoke to animals, which is an interesting conceptual relationship, but it's like a pun in the literary world that wasn't intended, right? So it was named after Eliza Doolittle. And if you remember the Pygmalion story, the idea was that there was like, you know, some high British asshole was going to teach some quote unquote poor, you know, uneducated woman. I mean, the whole thing is horrible.

Actually, if you read it, as I understand it, it's not quite as horrible as it sounds. They're really like she wins the day, right? And, you know, she's like street smarts and he's just ed smarts. And so she wins. But anyway. I think that Pretty Woman is like a modern take on it. Oh. Street smart.

Like, I mean, they make her a prostitute. She tries to get brought up. Yeah, yeah, that's right. Oh, interesting. So why would you call it Eliza? Because it didn't learn, right? Yeah. But it turns out it did.

This is a big deal. It can learn. Maybe Weizenbaum saw himself as the professor from Pygmalion teaching his robot Eliza. Anyhow, the point is this 1964 piece of code would learn, at least to a certain extent. So if you look at the original code, there's is this 1964 piece of code would learn, at least to a certain extent.

So if you look at the original code, there's a whole teaching piece, which is actually mentioned in the paper, but only in one line. It is never brought up again, and nobody recognizes it, that you could type, and I forget the command, but if you look at the code, you could type like learn or whatever. And then you could type in those S expressions. You could train it in real time as it was going. Totally lost to history, right? But it's a huge piece of the code. One other thing they found was the student of Weissenbaum's, Paul Howard. He made the whole thing programmable.

So you could put mad code, mad slip code into the script. So it actually became a programmable script. In other words, the script could have code in the script itself. So instead of just sentences, you could say, okay, at this point, we're going to modularize this and make it available to talk about relativity in this domain. Okay. Okay. It was fantastic. It was an amazing job. So it turns out Eliza's a bit more like a framework for writing chatbots. The doctor script that Jeff and so many others knocked off, that's sort of like their chatbot hello world. So now Jeff and this group, they have the original Eliza and they set up a website. Jeff's collecting copies of it for his family tree. It's a huge

undertaking called Elizagen, Eliza genealogy, and it's still a work in progress. But Elizagen can tell you, for instance, that Richard Stallman's doctor that's found in Emacs is most likely based on Cassell's version. And Peter Norvig's version also comes from Cassell, possibly via some intermediary. And that random ELISA you see on GitHub, in JavaScript, or Rust, well, if it has 36 responses, it might be the nth generation of the version printed in Creative Computing, that version that Jeff wrote. Jeff always felt that his version wasn't a true ELISA, but really no version was.

And you might say, who cares? Who cares if people got it a bit wrong? It wasn't written in Lisp and it did more than people thought. But the thing is that the impact of Eliza has been huge. The positive impact it had, although some people might say it was a negative impact, I think it was positive, was to make people believe that AI was going to go somewhere, right? It made me believe AI was going to go somewhere. It made a lot of people believe AI was going to go somewhere. It's like, okay, you can prove like theorems from Whitehead and Russell or whatever it was. That was what Simon and Newell were working on.

Okay. And maybe playing chess. It was checkers at the time. Samuel's checkers player, which learned, by the way. But, you know, this was real. This was Hal, right? Hal is, of course, from 2001, A Space Odyssey. The award-winning movie had Marvin Minsky as a consultant, and Minsky was at MIT with Weizenbaum and surely saw Eliza.

2001, I think, was strongly influenced by the ability to actually have conversations with computers that Weizenbaum had demonstrated to Minsky's satisfaction. The most interesting part of 2001 is actually when David Bowman is turning Hal off. Do you remember this? He's floating around in this zero G in Hal's memory banks, and he's pulling memory banks out, all right, in zero G, floating around in this thing. It's an amazing scene. And the cool part of it is that Hal is basically a neural network and he's degrading gracefully. So he starts singing Daisy, you know, Daisy, Daisy, give me an answer true. I'm so crazy over the love of you, blah, blah, blah, blah. I don't remember. It's something about a bicycle

built for two, right? But it starts slowing down as he pulls memory banks and it doesn't just turn off. He doesn't turn how off. It starts slowing down. Daisy, Daisy, right? And then it like reverts to its early education. Starts saying, oh, I want to introduce you to my teacher.

This is Dr. Langley. So for me, and this is modern, right? This is totally modern still. The natural degradation of neural nets, right? It learned, right? This is totally modern still, the natural degradation of neural nets, right? It learned, right? It learned this stuff. It trained its neural nets. A human trainer, a father, a mother trained how.

For Jeff, chatbots are a part of history, but also they're the future because of what they'll hopefully teach us about ourselves. How can an artifact, a computer, reason about things, right? How can it interpret the world as a set of things that it can then reason about? Okay, and take action on and explore and talk to, whether it's the scientific version of talking to that version of discourse or the humans talking to each other version of discourse, right? How you learn about other people, how, how we hopefully have discourse and democracy is we have discourse, which we're not doing right now, right? That's the problem. The problem is we're not having discourse, right? We're, we're like yelling at each other, not saying what makes you think that, right? I mean, in a nice, interested way, like Eliza.

That was the show. But it's not the end of the Eliza story. Jeff is still working to track down versions of Eliza. Actually, Ron Garrett from episode 76, Lisp in Space, he helped him retrieve an old Lisp version from an Apple II floppy. If you haven't heard Ron's story, you might want to check out that episode. But yeah, thank you to Jeff for tracking down ELISA

and showing us how we got it all wrong. You can find out more at ELISAgen, link in the show notes. And as I alluded to, this isn't just Jeff's story. He would like to thank Miles Crowley, the MIT archival librarian who led him through the archives, and also Dave Berry, Anthony Hay, and Peter Milliken, whose work on understanding the code was central to this project. Also the Weissenbaum Estate and Pim Weissenbaum for granting the open sourcing of the archive. And if you want to learn more about the Turing test and how it relates to chatbots, and maybe how we've been looking at the Turing test wrong as well, I'll be talking to Jeff a

little bit about that as a bonus episode for Patreon supporters. Also a link in the show notes. And until next time, thank you so much for listening.