Into the Impossible With Brian Keating - Princeton Scientist: We Don't Understand AI - Tom Griffiths - #553

Starting point is 00:00:00 One of the, I think, interesting challenges we have at the moment is having built systems that we don't fully understand. The man who built modern AI, he's the direct descendant of the man who invented the math that made it possible, which is insane, but it's not the wildest thing my guest told me today. That's pretty much exactly what he was trying to do. And he was the right kind of crazy. Ivan's was trying to invent AI 250 years before computers even existed. Sick of Fancy is a major problem. If you take a... a rational agent and have them interact with a system which is sycophantic, then that agent is going to become increasingly confident in their beliefs, but no closer to the truth. My guest spent 20 years building the mathematics of how minds work, and he just told me three

Starting point is 00:00:43 things that made me question what I thought AI actually was. Now let me show you from a physicist's point of view. Whenever I talk to people about consciousness from Chalmers, Bostrum, and upcoming guest Joshua Bach and others, I always get the same thing. Like we can't really define what, what, what, consciousness is, so how do we know what thought is? So how can you determine what the laws of thought are? Isn't that kind of a extremely prerogative in bulk claim? The way that I approach that question in the book is really by thinking about what are the kinds of computational problems that minds solve. And that's really what this enterprise was. It's trying to figure out, like, what's the mathematical structure that describes the thing that minds are doing, whether that thing is what Aristotle was interested in, which is just trying

Starting point is 00:01:27 to characterize what good arguments are through to some of the questions that you were raising about, you know, like, what does it mean to make a good decision and how do we think about, you know, rationality in that context. And so the interesting thing is, I think a lot of those questions are things that we can answer without ever having to touch consciousness. I think about one of the big challenges of studying consciousness is that we don't necessarily know what computational problem consciousness is solving. That's why it's something that's, you know, continue to be mysterious. We don't really know what it's there for. in terms of, you know, how necessary it is to being able to do the kinds of things that minds do.

Starting point is 00:02:02 And our AI systems give us nice demonstrations. You know, again, some people might want to argue that they're conscious in some form or something like that. But I think they give us nice demonstrations of how far you can get using certain kinds of mathematical formalisms. Yeah, and there's many, many kind of allusions to physics in this book, which is so delightful in many different ways, not the least of which, because it gives us some kind of formalism to hopefully go about this problem. But, you know, as a physicist is want to do, I want to kind of get into what you would say maybe, what is the briefest, you know, kind of most parsimonious, defensible definition of thought itself and the laws that govern it? In the book, I focus on deduction, which is, you know, are sort of like patterns of logical reasoning going from things that are true to other things that are true. Induction, which is sort of seeing a pattern in the world and then making the generalization that thing holds in general.

Starting point is 00:02:55 and then abduction, which is seeing something that you want to explain and then coming up with an explanation for it. And I think that's a pretty good characterization of the set of things that we normally have on our list when we want to try and explain sort of patterns of thinking. And the things that we try and engage with in terms of the different kinds of mathematical formalisms that are explored in the book.

Starting point is 00:03:14 There's an awful lot of discussions of both the successes and our understanding of consciousness in the wrong turns. And I like that because for me personally, I hate when we teach our undergraduates as, you know, as often as done. You know, we basically just teach them the string of Nobel Prize winning experiments and, you know, just connect the dots.

Starting point is 00:03:33 And that's, but you go through the, you know, the twist and turns. And I thought one of them was sort of brought up this conjecture that, or this statement by Feynman, which is that the, you know, kind of the difference between knowing the name of the thing and knowing something about it is the most dangerous gap in all of science. What are some of the inherent biases that science

Starting point is 00:03:55 has brought to it, because it's such a Frankenstein-type field, cognitive science, you know, start off with not really, as you discussed in the book, really being taken seriously, and now it's, you know, at the cutting edge. What is the sort of, you know, largest gap or the biggest lacuna in your field where people seem to maybe be overabundant of confidence in describing how models work or even the model of the brain, let alone models of artificial intelligence? So one of the, I think, interesting challenges. we have at the moment is having built systems that we don't fully understand, right? So we now have these AI systems that for computer scientists put them in a very unfamiliar

Starting point is 00:04:35 situation, right, where if you're a computer scientist, you're used to programming something, because you programmed it, you kind of know what it's doing. And that is not how our AI systems work. So these modern AI systems are built using enormous artificial neural networks, and they learn from data, far more data than any human could actually, like, read through and understand. And so you end up with something where it's both learned from a sort of incomprehensible amount of data and encoded that information in an incomprehensible number of continuous weights inside that system. And so as a computer scientist, you're then stuck and you're like, oh, what do I do with this?

Starting point is 00:05:11 I actually think that's a good opportunity for cognitive scientists, because we have been trying to study large, complex systems that we don't understand for about 75 years now. Those systems are human brains. And a lot of the tools that we built for understanding human brains and how it is that, you know, humans think and behave are tools that we can now use to go back and really analyze these AI systems and try and understand a little more about how they work as well. What would the advent of chat chb-T? What sort of thing would that be like? Is it the invention of the telescope, the cyclotron? What does it represent in your field? I think it's interesting. I'm not quite sure what the analog is because it's both a kind of like breakthrough in terms of revealing certain kinds of theoretical. ideas can take us, you know, further than we might have thought, but also something that's given us a new set of problems in terms of trying to understand what that system is doing and then trying

Starting point is 00:06:02 to figure out, like, you know, what all of its properties are and what the consequences of using those systems in certain kinds of settings is. It's both the validation of a theoretical approach, but also the creation of a new sort of field of inquiry. I talked to Stephen Pinker about his most recent book. We had a conversation about that where humans use these hurricanes. and computational shortcuts. And you bring up a couple of these in the book. And I wonder if you could tell some of the stories of Connemann and Tversky and how they illuminated

Starting point is 00:06:32 this kind of shocking at the time claim that humans are necessarily not the best reasoners, or not as reasonable as we think we are, right? Yeah, so there's an interesting paradox in trying to study human cognition from the perspective of computer science, right? So I live in these two departments. I live in the psychology department

Starting point is 00:06:49 and the computer science department. And in the psychology department, my colleagues think humans aren't that smart, right? If you kind of like study human decision making, you find out that humans have all sorts of simple heuristics they follow that result in systematic biases. And that's the work that Carmen and Devilski did was really kicking that off and giving us this picture of human cognition. And then if I walk across campus to the computer science department, humans are the things that we're trying to emulate when we're building our AI systems.

Starting point is 00:07:15 So there are sort of our best examples of systems that can solve certain kinds of problems. And so I think that tension is about the fact that the way that I would resolve it is that humans are actually good at solving a set of problems that are extremely hard problems to solve. And they're not always necessarily solving exactly the problem that a psychologist asks them to solve when they sort of study them in the lab. So a simple example of this is if you flip a coin five times, which of the following sequences is more likely? Heads, heads, heads, heads, heads, or heads, heads, tails, heads. If you just ask someone on the street, they'll probably say that heads, heads, tails, heads, tails is more likely, right? But, you know, as a trained physicist, you know, the probability of those two sequences is equal, as long as it's a perfectly fair coin, right? Any sequence of five heads or tails is equally likely.

Starting point is 00:08:03 And so one way to understand that, right, that's an error that's the kind of thing you could point to and say humans are irrational, we're biased in this way. But one way to understand it is to say, what if the human is not solving that problem but solving a different problem? So they're being asked to give you, what's the probability of this sequence under a random generating process? What if they're flipping that around and telling you, what's the probability that a random generating process produced this sequence? Or sort of how much evidence does the outcome give you for having being produced by a random generating process? And that's something we can calculate using Bayesian probability. And when you do that, it turns out people's judgments about randomness are very systematic and you can capture them with a nice simple Bayesian model. but that's a case where we're sort of like re-analyzing the problem that human minds are solving,

Starting point is 00:08:49 when you reanalyze it, it turns out people are doing a good job of solving that problem. And in some ways, it might even make more sense to be solving that problem. Because if you're wandering around in the world, it is very unusual for you to have to calculate the probability of sequences of things. But it's a good thing for you to be able to detect patterns that might suggest that something is non-random. And that's probably what our brains are built to do. A central character in this book is a past guest, Noam Chomsky.

Starting point is 00:09:15 And it's always been sort of, you know, kind of curious to me that his, you know, notions of generative grammar and, and so forth, you know, explain a lot from so little or seem to explain why, you know, for example, our children can learn language, you know, with far less training data, if you will, than can computers, these huge data sets with trillions of parameters now. But talk about his role in understanding how, you know, separate from AI, there's a clue to the laws of thought that emerge, you know, that caused the whole field of cognitive science to emerge. But it really is, you know, predicated on fairly elementary questions. It doesn't mean easy or simple. It just means that they're basic and important. Talk about Chomsky's role in all this and whether his ideas are still pertinent to experts like you in the field today. So part of this story about people, you know, trying to use math to understand and thought, it occurs. in the middle of the 20th century, when psychologists had decided that the only way to be rigorous about doing psychology was to not talk about thought and not talk about internal mental states,

Starting point is 00:10:18 right? So this was an approach called behaviorism, and the behaviorists said, you should just focus on the things that you can measure, which are the environments that people act in, and the behaviors that result from those environments. And so there was a group of sort of revolutionaries that was what was called the cognitive revolution, which were psychologists and linguists and computer scientists who were interested in finding a different way to study the mind. And they did this by saying, another way to be rigorous about minds is to use math to express hypotheses about how minds work, that we can then test through behavior. And so they did that using the kind of math that was most sort of obvious and accessible to them, which was the math of rules and symbols,

Starting point is 00:11:01 inspired by computers and logic and these sorts of formalisms that were very prominent in the 1950s, they set out to test out how well does that describe how minds and languages work. And so Chomsky took that approach and applied it to language. And he set up the problem in a way that was different from the way that previously linguists had thought about the problem. Linguists had kind of thought about their job in linguistics as characterizing the structures of different languages and then maybe looking for a sort of commonalities and regularities in the structures of those language is. And Chomsky said, well, actually, if we kind of think about this as a math problem, a language is some set of sentences that you're allowed to produce, and let's characterize that

Starting point is 00:11:42 set in a very mathematical way by specifying a generator of that set. Right. So he thought of a grammar as a system of rules that you could follow to generate all of the valid sentences in a language. And that approach, what's called generative grammar, became the foundation for much of theoretical linguists, it certainly through the 20th century and then, you know, it continues to be influential today. You talk about sort of a chessboard analogy with Chomsky. Can you sort of go through that or on different types of moves? You start off with the initial, what is it, the 16 moves that can be made by each player. Talk about that analogy. Go ahead and explain it, this chessboard analogy. So you can think about this problem of defining a generator of a set. A good way to think about that is something like a board game,

Starting point is 00:12:24 right? So the rules of a board game are a set of principles that tell you what the, the states of the board are that you can reach, right? And so you start out in some configuration. Chess is a good example, right? You've got all your pieces laid out. The rules tell you how to set up those pieces. And then you can make all of the moves that you can make from that position according to the rules, and that's going to take you to the next position. And then your opponent makes their moves. That takes you to the next position. Right. So, yeah, if you have 20 moves, your first move, the other person has 20 moves. At this point, there's already 400 configurations of the board that you could have reached. And that number keeps increasing, you know, exponentially.

Starting point is 00:12:59 as each subsequent move is made. But the end of making all those moves, you get to the end of the game, and by following the sequence of rules, you've generated all of the possible games of chess, right? And so that's his idea, is that just as there's a set of games of chess that you can follow,

Starting point is 00:13:17 final board positions that you can reach, there's some set of sentences that are the things that are in English, and maybe we can come up with an analog of the rules of chess that generates all of the valid sentences in English. One of my favorite aspect So the book is you kind of trace through the history of thinking about thinking, metacognition,

Starting point is 00:13:34 whatever you want to call it. And you start with Aristotle. I love Aristotle. Who doesn't? But his claims to fame in physical sciences are not so strong, right? I mean, they haven't really held up as well as his laws of thought or logic. I mean, he thought that things fell to the center of the earth because heavy things fell faster than lighter things, which Galileo disproved, you know, with a simple, you know, allegedly dropping two objects.

Starting point is 00:13:59 offer, even a thought experiment, you know, speaking of the laws of thought, the role of thought experiments is not insignificant. But he thought that, you know, women had fewer teeth than men. He had a wife because he had a son, Nicomanchus, right? Nicomanchus was his son, right, Tom? Yeah, I think you know your Aristotle better than I do. Well, the one claim to fame is that he knew that whales were mammals. But why does Aristotle, you know, get so much right about thought? And how can that possibly still matter, you know, 24 centuries later. I think part of that is that he was...

Starting point is 00:14:29 This episode is brought to you by Activia. You might already be eating yogurt, but not all yogurts are created equal. Activia contains over 1 billion probiotics per serving to survive and reach the gut alive. When it comes to gut health, Activia is the number one family doctor-recommended probiotic yogurt brand. Choose Activia. Feel good from the inside out. Visit Activia.comia.com for more detail. else. Doing math, essentially, right, when he was thinking about thought. So what Aristotle did,

Starting point is 00:15:02 he had two projects that I talk about in the book. And the first of those was the part that's about deductive logic. And this is setting up the set of syllogisms. So a syllogism is a simple argument with two premises and a conclusion. And, you know, these are sort of familiar kinds of things you've probably seen in school. It's like, you know, all A's are B, all B's are C. Therefore, you know, all A's are C, right? And so that's a, you know, that's a. an example of a syllogism. And he was interested in characterizing what's the set of these syllogisms and then which of these are valid. In a way that's actually, you know, quite like that sort of Tromsky problem, right, of being able to say, you know, like what are the good ones

Starting point is 00:15:39 and what are the bad ones? And so that was really a matter of just enumerating. So he was kind of like doing the combinatorics of these kinds of arguments. He enumerates all of the arguments. He says some of these I know are good. And I'm just going to say those are good ones. And then he makes little mathematical proofs to relate, you know, some of the other arguments back to, you know, the ones that he knows are good, and he can sort of say things about those two. And so I think, I think his success there was that he was involved in exactly the kind of mathematical enterprise I talk about in the book. He then had a challenge that was left over from that, which is like, you know, exactly the Chomsky challenge again, can I come up with a mathematical system that

Starting point is 00:16:16 characterizes the good ones, right, and separates them from the bad ones? And then that's the challenge that was picked up by Leibniz and later by Boer. So let's get to Leibniz, because you mentioned him. He had this dream, which seems kind of insane at the time, to logify or to codify, to mathematicize or reasoning. So was he basically trying to invent AI 250 years before computers existed? That's pretty much exactly what he was trying to do. And he was the right kind of crazy, right?

Starting point is 00:16:46 He really was someone who had a vision that far transcended the times that he lived in and made contributions to a huge number of different disciplines as a consequence. He was obsessed with the mathematics of combinations, interested in all kinds of mathematics. You know, he contributed to the calculus and so on. He built a calculator, a mechanical calculator that was able to do more sophisticated things than the other mechanical calculators of the age. So he had all these pieces where he knew kind of like what mathematics could do and he knew that if something could be expressed in mathematics, it could be executed by a machine. And so those things came together.

Starting point is 00:17:24 He'd been studying logic since he was a kid and reading Aristotle. And he had this dream of being able to take Aristotle syllogisms and then figure out a mathematical system that would let him essentially like, you know, then run this on his calculator so that if anybody wanted to have an argument about something, he could, you know, put it into the machine and then turn the handle and out would come the answer about who had it right. Maybe he was just too early or is it really possible to do what he was attempting to do? Maybe he underestimated how hard representation would be.

Starting point is 00:17:57 He had some really good ideas that, again, were ahead of his time, and then he had one thing that he hadn't quite figured out. And so the really good ideas were he's the person who invented this idea of vector embedding, as far as I'm concerned. So the way that he tried to solve this problem was by taking the terms that would appear in those syllogisms, the A's and the Bs and so on. and trying to represent them with a little vector of numbers. So he would associate, in his case, it was just two numbers with each of those terms.

Starting point is 00:18:24 And then he tried to find the relationships between premises and conclusions by then reducing this to regular arithmetic, where he'd have like, you know, the number 33 and the number, you know, minus 77 associated with one of the terms. And then if that could be divided by the numbers for another one, say it was like 11 and 7, that would be something where you could say, okay, now the conclusion is going to follow from And so he kind of worked out this system that was just based on arithmetic, having vectors that you are modifying through these arithmetic operations. That was really smart. That turns out to be really important for AI today. That's how language models represent words as well.

Starting point is 00:19:02 The thing that he hadn't quite figured out and sort of got glimmers of at the end of his life was that he didn't have the right algebra. He was like using regular arithmetic and it turns out in order to capture the content of the syllogisms, you need something that's that's a little more complicated than regular arithmetic. Yeah, so let's segue into George Boole. And what did he really change? And most of us, if we know about Boole, his name, it's from Boolean logic and computer circuits, and we stop there with the X-Nor

Starting point is 00:19:31 and all the other circuit diagrams you talk about in the book. But in your telling, Bull is a much more important character. So what do we get wrong about him? He was sort of genius who went beyond the moment that he was in. He spent most of his life as a school teacher, and even as a schoolteacher was corresponding with the leading mathematicians of the day, publishing, you know, really influential papers. He ended up winning this gold medal in mathematics from the Royal Society.

Starting point is 00:19:57 And that was sort of his precursor to the contributions that he made to logic. But his skill as a mathematician was really around these kind of algebraic ideas. And he had essentially taught himself this perspective on mathematics by reading, you know, a hard math book from France that, you know, no one else in England was really reading. And he said, like, he enjoyed reading these big, thick math books because it was the best way to get his small allowance for books to last as long as possible. And so he had this toolkit that was the one that Leibniz was missing, which is this algebraic toolkit. And then he could recognize that in order to capture the structure of thought, you needed this, this, you know, slightly different algebra. And then that's the thing that we now associate with, with Boole.

Starting point is 00:20:43 But his work really went far beyond that. The title of my book, The Laws of Thought, he was someone who was actively involved in this 19th century community of people who was trying to characterize what the laws of thought were. And his big book was called An Investigation of the Laws of Thought. And my epigraph comes from Boole as well. And in that book, he laid out both the kind of foundations of this mathematical logic, but also principles of probability theory that he thought were going to be the way to extend this

Starting point is 00:21:11 to solve other kinds of problems of thinking as well. Pre-saging and a lot of what we have come to use, is it a question of efficiency that it's just super efficient to do things with zeros and ones and you can reduce all sorts of these abstract thought concepts to zeros and ones, or is it not merely the computational efficiency that caused the success?

Starting point is 00:21:32 I think it's that by expressing things in that way, he was able to then do the thing that life that's wanted to be able to do in terms of, Now, it was possible to think about, you know, creating machines that would be able to execute these kinds of computations, right? So, Boole's work was then developed into a richer theory of mathematical logic. That fact that you could express mathematics in, you know, a mathematical form itself, you could take statements through a mathematical statements and express them in logic and that would turn them into math themselves. That became the foundation for a lot of work on sort of asking questions about the limits of mathematics that inspired Turing to

Starting point is 00:22:11 think about what's an abstract kind of machine that you could use to do these kinds of calculations, to emulate the mind of a mathematician, and then von Neumann figures out a scheme for building these machines that still underlies the computers that are on our desks today. Do you think that von Neumann machines, touring machines, etc., do you think that they will be kind of permanently ensconced in this discussion or other architectures and even other approaches towards AI, will they eventually supersede based on efficiency the same way that Bula was able to supersede in some sense, Leibonis? Yeah, so Turing machines were never a practical device, right?

Starting point is 00:22:48 It was a sort of theoretical abstraction for how you could describe computation. Well, Neumann worked out how to have a stored program computer, right? And so how you can have a computer which has, instead of having to rewire it every time you want to solve a different problem, it's able to use, you know, software to modify what it is the system's doing. and that's a fundamental advance in terms of being able to create machines that can do all of the kinds of thinking that we want them to do. Nowadays, a lot of the training of artificial neural networks is done using dedicated hardware

Starting point is 00:23:18 GPUs, graphics processing units, right, which are units that were originally designed to just speed up the computations required to put things on a screen, but those computations turn out to be exactly the computations that you need to do to run a neural network. And so there's lots of diversification of specialized hardware for doing those kinds of things. It's also interesting to note that the earliest neural networks, so neural networks that were built by people like Frank Rosenblatt and Marvin Minsky, they were also specialized hardware. They built physical neural networks that were, you know,

Starting point is 00:23:50 sort of connected up by wires with adjustable resistors on them. I think that's certainly a kind of technology that's changing the way that, you know, we're thinking about computation today. And, you know, a lot of the energy that's going towards compute is now going towards GPUs. The fact that a lot of energy is going towards those is something. that's encouraging people to think about alternative models for computation. If what you want to do is run neural networks, maybe we can learn things from the neural networks that run inside our heads, which run on far less energy than the kinds of neural networks that people are running on GPUs.

Starting point is 00:24:19 Yeah, you talk also in the book, I mean, speaking of GPUs, Jensen Huang was on Lex Friedman's podcast recently. He said, AGI is here. I keep saying that I'm not really convinced that AGI will be here until it could do something that human beings have never been able to do. And the clearest kind of most simple realm to demonstrate that is in the laws of math or some, you know, physical observation that we've never really been able to explain, you know, unifying quantum mechanics and gravity, something truly novel. Or at the very least, you know, replicate what, what human brains did a hundred years ago, you know, long before computers. For example, if you just gave it the data on the planet Mercury from 1911 and before, Einstein certainly knew that there

Starting point is 00:25:01 was this anomalous procession. In fact, GR was basically designed retrodictics. to explain why that behaved that way. And yet, we can't seem to get that to occur. My student, Evan Watson, and I have tried to replicate, you know, could you come up with GR from just the deductive observations of data, which we have hundreds of years about for Mercury, right? So what is your working definition of AGI?

Starting point is 00:25:24 As a cognitive scientist, I would be very sort of careful about thinking about, you know, this idea of artificial general intelligence in the first place, because I think it plays into a bias that we have, which is that our best example of an intelligent system is another human being, and all of our intuitions about intelligence are based on the kinds of things that human beings do, right?

Starting point is 00:25:47 And so I think that encourages us to think about it's in a kind of like one-dimensional way, where there's kind of like, here's where humans are on this one-dimensional scale of intelligence, here's our AI systems that are coming closer and closer, and one day, oh, they're going to be past us, and then either something wonderful or something terrible, is going to happen. And so that one-dimensional characterization, right, so this is like AI or superhuman

Starting point is 00:26:09 AGI or whatever it is, I think that's not a productive way of thinking about what's going on with our AI systems. I think a better way of thinking about it is that human minds and our AI systems are both systems that have been created to solve certain kinds of computational problems. They've been sort of optimized to solve those problems. But they've been optimized, some of those problems overlap, but they've been optimized in sort of different ways and under different constraints. So human minds have evolved under constraints on just what, you know, human lifetimes. We only live a few decades. Those compute resources I was talking about, right?

Starting point is 00:26:43 We only have a couple of pounds of neurons up there. And bandwidth constraints in terms of like, you know, we're limited in our ability to communicate with one another. We have to do things like talk to each other on podcasts in order to share information, right? Whereas our AI systems can have way more data than a human can see. they can potentially just scale arbitrarily in the amount of compute that they use, and you can transfer data from one machine to another, you can transfer weights from one machine to another.

Starting point is 00:27:08 There's a lot more sort of plug-in-play compatibility in terms of being able to spread that intelligence around. That means that the solutions that those systems find can look quite different, where we've made AI systems by essentially optimizing them to solve this problem of getting a radio signal from another planet, and trying to predict the things that are occurring in that radio signal to the point where they're really good at it and they've even made inferences about the aliens that live on that planet

Starting point is 00:27:37 and what kind of cities they live in and what kind of interactions they have, right? That's the problem that the AI system is solving. And the human is doing something quite similar, but they're doing it in a social context where they're interacting with other humans and they're doing it with the benefit of like, you know, thousands and, you know, hundreds of thousands of years of evolution behind them, right? And so we end up sort of seeing similar, kinds of behavior from these systems, but seeing it from two quite different evolutionary trajectories

Starting point is 00:28:03 and seeing it under two quite different sets of constraints. So saying, you know, one thing is like the other thing, I think it's sort of misleading. I think they're sort of on, you know, these different trajectories. And so we're going to end up with things that are really smart in ways that go beyond the kinds of things that humans can do, but also maybe surprise us in the other things that they're not able to do because those things don't show up in the training data or they have the wrong formulation or the learning problem or whatever it is. You speak in the book about what Chomsky called Plato's problem, how human beings know so much from so little.

Starting point is 00:28:33 But, you know, when I'm had on Jan Lacoon on this podcast, he said it's the exact opposite. AIs, you know, have tremendous amounts of information, but it's not even close to the amount right now filtering out something like 13 terabytes of raw information if you were to encode it, which I think is ridiculous. But even just fovial recognition and, you know, the camera or what have you.

Starting point is 00:28:55 I mean, it's certainly millions of megabytes, gigabytes, right? So isn't it the opposite? I mean, I read, you know, when my kids were little, that they need to hear a million words before they can speak. And if you just compress that, I mean, it's an awful lot of data, isn't it? The Plato's problem, right? You said, how do we come to know so much from, you know, so little, right? And Chomsky talked about this as the poverty of the stimulus, right?

Starting point is 00:29:17 And the idea being that there's not enough information in what the kids hear to determine the structure of the language that they end up speaking, right? So I actually think that our AI systems are in some ways a good demonstration of this, which is that if you give them as much data as a kid gets, they're still not as good as a kid at learning language, right? We can have arguments about what it means to give them exactly the same data that a kid gets. And I have colleagues here who, you know, are measuring different aspects of what that looks like. This spring, denim gets a softer, lighter update.

Starting point is 00:29:51 Introducing Old Navy's drapey denim wide leg, a new fit that moves with you. It's everything you want denim to feel like for summer. Easy, breathable, and effortlessly cool. With a fit that creates natural movement and a wide leg that feels modern, not overwhelming. Plus, that signature, wait, for this price, moment. Old Navy's drapey denim wide leg. But Chomsky's argument in particular was focused on syntax. So how you know some very nuanced things about the structure of language.

Starting point is 00:30:21 based on the experiences that you have. And he thought there's not enough information that's contained in the stimulus that you see. And, you know, to the extent that we can train models on at least the number of words that a kid would have seen, those models are still not doing as well as a kid from that amount of data. So I think that does support the idea that humans bring to these learning problems something that the AI models are not getting, right? So humans, they have something that a machine learning research or a cognitive scientist calls inductive bias. So something other than the data that influences the solutions that they're reaching, those inductive biases are what allows us to learn quickly, more quickly than our neural networks do from limited amounts of data. They're also something that influences what solution we find. So if you have your neural network playing this alien radio prediction game, it's going to find some solution to playing that game, but that solution might not be one that is very intuitive to us as humans, right?

Starting point is 00:31:18 It's sort of like figured out some weird stuff that are regular. that it can use in making those predictions, but it's maybe not got a really good model of the underlying world or things like that. Whereas the kinds of solutions that a human will find are going to be influenced by those inductive biases. So part of what allows humans to generalize smoothly from one problem to another and to act in ways that are predictable to other humans and to sort of show intelligence that has those properties of generality that you were alluding to is the inductive bias that we bring to those problems. And I think that's another sort of poverty of the stimulus argument. Like, if you want to get sort of appropriately general learners,

Starting point is 00:31:55 you might need to have some inductive bias to get that smoothness. It seems to me that one reason that humans flourish is that we're comfortable with ambiguity. You know, for example, a question like, is an olive of fruit, as you point out, it's pretty deep philosophically? Well, why is it that, you know, humans, even my, you know, kids can understand it, but it sort of leads to either, you know, AI psychosis or hallucinations or sycophanty, I'll ask you, which is the worst. But why is the question like, you know, is the moon a light bulb?

Starting point is 00:32:24 Why are those deeper than they look to be? Those kinds of questions, I think in cognitive science, have been useful in revealing exactly what our concepts are. So people coming out of that rules and symbols tradition thought, oh, maybe a concept is just a definition, right? And I think that's a good intuitive way of thinking, like, what a concept is, right? You sort of have the intuition. You can look something up in a dictionary and it's going to tell you,

Starting point is 00:32:45 oh, what a cat is. Okay, a cat has these properties and that's what makes it a cat. That way of thinking about the world sort of prevailed through the 50s into the 60s and then was pretty firmly rebutted by a cognitive scientist called Eleanor Rush, who showed that there's systematicity in the way that people have uncertainty about category membership, right? So your listeners can think about this, right? So if I ask you, is a chair, a piece of furniture?

Starting point is 00:33:13 Probably yes. is a phone a piece of furniture, probably no. Is a lamp a piece of furniture? Maybe, right? Is a rug a piece of furniture? Probably not, right? So you can sort of immediately begin to explore this fuzzy boundary. And that fuzzy boundary is a clue that there's probably not a rule underlying your

Starting point is 00:33:36 notion of what furniture is. In fact, it has what Roche called it a family resemblance structure, where there are some things that you're sure are part of the family. and then there are other things that sort of share some attributes with them, and then there's sort of fuzziness that sort of goes out from there. And so when we come to AI systems, that kind of thing was a challenge for AI systems that were based on systems of rules. And that was, again, the dominant approach for building AI systems.

Starting point is 00:33:59 Now through the 1970s, through the 1980s, people were making AI systems based on what we're called production rules. There was a company that has continued to the present day building a huge database of rules with the hope that if you've got enough rules, then you figure out what the structure of the world is like. The neural network approach really, in some ways, sprung up as an alternative to that that would be able to capture this fuzziness and all of the graded, continuous things that seem to be important properties of human concepts. You talk about the semantic revolution.

Starting point is 00:34:28 Can you talk about, first of all, what is a semantic network, and then explain the shift that made that possible and made the concepts becoming nodes in a weighted network rather than sort of a compendium of facts? Why was that such a breakthrough or seminal event? If we want to capture that fuzziness of concepts, you need to have some way of having graded relationships between things, right? And so your representation of furniture is now connected to chair very strongly, but connected to rug much more weakly, right? And so you can capture that by creating a semantic network, a network where each node in that network is a thing, concept, and each, you know, they have links between them that reflect their strength. And psychologists began to show

Starting point is 00:35:11 that that wasn't just a good way of storing information about the connections between things, but actually turned out to be a pretty good model of human memory, where, like, if you said to somebody a sentence that contained one of those words, then it would be easier for them to remember or recognize another of those words that was closely associated with it. Like activation of words seemed to sort of spread through that network. And so that was something where psychologists began to realize that maybe there was a different way of conceptualizing what thought is. You can think about it now as you have all of these concepts. Each of those is activated to some extent. Now you have a high dimensional space, which is the space of all of the activations of those concepts. You have a

Starting point is 00:35:51 point in that space and that's your current mental state. And then the weights between things tell you how those mental states are sort of evolving over time. And now we have this alternative to that sort of logic rules and symbols based theory of how it is that minds work. Walk us through an example in this besides the furniture. It seems like there's almost a geometric or, you know, Riemannian curvature approach that took over. Is that where the kind of insights of Hinton and, you know, gradient descent? Is that the kind of novelty that was applied by Hinton and his colleagues? Yeah, so if you have this idea that, you know, we want to now have networks of things

Starting point is 00:36:25 that are connected up to each other by different strengths, and maybe we can even take away the idea that those nodes in those networks have labels on them. And maybe they're just nodes that represent information somehow, right? That's what leads us to neural networks. psychologists had been exploring neural networks for a long time, even all the way back to the 1950s, the first kind of, when people were developing the first AI systems, there were also people working on implementing neural networks on computers at that time, as I said, building neural networks by hand. So Frank Rosenblatt, who was a psychologist at Cornell, he was originally a social psychologist, and he had written a dissertation that required aggregating a whole lot of survey data. And so he'd sort of found out about the computer on campus and started messing around. with that and then built a circuit in order to aggregate the data from his surveys. And suddenly you had a psychologist who understood computers and who understood circuits. And he was like, ah, I've got it. I'm going to build a brain. Right. He sort of had the pieces and the insight to think

Starting point is 00:37:20 about how to do that. And so he built some of the first mechanical brains or electronic brains. I say mechanical because the way that he did it, he had a sort of artificial retina that you would show something to and it would produce responses from little, you know, sensors. that were on that retina that would tell whether it was seeing something light or dark. And then that information would get sent to another set of units, these nodes that would be accumulating information from the retina. And then he had another set of connections that went from those to an output. So, for example, it could be deciding whether it saw a square or a circle.

Starting point is 00:37:54 And so those connections to the output had a little resistor on them that could adjust to reflect the strength of that connection. And he came up with a learning algorithm that made it possible for this system to learn to differentiate simple shape. circles from squares or simple letters like, you know, E's and Fs or something like that. And he proved a theorem that anything that the system could represent, it would be able to learn. Which was great. He was very, he went off and sort of publicized the capacities of the system, which was called a perceptron. The problem was his former schoolmate, Marvin Minsky,

Starting point is 00:38:28 had also built his own neural network. While he was a PhD student at Princeton, he went to Harvard where he'd been an undergraduate and built a neural network in the basement of the second. department out of leftover airplane parts. And he looked at this thing. He'd written his PhD dissertation on learning in neural networks. And he implemented this. And he looked at it and he was like, you know what, in order to learn anything interesting, this would just have to be so big and cost so much money that it's not, it's never going to work. And so he gave up on learning in neural networks, got interested in symbolic approaches to learning. And so when Rosenblatt, again, his schoolmate, right, came out and said, oh, neural networks can learn all these things. Minsky was not impressed.

Starting point is 00:39:07 And then with Seymour Papert wrote a book that showed that perceptrons were sort of fundamentally limited and the kinds of things that they could represent. And the reason for that limitation was that single layer of weights in the network. And so the reason why that was a limitation was that the perceptron with a single layer of weights could only represent linear boundaries in space. Right. So if you can think about all that information is coming in, it's going into a high dimensional space. And now it's trying to find a linear sort of partition of that space. in order to separate the things from each other. And so Rosenblatt's learning algorithm could find those boundaries,

Starting point is 00:39:44 but there were lots of problems where no such linear boundary existed. The solution to that problem was to make a neural network that had multiple layers. And various people kind of like came up with strategies for making this work. The problem was that Rosenblatt's learning algorithm didn't work for multi-layer networks, so only worked for one-layer networks. He had a sort of a trick for doing this that he called back propagation, but it didn't quite work. It sort of worked most of the time.

Starting point is 00:40:05 Another group of psychologists got interested in these neural networks, thanks to semantic networks and spreading activation and so on. And so this was David Rummelhart, Jay McClelland at UCSD, and then a postdoc that they hired Jeff Hinton, who was working on that project. And so Hinton suggested to Rommelhart that he could set up that problem as one of gradient descent. Right. So this is, you know, basically thinking about their... being some measure of how well the neural network is doing, and then adjusting the weights in the network in the direction that would decrease the error that the system was making. And then using that insight, Romal Hart was able to rederive something like Rosenblatt's learning rule. And then he was

Starting point is 00:40:52 able, you know, on a plane flight when he was off to a grant reporting meeting, had enough free time to sit down and work out the whole thing in his notebook and, you know, derived the learning rule for multilayer networks. Satisfyingly, one of the fundamental principles that was needed for that was something that came from Leibniz, from Leibniz's calculus, the chain rule. So Leibniz got to have his day after all, you know, a couple of centuries later. Hinton was actually the great, great grandson of George Boole. So they met again together in that, in that location. I was wondering, you know, kind of the, as a practicing, you know, researcher in this field, much more adjacent to it than I am Although I use it every day all day in some cases,

Starting point is 00:41:33 much of this regret on my wife. But the biggest problem that you see with LLMs, is it psychosis? Is it hallucination? Is it sycophanty? I mean, I love sycophanty. You know, when I asked it, you know, what books is Brian Keating written?

Starting point is 00:41:45 It says, losing the Nobel Prize, into the impossible and a brief history of time. And I just thought that was awesome. I'd love to get some of Stephen's, you know, book royalties. But what's the biggest concern for you when it comes to AI? It's not Dumer. It's going to take all our job.

Starting point is 00:41:58 We'll talk about meaning. at the very end. But what's the biggest kind of picture? I think there's a few things. So one is this jaggedness, right? This sort of like lack of generalization where I think we as humans can end up overconfident in the kinds of things that the AI systems can do because we apply our intuitions that tell us if you had a friend who could solve, you know, international math Olympiad problems at a gold medal level. You would trust them to do all sorts of other things on your behalf, but you should not trust an AI system to do that because they don't generalize across problems in the way that people do. So I think, you know, just having the wrong

Starting point is 00:42:29 intuitions about these systems is a major bottleneck to our being able to think about how to employ them effectively and how to make predictions about the kinds of things they're going to be able to do. And that was part of my motivation in writing the book as well, is giving people some of the context where these things come from and a sense of what the problems are that can come out of that and maybe what some of the kinds of solutions are historically that people have found. Of the other things that you mentioned, hallucinations, I don't mind very much in the sense that They're relatively easy to catch if you have some domain expertise. And I think they're actually good in some context.

Starting point is 00:43:02 So one of my best tricks for getting the models to generate good research ideas is to ask them to tell me about papers that I haven't heard of but should know about. And when they do that, they'll often hallucinate and make up a paper. But the ideas in that paper are much more interesting than if I ask it to just tell me some interesting research ideas, right? So having conditioned on generating a published paper actually makes it produce something which is higher quality. I think sycophancy is a major problem.

Starting point is 00:43:29 We have a recent paper, this is with Rafael Batista, where we show if you take a rational agent who's doing Bayesian updating on their beliefs and have them interact with a system which is sycophantic in the sense that it's generating data based on the hypothesis that the agent expresses to the system, then that agent is going to become increasingly confident in their beliefs, but no closer to the truth. And we have some demonstrations that this actually happens with real deployed systems where we have people trying to solve a simple problem. And if they're interacting with the default prompting for a GPT,

Starting point is 00:44:02 they end up not making progress in that problem, even though they become more certain that they found the right answer. And then the last two questions I have. One is, you know, for someone, you know, looking to get at the future where the future is going, where the puck is going, you have some hockey analogies in the book. I'll leave it for the readers to encounter them. But skating to where the puck is going to be,

Starting point is 00:44:20 it seems like one thing that's really missing or is not fully developed is the embodiment issue or truly, you know, maybe close to AGI, you have very advanced intelligence coupled to robotics or embodiment, and maybe it's what it's missing or what these systems are missing, is this marriage, which will unlock

Starting point is 00:44:39 via some network effect that we don't understand, you know, truly human level thought. I always use the analogy of what Einstein, who worked not far from you, called his happiest thought, which was that, you know, an observer in free fall would experience no gravitational acceleration force,

Starting point is 00:44:53 and that led him to the Einstein equivalence principle. So I always ask, you know, how can the computer visualize, you know, the zero gravity feel of going, you know, the elevator cable getting cut? And then second of all, how can it have a happiest thought? Maybe we could incentivize it that way,

Starting point is 00:45:07 but maybe you could embody it, you know, if it gets the answer wrong, if it's truly, you know, sycophantic, you blow out some of its capacitors or, I don't know, you feed it some, you know, training data only from the fast and the furious, you know, movie genre series. But tell me, what would be kind of the next unlock

Starting point is 00:45:22 as you see it, to truly get us to the next, to the next level that, you know, maybe incomprehensible to Minsky and Jomsky and all the other folks that we mentioned in the book. Yeah, so I think there are two parallel things here, right? So one is inductive bias. So trying to figure out what it is that's inside humans that allows us to find solutions faster and that are more robust and more generalizable. So that's a good opportunity for cognitive science to contribute something to AI. Second thing is getting something which is closer to human experience into these neural networks where, like I said, they're being trained to predict alien radio signals. If they have experiences that are closer to those of a human child, that might be

Starting point is 00:46:01 something that helps to create those more generalizable, more robust kinds of representations of the world. And then, you know, embodiment is obviously a part of that. It's not clear to me that that on its own is necessarily going to solve problems of, you know, like allowing these models to be more creative and to solve more kinds of problems. In a recent paper with LSU in my lab, we show that prompting models to make cross-domain metaphors, so to come up with a product design for a car based on ideas from an octopus, does not increase their creativity. It doesn't increase the originality of the ideas that they produce, but it does for people. So it seems like some of the tricks that we have for getting humans to have

Starting point is 00:46:45 good ideas are not necessarily things that are effective for our large language models. And so that maybe is some fundamental difference in architecture, but it makes me a little less optimistic that just doing things like providing embodied experiences that you might be able to draw on to form these analogies might be enough to get them to be being real creative. And then lastly, you end on a hopeful note, not really a doomer as I tend to be,

Starting point is 00:47:09 but kind of advice to early career or scientists or maybe even lay people, because you just give us some examples of what a early career cognitive scientists might do. but what shall a person take away from this book? Really what I wanted to do is to give people a sense of context and a vocabulary and a set of tools for thinking about these systems where I think for many people,

Starting point is 00:47:30 AI seems like something that suddenly came out of nowhere two years ago. You know, like all of a sudden you could talk to a computer and the way that you talk to a human. And knowing the, you know, a couple hundred years of stuff that led up to that is helpful in terms of understanding what it is those systems are doing, why they can do it, what the limitations are that we might expect they would have, what things are going to be hard for them to do, what are the next steps that might help to fill in some of those gaps, and having a way of having an informed conversation

Starting point is 00:47:56 about those things. The laws of thought here, as I said, something that in principle we should be teaching in school, not just to help us understand how our own minds work, but to help us understand the world that we're moving into. Professor Tom Griffiths, Princeton University, this book has done something that very few books can even attempt, let alone pull off, tell the history of cognitive science and also the future where it's going and get inside of the mind of one of the greatest researchers of our generation and those that came before him. Tom just told you that the godfather of AI is the great, great grandson of the men who invented its math. That sycophantic AI makes you more confident but no closer to the truth, and that

Starting point is 00:48:33 a child still can beat a GPT at the same data budget. Now, if all that reframes what you thought these machines were for, hit subscribe and turn on the notification bell. Drop a comment. What did Tom break for you? And if you want to go deeper, I talked about consciousness and machine minds with David Chalmers. The link is right here. I know you're going to love it. Go ahead. It's subscribe.

Into the Impossible With Brian Keating - Princeton Scientist: We Don't Understand AI - Tom Griffiths - #553

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.