Into the Impossible With Brian Keating - Meta NeuroScientist SHOCKED Me: Scale Alone Won’t Create Consciousness!

Episode Date: March 16, 2026

Please join my mailing list here 👉 https://briankeating.com/list  to win a meteorite 💥 David Sussillo spent years training neural networks at Google Brain and now leads research at Meta Reality Labs. His verdict on the path to artificial consciousness might surprise you. In this conversation, we discuss why scaling transformers alone won't produce conscious AI, how recurrent neural networks differ from the transformer architectures powering today's LLMs, the FORCE algorithm he developed to train RNNs, what his work on Meta's EMG wristband reveals about AI and biological signals, Rich Sutton's Bitter Lesson and whether we're locked into a GPU-transformer paradigm, and why studying the brain may be the only way out of AI's current limitations. David is also the author of Emergence, a braided memoir that tells the story of his path from foster care to the frontier of neuroscience and AI research. David Sussillo is a research scientist at Meta Reality Labs and an adjunct professor at Stanford University. He completed his PhD at Columbia University under Larry Abbott and previously worked at Google Brain. Key Takeaways: 00:00 Scale alone won't produce consciousness — here's why  00:25 The "bitter lesson": big data crushes every algorithmic invention  01:15 How hallucinations were discovered early at Google  05:00 The lock-in problem: why LLM + GPU may cap AI's ceiling  08:25 What Meta's neural wristband is actually trying to solve  11:20 Rich Sutton's bitter lesson explained  14:15 "More is Different" — why you can't reduce complexity upward  28:30 The book Emergence — two stories braided into one  36:00 Why mentorship matters more than intelligence for at-risk kids  47:00 Stability over intensity: what kids in foster care actually need  ➡️ Follow David Sussillo  🌐 Website / Research: Meta Reality Labs  📚 Emergence (memoir): https://www.amazon.com/Emergence-Memoir-Boyhood-Computation-Mysteries/dp/1538768577  🎓 Stanford / Columbia / Carnegie Mellon Alumni ✍️ Email: [e-mail]  🏄‍♂️ Twitter: https://x.com/SussilloDavid Join this channel to get access to perks like monthly Office Hours: https://www.youtube.com/channel/UCmXH_moPhfkqCk6S3b9RWuw/join 📚 Get my books: Think Like a Nobel Prize Winner, with productivity tips from 9 Nobel Prize winners: https://a.co/d/03ezQFu Focus Like a Nobel Prize Winner, with life-changing interviews with 9 Nobel Prizewinners: https://a.co/d/hi50U9U My tell-all cosmic memoir Losing the Nobel Prize: http://amzn.to/2sa5UpA The first-ever audiobook from Galileo: Dialogue Concerning the Two Chief World Systems: Ptolemaic and Copernican https://a.co/d/iZPi9Un Follow me to ask questions of my guests: 🏄‍♂️ Twitter: https://twitter.com/DrBrianKeating 🔔 Subscribe https://www.youtube.com/DrBrianKeating?sub_confirmation=1 📝 Join my mailing list; just click here http://briankeating.com/list ✍️ Detailed Blog posts here: https://briankeating.com/blog 🎙️ Listen on audio-only platforms: https://briankeating.com/podcast #universe #podcast #briankeating #intotheimpossible #science #astronomy #cosmology #cosmicmicrowavebackground #intotheimpossible #briankeating Learn more about your ad choices. Visit megaphone.fm/adchoices

I don't think scale alone, just transformers and lots and lots of data from the internet is going to get you to any form of consciousness. My cat is conscious, it snuggles. It's there, it's present, it's embodied. I was a bet that, like, there was still something to learn from the brain that would inform AI, I have to say, that I wasn't not successful. These things would just generate complete nonsense. If you add three periods to it or a backslash, they would just go crazy.

Every time there's algorithmic invention, big data just comes and crushes it. David Cecillo, neuroscientist, former Google Brain researcher, Columbia trained PhD, who's now at meta-reality lab. So what he told me about the limits of AI might change how you look and maybe feel and fear about what's coming next. Let's go. David Sassillo, welcome down to San Diego all the way up, down from the Bay Area. Thank you for having me, Brian. Appreciate it. You're the author of the second most difficult type of book to review and to cover on this podcast, which is a memoir, the first one being a novel, right?

You can't really get a novel fully described without spoiling it unless it's decades old. But this one, a memoir, is really just a fascinating work of pure love, but it's also a work of science. And we're going to get into all the mysteries and magic of it. But first thing I want to ask is, when I think about, you know, how we're using LLMs and I use them every day and I love them and then I have a great deal of fondness for how they deliver, you know, just incredible products that saved me a lot of time and resources, However, they have a stunning number of lacunae of flaws and foibles and gaffes that they make. And the first one I think about is sycophanty, which I actually like. I like it my AI to be a little bit, come on, come on. You've got to have a little bit of sycophanty, right?

And then the other one is hallucination. So I asked it once, you know, asked Chatshipping, you know, what books is Brian Keating written? Oh, losing the Nobel Prize, into the Impossible, a brief history of time. No, I'd like to have 1% of Stephen Hawking's book sales, but I didn't write that book, right? So is there any hope that, say, R&Ns might be more immune to hallucination or sycophanty, but let's just stick with hallucination, then, you know, other approaches for, you know, the typical approaches that are used in our, you know, LMs and so forth? On the face of it, I sincerely doubt it.

So the hallucinations, we wrote a paper about it in 2018, and we were looking at neural translation using R&Ns. So I don't think so. I think... Can you explain what neural translation? Neural translation is taking English to German, say. And so it's just translation and the neural language. Yeah, natural language.

And it's just you use a neural network to do it, right? And so what we discovered is that these things, I mean, it was at Google, they knew that these things would just generate. This is really early days. Just generate complete nonsense. So you could have, you want to translate into German. Hello, my name is David. If you add three periods to it,

or a backslash, it would just go crazy. It would just go crazy. These things, they've made major, major progress on this since then, right? But I don't see any reason why, in fact, I would expect it to be, Arnans to potentially be worse here. I think it's a very sophisticated problem that, in all likelihood, has an answer that involves many, many hacks, you know. And I don't do that research, so I want to be careful about, yeah.

Are there any particular, you know, in the consumer-facing side of things that the audience might be familiar with? you know, killer apps where R&Ns are superior to other forms of approaching the similar problem. What happened is the transformer happened, right? And so that just for the audience, the transformer is this much more, I want to say sophisticated. It is a more computationally intensive black box that utilizes this idea of attention. And so you can look at just the entire sequence of your input. And so that is a very powerful box. And so, you know, it really kind of coincided with my transition from Google that RNNs became out of fashion.

AI is really heavily dominated by fast-changing fashions. So it would not surprise me at all. There's been lots and lots of work on making RNNs work better, the so-called state-based models now, making these models more powerful, more efficient. So what really is going on is the transformer very cleverly, in fact, was devoutes. was built to work perfectly with a GPU. You're running uphill on that one. Yeah, and I've talked about this before,

but maybe I'll run my theory beyond you and see what you think. That actually, I think that we're in no danger of artificial superintelligence because the architecture that's been so successful is dominating the landscape from consumerism, which is all that matters, really. Attention is all you need,

but actually consumers are all you need for most things because then it'll eventually get propagated through the government, through military and it'll be education, it'll be impossible to extract it. And it's a phenomenon called lock-in. Like the most, the first mover advantage is often, you know, a disadvantage long-term. Like the QWERTY keyboard was designed such that the keys didn't stick together on a typewriter, which you barely know about probably.

I remember typewriters. But if you used, you know, like S&T too often, they would conflict and the hammers would stick together and you'd ruin your efficiency of typing. so they change from the more efficient form of keyboard organization, Dvorak to QWERTY, right? And now we're locked into it. My kids will never know. And your kids, you know, whatever, your students, your mentors.

They'll never know another type of keyboard. And for that reason, the first mover is not always, in fact, it's sometimes the worst solution. Like, you know, there are VHS and DVDs and all sorts of examples of this. My feeling is that the LM plus GPU is such a marriage made in heaven that nothing's ever going to break out of it. it and therefore we're not going to get to the true definition, which I told you about before we started, my definition is the Einstein test. Can an AI predict based on the pre-1911 laws of

physics that were known at that time the data from the planet Mercury could it come up with relativity? If it can't, then, you know, it seems to me it's good at checking things, incredibly useful, but it's not going to be super intelligence, let alone general intelligence, or maybe general intelligence, let alone superintelligence. What are your thoughts about this locking phenomenon? I mean, are we a victim of the success of these GPUs, married to LLMs, which were invented to solve the problem of doom that you and I loved in the 1990s, right? That's right, yeah. Who's to say that this will lead us to this paradise of superintelligence? I'm definitely sympathetic to the argument you're making about being locked in because history matters, right?

The details of the exact random choices that were made a decade ago. I have to say my experience with the AI community and industrial research is if you were to demonstrably prove, that some other architecture were better than they would all switch as soon as they possibly could. Like a major improvement. Of course, the problem there is how do you do it? And so this was a major problem at Google

when I was there. This is all at this point, public information. It's really history at this point. We had this new technology, deep learning. Of course, deep learning is just stacked neural networks. And so we had this research group together, Google Brain. And so we had all these people just innovating and innovating all these techniques.

That's where the transformer came from, right? And so what they did is they brought in all of the more applied research scientists who worked on the different, like the page rank or the neural translation or the ad placement or whatever. And they would just sit in and kind of learn how we were doing things.

And then they would go back and apply it to their systems. And it was, I think, a brilliant strategic move on Google's part to do that. It really, really changed everything for them. And so coming to the point, there was significant uphill battle even showing, That deep learning, which has taken over the world, was actually better than those systems because of this kind of entrenched advantage that you're talking about. So I think it's a real thing.

But for all of that, if you could demonstrably show that doing X, Y, or Z thing would be way better, I think they would eventually pivot. It's interesting because, you know, NVIDIA is a pretty big battleship to turn around, right? I guess, you know, with the TPUs that Google uses different story, right, because they're making their own, and so they can pivot and GROC is making their own. But it's still fundamentally the same idea. Massive matrix multiplications in parallel as fast as you can get them. And so what is meta's interest? I mean, the metaverse, you know, came and went without much fun, fair, but a couple dozens of billions of dollars put on the bonfire.

I had my meta-goggle glasses on in the other room. You can show me the version six later. You promised me exclusive view at those. That's not true. That had the holographic, you know, 3D rent. Oh, so you admit that they do exist. I fell into the trap. the interest in meta beyond

llama and sort of, I mean, they're also a behemoth. So what are their interest in the brain and neural networks? What's your role there and what's their kind of direction and why we should be excited about their future? Yeah, so broadly speaking, I want to be careful here. Yeah. I think just like every other company, Meta realizes

AI is here and they want to make the best products available that they can using those technologies and develop even in better and increased technologies as they do it. For my part, I actually work on a much more applied piece of technology, which is this EMG wristband,

which is meant to drive the glasses. Ah, the oculet, or the glasses now. The so-called neuroband. Right. That's an applied problem whereby you read out the electrical signals from the wrist muscles, and then you decode them to, say, hand right, or do gestures up, down, left, right,

this kind of thing. So taps and clicks and button presses. And so that's an applied problem, but it's an interesting problem because how do you get something to work on every single person on the planet, right? That's actually a pretty deep applied problem. Does it need to, though? I mean, if it works for the billion most successful consumers on Earth, that's a pretty good market.

I'm sorry, so that's, to me, that's the end went up to infinity already. The physicist, cosmologist in me. We think of billions, right? Yeah, that's right. So we actually were able to publish some of this work in nature. And I think that the real take-home message is that from a science perspective, nobody had shown, to my awareness, that like the deep learning recipe would work on biological signals. Right.

And so those kinds of scaling curves, I can't stand scaling law because is it really a law. Those scaling curves are true for humans in terms of reading out wrist movements. The more you had you get in these log linear plots, the better you do on your error. And so that's a real result. And that result has real implications for academic BCI because it means that, now, I want to be careful here, the muscles are a very simple system compared to, say, brain data. But for all of that, your muscle activity is action potentials, right? So it is a neural signal. So it's a very encouraging piece of data point for the program of brain computer interfaces.

So you didn't mention on scale and you talked about this in the book, that bitter lesson, you know, you talk about what is that? What's your beef with scale besides my own beefs is stepping on the scale? Talk about what is this, the limits of scale? Bill, I mean, these are scaling.

We have all sorts of power laws and Moore's laws. Yeah, so the context here, Rich Sutton, very famous AI researcher and reinforcement learning. He kind of wrote this thought piece in 2019. That is, you can't really talk about the future of AI without referencing this. So here we go, because it's actually a very, very good thought piece.

And he basically says that every time there's algorithmic invention, like someone actually thinks through an algorithm, just big data just comes and crushes it. So if you have cheap compute, lots and lots of data and neural networks, then you're just going to win every battle. So we should just give up and the lesson's bitter because we all, of course, value our own expertise, right? So to me, this moment, things are changing a little bit already because they're using RL techniques to improve language models. That's our reinforcement learning. So they're continuing to learn.

There's a lot you could say there. The idea that scale alone, that's what we're really studying. That's what today's moment is. What is scale? How far can scale take you? And I think from a consumer perspective, it can take you very, very far. Very far. right so the point that i make sort of in the afterward of the book is that predicting the future is

always a risky business but i don't think scale alone just transformers and lots and lots of data from the internet is going to get you to any form of consciousness um you know that's just somehow my cat is i would argue is conscious it snuggles it's it's there it's present it's embodied all of these things so i just don't think we're going to get there you know and and you know you know So what's really happened is historically, neuroscience really informed the beginnings of AI, really, truly. And then now we're in this world where AI is taken over. When I was at Google Brain, I was a bet that there was still something to learn from the brain that would inform AI. I have to say, I wasn't not successful.

I'm proud of the work I did there, but I was not successful in that program. Now we're at this point where, like, it's not just true for neuroscience, but for all of science. I think AI applied to those sciences is a major thing that we should be looking at. A lot of people are, right? You know, like my dream job would be like, let's go AI the heck out of neural data and figure out to make huge neural models and then go study how they work. So that to me would be very powerful. And then, you know, just to close it fully, it's my guess that one of the ways out of this scale is the only solution is to actually use AI to study the brain, pull some insights and or data extracts from the brain from doing so,

and then potentially apply those to AI models. The book begins with a wonderful kind of digression from famous physicist, Philip Anderson, and his famous quip that, you know, more is different, which, you know, on the face of it sounds pretty banal and self-evident, but it obviously resonates with you. And then the concept of complexity emerging as a title, indicates, you know, looking back on the highly improbable set of conditions that led you to be, you know, sitting here talking about this wonderful memoir slash science book. And we have to talk about the last

chapter because I do feel like that could be a separate book. But we talk about that later. What do you make of that statement that more is different? I mean, is it is the converse true? You know, less is similar? How do we reconcile these pithy statements with with kind of the deep complexity that I may be hiding in this kind of little three word sentence right when I think of more as different I think of it as a catchphrase for the idea this program that you tell me when I go wrong here because I'm a visitor here

that this program that physicists had that if they just understood the basic interactions of particles then the world would be understood and that was just grossly naive it's just not true at all and it was the second or third time it wasn't true Newtonian kind of clockwork universe ideas two centuries earlier before, you know, Anderson wrote this and the, you know, people used to mock physicists for, you know, having so-called solid state.

They call it squalid state because it was this, you know, dysfunctional aggregation. People trying to take the microscopic and extrapolate to macroscopic. Exactly as you're saying. And now we call it condensed matter physics, which is even worse name in my opinion. But yeah, so go on. So, yeah, this. So the idea, it's very straightforward in like the physical hierarchy of man. matter. So like, just because you know how atoms work doesn't mean you understand chemistry,

just because you know how chemistry works doesn't mean you understand biology. You can keep going, right? In biology of the brain and so on. But biology is a bigger leap, but biology, excuse me, neuroscience to society. That every single one of those levels, you need theories at that level, right? And that's really what he was getting at. And so I really enjoyed that thought piece because it spoke to me about my own experience in the world of if I wanted to study, have a scientific program to understand how I got from A to B, how would I do it? And basically the answer is I couldn't.

And I kind of just have to sit with that answer. The last section of the book, you call it an afterward, which is normally a place, you know, people kind of wax rhapsodically about the contents of the book. You actually go into basically a very technical digression about the applications of the different types of technology and the influence that AI is having on the world.

I wonder, as I said, it's six pages long or something, but I felt like it could be a book in itself. Totally. Did you, maybe you have plans for, you know, follow-up, you know, emergence too, you know, in 3D, I don't know, Ms. Pac-Man could have been in 3-D. What are your thoughts? I mean, the main, you know, kind of takeaway is,

I see you as an optimist. Yes, I am. And it's in contrast. A lot of people we've had on the podcast, Max Tagmark, others, not so much. So what do you make of this kind of battle brewing for AI safety, alignment, all sorts of other things, if you would? So first, to address the questions about the book, it's a braided memoir. That means there's two stories, the story of my life.

And I also tell the story about how neuroscience and some ideas from complexity theory had led to, well, it's really neuroscience, led to AI. Right. And so at the end of this book, there's a conclusion to both stories. And the last chapter of my personal story ends on what happened to myself and all these different kids. And then the afterward is really meant to be the conclusion to the AI neuroscience story. So that's how that works in the book. In terms of, you know, the second part was you're just asking for prognostications on where this is going? Yes.

Yes. I'll be honest with you. I don't know where it's going. I am an optimist. You know, like something happened in the last few months that everyone is really on edge about this stuff. And I think it's probably because the models got better, right? And now this idea of just big data, there's reinforcement learning going on. These models aren't sort of improving based off of these things.

But I think there's, to me, what's going on right now, the zeitgeist of this very moment is that people in positions of power, decision makers, managers, CEOs, they're all just finally catching up with it. So we're having what I believe is really a very human moment, right? And that, first off, these things, they're not robotic. They don't interact with the world as we do. You know, to come here, I had to coordinate with 17 different people. You and I are having a conversation.

There's humanness in everything we do. And, you know, everyone's worried that, especially in software, like, I certainly think software is one of the first things to really going to be impacted by this because so much the infrastructure is ready, automated. So much of software's text and so on and so forth. But like, anyone who's been senior in software engineering knows that. senior software engineers are largely talking to people all the time. Yeah. Design jobs. You're just, I'm having trouble swallowing this like, you know, we're all going to lose our jobs.

I definitely think we should be paying attention. I think everyone should be. Some jobs, yeah, for sure. Yeah, definitely some jobs. But like, rather, I think we should be adapting to these technologies. And I also look at it a little bit like a pointillistic painting, you know, that for a long, long time, programming was like doing one of these paintings point by point by point. And through time, it's gotten better.

We used to program assembler. Then there was C. Then there was Java, whatever. Then there was Python. So we slowly abstracted away the machine. And so I just think that, you know, one view of this is that we're abstracting now to just say, hey, go do this thing or go do that thing. And then figuring out ways to combine those. Let me end with a caveat.

I can't predict the future. So I don't really know. I'm in it just with the rest of us, which I think is really cool and exciting. Yeah, it's a fun place to be. Hey, everybody. I'm usually the one that asks my guest. to judge their books by their covers, but today I'm asking myself to judge my own book by its cover. My newest book, Focus Like a Nobel Prize winner, is chockful of advice, life tips, and focus

and productivity tips from nine of the world's greatest minds. Nobel laureates ranging from economics to peace to physics, of course. So go to Amazon and get the Kindle copy today. Okay, so let's pivot to this force algorithm network. Let's go through it. And then keep in mind, you know, most of the audience, astronomy literate and very scientifically literate. And very scientifically literate, but may not be as neuroscientifically literate. So take us through, what is force, what does it do, how is it involved, and maybe how does it differentiate

itself from things we might be familiar with? Yes, sure. So let me back up to back propagation as the means to train feed forward networks. This was a thing that was developed in the 80s, and because of the slowness of computers, it really never took off, and we had this

so-called AI winter in the 90s. So it turns out, let me just, just one of the punchlines is like, well, we just use back prop these days on big complicated networks, even recurrent neural networks. But when I was doing my PhD, we didn't know that. There's a lot of, like a lot of tricks of the trade that were developed that have slowly just become part of the collective knowledge around neural networks whereby things just work now. Plus we have GPUs. Yeah. But in 2000, say, 2004, 2005, we really wanted to understand how recurrent neural networks functioned. This is again artificial neural networks with recurrent connectivity that make them nonlinear

dynamical systems because we want to understand how brains function. And unless you're working in sensory area, so vision processing, auditory processing, that there the idea is that the sensory processing can really be well approximated through feed forward cascades of information transformation. If you're not in that realm, if you want to think about motor control, if you want to think about executive decision making, then you really need to have recurrent networks. So that then boils down to a task of, well, how do you do it? Okay, we make this little artificial network. And then what, well, how do you train it?

Well, back prop, it's known that back prop doesn't work for RNNs. It turns out it does within limits. But there's proofs about this. There's an unstable eigenvalue no matter what you do. And the Jacobian goes backwards, blah, blah, blah. It just isn't going to work. And so what forced learning was, is my PhD thesis with Larry, was in a attempt. to solve that problem.

And what we did is we built off of some work in the echo state literature. You basically give up on training the internal connections of a network and you just train the outputs. And the idea is you have this very rich non-linear system. So if you drive an input into it, then you get all these harmonics and phase shifts of that.

And so you can reconstruct those through a linear regression to get whatever target you want. We figured out a way to close the loop so that even though you're training the outputs, you're effectively training the outputs, the inputs too. So then you run into problems of stability and you run into these problems in and around chaos and how do you control chaos. And so forced learning turned out to be this really powerful way to solve those problems. The upshot there is that, you know, around 2008, 2009 for the first time,

again, with the caveat that LSTMs were always out there and nobody knew, that was the first time that this community of neuroscientists and computational theoretical neuroscientists could train recurrent neural networks for the purposes of making contact with brain data. And therefore, of course, the goal is to try to understand how brains were. And let's start with monkey brains. At least that was the inspiration. That was with Omer, right? Yeah.

So the idea with Omerie was, so now we could train these networks. But what good is a toy model when you don't understand your toy model? Right. So the tried and true tradition in kind of all of science is you get some data. You study that data through lots of effort. and rigorous analysis, you build a model by hand. You decide what you think is going on in that data. You build the model, you test it on some held out data

of whether or not it explains or predict something. And then you say if the explanations make sense and the predictions work, that's my model. There's a long history of doing that in neuroscience too. And there's been progress doing that too. I would argue that today, the field is in a place where we have these high dimensional electrodes that we record with.

So now we're getting thousands of units it's recorded thousands of neurons. Drowning in data. Yeah, just drowning in data, not really understanding how to make sense of it. And so now really the argument is let's train neural networks to see what they do. That then requires you to understand how those recurrent neural networks function, these artificial networks. And so what had happened is in an effort to try to understand any of this, I took a graduate

nonlinear dynamical systems class. And we studied CAM theorem and we studied, we studied all this stuff. We've studied, but we learned this really basic stuff. Like if you have a simple fix point and a tractor in this landscape, then first off, the activity around there doesn't move. You can linearize around it. And so then the question becomes, well, a linearization, there's no guarantee about how wide in the volume. I just like, hey, can we do this? You're like, would that dumb text?

That's not dumb. Very simple technical methodology work. And so I was asking around. And I was like, no, probably not. And Omen was like, why wouldn't it work? And so we started collaborating on it. And so we developed, you know, when we really drove the technical side of this, some optimization methods to find these fixed points and linearizations in these networks.

And what we discovered, at least for simple cases, and simple gets is a set that grows with every year, you could understand the functioning of a recurrent neural network by virtue of its fixed points and its linearizations. So that compresses the problem and you don't have to worry about the global landscape? I mean, I want to be careful that. You may have to worry about it. But what I think we're discovering is that they're in the world of dynamical systems, and there's really crazy things out there, like the Lorentz Attractor, or like just these iterative maps that have crazy chaotic behavior.

These techniques aren't going to be altogether helpful there. I think computation is a regularizing thing. The fact that this dynamical system does something with computation, it makes it simpler. It makes it regular. I want to be, this is more simple as complicated. Ironically, yeah.

Anyway, so like, like linearity is a good idea until it isn't. Yeah, and then it breaks, right? Yeah, until it isn't. And so if you sort of view it these systems as maybe like switching linear, it's linear over here because, well, you need to add up numbers. And then it's linear over there because you need to do an oscillation, whatever, right? So then it becomes, you've reduced the problem to saying, well, how do I switch?

these systems from being an oscillator to being an adder. And so in that sense, I think we're lucky. And so the complexity is really in the fact that if you have a very powerful computer type system, let's say a calculator, let's say you trained an R&N to be a calculator, right? What would be those structures? Because they would probably be fairly sophisticated in order to handle 12 digits of multiplication, et cetera, et cetera. What I would say is it's not in the in the chaoticity of the system that I'm worried about. It's in just the overwhelming complexity of size. Just you're overwhelmed. The thing I wanted to ask is what you're never supposed to do. Whenever you read a book, you're not supposed to judge it by its cover.

Hey, book lovers, we're judging books by the covers. We know we're not supposed to do it, but it's a the impossible, there's nothing to it. Let's take a look and judge some books. So I want you to take the audience through the title, the subtitle, and the beautiful, really beautiful cover art. The publishing agencies are like, you should really lower your standards on book covers because the process is hard and it's just the iterations and who knows what's going to happen. And the first one they showed me, I was like, that's it. I love it. This is gorgeous. The title is emergence and that's meant to be used in two different ways. One is the story of my life coming up from fairly difficult circumstances. My parents were both drug addicts and ending up in science and having a career neuroscience and AI. And the other is meant to invoke the nonlinear

phenomenon of emergence, the idea that when you combine simple parts together, they create things that could never really be predicted. That has sort of informed how I think about my work. And I thought of it as a nice scaffolding for how to tell a story about somebody who came from really impoverished beginnings to where I am today. Yeah, the title emergence has come to mean a lot of different things. And in particular, your research, which we'll get into, kind of presage this moment that we're in, where neural networks, which were the backwater,

I remember we're about the same age, I'm a couple of years older than you. But I remember in the early 90s, mid-90s in grad school, people working on neural networks, oh, it's nonsense, the brain is on a computer. It's completely different, something different altogether. How does it feel to witness those techniques emerge as indisputably the victor?

I'm not saying it's a good thing, and I'll get into that later, but to just effectively take over, literally seems to be eating the world. I think it's very exciting. You know, for my part, I feel like, you know, I was a contributor to that, but there are lots of contributors

and many much, much with greater impact than me. But for all of that, yeah, I feel like it's an interesting and exciting moment and I'm happy to be alive while it's happening. I want to study these things. I want to study the brain. I want to use these things to study the brain. So I'm excited about it.

How do you react to the explosion of the technology, not just in the research that you and I do, sometimes a backwater compared to what the impacts are on society. Certainly impact of cosmology on society isn't very big. But how is it that these things are influencing? Supposedly people are not needing doctors anymore. Software service companies are going to go out of business. Market cap gets destroyed every other day when Anthropa comes out with some new product.

People are forming synthetic relationships. Some people allegedly have been accused of self-harm perhaps because of the influence of these bodies. How does that impact the research that you do? Or do you just kind of ignore it? Or how does it impact you? The thing that I'm really focused on right now is just broadly the disruption, right? What happens when you have this technology that for all the things that you just said is impacting people's lives in ways that never really happened before?

And so there are a few precedents. I mean, we can look at what happened with the Internet. We can look at what happened with social media and so on and so forth. And I don't want to be a Polyana. I think there are some potentially very terrible outcomes if we're not careful. But I've got to say, you know, if I had a chat bot when I was seven or eight, it would have been amazing. I'm very bullish on these technologies to help educate kids, to help educate people in third world countries and so on and so forth. So I guess that would be my answer.

How when we look at the types of neural networks that you study and you'll hopefully define how they work. But in the presence of noise, you know, the human. and brain is a supercomputer that works in a wet, sloshy environment. You know, some work better than others. But they're basically very simple organs. They can be decomposed into neurons and axons and all sorts of nerve functions. And many other creatures have them, but don't have the ability to create things the way that we do. How do you see that in the context of emergence?

These structures emerging to be the most complicated object in the known universe. I'm an expert in that latter subject, but not the former except for my name is an anagram for brain. Nice. But how is it that the most incredibly complex thing can emerge from something? And maybe take us to a simpler level, perhaps. How does cognition, language instead of pattern recognition, emerge from these RNNs that you study and first define what they are? Yeah, sure.

So I would just push back that I don't think of a brain even as an organ is a simple thing at all. No, I don't think at all. Okay. So what the last, I guess, 7,500-ish years of thinking about brains from a computational point of view really shown us is that like when you abstract away the details of all of the biological detail and you say boil down a neuron which is a very monumentally complex biophysical system get rid of the DNA get rid of the

molecules and all of the ion exchange that's happening to actually propagate electrical activity and just you know say this unit is a simple unit that we're going to call it an artificial neuron which I think many people would be familiar with where it receives these inputs if the sum of those inputs are greater than some threshold value you emit your own output otherwise you don't right so that's the artificial neuron and what the last 75-ish years have shown is that when you connect those things this idea of connectionism that the magic of all of those types of computations that you're talking about can

emerge because those kinds of neural networks can compute anything and so the kind of thing that I've studied is a so-called recurrent neural network oftentimes in neural networks you have basically three varieties now you have the feed-forward network sort of just an assembly system for the information comes in. You transform it, you transform it, you transform it, you get an output. You also have transformers, which are the new hot thing that everyone is building the AI off of.

I'm going to skip that. And then you have recurrent networks, which are relevant really for understanding how brains work, which is that the connections from the neurons feed back other neurons that are connecting to them. So you create loops. And in the process of doing so, you create a dynamical system, dynamical system that can sustain activity that can hold a memory and so on and so forth. And so those are the kinds of the networks that I've focused on.

And what the brain really is is just this enormous high-dimensional non-linear system, at least at the electrical level. When you think about non-linear systems, we think about sensitivity to initial conditions. I want to get into your initial conditions in a moment. But talk about the hallmarks of chaos, of noise, of disorder in the brain or maybe just in the R&Ns. What is the role in that feedback loop of what's the perils of possibly injecting unwanted signals, biases, noise, and that other? Yes, so things can just go off the rails, right?

So unless you have a system that is highly controlled, potentially with feedback loops for control of that system, then things can just go off the rails. That could be true in a very small, low-dimensional, you know, three or four-dimensional system of equations. Certainly could be true in a high-dimensional system like a brain. There are classic results now from the 80s, I'm Sam Polinsky saying that if you have a big enough recurrent system and it's randomly connected, it will either have no activity or it will be chaotic. So you kind of have to contend with that. We don't know all the ways that a brain is doing it, but it's a very deep subject and we should study it. And, you know, for training artificial networks, we have some ideas. When many people, you know, first got exposed to the notion of this nonlinear dependencies, it is in the context of what's called chaos theory.

I remember in the book, Chaos. came out when I was a freshman and you were in high school, you know, I was in college at your bitter rival, Case Western, Go Spartans. That book came out, James Glick, pioneering, you know, kind of monumental book. But one of the things that, you know, really appeals to me about your memoir

is that you show the candid way in candid detail how people that started of basically in identical circumstances came out to be so, so different. That obviously isn't lost upon you. I mean, you spend a good deal into the book writing about it. When you look at that,

what are your kind of, you know, aside from wanting a chat about when you were seven, I mean, what sorts of things could we do to engage, you know, so that people are maybe less sensitive to initial conditions? Or is it irreducible? I have a feeling about my own life, a little bit of like there, but, you know, for the grace of God, you know, would I be in some other place, right? Very potentially bad place, you know. So I follow the outcomes of my best friend in second grade. I follow the outcomes of some of the people I met in the group homes that I lived in, my sister's outcome. Esther, what I do, and we just see where things, what happens to these kids, and the outcomes are all over the place. And some of them are quite sad. In that sense, not being able to understand, you know, why I ended up the way that I am. I can definitely point to things. Don't get me wrong, I can definitely armchair and be like, yeah, well, I think this, that, and the other thing.

And we should do that. But like, when I really sit down and think about it, I have to say, it feels like we're just blown in the wind in the chaotic system. I really, I really, I really mean it. You know, like, I have this sense of gratitude that I, that, that, that, that, that, that didn't happen to me. Let's just take the reader and listener back rather. So you raised basically what we used to call an orphanage. We don't really call it that anymore, right? That's right? But essentially that's what it was. Your parents are both drug, drug addicted, uh, dealing with that as you were from birth, basically, and presumably, maybe even from before birth. And you have an older, had an older sister, tragically, she did pass away.

I mean, it's so improbable. I mean, it is like, talk about playing Pac-Man, you know, playing an arcade game. It's like spawning into, remember the three-dimensional version they tried to make a Pac-Man? It's so improbable that you're sitting here today working meta-reality labs, you know, Stanford adjunct professor, having gone to, you know, gotten your PhD at top, neuroscience program, perhaps in the world, Columbia, and Carnegie Mell and all these other things that are so improbable. But it's kind of depressing if it's like just.

because you're an awesome human being, that's how you did it. Like everyone, if you're as awesome as David, you can do it too. But I always feel like that's kind of satisfying. It's a set message. But maybe it's true. I mean, don't sugarcoat. No. So, yeah. So what I would say is if I look at the things that made a difference in my life, I think many of them are reproducible for their circumstances. I was born very smart. There's just a genetic gift I had no control over. Being smart is a useful thing because you can actually examine the world around you in ways that are beneficial. But, you know, when I was in, when I was in college, there was a guy who threatened to blow up the physics department. And he got kicked out. And he was a pretty bright guy. So there's,

there's clearly more variables at play here. What I would say is that what mattered for me is having a sense of the future. I had an out. Early on, my teachers would make me the tutor of the class or, you know, I was into the gifted and talented program. It became clear to me that there was this thing called intelligence. I had it and maybe there was a way out. We discussed it in the group home with some friends. So I would say if you had some kind of talent that other really because what we're fighting here is neglect, right? We're fighting neglect. And so if you have some kind of talent that, you know, an adult takes a notice of you and then gives you a sense of perspective into the future, hey, you know, if you do this, you can get out of this. I think that

that's very helpful. So it's not, it wasn't central that I was smart is what I'm getting at. I think that people involving themselves in my life. There were multiple people. Some of my aunts and uncles really got involved was really important for me. I look at some of the, however, impoverished the institutions that I grew up in, I mean, it was better than being on the street. I grew up religious. I think that some of the way to, how I structure my behavior now is a function of those

religious experiences. Religious experience has a specific connotation. as a function of those ways of living. Those were all things that were helpful to me. You and I were speaking before we started recording about your religious upbringing. I started some of my thoughts in religion as well. But it is sort of true.

I mean, there aren't many like the Richard Dawkins' orphanage, you know, or there aren't many like militant atheist orphanages and so forth. And it did sort of give you this insight that there's more to life than maybe just science at the early age when you were really interested and computers were coming online. You kind of had this amazing emergence, confluence of events that made you who you are.

But I also don't want to make it only seem like because you're so brilliant that that's how you succeeded. Obviously, the brilliance and your sister was very brilliant, too, as I understand from the book, at least. But then, curiously, at least from my perspective, you seem, you come down quite positively on something that's completely under your control, unlike nature, you know, nurture. And that's in the, you know, cognitive behavioral therapy and the therapy, you know, psychoanalysis that you underwent. Most scientists are not

positively predisposed to psychoanalysis. Popper, you know, was originally kind of developed his scorn for pseudoscience from Freudian psychoanalysis, which you talk about in the book, as kind of the hero, but you also mentioned its shortcomings. So talk about that. How did you reconcile that as a scientific,

you know, I always forget, what are we, right brain, left, I mean, you might be both. How did you reconcile that kind of skepticism that I presume you had with like this more touchy-feely kind I think it's the left brain. Side of things is psychoanalysis. I got out of high school and I went to Carnegie Mellon.

And I had four pretty good years there. And then I left high school. And my life kind of just fell apart. I was living alone for the first time. I think I was working really, really hard in college. And that was like kind of burned me out a little bit. But more importantly, it kept me from like actually taking a moment to take a deep breath and see like, whoa, I had this crazy childhood.

And so when I was in Boston, I lived for a year in Boston. And things just totally fell apart for me in a way that was really, really painful. And so because of that, I started reaching out for help, you know. And so my uncle is a psychoanalyst in New York. And so I asked him for a recommendation. And he gave me one. And I wanted to engage in this process. I was sick.

I needed help and I knew it. It's amazing you had that self-awareness. Well, it was pretty bad shape. Because it could have been much worse. I was borderline ready for a hospital. So coming to your point, I just engaged with the process. And what I would differentiate, and what I think, honestly, a lot of people, a lot of professionals really have two different ideas. One is like the things that Freud said, many of which have been refuted even by that community, versus the process of psychotherapy, right?

Just lots of new theory about what that is, right? And so what I would say is that it's just a very complicated process. So first off, there can be almost no doubt that this really helped me, right? There is no doubt in my mind. How do I reconcile that? I just think of it as trying to make, can science really make sense of such a complicated mode of interaction? If we can't understand what like a single Prozac molecule does when we can actually see what

that molecule is, how am I supposed to make sense in a scientific sense when you and I are talking right now and the sort of impacts that this conversation will leave with us on each other afterward, we're done. From a scientific perspective, that feels like quasars in another universe. I don't even know. It feels like really hard. And so because of that, I just accept that. You know, I know sort of empirically that this is worth for me.

You've written the book, and it's a beautifully written book. It's multiple books. It's not just one, you know, three. It's really two books braided together. Yeah, it really is. And it's, to do that. And I haven't read this book, but I used to have a sponsor called Short Form. And I got their summary of a book called Educated by Tara.

Westover. Westover, yeah. who's also raised, like, militant young Earth creationist and, like, they didn't take aspirin and things. But there's not much for my audience and say that book. I mean, I found the short form summary, perfectly, you know, adept at summarizing when I needed to know. But it's kind of bold to write a memoir, you know, for your first book, especially as an accomplished scientist and researcher and working in industry and business. I don't talk about that, too, that transition. What about the story you felt needed to be told?

And what sort of is the massive takeaway? I'll tell you what I feel. I feel like it's tragic what happens in foster care. And in orphanages, to the extent that they exist. I mean, as I said, I give a lot of credit. I'm not, you know, I was raised Catholic, but I'm not practicing Catholic. I'm practicing Jew now. But they do such a better job than any other organization on Earth, the Catholic Church,

taking care of young children. And yet, it's far for perfect. Siblings get separated. I mean, that's the most tragic. I have three brothers. That's one of the most tragic things. As much as I fight with them, I would not want to be separate from. And you were separated from these people that were your family, and you'd later reunite with them.

And some of them, starting from the exact same initial condition, branched out, none of them are Stanford professors. So what are your takeaway? What did you resolve differently after, you know, writing this book? And how do you expect the reader to react after reading the book? Yeah, sure. So I'll be honest with you. I wrote the book. So I've been in psychotherapy 20, 30 years, right?

Like, I have largely processed this material. I wrote it because I thought it would be a good story. As I wrote it down, that's when I started thinking about, like, well, what are the take-home messages here? And in my opinion, the take-home messages are just shining any light on the kind of experience that I had. I mean, foster care, I'm not an expert in this topic, but as far as I understand things, foster care is largely a failed experiment. What could we do different, you know, or how could we do something differently? It's worthy of having that conversation. I think many people could just sort of understand, like, what were the things that helped me, right?

There are moments of intervention, and why did those things help me? I think that's something that could be understood as well. But tell me, this foreshallengar, and talk about Larry Abbott, who I know from his pioneering work in the Axion, which is not the axon, very different, right? Tell me about working with him and the impact of mentorship, which I also see is a theme in this book. It's a story of how just a chance encounter. You might think it's a tiny thing, but anyone can do it. I always say, I don't know. You don't have kids, do you?

I do not. I've got some if you want to borrow some. But anyone can be a father or mother. You could be a biological father. It could be, but some biological fathers, our fathers weren't great fathers. You reconciled. I reconciled it eventually towards the end of my father's life. But anyone can be an ideological or psychological.

Father, mentor, mother, figure, big brother, big sister. But it's gotten kind of corrupted by Michael Jackson, the Catholic Church. You know, all sorts of other things. But tell me what the role of mentors. And let's talk about Larry and the lab work that you guys did together. I mean, Larry's just an incredible guy. I might literally take a bullet for that guy. Like, I love this man.

Yeah, it shows through in the book. He, of course, the Axion, he did this little study on the Axion way back in like 80, went ignored for three decades. We're trying to detect it now. It's a time. It's 500 citations a year, right? It's just incredible. So good for him, you go, you know.

But he pivoted in his 50s, I guess around 2000 to working on neuroscience. I guess he pivoted a little bit early, actually, in the 90s. But anyways, I came across him in the 2000s when I was educating myself in a PhD. And through just sheer luck, I ended up in his lab. And it turned out to be this fabulous experience for me where he and I really just felt very, there's a simpatico with us in terms of how we thought about things and the way we worked. And he had this simulator. He would actually use the computer to do a lot of his work as opposed to doing just math.

And so I really related to that because it felt like the video games and the programming that I was skilled in. And so we really hit it off there. And what I would say just, just, you know, before we get into the technical stuff, what I would say about working with Larry is like our relationship just grew. And so there was a stability there. Larry didn't come out and said, you know, hey, Dave, I'm going to be a father figure for you, right? It just sort of happened. I don't even know if he would characterize it that way. But like we have a very strong friendship and mentorship at this point.

And it just developed and grew over the years. And so he's been a mentor to me even today. I'll give him a call, you know, three or four times a year. That, I think, is hugely important to kids especially. I don't think you have to take on the full burden. You just have to be this point of stability where a child can pivot around that, can rely on that can build upon it because what really happens in foster care is that you just live with so many people. You are effectively anonymous, right? You just are. So I lived with literally, I lived with

12 different sets of parents from the age of 7 to the age of 18. And nobody even knows what happened to you in the third grade when it will by the time you know, nobody even knows. How are your grades? I guess there's a record of that. Did you get your ass kicked in school? I mean, like just on and on and on. Be down. Yeah, beat down. So in whatever you do, do when you interact being a point showing up is really the thing i would say the single biggest predictor of you know success in a lot of ways is if you had a male role model growing up in in your life and a father figure in particular possibly because women are so much more biologically and sociologically conditioned to be around the kids it's rare that women leave their kids completely but the fathers it's very common

and that's number one predictor of you know the the boys you know next role model is most likely to be a prison guard right i mean that that that that happens in society. Yeah. I just think the barriers to entry need to be lowered. We need to do due diligence, you know, and obviously make sure screenings are done such as a mentor, you know, big brothers, big sisters, whatever, are safe, you know, environments because that's the absolute worst thing that could happen.

But I want to end with a beautiful quote from the book that you made, which is, again, this interwoven memoir, science, nonfiction with a factual life that's unique. There's really nothing else like this. If you liked Tara Westover, if you liked educated, if you like Rob Henderson's book, a recent one about his life in the foster care system, I believe, as well. I haven't read either one of them, but short form, if you're willing to come back on the Into the Impossible podcast, we'll have you back on. But you say beautiful prose, the promise, the commitment was a promise to myself that I would not end up like my parents. It was a vow to escape the chaos to find a different path. It was a declaration of war against the entropy of my own life.

It's really beautiful. Think about how you can use both sides of your brain, both of these hemispheres here, which we separated. I think it's three different had Alice in Miotrion, and we were taking it apart, and he was showing me different things about it. David Cecil, thank you so much for coming down to San Diego, and enjoy the rest of your day.

And everybody, this book is now out by the time the interview comes out, and the audio book is read by David. So you'll enjoy that. Better than the version I made myself read by Snoop Dog. That's another topic. Thanks a lot, David. Thank you for having me. Yeah, bye-bye.

Hey, you made it all the way to the end of this episode, so I know you're going to find fascinating this conversation with Dr. John C. Lennox. One of the greatest and most illuminating mathematicians and thinkers of our time. He's a delight to talk to, and I know you're going to enjoy our warm, loving conversation that we had about the perils and pitfalls of trying to turn AI into our next god. Click here for that. Don't forget to like, comment, and subscribe, and share the episode with a friend. See you next time. I'm Into the Impossible.