Theories of Everything with Curt Jaimungal - How Free Energy Constructs Our Reality | Michael Levin and Karl Friston

Episode Date: February 14, 2025

Curt Jaimungal is joined by Michael Levin and Karl Friston. This conversation incorporates insights from physics and information theory, particularly regarding self-organization and the significance of entropy and free energy. As a listener of TOE you can get a special 20% off discount to The Economist and all it has to offer! Visit https://www.economist.com/toe Join My New Substack (Personal Writings): https://curtjaimungal.substack.com Listen on Spotify: https://tinyurl.com/SpotifyTOE Become a YouTube Member (Early Access Videos): https://www.youtube.com/channel/UCdWIQh9DGG6uhJk8eyIFl1w/join Timestamps: 00:00 Introduction 3:43 The Free Energy Principle Explained 5:41 Creativity and Adaptive Utilization 11:56 In-Painting vs. Out-Painting 15:33 The Unreliable Medium of Biology 20:05 Aging: Noise or Psychological? 25:25 The Nature of Selfhood 45:10 Distinctions in Organic and Psychological Disease 48:54 Goal-Directed Systems and Aging 52:32 The Dynamics of Life and Death 55:49 Continuous Self in a Changing Form 1:01:35 The Constructive Nature of Science 1:08:02 Inferring Actions and Counterfactuals 1:11:40 Closing Thoughts and Future Conversations Links Mentioned: - Michael’s website: https://thoughtforms.life/ - Karl’s publications: https://www.fil.ion.ucl.ac.uk/~karl/ - Michael’s previous appearance on TOE: https://www.youtube.com/watch?v=2aLhkm6QUgA - Karl’s previous appearance on TOE: https://www.youtube.com/watch?v=uk4NZorRjCo - Michael on Anthrobots: https://www.youtube.com/watch?v=hG6GIzNM0aM - Michael’s paper on stress sharing as cognitive glue for collective intelligences: https://www.sciencedirect.com/science/article/pii/S0006291X2400932X?ref=pdf_download&fr=RR-2&rr=911840d57c51eace - Karl and Michael with Chris Fields on TOE: https://www.youtube.com/watch?v=J6eJ44Jq_pw - Karl Friston on the ‘Free Energy Principle’ on TOE: https://www.youtube.com/watch?v=2v7LBABwZKA - Michael’s recent paper with Chris Fields: https://www.sciencedirect.com/science/article/pii/S1571064525000089?dgcid=coauthor - Top-down models in biology (paper): https://royalsocietypublishing.org/doi/epdf/10.1098/rsif.2016.0555 Geoffrey Hinton on TOE: https://www.youtube.com/watch?v=b_DUft-BdIE Support TOE on Patreon: https://patreon.com/curtjaimungal Twitter: https://twitter.com/TOEwithCurt Discord Invite: https://discord.com/invite/kBcnfNVwqs #science #podcast #reality #mind #consciousness #theoreticalphysics Learn more about your ad choices. Visit megaphone.fm/adchoices

We've all conversed extensively about information previously, but what is the meaning of this information? What precisely is the distinction between information and meaning? For me, what I'm really interested in, and I'm not going to try to give a mathematical definition or anything, but what I'm really interested in is this polycomputing perspective where you focus on the observer. So there's some given set of physical events and there are one or more observers that are choosing how to parse what they're observing in different ways.

So that might be time scale, that may be granularity, and then the ways in which they fill in things that aren't actually in the data at all relative to their expectations, and most importantly, the way in which they will use the patterns in that data to do something moving into the future. So adaptive utility of whatever's there, that to me is the most interesting thing about information. It's on the receiving end and how much processing and

creativity is used by agents to do something interesting with it. Carl, do you have any disagreements as to how information becomes meaningful to a cell or an organism or to a society? What's precisely this difference between information and meaning? Yeah. I just recapitulate what Mike has just said, but using the language of a physicist. So I think you'd have to start just by acknowledging that most of the physics that's brought to the table to explain this kind of self-organization that has some meaning for the things that are self-organising rests upon

information theory. So you have very elemental concepts such as self-information, which is just the implausibility of an event as enumerated by the negative log probability. The average of self-information would be entropy and that's an important measure. And then you can work up to sort of things like free energy as an information theoretic measure of the quality of any self-organization. But I think to sort of speak to Mike's point and to your question about the distinction between information in an information theoretic sense,

possibly even a Shannon-esque sense, and meaning, I think you really do have to take a relational or observer position. So from my point of view, under the free energy principle, the whole point is that if you've got a separation between the observer, and the observed, you now can understand the dynamics and self-organization of the observer as holding probabilistic beliefs or mathematical or Bayesian beliefs about the observed. I think that's where the meaning arises. So it's not the information theory of the observer, for example, neural activation, when I see a particular visual scene.

It's what that, what that pattern of activity means for encoding and representing the meaning of the sensory input, the visual input that caused that activity. So it's, if you allow me to use beliefs in a mathematical sense as Bayesian beliefs, then it's not, it's the, the meaning is inherent in what those beliefs are about and because you've now got the separation between the observed and the observer they are the beliefs of the observer about the observed which could be the rest of the universe it could be you another person that constitutes my niche but you know what I am observing so I think there's a really important sort of distinction between meaning and information,

which really takes you beyond Shannon information and information theory, and really forces you to make a distinction between the observer and the observed. Could I try something out here? I want to see what Carl thinks about this. Just to see if you like the whole premise or what you think. I've been thinking about information on a behavioral timescale,

a developmental timescale, and an evolutionary timescale from the perspective of this bow tie architecture idea. The idea that as an embryo or as a cognitive being, you don't have access to the past, which you have access to are the memory engrams that the past has somehow left in your brain or body. And that at any given moment as an active agent,

what you have to do is interpret those traces and then they make use of them in some sort of, you know, forward aimed behavior. I've been playing around with the idea that whereas the left side of that bow tie where instances of experience and so on are then compressed somehow into a generative model that sits at the center of this bow tie.

That maybe is algorithmic and mechanical maybe. But the right side seems to the interpretation of your own memories and the information that you've got from the genetics or from anything else, seems to be fundamentally creative. Because you've lost information, it doesn't seem to me that you can do that in

an algorithmic way, it seems like that's what we would call creativity, is being handed something and then you have to come up with different ways to make it relevant and make it useful to your future behavior. Do you think that kind of architecture is useful? That way of thinking about it is useful. If so, what are your thoughts on that creative aspect?

How do you uncompress these things where the environment has changed, you've changed, many things have changed, and now you have to make some kind of use out of this. So how do you see that, the right side of that funnel? That's a great question. I think the answer to that question would come in two parts. The first part would be a bit of a technical part is to fully endorse this notion of a sort of bow tie. You didn't mention light cones so you should at some point mention quantum light cones. I'm going to sort of just take another perspective on that picture and that perspective would be the kind of perspective that the people like

Teletishby would have taken which is the information bottleneck which basically emphasizes the importance of compression and one way of looking at this sort of bottleneck is over time very much in the picture that you were painting where the present is the bottleneck and just incidentally it is the simplest of Markov blankets that separate the observed from the observer but in this instance the Markov blanket is the present that provides a separation between the past and the future so it's

where it all plays out. I think compression is an important notion here because one way of reading the interpretation of another kind of bottleneck, which is just the sensory input available to a particular system or organism or cell is to infer the causes of that, you know, that bottlenecked input, that compressed input to decompress to disentangle and to find creatively perhaps I would say

more constructively, I think creation in a future pointing way. I think that's the second part of my answer, but certainly in a constructive way to reconstruct on the inside, the inside of the observer, the best explanation for this particular compressed sampling of the world that is available through your sensory epithelial through your, your, your cell receptors. And the maths of that is really interesting because you can also, well, in terms of interpretation, I would read that as imprints, so filling in the gaps, finding the best explanation for this

partially observed bottlenecked sampling of the world. That just is inference and that's important to say out loud, because any system that can be described as performing inference can also be described as performing a gradient descent or having some dynamics in which the the variation free energy is minimized simply because that is exactly the quantity used in statistical inference and variational inference, say for example a variational autoencoder that itself has a bottleneck architecture if you think about it. So if you can describe self-organization as a gradient flow or a dynamic that looks as if it is interpreting and inferring, then

you now have, I think, a formal link between the notion of creativity and the notion of the bottleneck from the point of view of compression, because as people like Jürgen Spithuber would emphasize or indeed people, the architects of the so-and-so induction and complexity, minimum description length, ways of describing this art of interpreting through inference. They would all, interpreting through inference. They were all, all of these interpretations would lend themselves to a compression notion, which basically provides one heuristic on why bottleneck architectures are so predominant.

Say machine learning. So you take a transformer architecture, a variational autoencoder, anything that has some encoder and decoder kind of architect, what's it doing, it's forcing it through a bottleneck, you're compressing it down in a way that it forces you to find a simple explanation for this content or for this data. So you could call that creative. But I think you were talking about a slightly deeper kind of

creativity, which is the future pointing, the planning aspect, what it means to me in terms of how I now go and sample that data and how I act upon that data. And I think that that's the essence of the creativity. From the physicist's point of view, my perspective, that creativity is really acting in a way to get the right kind of information that allows me to find simpler explanations for my lived world. If you can do that, you've got, if you like, a creative generalized homeostasis at hand. So on the biology end, you

talked about filling in the gaps, right? And so I see two aspects to this. One is the sort of in-painting idea, where it's gaps with, you know, sort of they're more minor in a way because they're within an existing structure. And you sort of fill in the things you don't know. But overall, it's within a structure. And then what we see often in the biology is a kind of outpainting where you're going

further into areas where the past has not prepared you for. And just as a simple example, the anthropobots, we recently, right? So these are these self-multi-little creatures that are formed from human tracheal samples. They self-assemble. We recently studied their transcriptome, and we found out that about half of the genes in the human genome are differentially expressed in these anthropos. And so we haven't touched the genome.

There are no synthetic biology circuits. There are no nanomaterials. There are no weird drugs. It's just they have a new lifestyle, a new environment, and they have radically altered the way that they express the genes that they've been given from their past experience as human embryos and then humans.

And the xenobots too, there's some amazing stuff that we fished out out of novel transcriptional motifs that Zenobots express now in their new lifestyle that actually allow us to communicate with them in a new way. This will all be forthcoming shortly. And so just this idea that there's the in-painting filling in of gaps where your environment is in small ways different than what you knew, but basically the old tricks still work.

And then there seems to be this is what I meant by the creative, right? It's that you have to go beyond what you've already seen and use the affordances you have, the things that the genome does give you, in novel ways for a new lifestyle that is clearly not random, clearly not just a trial and error. It's got some sort of a coherence to it that's actually quite adaptive in these new environments. So that's what I'm really interested in is that kind of

problem solving where it goes beyond filling in and kind of out into completely novel scenarios, you know. Is this not the same problem that you and I shared Michael when we were birthed? Yes, yes, by the way, absolutely. I know, I 100% agree. I think that the reason that these living things have all that plasticity is because they're never actually committed to the standard embryonic story anyway. They'll take that story under default circumstances, but I don't think they, with very small exceptions, maybe nematodes, you know, where all the cells are numbered and they kind of all do the exact same thing. The vast majority of us, all of living things, I think have to solve this issue from the beginning. Yeah, I agree with that.

So that speaks to a sort of interesting application of scale-free polycomputation. So my genotype then is basically my conspicuous or my species memory of the kind of world, but not the particular world that I have to creatively maintain myself in and self-organize within. So are you saying then that the genome basically equips me with a kind of plasticity and the ability to learn an unrealized environment and unrealized as yet world which we have to go construct with each new generation. Yeah, I would say that the genome is not so much a memory of the past world as much as it's a memory of various tricks that together can comprise a problem-solving agent that then might be exposed to the same things that you were exposed to before, but actually might not be. And long-term, you know that you can't just stick

with what you have because you know you're going to be mutated, your own parts will change, the environment will change, right? So most recently, I've been really working on the implications of biology as a very unreliable medium and the fact that unlike current computational devices, it's not about error correcting in the sense of let's stick to the same meaning of the data that we had before.

Conversely, we know that nothing we see is reliable and our past ways of dealing with things cannot necessarily be counted on. And so what the successful life forms do is create problem solving agents that do their best, you know, kind of beginner's mind kind of thing from scratch. So I would say, yeah, I think more and more that's what I think the genome is actually giving you is a bag of information processing tricks as opposed to specific

solutions for specific past problems. So that's one step up from Stuart Kaufman's selection for select ability or second order selection. It's actually, it's not just, um, selection for select ability. It's actually selection for learn ability. Yeah. And we see, right. If we, um, so, so I've had, uh, I've had a couple of papers now with Laxwin,

Srishe, and Ben Hartle where we actually model what happens when you take seriously the idea that the layer between the genome and the phenotype is actually intelligent, meaning it's not just a complex mapping, well, certainly not a linear mapping, but it's not even just a complex mapping, it's actually a mapping that's a problem-solving system. When you do that, the dynamics of evolution are quite different, because what it ends up doing is by being able to fix, as we see in embryos, they're able to repair all weird things that happen to them.

By doing that, it hides information from selection about what was actually in the genome. Because when selection sees a good embryo, well, was that good because your structural genome is good? Or because it was actually quite a mess, but you fixed it along the way, right? And so when that happens, a lot of pressure gets taken off of the actual genotype, but all the work tends to be done on the competency itself, on the problem solving capacity. And if you take that all the way to its conclusion, which I think is what Planaria have done, is basically because of their lack of Weissman's barrier in the asexual strains and the fact

that they accumulate every mutation that doesn't kill the neoblast, just spreads into the next, you know, into the regenerative body. Because the hardware is so unstable, they've basically put all their effort into creating an algorithm that will build a proper worm no matter what happens to the hardware. And then the rest of us are sort of, you know, with salamanders and then mammals and then C. elegans, you know, we're sort of at different levels

on that continuum. How seriously do you actually take your genome, right? I think C. elegans takes it extremely seriously. I think planaria don't pay that much attention to it at all, and which explains a very weird observation. Well, actually two observations. One is that people have been trying, I think for at least 30 years, probably longer to make transgenic planaria,

and it just doesn't work. There are no transgenic planaria, and just doesn't work. There are no transgenic planaria and also there are no mutant strains of planaria. Unlike every other species where you can get Drosophila with curly wings and mice with weird tails and whatnot, there are no mutant strains of planaria except for there's two weird lines of planaria, ours that are not genetic, right? The double-headed form and the cryptic confused form, and those are not genetic. And so I think what's actually happening is planaria are, and we have computational experiments that show this,

is that there's this like runaway ratchet for taking pressure off of the structural genome and onto this competency, the intelligence of that layer that interprets the genome to give you something. I think that's what's going on. So I'd like to touch on something you had mentioned earlier, Karl.

You said that the present moment is the simplest Markov blanket. Can you explain? As you know, on theories of everything, we delve into some of the most reality-spiralling concepts from theoretical physics and consciousness to AI and emerging technologies. To stay informed, in an ever-evolving landscape, I see The Economist as a wellspring of insightful analysis and in-depth reporting on the various topics we explore here and beyond.

The Economist's commitment to rigorous journalism means you get a clear picture of the world's most significant developments. Whether it's in scientific innovation or the shifting tectonic plates of global politics, The Economist provides comprehensive coverage that goes beyond the headlines. What sets The Economist apart is their ability to make complex issues accessible and engaging, much like we strive to do in this podcast. If you're passionate about expanding your knowledge and gaining a deeper understanding of the forces that shape our world, then I highly recommend subscribing to The Economist. It's an investment into intellectual growth.

One that you won't regret. As a listener of Toe, you get a special 20% off discount. Now you can enjoy The Economist and all it has to offer for less. Head over to their website www.economist.com slash Toe, T-O-E, to get started. Thanks for tuning in and now back to our explorations of the mysteries of the universe. Can you explain? Simply in virtue of the fact that everything I need to know about the future that can be determined by the past is written on the present.

So this is just a definition of any Markovian process and assumes of course that we live in a Markovian universe as described by things like the L'Oreal equation. So reading a Markov blanket as a set of states that provide a separation in terms of conditional independencies, that's the definition of a Markov process that the current observation has within it, everything you need to know that is useful in predicting the future. And in a sense, that's the that's one of the foundations of the information bottleneck

approach to this and notions of predictive information that, if you like, is passed through this particular bottleneck which is the present, which I, you know, could be a mathematical, a useful mathematical device when thinking about the kind of what Mike was talking about. And I should say, it's always a delight to listen to him. He's encyclopedic when it comes to your biotic self-organization. There's a number of examples that he has at his fingertips, it's really illuminating. But what struck me is that Boncourt, he was almost arguing for a return to Lamarckian evolution, which the EvoDivo people would love, the psychological or a combination of evolution and psychology that puts sort of cultural niche construction into the mix

which is all about problem solving in a creative way. I would just write that down as your inference towards good models that have high explanatory power. But it may be that that sort of evo-devo perspective on, I think I might call it polycomputation, may be nicely articulated through this sort of predictive information notion when thinking about how generations and phenotypes within different, how you get sort of, what is past transgenerationally? Is it the knowledge about how to solve problems or the knowledge about, the knowledge of how to at a higher order, solve any problem that, that solve problems that have not been, have not been solved before. I was very badly expressed, but I'm trying to get to a third order selection basically. Carl, your model has noise as something to be minimized or error as something to be minimized,

but I'm curious if this noise is also a source of creativity and novelty. So I guess, Michael, you can speak on that even though it's about Karl's model. I want to know what you think of that. Yeah. I think that the question of what's noise and what's not is very much in the eye of the beholder. And so trying to cut up the, you know, and I think Carla will say that it also has to do with what you were expecting to measure and the things that are outside of that you might interpret as noise or you might take that as an opportunity to compute something

else. So I think noise is interesting, but I think it's just part of the bigger issue that even in the deterministic system a finite observer that has no actual noise, a finite observer is going to have to pick some things to take very seriously, and then some other things that it's going to treat as noise in the sense that it's going to just coarse-grain over it and ignore it and find invariants that are not paying attention to that at all.

Yeah, I have to sort of qualify responsiveness in terms of what kind of noise we're talking about. You know, you could be talking about the random fluctuations that are at the heart of the probabilistic statistical structure of the universe. So for example, we mentioned before that we can reduce the universe to a launch band equation, which just says that the states of the universe evolve in time as a function of the states of the universe plus noise. And those are the random fluctuations that provide a description of the universe in terms of probability distributions, they underwrite quantum mechanics, for example. So noise in that sense is foundational in terms of equipping this universe with a probabilistic description and the notion of inference and meaning in the kind that we talked

about before. Or you could have meant prediction error which I think is what Mike was speaking to. So prediction error is the surprise or the mismatch that would ensue if the way that I make sense of this world is not able to fully accommodate the sensory evidence or what the world tells me. That kind of prediction error has got nothing to do with the random fluctuations that I was just talking about. This kind of prediction error plays an essential role in belief updating.

So technically this the prediction error is actually this the variational free energy gradient and if everything if we self-organize in a way that looks as if we are being if we are interpreting if we are being creative and creating good explanations for our sensorium, then, and we are describing that as a, a gradient flow on variational free energy. What that is equivalent to saying is all my belief updating and all my self organization is literally driven by prediction error. So when the absence of prediction error, I would be dead or I would at least be at thermodynamic

equilibrium. So it is prediction error that drives the dynamics, the itinerancy of biotic self-organization. So that kind of noise, prediction error, is absolutely, it shapes self-organization and it underwrites the meaning of the kind that we'll be discussing at the opening of the conversation. Let me see here if I can tie some threads together such as the bow tie, Michael, and the self-organization you've just mentioned, Karl, in the present moment, how necessary is the concept of the now or even the experience of the now to the self? Well, let's see.

So what I've been thinking about is extending the, and Chris Fields and I have done some work on this together, is extending the notion of communicating laterally with other beings at your own sort of temporal level, applying that to communication with your past self and future self. So this idea that what you have at any given moment is a set of, let's say, engrams or information structures that are basically communications from a past self that has done some work on niche construction of your environment to make it easier or harder for you to do things now,

and you're about to do that to your future self by actions that you take now. So I think that center of that bowtie is really critical because it defines the need for the self to be an ongoing constructive process of storytelling and interpretation and searching for meaning and this, you know, this constructive aspect that Karl talked about, what I called it creative, but it really defines the job

of the self as this, basically, well, to use recursively the same word in a definition, I guess, is it's a self-telling story, in other words, right? It's a continuous process of trying to understand what you are based on what you've been communicated to by your past instances, and this happens, I think, it happens all the time. I would agree entirely. Ask a question back to Mike about that particular formulation. Before so doing though, it's interesting to pick out how often the word self appears in both formal and narrative accounts of being so we were talking before about self information.

the negative log probability of finding something in a particular state or an event happening. And of course the thing that you are attributing that self-information to, operationally, is the self, but in a really trivial and unmeaningful way. Self-organization is another phrase we often use. So we are implying that there is a certain kind of selfhood from the point of view of an observer, which does not necessarily imply that the thing that is self-organizing itself has a notion of selfhood. So I suspect that what Mike was talking about was this rather almost counterintuitive, but I think appealing notion that the observer is somehow creating an explanation for her observations that entail or rest upon a notion that the observer is itself a subject of observation.

So self observation, which is quite remarkable. And, you know, in my world, and we've spoken about this a number of occasions, it is just a fantasy. It's just a story, a narrative of the kind that Mike was trying to intimate that sort of ties the past and the future together to give you a sense of continuity. It is a story because you can certainly lose that sense of selfhood and certainly in certain psychiatric conditions where you don't have a notion of self. So it's quite a friable kind of story that we tell ourselves that you're implies a certain degree of persistence and continuity as we as we take our present of species present to move a market through this universe. One question that i've come back to my comp. Is to have this sort of continuity of self that transcends the moment. Would you need to have as part of your apparatus for explaining the world a notion of you changing the future,

a notion of acting upon the world and knowing the consequences of your action. So in my world, there's a bright line between authentic agents who actually imagine in their head as an observer of the world, what would happen if I did that? And as soon as you imagine these counterfactuals, you're now in the interesting game of selecting one or other to actually realize physically. So my question, Mike, would be, is the kind of selfhood you were talking about implicit in anything that self-organizes or is it a special gift or

attribute of certain possibly quite complicated systems that are able to predict their future or specifically, their counterfactual futures conditioned upon the way that they might act upon the Lord. Mm-hmm. Wow. I think that what you've described is a special kind of advanced kind of self, but I do see simpler versions of it going far down below the explicit recognizable ability to model forward various things that are going to happen and then choose one. I think that there are simpler systems that do versions of that as a, I don't know what the vocabulary for this is, but without having an explicit mechanism to do so, some of that ends up happening anyway as a byproduct of simpler things that they're doing.

Okay. I want to tie this, maybe this will be part of the answer to this. I want to tie this to a weird inversion that I've been playing with recently, and Chris and I just had a paper on this too. We'll see what you think of this. Under standard circumstances, we tend to think that thoughts and thinkers are quite different things.

Let's say in the Turing machine thing, you have the actual agent is the physical machine, and then you have passive data. You have patterns in some excitable medium, and the agent, it modifies those patterns. Perhaps an observer could see patterns in its cognitive medium, right, you know, thoughts of various kinds as patterns in that medium. And so what I've been thinking about is flipping it or more accurately dissolving

the distinction and asking to what extent can we ascribe the agency or to take the perspective of the pattern itself as opposed to the machine, to the thinker. So the idea is if we look at it from the perspective, okay, so on the Turing machine side, if we look at it from the perspective of the data is actually in charge. It's active data. The machine does whatever the data says. And the machine is in some way just a scratch pad in the world for what has actual agency, which is the pattern in the data. And then you could imagine sort of a continuum of different degrees of sophistication of these patterns. I mean, we know in different mediums, some or different

media, some patterns just sort of come and go and they dissipate immediately, others hold themselves together. Others might compute or process information. But in all of these cases, it's the actual pattern in the medium that's doing this. And so I wonder to what extent we can say like in an embryo or in a cognitive system, you could say that it is maintaining a story by

operating on these data and sort of concocting a consistent story that goes through time. But to what extent can we actually take the perspective of the pattern itself and to say that it's the pattern that propagates across and that the degree of continuity you have, and again, it's not a fixed thing, it continuously changes,

but the continuity is provided not by the physical machine that's doing it, not by the thinker, but by the pattern that in some way pulls along or resonates with the structure that either can or cannot maintain it. And if it cannot maintain it, then you get birth defects and psychiatric disruptions and so on. But in the ideal case, it's the story itself. And the reason I bring this up is not only because

there are some actual biomedical implications of thinking of it this way, in particular on bioelectricity, asking whether it's the body that tries to operate on bioelectric patterns or whether it's the patterns that are actually driving and pulling the biochemistry with it. There are some very specific experimental implications of that. But, you know, part of, I think you and I are in agreement on the need for the self to be a process, but I'm worried that it might sound to people listening that it's a kind of, and at least from my end, it's not meant to be a kind of deflation

or a nothing but-ism, you know what I mean? That it's like, well, it's a story and in a way that tends to reduce it, like it's just a story, it isn't real. And for me, I think it's the best kind of story there is, which is a, you know, a self-referential loop that actually locks itself into existence. And it's not the physical machine that might or might not dilute itself with that story. But actually it is the story is the thing. And the rest of it is just trying to sort of keep up and implement it.

But the structure of the pattern is a thing and to me it's a real and important thing. So what do you think about that, this idea of taking the perspective of the pattern as opposed to the machine and thinking about things like you can have fleeting thoughts and then you can have sort of persistent thoughts that do a little bit of niche construction in the brain are actually quite hard to get rid of, right? You know, I'm sure more about this than I, but Gio Pizzullo and I just wrote something on niche construction by mental constructs, right? And then you have things like, you know, maybe a personality fragments and then of course, full human selves and so on. What do you think about this flipping

the perspective from the view of the story itself? I think your notion of dissolving the distinction is probably the best, easiest way for me to think about this because for me, it is just a pattern. I struggled to find the machine that me it is just a pattern. I struggle to find the machine that is not part of the pattern. I say that in the spirit of mortal computation, which I know you would be fully committed to, and meant in the sense that Jeff Hinton originally introduced and Alex or B does has certainly promoted

recently. So from that point of view, mortal computation is just an acknowledgement that it is the patterns that matter. And that's where the dynamics are. That's where the self-organization is. And there is a substrate could be a sort of you know an electrochemical substrate or it could be possibly a von Neumann architecture but if you move into the world of von Neumann architectures and Turing machines I think you're outside the world of natural physics and natural intelligence and you're talking about simulation machines which we have built and it may be that that's what requires the dissolution, the realisation that simulations

of self-organisation and pattern formation on a Turing architect, on a Newman architecture with reading right to memory that become renders the software immortal because you can run that pattern formation on any Turing machine is a distraction and that the you know the really interesting questions and the physics of all of this and the interpretation and certainly some of the understanding practically the implications for biotic self-organization are actually in the patterns. So when I was talking about states before, I was just talking about the particular pattern at the moment without any scaffolding or any structure in play to, if you like, do the computing.

So for me, the pattern is the data. It is the dynamics. It's all the same thing. And then the trick is, where does the self come from? Well, it's when one pattern somehow distinguishes itself from another pattern. How do you do that? You have to separate them. How do you do that? Well, you define the inputs and outputs or

define a marker blanket or if you were Chris Fields, you'd you put a holographic screen in between them. But it's all about the patterns. So that's how I would say having said that, there is an interesting because you mentioned niche construction. There is a conversation with people thinking at a slightly higher level in terms of communication and hermeneutics and language. There might be a related kind of argument that the culture, there might be a cultural environment out there. So I'm thinking now about the separation between the observer as a phenotype and the eco niche. And certainly from the point of view of the free energy principle, if you read the observer making sense of data supplied from the environment as this constructive active self-organization

that can be read as creating explanations then the converse is exactly true there's an exact mathematical symmetry that the environment must also be learning from and making sense of the things that are observing it. So the environment observes the denizens of any particular eco-niche. On that view, you could actually argue that the environment could be a cultural environment, it could be an environment of means of knowledge, of books for example. I can't remember whether you've written about this, but I suspect you have. But you could look at books as the agent, as particular patterns, that are actually on the point of view of natural selection, much better than we are in terms of lasting longer than your typical human being.

And you're reading the book as actually learning from at a very slow time scale those symmetries and invariants of the its environment which of course are the little human beings that create printing presses and write stories. So that may be important that distinction which I think is distinct from the mortal computation argument, where there is no distinction. The substrate is a pattern. The pattern is the substrate. Once one puts time into play, that would be one sort of way of

dissolving the problem, but there may be something beyond that that you were intimating, which is the, the, the patterns could be of an encultured sort and could be treated as an observer of the culture creating or co-constructing artifacts that generate this particular environment. What if anything does that say about this, you know, the distinction and the argument between organic and psychological disease? Like does that inform that issue? Again, for me that distinction is somewhat dissolved by the very arguments that you're making. So in the context of mortal computational self-organization read as inference, then

we are talking about the organic, the organismal self-organization, sub tending belief updating. Um, and as such, you're talking about psychology. So for me, the two, um, perhaps they have dual aspects of the same thing. Um, you know, in the sense that, you know, there is this information theory could apply to neuronal dynamics and neurochemistry and electrochemistry and pharmacology. Um, but, or, uh, But this is exactly the same mortal substrate that holds the meaning in terms of standing in for the sufficient statistics or beliefs

about the causes of this system's sensations. So I mean, that plays out in the context of psychopathology very, very clearly in terms of something you were talking about earlier on, which is sort of attending to this and that, even in a deterministic system. So attention in this instance is just another aspect of this sense making. And if you break that, then you have a quite a powerful mechanistic explanation for a large range of psychiatric and neurological conditions that can all be cast in terms of some kind of false inference that is all explained in terms of a certain disattention or a failure to attend to the right kind of things or ignore the

right kind of things you know ranging from hallucinations and delusions right through to say dysmorphophobia you know abnormal beliefs about my body for example and I'm coming back to your favorite example of course cancer just having beliefs about the boundaries of my cell or myself which could in a Freudian sense be associated with ego boundaries so inferences about what the boundaries are which again brings us back to sort of and the other, whether this interesting notion that the self can be observed. You have mentioned the paper about can a self, can you ever know yourself with Chris and the Godelian arguments that you've carved. I thought you might bring that to the table.

Yeah. Okay. That's very interesting. One place where that distinction, this psychological versus organic disease distinction is coming up for us now is in a very practical way is in our efforts against aging. One standard set of theories about aging are basically noise-based theories that just over time something, it might be the genome, it might be something else over time, something

accumulates errors and that it's basically the, it's an external source of aging if you will. It happens because there's damage that accumulates over time. We've been driven by some wild results in modeling of goal-directed morphogenetic systems. We've been driven to a different idea. I'll bounce this off of you and see what you think of this. This idea that what we see in our computational model is that when you have a system where a bunch of competent subunits cells are bound

towards implementing a particular goal. So there are stress-based mechanisms that keep them towards a particular goal. Once they've accomplished that goal, which is to build the correct morphogenesis of the body, then they hang around for a while and everything is fine. But then eventually, it starts to de but then eventually it starts to degrade. And it starts to degrade for no external reason. There is no damage.

We've not introduced any progressive decay of thermodynamic, whatever. There isn't any of that. It starts to decay for what looks to me, and this is early days, I don't know, but what looks to me like a very psychological reason. It's a goal-directed system that has reached its goal and I don't know, but what looks to me like a very psychological reason. It's a goal-directed system that has reached its goal and now doesn't know what to do anymore. And it starts to dissociate. There is no new goal.

It's already done the thing it was supposed to do. And it's very weird that in a system like that, it doesn't just stabilize. It sits there for a little bit, but then eventually it just starts to degrade. And so to me, that, I mean, I like dissolving all kinds of distinctions for sure, but then eventually it just starts to degrade. And so to me that, I mean, I like dissolving all kinds of distinctions for sure, but to me it seems it's critical to how we think about interventions, this idea that, you know, is aging because external errors in the hardware accumulate, or that no, actually even in the absence of anything wrong with being in the hardware, you're going to have a fundamentally

psychological reason for this. In this case, psychological, not of the brain of the creature, of the somatic intelligence. It's a kind of existential boredom of the somatic intelligence, if you will. That led me to some weird thoughts, and I wonder what you think about this. If you can imagine a sort of,

I don't know if we've already, I can't recall if we've already talked about this, but a sort of Judeo-Christian notion of heaven, right? You sort of show up, everything is great, and it's going to be great for an infinite amount of time. Now, the question is, you've got a snake, you've got your pet dog, and you. My intuition tells me the snake is going to be completely fine,

ad infinitum maybe. The dog, probably if the conditions are very good, every day is like every other day, I'm not sure if the dog's going to have any problems over. In this environment, we've done away with brain degeneration, we've done away with aging of the physical, there's no physical aging. Now what happens to the human?

I don't know, we could keep ourselves busy for the first 10, you know, there's no physical aging. Now what happens to the human, right? I don't know, we can keep ourselves busy for the first 10,000 years, but what happens a billion years in is it, do you think it's possible to stay sane for extremely long periods of time with no organic damage just by virtue of the, of, I don't know, some sort of psychological need for further change. What do you think about that?

Is that anything? Yeah, it's fascinating. I personally think that you couldn't and you wouldn't want to. So from a purely mathematical perspective, what you're talking about is oscillator depth. So attaining your steady state your goal and staying there this is exactly equilibrium physics death and you become a closed system

and you lose that which is characteristic of biological self-organization you could actually say any kind of interesting non-equilibrium or out of equilibrium or far from equilibrium non-equilibrium or out of equilibrium or far from equilibrium dynamics. So to stay in one place violates the kind of solenoidal dynamics or Red Queen dynamics that is definitional of life. So I don't think you can elude death. Death is just part of a life cycle. It's part of the so-called dynamics. It's part of this itinerant attractor manifold that defines the self-organization and open systems that we'll be talking about. So I would actually contend that all the snake and the dog and you and I would not survive terribly long even in heaven in the same form.

But if we did, we would technically be, we would no longer be sort of, well, of course, we wouldn't be living because we would be dead, but we certainly wouldn't have the dynamical itinerancy and the normal laws of physics would not apply nor would the way that certainly I understand self-organisation as a constructive way of existing and living in a world in a world to which you are open. So as soon as you're open as a system, you necessarily and you have a steady state. In other words, you have characteristic states that define you as you and me as me. You necessarily have to have these solenoidal

dynamics, which means that your attracting set is of the kind you have to keep moving, much like a shark or much like oscillation, but at every level you've always got to keep moving simply because you've got this solenoidal aspect to non-equilibria or dynamics of open systems. This is also mathematically called detailed balance. So if you have detailed balance, you're dead. Um, and that's the situation where you have this, um, equivalence between forward and time reversal dynamics. So it doesn't matter whether you can go backwards or forwards in time that you look the same.

So that's being dead and that's exactly, um, not changing. So, you know, I think there's a mathematical reason why, why one should not try and elude death and death death is part of the natural cycle. I'm not sure if you were able to. I'm immortalize one in the sense that you are no longer changing. I think that will be from a mathematical perspective a a certain kind of death. a certain kind of death. Very interesting. So, yeah, so we are, in therapeutic terms, we're then focusing our attention on renewing.

So, okay, clearly you cannot stay the same. The question is what can you renew to give a new sense of purpose to the somatic system? And that at this point is what we're going to try for. Well normally you and me, people, things like you and me have children. So would you consider that to be a changing form of yourself? So earlier you said you cannot last forever in the same form, which implies that maybe you can in a changing form. Now it's difficult for me to think of a snake or dog and especially us that stays the same.

We're constantly changing even throughout this conversation. So is there a way, Karl, that we can live forever in a changing form in such a manner that you do have a continuous self? Because you can change so much that you're no longer the same self a minute from now. Yeah, well, I was just thinking that, or thinking about that when Mike was talking about this sort of selfhood that makes sense of the deep past and deep future. Part of my self modeling and I identify as a human. And of course, if I identify as a human, then course if I identify as a human then myself could actually be transgenerational and on that view on that particular

scale yes you can live forever if your conspecifics procreate in the right kind of way so from a purely sort of maths perspective this is the cycle of reproduction say for sexual reproduction creatures, is just another expression of one of these solenoidal dynamics in open systems. It's a particularly long one, but it's a necessary feature of any pullback attractor or attracting manifold that defines the self-information that we're talking about before. So into that self- information is a probability you're close to or on that manifold that contains these itinerant trajectories at every scale and of course that also

means that there's always a scale higher. So there will be cyclical motions of the heavenly bodies that are necessary to actually have a yearly cycle of climate that are necessary to have a diurnal cycle of day and night that have, you know, that contained my sleep-wake cycle, that contained right the way down to respiration, right the way down to fast oscillations of, you know, of our dendrite in our favourite cell in the hippocampus. All of these contextualise each

other, but the one recurrent theme is this revisiting states that I was once in simply because if my attracting set or the set of states that characterize me or my niche or my niche's niche was not attracting then I would just exponentially diverge and dissipate and of course that would not be self-organization. So I know we all have to get going before we do the last question. Michael, you mentioned that you were concerned that you don't want people to think of the self as merely a story. You called it nothing but ism.

So I want to know why is it you're concerned about that? What's underlying that concern? And while Michael is answering, Carl, I want to know, is there anything about what you've said during this conversation or more broadly the free energy principle that Your concern will be misrepresented Here's here's the thing I Receive pretty much daily emails from people who say things like I've read your paper I now and I now I understand that I'm a collective of cells and furthermore, a self-constructing or whatever,

and I don't know what to do with myself. I don't know what to do anymore. I'm depressed. I think we have to come to grips with the fact that some people expect these important things about themselves to be magic. When the lid is lifted off of some of the processes that underlie them, they feel like something important has been lost. In particular, these stories of collective intelligence,

of processes, of self-constructing patterns, all this stuff. Some people interpret this as a deflationary story, that they've lost something important that they had before. And I think it's dangerous. There's only a segment of people that like a very particular, you have to be a very thoughtful person in a certain sense to be destabilized by stuff like this. But there are people who are in that zone who read these things,

and they think that what science has now shown is that they somehow don't exist, basically. That's taken to its conclusion, that's where it goes. And so I don't think that's a good conclusion, and I've made materials about it that I send people to say, this is not the conclusion you should draw from this. But yeah, I mean, I am concerned because I think the stories that we tell as scientists need to not only be the best picture of the world as we have, but actually be useful and constructive,

not just for engineering, but actually for living, for people's lives. And it's an assumption of mine that to the extent that we can tell better stories that they're actually going to be more useful to people, not destabilizing. I could be wrong, but. Well, just to pick up on Mike's comments, he started by talking about self-organization as a quintessentially creative process and ended by explicitly talking about the constructive aspects of this so-called deflationary account. So I would fully concur with

him. I mean what we're talking about is a really beautiful constructive process and much of the scientific account of that is really to try and understand the the fundamental the mathematical architecture and how to interpret that but that that scientific process is itself a creative constructive act that complies with the very principles that we've been talking about. So I don't get emails from people who find that this is somehow demeaning their presence in their sensed world and all the issues that attend the sense making and decision making and navigating that world in the right kind of way. And of course, acknowledging that the world is largely comprised of other people like you and me. So there's a lot of sociality imbued in being creative and constructive

in the way that you explain your world. In response to your question, is there anything that might have been misinterpreted? I'm sure the majority of it will be misinterpreted and that's fine. So that misinterpretation can be read in terms of there being a prediction error, somebody pursue that and that will drive their belief updating and a conciliance of our respective world models or somebody pursue that, and that will drive their belief updating and a conciliance of our respective world models or generative models. But in closing, don't take anything I say too seriously. Probably the safest thing to say.

What does it mean for consciousness to be scale-free? Well, if I was Mike, I would say that that basically you could find processes that have the attribute of things that we consider to be conscious processes expressed at multiple scales so that you could in principle associate conscious processing or sentient processing to a single cell right through to again the motion of the heavenly bodies. Earlier you talked about consciousness and you said something about the counterfactual of what would happen if I did so and so. So what's the difference between that and a hypothetical?

I don't think there is a hypothetical. Well, it is a hypothetical and but with a particular future pointing aspect that also depends upon your actions. So it's a hypothesis about what would happen in the future if you acted in this particular way. So it's a special kind of hypothesis that is conditioned upon or rests upon this course of action versus that course of action. So very simply, it's just our ability to plan. And in planning, we are dealing with hypotheticals because the future has not yet revealed itself.

I think that's what Mike is getting at when he talks about the right-hand side. What the future part of, say, a light cone or an information bottleneck centered on the present? I see. Okay. I was confused about the terminology because not all hypotheticals are counterfactuals. And I thought you said counterfactual. Well, I did, but I use the two interchangeably because I haven't had your philosophical training. So, okay.

Now, before we get going, what do you think is the simplest kind of, of physical system that's capable of curiosity and prediction and exploration? Um, I think, I think Mike is correct that all systems that are non-trivial in their scale will possess behaviors that can be described by an observer as curious and sentient behavior, that they look as if they are exploring. If you're actually, if you're asking will the If you're asking, will the internal dynamics, which of course we can never know, but if we could know of the system, are planning in a way to reveal information, to be novelty seeking or sensation seeking or to be curious, I think they are limited to certain larger scale kinds of systems. So you have to have certain statistical architectures in play before you can describe

self-organization in terms of this kind of planning and selection from or looking as if you are selecting from different futures simply because if you're sufficiently big then right on the inside you don't have direct access to your action upon the world, your outputs, if you like, or if you're Chris Fields, you don't have direct access to what you are writing to your holographic screen, because you're so big, there's lots of intervening states. And at that point, you start to, you are now obliged to treat your real realized and real actions as random variables. So now you start to infer what you are doing.

And as soon as you infer what you are doing, which is not the same thing as what you're actually doing, which is not the same thing as what you're actually doing. You're only able to use the consequences, the sensory consequences of your action to infer, oh, that's what I'm doing. But as soon as you have that capacity to infer how you are acting, you've now got the opportunity to infer the consequences in the future of acting like this or acting like that. So that's what I meant by sort of counterfactual futures that are hypothetical in the sense that they rest upon the hypothesis that I will do this.

I have another hypothesis, I might do that. And then you evaluate the two hypotheses in terms of what's the probability that something like me would experience the outcomes of acting like this versus acting like that. So that's what I was implying by what people in machine learning might call planning as inference, which is not the same as controllers inference. So controllers inference can be done by thermostat that is so small it has direct access to switching on the heater or switching off the heater. But when you get to something of our size, deep inside your brain for example, there are multiple connections and states that intervene

between your brain and the muscles which cause you to move or cause me to speak or your autonomic reflexes. So now we have to think about in a creative constructive way, what are we doing? What is the course of action, the path of action that we're currently committed to? Are there any better ones? And perhaps I should be doing this. Oh yes, I'll do that. Now I'll change my mind.

And then you prescribe the predictions to those, uh, the predictions consequent upon selecting that particular path into the future. And then those ultimately drive your reflexes there by supplying the evidence. Oh yes, I'm acting like this. Is it the case that once you have the ability to infer about your own actions that you have the ability to infer about your inferences and then that goes on ad infinitum?

Absolutely. Yeah. Well, you should, so there are lots of ways you could take that conversation. You could take it into higher order thought theory of consciousness and talk to people like Steve Fleming, a young colleague of mine at University College, London. Um, uh, you could also talk to people like, um, last Sanford Smith and Thomas Metzinger in terms of mental action. Um, so, you know, thinking about thinking or metacognition, um what would happen when you get these big objects that are in a layered way separated from the sensorium or the sensory epithelia.

So now you can have notions of action on the inside. And if you were speaking to a psychologist, this will be like attention. So you can pick out various parts of your internal dynamics that are doing the sense making in the exactly this creative or constructive way we're talking about earlier on, um, you can pick out by using attentional action or mental action, like action on, on the inside, absolutely. And it's exactly these kinds of things that now speak to issues in consciousness research. So to be able to, you know, there are people argued that to be conscious requires you to have this counterfactual ability to mentalize

or to not mentalize, but certainly to represent in your internal dynamics counterfactual futures under different actions. And those actions can be attentional. They can be covert actions. They don't actually have to be movements. Carl, Michael, thank you so much. It's been a pleasure. We could probably keep going for a few hours and maybe next time we will,

but we all have to get going. So thank you. Lovely. I look forward to the next time. Thank you so much, brother, dear. Awesome. Thank you so much. I really appreciate it. Thanks, Carl. Always an amazing pleasure. Thanks, Biker. Thanks for pulling us together. Bye. I've received several messages,

emails, and comments from professors saying that they recommend theories of everything to their students, and that's fantastic. If you're a professor or a lecturer and there's a particular standout episode that your students can benefit from, please do share. And as always, feel free to contact me. New update. Started a sub stack. Writings on there are currently about language and ill-defined concepts

as well as some other mathematical details. Much more being written there. This is content that isn't anywhere else. It's not on theories of everything. It's not on Patreon. Also, full transcripts will be placed there at some point in the future. Several people ask me, hey Kurt, you've spoken to so many people in the fields of theoretical physics, philosophy, and consciousness. What are your thoughts? While I remain impartial in interviews, this substack is a way to peer into my present deliberations on these topics. Also, thank you to our partner, The Economist.