Theories of Everything with Curt Jaimungal - Consciousness vs The Ruliad | Stephen Wolfram Λ Donald Hoffman

Episode Date: June 26, 2024

Stephen Wolfram, welcome. Hello there. Donald Hoffman, welcome. Thank you. It's my understanding this is the first time you both are meeting. That's correct. Indeed. Yes, people say to me, you know, about things I worked on in physics and so on, oh, that's

related to things that Immanuel Kant did. And they say, it might be related to things that Donald Hoffman has done. Well, Immanuel Kant I'm too late for, but Donald Hoffman, we get a chance to actually talk about things. That'd be fun. Absolutely. Don, do you see yourself as Kant 2.0? Well, I'm not nearly as smart as him, so it'd be a lesser version, but similar.

It's idealism, but with some mathematics behind it. How about, are you a Leibniz 2.0 as well? Or are you, I don't know? Yeah, much, much less smart than Leibniz, that's for sure. But yeah, it's very, very similar. I like Leibniz's monodology. There's a lot of good ideas in there. And the work I'm doing on conscious agents, in some sense, I can view it as simply a

mathematization of Leibniz's ideas. Interesting. I still have to, you know, people have told me for four decades that things I'm doing are sort of Leibniz related, and at various times I have tried to understand Leibniz's monad idea, and I've usually failed. Although one thing that helped me a lot recently was realizing, and maybe you can comment on this, that Leibniz didn't imagine that you could have mind made from non-mind. So for him, a monad, if there was ever going to be anything mind-like about it, it had to start by being a mind, so to speak. Right. He has his analogy of the mill, Leibniz Mill analogy, right? So he's looking at what we'd call the hard problem of consciousness from a physicalist point of view. And he gives it just

one paragraph in the monology. That's all he thinks it deserves. And he basically says, look, if you're trying to get consciousness out of some kind of physical system, it's like going inside of a mill and going down and seeing all the gears and so forth, you know, whatever it is, the gears are not going to give you an explanation for what is going on in consciousness. And so he, you know, he felt that whatever mechanical physical explanation we give will fail. He figured one paragraph was enough and he moved right on. Don, what would be your position on that?

Well, so physicalists have been trying to give theories of consciousness quite strongly now for the last three decades. Right? So we have integrated information theory, global workspace theory, orchestrated collapse of quantum states and microtubules and so forth. But, and I know that the players and their brilliant people and their friends,

and they know what I'm gonna ask them every time I talk with them or get on stage with them is what specific conscious experience can your theory explain? Taste of chocolate, the smell of garlic, the taste of mint. What? I, you know, we're interested in scientific theories that explain specific

conscious experiences. What experience can you give me? Humans can experience around a trillion different experiences. So it should be like shooting fish in a barrel. There's a trillion experiences. Which ones have you done? And the answer is zero. And so right now

we have no example of a physicalist theory that can explain even one specific conscious experience. So for example, what I would ask for example of integrated information theory. They say there's going to be some causal structure that's the substrate. And if you have the right causal structure, then they say you can represent that in a particular with a matrix. The matrix represents that causal structure. So, okay, great. That's your theory. What's the matrix for Mint? Just how big is the matrix? What what, you know, if it's an n by n matrix, what is n and what are the n by n, the n squared entries?

So, so, you know, it seems like we have an easier problem in the last year or two than we've had in the time before that. Because now we have LLMs that can talk to us a little bit like we talk to each other. And you know, for humans, it's both practically and ethically not possible to kind of take our brains apart and see what's going on inside. But for an LLM, so far, it seems ethically just fine to do that. And, you know, so what would you imagine? I mean, you know, you've got your LLM and it's, you know, it's talking to you and it's

discussing the kind of tea it likes and all kinds of other things. What would be the kind of thing that you would think you should want to identify that is its internal experience? Nothing. My guess is that they don't have any internal experiences. And what our LLMs right now are doing are just sophisticated correlations and computations. They're looking for statistics. And how convinced are you that you are more than that? Well, I would say that I have the taste of mint and the smell of garlic and I can hear

the middle C on a piano. And right now... Can you convince me that you can hear those things or feel those things? Oh, absolutely not. And you can't convince me either that you have it. So it's a matter of me just believing that you're relevant similar to me in certain ways. So I absolutely agree that there's no proof of anybody else. Solipsism is certainly a logical possibility. Right. But so, but you believe that I might have those internal experiences, but you don't believe that the LLM could have those internal experiences.

Well, so it's a little more complicated. So I'll say a little bit more about, so I think that our experience of space and time and physical objects is Just a headset So it's it's my consciousness is created a headset to interact with other consciousnesses. And so when I look at You on the screen for example All I see is pixels and the pixels on the screen, I wouldn't want to say are conscious. But through the pixels, I'm getting a portal into, I think, your consciousness.

I can guess what you're thinking about and guess what your beliefs might be right now and so forth, you know, probabilistically and not all completely accurately. But I wouldn't want to say that pixels are conscious. They're just part of my headset that's given me access to the consciousness. So I want to say that consciousness is fundamental. It's the fundamental existence. And what we call space-time is a fairly trivial headset that some consciousnesses use, but probably most don't.

Probably there's a variety of much more interesting headsets out there than just a four dimensional one that we're using. And so no thing in the headset is conscious, just like the pixels on the screen are conscious. And some pixels give me access to your consciousness, and others don't, but I wouldn't want to call some pixels conscious or not.

So ultimately, I think I'm always interacting with consciousness. I'm always interacting with consciousness. So if you see a frog, are you interacting with consciousness inside the frog or not? Not inside the frog, but I mean, it's sort of like the frog is like the pixels on my screen that's giving me access to certain aspects of consciousness. So, but the frog internally has a feeling of, I don't know what it might have a feeling of a mosquito or something like that, a feeling of, it has an inner experience.

All right, yes, yes, we understand Stephen. So let's say there's a frog feeling sub one and then a frog feeling sub two. Well, well, yeah, I would say that there is a conscious experience that I'm interacting with a conscious experiencer or a series of experiences behind the frog. And in the case of the LLM, there, there is going to be beyond the, the headset conscious experiences, but it's not going to be what we typically think of as somehow a physical machine gave rise to consciousness. It's rather that even the very components of the

computer that are running the LLM are like pixels on my headset and behind that is consciousness. Absolutely. So, you mean, so it's like the LLM is the digest of eight billion souls and that's the way you see it? That's right. So, it's really a bunch of conscious agents outside there, outside of space-time, and we are opening different portals into consciousness beyond our headset. Just like pixels. So we humans are sort of the ultimate seat of those elements of consciousness in your view.

Is that right? Not at all. We're probably among the less sophisticated ones. So what's an example of a more sophisticated one? Well, our headset has only got four dimensions. Why not have consciousnesses that are using headsets with a billion dimensions? Pete Uh-huh. But I mean, but then, I'm a little surprised, you know, in we, I mean, so, do

you view our sort of, if you imagine, first of all, do you believe there are laws of physics, for example? Or do you believe that there are, in other words, are there things that are laws on top of which our brains and the electrochemistry of them and so on operate? Or do you think that it's sort of, is there some substrate underneath? Or are you somehow imagining that your scientific theory is built? I mean, okay, for example, we could imagine that you never had a theory of physics. All you had in physics was a collection of experimental results. And you would have a bunch of, you know, you could even imagine sort of axioms about how, oh, I've seen that this thing correlates with that thing.

And we would have this kind of sort of observational version of physics that never had anything sort of underneath. I'm just curious, how do you imagine kind of the nature of kind of what's happening in brains relating to sort of the substrate or the potentially laws of physics? Right, so my view is very, very similar to what some high energy theoretical physicists are doing right now, which they're looking for new principles and structures beyond space-time. So this is Neemar Khani-Hamed and a bunch of other people. There's the European Research Council just announced the Universe Plus project and putting 10 million euros into what are called positive geometries beyond space-time and beyond quantum theory.

And they just had their first workshop in February where they brought together about 100 PhDs in mathematics and theoretical physics. what they've discovered are these new structures beyond space-time called positive geometries, amplitude hydra, the sociohedron, cosmological polytopes, that their volumes encode scattering amplitudes. Yeah, yeah. I used to do particle physics when I was a kid. I know. These things are a bit more recent than that, But I mean, I think the thing to understand about sort of particle physics and where it's gone is, I think what you're describing is kind of the limit of what one can view as

the S-matrix approach to particle physics. I mean, you know, the one view of what happens when sort of particles interact is you see all the details of what's happening and the mechanism of the interaction. Another approach that Heisenberg introduced was just this, we don't know what's happening inside, we're just going to say what are the things that are coming in, what are the initial states, what are the final states, and we're just going to define this thing, we call it the S-matrix, that describes the transformation from initial states to final states without having to address this question about sort of the mechanism

of what's happening inside. You know, a thing that I learned recently is a piece of history of science trivia, but it's interesting to me at least, is about how Heisenberg ended up coming up with the S-matrix. So, the, you know, one of the things that's relevant to, you know, my efforts to understand fundamental physics is the question of, you know, what's discrete, what's continuous? You know, back in antiquity, you know, people were arguing about everything, you know, is matter can, this speech or continuous and so on.

That finally got resolved at the end of the 19th century, basically. And yes, you know, matter is discrete, it's made of molecules, we can see Brownian motion, all those kinds of things. And then very soon after, you know, light is consistent with being thought of as being discrete. At the time, 100 and something years ago, most of the obvious physicists were convinced that space was also discrete. But they kept on trying to make that work from a mathematical point of view and particularly make it compatible with relativity, But they kept on trying to make that work from a mathematical point of view, and particularly make it compatible with relativity, and they kept on failing. And Heisenberg, as I recently learned, was kind of in the middle of that whole effort

when he said, I just can't make this work. And he said, forget it all. I'm not going to try and describe the mechanism. I'm not going to describe what's happening in space. I'm just going to set up the S-matrix and say, you know, this is given this initial configuration, what, you know, how will that translate to final configurations? So I think, you know, it is certainly possible to, you know, in terms of what we experience in the world, there's no question that you can describe our experience of the world just in terms of kind of the initial states, the final states.

You can describe just, you know, as I was mentioning, sort of an axiomatic physics, where all you describe is what relates to what, and you don't really talk about what the underlying substrate, the mechanism of physics is. I think that's a, and it's interesting, you know, in our models of physics, at some level, that's what's happening. At some level, what really matters are things like causal graphs that say how one event relates to another event. The question of how are those events kind of... When we start setting up things like space and time, space and time are very different in our kinds of models. But when we sort of say, this is a lump of space, that's something which is something we can do as

a convenience for understanding what's going on. But ultimately, in terms of our experience, what matters is this causal graph of relationships between events. So I certainly on board with the idea that what matters to us is just this causal graph of events. For example, the very construction of space, for instance, is something that I view as being kind of a coincidental feature of our scale in the universe. That is, you know, the fact that we say there's a state of space at a particular moment in time. Yes, you know, I look around this room.

It's, you know, I can see maybe 10 meters away. I, you know, light gets to me in a microsecond from 10 meters away, but it takes me milliseconds to process what I saw. And so I've kind of integrated this whole, you know, I've aggregated all those photons that are coming in and I can reasonably say there's a state of space that I can talk about and then that might change over time. Whereas if I was, for example, oh, I don't know, if I thought a million times faster

than I do, then, you know, I wouldn't know, if I thought a million times faster than I do, then, you know, I wouldn't probably integrate space and I wouldn't talk about space. And if somebody told me, oh, by the way, there's this way of thinking about the physical world that involves the idea of space, I'd say, well, that's kind of interesting, but it's not necessarily something that is relevant to my particular way of observing the world. So I mean, I think that's a, I don't disagree that the construction of space is a feature of certain details of us being the way that we are, so to speak. I agree. And that seems to be what the, this universe plus the positive geometry approach to physics is after as well. One of their banners is

they say space-time is doomed. It cannot be fundamental because it ceases to have operational meaning beyond the Planck scale. So they're actually looking for new foundations for physics entirely outside of space-time and remarkably entirely beyond quantum theory. So these new structures, for example, have no Hilbert spaces. And they're saying there are no Hilbert spaces here, there is no unitarity and so forth. This is beyond quantum theory. But they want to get space time and quantum theory emerging together from things like the amplitude hedron and so forth. Yeah, I'm not sure that's the best way to do it. I think one of the things, you know, I have to say, I did physics when I was much younger, and then I didn't do physics for a long time, and then kind of got back

into doing it when I realized that a bunch of things that I'd figured out for other reasons were sort of converging on giving us a view of how physics might work. And it's been super exciting to me to actually, you know, I think we got it. I mean, I think we know how it works. And I think Nima and folks like that, I know, know something about what we've done. But I think that the paradigm is, it's interesting because the paradigm is a bit different from traditional mathematical physics. But there are very beautiful connections to lots of work that's been done in traditional mathematical physics.

Sorry, just a moment. The paradigm of space-time is doomed or the paradigm of the amplitude hedron? No, no, the paradigm of what people call Wolfram Physics Project. I mean, it's something where sort of the foundational machine code is very computational. The, what that turns into at the level of things that we can do experiments on and so on is

sort of looks much more like traditional mathematical physics. And what's really cool is that a bunch of limits of our model clearly are kind of map into things that have been studied in traditional mathematical physics. And that's kind of what you would hope would be the case because what we're trying to do is deal with something that is sort of a lower level machine code of the structure of the universe, the structure of reality, basically, than the things, you know, I'd always thought of what I was doing as kind of going underneath space and time to something which is sort of more fundamental than space and time. I mean, I think it has

not helped the progress of a lot of kind of physics, that people have sort of had this idea that space and time are the same kind of thing, which is kind of, I think, you know, in terms of doomed ideas, that is a doomed idea. That was a thing that, you know, Einstein didn't have that idea. That idea came in when Minkowski said, it's really cool that there's this quadratic form that we can write with space pieces and a time piece and they all sort of fit together. And that's how kind of this concept of space-time was born. And I think it's sort of a mistake because I think that time, as I see it, is this kind of progressive application of computational rules, and space

is this thing that you can reasonably construct as a way to describe what's in the universe. It's kind of the structure, the sort of a data structure of the universe that you can slice into pieces of space. You could slice it into quite different things as well. And I think it's a feature of observers like us, I think, that we believe in space. I mean, same thing that, the fact that we believe in fluids,

as opposed to just saying they're all a bunch of molecules bouncing around. That's a feature of observers like us, and not necessarily a feature of all observers. I mean, I do think, by the way, in terms of dimension, you mentioned, you know, is three plus one dimension kind of the fundamental thing? I'm sure the answer is no. And, you know, my guess is that there is some totally obvious feature guess is that there is some totally obvious feature of the nature of the observations that we make that leads us to believe that the universe is three plus one dimensional. I mean, you know, in terms of sort of the computational kind of representation of the universe, it really doesn't make much difference that it's three plus one dimensional. We could

as well be, you know, exploring it on some one dimensional space filling curve or some such other thing. It's, you know, I think that's a, I'd love to know what feature of us makes us believe that it's three plus one dimensional. Right. Right. So you have your Rulliad, right? Which is all the possible different computational rules and our projection of that Rulliad into a three plus one dimensional space time is just one of an infinite number of different projections you could take, right?

Yes, yes indeed. I think that the thing that has been very exciting to me and was not something I saw coming at all was the way in which one can, given the idea of the Rulliad, the way in which one can derive the known laws of physics. And that's something that, you know, if you'd asked me five years ago even, would there be a way to derive general relativity, derive quantum mechanics, I would have said, well,

there might be an underlying theory from which those emerge, but I don't think that there would be any way that would make those theories necessary. Those are theories which just happen to be the way they are because the universe happens to be the way it is. If you'd asked me about the second world thermodynamics, for example, I would have said, as people have said for a hundred and something years, that yeah, it's probably derivable in some way, but we don't quite know how to do it. But the thing that's been super surprising to me is that general relativity, quantum mechanics, and it turns out the second law all seem to be derivable. What's the assumption that you need to derive them? Well, the Rulliad

doesn't really have assumptions. The Rulliad is just this abstract thing that you set up. The assumptions have to do with what kind of observers we are. And, you know, it seems like there are two critical assumptions, although I'm guessing that there are actually more assumptions that I haven't correctly identified yet. But the two that I have identified is, you know, we're computationally bounded. We don't get to trace every detail. We only get to notice certain aggregate things. And we believe we're persistent in time. We believe that, which is something I'd be really interested to talk about, is kind of the, you know, to me, it seems like a crucial feature of observers like us is that we have this persistent thread of experience. We don't, and we have a single persistent thread of experience.

It's not the case that we kind of are, you know, have our multiple thoughts kind of branching out in all possible ways, nor is it the case that we are sort of here just for a moment and then it's a different us at the next moment. We kind of have this perception at least that we have a sort of consistent thread of experience. And anyway, I mean, from the point of view of physics, the big surprise to me is those two assumptions seem to be sufficient to allow us to derive the laws of physics that we have. Now, clearly, if those assumptions were changed, if we were observers different from the way we are, we would get different physics. We might not

be able to communicate with those other observers who have such very different qualities, but those other observers, were we to be able to get inside them, their view of the physical world would be different. Adam So, there's an infinite number of different views of the world that could be, and general relativity is just one of the infinite varieties of them, right? For observers like us. Well, for observers like us, general relativity is inevitable. But not for observers, just for observers like us. Yeah, for observers like us. But there are conceivably other observers

that don't observe general relativity. Well, the problem with that, and the reason this is tricky, is, you know, I view, for example, you know, the weather as having a mind of its own. But the weather, one might think, and this is a question of what its internal, quote, conscious experience is, might have an experience of the world that might not have general relativity as one of the things that it experiences. The problem is that, as we were talking about before, you imagine that I'm enough like you,

that you can kind of get some idea of sort of what's going on inside. But in the case of the weather, I don't think it's enough like us that we can have a good projection of what its internal view of things is. So, while it may be, while we could abstractly think of it as an observer, it isn't an observer with which we can kind of have, where we can translate its kind of internal perception of things into our internal perception of things. Now, what is your view on conscious experiences and their relationship to the Ruliat? Is the Ruliat more fundamental than conscious experiences or is consciousness more fundamental than the Ruliat in your view?

Well, I don't know. I think the Ruliat is just an abstract object. And, you know, the fact, and it is my sort of assumption, perhaps, but it's working really well, that sort of everything that exists is somehow part of the Ruliat, which means we are too, which means that the Ruliat is sort of a substrate for everything that we are. Now, the question of whether you can go into the Ruliat and say and point at something and say, that's the Don Hoffman set of Eames and the Ruliat, so to speak. And then what special features that might have. That's something, I mean, we know a certain amount about that. There's a lot more to figure out about that. But if

you're asking, is there something, I mean, this is a complicated thing about what science is and what the point of science is and so on. I mean, there's the universe doing its thing and there's us having some narrative about what's going on in the universe. And I think, you know, science, I think, is about sort of taking not what the universe does, but trying to develop a narrative that we can play in our minds that can say things about what the universe is doing.

In other words, it's not... I think you mentioned the concept of a headset for us to perceive what's actually going on out there, so to speak. And I agree that what matters to our science is what we perceive. I mean, the things that are not... And if you look at the history of science, what has happened in the history of science is we've been progressively able to perceive more kinds of things, you know, telescopes, microscopes, electronic amplifiers, all these kinds of things. And we have then, you know, found ways to describe the world that we can then see, so to speak.

And I wouldn't be surprised if in the future there'll be more kinds of sensors that we somehow manage to transduce into the things that we, you know, into the built-in senses that we have, and then we'll want to describe more things about the world. But I think in your question of, I mean, for me, I've always thought of consciousness as an incredibly slippery concept. And so I've, you know, I haven't been that interested in kind of exactly how do I define it, and etc., etc., etc., perhaps to my detriment. But the one thing that was interesting to me a couple of years ago was realizing that

I needed sort of pragmatic definitions of consciousness in order to understand more about how physics works. In other words, it's, you know, for example, it's kind of like, if I say, okay, there's consciousness and observers like us have consciousness, what are the operational consequences of that? For example? So, for example, one of them, I think, is this thing about single threads of experience. I think that's a, now, whether you say that's a defining feature of consciousness or not,

I don't know. That's a question of what you mean by the word. But I think there's a significant feature of us as observers that we have this concept, that we have this belief that we have a single thread of experience. I mean, I don't know, you know, I've sort of wondered what it's like, you know, if you could be in a kind of a multi-way trance, so to speak, where really your brain is thinking about two different kind of, you have, you know, you have two different time narratives going on in your brain. I mean, I can't imagine what that would be like. But if one had grown

up with that, maybe one would have some sort of internal, you talk about what the internal feeling of something is. I'm curious what the internal feeling of an observer a bit different from me would be like. These are all very interesting topics. I'm interested in something very, very simple, like the taste of mint. And so the taste of mint as a conscious experience. So to keep a really, really simple set of all the threads and so forth, just a single specific conscious experience that an observer might have and how that would be related to the Ruliat. So for example, would you want to say that there's a computational substrate in the

Ruliat that is, for example, necessary and sufficient for the experience of mint to occur or not? You know, I don't know what the experience of mint is. I mean, you know, in other words, I have some experience of it. If you say, let's kind of scientificize that experience, okay, what do we do to make it? And you know, this is the question and part of what science is and what science aspires to be. Because there are, you know, if we say, how do I make that something that you can also sort of observe, you can also be part of? Because the experience that I have internally, as we discussed before, is not something other than by extrapolation, you don't know what that experience is.

Right. So the question is, can I make a transportable version that is kind of a community science version of my experience of mint? Or is it just something that happens inside me that can never be broken out of me, and which is therefore not in some sense, you know, it isn't community science, so to speak. It isn't what we usually think of as being what we usually aspire to have in kind of the operation of science. So if I say, how do I break out that experience? Well, I could start saying, you know, and by the way, it's going to get complicated very quickly because you could say, okay,

I experienced this, a bunch of neurons in my brain are chirping away, and what does that mean, that these neurons are chirping away? Well, we can say, no doubt, the neurons in my brain that chirp away at the taste of mint would be different from the neurons in your brain that chirp away, and we don't even know how to map neuron number. If we were nematodes, we might know how to map our neurons, but we're humans with a lot more neurons and we don't know how to map our neurons and there won't be a unique mapping from one brain to another. I mean, in other words, a nematode, sort of interesting thought experiment, could one nematode communicate scientifically to

another nematode its internal experience of the taste of mint. Because after all, the nematodes have a fixed set of nerve cells where we can say cell number 312 fired in this case. And then the other nematode would say, oh yeah, I know what cell 312 firing feels like, but it's a very different thing with us. In order to communicate a concept from one human brain to another, we kind of have to package it in a robust form that will allow that communication. And the number one robust form that we have is human language, where we're taking all those random nerve firings that you think of when you imagine the taste of mint and you're packaging

those up and you're saying to me the taste of mint. And that's unpacking in my brain and maybe I get some notion that is something, some correspondence. I don't know what the correspondence is between your version of the taste of mint and my version of the taste of mint. Although if we were nematodes, we might know, because it might be the very same nerve cell that was firing. But we have a more general notion of concepts than that.

And I think to this idea of being able to take a bundle of neural activity and package it up in a robust form so that it can be moved to another brain and unpacked. I think that that's probably one of the key things that our species discovered, which is that you can have things like words that are kind of transportable from one brain to another. And I guess there's sort of a fun analogy, which is, you know, when you have a particle like an electron or a photon, a quark or something like this, one of the things it's doing is it is a carrier of existence through space and time. That is, the electron is a thing

that you can identify as being the same electron when it moved to another place or another time. That's similar to this idea that concepts are also transportable things. In our view of the way this works, an electron is something capable of pure motion in physical space. A concept is something capable of pure motion in real space. I mean, by pure motion, what I mean is it is not obvious in our models, for example, that a thing can move without change. So in physical space, you know, you move a book around, for example, and if it's near

a space-time singularity, the thing will be distorted like crazy. But most of the time, we say, I move a book from here to there, and it's still the same book. And I think that this possibility of pure motion in our models is something that you have to kind of establish abstractly that that's possible. And by the way, the idea that there is pure motion again depends on observers, because that book, you know, you moved it and some things about it changed. I mean, in our models, it's made of different atoms of space when it moved to a different place. And yet to us, it's the same book. And so similarly I would say, you know, when you talk about the concept of mint,

of the taste of mint, experience of the taste of mint, it is a non-trivial fact, if it's true, that that is a transportable thing through time, that there is a consistent, persistent thing that is the engram or whatever it is that represents that concept and that it is robust. And I think that if you, you know, the version of it that's locked inside your brain at some moment in time, I don't think that's transportable.

I don't think that's science-isable. I think that's a thing that you would say it is, I mean, if we were thinking about it in terms of an LLM, it would be some little, you know, some activation of some neuron at some moment and, you know, then it's gone. And we wouldn't say, you know, and we would argue, was that a conscious experience of the LLM? Well, it isn't robust. It's not something where we can pick it up and say, look, it's a conscious experience because it was a fleeting thing that just was there at that moment and then disappeared. And I would claim that absent some way to robustify what you're talking about, there isn't really a way to extract.

I mean, if you say, show me that conscious experience, what is it? You know, physicalize that conscious experience. You can't physicalize it. So what does that mean, so to speak? I would claim that it is not an obvious fact that things can be made robust enough to be sort of picked out as a separate thing. I mean, I'm sort of reminded of, I have to say, in your kind of what is that essence of a conscious experience, I'm reminded of something that I kind of feel silly about myself, because, you know, when I was a kid, 1960s and so on, it was, you know, you would run into people who would talk about sort of the eternal soul. And you know, if you were kind of a physics-oriented kid as I was, you would always say things

like, but how much does a soul weigh? You know, how can this be a real thing? You know, how much does a soul weigh? How can this be a real thing? How much does it weigh? If a soul departs a body, does that mean you lose a microgram or something? How much does it weigh? There must be some sort of... If it's real, it must have those physical attributes. Of course, I realized later on that that's a very silly thing to have thought because you know, computation, the idea of sort of an eternal soul is kind of a sort of primitive way I think to talk about abstract computation and it would be a very foolish thing to ask

sort of how much does the abstract computation weigh. And I kind of suspect, and I'm not untangling it in real time as well as I might, but I'm kind of suspecting that your kind of notion of the intrinsic conscious experience of something and you saying, look, you can't pull it out and physicalize it, is the same kind of mistake. So let me see if I can paraphrase. So in your ontology, the Ruliad is fundamental or close to fundamental and the rules there, the computations, but any color, shape, motion, taste, experiences, those conscious experiences are not part of

the fundamental ontology that you're considering. Is that correct or have I misunderstood? Okay. So, you know, in matters this fundamental, there are inevitably many different ways to look at the same elephant. Okay. Okay? So, the Ruliyad and for example, its representation in terms of computation and rules and so on is the way that I understand best and that I think people in general understand best. It is probably not the only way to think about it. So for example, just as mentioned before, you can think about physics either as a kind

of an underlying mechanistic structure that makes things happen, or you can invent kind of an axiomatic physics where you just say, this is a thing that's true, that's a thing that's true, and then you have to fit all the pieces together. So similarly, when it comes to the Rulliad, there is certainly, I like to think of it from sort of the bottom up, of I can represent it in terms of computations and things like this. But in the end, sort of observers like us are making various observations about it. And one could imagine reconstructing it. It's I don't know how to do it exactly. But one could imagine saying, all I know is what I observe. And that is, that's my reality, so to speak. And now from that reality, I can, you know, I could imagine a theory in which there is

this really odd thing with computations and so on. In other words, my way of thinking about it, the way I prefer to think about it, just because I guess that's the way my particular mind is built, is from this kind of hard structure of computation building up to something where I might hope to be able to find somewhere in the Ruliat a thing that corresponds to a brain with a feeling of mint and things like this. That's the way that for me is the most sort of, it gives me the most sort of hope of being able to make scientific progress.

But I don't think that's the only way to think about it. I think you could as well say, all I'm going to do, like the S matrix, for example, you know, forget the mechanism. All we want to know is the transformation from initial states to final states. And we're going to just say there's this thing called S that represents that transformation. And we're then going to talk about the properties of S. I mean, this was in the late 1950s, early 1960s, this was kind of what people thought was going to be the way that particle physics worked.

In the strange cyclicity of science, those ideas have come back again, but at the time, there was sort of a competition. Would we describe the world by saying, there's just this S matrix and we're going to figure out properties of the S matrix by having, I wouldn't call them conscious experiences, but particle accelerator experiences of the S matrix? That's door number one. Door number two, are we going to figure out the mechanism, you know, how all these particles are structured and how they, you know, what the little interaction vertices are and all this kind of thing? That was door number two. In the 1970s, door number two one in particle physics. But I think it's not the case that, you know, we're seeing

in fact a return to more of the kind of S-matrix approach to saying we don't really know what's going on inside, but we can describe certain constraints based on what we observe. And I absolutely think that there's a way of constructing kind of sort of the Rulliad. You could invent the Rulliad as the afterthought, having started from something which is just axioms about observers. And my particular way of thinking about it, I like to start from something that I can run computer experiments on and that happens to, you know, that I at least imagine that

I have a reasonably good handle on from the way my mind works. But I don't think it's the only way to think about it. Right. You know, for me, if you say, you know, only start from things that an observer can observe, which is kind of the S-matrix idea. Only start from things that are sort of externally observable. Sure, one could do that. I don't know how to set up that formalism. I mean, you know, I've got some ideas about that, but that's, I think, for me, it's much more difficult than the bottom-up approach. But I don't think it's, I think both approaches are perfectly viable. It's just a question of if one's goal is to have kind of a narrative description

of how the world works, one can make a choice between those approaches, you know, which is the way that is most likely to lead to a narrative that, for example, I understand. I mean, now again, this is, and for me, the narrative that has to do with Ruliat and computation and so on is easier to understand. It is more grounded for me than a description in terms of kind of starting with consciousness, so to speak. Right. So, I think I'm understanding better. So, to me, of course, I love the computational approach and the mathematically precise approach,

that's what we need to do in science. And I guess what I'm doing is saying that the computations and the mathematics are describing the activity of consciousness as opposed to the activity of something that's not conscious. In other words, what I'm doing is biting the bullet up front and saying fundamental in my ontology are things like observers that have conscious experiences. Because every observer, if you imagine an observer that has no conscious experiences, it's not really clear what we're talking about.

An observer with no conscious experiences is nothing. I don't know what that means exactly. You and Leibniz seem to have a lot in common. Probably so, except that he was much smarter. One of the things that I only very recently understood about Leibniz, as I mentioned earlier, is that Leibniz could not imagine a way that mind could arise from non-mind. And I think you think the same thing. That is, you can't imagine a way that mind can arise from non-mind.

I can imagine how cognition, intelligence, and things like that could arise. But conscious experiences, what we call qualia, I would be delighted to see the first scientific theory that ever tries to do that. Right now, there's nothing on the table. Well, I mean, so what would, I mean, this question of what can arise from what is a, first of all, you have to know what the thing you're trying to get to is. Like people say, can life arise from non-life? And again, it's a messy business because what do we mean by life? If we mean the specifics of life on earth with RNA and cell membranes and all this kind

of thing, that's one question. If we say the thing we scoop up from the Martian soil and it does something amazing that we've never seen before, you know, is that life? Is that not life? You know, it's, I think we have to know. And I think one of the difficulties about what you're talking about is if you say, can conscious experience arise from something other than conscious experience, if we don't know, if we don't have a general description of the target, it's very hard to answer that question. Just like if we say, can life arise from non-life, and we have only one example of life here

on earth, and if you say, can conscious experience arise from something that isn't conscious experience, and you ultimately have only one instance of that, which is what's happening inside you. You don't even know that I have that same conscious experience. So, you're trying to explain kind of an N of one thing of how does the thing that you feel internally arise from something that sort of isn't you and so on? How does that arise? And I think that's a, I mean, I'd be very interested to understand how one would, you know, how one would get a positive answer to that. In other words, forget, you know, oh, there isn't a good enough theory and we don't know the electrochemistry and we, you know, we can't

see how aggregates of neurons behave and so on. You know, there are obviously issues there, but there's a different question, which is, you know, how do I, what's the signal of success? Right. So, one issue here is that as an observer, all I have are my conscious experiences. I actually, the notion of something physical beyond my conscious experiences is actually the leap. Right? Absolutely. Absolutely.

Right. The only thing we have is what we, you know, it's the, you know, cogito ergo sum type story. Absolutely. And so we're on the same page on that. And I agree, I don't know that your world of experiences is anywhere similar to mine. I can never know that. But what I do know is that consciousness is what I know firsthand. What I call inanimate matter is an extrapolation. What's directly available to me are experiences, conscious experiences, and what I call an

unconscious physical world is an extrapolation that I'm making. What I only have are my conscious experiences. I have nothing else. Let's go back to, I hadn't thought about this before this conversation, but let's go back to the nematodes, okay, which have precisely defined neural nets, where there really is a way to say, nematode number one feels this. And do you believe that if I could accurately measure the electrochemistry of the nematode, that I would capture that's the whole story?

Or do you believe that there's something that is kind of beyond the physical that is kind of not captureable by any physical measurement that is what, you know, is something about what the nematode feels? What we call physical is going to be something inside a four dimensional space time, which is going to be just what I as a particular observer can observe because I am the kind of observer I am. The reality beyond that four dimensional space time that I happen to observe is infinitely complicated.

And I may need to go to that other deeper reality to give you a full, so in that sense, what I can do in terms of a physical thing inside space time is probably trivial and probably inadequate. I understand. So, I mean, this is at some level, you know, I could unkindly say it's kind of a Victorian theory, okay, because it posits that there is what we have physically in our minds, and what we can sort of tell is there. But then there's a spirit world which is beyond that, that might be, for example, in the Ruliad, for sure, in my

view, we see just tiny little slices of the Ruliad, and there's much more there. For the things that, I mean, the, okay, so one of the questions is, is it enough for doing physics that we sample only that tiny slice of the Rulliad? It might not be. It might be the case that we would sample that slice of the Rulliad and miracles would keep on happening. Weird things, weird random things would keep on happening that kind of poke in from other parts of the Rulliad that we weren't able to sense, so to speak. And that, in other words, that it isn't a closed system, that the part of the Rulliad

that we are slicing, the slice that we're taking isn't closed enough. And so we constantly are being exposed to other things. So an analogy, in fluid dynamics, for example, most of the time it's okay to just think of a fluid as with a velocity field and things like that. Occasionally, you actually, if you're making a hypersonic airplane, you have to care about the fact that the fluid is made of molecules. But that's a rare case. But it could be that there are things about the world, perhaps even your consciousness

things about the world, where aspects of the Ruliad poke through and it isn't self-consistent to just look at the slice we know that we can observe. So that's an interesting question of to what extent is the pocket of reducibility, as I would call it, the kind of slice where we can say things about what's going to happen. To what extent is that closed? And to what extent does it have things feeding into it? By the way, there's an analogy of this in mathematics, which is kind of to what extent can you do mathematics at the level of kind of talking about things like the Pythagorean theorem?

And or do you have to, can you talk about the Pythagorean theorem or every time you mention it, do you have to go back and say, oh, and the definition of real numbers that I'm using is this and there's the following axioms, et cetera, et cetera, et cetera, which is kind of like going down to the level of molecules and talking about the fluid. So I think it is a non-trivial claim, but a thing that I think we are deriving in, for example, our models of physics, that there is a sort of self-consistent layer that can be talked about merely in terms of general relativity and quantum mechanics and so on, without looking down below at the details of the whole hypergraph and all these atoms of space doing all their complicated things.

It is a scientific claim that it is enough to merely look at this kind of continuum level of general relativity and so on. By the way, a thing that we would really love to do is to see things, other things poking through. I mean, that's what, you know, when people observe molecules, you know, they have water of fluid, but yet they saw that these little grains of pollen were kicked around and brown in motion. And that showed there was something below just this fluid description of water. And we'd love to find the same kind of thing

for physical space. And that's one of my big activities right now, is trying to see, you know, is there an effect? Are we going to be lucky? Because molecules, people were pretty lucky. Molecules were big enough that you could actually see them in 1900, so to speak. Whether we will be able to see the atoms of space, so to speak, in my lifetime, I don't know. It's a question of what the scale is and how clever we are and so on. But I think that this whole idea of whether we are in a consistent bubble, so to speak, or whether we have to appeal to things sort of beyond our physics is a reasonable question. I mean, that is, there are things where I'm hoping that there are observations that we

can make with telescopes or maybe with other kinds of systems, but that there are observations that we can make with telescopes or maybe with other kinds of systems, but there are observations that we can make in which the nasty, spiny parts of the Rulliard will kind of poke through our usual continuum view of space. And so what you're asking, I think, is in the case of conscious experience, is it enough to merely talk about kind of the laws of physics that we know, or is that a place where there's a poke through from something beyond kind of the laws of physics that we know? I think that's a very important and useful question. And there's also another way of looking at this issue and that is if we we're trying to build a scientific theory and we're trying to find as few

assumptions as possible for our scientific theory. We believe in Occam's razor and so and we both agreed that that as an observer all I know are my conscious experiences. So whatever conscious experiences are they're all I know as an observer. So in the ontology that I'm going to assume in my scientific theory, I have a big choice. I can either put conscious experiences in that ontology as found as foundational or not. And if I choose not to, then I've given myself the scientific duty to explain where those experiences come from. So I either postulate that they are,

I say, upfront, these are part of the ontology, these are the assumptions I make, or I say, no, they're not part of the assumptions, I therefore have the duty to explain consciously. So it's my choice. Now, I would like to just stop you for a second there. I mean, it depends on what kind of science you're doing. Okay. If you're doing psychology or something, if you're doing a science that is about that, then for sure. But one of the things that happens in science, it's not obvious that it would be possible, but it has proved possible is that you can

separately look at physics, biology, chemistry. You know, they have interfaces, but you can choose to concentrate on one aspect of the world. And, you know, an obvious question is, is there a, you know, you might make the claim there is no meaningful science that can be done without entraining consciousness in it. That would be a potential claim. That is not what has been the observation of the last few hundred years of science. The last few hundred years of science has achieved a lot without solving the problem that you say nobody has solved, and I agree nobody has solved. So, it's a question of what it is that you think you're

going to do in your science. Now, when you talk about Occam's razor, and, you know, I don't know why Occam's razor is true. I mean, it's an interesting criterion. In a sense, the Ruliat denies Occam's razor, because the Ruliat has everything, all these kinds of things going on in it. At some level, from the point of view of abstract aesthetics, the Rulliat is lovely because it assumes nothing. But, you know, from the point of view of, you know, is it saying, oh, the description of what's

happening, for example, let's take an Elkham-Razor argument about what happens in a fluid. The Elkums-Razor argument would probably be if the fluid is flowing from here to there, all the molecules inside it must be flowing in exactly that direction. That would be wrong. In other words, so, you know, and in fact, what's true is there's very complicated stuff going on. It just happens that the level of looking at the whole fluid, it can be described by saying the fluid goes from here to there.

So I don't think, I mean, I think it would be a mistake to say that there is something kind of, there's any necessity. If there's an Occam's razor that means anything, it means something because of the way our minds work. I mean, one key feature of our minds is that they're very finite. And, you know, we take all the stuff going on in the world, and we're trying to make a narrative about what's happening that is simple enough that we can stuff

it in our minds and make inferences about it. And for that, Occam's razor is very useful. Occasionally, things will poke through and be like, you know, Occam was wrong type thing. But you know, I think it's a feature, you know, I think perhaps one could even argue, you know, I've been on sort of the hunt for things that observers like us just take for granted. And I think in some sense, the simplicity of explanation is something that we implicitly take for granted. And I think in some sense, the simplicity of explanation is something that we implicitly take for granted. Let me see if I understand you correctly. In the same way that we observe general relativity because of the kinds of observers we are in the Wolff model, and in the same way that

we see quantum mechanics because of the kinds of observers we are in the Wolff model, we also, many people, many philosophers, many cognitive scientists, for instance, Don, are willing to say, look, we can move beyond space time and we can find something that can give rise to the physics that we have. And then in part, by doing so, they appeal to Occam's razor. But you're saying that also Occam's razor itself may be something that we find appealing because of the kinds of observers we are. Yes. That's interesting. Yes, I don't think Occam's razor, the Ruliat does not know Occam's razor.

Now what's interesting about what you said is that by assuming nothing you assume everything. So in some sense when you take Occam's razor to its pinnacle, you then undo Occam's razor to the utmost. Well, yes, in some sense that's right. I mean, in some sense, you know, by assuming nothing and getting the Ruliat, you have something where, sort of, to recover Occam's Razor in your observations of the Ruliat is something, is then a different sort of, a different adventure.

It's some, but I mean, I want to come back to this idea. I mean, this, you take the point of view, I think, that there is a desire to construct conscious experience from something else. And, you know, I agree, as I said, that it is like, you know, mechanism versus it's like the particle mechanism versus the S matrix and so on. There is, there are no doubt, complementary descriptions of what's going on. Which one is the easiest to build a formalism around is a matter of taste probably. And you know, for me so far, I found it easiest to talk about the Ruliat and so on and build up from that side of things. Now I will say that I'm pretty sure that there's a way of formulating a lot of the things that I've said about the

Rulliad, the principle of computational equivalence, computational irreducibility, all those kinds of things as essentially axiomatic statements about observers. That in other words, that one can, an alternative to going sort of bottom up is to simply say, for example, you know, Einstein did this in formulation of special relativity. He simply said, you know, there's, the observers can't determine sort of simultaneity in an abstract fashion. Observers have these limitations. And he took that as axiomatic. And from that, he constructed a physical theory. And, you know, and that was a sort of observer-first construction of a physical theory. In our way of deriving spectral relativity, it is not observer-first. It doesn't work

that way. I suppose it makes one kind of observer-related assumption, which is it says the only thing that we can in a sense pay attention to is the causal graph of causal relationships between events. We are not in a position to independently discuss the relationship of atoms of space. The only way that we can sort of say anything about atoms of space is by their effect on other things and ultimately, implicitly by their effect on us as observers. But now observers like us, as we've discussed, have conscious experiences or we have nothing, right? If we have no conscious experiences, we have nothing that we've observed. I don't quite understand that. So let's walk through that for a second.

Okay. So, I mean, one of the problems that I'm having is you mention sort of conscious experience and you know, I certainly have this internal feeling that I'm having conscious experiences, I try to imagine what it would be like if at some time in the future, well, here's a few different cases. So let's say somebody does molecular scale manufacturing of a brain just like mine, and somebody can scan my brain and, you know, reconstruct every molecule, then first question is, does that sort of copy of my brain also

have conscious experiences or not? So you're asking me. No, I'm asking you. Does that, in your view of things, would a, okay, so I'm going to, you know, we're going to go through several different levels because one thing would be a, a, one question would be, is a molecular scale copy of my brain able to have conscious experiences? Now, you could say no, you could say there's more there, there's other pieces of the rouillade that are poking into your brain that aren't pieces of the rouillade that are poking

into your brain that aren't part of the canon of physics that we know right now. That will mean that the thing you copied of just molecules, you didn't copy enough. You could say that. I don't know if you are saying that. Well, my analogy would be more like, again, the zoom screen. So right now, I see pixels on the zoom screen. Some of them are of your face and some of them are of inanimate objects in the back. And I could try to get a mathematical model of how the pixels of your face dynamically behave versus the pixels of the books behind you that behave in different ways. And I wouldn't want to say that therefore because I

understood I've got a model of how the pixels on your face are doing some other complicated computation different from the pixels of the wall behind you. That doesn't give me any real insight into the nature of the consciousness itself because in every case all I'm dealing with is just an interface. I'm not dealing with the consciousness itself. I'm seeing consciousness through an interface. So space-time, I'm saying, is nothing but another zoom screen. I understand. Can I make a comment here? So there are two ways of copying. We can copy Stephen by duplicating the window right now. But then there's another way where if you clone Stephen or if Stephen happened to be cloned, so if there was an embryo and it's split, now would you

say that look this embryo is operating in space-time so in some sense this embryo is operating at the level of the pixels on our zoom screen we could say that but then we would also see the two Stephens and say that both Stephens are conscious. I don't think that's what Tom is saying. I know that this may not be what you're saying Stephen but I'm curious so what would be the difference in the embryo case splitting versus copying Stephen that makes one not conscious? I think we need to define this more. You know, I think one thing is, if I had an identical twin, you would obviously believe, I think, that my identical twin, if the identical twin

was alive, would be just as conscious as me. Is that true? Right. I would, I would, I think it would be the best inference to make. Is that, you know, if it's fine. Okay. So the identical twin is conscious. Now let's imagine that in some future state of molecular manufacturing, I can make a molecule by molecule copy of myself. Would the resulting molecule by molecule copy of myself be as conscious as I am or not? Well, so the answer is going to be partly about what we think about this space-time

interface and its relationship to consciousness. So if we're taking a point of view in which space-time particles somehow give rise to conscious experiences by their complex interactions, then from that point of view, of course, I would then say, well, if those physical interactions in your body gave you consciousness, then presumably identical ones in another space-time body that's identical to yours, but also. But what I'm denying is that physical objects inside space-time actually give rise to consciousness. That space-time itself is nothing but an experience of consciousness. So what we call physical-

I don't think that's the same issue. I mean in other words, I think that that first point is if you know You it is not obvious that if I copy a proton for example That it could be the case that there's a special proton that is a proton in a conscious mind That is different from proton from other protons and it could be that when I copy the proton It it is no longer a conscious proton. It's a it's kind of a you know, it's a it's a lame dead proton You know, I copied the thing but it wasn't it wasn't conscious anymore Just like it could easily be the case that if I copied every molecule, that the thing

that I get wouldn't be alive. It wouldn't be operating, so to speak. You can have a simple analogy. If I copy a lump of computer memory, but that lump of computer memory is not being... There's no program counter that's starting to execute instructions in it. That lump of computer memory, while it is a copy of another lump of computer memory, it's not alive in the same sense that the original computer memory was. But so I think the first distinction is whether there is, whether the sort of electrons and protons and so on in me, if I were able to copy them, would they be, if I were able to make kind of a physical copy of them? And yes, that physical copy will be something that I would perceive as being in a different

place in space-time. I don't think that's the most important aspect of it. I don't think that's that important for your theory, actually. But I think the first question is, you know, did that copy that got made, was it, you know, did it preserve its consciousness or not? Well, maybe an even prior question is, do we believe in local realism in space-time. So I would want to argue that local realism is false and that, and even stronger, that in fact, particles only exist in the act of observation and otherwise don't. So to be really out there, I'll say neurons only exist when they're perceived and neurons do not exist when they're not perceived.

So local realism is false. And therefore this whole line of questioning goes away. All I have as an observer are my conscious experiences, period. When I talk about inanimate objects and particles and so forth, I'm now extrapolating from my first-hand evidence of conscious experience to something that I don't see. I get it. Right. They don't exist unless I actually perceive them.

So, I'm just to clarify what you're saying. I mean, it is the case that you could imagine constructing a science by talking only about conscious experiences and how those conscious experiences relate to each other. One could imagine building a science that way. And, you know, even there are little shadows of that in things I've done. There are shadows of that in the way special relativity is set up and so on. But it is... And it's, by the way, explicit in Chris Fuchs quantum Bayesianism. Okay. And in cube, so they call it cubism, but basically-

Yes, I know that. That's right, so there in some sense, what he's basically saying is, the observer is everything and all of quantum mechanics is just the handbook that the observer uses to interpret their experiences. But the thing to understand is, this is, it's a classic issue in lots of areas of science. You can describe things by mechanism or you can describe things by kind of what's achieved

in the end. So, for example, if we're doing mechanics, we can describe the equations of motion for something, you know, a ball going through the air. We can describe, we can say there's an equation that says what the ball will do at the next moment in time, or we can say we're using an action principle and there is an overall constraint that the motion of the ball should minimize the action quantity by the trajectory it chooses. So this is something, I mean, it's been there since Aristotle and probably before, these different forms of explanation of things.

My contention is that, and so I'll be very clear about it, that there is no mechanical explanation for any conscious experience. Not possible. Okay. I mean, so- So, Leibniz was right. I'm arguing Leibniz was right with his argument from the mill. And that right now, the work that's been done in cognitive neuroscience on the models of

consciousness, these are all my friends and colleagues that are working on this. I always ask them, okay, you're proposing a neurobiological mechanism. So what mechanism gives rise to the taste of mint? I understand, but you've got to have an end point. You've got to have an end point to make that a meaningful thing to talk about. You've got to be able to define what success means. In other words, what kind of an answer would satisfy you? Well, so these theories themselves tell you what they would say would be the answer. So for example, integrated information theory says you have to have the right causal architecture and you can specify it with a matrix.

I think these theories, I'm not a big fan of these theories. Okay. Well, normally. can specify it with a matrix. I think these theories, I'm not a big fan of these theories. I think what they're doing, it's kind of like they're describing something which is sort of the whole elephant and they're describing how it flips its tail in a particular way. That may be a little bit unkind, but I think that a case that perhaps is easier to pin down is things about the definition of life, where it's a little bit less controversial because there isn't this kind of inner experience type thing. It's like, what does it mean to be alive?

Is it self-reproduction? Is it beating certain thermodynamic things? Is it something about, you know, what is the, you know, what is the kind of the definition? And it's a mess. There isn't because in that case, as I said, it's a, you know, we have an N of one, but at least we've had 10 to the 40th organisms that have lived on this earth. In the case in what you're describing, you have really an N of one, because it's only you yourself internally who can definitively have something to say about what conscious experience is.

So I'm still, I'm fighting on this issue. I don't think, I'm not saying there isn't an answer, but I don't think you've given it, which is how do you define success? In other words, what, you know, let's say, I'm, and I'm going to, you know, that question of how do you define success, you have rather dismissively said that my friends,

the LLMs are all all merely regurgitating the things that went into them, so to speak. But you claim that we are not, so to speak. So, my question will be, if you can define a notion of success for consciousness as experienced by you and as extrapolated by you as experienced by me and other humans and so on, then the question is, that definition of success of, did you manage to derive that? Can I then, is that definition of success transportable enough that I can really apply it to an LLM? And perhaps the answer will be, you know, bzz, the LLM is not conscious.

But right now, you haven't given me anything that is concrete enough that I can take it and, you know, fit it onto the LLM and say, do you win or do you lose? Right. So I owe you a mathematically precise theory of consciousness, a scientific theory of consciousness that could try to do that kind of thing. Perhaps. That's what we're trying to do. We have a theory we call the theory of conscious agents.

And we have some papers that we've published where we have a mathematical model that uses Markovian dynamics in the model. And what we're doing right now is to try to answer your question, right? So I agree with you. What you're asking for is exactly what we have to do. And the way we're going at it is as follows. The high energy theoretical physicists in the last 10 years have discovered these positive geometries beyond space-time and quantum theory. And behind those positive geometries, they found these combinatorial objects that classify them. They're called decorative

permutations. And so this is just in the last 10 years. So we've taken off the headset, the space-time headset, and we've gone outside for the first time and we're finding these obelisks, these positive geometries outside of space-time and these combinatorial objects. So what we're doing to answer, to actually respond to your question is we're saying, let's start with a mathematical model of consciousness, qua-consciousness. So it's like a network of interacting conscious agents. So it's a social network and it's governed by Markovian dynamics. And what we're doing then is saying can we take this Markovian dynamics and first

show that we can project onto the decorator permutations that the physicists have found and then from there project onto the positive geometries. If so then we can project all the way into space-time and then we would actually be able to make testable predictions inside space-time from a theory that says consciousness is fundamental and we start there. So we've already, we published a paper last year where we actually showed some new mathematics apparently about Markovian dynamics and we showed how they can be classified with decorative permutations. So we published that and now what we're doing is showing, we're trying to show that we can get the positive geometries,

like the amplitude hedron, as projections of Markov polytopes, which are the spaces of all possible Markovian dynamics. So what you're asking for is exactly what should be asked for, and what we're trying to do is to show that we could get all of physics, plus more, from a theory of conscious agents being assumed to be fundamental be asked for and what we're trying to do is to show that we could get all of physics plus more from a theory of conscious agents being assumed to be fundamental outside of space-time and projecting through decorative permutations positive geometries into space-time where we can make our empirical test.

So that's what we have to do, but if we don't assume that consciousness is fundamental in the foundations of our theories, then we either have to dismiss consciousness and say it's not there, or we have to give a theory in terms of unconscious entities about how consciousness emerges. And if we try to do that last, I claim that it's not logically possible to start with unconscious ingredients and to have consciousness emerge. Not possible. Pete That is not my intuition. Okay?

I mean, you and Leibniz have the same intuition. I think the reason I disagree with Leibniz's intuition, if you'd asked me in 1980, do I disagree with Leibniz's intuition, I would have said, I don't know. I don't know how you would get a mind-like thing to arise from a non-mind-like sort of origin. But then, by 1981, I was starting to do all kinds of computer experiments and so on about what you know, what simple rules can actually do And it really surprised me in other words

What could emerge from something that seemed like it was too sterile to? Generate anything interesting. I was completely wrong and I you know, it's amazing that even after all these years You know I I do the experiments on different kinds of systems and I keep on being wrong. And I keep on thinking, this thing is somehow too simple to do anything interesting. And my intuition keeps on, even though I think I've now developed much better intuition about this, it is remarkable the extent to which much richer things than you might imagine can emerge from simple causes, so to speak.

And so I think that's a foundational piece of intuition that I've developed. It's kind of fun for me because this idea of computational irreducibility that I actually introduced about 40 years and a week ago. It is interesting to me that when I talk to some younger scientists and so on, for them, computational irreducibility is obvious. The world could not be any other way, which is how I feel about it too. But it's a thing where if you grow up with this idea, it's kind of an obvious idea in the end. It becomes obvious after you've kind of ground on it enough. But I think that that idea was something certainly not known. You know, that's

a new piece of information, a new piece of sort of intuitionally relevant information. So that affects my thinking about this. Correct me if I'm incorrect, Don. I don't think you're disagreeing with what Stephen just said. Stephen, what you had said is that look, we can start with something that's simple, mechanically simple, and then get to something that is extremely mechanically complex, such that we would never think, looking at the complex case that it could be made of these elementary elements. And Don is saying that's correct.

But notice the word mechanical there. You can get something that's simple mechanically and give rise to something that's complex mechanically. But that's a different question than jumping onto logical categories. Right. So the claim is that there's a spark of consciousness that can simply not be reached mechanically. That's the claim. That's the claim. That's the claim.

Right. So, okay. So, it's an interesting claim. It's a claim that I think the structure of the science that we have is not going to be able to talk about it. In other words, you can say, let's turn science on its head, and let's say that's our basis, and then let's see what we can construct about the rest of science. That's a perfectly intellectually valid thing to do. But if you're going to ask, given the fact that you're not able to give a kind of a science-based definition, you're not going to be able to get to what you want from kind of, you know, you might very well be able to, as I keep on saying, you know, from a theory in which all that's real is what observers observe, I have no doubt that you can go from such a theory to

deduce how things have to work in the world, and even to be able to say, given this way of how things work in the world, we could come up with kind of a sort of a meta theory that corresponds to space and time and all these kinds of things that is a good description of what we have derived from this underlying theory that has to do, that starts with observers. Actually, I want to ask something about that in what you described. Do you think that there could just be one observer? In other words, do you think it's important to the nature of observers that there are many of them and that they have some correspondence to each other? Or do you think that if in fact it was the case that you were the last human alive and

there's no other sort of, I don't know what, I don't think intelligence relies on life forms, but imagine that it did and you were the only thing in the universe that was like you and quote, conscious. Is that an okay situation or is there something that would not work in your theory? Would your theory require that there's a whole flock of observers there? Yeah, so it was quite striking that the paper we're writing right now that we'll be publishing hopefully later this year, we've discovered a new logic on the space of Markovian kernels.

So we were able to associate a Markovian kernel to each conscious observer. And the Markovian kernel is basically is describing given that my current experience is red, what's the probability the next one will be green and so forth and you can write down a matrix of it. It's what we call the qualia kernel. And, and I think that's a horrible problem with that, but we'll come to that in a minute. Okay, sure, sure. But so then there's a question, can these conscious agents and the Markovian kernels combine to create new conscious agents with more experiences? And we discovered,

we'll be in this new paper announcing a new logic on Markovian kernels that we just discovered. You probably know about taking a Markovian kernel and taking a trace chain on Markovian kernels that we just discovered. You probably know about taking a Markovian kernel and taking a trace chain on a subset of states. I can immediately imagine what that means, but yes. Yeah, so I thought about a 10 by 10 kernel. I get it. And I look at three of the states, it's going to induce the dynamics on the 3 by 3, and

you'll get a new kernel on the 3 by 3 kernel that's induced by the 10 by 10. It's called a trace chain. So it turns out what we're going to publish is that one kernel being the trace of another gives you a partial order on all kernels. So it turns out that's a partial order. So it actually, for example, the trace of a trace is a trace. So it's transitive and irreflexive and so forth. So it gives you a logic.

And it gives you a logic with a least upper bound, a greatest lower bound, and so forth. It turns out it's a non-bullying logic of these Markovian kernels. There's no top. There's no top consciousness. There's an infinite number of directions that you can go infinitely far in terms of combining. It's locally Boolean. So if I take a particular Markovian kernel, all the kernels that are less than it in this logic form a Boolean logic, so it's locally Boolean.

So just a technical question here. So we can think of one of these Markovian kernels as defined by some matrix. Okay, so are these finite matrices or are these infinite matrices? Well, right now what we've been doing are finite, but in this paper we're only going to deal with finite. We'll then look at the continuous case and so forth beyond that. Okay, so I mean what you're saying is, given that I have a probability matrix that says, your Markovian matrices are kind of like random versions of S matrices in a probability matrix that says, I mean, you know, your Markovian matrices

are kind of like random versions of S matrices in a sense that they're saying, you know, given this vector of what comes in, this, you know, you multiply by this matrix and you get this vector of what comes out. And you're doing that purely in terms of probabilities. But now what you're saying is given, given, I'm just trying to understand the technical aspect of what you're describing, given such a matrix, you are saying there are, you can extract sub matrices by tracing out by, I mean, for anybody who's watching this who wants to know what that actually means, it's, it's your, your, you're just adding your, you're getting rid of those components by just adding up a bunch

of things and fixing, fixing what happens. So, so we've got, you know, some part of our matrix is still flapping around, you know, free as a bird, so to speak. And another part has been locked down. And what you're saying is, if you, if in all the different forms of locking down, they form a, there's kind of a like subsets of a set or something, they form some kind of partial order of you lock down this and you, you know, there will be pieces that are in, you know, you can lock down this part and if you lock down a part of that part, it's sort of, it's a proper subset there and you can have another part that is sort of, that's, I mean, yeah, I'm going to start spouting technical things about chains and anti-chains and

so on, but which is probably not very useful, but at least helps me understand what's going on. Okay. That's right. But by the way, most Markovian kernels are not comparable, right? So, if I give you a Markovian kernel, almost every Markovian kernel is not greater than or less than it, right? It's quite an accomplishment to have any kind of relationship at all with other Markovian kernels, which gets at the diversity of consciousnesses and the relationship. But it turns out you can't combine consciousnesses unless where the states overlap, they have the same trace. You have to have the same trace on your overlapping states to allow consciousnesses to combine. You know, I'm hoping that there's more to consciousness than Markovian matrices. Well, because that's a shockingly minimal kind of view of what

I mean, and also to say I'm never a believer in theories that have probability as a fundamental component. Well, so there's two things there. So the first, though, I would bring up something called the theory of computational equivalence that I agree with you on. And it's a simple thing to point out that Markovian kernels are computationally universal. It's trivial. So the problem is, as soon as you've got probability in the picture, you're no longer dealing with pure computational rules. Probability is a statement. If you're looking at the manifold of all possibilities and you're just viewing probability as a parameter

effectively to sort of sample your space. So for example, let's say I say I've got a circle. There's a well-defined meaning to a disk, let's say, a region that's circular. And then I say, well, actually, I don't have a circle, I just have this probability distribution that allows points to be dropped anywhere in this region. Now I could describe the circle by saying that I have this probability distribution that in mathematical terms only has support within the circle, only has a non-zero probability within the circle. That will be a way of describing the circle. But if I am to talk about the

sequence of points that are dropped in the circle, then I've got a whole bunch more. It's no longer sort of accessible to pure computation. As soon as I can drop the points according to probability, I don't have a rule for where the points will land. Well, but there's a theorem in automata theory that the non-deterministic automata, Turing machines for example, have exactly the same computational class as the deterministic. That's a much more detailed issue. Let's unroll that. That's not correct. I mean, okay, deterministic and non-deterministic Turing machines absolutely have the same computational

power. But that is not the same statement as the probabilistic Turing machine has the same kind of computational character as a Turing machine. Let's, let's unpack that. Oh, I may have a different character. Sure. Sure. Right. Let's unpack that. We've got a Turing machine. A Turing machine has definite rules. You started from some initial state. It goes crunch, crunch, crunch and generates, you know, its succession of states. But now let's say it's a probabilistic Turing machine. And that means that it's what it

does at every step is not definite. It's determined by some probability, but it does, it does something. It's just, we don't know what it will do. And it has a probability of, you know, 30% of doing this and 70% of doing that. But at every step, it does something. Right? So that's, that's the probabilistic Turing machine. We don't know what it's going to do at every step, but at every step it does something. A non-deterministic Turing machine is a different story. A non-deterministic Turing machine is asking, what are all the possible things that could happen? We've got many paths of history.

The Turing machine could go left, it could go right. We actually take all of those paths. We build up this whole, you know, we call the multi-way graphs of all possible paths. Okay. And the statement is that if what you're interested in is does there exist a path that leads to this or that thing, that's, you know, the full, put it this way, the multi-way graph can, is computationally equivalent to the single-way Turing machine. That's everything you can, every computation that you can do with the multi-way Turing machine you could in principle do with a single-way Turing machine, but it'll be a lot of effort. That is not true with a probabilistic Turing machine. So probabilistic Turing machine, that choice at step three that you picked to go left, that choice is

unknowable by an ordinary Turing machine. That came from outside the system. That was the probability, the heat bath, the random, the God was playing dice and it came out this way. That came from outside the system and you can't know that. So, as soon as you have a probabilistic theory, it's not the case. It's not the same story as non-deterministic theories. A non-deterministic theory, it's not the case. It's not the same story as non-deterministic theories. A non-deterministic theory, there's still a definite thing, which is the set of all possible non-deterministic paths, which is different from, you know, if you said, well, as I was saying, in the case of the disk, for example, if the way you're describing the disk is to just say, let me look at all possible ways that the points could be selected there,

then yes, it's a nice kind of computationally describable version of the disk. It's a somewhat roundabout way to describe it, but it's the same kind of purely computational kind of concept. But if you say, I'm going to notice where every raindrop fell on the disk, so to speak. That is a different story. You can't know whether, you know, if it's a probabilistic

thing, it is not from within the system to know where the raindrops fall. So I think, I mean, Are you unhappy then with the notion of a probabilistic fundamental framework? You don't like that idea? Yes, I don't think that will. I think that if you say that, you are wheeling in, you know, who makes the choice in the probabilistic system? In other words, does God make the choice in the probabilistic system? How does that choice get made? Because in something like in a multi-way system, in a non-deterministic system, there isn't a choice to be made. All

possible choices are made. There's no deity playing dice. Whereas in a probabilistic system, you have to have something from outside the system deciding what's going to happen. Unless what you're saying is you're merely using the probabilistic system as a proxy for this multi-way thing, but I don't think you're doing that because it's a fundamental feature of kind of, I think, what you would call conscious experience, that there is a single thread of conscious experience. Now, maybe I should ask you this question. I mean, insofar as you think you know what conscious experience is, is there a definite single thread through time of conscious experience?

Well, actually, from your discussion, I would actually say that the way I'm thinking about is the multi-way thinking of it, that all possible consciousnesses, in fact, exist. All the threads are there. Wait a second. There's two issues. One is, do they exist? And the other is, are you experiencing them? Because I don't think you experience them.

I don't think you think you experience them. No, I'm not experiencing your consciousness right now, for example. Right. But I think that a critical feature of our typical conscious experience is that we believe we are persistent and we have a single thread of consciousness. We think definite things are happening in the world. We think definite things, we think that we are thinking definite things. You know, forget about what's happening in the outside world, but we imagine that we have a definite train of thought, so to

speak. We do not imagine that we have kind of, oh, they're a superposition of a hundred thoughts that I'm having right now. Rather, we think, at least we have the impression, might be wrong, but we have the impression that, you know, somehow we are just having a single thread of experience. I mean, do you agree with that? Yeah, that's my subjective impression is that there's a single continuing me that has taken one path that I couldn't have predicted and so forth. So that's the way I feel about it. And I agree that if you bring probabilities into a scientific theory, that's where explanation

stops, right? Right. Explanation stops where probabilities begin. And then you can either be an objectivist or subjectivist about those probabilities and how to interpret those. So I absolutely agree that probabilities are the end of explanation. And so when probabilities appear in my Markovian dynamics of consciousness, I'm saying this is where my explanation stops.

We'll need a deeper. If you want to get rid of these probabilities, you're going to need a deeper theory than the one I'm offering you right now. But the one is, can I offer you one right now with probabilities in it, where you say, oh, okay, that's where your theory stops, and I say, oh yes, that's where my theory stops. But if I can use that theory of consciousness

and show that we can build up, forget these positive geometries, get space time emerging, then maybe you'll grant me the dispensation to hold off on the probability until I show that I can actually do this, and then we can go back and say, now can we get rid of these probabilities in the matrices or not?

Right, so I suspect in your concept of kind of, I think from what I'm understanding, when I asked the question, could there be just a single observer in the world? I think what you're saying is no, because you're building a calculus of the combination of observers. And yet, it's not a Boolean logic, so there's no single top observer. But there is, in some sense, you could talk about the whole of all the observers. If you want to say, is that an observer, I might say, yeah, maybe instead of one observer,

there's the whole observer, which is you can go infinitely far in infinitely many directions. So there's not an infinite one top. There's an infinite number of directions that you can go infinitely far. And so, this notion of conscience is really complicated, but I don't think I can say that there's one, but I can say there's a whole. Pete Okay. But so, are you, in your Markovian partial order, are you and I part of the same post set, or do we each have our own separate post sets? Well, we're part of this big whole post set, the Markovian post set, but we may be, and we're partly in branches that are partly compatible because we're talking and presumably something's

happening. We're not completely incompatible. All I know that Don is you're greater than Leibniz. That's partial order. Well, he has the advantage that time has gone by since Gottfried was around. Well, his IQ was at least double mine. So, but anyway. But okay. double mind. So, but anyway. But, okay, so I'm fairly confused here because on the one hand, we agree, I think, that the only conscious experience that you can have any definiteness about is the N of one conscious

experience that you are having. That's right. Okay? That's right. So now, in your theory, you're talking about multiple conscious experiences, multiple conscious agents or whatever, that have certain relationships. And so, I'm not even, I mean, so you're positing that you're taking your sort of empirical inference that there are other conscious agents in the world, and you're saying I'm

really going to believe in that because I'm going to make a theory that has many conscious agents in it. Is that fair? Yes, I'm also going to believe that the experiences that I have had in my life do not cover all possible conscious experiences. I'm going to admit that there are experiences that I don't have yet. And for example, as life has gone on, I've had brand new experiences I'd never had before. All of a sudden you go, oh, I'd never had that experience before at all. Right.

So why is that? I mean, but you're saying that somewhere in, okay, so in your theory, there are sort of, is it the case, first of all, is that time in your theory? Or is it merely the kind of, the partial order sort of, is it merely the pecking order of consciousnesses or is there some kind of progression there? What's interesting is that, as you well know, you can have stationary Markovian processes, in which case there is no increase in entropy from step to step. So there's no entropic arrow of time.

And so what I'm imagining is that the full dynamics of the whole consciousness is stationary, but I am a projection of that. So I'm a trace, so I'm a projection, so I've lost information and it's a theorem, pretty easy to prove that when you take a projection say by conditional probability where you lose information, the projected chain will have increasing entropy. So I'm proposing that there is no time for the whole consciousness and time emerges as well as space as an artifact of the loss of information and projection. So what we're going to actually try to show is that time and space themselves are all artifacts of projection and not an insight into the true nature of the deeper

whole consciousness. Okay, so let's unpack that a bit. So one thing we can imagine if we take a fairly traditional space-time view of the universe is we can imagine that there's this giant crystal that is the whole space-time history of the universe, right? It's just there. And then we can imagine that our experience of the universe is merely motion in the time direction through this crystal that is the space time, the representation of all space time in the universe. So I think what you're saying is,

you are imagining, and I want to unpack this a bit because I think there's there, you know, you're imagining that you have this thing. It's a partial order of Markov matrices basically. And by the way, I think it's not really fair to talk about it as a logic. I mean, it is a logic in some sense of universal algebra or whatever else. It is a logic in some sense of universal algebra or whatever else, but I don't think by saying the word logic, you're kind of making that sound like it has something to do with human experience. You know, logic as constructed by Aristotle is kind of this way of representing sort of the way humans construct arguments. And I don't think that kind of the mathematical structure that you're describing as a logic is, you know, it is, you could as some kind of mathematical definition that you're describing as a logic is, you know, it is

you could as some kind of mathematical definition, you could say it's a logic, but it certainly isn't logic with the same kind of import that Aristotle's version of logic has. You know, just to make that point, I mean, I think it's a... Well, if we think about, so think about probability measures as propositions. Uh-huh. Right? So they're propositions. And we can talk about the, when we talk about, it turns out we can put a partial order on

probability measures. This is something we did 30 years ago and it's called the Lebesgue logic. And so it turns out if you say one probability measure is less than another, if it's a normalized restriction of the other. Okay. That gives you a partial order on the sort of all. I can believe it. Yep. And so now the reason I would call that a logic is because I can think about probability measures as propositions, and here I am taking the and and the or and the conjunction, disjunction and negation and so forth. So in that sense,

I'm calling it a logic because it's logical relationships among propositions. But wait a second. I mean, you know, the notion of and and or, which I claim is a deeply derived notion. I mean, in other words, that is not a foundational notion. That's a notion, you know, processed through layers of kind of symbolic representations of the world by humans and all kinds of things like this. But be that as it may, I don't view logic as being in any way fundamental, but be that as it may, you can say, you know, there's the and of, you know, you've got, I'm still trying to understand, in your kind of Markov matrices, you can say, well, do you, are you associating propositions in some way with these Markov matrices or not? Yeah, the probability that if I see red now, I'll see green next is

0.01 and the probability that I'll see blue next is 0.03. That's the proposition. is.01 and the probability that I'll see blue next is.03. That's the proposition. Well, wait a minute. The proposition is, so you're saying the Markov matrix itself represents a proposition. Yeah, it does. It represents the statement. I mean, the Markov matrix is a collection of probabilities and the assertion, these are the probabilities, is the proposition.

That's right. And if you view it that way, then when you have these, when you put a partial order and you look at the meet and the join, then you could be thinking that these are in some sense logical relationships among propositions. And so why not just call it a logic? But if you don't like that term, Okay, fair enough. That's a better answer than I thought of. Okay, so let me just understand what you said there. So and maybe these explanations are helpful for anybody who's watching this. I don't know. But it's getting technical.

It's, I think, I mean, what you're saying is if I say the probability that I see red is 50% and whatever, and then another proposition is the probability that I see red is 30%, what you're asking is, and which I'm a little confused by, if I take the and of those propositions, I don't see how I construct that out of your kind of, I mean, those seem to be inconsistent to me. Those seem to be, you know, they're on sort of an anti-chain of your partial order. So how do I do an and of those things? Right. So the only way that you can take the AND of two probability measures is if they have the

same normalized restriction on the propositions that they overlap on. You have the same normalized restriction there, otherwise you can't take the AND. In other words, I've got to have if one Markov matrix says 50% probability of red and 8% probability of purple, and the other one says 50% probability of red and 8% probability of purple and the other one says 50% probability of red and 6% probability of yellow. They're incompatible. Then I can combine them, you know, if they didn't have things to say about purple and yellow going, you know, between each other.

So insofar as they're disjoint, you can combine them. Okay. Or if they speak about the same, if they both agree that red is twice as probable as blue, then they're fine. As long as you agree about the relative probabilities on things, then you can take the disjunction and conjunction and so forth. Okay, fine. Fine.

So then, all right, so I'm buying more that you can, I mean, I think it's a very weak logic, but you can set something up that has some of those attributes. But so now, I mean, the question is, you're imagining that... So one question is, what can you derive from what? So one of the surprising things about our physics project is that what I had not imagined is that you could derive so much from so little. And so, you know, you would think that the statement consider all possible computations, you know, the entangled limit of all possible

computations, you would think that you could derive absolutely nothing from such a thing. But the surprise is that, you know, as soon as you put these conditions about how observers can sample that, you suddenly start to be able to derive things. I think the simplest case to see that is the molecular dynamics case, where you can say, you've got all these molecules bouncing around, and we know that they conserve number, maybe they conserve momentum and things, that those don't matter that much. But we've got all this microscopic sort of randomness, computational irreducibility going on.

Just from the fact that observers like us are computationally bounded, we can now derive the second law of thermodynamics. We can start to derive fluid mechanics, things like this. So in other words, it's very surprising that from so little you can get so much. And that's you know, that's the thing that really I didn't expect at all. So what I want to understand for your what you're doing is, you know, I think you're also attempting to get much from little. I think you're also attempting to get much from little. Absolutely. So, I want to understand what you would like to get is things like, I mean, honestly, I

think you're more likely to get the Ruliat than you are to get space time in its usual formulation. I think it will be easier to get from the kind of thing you're describing to the Ruliat than to get to all of the technical detail of space time and so on. Let's just understand what it would mean. So again, I want to sort of posit this kind of axiomatic physics where all you're doing is you're saying, I make these observations and all I know is that I have certain axioms

about how these observations fit together, which I think is what you're, you know, you are positing certain axioms about how what you're describing as conscious agents fit together. That is, right, critically, critically, I think you're positing something which seems completely unobvious to me, which is all we know is the N of one. We have an internal experience of being a conscious agent, right? But you are positing a network of relationships between conscious agents in your partial order and things like this. And that to me, that's a big leap. Now, you might argue the Rulliad is a big leap too, but what you're doing there is you're saying, you know, all I know is what I have internally, and I'm talking about that as

a conscious agent. But now I'm going to posit about conscious agents that they have these interrelationships. And by the way, I'm pretty sure if there was only one conscious agent in the world, you know, the game would be over. You wouldn't be able to construct, there'd be no grist to construct a sort of an external model of the world. Because what I think you're doing, as I understand it, is you are going from the calculus of observers to construct an external model of the world, which is the opposite way around

from what I've been trying to do. So now I claim if there's only one observer, there's no grist, there's nothing you can do to build up that external model of the world. Just as I don't think you can tell in the solipsistic view of things, you can't really tell whether there's anything out there. You are taking your personal extrapolation that there are conscious agents like you that have certain relationships. You're taking that and building what amounts to what we might call it a scientific theory, we might call it a theory of the world somehow based on that.

So, if I think that there is what I call the whole, right, so it's this really infinite conscious agent, I can imagine it then choosing to look at itself through different traces. So I'm going to choose to look at myself through the trace, and this trace I'll call Don Hoffman and that trace I'll call Stephen Wolfram. And these are just different, so it's the whole looking at itself through a straw, through a straw hole, right? Because the whole is infinite and I've got a finite IQ. So in that sense, you wouldn't have the problem

of not having the ability to have interesting worlds and so forth if there's this infinite consciousness that's looking at itself through an infinite number of different perspectives, then suppose, so that's what I am and you are. So from this point of view, Don and Stephen are just avatars of this deeper whole consciousness. The whole is talking to itself through Adon and Stephen Avatar right now. Right, right, right. You know, that's bizarrely close to what I would say about the Ruliyad. So in other words…

I thought we were coming into this. I actually thought that we were going to end up pretty much agreeing that we're doing the same thing. I'm just calling it consciousness and you're not. Right. Well, but the thing that I don't get in what I'm doing, I'm imagining that there are these atoms of space, and I'm imagining that there's this hypergraph and so on. And do I know that these are real things? No, they're my way of describing the world. I mean, it's like, occasionally people will come and say things like, oh, you have a computational model of the world.

What kind of computer is it running on? That's a hopelessly philosophically muddled point of view, right? And so, you know, this is merely a description. And I think what you are, so, so let me, let me see if I can unpack your description. So you're saying your whole is the set of all these possible connections between consciousnesses and maybe, maybe you're even going, you know, and I think you have to go this way in order to avoid sort of the trap of probabilities and the dead end of oh, there's probabilities where we don't know what the particular role of the dice is. So you're going to end up with essentially a multi-way collection of all the possible histories and so on.

So you've got this whole structure that is kind of the, I mean, I think, okay, so I think what you're constructing, I mean, the object that you're constructing, it's, you know, that is a mathematical object. I think your sort of Markov chain thing is, is weaker than it should be. In other words, I think replace that with an arbitrary computation and you basically have the Ruliat. You have the same object. So in other words, he's inviting you to co-publish, Don. I don't publish things. Well, I think you raise a really interesting, very technical question that I think we should really try to address is the relationship between the roulette and what's possible with this infinite, this lattice of Markov.

I'm pretty sure that what you've got is, you know, with this partial order of Markov chains and so on, that's a definite mathematical structure. It is a much weaker mathematical structure than something where your relationship of taking traces is much weaker than an arbitrary computation. But I don't think it's a huge leap to say that's a particular sub-model that might capture some aspect of how it might be a useful phenomenological model of certain aspects of conscious experience that you have or whatever else. I think the more general my feeling is, you're slipping down a slope here. First you have to, and you're going to wind up with something.

This is one of the things that again has been a surprise to me. The Ruliat is the end point of an awful lot of generalizations. So in other words, there are, in mathematics, if you're looking at, you know, Grothendieck's work on higher category theory and, you know, infinity groupoids and things, that object is basically the Ruliat. That object, and in fact the, you know, Grofendieck's hypothesis about the inevitability of what amounts to topology or space or whatever from a thing of that kind is precisely the assumption that we are also making or the thing that we think we can give some level of derivation of that space inevitably emerges from observers in this

rule of ad and so on. So I think it would not be a surprise to me that the end point of an effort of generalization is the same object because is, you know, is that, you know, if you're thinking about kind of what I might call, I don't know what the right word for it is, but I'd be calling it axiomatic physics. I'm not sure if that's the right characterization, but it's a, you could think about it as a calculus of observers as opposed to, or a, you know, where everything is just in terms of the relationship between observers. But it's really critical, I think, to what you're talking about, that there isn't just one observer. I don't think you can, I don't think, as I said, talking of mills, I think there is no grist for your mill without a multitude of observers. That is, I think you can't, you know, because if you're going to be able to construct extent

and so on, you need that. And now the question is, if you, and so then I want to come back to your, you know, experience of mint. Well, let me first just agree with you on the two points you've made. First, I agree that I would be delighted if it turns out that the Markovian dynamics that we're doing and the partial order turns out to be equivalent to the Ruliat. I would be delighted. It won't be equivalent. It will be a subset. It's a small piece of it. I mean- Well, again, you can get computational versatility out of two or three Markov kernels.

All you need are two or three Markov kernels and you get computational universality. So that's why I think if there's a relationship, they're equivalent. Okay, fair enough. I don't think that quite makes sense because Markov chains are probabilistic and involve real numbers and you're kind of out of the game of computation theory by the time you're dealing with those kinds of things. Two or three kernels that are not probabilistic, they have only zeros and ones in the matrix,

get computed. Remember, the Markov kernels include zero one matrices and they include the deterministic ones as a special case. So we get that even just from those, we'll get the roulette. Hold on, hold on. It's not so simple because let's talk about how you actually apply. I mean, this is you've got these matrices and you know, if you say what you are constructing is a product of many matrices, which is not what I've heard you say. What I've heard you say is that you're taking these matrices and you're tracing out components and you're looking at the partial order of matrices. That's a different statement from the statement that you are taking proletes and matrices.

And I agree with you that it's not quite as simple as that. I mean, you can't, you know, to get, let me think about this for a second. Certainly with finite matrices, you will not get computation universality. You're going to need infinite matrices. And in fact, okay, so here's a construction that you could easily make. So you could imagine building a cellular automaton by just taking a vector that represents a one-dimensional cellular automaton, a vector that represents current state, and you have an infinite matrix. No, that's not going to work. That doesn't work. That only gives you a subset of cellular

automata. You can't get... So if you have a single matrix and you're simply doing matrix multiplication, there's linearity to matrix multiplication. And you're only going to get a very small subset, which by the way, aren't universal, of cellular automata. Now, if you say, oh, I'm going to make these matrices be like elements of a group, generators in a group, and I'm going to say I'm going to multiply these together in all sorts of different ways, then that construction, yes, you can get computation and analysis out of it, but I don't think that's what you're talking about.

And that's what we do. So I hadn't talked about that part of the theory yet. So I only talked about this one, what I call the qualia kernel. It turns out the qualia kernel is actually a product of three kernels. So we actually, in the basic formalism of the consciousness,

and I send you the paper on the consciousness theory, we have a decision kernel, an action kernel, and a perception kernel. When I take the product of all those, I get what we call the single qualia kernel. But what we're imagining is that there's this infinite social network and that there's actions, message passing and so forth that's happening. And it's all going to be done by products of Markovian kernels throughout this whole thing.

So it's going to be a computational universal network. And we're going to get, you know, some of the kernels can have no probabilities in them at all. They're just zeros and ones. And so... Yeah, get rid of those probabilities. You're going to have a much easier theory if you get rid of the probabilities. Because as soon as you have the probabilities, as you say, it's kind of, you're admitting, you know, incompleteness of your theory, so to speak. You're saying, there's, you know, I just don't know where these, where the dice rolls

are coming from. But let's not… Well, on the incompleteness of theories, I would say that every scientific theory starts with assumptions, and those are the miracles that the theory doesn't explain. Well, that's an interesting point. Okay, so that's the bizarre thing that I didn't see coming about the story with the Ruliat. It doesn't, you know, the representation of the Ruliat in a particular form, that is a sort of arbitrary choice that you can think of as an assumption. But the thing, the actual object that you construct,

I don't think that has... It's not the kind of a thing that starts from assumptions. It has been the experience of all of scientific theories to date, that all scientific theories have been... They've been models, and as a model, they are not the system itself. There's some projection from the system itself, some simplified narrative about the system itself. The thing that's bizarre with the Rulliad, and I'm still trying to wrap my arms around this thing because it really surprises me a lot. It is inevitable, and it is something that is just, it's a unique, inevitable thing that doesn't, it isn't like a, you know, you

say scientific theories have assumptions, because I think one's imagining, as one usually has done, that the theory is a model where it's assuming, oh, it doesn't matter that such and such is such and such a way. So, I think the, I mean, in our theory with the Ruliat and so on, the assumptions come in, and assumptions about what we are like as observers, which is a different kind of a, you know, and that's the underlying theory, the underlying reality, you might call it, is just the Ruliat. And in some sense, it's everything, but it tells you nothing. To have it tell you something, you have to take these slices, and these slices are particularized by, you know, features of us as observers, so to speak. Let me just ask a question about it. There are two questions. One is, does the Ruliat admit something like Gödel's incompleteness theorem that would

hold for the Ruliat? I mean, it's in some sense, even though you're talking about this infinite thing, with mathematics in general, Gödel says that any system that has the, you know, formal power of arithmetic, there'll always be theorems that are true that can't be derived within that system. I'm just wondering if the roulette has some kind of incompleteness as well. Yes. I mean, okay, this is complicated to untangle. Let's do it for a second, okay?

I mean, you know, Gödel's theorem is built on top of a bunch of assumptions about truth and so on. I think it is more useful to think about, let's see, where do we start here? The point is that sort of the thing that I think is the underlying phenomenon that Gödel's theorem is built on is computational irreducibility. Because what you might say is I'm going to start from these axioms of arithmetic and then any theorem I must be able to just finitely prove from those axioms. But in fact, there's no upper bound on how many steps you might have to take to get to the theorem that you care about.

Unfortunately, okay, so the basic point is computational irreducibility, which is kind of the core of Gödel's theorem, is absolutely alive and well in the Rulliad. In fact, without it, there wouldn't be time, there wouldn't be space, there wouldn't be lots of things. The fact that the passage of time is meaningful is a consequence of computational irreducibility. If it wasn't for computational irreducibility, the leading of our lives would be there would be nothing that was actually happening. It would just be, oh, we could jump to the end and say the answer is 42 or whatever. It would be, so that's a, you know, and similarly in the fact that

there is an extent to space that is also a consequence of computational irreducibility. So in those things, I mean, computational irreducibility is absolutely fundamental to the non-collapse of the Rulliad. The Rulliad would collapse without competition or instability. That's beautiful. That's beautiful. The fact that there's, you know, the fact that it has extent is a consequence of that. So now you can ask questions about, well, let's talk about mathematics for a second, because one of the things about the Rulliad that's again something I didn't see coming

is the Rulliad is not only the foundation of physics, it's also the foundation of mathematics. And so, and in fact, it has the bizarre consequence that, you know, in the sort of platonic view of mathematics, that there's a there there, so to speak, that what you end up concluding is if you believe that physical reality exists, you must believe that there is a mathematical reality that exists. Does it also work the other way around? If you believe in the sort of Platonic view of mathematics, then I think so. I haven't thought about it that way around because people are usually, people usually, maybe Don is an exception,

but people usually believe in physical reality. People usually don't have a problem with the notion of physical reality. Well, Plato would have said that the true reality is the Platonic reality, and then this one is the illusory one. Yes. Right. Yes. Fair enough. Right.

So, I mean, in, but, you know, just to understand how that works in mathematics and how that sort of how Gödel's theorem works there and such like. So in mathematics, and this is, yeah, in mathematics as it was formulated in the 20th century, perhaps not in the best possible way, but the formulation in the 20th century and from Hilbert and people like that was we put down these axioms and then we see what theorems we can derive from those axioms. So, for example, the particular case Goethe looked at was we put down the piano axioms for arithmetic, you know, X plus Y equals Y plus X and a whole bunch of other axioms, and then from those axioms we try and fit them together to derive

other theorems. And the question is, if we fit together those axioms, do we, you know, is there a finite path to every...where do we get, you know, we get certain things that we can construct from those axioms. One of the things that's tricky about Gödel's theorem, as it's usually stated, is it's not a question of what one can construct. It has this notion of truth, which is an overlay on top of what one can construct. So, you know, I can construct the statement X plus Y equals Y plus X. From that, an associativity of addition, I could also construct the statement

that X plus Y plus X equals Y plus X plus X, for example. That's a thing I can construct. And, you know, first question is, is every, well, this notion of what, let's see, I mean, of what, the question of what's true is more complicated than the question of what you can construct. And that was part of the point of Gödel, right? Is that the notion of truth transcends the notion of proof. Yeah, I think that's a technical detail, actually. I think that's a confusing feature, and it's confused people a lot. The real essence, okay, what did Gödel actually show? What Gödel did was he wanted

to take the statement, this statement is unprovable, which is a statement that doesn't seem to be a statement about arithmetic. And the remarkable thing that he did was to show that that statement can be compiled into a statement about equations about integers. That you can have an interpretation of that statement that is just a statement about equations about integers. That fact that you can compile that into a statement about integers was an early version of the idea of computation universality. That is that you can take this thing and compile it into this set of primitives.

And then having done that, then you can feed that statement. This notion of provability is then something that tangles itself up through that statement. But the remarkable thing is not that that statement doesn't, you know, that statement is a kind of a paradoxical mess. The remarkable thing is that that statement is actually a statement of arithmetic. That's right. And but what's interesting though is that, you know, someone like Roger Penrose, for example, looks at this

and says what he takes from Gödel's incompleteness theorem is that I that something about me that allows me to understand what this formal system cannot do. I can understand the truth of this thing, but I understand it and the formal system cannot. So that's really the key. For Roger Penrose, that was sort of the big take-home point from this and I and I would agree But it sounds like you disagree

Dawn and and Stephen would you say that it's correct to characterize you Stephen as a Computationalist and dawn that you think there's something more to reality or to consciousness than mere computation Yeah, yeah, I'm I'm suggesting that girdles incompleteness theorem suggests that the notion of truth transcends the notion of proof. So I'm all for the Ruliat and I'm all for mathematical models, but I'm suggesting that there's something deeper. Yeah, but I think the problem is this notion of truth is a complicated, derived human concept. And I don't think it's the right thing to think about as a foundational thing. I think that

constructing things is a much more useful foundational idea. So for example, in talking about kind of, I mean, obviously what Gödel was trying to do. Gödel was kind of a Platonist, and what Gödel was trying to do was to blow up kind of Hilbert's idea that there wasn't a there there for mathematics, so to speak. But I think this notion of, for example, to have truth, you have to have a notion of falsity. What is the notion of falsity in something where you're constructing things? Well, here's what it is in our... So, you know, I've had the nice opportunity to, you know, Gödel talked about the arithmetization of meta-mathematics. I actually even put together a book that I talked about the physicalization of meta-mathematics. The fact that the network of all possible

theorems, how they prove, youms, how you can prove one theorem for another turns out to be the same kind of construct as the way that physical space can be constructed in the universe. Both of these things are the rulliad. And then the question of how we perceive mathematics is a question of what we are like as mathematical observers. Mathematical observers are rather different from physical observers. You know, a mathematical observer, you know, a view of a mathematical observer is, a mathematical

observer doesn't care so much about time, but a mathematical observer is just trying to put a bag of theorems into their mind. They say, this is true, this is true, this is true. That's the notion of truth is this is a theorem I'm going to say is true. It's a thing I'm going to put in my mind, I'm going to say it's true. Okay, so now the question is, what is falsehood? In other words, what is, you know, I've got these theorems, I'm foraging in the forest of theorems, and I keep on putting more things in my bag. Turns out that the, I think that what falsehood is in our models is what you get from kind of this medieval concept of the principle of explosion. If you have something which is false from a falsehood, you can derive every statement.

Right, exactly. And so, then what happens in our models is that normally you're putting these theorems in your bag and you're saying, these are the ones I think are true, but suddenly you put a false theorem in your bag. And then what happens is then everything is true. And so what goes wrong? What goes wrong is if you have a finite mind, your mind is exploded at that point. You can't fit.

So, in other words, it's a, you know, that's a, so I'm kind of describing a more, a kind of physicalized version of the notion of truth and so on. And I think, you know, this idea that, I mean, there's sort of the glib statement, which I don't even know where it came from. I've never really traced this history of, you know, statements which are true but unprovable. I think it's a super confusing way to think about Gödel's theorem. But, you know, I think, I mean, this whole question about whether, I mean, you know, this, I still want to come back to, because I'm really interested in this question about what

your statement about the experience of mint, and your kind of... I think the theory you're constructing is a theory that extrapolates far away from your internal experience of mint. Your theory talks about the interaction between observers and between consciousnesses and so on. In some sense, it's kind of a flip around of the theory that starts from the particles. It's a flip around to a theory which talks only about the effects. And, you know, only about the observers. And then I want to get the particles.

One of our goals is to show that we can actually model the momentum distributions of the quarks and gluons inside of a proton, starting only with this dynamics of consciousness outside of space. Yeah, you won't get there. That won't work. But I think you'll go, if you can, you will probably be able to get from this kind of formalism, my guess is that you will be able to get basically to the Ruliat. And then, you know, then it's, I'm all in favor of more people pushing to get from the

Ruliat to the momentum distribution of, you know, to the structure functions of protons and distribution and momentum distributions of protons. That's a heavy lift. I don't think we're going to see that in a short time. I'll just give you a little idea about how we're trying to do the lift. So we're proposing that particles in space-time are projections of communicating classes of Markov kernels. The notion of mass of a particle is a projection of the entropy rate of the communicating class, and the spin is a projection of the determinant,

and the momentum is a projection of the number of asymptotic events inside the communicating class. In other words, we're building up a dictionary that says these physical properties are projections of these properties of the Markovian dynamics. And so we'll see. And then in that context, we're going to try to get the momentum distributions inside the proton. I don't think that's the right target.

I think there are other, I would, if I were doing this, I would try and get general relativity. I think general relativity is a much lower hanging piece of fruit for what you're talking about. I think that the problem with particle physics is, you know, knowing what a particle is is kind of complicated. And I think that the kind of the structure of space-time, the overall structure of space-time is much easier to try and get to. But I mean, taking a little bit apart what you're saying. Fair enough. By the way, a point well taken. Thank you. I mean, it's, you know, I think that, for example, in our model, you know, one of the things that surprised me a lot was the very easy interpretation of what energy is.

So it turns out that energy is basically the amount of activity in this network. I mean, more formally, if you make the causal graph, it's the flux of causal edges through space like hypersurfaces. Momentum is the flux of causal edges through time like hypersurfaces. Which by the way, is something I could imagine you being able to get as well. I mean, you being able to make that interpretation. I think the thing that surprises me and what you just described, so, you know, let's talk about entropy for a minute, because entropy is another one of these often misunderstood, you know, constructs. I mean, you know, entropy, what's the definition of entropy? I mean, in a sense, entropy is

basically you take a system, you know certain things about that system, and then you say, how many states are there in the system that are consistent with the things we know about it? And you take the log of that and that's the entropy. So let me understand, when you talk about, you know, when we talk about entropy increasing, it's a, I mean, again, this is another layer of complexity in what we're talking about because what we're doing is we're saying the number of states of the system consistent with what we observe is increasing, let's say.

But if we have a system which is a determin deterministic system, and we know everything about what it's doing, and it's also, let's say, a reversible system, so we can always take a state of the system and find previous states of the system as we can find future states of the system, in that case, if we could observe everything about the system, its entropy would always be equal to one, zero, rather, because there's only one possible state of the system. It's the state of the system, future state of the system, and so on. So what leads to our perception of the increase of entropy is that we are not observing every detail of the system.

We're instead observing only certain features of the system. And with respect to those features, we say given these features, there are more and more states of the system consistent with those features. So can you say again what, because I didn't understand what you meant by, so you were saying something about entropy being related to something else. Well, the entropy, so one proposal is that the mass of a particle is a projection of the entropy rate of a communicating class. So the entropy rate, you know the definition of entropy rate for Markov kernel? Tell me, tell me it.

Okay, yeah. So for anybody else who's watching, even if I know it, the chance that everybody watching knows it is incredibly low. Well, the toe audience is quite technical and they not only can keep up but enjoy it. So indulge. So I have a recurrent communicating class. It's got a stationary measure. So it means there's a long-term probability of being state one through state M. Okay. So I got the stationary measure and then each row of the matrix is a probability measure

and so it has an entropy. Hold on, hold on, hold on, hold on. Let's unpack this a bit. So, we've got this matrix that says here's a vector of what's happening right now and a vector of probabilities for right now and we're going to apply this matrix to get a new vector of probabilities for the next step and we're going to apply this matrix to get a new vector of probabilities for the next step, so to speak. Right, right, right.

Okay? And now you say, let's apply that matrix a zillion times. And the result of that is we're going to go to some limit, and that limit is the stationary measure, as you're calling it, that there is a limiting matrix in which every entry in that matrix has some particular value that corresponds to the ultimate limiting set of probabilities. Being in that state, that's right.

Okay, I got that. Now what? So the stationary measure gives you the ultimate probability of being in state one, state two, and through state n. And then now if you're in state one, right, there's a transition row. There's a probability measure about where you're going to go next. Yep.

That probability measure, you can take its entropy, right? So you can take the probability measure, take its entropy. Now you just multiply that entropy by the stationary weight and add them all up. So that's all you... So it's a weighted sum of all the entropies of the rows weighted by the stationary weight. I mean, here's where I'm getting into trouble. Because yes, at a mathematical level, you

can compute sum of p log p for all these entries in the matrix. What the interpretation of that is, and maybe you don't need an interpretation of that. But for me, the entropy, again, this is by putting probabilities in, you're kind of cooking things in a certain way. For me, when I'm talking about entropy, I want to know what are those individual states? It's kind of the frequentist version. I'm not just saying there's a probability, I'm actually saying what are the things underneath that probability. So, you're, but I don't know whether... And I'm not. I'm taking these probabilities as the foundations of this particular theory.

Okay. So, it's a purely mathematical thing that you're doing. So, it's not, there's no interpretation of entropy here. It's merely the mathematics of... That's right. ...P log P's right. Okay. That's right. And of course, entropy rate, for example, is a big deal in communication theory. If the source has an entropy rate that's bigger than the channel capacity, you get distortion and so forth.

So it's that kind of thing that comes up in communication. It's always fun to trace those things through for like 5G and see how the fact is all these things that people said, it's a theorem that you'll never be able to communicate faster than this. And then somehow we managed to have cell phone channels that break all those theories. Anyway, that's a separate, different discussion. But okay. But so I'll just say one little fun thing that comes out of this. If we define the entropy rate, the mass to be a projection of entropy rate,

then that forces us to make certain predictions. So a mass zero would correspond to an entropy rate of zero. And that would correspond to a Markovian matrix that has only zeros and ones in it. A single one in each row and all has zeros. And well, so we know that in space time, massless objects must move at the speed of light. So it better fall out of our theory that you get the maximum travel speed in our theory for the things that have zero entropy rate.

And it turns out if you look at what's called the commuting time between states in a Markov kernel, the maximum commuting time, the in a Markov kernel, the maximum commuting time, the fastest commuting times, so the smallest commuting times, the fastest travel times, are for the ones with have zero entropy rates. So we actually get that. And the maximum speed is one state per step of the chain. Hold on. You're commuting lots of different concepts here. I mean, when you're talking about things traveling from here to there in this Markov chain,

it's like you have a vector and this thing is kind of moving the probability measure from one part of the vector to another. Right, that's- Yeah, you're going from one state, from one conscious experience to another conscious experience. And the question is how fast can the conscious experiences change?

Right, but by conscious experience here, you are taking what I would consider to be a kind of a, you know, I hope that in a sense, I feel my conscious experience is a lot richer than, you know, than this, lot richer than your probability vector. This is, again, one of the things that is difficult about this, the intuition about all these kinds of things. For example, in this idea that you can have richness of things emerge from simplicity. Or another thing that took me a long time to come to terms with, I'm not sure I completely come to terms with it even now, is that the universe

is an unbelievably profligate waster of computational resources. And, you know, I had always imagined that there would have to be a definite history in the universe, that it couldn't be the case that the universe is just sloughing off these immense numbers of different histories, most of which are completely irrelevant to us. So, you know, I guess my question here is, you're imagining that you're summarizing conscious experience. I mean, you know, you first, you started off by saying, look, conscious experience is this very rich thing that people can't reproduce from theories and so on. And so, what you're doing is you're flipping that around, as I understand it, and saying conscious experience is the axiomatic starting point. And then we're going to try and erect a theory around that starting point, which I think is a perfectly

reasonable thing to do. Okay, I don't have a problem with that. That's right. As long as I can make physical predictions that are testable inside space-time, right? Yeah, but I think, you know, the question is what goes into it, right? Because as soon as you're saying you've got these families of Markov chains and so on, you know, that's real content. That's not, you know, that's a model like I say, you know, the universe is made of hypergraphs and somebody else says, no, it's made of cream cheese or something. You know, you're positing something definite. The atoms of your ontology, so to speak, are these conscious experiences or whatever. I mean, you know, I find that so, by the way,

I mean, to either support or attack both of our points of view, you know, I can no more pick up an eem, one of our sort of atoms of existence and say, here it is, then I claim you can pick up that conscious experience and say, here it is. Right, right. So, both of us are in the situation where we have to say, look, the effects of what we're talking about are all very good, even though the thing we're ultimately talking about is not a thing we can pick up. Now, you know, to me, the problem, one of the things that's nice about Eames and hypographs and Rulliads and things like that is they're extremely non-human. So, we do not have sort of, we don't make the mistake of saying, oh, it's truth, it's falsity,

it's, you know, experience, it's this, that, and the other, because they are by construction, in a sense, they are deeply abstract and deeply non-human. So we don't come to it with a prejudice about how things should work. What worries me about starting from sort of consciousness as the element, so to speak, is that many, we think, we imagine, and in fact, even the way you're talking about the sensation of mint and so on, is we come with a bag of prejudices about how that all works. And so, it is a challenging thing to erect the science without being sort of pulled in the direction of some prejudice or another.

Fair enough. And I think that that's a very important point. And what I would say to anybody who wanted to do the research along the lines that I'm doing is to, I would say, the set of experiences that you've had is measure zero compared to the set of experiences that are out there. So don't make the silly mistake of taking your own experiences as comprehensive of all experiences. Really, in some sense, use your experiences to get going, but then follow the math. Don't follow your experiences. That's a very challenging thing to do. Living paradigms is, you know, I got to say,

in my life, for example, you know, I started studying simple computational systems, I don't know, 40, 45 years ago, basically. And you know, it took me embarrassingly long to realize things that were plainly observable in experiments I did. I mean, I, you know, just it happens to be the a few years ago, it was the 40th anniversary of my, not my discovery of this rule 37 automaton that does all kinds of cool, complicated things. I, it would be nice if I could say it was the discovery. It wasn't. It was the discovery of it was three years earlier. It took me three years to understand what the heck was going on and to not ignore it. And I think this is the, you know, it is a huge challenge to kind of rise above one's kind of, one's assumptions about what's going on.

And I mean, maybe one thing I could ask is- I'll just say that's a clue to what it means to be an observer. That it is hard to rise outside of one's previous impressions of things. Exactly. outside of one's previous impressions of things. So, a question would be, you know, observers like us, human observers, things like that, we have an internal experience of it, we have a way of projecting what human observers might be like. You know, when we go to observers with very different, human observers with very different

backgrounds, very different kind of belief systems, kind of ways of thinking about the world. You go, we're talking about the spirit world, animism, whatever else, or we're talking about all sorts of Eastern philosophy, ways of viewing the world. Even then, it can be difficult, I think, at least it has been for me, to wrap one's simple Western kind of scientific mind around these kinds of different ways of thinking about the world. That's right. That's right. I agree. I've faced the same thing. But one thing that trying to do that has, I've come to conclude is that, I love science, I love mathematics, I love concepts and being precise and everything, but I've concluded that reality, whatever it is, infinitely transcends anything we can describe.

And that's a very humbling, humbling thing. Pete Yeah, well, right. You know, I have to say I've had this experience now, you know, with the Ruliat and thinking of myself as this little tiny bundle of Eames in the Ruliat. I would like to be able to characterize what bundle of Eames is a thing like me versus what bundle of Eames is not an observer like me. I don't yet know how to do that. It will be interesting to understand, for example, and this is why I'm asking a little bit about do there have to be many observers, because, you know, for example, that gets you into, oh, you need kind of self-replication. You need some kind of, you need some way of replicating the number of observers. Do you

need the observers to be non-identical? Probably you do. If all the observers are in lockstep doing exactly the same thing, they're not very interesting observers. And one of the things, again, I sort of haven't seen coming, but I've now realized is relevant, is, you know, I happened to, well, I just recently did some things about sort of foundations of biological evolution, which surprised me a lot because I've thought about biological evolution off and on for four decades. And I'd always thought, you know, I'd always had a hard time coming up with sort of a minimal model for what was happening.

And I finally have this very minimal model with a cellular automaton with a few simple rules and you're asking, you know, the fitness is something like how long does the pattern live before it dies out? And what you find is that, you know, with that tiny genome, a very sort of small number of bits in the rule, it turns out you can evolve, you can adapt to produce these long lived things that are unbelievably complicated. When you say, what's the narrative scientific explanation of why the thing lives a long time, there really isn't one.

It's just the bits do what the bits do, and the answer is it lives for 10,000 steps or something. But one of the things I've been curious about is whether what it takes to make an observer does what it takes to make an observer relate to things that we are used to that are very routine to us, like the idea of life, the idea of sort of replicating multiple, similar but not identical copies of minds, things like this. Is that thing that is routine for observers specifically like us actually something that is sort of critically important in the notion of an observer like us? And, you know, as I say, the big

surprise for me has been the derivation of core laws of physics just from very coarse statements about observers like us. And as we get finer statements about observers like us, what more might we be able to derive? And you know, I'm sort of curious about whether, you know, for example, the thing that I find surprising is the existence of the Ruliat, I think, is inevitable. The existence of us as observers within the Ruliat is something that you have to derive. It's not self-evident.

In the abstract, it is not obvious from the existence of the Ruliat that there will ever be an observer like us. It's something that is presumably, in a sense, mathematically derivable. I don't know how to derive it. But that's the, you know, to simply say as an axiomatic matter, if there is an observer like us, then the observer like us will observe physics of the kind we observe. But the question is, can we derive from the very nature of the Ruliat that there must be observers like us? You know, that's something which I think would

be interesting. I think it will be doable. But then we can ask questions like, okay, there are observers like us. You know, for example, how common are observers like us? You talked about a set of measure zero of our ways of observing the universe. This relates to... We had an earthquake here. I thought that was an earthquake. We had a big earthquake here just now, but I'm good. We're getting closer to the truth.

That's the sign. Yes. These are earth-shaking ideas. Don, you're okay. Let's Um, these are shaking ideas. But Don, you're okay. Like, let's just make sure you're okay. Yeah, we're fine. And the people in your home. Yeah, they're fine.

I think our cats are probably scared, but that's, that's a different thing. All right. And now the question is, what is the cat's perception of the physical world? Well, our cat's perception. That's right. It was very different from mine. And right now they're probably under the bed hiding because there's something that just growled or just something really nasty to them.

Right. So one thing that would be nice to be able to derive is what is the density of observers like us in the Ruliat? Yes. By the way, we have the same problem in my framework, right? I'm saying that space-time is just one of an infinite number of headsets. So what we're perceiving as observers like we are is just one out of an infinity. And so I'm going to try to model this particular little headset and its properties and protons and so forth. But then once we do that and sort of establish that we can do that, then I want to look and say, there's an infinite number of other things to explore. What are the other headsets that

I can't even concretely imagine, but I can use mathematics to try to imagine them? Yeah. Don't go off to protons, go off to general relativity. You'll get to general relativity. You have a serious chance there. I think protons are hopeless, but just, just I'll give the sutta. No, but in any case, this point that you're making that, you know, in the Ruliyad, it is not difficult to kind of construct what an observer different from us would observe. And to give an example of that, one of the things in the sort of computational universe of all possible programs, one of the things that is a little bit of a different issue but related is there are programs that we know we care about.

And we're kind of, you know, there's a certain, like in mathematics, there are theorems we know we care about. There's an infinite space of all possible theorems, most of which we don't care about yet, at least. And if we look at the computational universe, there are certain rules that we might have used in technology or whatever else that we know we care about. And then there's an infinite set of other ones. One thing that's interesting about the computational universe, or for that matter, the Ruliat, which is closely related, is that it is very straightforward for us to do the experiment of just jumping any way we want in the computational universe. We just pick a program at random, start running it, see

what it does. Most of what it does is deeply alien to us. Exactly. Exactly. Pete And so, the question is, you know, in a sense, the view of what we're doing is we started from the place where we are on this earth with life as it is and so on, and we're gradually expanding, we're gradually colonizing more of what I would call rule-y-all space, kind of more of the space of possible paradigms and so on. We're gradually also sending out spacecraft that colonize physical space. But what we can do, which is very disorienting, is we can actually jump to random places in the Ruliat and see what's there.

But we don't have a connection. In other words, this notion, and I think it may be relevant to what you're doing as well, is to build up something which we can have a real experience of or something, I'm not sure if experience is the right word, we kind of have to go in steps. Like, first we understand this, we get familiar with that, then we go to this and so on. We're not able, if we're just throwing out there anywhere in real space, it's just totally disorienting. And I think, you know, it's kind of a, so. Yeah, you can't grok it.

There's a grokking thing and you can't grok it if you don't get there in the right way. Yes, yes, yes. And I mean, you know, it's like people say, are the AIs going to sort of discover, are they gonna jump sort of to science that we don't, and this is the same issue, that what is, the question of what is science? If science is the construction of narratives

that humans can understand about how the world works, it's not all that useful to have something, you know, it's a different problem to just say we can go out there and get to these things that are deeply not connected to humans. So, I'm curious in your view of things, if you're starting from consciousness as atoms, so to speak, to what extent, I mean, if you were just starting from consciousness as atoms, so to speak. To what extent, I mean, if you were to start from cat consciousness, would you build the same theory? In other words, if it was consciousness, or let me be more extreme, if you believe, and

maybe you don't, that the weather is in some sense conscious, then if you were to build your theory based on weather consciousness or cat consciousness or nematode consciousness, would you build the same theory or would you build a different theory? Well, I can tell you how we built this one. We said there's lots of things that you could do to talk about consciousness. There's lots of things. We picked only two. We said there are experiences and probabilistic relationships among experiences. And we said those are the only two things we're going to take. And the reason was I was, you know, Occam's razor. Basically, the fewer assumptions,

the better off you are. And so I decided to, I can't get a, if there are no conscious experiences, I can't do anything. And I need at least probabilistic relationships and let's just see if we can do it with that and nothing more. So I tried to get as general a theory with as few assumptions as possible. So the answer is as best as I can understand, I would say I would get the same theory of consciousness no matter where I started because I tried to get the minimal things that you could possibly have. But again, that may be just betraying my lack of being able to think outside of my little box.

Pete Well, I mean, you know, my guess is that there's a certain category of, of, yeah, I mean, you're erecting a theory based on, you know, calculus of observers. And it is, you know, it's a change of basis, so to speak, to think about a different kind of observer. Whether the theory you end up with after that change of basis looks the same is not, I don't know. And that's a question in part, it seems to me, the translation from one kind of consciousness to another, which, by the way, we have been singularly unsuccessful at achieving. I mean, I doubt you can have a philosophical discussion with your cat.

Right, right. Exactly. I completely agree. I completely agree. And so, I think that it's easy for me to think that I've got a general theory of consciousness and absolutely not. I can only, in some sense, have a theory of consciousness of the kind that I can grok. And what I can grok right now may be absolutely trivial compared to what's in the whole really outer, the whole space of conscious agents. But so, let's talk about AIs for a second, because the thing that you're doing, you know, in a sense, you could now, you know, you don't know, we don't know, we're all saying, you're saying nobody knows what

consciousness really is and so on. So, you're going to take it as an atom, you're going to take it as just the starting point for your theory. But in an AI, we can take it apart any way we want. We can't take about human brains, there are things we don't know, you know, are the microtubules important, you know, et cetera, et cetera, et cetera. There's a bunch of stuff we don't know. For, you know, your friendly LLM, we know every bit. So now the question would be, if you start having LLMs that can interact with each other, for example, and can have, you know, would we build, since you've said you expect that, you know, based on a cat consciousness or whatever else, you'd probably get the same kind of theory.

So my next question would be, let's take an LLM consciousness, which maybe, you know, maybe there's something wrong with it, maybe there isn't, but let's just take that as a basis. We can still talk about the relationships between LLMs. We can talk about kind of there, you know, you could talk about approximating what happens with LLMs using your Markov chains and so on. And now, but now the question is, now we've got a foundation, which is a foundation that sort of relates, it's a computational foundation. We're no longer having to say there's this mysterious thing that we're just taking as axiomatic, we've actually got something whose

axiomatization kind of goes all the way down to my kind of axiomatization, so to speak, or the computational foundation. So I guess the question would be, if you were to take a bunch of LLMs and you were to say, you were to make a model, as you have made a model of how, you know, consciousnesses like us interact, you could say, you could ask the question, if you do the experiment on LLMs, will they... Okay, so I mean, you've got an assumption about how consciousnesses like us interact, which is sort of...and you're saying you're going to make that experimentally testable by deducing from those interactions between consciousnesses what

the inferred space-time structure is. So now we could do the same thing with LLMs. We could say, you know, we take these LLMs, they're interacting a certain way. Could you erect from the observation of interactions between LLMs, sort of a structure of space-time. So for example, let's take a... And, you know, that's an interesting thing to imagine because if you actually think about a bunch of LLMs, they're probably on the internet.

And the internet doesn't live in, I mean, it is ultimately built in space-time, presumably, but the connectivity of the internet is not the structure of a three plus one dimensional space. Right, right, right. Exactly. So now the question would be if our conscious elements are AIs living on the internet interacting

by the rules of the internet, so to speak, which are a bit different from the rules that we I mean, I don't know whether they're in your model, whether they're different. The question would be those agents erecting their model of space-time, what is that model of space-time? Paul L Right. Well, and I would imagine within our framework that there is an infinite number of different space-times that could be in principle constructed. But by the way, the positive geometries that the high energy theoretical physicists have,

something like the amplitude hydron, it turns out that one of the parameters and it has parameters N, K, M, and Z. One of the parameters M for our space time is four. But their positive geometries allowed M to be any positive integer you want. So instead of a four dimensional space time, they can have positive geometries for them to be any positive integer you want. So instead of a four-dimensional space-time, they can have positive geometries for a billion-dimensional space-time.

So, in other words, already in the new structures that Neemar, Connie, Ahmed, and others have found beyond space-time, they're realizing that our space-time is just a parameter four, but there's a whole range of parameters that they've discovered are possible, and so other headsets are effectively possible. So my answer would be there's an infinite number of them and that's just in our first step out of space-time we're finding this. I presume there'll be even more dimensions of variation that we'll find. M equal four is just the first, right? So my assumption is that the reason we believe space is three plus one dimensional right now in the history of the universe is because of some aspect of us as observers. That's my

belief. I can't, you know, I haven't established that. This is dramatic. We're getting another, this is, yeah, this is earth shaking stuff. Well, are you in a place where there's some kind of warning if the fault is going, speed of light being faster than seismic waves and so on? Well, I'm in Southern California. We're used to earthquakes here. Okay, so this is not out of the ordinary. Well, this is unusual.

We've been having a few earthquakes in the last couple of days. So it's unusual. Yeah. So Don, how about I do a summary for yourselves and I'll tell you how I see the conversation so far and hopefully I do so in a straightforward manner. So it started off with you, Don, asking Steven, look, can you give me a scientific account of the taste of mint or the scent of garlic or whatever it may be and what is a theory of consciousness that has a scientific basis like go ahead Don go ahead Stephen try me do it and then Stephen's like okay I can't give you an answer to that because you have to know where you're going so you have to

define consciousness in order to get to it from some other place. In other words, there's an adage that says something like, if you don't know where you're going, you'll never get there. And then Don, you say, okay, well, Stephen, you have it backward. It's not that consciousness is this place you have to get to. It's rather what you know most intimately. It's where you start. And then this material world that you think is a fundamental notion is actually the derived one. And then Stephen says, okay, so fair enough. However, Don, you claim you

have a scientific account of consciousness. So how can you scientify this? I believe you use that word, Stephen. So how are you going to do that? And even worse than that, Don, if you take your intimate notions so seriously, then where are you getting this proliferation of consciousnesses from? When all you intimately know is this N equals one, but yet your theory has multiple consciousnesses. So then I believed on, you said something like, well, you could have an N equals one if you take it to be the totality of consciousness and we're instantiations. By the way, I think that's a really, we didn't pursue that particular point about the, you know, the uber-consciousness, so to speak, which feels like kind of the God theory. You

know, it feels like kind of the limit. You said there's no upper bound, so there is no, you know, it's kind of like, you know... But is there a lower bound? If you believe that the N of 1 story is that infinite limit, you are claiming you are God, basically. That's basically what you have to say, is that if there's an N of 1 and there's only one, because you only know that one thing, but you are also then that one thing, that unique thing, is this upper limit, this infinite limit of this whole sort of pile of progressively uber-uber consciousnesses.

Yeah, I'm willing to go there, but I'm taking you with me. But I'm saying that you and I are both God looking at the self, talking to the self through two different avatars or three different avatars. I think that limit thing is Basically your version of the rule yard. I mean, I think that that that's what you know, I think Anyway, that's which is kind of interesting. I mean, it's it's it's always good when you know when we can as I say It is for me. It has been the the the limit of many kinds of Parts of thinking okay, Kurt, sorry, we interrupted you. Good service, Mr. Forker.

This sounds like a foreign notion, but many people say all there is is the universe and these glasses, the cell phone, yourself, your eyeballs, they're expressions of the universe. So this is just the similar sentiment in different language. Is that correct? Pete I think we're going into a different direction here. But we're going to, we're going to go on for another hour. Kurt All right. Pete I think you gave a pretty good summary, Kurt. I mean, I think the only part that perhaps you left out is this, you know, these two different complementary ways of viewing the

world. Do you go from the Eames up or from the conscious observers down, so to speak? Yeah, and I was also going to say that, Don, you'd then talk about Markovian dynamics, giving rise to consciousness and Stephen believes that's, at least initially, that's too simplistic to reproduce the intricate experience that we have while caveating that Stephen, you know full well the power of rudimentary simple items giving rise to what looks convoluted and elaborate and as you helped pioneer computational emergence so you caveated with that and

then there was some some really add pushing of Stephen like a you're with a leather jacket at the back of a Mathematica conference saying like yeah You got to try some some some Rulliad. You got to take sniff of this, sniff of this causal way graph. You won't go back. Here's a multi-way. It's on the house. Right.

And then there was an earthquake and that's kind of the summary. And then the actual God intervened for the blasphemy that we engaged in over the past three hours. I really enjoyed this conversation and I would, I would welcome a chance to talk some more and explore this further. Yeah, very interesting stuff. And now I think I understand just a little bit about what you've, you know, I bought myself a copy of this book. Oh, oh yes. I know it's very old. Very, very old. It's very, very old. And I didn't read it yet.

So now maybe I probably have to look at the 30 years after version. John Wheeler cited that book and is it from bit paper? Ah, that's interesting. Unfortunately, I met John Wheeler only once. I mean, I exchanged letters with him a bunch of times, but I met him only once when he was 95 years old. It's kind of a sad story because I'm talking to him about a bunch of things and he looks up and he says, you know who you should talk to about all this stuff? It's a chap over at the Institute. His name is John von Neumann. Oh, wow.

And I said, unfortunately, he died before I was born. Oh boy. Yeah. That's, that's sad. Yeah. Yeah. Anyway, not, not to end on a down note. That's right.

Well, thank you all for spending three hours with myself and with the audience. Yes. The audience that will eventually see this and take care. Yeah. Great. Firstly, the audience that will eventually see this and take care. Yeah, it was a good, great pleasure. Firstly, thank you for watching. Thank you for listening. There's now a website, curtjymongle.org and that has a mailing list. The reason being that large platforms like YouTube, like Patreon, they can disable you for whatever reason, whenever they like. That's just part of the terms of service. Now a direct mailing list ensures that I have an untrammeled communication with you.

Plus, soon I'll be releasing a one-page PDF of my top 10 toes. It's not as Quentin Tarantino as it sounds like. Secondly, if you haven't subscribed or clicked that like button, now is the time to do so. Why? Because each subscribe, each like helps YouTube push this content to more people like yourself, plus it helps out Kurt directly, aka me. I also found out last year that external links count plenty toward the algorithm, which means that whenever you share on Twitter, say on Facebook or even on Reddit, etc., it shows

YouTube, hey, people are talking about this content outside of YouTube, which in turn greatly aids the distribution on YouTube. Thirdly, there's a remarkably active Discord and subreddit for theories of everything, where people explicate toes, they disagree respectfully about theories, and build as a community our own toe. Links to both are in the description. Fourthly, you should know this podcast is on iTunes, it's on Spotify, it's on all of the audio platforms. All you have to do is type in theories of everything and you'll find it. Personally, I gain from rewatching lectures and podcasts. I also read in the

comments that hey, TOW listeners also gain from replaying. So how about instead you re-listen on those platforms like iTunes, Spotify, Google Podcasts, whichever podcast catcher you use. And finally, if you'd like to support more conversations like this, more content like this, then do consider visiting patreon.com slash Kurtjmungle and donating with whatever you like. There's also PayPal, there's also crypto, there's also just joining on YouTube. Again, keep in mind, it's support from the sponsors and you that allow me to work on

toe full time. You also get early access to ad free episodes, whether it's audio or video. It's audio in the case of Patreon, video in the case of YouTube. For instance, this episode that you're listening to right now was released a few days earlier. Every dollar helps far more than you think. Either way, your viewership is generosity enough. Thank you so much.