Theories of Everything with Curt Jaimungal - The Mathematical Accident That Changes Everything

Episode Date: July 1, 2025

Special offer! Get 20% off discount to The Economist and all it has to offer! Visit https://www.economist.com/toe Harvard physicist Jacob Barandes returns with a groundbreaking insight that could reshape quantum theory. By questioning a single hidden assumption, Jacob bridges the gap between classical probability and quantum mechanics. This ‘mathematical accident’ challenges the foundations of Bell’s Theorem, dissolves the measurement problem, and opens a path to a realist interpretation of quantum physics. This episode is a rigorous journey through stochastic processes, non-locality, and the future of theoretical physics. Join My New Substack (Personal Writings): https://curtjaimungal.substack.com Listen on Spotify: https://open.spotify.com/show/4gL14b92xAErofYQA7bU4e Timestamps: 00:00 Introduction 1:01:46 Teaching Black Holes to Graduate Students 1:04:59 Coordinate Systems in Space-Time 1:06:58 Teaching Black Hole Coordinates 1:10:11 Insights from Nima 1:13:41 Nima's Course on Quantum Mechanics 1:16:22 Quantum Foundations and Cosmology 1:18:48 Transitioning to Quantum Gravity 1:23:10 Philosophy's Role in Physics 1:26:10 Leaving String Theory 1:33:39 Interpretations of Quantum Mechanics 1:37:02 Challenges of String Theory 1:42:49 Quantum Field Theory Insights 1:50:30 Foundations of Quantum Field Theory 1:53:47 Particle Existence Between Measurements 1:59:44 Speculations on Quantum Gravity 2:01:41 Legacy and Contributions Links Mentioned: •⁠ ⁠Press release of the 2022 Nobel Prize in Physics: https://www.nobelprize.org/uploads/2022/10/press-physicsprize2022-2.pdf •⁠ ⁠Eddy Chen & Barry Loewer on TOE: https://youtu.be/xZnafO__IZ0 •⁠ ⁠Jacob Barandes on TOE (part 1): https://youtu.be/7oWip00iXbo •⁠ ⁠Tim Maudlin on TOE: https://youtu.be/fU1bs5o3nss •⁠ ⁠What Is Real? (book): https://www.amazon.com/What-Real-Unfinished-Meaning-Quantum/dp/0465096050 •⁠ ⁠David Wallace on TOE: https://youtu.be/4MjNuJK5RzM •⁠ ⁠The Copenhagen Interpretation: https://plato.stanford.edu/entries/qm-copenhagen/ •⁠ ⁠Bohmian Mechanics: https://plato.stanford.edu/entries/qm-bohm/ •⁠ ⁠Everettian Quantum Mechanics: https://plato.stanford.edu/entries/qm-everett/ •⁠ ⁠Jacob Barandes on TOE (part 2): https://youtu.be/YaS1usLeXQM •⁠ ⁠Jacob Barandes on TOE (part 3): https://youtu.be/wrUvtqr4wOs •⁠ ⁠The sky is blue (paper): https://arxiv.org/pdf/2205.00568 •⁠ ⁠The Emergent Universe (book): https://www.amazon.com/Emergent-Multiverse-Quantum-according-Interpretation/dp/0198707541 •⁠ ⁠Complex Coordinates and Quantum Mechanics (paper): https://journals.aps.org/rmp/abstract/10.1103/RevModPhys.38.36 •⁠ ⁠Kurt Vonnegut’s lecture: https://youtu.be/4_RUgnC1lm8 •⁠ ⁠Max Born’s memoir: https://archive.org/details/myliferecollecti0000born/page/n5/mode/2up •⁠ ⁠Hugh Everett’s unpublished dissertation: https://ia801909.us.archive.org/20/items/TheTheoryOfTheUniversalWaveFunction/The%20Theory%20of%20the%20Universal%20Wave%20Function.pdf •⁠ ⁠La nouvelle cuisine (paper): https://www.cambridge.org/core/books/abs/speakable-and-unspeakable-in-quantum-mechanics/la-nouvelle-cuisine/6FFC85D84585D9C41AA4A1185BF5290E •⁠ ⁠The Great Rift in Physics (paper): https://arxiv.org/pdf/2503.20067 •⁠ ⁠Quantum stochastic processes (paper): https://arxiv.org/pdf/2012.01894 •⁠ ⁠Bell’s Theorem: https://plato.stanford.edu/entries/bell-theorem/ •⁠ ⁠Neil Turok on TOE: https://youtu.be/zNZCa1pVE20 ***For full resources please visit https://curtjaimungal.org SUPPORT: - Become a YouTube Member (Early Access Videos): https://www.youtube.com/channel/UCdWIQh9DGG6uhJk8eyIFl1w/join - Support me on Patreon: https://patreon.com/curtjaimungal - Support me on Crypto: https://commerce.coinbase.com/checkout/de803625-87d3-4300-ab6d-85d4258834a9 - Support me on PayPal: https://www.paypal.com/donate?hosted_button_id=XUBHNMFXUX5S4 SOCIALS: - Twitter: https://twitter.com/TOEwithCurt - Discord Invite: https://discord.com/invite/kBcnfNVwqs #science Learn more about your ad choices. Visit megaphone.fm/adchoices

I don't think it's coherent. I don't think that answers the questions of what's going on in quantum foundations. Harvard's Jacob Arendez exposes the hidden assumption that has blinded us for nearly a century. One day, while preparing to teach quantum mechanics, Jacob made a startling discovery. Quantum theory and classical probability are separated by a single unnoticed presumption, Markovianity. Drop it and the gap between them vanishes entirely. Even more startling, this mathematical accident bypasses the so-called local realism constraint of Bell's theorem and dissolves the measurement problem, all without magic. This discussion ranges from black holes that reign to the mathematical beauty of indivisible processes culminating in a meditation on legacy, kindness, and why the next revolution in physics

demands philosophers and physicists working as one. Jacob, welcome back. It's always a pleasure to talk with you, Kurt. It's always a pleasure to have you, to host you. Thank you so much. Lovely being here in Toronto. It's a lovely city. And if you haven't been to Toronto, you should come visit.

It's great. Is Toronto real? Here, how about that? Is Toronto non-locally real or locally real? Firstly, let's focus on this term real. Has the 2022 Nobel Prize disproved so-called local realism? If you read the press release from the 2022 Nobel Prize in Physics that was given to Aspe, Clauser and Zeilinger for their work testing experimentally violations of a generalized set of equalities that began with the Bell inequality and evolved into things like the CHSH inequality.

If you read the press release for that Nobel Prize, it says that what they accomplished was proving that there cannot be hidden variables. That's not strictly correct. It's certainly not what Bell argued. There is a somewhat narrower claim about what they demonstrated with their experimental work. And that was to show that local realism is false. So we have to talk about what local realism means. Local realism is the statement that objects localized in

space have definite properties, have in some real sense, they possess some kind of definite way that they are, that is independent of measurements. And that somehow interact with measuring devices to yield results, maybe the measuring devices passively reveal those pre-existing properties, maybe there's a more subtle interplay. But the objects have something like real definite properties localized where they are in space. Just to be specific, you said real definite properties.

Can we just drop real from that? Can we just say that the particles have definite properties? Sure, sure. I was using real there more for emphasis. Real, definite, just to make very, yeah. The question about what do we mean something distinct by real from definite. I mean, you could try to parse those.

I don't have a definition of what real means in a metaphysical or physical sense have a definition of what real means in a metaphysical or physical sense that doesn't involve words that essentially mean the same thing as real. To say something is real is to say that it exists. But what does exist mean? To say that it is real, really there. I mean, it's, so this is again, one of those,

you know, we've had conversations before about how some very primordial primitive notions can be very difficult to define rigorously in a way that isn't logically circular. So for these kinds of notions, and these include things like reality and existence, they include things like if you really try to pin it down Exactly what we mean by probability

Exactly what we mean by consciousness You know, they're all of these very difficult questions and metaphysics We think we're talking about something when we talk about them, but they are very difficult to define free will is another good example very difficult things to define. So we'll have to take for granted for now, because we can't solve every problem in philosophy today, that we have some prior notion of what it means for things to be real, that they exist.

So we assume, according to local realism, that objects localized in space in some sense have real or definite or pre-existing properties or a way that they are an ontology, a physical state of being that is separate from what we see when we measure those objects. There could be some subtle interplay between that state of being of the object and measurement, but it can't just be that they have no state of being except through some kind of measurement process according to local realism, if that's the view that one takes. I know Tim Maudlin quipped about people's obsession with saying that there's no realism,

but yet they like locality. He would say, oh, it's not real, but thank God it's local. Right, yeah, yeah. And I think the author Adam Becker, who wrote a book, What is Real, about the historical development of quantum theory, I think he also put it really neatly. wrote a book, What is Real, about the historical development of quantum theory. I think he also put it really neatly. He said, without realism, there's nothing to be local about.

So the argument some people have made is, and this term local realism, I should say that not everybody who works in philosophy and foundations of physics, foundations of quantum mechanics, particularly likes that term local realism. But of course, one view is maybe we can detach the two pieces and save locality, but give up realism. Or maybe what we have to do is hang on to realism and give up locality. I have to say I'm strongly sympathetic to those

who would say that without the realism part, there's nothing to be local about. I mean, presumably, in some sense, you and I exist. If I'm not going to be solipsistic, then if I accept that I exist, I'm going to want to accept that you also exist, that others exist. I appreciate that. Yes, yes, you're very welcome.

You're very welcome. It's your finest quality. Thank you. Thank you. And if science is to make any sense at all, if even the textbook version of quantum theory doesn't make any sense at all, then measurement outcomes have to exist in some sense. If there are no measurement outcomes at all in any sense, according to anybody, measurement outcomes, we can talk about different interpretations of quantum theory and what it means to have

measurement outcomes. Many people who are watching this will know that, you know, on the many worlds theory, there are many measurement outcomes. There are also relational or perspectival interpretations or theories for quantum theory in which measurement outcomes are relational or depend on one's perspective. But if there are no measurement outcomes of any kind, in any sense, if those just don't exist, then it's a completely self-undermining picture of reality. That is, that sort of picture of reality undoes itself.

It means that without measurement outcomes, we can't even talk about how we do science or how we gather empirical knowledge of the world. And without empirical knowledge of the world, there's no way to account for what we see. There's no way to ground our physical theories. So I think some kind of realism about something is unavoidable if science, narrowly or physical experience more broadly, is to be coherent.

And if you think that there is realism about people and realism about measuring devices or measurement outcomes at least, then you can ask the question of where the line is or why only measurement outcomes? Is is. Why only measurement outcomes? Is there something special about measurement outcomes? Is there something special about people? What about our individual cells?

Do our cells get to have realism also? The cells that make up our bodies, the, you know, subcomponents that make up our measuring devices. And if you want to claim that our macroscopic, macroscopic meaning that the big world, the world that people walk around in exists in some sense, and you want to argue that this world is emergent in some way from some deeper level of reality, some deeper physical substrate, then it's very hard to avoid some commitment to the reality of that physical substrate.

I mean, you can't have emergence without a substrate out of which the emergence is supposed to happen unless you can provide a rigorous argument for how we're supposed to do that. And I have not seen such a rigorous argument. So it seems hard to deny some kind of realism to something going on in nature, even at a fairly low level. Exactly where we have to stop, if we have to stop at all, is an interesting question. So I just don't know how to make sense of the idea of giving up realism. I don't even know what that means.

I don't think it's coherent. I think people sometimes will say it and then they'll move on without maybe following the consequences of giving up realism to their logical conclusions. I think when you do that, you run into some pretty deep problems of self-undermining. So the question is, does Bell's Theorem rule out local realism to begin with? And this is where we can start to talk about some of the implicit assumptions,

some explicit, some implicit assumptions that lead to Bell's results. You will sometimes see writings of prominent physicists, sometimes in the popular press, sometimes in articles in scientific journals that are intended for people to see, in which they sometimes wax poetic about how quantum theory has transformed our view of reality, how pre-20th century physics was very different from the physics of the 20th century and beyond.

And what we've learned from quantum theory is that nature is subtle and mysterious. I grant all those things, nature is subtle and mysterious. But that we have to give up things like realism, that what it's taught us is that we can't be realists about nature, that we have to view all of physics in a purely operational or instrumental sense. It's merely about the operations we can perform. Things are defined in terms of the operations we can do to measure them or interact with them or work with them. And that scientific theories are only instrumentalist in the sense that they're just models or mathematical vehicles for relating preparations of measurement setups to the outcomes of measurements.

I don't view this as some kind of progress of science. It's a particular philosophical metaphysical viewpoint toward what scientific theories are supposed to be like. And I object and reject the idea that this is somehow the lesson that modern physics, including quantum theory has taught us. It's true that on the assumption that quantum theory is a good, successful physical theory, as abundant experimental evidence strongly suggests,

that our understanding our picture of reality does need to be modified compared with the physics prior to the beginning of the 20th century. But that just means that there's work to be done, work to be done by philosophers and by physicists to try to understand that world rather than give up and embrace instrumentalism or operationalism. I think that instrumentalism and operationalism are perfectly fine, practical approaches to making use of our physical theories. And until we find an acceptable picture of what's going on, I think they're perfectly reasonable temporary places to be,

but I don't think they're the end goal. So how do we avoid speculation then? If we're not going to ground ourselves with what's operational or measurable? So I have an aversion to speculation. This is something that we've talked about. I think that the success of quantum theory, quantum theory is a very intricate theory. It's got a list of axioms, which we've talked about. It makes a huge number of very non-trivial predictions that have been confirmed to incredible accuracy in experiments over many decades. Oh, and I should qualify what I said as not just speculation,

because speculation could just be idea generation, which is what one should do in a meeting or in a room with yourself ideating. So I mean to say more wild generation of theories and then speculation, a top speculation, a top speculation. Yeah. So it's a great point. Yes. Idea generation is great, but what I don't typically like is wild speculation that doesn't eventually ground itself in some kind of rigorous scrutinizing. Fortunately, quantum theory is such a rich, intricate theory that it places a lot of constraints on the kinds of world pictures that you might try to write down. Now, of course, when you try to create any physical theory or try to put together any

world picture of the theory, you're gonna be guided by some extra empirical criteria as well. You don't wanna multiply assumptions beyond what's necessary in some vague sense, that's Occam's razor. There's, you know, given the choice between different kinds of world pictures

or physical theories or what have you, all else equal, you'll want to use pictures that are more perspicuous, clearer, simpler. There's a kind of elegance that we're often looking for. But above all, you want to meet certain criteria that I think are just deal breakers. One is your world picture has to be consistent with the empirical predictions of the physical theory. It's gotta get the experimental predictions right. We call that empirical adequacy.

That is an absolute must for any world picture or interpretation or formulation that you're looking for. You want this interpretation or picture not to yield ambiguous statements about things that in principle, maybe in practice difficult, but in principle you can go out and look at. Ideally you want to understand how things like our macroscopic everyday world,

at least in schematic terms, can be seen to arise from the world picture that you're proposing. So as a counterexample, the Copenhagen interpretation just takes for granted there's a classical world. And going back, I mean, many people have said this more eloquently than I have, including Hugh Everett, who gave rise to the Many Worlds interpretation.

All the way back to his doctoral work in 56 and 57, he complained that the Copenhagen interpretation made it impossible to understand how you could, there's any way to understand how the classical world emerged from a deeper level of reality because it simply took the classical world for granted. And I agree with that critique. So understanding, at least in schematic terms, how our macroscopic everyday reality is supposed to emerge from that world picture.

And finally, you should avoid the need for a very long list of epicyclical additional speculative metaphysical hypotheses and ad hoc assumptions that should have gone on forever. I think these are deal-breaking conditions and given the success and intricacy of quantum theory and its predictions and given those criteria, most things you would propose just don't work. They don't meet those criteria and so you might worry as I spent a lot of time worrying, that maybe we don't have a case of underdetermination, that quantum theory underdetermines its possible realist interpretations,

but that quantum theory overdetermines them, that there simply aren't any sensible, realist oriented formulations or interpretations, world pictures, things the world could be like, such that the theory to describe and make predictions about the world is quantum theory. That was a concern that I had for a very long time. There are other interpretive approaches to quantum theory. We've talked about these and many people who are watching this probably have heard of these or know something about them. The Copenhagen interpretation we already mentioned. There's the strict textbook, don't ask questions, shut up and calculate approach that just goes directly from Drac and von Neumann's axioms.

There's the de Broglie-Bohm pilot wave theory or Bohmian mechanics. There's the Everettian many worlds type picture and arguably there's more than just one. Some people who work on this would say there's only one, but people formalize this story somewhat axiomatically differently. So arguably there's more than one such picture and there's more. I mean, there's a whole... There's this handsome guy who came up with indivisible stochastic processes. I don't recall his name though. Yeah. I know that guy.

I think I do. I think I know that guy. But does any of us really know ourselves? That's an interesting question. You have all these pictures, but what motivated me to work on, and we're talking about this indivisible stochastic approach, was I didn't think that any of the previously existing approaches met those requirements. And we can go through and talk about why I don't think they meet those requirements.

There's a sense in which the textbook approach and Copenhagen meet the empirical adequacy requirement by construction. They're just phrased, they're just constructed around, you know, the measurements, which they're just taken to be, you know, to be primitive axiomatic features of their descriptions. And so they're, of course, they're gonna be empirically adequate, you know, for most everyday purposes. But they do lead to some ambiguities.

And then we run into the ambiguity problem. The Wigner's Friend Thought experiment, which you and I have talked about a couple of times, is an example of where you run into some ambiguity. You have two observers, one inside of a sealed box, one outside of the box, and then there's this question, do both observers agree that the one in the box conducted a measurement and we should apply the measurement axioms, did not conduct a measurement, it should be modeled using the non-measurement axioms. And this is just an ambiguity that shows up in these circumstances.

Now when other physical theories break down, like general relativity breaks down at singularities, Newtonian mechanics has places where it has singularities, maybe we'll talk about some of those, electromagnetism has singularities, maybe we'll talk about some of those, auto-magnetism has singularities. We just accept that these physical theories are incomplete in some sense, they're effective descriptions. It's not the end of the world. It's interesting that there was so much resistance in the history of the development of quantum theory to accepting that quantum theory also can have some limitations according to the

textbook or Copenhagen approaches, which are subtly different. Speaking about the Copenhagen interpretation, are there modern physicists who adopt that view? Just a moment. Don't go anywhere. Hey, I see you inching away. Don't be like the economy. Instead, read The Economist.

I thought all The Economist was was something that CEOs read to stay up to date on world trends. And that's true But that's not only true what I found more than useful for myself personally is their coverage of math physics philosophy and AI Especially how something is perceived by other countries and how it may impact markets for instance The economist had an interview with some of the people behind DeepSeek the week DeepSeek was launched. No one else had that. Another example is the Economist has this fantastic article on the recent dark energy data which surpasses even scientific Americans coverage, in my opinion.

They also have the chart of everything. It's like the chart version of this channel. It's something which is a pleasure to scroll through and learn from. Links to all of these will be in the description, of course. Now, the Economist's commitment to rigorous journalism means that you get a clear picture of the world's most significant developments. I am personally interested in the more scientific ones, like this one on extending life via mitochondrial transplants,

which creates actually a new field of medicine, something that would make Michael Levin proud. The Economist also covers culture, finance and economics, business, international affairs, Britain, Europe, the Middle East, Africa, China, Asia, the Americas, and of course, the USA. Whether it's the latest in scientific innovation or the shifting landscape of global politics, The Economist provides comprehensive coverage, and it goes far beyond just headlines. Look, if you're passionate about expanding your knowledge and gaining a new understanding,

a deeper one of the forces that shape our world, then I highly recommend subscribing to The Economist. I subscribe to them and it's an investment into my, into your intellectual growth. It's one that you won't regret. As a listener of this podcast, you'll get a special 20% off discount. Now you can enjoy The Economist and all it has to offer for less. Head over to their website, www.economist.com slash toe, T-O-E, to get started. Thanks for tuning in.

And now let's get back to the exploration of the mysteries of our universe. Again, that's economist.com slash toe. Speaking about the Copenhagen interpretation, are there modern physicists who adopt that view? So I give a survey out to incoming graduate students and I ask them their views on quantum mechanics, is the measurement problem a serious problem? What's their preferred interpretation? The last time I gave out this interview was for the students coming into the physics PhD program in 2024. I got all of them to

answer it and about half of them put either the Orthodox or textbook approach or the Copenhagen approach and then the other half picked various other approaches. A pretty big chunk regarded the measurement problem as either a major or minor ongoing problem. So it's very interesting to get a snapshot of graduate students. When it comes to practicing physicists, senior physicists, it depends on whom you ask. People have a lot of different views. In 2019, Harvard hosted Anton Zeilinger, who is the Zeilinger who shared the Nobel Prize, the Physics Nobel Prize in 2022 with with Alana Spey and John Clouser for his work testing and finding experimentally violations by quantum theory of

certain inequalities that are related to the Bell Inequality. Harvard has an annual Lee historical lecture when we invite a physicist who has been witness to many important and great events in the historical development of physics and who's contributed to them, great events in the historical development of physics and who's contributed to them, talks about their life, their contributions, what they've seen, and their broader views at this point in their career. In 2019, Harvard invited Anton Zeilinger to be the historical lecturer. This I believe was recorded, and if there is a recording, Kurt, I can send you a link to it and people can watch it.

It's amazing to see Ze I linger talk about his work and his career. So that would be a really interesting thing to share. Toward the end of that lecture, I'm talking like an hour and nine minutes in something like that. He talks about his own interpretational views about quantum theory.

And he very explicitly embraces the Copenhagen interpretation. We have a classical reality. All statements about measurement preparations and results and measuring devices and people are all phrased in classical language. And the role of the quantum state is that it is a mathematical tool for relating our classical measurement preparations. Classical in the sense that the measurement preparations are prepared by classical measuring devices to study quantum things. The quantum state encodes and represents mathematically those classical devices that prepare our measurements and represents predictions about the results that show up on those classical devices.

And the collapse of the quantum state is not some physical process, but is merely a change in representation because classical observers or classical devices are updating their information and that there's therefore no measurement problem at all. So these sorts of views exist. I'm not painting straw men when I talk about people

adhering to the Copenhagen approach, including people who know a lot about quantum mechanics and have done incredibly important work in their careers on quantum mechanics. I have the utmost respect for people, like certainly Anton Zylinger, I mean, he's a Nobel Prize winner, and has done absolutely central work

in our understanding of quantum foundations. But this is a point at which, very respectfully, I and many others disagree. Bohmian mechanics has proved to be very difficult to generalize beyond systems of fixed numbers of finitely many non-relativistic particles. There have been attempts to generalize them. There are amazing people thinking and working on these things.

And we've talked about some of them, Shelley Goldstein, Ward-Streuva and many others. I mean, I could list everybody over the years, Detlef Dürr, Nino Zanghi, Roddy Tomolka. I won't leave anybody out, but a lot of people have worked on these approaches. And trying to generalize to the fully relativistic case, trying to generalize to be able to accommodate quantum field theories, in particular, fermionic field theories, interacting field theories,

the kinds of field theories that we see in the standard model has proved to be very, very difficult. And if Bohmian mechanics can't achieve an ability to describe those real world models that work so well, the standard model is the most well-tested physical theory that we've ever had, then these Bohm type theories simply don't achieve empirical adequacy.

And this is something that David Wallace, who's a strong proponent of the Everett many worlds approach, has argued in a paper and in a series of talks that are centered around why the sky is blue, Rayleigh scattering. Rayleigh scattering is the reason why the sky is blue. This is a relativistic problem and it's one that Bohmian mechanics seems to have a lot of difficulty handling and his argument is if it can't handle a question like why this guy is blue then this is a clear sign of empirical inadequacy. Now that could be fixed up. It's possible that

people will develop Bohmian mechanics to a point at which we'll be able to accommodate modern realistic field theories. But until we reach that point, the theory doesn't achieve empirical adequacy. My problems with the Everett approach are, and people have a lot of views about this, a lot of work is still being done on this. But I'm just still not convinced that there's

any way to get probability out of this deterministic picture. We have all these branches, and on the branches we have lots of copies of observers. Some are rational by some definition, some are irrational, but there's no connection between whether they're rational or irrational and what happens to them because in the many worlds ontology everything happens. So we can't somehow argue that observers ought to be rational and ought in some subjective decision theoretic way, assigned probabilities according to any given rule. There have been a lot of arguments to try to get that off the ground, going back to

the work by David Deutsch in 1999 and then David Wallace in his 2012 book. The proofs are very intricate. They get longer and longer. But I think you can't ultimately overcome this problem. And there's more work. I mean, Simon Saunders is trying to bring back branch counting using coarse graining and appeals to analogies with statistical mechanics. And there are other approaches also, but none of them appear to achieve the aim of getting

probability to show up. Arguably all the arguments, if you carefully look at them, are circular or involve a very long list of speculative metaphysical hypotheses when you look at the fine print. And if you can't get probabilities out, then again you're not achieving the empirical adequacy requirement. And I know there's some people who might say something like, well, you know, can't we just impose a measure on all the branches? Just impose a probability measure and declare them to be probabilities. The problem is that imposing a measure is something you would do in the axioms.

You'd have to add an axiom, but in modern approaches to Everetti and quantum theory, the branches to which you would presumably be imposing these probabilities are emergent. They're approximate. They're not fundamental in the axioms. And so yes, they can arise. You can get branches.

Axioms describing fundamental ingredients can produce contingent. Contingent means not necessary, but things that could show up. Tables, chairs, in this case, branches, these can show up later on, but you can't assign properties from the axioms to emergent contingent derivable things. If I have a theory of chemistry, I can assign properties to my atoms.

I cannot, after chairs emerge, axiomatically assign properties to chairs and declare chairs must all be yellow, for example. Right? And if you, if the branches are supposed to be emergent things, then we cannot assign them a measure or probability, a probability measure after the fact in our axioms. We have to somehow get the probabilities out without assuming probabilities at the beginning. And then there's just no deductive argument that's gonna get you to the probabilities.

Anyway, this is a long-range debate that people have been having. People are arguing about this still, but I'm just not convinced we can ultimately get this to work. This problem of probability, I think, is really a fundamental obstruction. And then separately, the Everett approach

to get these arguments off the ground in the first place entail lots of additional assumptions beyond the core assumptions that you just have a Hilbert space and a state vector. I mean, even some very basic ones like, it seems intuitive that when a branch has a zero in front of it, because when you expand the universal wave function or state vector as branches, they've got numbers in front of them.

These are the numbers that you want to somehow argue should be squared to get probabilities. Prior to assigning that probabilistic understanding of those numbers, it seems intuitive that if any of them have a zero in front of them, then they don't exist. But even that's actually not totally obvious. Yeah, we talked about this. Yeah, we talked about this. Why would a zero in front of something? There's a way to think about the evolving wave function as a system of classical harmonic oscillators just by changing a representation. This is work by Stroci and Heslot, Strowci in the 60s and Heslot in the 80s.

And from that standpoint, having a zero in front is just like having an oscillator that's not oscillating, but that doesn't mean the oscillator doesn't exist. So there's a lot of these extra assumptions you have to carefully add. I don't know how to justify all of these assumptions. There's just too many of them. It's not that I have a problem with having any metaphysical assumptions. We need some metaphysics to get off, you know, to get out of bed every morning. But once you have that many, it each one reduces the credence,

the credibility, my belief in the success of the picture. So I actually think that the constraints we have from the empirical success of quantum theory and just this short list of deal breaking conditions put so many constraints on candidate world pictures, interpretations, formulations, theories that are supposed to stand in and give us the world picture we need for quantum theory that almost anything that you guess is gonna be wrong.

You'll be able to show that it runs afoul of one of these conditions. So I actually think this is exactly the kind of circumstance in which we can make progress by maybe some initial speculating, but then carefully checking that things work together. And this is what led me to the work that I'm currently working on. I think that this approach I'm working on, this indivisible, sarcastic approach, meets those requirements. Now I don't know for a fact

this is the unique approach that will work. For me, this is just an existence proof of proof of principle. And maybe some of the ideas that went into this indivisible approach will inspire other approaches that will also do a better job of meeting those conditions. So I think progress actually can be made here.

I think we are exactly the kind of situation in which we have the kinds of constraints on us that can lead to progress. If it turns out that this indivisible stochastic approach is not unique, well, then we're back to having an under-determination problem. But under-determination of scientific theories or interpretations given data, I guess under-determination of theory given data and underdetermination of interpretation given theory, these are old problems. These are problems that go back to the beginning of science. And if we can just revert back to those problems, I would still feel that we've made some progress.

So when we first spoke, we talked extensively about Indivisible Stochastic Processes. And I will place a link on screen and in the description of part one, part two, part three with Jacob. And in the first part, if I recall correctly, you described a cliff, two cliffs, and it was as if they had merged and there was no longer a gap between them. Can you tell me about that moment when you were developing Indivisible Stochastic, the framework? What that looked like? What time of day was it, where were you, what did you feel, what were you thinking? Well, I was sitting at my desk and I was typing on my computer. And I was trying to,

so I needed to teach a class, and we talked a little bit about this, to students who I was not assuming had any prior exposure to quantum theory, but also complex numbers, linear algebra. I wasn't assuming they'd seen any of these ingredients, but they had seen some probability. And so I felt like I could talk about

the theory of stochastic processes. So let me give a little bit of historical background that I don't think we've talked about before. Please. So when I was in college, I spent a summer at Fermilab, which is an experimental physics laboratory in the United States in Illinois.

I learned a couple of important things from this experience. One was how to drive on highways because there were a lot of highways like we would pull out of where we were sleeping, you know, the place, the rental places where we were all staying, you'd pull right out onto a four-lane highway. So it was a real chance to get a lot of driving experience. That was one thing I learned. Another thing was the sky is really huge in Illinois. Like, you know, as someone who spent a lot of my time growing up in New York City, and then in the suburbs of New York City, lots of trees, you don't see a lot of the

sky all the time. Where we were staying in Illinois, the sky was enormous. That was a really extraordinary thing. I also learned that I was not destined to be an experimental particle physicist. So that was an important thing to learn. Because? Wasn't particularly good with my hands. Ultimately, I think I never managed to use a soldering iron. But, you know, I would go to meetings and we'd talk about getting the experiments, you know, what was the latest going with experiments. And just it it it just wasn't what I felt most interested in

I had enormous respect for people who did that work It was incredibly interesting and what they were able to make matter and energy do was extraordinary It just didn't have the right I just have the right connection to it I found myself thinking a lot more about theoretical and philosophical questions So I learned that also. And the other thing was I decided to learn linear algebra. I wanted to get ahead. And so I found a curriculum that was being taught at my college curriculum.

And I saw the book they were using. I found some homework exercises. And I had a summer. And I was away at Fermilab, so I decided to spend the evenings learning linear algebra. When I finished doing all the homeworks, I went to a professor, Dave Beier, really nice guy, really amazing guy, mathematician. And I told him I had done all this work

and I was wondering if he would grade it for me, if I could get some kind of letter grade. I figured after all this work, maybe I can get a letter grade and put it on my transcript. He said, okay, but only if I did a final exam, a take-home exam. I said, okay. So he gave me a take-home exam and I did the exam and I did okay. I did okay on the exam. But when he sent it back to me, I was a little bit disappointed in my final grade and I

felt a little bad because I put so much work into the course. So I asked him, okay, on the exam. But when he sent it back to me, I was a little bit disappointed in my final grade and I felt a little bad because I put so much work into the course. And so I asked him, you know, okay, so I got this final grade, that's fine. You know, but I'm wondering, is there anything I can do to boost the grade? Could I do like an extra project or something like that? He says, actually, yeah, you can do an extra project if you want maybe to boost your grade. This project is on stochastic processes. I had never heard of stochastic processes before. Back in high school, I had played around with probabilistic simulations for various things, but I'd never done anything like a formal study of probabilistic theories.

So this was my first opportunity to see this, and I think what's interesting about this is if I had done better on this exam, I never would have had this encounter, right? So, you know, this is, I think, a general lesson. Sometimes you have an experience and you feel like it's bad news, and it turns out actually might be really good news. Here I'm quoting Kurt Vonnegut. And those of you who don't know, Kurt Vonnegut has this famous lecture he used to give on good news and bad news.

And we never know what the good news or bad news is at the time. Sometimes we only find out the answer much later. And he's got a lecture on YouTube people can go watch. It's very interesting lecture. No Frills delivers. Get groceries delivered to your door from No Frills with PC Express. Shop online and get $15 in PC optimum points

on your first five orders. Shop now at nofrills.ca. So at the time I thought it was very bad news that I hadn't done so well on this exam. In the end, it gave me an opportunity to learn the rudiments of the subject. And this is an old story. I mean, Max Born tells a story in his autobiographical memoir about when Heisenberg was visiting from Munich to Gurtungen with Max Born. And Heisenberg eventually defended his doctoral dissertation and he was asked a question that he couldn't answer.

This is Born retelling the story. It's in My Life, Recollections of Nobel Laureate, which is Max Born's autobiographical memoir. And he tells the story about how Heisenberg could not get this one question. It was proposed to him by Wilhelm Wien, who was an experimental physicist. And Wien was quite upset that Heisenberg couldn't get the answer and wanted to fail him. And his other two committee members said that they couldn't fail him. He was the best theorist they'd ever seen. And so they agreed to give him the lowest passing grade. And Heisenberg was just mortified. He felt so depressed after he got this, he barely passed his final exam

for his doctoral work and he went and he sort of sulked for a long time. But then he thought about this question that Vilhelm Wien had asked him and it turned out the question was closely related to the uncertainty principle. And so all this time he spent worrying about this question he couldn't answer may have helped inspire him to come up with the principle that is the most famous thing attached to Heisenberg. Interesting. Heisenberg's the principal. So, again, you never know whether something you think is a failure is ultimately going

to end up being the thing that is important for your career. So I learned about stochastic matrices in a very rudimentary way. I considered some very simple stochastic systems that I was given to do for this final project. I learned about their connection, you know, long-time behavior of stochastic processes and how this is related to their eigenvalues. But these were all very new terms to me because I had only just learned linear algebra that summer. Fast forward a number of years to the end of my doctoral dissertation

in which I'm writing simulations, numerical simulations, to handle systems of, these intricate systems of black holes that show up in certain low energy solutions to quantum gravity. And to model them, we ended up having to use Markov chain Monte Carlo simulations. And so I was revisiting this theory of stochastic processes. I was remembering all the stuff I had done way back in, you know, early on when I was in college that summer. How many years at this point is there in between? Something like eight years,

nine years, something like that. I have to be more precise. I don't know exactly when I started doing the numerical simulations, so it's a little hard to say. How much of it did you recall? Is it just assemblance, like the gist of it or are you recalling precise formulas? I remember the gist of it, but I had scanned.

I had photocopied all my notes. And so I went looking for my old notes for this project that I did. And so I had an opportunity to see it again. Interesting. But now this is so many years later. It's after I had done an undergraduate degree in physics and math. It's after I had studied a lot of theoretical physics and philosophy. It's the end of graduate school.

I mean, I was coming at it from a completely different vantage point. And it looked very rudimentary to me now, given, you know, all the work that I had done in a year since. But I was so fascinated by this. Because going back to that time, I mean, I saw these stochastic matrices, this sort of theory of stochastic processes, before I had learned quantum mechanics, before I really knew anything concrete about quantum mechanics. And seeing it again, after all these years of having thought about,

having taught quantum mechanics to students as a teaching assistant, you know, having taken so many more quantum mechanics courses, having used quantum mechanics so much in all of my other work, I was struck by some of the formal similarities between the theory of stochastic processes and quantum theory. These are both theories that involve probabilities. They're both theories that involve processes in which the outcomes are indeterministic

and are described using some kind of probability. They're both theories in which we outcomes are indeterministic and are described using some kind of probability. They're both theories in which we encode those probabilities in vectors, in vector spaces. Stochastic processes, it's probability vectors. Quantum theory, it's state vectors or wave functions, or somewhat more generally density matrices. Time evolution is given by square matrices.

For a stochastic process, there's stochastic matrices. These are matrices whose entries are non-negative in the column sum to one. If we think about multiplication as being multiplication with the matrixes on the left and the probability vectors on the right. And in quantum theory, it's unitary matrices, unitary operators that carry out the evolution. For the theory of stochastic processes, some of the things we might want to ask about are random variables, functions on a sample space. The random variables are the things we could observe.

They're the observables. And in quantum theory, observables are represented by operators or matrices, self-adjoint operators or matrices. We call them observables also. So there are all these formal resemblances. And so now fast forward to me trying to prepare for this class. Try to teach to students who, like me, didn't know linear algebra at the time, didn't know very much about complex numbers, didn't know about quantum theory, but knew something about probability.

And I thought, well, maybe I can adjust the formalism of the classical theory of stochastic processes, and maybe I can adjust the formalism of the classical theory of stochastic processes. And maybe I can adjust the formalism of quantum theory to make them look a little more similar. Maybe I can figure out some kind of mapping between them, or change how I write them to make them look more similar. And I was, the goal, the goal was very clear. My goal was to bring the two theories as close together as possible

and then be able to tell the students, okay students, here's what you need to jump the gap. And by bringing the two theories close together, the hope was that I could make that gap a little bit less mysterious, a little bit less arcane, a little bit less extravagant. Maybe I could boil it down to some relatively simple, transparent change.

Maybe a generalization, maybe dropping an assumption, maybe modifying an assumption. Something simpler than just listing all the Dirac-Phonomen axioms out of the blue like we often do. And the surprising thing was that the gap disappeared. And suddenly I had one mathematical formalism for both the theory of stochastic processes and for quantum theory. And I was very confused how this had happened. It took me a little while to realize

that I had implicitly given up the Markov assumption. The assumption that for the stochastic process, what comes next according to the process is determined solely by the system's present state or configuration. That's the Markov assumption. I had allowed the probabilistic development of the system to depend on past details, details that were not mediated solely by what was going on at the present. And once I realized I dropped this, immediately what I did was check the literature and see,

surely someone else has tried to model quantum theory using a non-Markovian stochastic process of some kind. And the answer was there was almost nothing in literature about this. Previous efforts to replace quantum theory with something like a stochastic theory, which go back to the work of Fritz Bopp in the 1940s, who, by the way, is mentioned in Hugh Everett's correspondences, his 1957 letter to Bryce DeWitt, his extended unpublished dissertation from 1956.

He talks about Bopp's stochastic theory, says actually he thinks it's a fine theory if it can be fully developed. His objection is not to indeterminism. He doesn't prefer determinism over indeterminism. Everett says he just wants a theory that does one thing and not the Dirac-Vinolan axioms which seem to be deterministic in some situations and indeterministic in others. He's very clear about this and I recommend and you can put a link to his full 137 page unpublished dissertation. It's available online. People can read about this. The link will be on screen.

So this will be interesting for people to look at. But then work by Imre Fenyes in the 50s and Edward Nelson in the 60s through the 80s. But they had assumed that the dynamics should be Markov. This Markov assumption was an ingrained assumption that many people had. Now, at least as far back as 2011. Here I'm pointing to some work by Shelley Goldstein, Goldstein, Tosk,

Norsen and Zanghi. They have an article in Scholarpedia on Bell's theorem, which you can also link to. They note that Bell implicitly assumed Markovianity in deriving his theorem. There's a much more recent draft article by Tim Maudlin called

The Great Rift in Physics, Relativity and Quantum Theory, in which he also highlights this Markovian assumption that's implicitly made in Bell's theorem. Especially, this is especially clear in the 1990 formulation of Bell's theorem. La Nouvelle cuisine is the name of that paper. You can look at that as well. Bell relies crucially on situating his so-called screening local beables in a space-time region of finite thickness. It has to be finite thickness

so it doesn't intrude on the light cone of the other thing. And if you have a non-Markovian set of laws, you can jump over those regions through the laws, but in a manner that stays within the light cones, respects light cone structure. So it provides a way around some of Bell's conclusions while respecting the so-called relativistic causal structure of spacetime.

But what I think was lacking was a concrete realization of this loophole. Loophole may not be quite the right word because, you know, Bell has premises. Some of them were explicit, some of them were implicit. Those premises lead to Bell's theorem. Here, we're denying one of the premises. That's not quite a loophole as much as showing the theorem is doing its job, right?

The theorem says with these premises, you get this result and I'm denying one of those premises. So that doesn't mean the theorem is broken. In fact, it's showing the theorem is doing its job as a no-go theorem, highlighting what the premises are. Although arguably by not making this Markov assumption explicit, there was a loop, but you could argue that's a loophole in some sense.

But what we lacked was a concrete realization of a comprehensive formulation of quantum theory that was built on non-Markovian laws. And I had inadvertently stumbled into such a theory. The particular kind of non-Markovianity involved is called indivisibility. That term was introduced, and we've talked about this before

in the quantum information literature as far back as 2006. And then specifically in the context of classical looking stochastic processes in a review article in 2020, it was the pre-print 2021 is when it was published by Simon, Melson, Kavan, Modi. Right, it's recent. Very, very recent, right? We're talking just a year and a half or so

before I had independently arrived at this. Also to be technical, is it being non-Markovian or is it just not being Markovian? Right, so this is a subtle distinction. An indivisible stochastic process is one in which the laws let you probabilistically predict what the system is going to do as a function of time from some starting conditioning point.

And you may be able to predict what the system will do probabilistically at various choices of later time. But if you pick an arbitrary time in between, the laws may not give you what the rule is to go from an arbitrary intermediate time to later time. And that means that you have a failure of your ability to iterate the laws. That is, there's a failure of the laws to divide in time. Now, that's a form of non-Markovianity. It's a particular form of non-Morkovianity, and it's a particularly unstructured form

of non-Morkovianity. So what this means is that you can fill in the story if you want with a detailed set of probabilities for all the possible trajectories that could be taking place behind the scenes. The laws of theible process don't appear to fix just one choice of such non-Markovian description. My colleague Alex Meehan, who's a professor of philosophy

at University of Wisconsin-Madison, who may be also someone you might talk to, very interesting philosopher of physics, super great, big fan. He suggested that we use the term realizer for a given non-Markovian process, a specific realization, a specific process that assigns probabilities to lots of intervening events. And an indivisible process is not one such realizer, it's an equivalence class. It represents a whole collection of different realizers, different specific non Markovian processes that all agree on the rudimentary laws that define the indivisible

process. So the individual process identifies a couple of basic laws that fail to divide and doesn't fix all of these additional things that you might want to impose. And all those additional things, if you impose them, give you a specific realization or realizer. Sorry, what's a realization? Right. So let me be, this was sort of an overview, but let me now be a little more precise.

So one thing you could ask about such a process is, well, okay, behind the scenes, behind the scenes, if I imagine this process is truly unfolding and I run an indivisible process, let's say 10,000 times, can't I, if I imagine being able to see behind the scenes, can't I just count up all the trajectories that do this and do that? And then just by adding them up and dividing by the total number of trajectories that do this and do that. And then just by adding them up and dividing by the total number of trajectories, start assigning fractions,

probabilities in some frequent sense to lots of statements that are not specified by the limited laws I gave you. Well, the answer is you could. You can do that. What you'll find is that there's not a unique such choice of frequency ratios to write down. The laws of the indivisible process are very rudimentary. They tell you how to predict what the probabilities will be starting at certain conditioning times to later times.

The later times are adjustable. There's no assumption that time is fundamentally discrete here. But that's kind of all the laws give you. They don't give you detailed information about the probabilities to assign to the detailed trajectories that are going on behind the scenes. If you impose what all those probabilities should be, all those extra probabilities behind the scenes, all of them, completely specify them, which is an infinite amount of information and very unwieldy and not practical.

But if you could somehow imagine imposing all of them and fixing absolutely every probabilistic detail, you would have one realizer. But nothing fixes that particular set of choices. You could have assigned different frequencies, different probabilities to all those behind the scenes details, give a different set of probabilistic statements

for all those things, consistent with the same indivisible laws that we started with and that would be another realizer and you can argue that there may be many, many, many realizers but because all the empirical predictions of the theory come out of the rudimentary indivisible laws all these extra details do not appear to have any empirical content and an indivisible theory simply doesn't specify what they are.

It leaves them undetermined. And so an indivisible process represents a whole class of possible realizers. So the difference between an indivisible process and a non-Markovian process is that a non-Markovian process is one such realization where you've assigned probabilities to every possible statement, every trajectory, every detail. An indivisible process is less sharply defined. You define some features and those features are sharply defined. But what's not empirically significant is left unfixed. So an indivisible process in a way represents a whole class of non-Marcovian processes.

And the nice thing about this is from the point of view of someone who isn't even thinking about quantum theory, maybe you're a statistician, maybe you're a statistical modeler, you might have thought, well, I'd like to model non-Markovian systems. I'd like to model systems in which the more you know about the past, the more you can say about the future. But if your system is arbitrarily non-Markovian, if there's no bound on how much previous information will determine what happens in the future, you might think that you'd need to specify

an infinite amount of lawful information, an infinite amount of information about the laws of your model in order to make any predictions at all. It might seem that fully non-Markovian processes are just too impractical to use. These indivisible processes give you a way to handle that problem with a limited set of laws. You can make predictions with these models, you can make empirical predictions with these models. Even though the models are inherently non-Markovian,

you don't have to specify an infinite amount of information. You might have worried if I only specify a small amount of information with the laws, will this theory be able to make any empirical predictions at all? And the answer is well, as I've shown, it's equivalent to quantum theory. And quantum theory certainly makes a lot of empirical predictions. So you might have thought what you had to do was make a Markov approximation.

Just take all that past information, truncate it somewhere at some order, or truncate it all the way down to just the present state. We know when we do that we're making approximations, but maybe we thought that's all we can do because otherwise the problem's too intractable. An indivisible process gives you another way to make non-Markovian processes tractable without making all of those approximations.

The cliffs, let's speak about cliffs. The cliffs between being a theoretical physicist or a quantum physicist and being in finance, it's actually not that large. And you know, many, I'm sure, that have made that jump. I have gone both directions too, yes. Okay. Now, given that you have a new mathematical tool, indivisible stochastic processes that are a subset of

non-Markovian that doesn't require infinite amount of information and Markovian and non-Markovian processes are useful in finance. Do you imagine that your indivisible stochastic approach not only will aid quantum theory, quantum foundations potentially, but finance? Yeah, I mean, I came at this out of an interest in trying to make sense of the fundamental nature of the physical world, to make sense of quantum theory, to answer this question, can we find a world picture that underlies quantum theory? But I would argue that good foundational work in science and in philosophy

can and should have practical implications for other fields. On the one hand, having this indivisible picture might give us finally a world picture underlying quantum theory, but you can read this correspondence in the other direction. Right. If you have some thorny system in which you have to worry about memory effects or other forms of non-Murkovia entity and didn't know what to do and don't want to make various kinds of approximations, this might give you a new set of tools for studying those kinds of processes,

whether you're working in finance or you're working in biostatistics or you're working in neuroscience or you're working in machine learning. I mean, indivisible processes are new. They only entered the research literature in the way that I'm describing them for what we would normally call something like a classical looking stochastic process in 2020-2021. And that paper didn't really explore what you could do with them. Now we have this tool and the question is what can we use it for? And I think that's a really exciting thing to have a brand new tool. It's like having a blank page that we can write on, right? I mean, it's five years old. So attention is all you need was 2017. Chat GBT came out five years later. Who knows what's going to happen this year? Who knows

what's going to happen going forward? That's right. But, you know, I would be very excited to see these methods find use in other areas. Finance, biostatistics, neuroscience, you name it. I mean, all these areas, I think, you know, we don't know where these things could be useful. I'm excited to see where they could be. Hi, everyone. Hope you're enjoying today's episode. If you're hungry for deeper dives into physics, be useful. I'm excited to see where they could be. part of a thriving community of like-minded pilgrimers. By joining, you'll directly be supporting my work and helping keep these conversations

at the cutting edge. So click the link on screen here. Hit subscribe and let's keep pushing the boundaries of knowledge together. Thank you and enjoy the show. Just so you know, if you're listening, it's C-U-R-T-J-A-I-M-U-N-G-A-L dot org, KurtJaimangal dot org. T-J-A-I-M-U-N-G-A-L.org, KurtJaimungal.org. This came from you tearing apart your curriculum and thinking how can I attack this course in a new manner to make

it more elementary for people to understand for first-year graduate students, or maybe this was undergrad. This was first-year undergraduate students who were taking this class. I want to speak about first-year graduate students. What's an example of something that you've consistently found is difficult for graduate students who are taking this class. Okay, I want to speak about first year graduate students. So what's an example of something that's been, that you've consistently found is difficult for first year graduates to understand,

but you found some clever analogy or some interesting hello world example that makes it finally click for them? So, I've taught a number of courses aimed at first year graduate students in physics. I taught Jackson's classical electrodynamics for six years. And I've taught graduate level general relativity for over a decade. So there are a lot of examples of things, I mean there's many I could list, where just through experience, repetition, teaching over and over again, hearing people's questions, and having lots of opportunities to refine my approach, I think I found some things that that are helpful to talk about.

I think one example I like is talking about black holes. So when I teach general relativity and we get to black holes, I set aside half of a full class where I don't do any lecturing. I just stand in front and ask students to ask me all their questions about black holes. Like an opportunity to talk about the physics of black holes, the metaphysics of black holes, whatever they want to ask. Because black holes are incredible. They're mysterious, they're exotic. For a lot of young people, hearing about black holes is the kind of thing that triggers their interest in science, maybe even in physics. And so to finally be in the class where we've mathematically formulated what black holes are,

we have the tools now at our disposal to start answering technical detailed questions about black holes is a really exciting moment. And I don't wanna run through it too quickly. I wanna give students an opportunity to dwell and really ask questions about it. And so I don't know that I have exactly like a trick that I do,

except the trick being this sort of methodology, this methodology of just opening up space during a period of time during class to just talk about things, to field all their questions no matter how silly they may seem. Tell me about some of the most common questions. Well, so one question students often have is, if there's so much time dilation near the edge of a black hole, how does anything actually fall into a black hole? I mean, you draw these space-time diagrams that show

the trajectories of infalling objects. Space is in one direction, time is in another direction. And you draw these pictures and it kind of looks like the trajectories never quite reach the event horizon of the black hole and they never quite fall in. And this is really mysterious to a lot of students. And so it's an opportunity for us to talk about coordinate transformations, coordinate representations in general relativity. How will we derive the solution for the simplest kinds of black holes

or some more complicated ones? We're using coordinate systems that are particularly good for solving the basic equations of general relativity, the Einstein field equation, in order to write down the correct formula for the shape of space-time. But it may not be the best coordinate system to understand the experience of observers

moving around in space-time. As we talk about how you can change coordinate systems, there's no canonical preferred default coordinate choice in general relativity. You can change your coordinate systems, and it turns out the kinds of coordinate systems that we often use when we first start talking about black holes are not the best kinds of coordinate systems to understand

what the experience is of probes or observers who fall into black holes. There are better coordinate systems. And even if you want to say, well, I want a coordinate system that for which the clicking of a wristwatch, you know, has its sensible meaning for observers very far away. So suppose you have an observer very far away who drops in a probe to fall into the black hole. And you can ask, what kinds of coordinate systems can I write down that far away from the black hole, tick in time

at the same rate as the far away observer's wristwatch, you'll find there's not a unique coordinate system you can write down. The coordinate systems we will sometimes write down when we're first trying to solve the Ancet-Field equation do have the property that for observers far away, the notion of time for that coordinate system coincides with the ticking of the wristwatch, but these coordinate systems can be very badly behaved near the edge of the black hole. There are other coordinate systems you can write down

that are just as legitimate, that likewise agree with the ticking of the wristwatch outside far away from the black hole, but that allow us to see what happens to things that fall into the black hole. An example is Gullström-Ponlew coordinates. But there are other coordinate systems you can write down. People can learn about this.

Gullström-Ponlew coordinates have a beautiful name, they're called global rain coordinates. Because they're based on free-falling trajectories like raindrops falling down the black hole. I think the image of a black hole floating in the silence of empty space with raindrops quietly falling on it is such a beautiful poetic image. But, you know, so there are, so showing students that there are other coordinate systems you can write down and that you shouldn't take any one

particular coordinate system to be completely serious as the only way to understand what's going on in a picture of space time. That's the kind of thing that I find can be very elucidating to students who are confused about what goes on with black holes. So that would be an example of something I would point to as something that can be very illuminating for students who are seeing this for the first time.

What's the difference between being coordinate independent and being generally covariant? Well, the general in general relativity refers to, well, when Einstein introduced that term, my understanding, and this is now an historical question, so this needs to be fact-checked, but my understanding was that Einstein was in part motivated by a desire to understand how to incorporate gravity

into special relativity, but was also in part trying to understand how to be able to handle relativity in much more general coordinate systems than the Cartesian-Minkowskiian coordinate systems that one uses in special relativity, these rigid straight line rectilinear coordinate systems. He wanted to be able to talk about the kinds of coordinate systems that might be adapted to observers that were in various kinds of acceleration or freefall. Or just be able to talk about the theory with a more general set of coordinate representations. Here, these coordinate systems are being put on four-dimensional space-time. So space points in three different orthogonal directions. You have to use your imagination and imagine the fourth dimension for time.

And space-time can be curvy in some intrinsic sense that doesn't require the existence of an extra dimension for the curvature to happen. And we want to describe all of the points, the events, the things that can happen at certain points in space and at certain times. We want to be able to describe them numerically, and so we put down a coordinate system like graph paper. But you could imagine different kinds of coordinate systems, different kinds of graph paper. And if you wanna be very general about this, you wanna be able to handle general coordinate systems, that's the general in general relativity.

General relativity is able to handle arbitrary coordinate systems, general coordinate systems. And then you want the basic rules of the theory general coordinate systems, and then you want the basic rules of the theory to maintain the same meaning, to have a sensible consistent meaning as we imagine transforming between coordinate systems. We want them to be covariant. The word is covariant for having a certain kind of consistency as we go between coordinate systems. Not invariant. We're not saying that things look exactly the same in every coordinate system.

Covariant is a bit of a weaker statement. It just means that there's a certain kind of technical formal meaning of the ingredients of the theory, the laws of the theory, the mathematical objects we use to represent things in the theory. We want them to maintain their integrity in some sensible way as we imagine changing coordinate systems.

And general relativity has this feature that the theory retains its structural form as we imagine changing coordinate systems in a very general way. And so general covariance is related to our ability to change coordinate systems, but it's a statement of the rules or laws of general relativity that when we do change to general coordinate systems, they maintain a certain kind of conceptual integrity. Your PhD supervisor was Nima, Nima Arkani Hamed, correct? I worked with Nima Arkani Hamed for the first few years of my PhD.

Then Nima took a job at the Institute for Advanced Studies at Princeton. I transitioned from working on particle phenomenology, which was primarily what Nima worked on, to quantum gravity. So I did the second half of my time in graduate school, the second half of my research in graduate school, and ultimately my dissertation with Frederick Deneff who works in quantum gravity. Well, I wanted to know what's something you learned

from NEMA that stuck with you to this day? I think anyone who talks to NEMA for five minutes learns a lot of things. So, there's a number of things I learned from Nima. One is I learned a lot about tennis. We played tennis, which was great. He was a little better than me, so he always ultimately won every match we played, which was very frustrating. I mean, I learned effective field theory from NEMA. So effective field theory is a particular paradigm for how we think about quantum field

theories, not as necessarily fundamental, exact descriptions of nature, but as theories that provide us with progressively more precise approximations that allow us to make progressively more accurate kinds of predictions about the things that we see. And thinking about quantum field theory in terms of effective field theories, it means we don't take our theories necessarily too seriously as strict statements about what's going on in nature, and that

our theories serve us rather than us serving our theories. We can adjust the features of our theories to provide a way to describe different regimes. And, you know, the larger theory of effective field theory goes back to very important, you know, physicists, people like Steven Weinberg and Howard Georgi and various people who played very important roles in the development of quantum theory. I'm leaving lots of people out. I mean, effective field theory was developed by a lot of people.

But NEMA was a very strong proponent of thinking in terms of effective field theory. That had a big impression on me. When I took quantum field theory with NEMA, I'd seen quantum field theory before. I took it again with NEMA because I figured I would learn a lot from him, and I did. And his approach to quantum field theory was heavily based on thinking in terms of effective field theory. And, you know, in understanding the relationship between particles and fields through representation theory, which was also really very revelatory for me. Another very important thing I learned from Nima was that I really wanted to be a philosopher

of science, philosopher of physics, rather than a theoretical physicist. Because some of the things that Nima said about the foundations of quantum mechanics, I found very mysterious. And some of the questions that he raised partly inspired my shift in thinking, which wasn't so much a shift as much as a return to the kinds of philosophical thinking I'd always really wanted to engage in. In a lot of ways, physics in my undergraduate years and in graduate school was an extremely enriching, worthwhile, exciting, mysterious, wonderful detour that gave me, I think, a lot of skills and tools to address the kinds of questions I was really most interested in at the intersection of philosophy and physics. And some of the conversations and things I learned from Nima inspired me to go in those

directions. So I'm very appreciative of my getting to know him. And I think anyone who spends a lot of time with him, you just learn a lot from talking to him. I mentioned he had a view on decoherence, and he had some proof about how decoherence solves the measurement problem or an argument, but it didn't quite work out and you wrestled with it for days or weeks maybe. Yeah, so Nima taught a course early in my time in graduate school. I don't know if I should say this, but it was a, so the course was called a Physics

283b, it was called Quantum Mechanics in Space-Time. And so it was an upper level advanced graduate course. Everyone who went in to take the course was going to brace themselves. We're all bracing ourselves for what we expected to be very difficult homework assignments and exams. Every week, Nima would say, I'm working on your homework assignment. It'll be ready soon and it's going to be difficult, so just stay tuned.

He kept saying that every week, kind of like the Dread Pirate Roberts from the Princess he kept saying that every week, kind of like the dread pirate Roberts from The Princess Bride saying that, you know, to Wesley, you know, you know, good working with you, I'm most likely to kill you in the morning. And he said that to him every day for five years. So and I think Neema will appreciate the Princess Bride reference because he also liked making Princess Bride references. He once accidentally drew a picture of someone with six fingers and said, too much, Princess

Bride. But so every week he would say he would get us a homework assignment. And then the last day he said, it's been great working with you. And we all left him. We never got a single homework or exam the whole course. We all got A's. It was the it was the best course. I learned so much from the course. It was mostly about quantum field theory and curved space time, which is a very intricate, really interesting subject. Wait, you didn't even have to do an exam?

No, nothing. We all just got A's. So I learned a lot and I got an A with no homework of any kind. It was fantastic. But of course, while you were there in class, we were all working very hard to understand what was going on. So it was a very enriching class. But he began the class by talking about quantum foundations. And the motivation, and I think this is a very important motivation for people to think about it, and this continues today, is that he was trying to talk about quantum theory in the cosmological context. How quantum theory should be connected up with our best theories about the universe as a whole.

And when you're trying to understand the universe as a whole, you only have, as far as we know, or at least when they have access to one observable universe, there are no external observers as far as we know who can do measurements on the universe. And so you begin to run into some rather important fundamental questions about quantum theory. And Hugh Everett talked about these questions in his, in the published version of his dissertation in 1957. He specifically talked about cosmology and how difficult it was to talk about quantum theory in the context of cosmology where you don't seem to have external observers anymore.

So these kinds of questions have been going on for many, many decades. And I first began to see some of these concerns in this class that Nima was teaching. And Nima argued in that first class that there was a way to make sense of quantum theory in many real world scenarios. Maybe not necessarily in the full cosmological situation, but in many scenarios by deriving everything from decoherence. The way he put it is you need to posit the eigenvalue-eigenstate link, which just connects a thing called eigenvectors in the Hilbert space with numerical values called eigenvalues, the things that we should see in experiment when we measure things,

unitary evolution, so the idea that a quantum state evolves according to a smooth linear kind of rule, essentially the Schrodinger equation in the simplest cases, and the probabilistic predictions of quantum theory, the Born rule, that results should occur with certain probabilities when we measure them. And he claimed in that lecture that you could derive the second two things solely from the first thing. As long as you put the right kinds of measuring devices

into the story and understood how decoherence worked. And some of this was inspired by a lecture that Sidney Coleman, who was a quantum field theory professor at Harvard, gave in 1994, which is on YouTube. It's called Quantum Mechanics in Your Face, and there's also a transcribed version on the archive. You should read to both of those. Yeah, I'll place it on screen.

That's right. And, you know, and he made all these statements, and he was very persuasive at the time that you could, you really could get all the whole theory of quantum theory out of just the eigenvalue, eigenstate, link, and decoherence. And I should say that I've spoken to NEMA since then and he doesn't remember having said exactly those things. He said he doesn't have those views and doesn't remember having them at the time. And maybe, I mean, my notes were very accurate, so I definitely took those things. He said he doesn't have those views and doesn't remember having them at the time.

And maybe, I mean, my notes were very accurate. So I definitely took those notes. So it may be that he didn't tell the whole story. It may be that I misunderstood what he was saying. It may be that he briefly had one set of views and did another. I don't want to attribute anything to Nima on this count. But at the time, that's what I took from this lecture. And I spent a long time trying to understand how you could get the second two postulates from the first one. And ultimately I decided that you just couldn't do it. And this isn't, I'm not the first to have this problem. I mean, lots of people, when they've sat down to really try to understand any of these things to fit together,

have found themselves in a similar circumstance. So these kinds of thoughts inspired me in part to want to understand quantum foundations better. Now there were other things that happened while I was in undergrad and grad school that pushed me in part to want to understand quantum foundations better. Now, there were other things that happened while I was in undergrad and grad school that pushed me in this direction. But that was definitely one of the things that inspired me in that direction. So Sidney Coleman had a bravado and a confidence when he was giving his lecture.

So is the statement from NEMA that he gave him the lesson the same as Sidney's? It was similar in some ways. I'd have to look at the notes in detail. It's been many years since I took this class from NEMA and I don't want to attribute anything to anybody that they didn't really say. Well, my question was going to be, I don't know if you recall Coleman's lecture, but if you did, what was the critique? Well, Coleman's lecture was based on, I mean, I have the utmost respect for Sidney Coleman's work in quantum field theory, which was extraordinary.

And, you know, some of the most important quantum field theory people who lived talked about Sidney Coleman being the person who taught them more about quantum field theory than anything. I mean, Steven Weinberg, for example, who passed away just a couple of years ago, Nobel Prize winning quantum field theorist who helped construct the standard model. He was at Harvard and at an event to honor Sidney Coleman back in 2005, I'm pretty sure I remember Steven Weinberg saying that he learned more about quantum field theory from Sidney Coleman than anybody else.

And anyone who knows Steven Weinberg learned more about quantum field theory from Sidney Coleman than anybody else and anyone who knows Stephen Weinberg's textbooks on quantum field theory will realize what an amazing statement that is but Sidney Coleman's lecture on quantum mechanics was It was I think not at the level of rigor some of his work in quantum field theory he basically tried to say that... This episode is brought to you by Dzone.

For the first time ever, the 32 best soccer clubs from across the world are coming together to decide who the undisputed champions of the world are in the FIFA Club World Cup. The world's best players, Messi, Holland, Kane and more are all taking part. And you can watch every match for free on Dazon, starting on June 14th and running until July 13th. Sign up now at dazon.com slash FIFA. That's D-A-Z-N dot com slash FIFA. His view of quantum mechanics was just quantum mechanics. He even said it's quantum mechanics, comma, stupid, as if anyone who thinks differently is stupid. The tone he takes in the lecture

is very dismissive toward philosophers. Now, I don't know, I didn't know Sidney Coleman personally. I never got a chance to meet him. I heard wonderful things about him. So I can only go based on what's in the lecture, but that dismissive attitude toward philosophers is if physicists can do all of this themselves

and philosophers don't have anything meaningful to contribute, is not, I think, a good message to be sending to young people. I don't think it fosters the kind of interdisciplinary dialogue that can be very fruitful for all of these disciplines. And he says clearly in the lecture that he attributes his views to the picture of quantum theory that comes from Hugh Everett. So he takes a point of view on quantum theory that is at least in some way inspired by Everett,

although he doesn't specifically commit, I think to a full many worlds picture in lecture, although my memory could be slightly mistaken. But he appeals to various arguments to get the probabilities out of quantum theory that are no longer widely considered definitive. So he has various arguments involving infinite lists of experiments and in some notion of doing an experiment infinitely many times you get exact you get exact results, but of course you can't really do infinite measurements. That's why a lot of these arguments have fallen out of favor.

So he also makes some some statements about he uses the GHG theorem, which is a somewhat more modern, non-probabilistic version of Bell's theorem that's related to non-locality to make certain strong arguments about what's possible. I think he also does something that I think requires a little more attention. So on the one hand, you can talk about probability theory. On the other hand, you can talk about kinematics. Kinematics is the way that we mathematically represent the configurations of a system, the states of a system, the trajectories of a system,

the way we describe how it is and what it looks like. And then there's the dynamics. Dynamics are the rules for how configurations or states change. Those are the dynamical rules, the rules that tell us how you go from initial states to later states and so forth. F equals ma is a dynamical rule. The Maxwell equations are dynamical rules. Whereas the kinematics of St. Newtonian mechanics is the statement that there are bodies in space and we represent their locations using coordinate systems. That's kinematics. And of course,

probability is probability. Now, there's a view that what quantum theory did was transform probability, kinematics, and dynamics. They're all different. They're all non-classical. They're all something totally new. To the point at which I think some people conflate all of these things and just say all of it is quantum. And I think we'd go farther and say that if you think any of them is not

quantum, you're saying that none of them are quantum. I think that's too strong of you. For example, in the indivisible stochastic approach, probability theory is ordinary probability theory. We're using good old fashioned ordinary probability theory, which in talking to statisticians, they seem to be like the fear among I think some statisticians is that quantum theory doesn't use ordinary probability theory. And so it's not something amenable to some of the techniques that statisticians like to use. In the indivisible approach, we're using ordinary probability theory. If you want to call it classical probability theory, it's classical ordinary probability theory, all the probabilities that show up are just of the usual kind.

theory. It's classical ordinary probability theory. All the probabilities that show up are just of the usual kind. The kinematics, the configurations of things are in some sense kind of classical. Systems just have arrangements, configurations, maybe in physical space if you think physical space is the right way to think about things. If you're modeling a system of particles, this is arrangements of particles in physical space. If you're talking about fields, it's patterns of field intensities in space. These are the kinds of configurations that we would describe classically for the most part. What we're altering is the dynamics. The dynamics is no longer going to be a Markovian deterministic differential equation or set of differential equations. The dynamics will take the form of

these, will now take the form of these non-Markovian probabilistic kinds of laws. So that's what's new. The laws are now different. And I think in Sydney Coleman's lecture, he can place all these things. He says that if you think about any of these things in a non-quantum way, then you're just being too classical. You're just thinking everything is classical.

And I think if you read the transcript or watch the lecture, you'll get that impression. So I guess my feeling is, you know, with enormous respect, of course, to Sidney Coleman, I don't think that lecture answers the questions of what's going on in quantum foundations. And I think that for at least some time, I think it gave people the impression that these problems were all solved. They were all just a mistake of people working in philosophy.

There's one point in the lecture when Coleman even looks very frustrated. He's like, I can't understand why all these people, they draw all these diagrams and they get so confused. I don't understand how they could be so confused about such a simple point. And it's got a very condescending tone toward people who work in philosophy that I just don't think is helpful. I don't think any of us need that kind of attitude. We need to bring people together. Did these disciplines work much better when we understand the work that we do and we bring our work together? You get a kind of cross-pollination that lays the seeds for future discovery. And, you know, I should say that from people I've spoken to who knew Sidney Coleman, he was not anti-philosophical. He was very interested in a lot of this stuff.

So the best I can say is maybe that somehow didn't come across in this lecture. And for whatever reason, that was the case. But I think that that's worth noting. Why did you leave string theory? I was drawn to these foundational questions at the intersection of philosophy and physics, I was just fascinated by the problems that happen trying to make sense of quantum theory. I was dissatisfied with the interpretations and formulations that were available. And I just, I got hooked, you know, you get hooked on something and you just want to probe it and understand and make sense of it. And I just found myself much more drawn to those kinds of questions.

I'm particularly drawn to the kinds of questions that philosophers of physics will say is the kind of work that they do. We tend to be interested in the mathematical and structural features of our best, most successful physical theories, some of which are fundamental or candidate fundamental theories, some of which are somewhat more prosaic theories that describe everyday physics. We want to know what it is that physicists or more generally scientists are doing when they do science. We want to think about what is the success of these theories tell us about the structure of what's going on in reality. You can't do anything without some metaphysical posits. At the very least, that our experimental devices exist, right?

I mean, there's some very basic metaphysical things that we assume to get things working. We have to assume scientific induction works, that we can take evidence of the past and use it to say something about the future. There's no way to justify that. I mean, back to David Hume, you know, hundreds of years ago who pointed out that any attempt to justify that by using past experience is a circular argument because if you say that induction is successful,

we should believe in induction because it's worked so well in the past, then you're using induction to support induction. And a lot of people have written about this. I know that you had John Norton on your podcast as well, and he's written a whole book on how to make sense of induction in his book, The Material Theory of Induction. So, you know, these are very difficult problems.

You need something to get off the ground before you can begin to even do science. And philosophers of physics, philosophers of science are interested in those kinds of questions also. We're interested in what our best physical theories have to say about traditional questions in philosophy and in metaphysics. That's what I call physical philosophy. And at least some of us are interested in taking

the methods and tools that one develops in analytic philosophy and philosophy of science and applying them to make progress on outstanding questions in physics. That's what I call philosophical physics as opposed to theoretical physics or mathematical physics or experimental physics or applied physics or computational physics. Right. And trying to make sense of quantum theory, I view very much as a form of philosophical physics. We have a physical theory quantum theory. There are places where it is either ambiguous or our other attempts to make sense of it run into problems of empirical adequacy or one of the other problems that we've

talked about earlier. This is a physics problem and it may be that the ideal set of tools to address it are the kinds of tools that Einstein was using when he interrogated the meaning of inertial reference frames. An interrogation that led to enormous consequences. This whole development of relativity comes out of him trying to develop a more rigorous understanding of inertial reference frames. The tools being? Tools of rigorous scrutiny, careful argumentation from as clear definitions and as clear, clearly stated premises as we can formulate, looking for implicit assumptions that might not have been noticed,

looking for connections, questioning standard assumptions, looking under rocks metaphorically speaking for interesting things that may be looking under them. Can you give an example of that? The looking under the rock? Sure, yeah. So, I mean, in a sense, that's what Einstein was doing when he was probing the meaning of an inertial reference frame.

I think one could very well have had an attitude back in the early aughts, not the 2000 aughts, but the 1900 aughts, not the 2000 aughts, but the 1900 aughts, that physics was about the atomic theory and it was about electromagnetism and it was about thermodynamics. And here Einstein wasn't trying to build new models of that kind. He was asking about the scenery, like inertial reference frames. Inertial reference frames, that's background stuff, that's scenery, that's not the protagonist of physics, but he uncovered that rock and out came relativity.

So, I mean, that's, I think a fantastic example of how looking under rocks can lead to new insights. We also look for connections between theories, connections that other people haven't noticed. I mean, this project came out of me trying to build a connection between the theory of stochastic processes and quantum theory. What kind of work is that? It's not the kind of work I think that we usually associate with theoretical physics where you're building models with any given paradigm or mathematical physics where you're

trying to take statements that physicists have made and prove them rigorously turn them into theorems. Or apply mathematics to physical problems. Or develop new areas of mathematics inspired by what's going on in physics. You know, it's not experimental physics, it's its own kind of thing. Definitions, premises, building very, very careful arguments, uncovering hidden assumptions, looking under rocks, building connections between things, analyzing the conceptual and mathematical structure of our best theories, understanding their moving parts. Spending time doing that is really exciting and sometimes this leads to surprises. But scrutinizing, I mean that's,

um, scrutiny is a particular methodology of doing things, right? We're not necessarily doing experiments, we're not just building models, which is of course, you know, building models is a very important and very difficult and challenging thing to do, and obviously central to how we formulate physics. But taking things we already have, and really trying to pin down all the details, trying to make sure everything fits together, there are no gaps and no logical

problems. Trying to pin all that down is just a different approach. It requires a different set of tools, different inclinations. I mean, there are different people who like doing different things, who have different talents, different tendencies, different inclinations. Some people like doing one kind of work. This is the kind of work I like doing. And arguably it is born fruit for science.

Last time we talked, we had a lovely discussion with Emily Adlam. And during that time, we listed many important contributions that have been made to physics that have come from taking that philosophical lens and applying it to our best physical theories, from EPR and entanglement to decoherence to quantum advantage, Bell's theorem, the no-cloning theorem, the no-signaling theorem, and arguably later work that is connected with things like quantum teleportation and quantum cryptography and

quantum information. So I think there's a real track record of this being incredibly useful to science and worthwhile for people to work on and also for people to fund. So I have a question about the interpretations of quantum mechanics. There's plenty of hubbub about interpretation of quantum mechanics, but if it's already understood that the greater theory, the more fundamental theory will be a theory of quantum gravity, furthermore that the only or the most well-developed UV complete non-perturbative approach is string theory, then why isn't there more work done on the quote-unquote interpretations of string theory?

Yeah, so I would recommend that people look up the work of Nick Huggott, for example, a philosopher of physics who's given a lot of attention to try to understand the metaphysics and philosophical questions surrounding string theory. I did some work with some of my colleagues, some preliminary work on string theory. We just had a lot of other things we were working on and we got pulled away from this. I think partly the reason there's been less philosophical work on string theory is because it's just newer than things like relativity and quantum mechanics. You know, relativity goes back over a hundred years, quantum theory goes back over a hundred

years. So there's just been a lot more time for people to think about these theories. String theory is of much more recent vintage. I think one issue is that string theory is still very speculative. We don't know that string theory is the confirmed best theory of quantum gravity. And I think that there's some reluctance to try to spend too much time trying to understand the metaphysical underpinnings of a theory that isn't substantially verified at this point. We don't have any real empirical confirmation of string theory at this point. I said before that philosophers of physics like to understand the mathematical structure and features and implicit assumptions and everything for our best, most successful physical theories. And by this, I mean theories that have proved the test of time, that have been empirically verified.

Because the work that we do for those theories, we feel will stand the test of time. Whereas if you're working on trying to understand the philosophical underpinnings of a speculative theory that might not last, well then your work on that might also not last. Now of course one could take the view that philosophical work, foundational work on string theory may help move quantum gravity forward. And I think that maybe partly would inspire people to think about these questions. But I would recommend that you interview some of the people who work on the philosophy of string theory.

I think Nick Huggott would be an excellent person for you to talk with. I think you might get some insight, and you could ask him what motivates him to work on this theory. I think the other main reason why fewer people in philosophy and foundations of physics work on string theory is that string theory is mathematically very complicated. I worked in string theory during graduate school, and there are very beautiful features of string theory. I mean, you work on it and you see beautiful connections. It changes how you think about quantum field theory in really profound ways.

I learned a lot about quantum field theory from learning string theory. But it is a very mathematically intricate theory, and I think that presents an obstacle for a larger fraction of people working in philosophy of physics to work on it. Now, the people I know working in philosophy of physics are very mathematically sophisticated. But there's an investment in time that one has to make in order to be able to work on string theory, a bigger investment in time, I think, than maybe the case for other theories people can work on. And so I think that's been a hindrance

to more people working on it. But there are people who have decided to invest that time and they've worked on it. And it's interesting to know, to think about what they've been able to see. I think one of the difficulties is that because string theory is in many ways still a very partial theory,

it's not yet quite ripe enough to be able to start making strong statements about the metaphysics. We have only a very sort of superficial understanding of exactly what the theory entails. And that makes it difficult to ask deep foundational questions about it at this stage of development. What did you learn about QFT from string theory that isn't string contingent? That isn't string contingent. That's a good question. I mean, so one thing I learned about quantum field theory from string theory is how to think about gauge theories.

I'm a little worried that if I get in the weeds here, this is going to become technical very quickly. The audience loves the technicality, don't worry. Okay. Well, so in string theory, one models the kinds of particles that transmit forces using certain kinds of strings. We use open strings to model gauge theories, and we use closed strings to model gravitons. The actual force carriers correspond to particular quantized

excitations of these strings. And, you know, one question you could ask is, well, if I have a particular, you know, quantized excited mode of a particular kind of a string, and this string is supposed to describe some kind of force carrier, well, I've got the particles, but how do I understand where the field comes from and the structure of the field? Why should these fields have certain features? Why should gauge fields exhibit gauge invariance? Why should the gravitational field exhibit diffusomorphism invariance and the equivalence principle?

Why should it treat inertial and gravitational mass as being similar kinds of things in the first place? And you know, in string theory, because you're working in a so-called first quantized formalism most of the time, you're working with individual strings, which manifests as individual particles rather than in the so-called second quantized formalism that's called string field theory, which is its own discipline and there are people who work in string field theory.

But a lot of the time you're working in the first quantized string particles as sort of individual excitations of these strings. Understanding how you get all the intricate rich structures that show up in quantum field theory from these strings, ports right over to thinking about the relationship between, not in string theory, the particles that correspond to fields and the fields that they're related with. So, for example, photons are associated

with the electromagnetic field. The Higgs boson is associated with the Higgs field. The electron is associated with what's called the electron field, which is a fermionic field. It's a very interesting kind of thing. Quarks have their own fields. Deutrinos have their fields. There's fields associated with all these particles.

And understanding the relationship between a field and its corresponding particles is a little bit subtle. One way to think about it is the field is a sort of prior idea and that you can argue using some elementary quantum mechanics, a little bit subtle. One way to think about it is the field is a sort of prior idea and that you can argue using some elementary quantum mechanics thinking of fields as connected systems of harmonic oscillators that quantum mechanics harmonic oscillators can only be excited

in quantized amounts. These fields can only be excited in quantized energetic amounts and each such quantized excitation, each quantized energetic excitation corresponds to one more quantum, one more particle of that field showing up. This is the sense in which, say, the Higgs boson is one quantized excitation

of the underlying Higgs field. But you may also ask the opposite question. If I imagine starting with some statement about the properties of particles, let's suppose I've got a particle with certain features. I've got a particle that's got this intrinsic inertial mass. It's got this intrinsic charge. It's got this intrinsic charge. It's got this intrinsic spinniness or spin or angular momentum. Can I predict what kind of field it will correspond to?

And can I even, in some sense, model the field as some appropriately defined quantum state involving these particles, as some coherent state of these particles. And if I do that, what kind of field do I get? What features does the field have? And what I took from string theory, which I guess you could have learned just in quantum field theory, but for me, it took learning string theory to see this connection, was that the properties of these particles, again, things like their mass and their charge and their spin

and these are the features of these particles, again, things like their mass and their charge and their spin and these are the features of these particles, can reveal to you the structure of the fields they correspond to. Interesting. For example, the fact that photons have zero intrinsic inertial mass and have one unit h-bar of intrinsic spin and have no charge. That you can use to show that the field that emerges from photons is gonna look something like the electromagnetic field and in particular, the masslessness of the photon and its spin being one

is connected with Gajian variance in this electromagnetic field. So that's the sort of connection. And again, you could understand this connection without string theory. If you pick up Steven Weinberg's books on quantum field theory, he also explores this connection. Although in a way that when I first learned it

from Steven Weinberg's book, I didn't find as intuitively clear as I did from string theory. So that would be an example, right? And string theory has generated a number of other spin-offs that people working in quantum gravity spend a lot of their time thinking about. Things like holography, you know. And you know, to whatever extent that you think that those ideas are worthwhile, they're certainly very interesting.

And arguably they stand on their own separately from string theory, at least some of them. I should have been asking about the interpretations of quantum field theory as well. Who are some people I should be speaking to? Interpretations of quantum field theory, there's a number of people who work in that area who might be worth speaking to Michael Miller, who's particularly convenient because he's at University of Toronto. Uh-huh.

You might talk to- Right, we talked about him yesterday? We did, yes. Okay. You might talk to Doreen Fraser, who's at University of Waterloo. So also not someone who's very far away. You might talk to David Baker,

who's given a lot of thought to quantum field theory. He's at University of Michigan. Well, now- Noel Swanson, Noel Swanson would be a good person to talk to, he's at University of Delaware. Hans Halvorsen at Princeton University. Both Noel Swanson and Hans Halvorsen work primarily with algebraic quantum field theory.

So they'd both be interesting people to talk to about how we think about quantum field theory. So I have a lot of people I would recommend. Okay, now that I've accidentally, inadvertently teased the audience about interpretations of QFT, why don't you outline some of them? Why don't you outline two of them? Right. Right. So quantum field theories live within the umbrella of quantum theory. So you have the same kinds of axioms. You have quantum states in some sense, although depending on how you formulate a quantum field theory,

you might prefer to work with what are called c-star algebras, we've talked about that, rather than Hilbert spaces, but similar conceptual basis for thinking about quantum states and some kind of evolution law and observables being represented by an algebra of, an algebra means a collection of mathematical symbols that are the observables. And you still have to use the Born rule to calculate things probabilistically,

or at least compute measurement averages of things. So the basic rules still apply. Some people make a terminological distinction between quantum theory, which is the more general idea, and quantum mechanics, which is specifically about particles. Some people use the word quantum mechanics to mean all of quantum theory. So you have to be very careful what they mean. When I say quantum mechanics, I usually mean quantum theory applied to models of systems of particles.

And I use quantum theory to refer to the more general framework of which quantum mechanics is an example and quantum field theory is an example and string theory is an example, right? These are all phrased within this larger formalism of quantum theory. So some of the foundational philosophical metaphysical interpretational questions that one considers in quantum theory are still present in quantum field theory. Quantum field theory doesn't let you get around the measurement problem. Most of the time in quantum field theory, you're practically speaking, if you're doing quantum field theory for practical purposes, for particle accelerator experiments,

you're mostly imagining some collection of particles coming in, in the far past, which for particles experiments is not really very far in the past. And then you consider some other collection very far in the past. Right. And then you consider some other collection of particles in the far future, again quote unquote far future, and you want to compute the, roughly speaking, the probability to go from one to the other. This is usually phrased in technical terms in terms of what are called scattering cross

sections and decay rates. You take these probabilities, you convert them into the kinds of quantities you would actually measure in a particle accelerator, but they're still based on the Born Rule. And you're still using the Born Rule to calculate those probabilistic predictions. And usually you're doing only one measurement and then you're calling it quits.

So you don't usually see the collapse anymore in the formalism. You don't collapse the quantum state and then take the collapsed quantum state and do something else on it. So the measurement problem is a little more implicit in the formalism for the most part. We call, by the way, the array, the set of all of the scattering amplitudes, the complex numbers that you then compute scattering cross-sections and to carry is that we call that that array of all of them, that data set of all of them,

we call that the S-matrix. And so a lot of the time in quantum field theory, you're trying to understand the S-matrix and compute the entries in this S-matrix or trying to study its analytic mathematical properties or whatever. So the standard problems are still there, even if maybe they're a little more suppressed,

given the kinds of things we usually work with in quantum field theory. Students are often surprised when they do quantum field theory that they're not using the Schrodinger equation very much anymore. So a lot of those older problems, like I said, are just less manifest. But in addition to all those problems, there are new problems that show up. Quantum field theories are phrased in a manifestly special relativistic language and there are there are new kinds of problems that show up there. In particular, if you do want to think about doing multiple measurements on a quantum field theory, well then you can't have one measurement be in the infinite past and one

measurement the infinite future because if you want to do another measurement, you can't do something after the infinite future. You have to actually take seriously that things happen in finite amounts of time. And now you're talking about doing measurements ostensibly in some kind of space-time picture. You're thinking about your measurements being events, in some sense localized in space and in time, in a relativistic context.

And so now you have to be very careful that you don't get paradoxes and violations. And there's a whole set of papers where people are trying to make sure they understand how measurements work in this sense of quantum field theory. And so this is a whole collection of things that people might want to look at. I can, after we talk, I can send you some references that you can put in the chat. But that's a source of real serious discussion. Another set of questions is there are different ways to formulate quantum field theories. I mentioned algebraic approaches, things that for, I mentioned a couple of people who work

on algebraic approaches to quantum field theory. Then there's what some people call Lagrangian quantum field theory, which is also closely related to effective field theory. We talked about earlier thinking about our quantum field theories as progressive approximations, where we're really just interested in computing S matrix elements for the most part. And understanding the connection between these different formulations of quantum field theory is an outstanding question. At present, we don't have an algebraic, algebraic quantum field theories are intended to be formulated in a mathematically rigorous way.

We don't have an algebraic quantum field theory version of the kinds of quantum fields that we use in the standard model. The standard model is our most successful scientific theory. That's the Standard Model that has the quarks and leptons and gluons and photons and the Higgs boson that we think gives us our best, most quantitatively precise description of fundamental physics. That's phrased in the language of effective and Lagrangian field theory, which is not for the most part mathematically rigorous, and for which it's very difficult to answer some of these foundational questions I mentioned.

Algebraic quantum field theory is much more mathematically rigorous, and we have more resources, mathematical theoretical resources, to probe some of these foundational questions. But because we don't know how to formulate the standard model in that rigorous algebraic language, this limits our ability to address some of these foundational questions for the kinds of realistic, interacting field theories that we work on. And so one area in the foundations of quantum field theory

is to try to bring these two pictures together, try to see to what extent we can understand how we use quantum field theory in practice, but in a somewhat more rigorous way that's more amenable to addressing foundational questions. One other question you can ask is, what is quantum field theory saying is actually real, real in the physical or metaphysical sense? We're talking about reality. Are we saying that fields are physical real things? That at every point in space there's like a physical intensity or directional or more complicated entity that's like really there

and that there are real patterns in these fields across space-time? Or are we saying something else? Now, prior to having some kind of realist approach to quantum theory, it would have been very difficult even to ask that kind of a question. But even once you think you have maybe a realist-oriented approach to quantum theory, like the kind I've advocated, or like, for example, Bohmians who are trying to do quantum field theory are trying to advocate, or Everettians or whatever, you then have this further question,

what do I take to be the ontology? What should the real thing be? And so one set of debates was, should we think about quantum field theories more in terms of particles or more in terms of fields? And some philosophers argued that thinking in terms of particles was untenable for a whole variety of reasons. Others have argued that thinking in terms of fields is also untenable.

If you talk to practicing theoretical particle physicists, you'll hear a variety of answers in that question as well. You'll even hear from some top people working in quantum field theory that we don't even understand exactly what quantum field theory is in some fundamental sense. So that's just scratching the surface. We could then go farther.

What happens when you put quantum fields not on flat special relativity type space time, Kowski space time. What happens when you put quantum field theories on a curved space time? Even fixing, freezing gravity, not letting gravity be dynamical, freezing space time is in particular shape. What happens to quantum field theory when the space time is no longer flat? What happens when you're accelerating in space time and you have things like the Unruh effect and you see the appearance of particles that show up? I can recommend to people who are interested in learning more about that

particular problem and learning more about sea straw algebraic formulations of quantum field theory to read a remarkable review, part review paper and also part philosophy paper by Rob Clifton and Hans Halverson called R. Rindler, there's an N in there, Rindler, R. Rindler Quantareel. And in addition to addressing that question, which they do in the later parts of the paper, the beginning of the paper is a really nice introduction to the C. Stier algebraic formulation of quantum theories generally, which they use to study this problem. And then, you know, within algebraic quantum field theory, you run into all these deep questions about unitary

and equivalence of different representations of quantum field theories. So, I mean, I could go, I mean, there's just so many interesting problems in the foundations of quantum field theory. And I know that people who work in the field are listening to this and are probably upset that I'm leaving something out. There are just too many things to mention. But I'm only leaving them out because of lack of time and because they're not occurring to me in real time.

I mean no disrespect at all, please. So I have a two-part question. Between measurements, are particles popping in and out of existence? Now, I know we've spoken about that on part one or part two, but I still want you to go over it. So that's half of my question. My second half is in many of the approaches of quantum gravity, we sum over different space times.

So between measurements is space time also fluctuating. Particle counts fluctuating and is space time fluctuating between measurements. What does it even mean for space-time to fluctuate? So, in non-relativistic quantum theory, we're usually interested in systems of fixed numbers of finitely many non-relativistic particles, and in that case, the particle number doesn't fluctuate. And if you model this sort of a thing in the indivisible stochastic approach, you get arrangements of particles, and those arrangements of particles are like snapshots. And those snapshots can look very different as time goes by.

They're probabilistic. But you would have a conserved number of particles. In a relativistic context, the number of particles is not generally conserved. Now, there are conservation laws you have to worry about. For example, electric charge is conserved, and that means that if you have charged particles, then if particles appear, antiparticles should appear so that electric charge is conserved. And that means that if you have charged particles, then if particles appear, antiparticles should appear so that the charge is conserved or something like that.

You have to be mindful of those conservation laws, but there's nothing in principle that stops particles from appearing or disappearing. Usually in the context of relativistic quantum theory, we're usually thinking about something something like a quantum field theory. That's usually the most natural way to handle relativistic quantum theory. There are first quantized approaches to relativ quantum field theory. That's usually the most natural way to handle relativistic quantum theory.

There are first quantized approaches to relativistic quantum theory. That's another thing I picked up from string theory because we usually do a lot of string theory working with the first quantized formalism, thinking in terms of specific numbers of strings. Although I should say that the number of strings can change because strings can break and attach.

We're thinking in terms of strings rather than fields made of strings. You can apply that relativistic first quantized formalism to particles and you can model particles with no intrinsic spin. You can model particles with various kinds of spin in a formalism that's very reminiscent of what one does in string theory, but applied to just point particles. So that's another interesting thing. But usually when we're thinking about reminiscent of what one does in string theory, but applied to just point particles. So that's another interesting thing. But usually when we're thinking about relativistic systems,

it's often easier to use quantum field theory. And in quantum field theory, particles, as we've talked about, are kind of emergent entities. You can excite the fields in different amounts, and when you excite them, you have different numbers of particles. And because the fields can be excited in different patterns, and the excitation patterns can change with time, particle number in general is not conserved. So particles in that sense are emerging as

fluctuations of the field as functions of time. So in the case of particles fluctuating in an additive existence, in some sense that's allowed in the relativistic case. Now in quantum gravity, here we're entering the realm of speculation. We don't have a fully realized theory of quantum gravity by any stretch. Arguably there has been some progress made in handling questions in quantum gravity by various techniques. One technique used to handle difficult problems in quantum gravity is functional integral techniques.

We use the path integral or partition functions, various things that capture quantum features of what we think is the right way to think about gravity, but not directly using the usual tools of Hilbert spaces and operators. There are even some circumstances in which things that might be very harder or very difficult to see in a Hilbert space oriented approach become easier to calculate or see in a functional integral type approach.

Even to the extent that there may be some situations in which the functional integral approach takes you to places you can't get to from the Hilbert space approach, which may reveal some very important things about whether quantum gravity is really meant to be based on Hilbert spaces at all. Personally, I'm suspicious that quantum gravity is ultimately going to be phrased in terms of Hilbert spaces for a whole variety of reasons, one of which is that Hilbert spaces seem to be really well adapted to the case where you have an external time parameter. There are ways to do quantum theory with Hilbert spaces without an external time parameter,

the Page-Wooders formalism, which I'd rather not get into in detail here, but it's a bit difficult to work with Hilbert spaces when you don't have an external time parameter, and in general relativity, you don't have an external time parameter. So there are a lot of reasons why you might be suspicious that Hilbert spaces are the right ingredients,

and so we use these sort of functional integral techniques to make certain kinds of predictions, to calculate things, to answer certain kinds of problems. And in these functional integrals, you can think of them as you take whole spacetimes, you assign whole spacetimes a complex number in amplitude, and then you sum these complex numbers associated with different spacetimes, and then you do various squaring of the answers

to calculate things. The metaphysical status of these kinds of operations is very murky. I don't have an intuitive grasp of what it means to add together two space times. Of course, I don't really have a grasp of what it means to add together two particle states. In the indivisible approach, we don't really have superpositions in a literal sense.

Our use of superpositions in the Hilbert space mathematics is just a way to encode the indivisibility of what's going on in the process. The particle really is in just one place or the other. So arguably, if you wanted to think about quantum gravity in an indivisible stochastic sense, you wouldn't really be summing over, you wouldn't really have true superpositions of different space times. There might just be one space time or something that space time emerges from, some other kind of more fundamental substrate out of which space time emerges.

And our use of sums of complex numbers over space time is no more telling us what's going on metaphysically than would be the case in the use of the path integral formulation of a quantum mechanical system of particles. But this is all very speculative. I don't feel like I have enough of a grip on what's really going on in quantum gravity to be able to give anything like a concrete metaphysical picture of what I think is going on. I don't think we're just at that stage yet in the development of quantum gravity. So what does it look like for an indivisible stochastic process

to pop in and out of existence? How do you model creation and annihilation of particles in your approach? So one thing is to take your configurations not to just be arrangements of a finite number of particles, but to broaden what you mean by your configurations. Now your configurations are not just arrangements, but also any number of particles in those arrangements.

Configurations with no particles, configurations with one particle, configurations with two particles, but in all kinds of various arrangements. When you do this in the Hilbert space sense, this is called the second quantized formalism. The Hilbert spaces you use are called Fox spaces. So one could model it that way.

Another way to model it is not to treat particles as fundamental, but to work with fields. And fields don't pop in and out of existence in the usual sense. The fields are just there, and they can be activated in various patterns, and some of those patterns we regard as emergent particles. So rather than seeing particles coming in and out of existence, we replace particles with a more fundamental substrate that is not winking in and out of existence.

Now, I should say there are some really interesting directions where people have considered third quantize theories, or higher order quantize theories in which they imagine, what if quantum fields can pop in and out of existence? It's a more profound sense. Okay. People who are interested might imagine what if quantum fields can pop in and out of existence? Let's have more profane sense. Okay. People who are interested might want to learn a little bit more about that. Just Google third quantization or higher order quantization, you'll find some

interesting things written about that. Now I don't want to claim to be an expert in those sorts of formulations. So I won't say more than to just maybe entice the audience to learn more about it. those sorts of formulations. So I won't say more than to just maybe entice the audience to learn more about it. You're a master of physics, of philosophy, and of history. You may even be... I think you're assuming way too much about me, Kurt, but I appreciate the compliment. You may even be a part of that history. So let's imagine 10, 20, 50 years from now there's a history book, there's a chapter on Jacob in this history of physics book. What do you want it to say?

That I was a really nice person to people I cared about and to people like everybody. That I tried to make the world a better place, that kindness was important to me, that I created spaces for the people I care about to feel safe and nurtured and loved. I mean, that's the thing that immediately jumps out at me. I mean, what more could anyone want to aspire to? I mean, of course, we all fall short of it.

I fall short of that all the time, but that's my hope is that I get better at that. And that's, I mean, I don't know if history books talk about those kinds of things. I hope any book that talked about me would talk about that. In terms of contributions to scholarship, I would hope that the projects I'm working on end up panning out.

Anything we work on could ultimately not succeed. I'm not claiming to have a theory of everything. My hope is to address what I think are some important problems at the heart of deep and foundational questions in physics. I hope that my work ends up succeeding and that the history books say that they succeeded, or at the very least inspired people to go in new directions that they might not have gone in otherwise. might not have gone in otherwise. You know, I hope that, well, I mean, I hope they talk about my amazing family. My family is amazing and I love them and I hope,

I don't know, it's a hard question to answer, I don't know. Jacob, thank you for spending so much time with me. It's always a delight, Kurt. Every time I come to Toronto, I'll always come and we'll talk more. And you have to come visit me in Boston again. It's a deal. Take care, man. You're a real gem, you know that? I'm a gem.

Talk about master. Like, you are a master at what you do. Hi there. Kurt here. You are a master of what you do. week, you get brand new episodes ahead of time, you also get bonus written content exclusively for our members, that's C-U-R-T-J-A-I-M-U-N-G-A-L dot org. You can also just search my name and the word SUBSTACK on Google. Since I started that SUBSTACK, it somehow already became number 2 in the science category. Now, SUBSTACK for those are unfamiliar, is like a newsletter. One that's beautifully formatted, there's zero spam. This is the best place to follow the content of this channel that isn't anywhere else.

It's not on YouTube, it's not on Patreon. It's exclusive to the Substack, it's free. There are ways for you to support me on Substack if you want, and you'll get special bonuses if you do. Several people ask me like, Hey Kurt, you've spoken to so many people in the field of theoretical physics, of philosophy, of consciousness. What are your thoughts, man?

Well, while I remain impartial in interviews, this Substack is a way to peer into my present deliberations on these topics. And it's the perfect way to support me directly. KurtJaimungal.org or search KurtJaimungalSubstack on Google. Oh, and I've received several messages, emails, and comments from professors and researchers saying that they recommend theories of everything to their students. That's fantastic. If you're a professor or a lecturer or what have you and there's a particular standout

episode that students can benefit from or your friends, please do share. And of course, a huge thank you to our advertising sponsor, The Economist. Visit Economist.com slash Toe, T-O-E, to get a massive discount on their annual subscription. I subscribe to The Economist and you'll love it as well. Toe is actually the only podcast that they currently partner with, so it's a huge honor for me, and for you, you're getting an exclusive discount. That's Economist.com slash To, you're getting an exclusive discount. That's economist.com slash toe. And finally, you should know this podcast is on iTunes.

It's on Spotify. It's on all the audio platforms. All you have to do is type in theories of everything and you'll find it. I know my last name is complicated, so maybe you don't want to type in Jymungle, but you can type in theories of everything and you'll find it. Personally, I gain from rewatching lectures and podcasts. I also read in the comment that toe listeners also gain from replaying.

So how about instead you re-listen on one of those platforms like iTunes, Spotify, Google Podcasts, whatever podcast catcher you use. I'm there with you. Thank you for listening.