Latent Space: The AI Engineer Podcast - 🔬 An Oscar, Two Asteroids, and the Algorithm in Your sklearn: John Platt on AI for Science

Episode Date: September 22, 2026

How often do you get to talk to a guest who has both an Academy Award and who invented textbook machine learning algorithms? John Platt has an Oscar, two textbook algorithms, two named asteroids, and an Erdos-Bacon number of 6. This was easily the most fun bio of all the guests we’ve read to date. And the result was an epic and fun chat covering Google’s Empirical Research Assistance (ERA), how AI can help battle climate change, and tons of great stories about the co-evolution of science and AI.John’s colleague Dave Bacon likes to tease John that his career has been defined by being twenty years early to the next big thing. This may be convolutional neural networks (some credit him with coining the term), fusion research, quantum computing. John and Google have been working on solving some of humanity’s hardest problems with AI and computation for well over a decade now. Recently John and his team set their sights on using AI to solve any scientific problem that can be written down as a score.Google’s Empirical Research Assistance (ERA)John’s team has taken on many hard scientific problems over the years. In solving these, they noticed a pattern, many scientific problems can be reduced to what John calls a “scoreable task”. Once you have the score function, the goal is to find some code that maximizes the score. The hard part is in formulating the score, but once you have the score finding the maximizer can still be quite a lot of effort.John’s team set out to automate solutions to this general problem. This came out of the idea of an “auto-Kaggle” AI, which can solve any Kaggle problem you can throw at it. Kaggle is owned by Google, so all the data was ready and easily available to them!The result is Google’s Empirical Research Assistance or ERA (paper, github, blog). ERA is surprisingly simple conceptually. Gemini (or your LLM of choice) keeps a running tree of past experiments (notebooks) and where they’re going. It’s a close cousin of Monte Carlo Tree Search: at each iteration the Upper Confidence Bound rule picks which notebooks are most promising to mutate. This is optimistic, not greedy, so sometimes even the fifth-best notebook gets chosen. Gemini then proposes mutations for each one, about ten at a time. The history of each branch is shared, so different leaves can learn from each other.“It’s almost like having a hyper-eager grad student who doesn’t sleep.”Evolutionary algorithms have been around since the 70s, but this works because Gemini actually knows where to look! What’s even more interesting is that there was a step change between Gemini 2.0 and 2.5, and this went from just not working to working great.ERA is so powerful that John and his team solved many outstanding problems with it, resulting in at least ten papers. Some of these were climate change related, which we talk about in the next section.So, we had to ask: if you have an optimization god how do you avoid fooling yourself? John’s answer is that ERA provides predictive models. It’s up to the scientist to make sure they’re truly descriptive. Some of this just involves good old-fashioned careful machine learning science. “It’s a power tool. It can slice your fingers off.” This led to some fun discussion about Kaggle competitions, and the fun ways people can overfit to datasets without meaningfully solving the problem you actually care about: Google’s contrail-detection competition was won by entrants who noticed a half-pixel error in the labels (is the origin at the corner of the pixel or the center?) and this turned out to be a part of the winning special sauce. Great for winning $15,000, not so helpful if you actually want to solve contrails.“People themselves will act like these LLMs and try to reward hack. It goes back to Goodhart’s law: any metric that becomes a target is no longer good as a metric.”His advice for where to start instead?“Always just fit linear regression. Just do it. Just do it. Just do it. Or SVM.”Tackling Climate Change with AIJohn and his team have worked extensively to mitigate the effects of climate change. We talked about several of their initiatives.Perhaps the most interesting result we talked about was reducing the effects of condensation trails (contrails) from airplanes. Those little streaks you see running behind planes somehow account for 1% of all human-induced global warming?!? Some of these trails of ice crystals can hang out for days. These crystals are black in the infrared, acting like a thermal blanket that traps heat day and night.It’s easy to understand what’s happening here, a region of atmosphere becomes “ice supersaturated”, and a tiny bit of exhaust seeds water vapor that instantly crystallizes. The scale here is astounding, with a single gram of exhaust resulting in ten kilograms of ice crystals.The solution to all of this is quite simple, in principle! We know what parts of the atmosphere are most likely for the trails to form. Just have the planes drop a flight level or two. Problem solved, right? Well, the hard part is accounting for how much warming was prevented. This is a counterfactual problem, parts of which stumped John’s team for over two years. They had a working model for the heat-trapping half, but not for the reflected sunlight. ERA was able to find a simple model with some confounders they hadn’t considered. Cracked it!Modeling climate generally is a hard problem. Climate is best thought of an attractor of many different possible weather outcomes. This makes it much harder to model.“Weather is where you are on the attractor, and climate is the statistics of the attractor. The problem with climate is that we’re altering it. The attractor itself is changing, it’s moving.”John and his team have worked on treating both the symptoms and the disease of climate change, with several other works in the area. Another fun example we briefly cover is FireSat, a way of using a constellation of satellites to rapidly identify fires before they grow too big to put out. For anyone living in California, you understand the problem. In dry years a small fire can result in hundreds of thousands of acres. If you could find this fire when it’s the size of a room, it could be put out. By the time it hits an acre we have a much harder problem.Where is this all going? Looking forward by looking backBy now it should be clear John has an incredible and unique view over the intersection of science, computation, and AI. John talked about a class on physics of computation he took with Richard Feynman back in 1982. This was when quantum computing was an ill-defined concept with no theory or experimental backing. John recalls every Tuesday was a guest lecture, and every Thursday was Feynman explaining why the Tuesday guest was wrong. John also recalls doing science back when there was essentially no compute, a million operations per second was cutting edge.What is John’s recommendation: the most important skill is developing deep domain expertise. There’s no other way to develop taste than to tackle hard problems. One surprising part of this is that John recommends spending time doing things the old fashioned way. Play with tools, and just implement things yourself.“You could drive up the mountain, or you could hike up the mountain, and maybe it’s okay, even fun, to occasionally hike.”Summing it up, John’s message to the audience is that there will still be a place for scientists, and that if anything it will just open up more opportunities for “the creative stuff, the rigorous stuff, the philosophy stuff.” But don’t forget to spend time doing the grunt work.“There just seems to be this strong impetus in the world to optimize and squeeze everything out. But you do lose something when you hyper-optimize. It’s overfit.”And whatever tools you end up using, John’s advice is the same one Feynman gave him forty years ago: you must not fool yourself, and you are the easiest person to fool.We had a great time talking with John. We hope you enjoy! Also in this episode* Fusion is three years away, not thirty, if you ask John. And why the Lawson criterion means every fusion approach has an Achilles heel.* Why superconducting qubits are still finicky.* The asteroid he named after his mom, which turned out to have a moon.* The looming helium shortage nobody talks about.* How NeurIPS started as people crashing a private workshop at Snowbird, and why Hopfield networks are all you need.* Being Carver Mead’s sysadmin on a VAX with an 80 MB disk the size of a dishwasher.* Finding asteroids in 1985 with film, a stereoscope, and a letter to Brian Marsden. The Vera Rubin Observatory found 11,000 in six weeks.* The Feynman effect: total clarity in the room, none once you leave.* Quantum echoes, the NISQ era, and why he thinks quantum is neither thirty years away nor tomorrow.* A startup that wants to inject mercury into a fusion reactor and sell the transmuted gold. “It might not work.”* John’s 20% time rule for his own group: do stuff for learning, and you don’t even have to tell him what. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe

Are you talking about introducing explicit priors that, you know, she based upon some human intuition or maybe in this case, LOM intuition? When you talk about multiple hypothesis testing, right, there's predictive models and there's descriptive models. A predictive model is like, let's say you just have a, you have some inputs and you have some outputs. And you just, I just want to build a piece of code that tries to just have the lowest error rate on some data set.

Dysistical model. A descriptive model is actually what science is trying to. to get to, which is, okay, it should be able to extrapolate because it has sort of the physics or the actual some description of reality that's captured within it. And then you can use it to extrapolate. Yes, Newton thought of apples and gravity, but gravity isn't actually about Apple right. If you take a 17th century machine learning model, like, oh, apples will fall, but how about planets? You know, I don't know. I have no data about planets. So who knows what they do, right? The distinction between those is a little bit blurry, right?

Because when a physicist or scientist comes, they use their intuition, or maybe even more than intuition. Like essentially there's maybe a solid pile of facts that they know about the world. And then they make sure that whatever model they build is sort of consistent with what's known. Welcome to Latenspace Science. I'm Brandon, joined by my co-host, RJ. It's a pleasure to have John Platt, you know, with us today. John is a Google fellow and head of Applied Science at Google Research.

He has really a fun background to, I guess, you described yourself when we're talking a few months ago as a mega nerd. Oh, Giganer. Giganer. Ginger. Gingard. You're excited and absolutely everything. And it really shows, yeah, you, was correct me if I'm wrong about any of this stuff. So you started college at 14 and started your Ph.D. at 18 at Caltech.

You were advised or co-advised by John Hotfield, right? Oh, yeah. Yeah, yeah. who just won a Nobel-Frizen, you know, two or two years ago. Yes. John created several, it was responsible for several textbook algorithms, one known as Platt Scaling, another one, sequential minimal optimization, which is the textbook algorithm for training SVMs, even today.

It's still, if you use SK Learn, it's there. John is discovered and named two asteroids, has a Oscar for technical developments from 2006. So if you've ever watched a Pixar movie, you've seen John. John's algorithms in work. John has an aeros bacon number of six or three, three and three from either side. And I'm going to skip over like 20 years of your career, but then jumping into Google, working at Google Sciences, you've worked on fusion, quantum computing, climate modeling, and many other topics. Is that more or less right? That's right, yeah.

Okay, okay, cool. Do I miss anything important for today? No, I mean, I've also done, you know, lots of applied math and signal processing. and all sorts of fun things like that. Yeah, yeah. I think you also, your Wikipedia has a fun story about patents and the iPhone too. The iPod. The iPod. Yeah, yeah.

Yeah, welcome. Thank you. Thank you for having me. Can you tell us about the era is the, I think, the way that the acronym is pronounced. And I know that there's a lot of different semi-related stuff out there, both within and outside of Google. So what can you tell us a little bit? about the details of error and what makes it special? Well, we've been doing sort of AI for science in Google Research for more than 10 years now.

And around 10 years ago, it was very much using, I don't know what you call it now, maybe classical machine learning models, you know, things like, you know, convolutional nets or whatever, and they were specific models to build to, you know, solve specific science problems. But about two years ago, we got very, excited about these more general LMs that have popped up in the last few years, and we were

wondering what can be done with them. And of course, a lot of people have been playing and trying to figure out what the right thing to do is. And we kind of stumbled into this mapping. And there's found that many different scientific problems

can be mapped into something we call scoreable tasks. So you can often phrase a scientific problem as, gosh, I really would like to have a piece of code that maximizes some score. And it's surprising the number of different sort of scientific problems you can make a lot of progress on by mapping into that framework. Well, one thing is a lot of scientists spend a lot of time sort of building models. They might be statistical models, or they might be, you know, physically based models. And if it's a statistical model, like in machine

learning, your scoring function is, well, I have some data set, and I'd like to have the fit, the model, on the data set go up. And we can talk about overfitting in a minute. But that was whatever question. And that's actually very interesting. So machine learning is kind of a subset of this sort of scorable task, right? But you could do other kind of things like especially Michael Brenner, who's the lead author on the era paper. He's very, very skilled because he likes to sort of knock out a scientific paper in an evening now with a tool.

So there's something in applied math called asymptotic expansions, which is you're asking, how does the ordinary differential or partial differential equation, but say ordinary differential equation behave. There's some parameter that has an epsilon in it, and you're trying to say, how does it behave as epsilon goes to zero? And it turns out you can turn that into an empirical task by essentially asking that it proposes some solutions

that are asymptotically correct. And you check to see if the asymptotic solution is correct for like epsilon equals 1e minus 4 or something. And then you check that fit, but then you ask you ask Gemini, which is the core AI underneath it to do the mathematical reasoning, try to solve the problem while also maximizing the fit to the data. So you can actually, there's a lot of sort of tricks you can do because it's not that the underlying thing that's altering the code or the underlying thing that's sort of making the decisions is not. a random process. It's an AI itself that is smart and knows about things and knows a lot about the world. You can get a lot of, solve a lot of interesting problems because that sort of core inner loop is an AI that has huge amounts of prior knowledge. So that's sort of the trick. So we've been running around trying to map lots of scientific problems into scoreable tasks

and trying to solve them. And it's really been kind of fun. And I'm happy to talk about the ones that I've been involved in at least. Yeah, I would love to hear about some of the more. So it's a statistical one is what everyone listening will probably know about what you just mentioned makes sense. What are some of the other interesting ones? Let's see, that are not statistical. Let's see, because we have interesting ones like one that we just put a paper up on

archive is, or actually I think it might be on GitHub. It's you often run into this in remote sensing because there's always a tradeoff. There's satellites flying above the earth. and there is a trade-off between how frequently they can revisit a spot in the earth, what their spatial resolution is, how big the pixels are, and their spectral resolutions are how many bands they have. And ideally you'd like to have a monitoring of the Earth that's constant and, you know, a frame every five minutes at hyperspector resolution,

at whatever, 10 centimeters. You can't get that. But for, for example, to monitor CO2, the atmospheric concentration of CO2, you can take data for one satellite. That's, for example, it's OCO2 or OCO3. OCO3 is actually attached to the International Space Station. So it gets you like a little strip of CO2 measurements that are highly accurate and pretty high resolution.

You can actually try to do, because a lot of it is in the infrared, weather satellites like Goz has some infrared bands, And it takes a picture every essentially five minutes, but the pixels are very large, and it doesn't have such great spectro resolution in terms of it wasn't designed to find CO2. So you just ask one to estimate the other, and you shovel other data in, like,

what's the current weather, what's sort of the long-term, you know, albedo? And so ERA came up with this very nice model that can do almost like super resolution, an informed super resolution of one satellite to another. So that's like one example. Yeah, yeah, okay. So any scientific problem that you can map into this framework,

so the input to error is sort of this mapping, and the output is code? Is that? Well, sort of. I mean, the input is you, the way we've got it set up in the product is you just start talking, right? And so because a lot of times it's non-obvious to how do this mapping, although experts like Michael Brenner know how to do it. So we actually, a rotan agent that actually helps you, it sort of talks to you to try to help you define what your scorable task should be.

So already there's sort of an instance of Gemini sitting there trying to help you write a code. So that's actually sort of almost like an intermediate result. You start talking to it about your problem and it tries to produce essentially a Python notebook. underneath that that has a score essentially a function with a with a scorable which essentially produces a score and then it starts to mutate that notebook in a clever way because again it's Gemini and it will try to sort of keep proposing code that tries to maximize the score so what is different about this and just a generally a general agentic system that can sort of optimize notebooks right now it's a

it's essentially its own in modern 2026 parlance we actually worked on this in 24 and 25 but the modern parlors is kind of a specialized harness that runs an algorithm which for era was Monte Carlo tree search so essentially it's keeping hundreds or thousands of possible

instances of notebooks and then it selects one I can explain how it selects one and it decides well how can What can I do, Gemini asked itself, what can I do to make that notebook be better? And then it will make a new one and test it and then put it back into the candidate pool. So you can imagine the candidate pool is actually tree structured because it's, you know, every candidate possibly has some children. And what you do is you pick based on something called the, it's actually a fairly standard algorithm in a firm reinforcement learning called Upper Confidence B.

So you essentially pick, it's an optimistic algorithm. So it tries to estimate, say, what's the 95th percentile outcome of mutation? It tries to estimate that. And it picks the one to the highest bound, the highest optimistic bound. So it doesn't always pick the best performing notebook. It tries to predict like what's the current performance plus two sigma of its guest. So it's always trying, so it hunts around. So it's high recall, basically.

High recall. It's trying to make its bet. so that it most efficiently tries to make progress, which isn't always greedily doing the best candidates. Sometimes it's the fifth best. We've also played around where it kind of recombines, it sort of takes ideas from two candidates and smashes them together and tries to make a third candidate out of that.

How does it seed the initial candidate pool? Well, that's the amazing thing, is that underneath Gemini is actually good at writing code. I mean, you just ask it, write me a thing, because you have a textual description of the problem. It isn't just, oh, here's your scoring function, start. You say a textual description of the function. And you might give it, in fact, we have under some things,

like here are five papers that people try to solve this problem with. And it's kind of smart. It actually goes and reads the papers and it will actually take a first stab at code. It might not be great, or it might, you know, sometimes it has bugs and it returns essentially minus infinity. but it will then try to mutate the code and say, oh, it'll try to make it be better.

So it's pretty cool. You don't actually have to give it, I mean, you can if you want, give it a starter code, but you don't have to. How many agents are you spinning up, I guess maybe not agents or how many different, you know, tree branches are you spinning up at each iteration? Oh, at every iteration? Well, there's a trade-off.

You'd like to do a lot of parallel work, but if you do too much parallel work, you can't learn from previous things. So right now we use about, the default is 10 parol. So you try to grow 10 leaves at a time. Okay. That seems to be about the right tradeoff.

When you say you can't learn from previous iterations, that means that the orchestrator, or is there some sort of, yeah, what's the, that you said that there is some step which is able to recombine or make decisions beyond just like the score. I mean, so yeah,

I guess maybe one of the opposite questions is, as a human, when you are doing some sort of a ML project, you don't just look at, like, oh, there's this one metric that we're trying to optimize. Oftentimes there's, like, orthogonal metrics, sometimes even insights such as, like, just watching, you know, training curves can sometimes give you intuition about what's going on or, like, looking into specific examples. Does it do any sort of introspection like this? Is there... Well, it has the history of... But by I meant why you can't do too many things. things in parallel is if you have 10 parallel searches at once, then what the number one can't

actually see what numbers two through 10 are doing. So if you do a thousand at once, then you're using a huge one of computation without a lot of cross learning. Whereas once you finish a little batch, you get the history. It's sort of obviously you have to prune it so it doesn't blow up the context, but you get the history of what it was thinking about as it was kind of writing the code and the results of the code. So it can learn from its previous attempts. Okay. And does it learn across?

Oh, yes. Yes. Essentially, it's essentially it's like one essentially shared context. Yes. So it is sort of thinking as it goes along. It's not like it's a thousand different, completely independent branches. You're really pushing Jimenez like long context abilities here. And you have to do, you have to do the right management and stuff.

Yeah, yeah, yeah. Okay. Oh, that's cool. I mean, going back to RJ's question. The key point here being the first key goal is to, I guess, identify what specific score that you are trying to optimize, right? Sometimes that is, I agree, like kind of the hardest part of the problem. And so I find it interesting that I'm not sure I'd always trust my agent to do that part. That part seems like the more human task in the loop.

It is, and often you have to be careful. In fact, a lot of what you do, it's kind of, it's very meta, I guess, everything we do is very sort of high level. You have to make sure that there's no one common thing is you come up with a scoring function where the agent does or you do it together. And then the iteration finds a way to cheat or whole. Like, oh, no, I didn't mean that. And so you have to go through and often sort of play and have a loop around it where you kind of iterate. Like, no, no, no, no, I didn't mean that. or you have to tell it in its instructions, okay, you know, don't do this.

So, yeah, there's often iterations. And so even with a Gentic help, you don't necessarily get the right scoring function from day one. And in fact, it's really neat because, I mean, in the old days, i.e. 2024 or something, you know, a lot of grad students would spend a lot of time doing scientific software. And it's just so much effort to write code at all that you kind of try maybe a few things, are a few things that are very related, and then you sort of stop because you have to write your paper, you have to do your next experiment. This thing is kind of underneath, kind of relentless, because it keeps trying and keeps trying, keeps trying. And so the people who use it are now spending all

their time, almost at the right level, almost at the scientific creativity level. What does it mean to have a cost function? You know what I mean? And so that's almost like the essence of the scientific problem. You're not so much now in the details of, oh, oh, I have to import this CSV file, or have to get this database to work or whatever. You're now sort of thinking almost like deeply philosophically about your actual scientific problem, not down in the grungy group of worrying about databases. In fact, one cool thing the agent can do

is actually suggest data sets to you. Like, oh, have you thought about maybe pulling in this dataset and doing a join? And so it'll make suggestions about like, you know, datasets you can join with, which is kind of cool. Going back to what you said a second ago, in terms of agents love to hack things or reward hack, do you have any fun stories or interesting stories about where things were comedically run off the rails? Boy, I'm blanking. I know other folks have read into it. I don't know if it's, I don't know if I have enough details to sort of say to sort of express the comedy of it. But it does. You kind of get surprised. I don't know if I have any really concrete. Sorry, I'm blanking.

No, it's fine. Yeah, I always like to think of machine learning. It's like, it's sort of like the old genie stories. That's right. That's right. That's what you wish more because you're going to get it. That's right. And you have that, and that happens very much with this.

You have to be careful. But on the other hand, it has some knowledge. The nice thing is that sort of Gemini knows a lot about many things, sort of more than any one person can do. So it at least knows, especially if you point papers, point, you know, like here, there are five papers that try to do this in some way. So to some extent, it does have that genie feel,

but to some extent it also sort of does same things. This is why, remember the whole idea of sort of evolutionary coding, it's been around since the 70s. Everyone's loved to do that. Like, oh, let's mutate LISP code or whatever to do things. But the reason why it just hasn't taken off is that random mutation in code space is pretty much worthless. I mean, just like, well, just like DNA. It's sort of like, you know, most things are harmful. So here it's like, oh, no,

no, no, we can actually find, it sort of knows, sort of underneath and knows interesting gradients to try, which is why the thing works, that the underlying loop itself is nai. So yes, it can, it can maybe overfit and have funny sort of genie problems like you, Lutu, but it also has some a lot of sanity because it sort of it knows about the world and it it has world knowledge in it so. The paper though you were doing Gemini 2.5 and I think you know Gemini has advanced

quite a bit. Do you have metrics or have you, you know, this is a tool that you're continuously using and it sounds like you're improving. Yeah. And I'm wondering like do you internally have you seen like almost like a phase transition in how effective this tooling has been?

How dramatic has the improvement been over the last? Like I guess you're Oh, well, I mean, a year or two, amazing. You know, there's every, even every half version of, I mean, essentially, I think it would have been impossible under Gemini 2.0. Really? Yeah, I think so. It wouldn't have worked.

So you started at 2.5 and that was like just the thing. Oh, no, we've been trying to experiment with these things actually for a while. Okay. And things just weren't working. And then they started to work. And then now they're just amazing. So the progress on Gemini major versions has just been stunningly amazing. Yeah, I think this is an experience a lot of people have been having where things were just seemed impossible, whatever, are suddenly becoming magically useful, like really quickly.

And so if people are, I even say this to scientists, because there's some people like, oh, I tried whatever 2.0. And I didn't like, oh, yeah, that was a long time. That was a year ago. That was like a long. That was an attorney. It was a journey ago, right? Yes. And in fact, all the, all the, we even have one of the preprints where we've sort of combined.

era with anti-gravity and that you know the whole that whole harness of anti-gravity is pretty amazing too and it's that that's the one where you can sort of pull in lots of papers and it can write lots of code for you and so yeah is that publicly available or is that the the anti-gravity yeah yeah anti-gravity is certainly publicly oh sorry sorry yeah the the era plus anti-grapa not yet okay okay I find this area really fascinating because like you said it's been there's been some form of code mutation out there since the dawn of computer science, basically. The canonical problem is sort of the overfitting or multiple hypothesis testing problem,

I think, which is maybe a little bit better matched to the problem where you're basically, my hypothesis is now that this algorithm work. Now my hypothesis, and so that you run the risk that sort of it has exponentially exploded, right? Because now suddenly I have like these, it's like hyper, hyper parameters that I'm optimizing. and so that you have this explosion of state space that you're exploring, and so that it seems much easier to sort of overfit to a problem. What are your thoughts about that? Because on the other hand, empirically, my experience,

I even tried the sort of open source version of era. I kind of strapped it into clot, and it's running right now, so I can't tell you how well it's working. Okay, I'm curious. Yeah, I'll let you know. But I'm just curious to know this is a question that's been in my mind about just general AI for science. And so what are your experiences with this sort of on the, sort of on the ground?

I guess there's two questions sort of embedded in your question, I think, right? Because when you talk about multiple hypothesis testing, right, there's predictive models and there's descriptive models, right? You know, can tell I've been doing machine learning for a long time, statistics. That's actually a really good point, though. Do you mind explaining that? I'm not sure that's something that everyone, our audience would be familiar with. Right, especially in modern days. People are trying to sort of obscure the two.

If you started with LLNs, I'm not sure that distinction would be meaningful. That's right. So a predictive model is like, let's say you just have a, you have some inputs and you have some outputs. And you just, I just want to build a piece of code that tries to just have the lowest error rate on some data set. That's just a statistical model, right? A descriptive model is actually what science is trying to get to, which is, okay, it should be, to extrapolate because it has sort of the physics or the actual some description of reality that's captured within it. And then you can use it to extrapolate because it's sort of like,

you know, yes, Newton thought apples and gravity, but gravity isn't actually about Apple, right? If he just fit, if he had taken as a machine, you know, the 17th century machine learning model, like, oh, apples will fall, but how about planets? You know, I don't know. I have no data about planets. So who knows what they do, right? So when you say extrapolated, okay, I didn't realize we're kind of going on a tangent. No, no, no, no. I am curious. Okay, so when you say extrapolative,

so there's different ways I can think about this. One of them is, you said, a model of physics or model of the world. Are you talking about introducing explicit priors that, you know, she based upon some human intuition or maybe in this case, LOM intuition? Or are you talking about this is the physics is actually learned by the model or the underlying process of the world is under the model? Well, the distinction between those is a little bit blurry, right? because when a physicist or scientist comes,

they use their intuition and they were about, or maybe even more than intuition, like essentially there's maybe a solid pile of facts that they know about the world, and then they make sure that whatever model they build is sort of consistent with what's known. Currently in era, it is sort of, it is L.M. intuition. Essentially, that's what I was trying to say

about having a good gradient underneath, that especially if you point it at existing papers, it will try to build models that are kind of sane underneath because if it's, again, especially if you give it guidance, like, oh, be sure to incorporate this and this or look at these papers to get these things. So it will, there is a, you can introduce a bias towards certain model choices

and it will have a bias because it just, its own little sort of world knowledge is accumulated inside of itself in sort of pre-training. Can you give us examples of what that might look like? is it modeling something in a way where it's actually, there's different, for example, if you're doing something with partial differential equations, there's these formalisms people have like neural operators,

for example, or where you can embed a, you can encode a differential equation in some sense, or I think that's called physics-informed neural networks or something. It's like one for fluid and kind of PDE-type modeling systems, or for, let's see, molecular systems, there's oftentimes this idea about equivariance. I mean, if the model is like picking up on these, you know, tricks which have been developed in literature, or are they adding some weights for some physical prior or something?

Yeah, kind of what does that look like when it introduces like a physics, when it introduces some sort of, you know, knowledge like that? I don't know if I have enough data to sort of say, oh, you know, 73% of the time it does this. Okay, but especially when you pointed at existing papers, it will try to, in fact, it will do very well at adapting the methods that are described in the papers for the problem. In fact, it will do an amazing job. You can actually often just recreate a reverse engineer paper. That's, again, what Michael Brenner actually likes to do this. He'll say, oh, that sounds like an interesting paper. We did this actually for the, we had this thing. It was actually kind of a hack. I suggested this to Michael where there's this one MIT professor who he came up with some code to do, essentially,

if you have a rooftop, if you have a rooftop with a fixed area and you want to sort of maximize the amount of solar power you capture over a day sort of solar energy, you can build up, which, of course, captures more sunlight, and you can sort of build, you can have a design a widget, you know, involving mirrors or struts or solar panels to sort of stock at whatever are angles or sizes you like. And usually with a maximum height and then try to figure, let it sort of explore that design space. And I believe, Michael, we can ask him,

I believe it actually just, I don't think he actually installed the simulator. I think the code just reproduced the code because it has a coding agent inside of it. Just reproduce the code from the paper and sort of figured it all out. So yes, it's very, It's very good, especially if given a pointer to what other people have done, it's very good at kind of like, oh, I haven't seen it try at Equivariant modeling.

That can get very hairy if you know about Klebsch Gordon coefficients. It's pretty nervous. They're fun, yeah. So I don't know if it'll do the true equivariant stuff, but it actually knows a lot. I remember actually when Gemini 2.5 came out. I know this is not exactly about era, but I remember sort of thinking, oh, this is a new world when like the day 2.5 came out because he said, hey, Gemini 2.5,

can you write me some boosted decision tree code? And it did. And it worked. It just did, yes. And I said, you know, yeah, this is a new world. So, yes, I think it's, for it to loop back to your question, I think, yes, if you give it sort of guidance about, oh, you know, it's important to put this kind of thing in, it will.

And so it won't necessarily, at least not that we've seen, discover completely new, Like if you didn't know about Klepsch Gordon Go Fish, I don't know. You didn't know about something. It won't completely discover a new kind of physical model from scratch. But it will certainly, if you tell it about interesting constraints about the world that are known, it will certainly follow. I don't know if I answer your question. So there's still room for humans for the next year or two.

Oh, in fact, going back, I think there's totally room for humans because I don't know. I mean, we have co-scientist that tries to help you come up with sort of hypothesis. this generation, but really I still haven't seen sort of the creativity and the philosophy and the sort of the careful rigor. You totally need humans. I don't see humans going away. They can make strange suggestions. And I've used co-scientists actually for an interesting problem in geochemistry.

And I learned about a new kind of ion. I didn't realize happened in magma. But I, and so it'll tell you interesting things. and you'll learn stuff, but I don't think it sort of substitutes for human creativity. You know, going back, we were just talking about two years, you know, you had Gemini 2-20 to 2.5, and this was like you already saw a leap, and now it's been another year or two, and now we're at 3.5, and, you know, there's, you're saying this is working much better. I mean, whenever you look at a graph, you know, you can, if something looks like an exponential,

it can either, you can either be in a signaloid or you can be at the beginning of a takeoff, right? I guess every exponential turns into a signaloid dimension. Every exponential turns to disozoid. But the question is like, where are we on that? I mean, I guess I'm a big believer in sort of the whole jagged. The frontier? Yeah, the jagged frontier. And so certainly, at least what I see, I mean, I don't know what's going to happen a couple of years.

But yes, there's some big spikes out in jaggedness in terms of coding ability and just gathering knowledge and finding related things. And that's huge and wonderful, which I think is great for scientists. So far, it's kind of less in terms of rigor and, um, We can talk about things like the International Methyth Olympiad and meth in general. But in terms of sort of philosophy and creativity, I think it's still kind of not. And maybe it'll maybe everything, some people are saying everything's going to inflate and pass, but I'm still seeing a lot of very strong jaggedness. So I can see, well, maybe it'll get to be extra good at coding and extra good at fitting models

and extra good at making suggestions and things. But I don't know. So far, not. So far you need the humans. Yeah, I want to get back to the question about the multiple hypotheses testing. Oh, sorry, sorry. You'll never really be real. Sorry, yeah, yeah.

Totally love the tangent. Multiple hypothesis testing is when you make, have a descriptive model, and you're saying this is the way the world works, and you have a bunch of data, and you take a billion darts and you throw, and so you have to be careful. There's something called false discovery rate, right? And so the question is, is this finding descriptive models, or is this finding sort of predictive models?

And fundamentally, the scientist is there to make sure that whatever is saying is descriptive. We haven't been able to make a system so far out of these pieces that can really sort of discover completely new physics or completely new science. But this is sort of a power tool to help you discover completely new science.

So maybe I'm trying to unask your question about multiple. But I think this gets at the heart Yes. Yes. But then you're saying, what about just pure over? Okay, let's set aside. It's not trying to figure out a descriptive model of the world that's still up to the scientist.

But what about just plain old overfitting? Yes, you have to be very careful because it's a power tool. It can, I shouldn't probably say it can slice your fingers off. You know what I mean? You have to be very careful and you have to be very rigorous. In fact, now you have to be more careful and more rigorous to not fool yourself. You really, really need to be just excruciatingly careful about having. you know, very hidden holdouts you don't look at.

You have to be just super, super rigorous to make sure that you don't completely, because it is a total power tool. So the question to how do I not slice my fingers off is you need to use the same techniques, but be very careful with them? Yes. That's a very clear answer that I'm going to get before. Thank you. Yeah.

Yeah, I actually don't think I've heard any guests say that. Yeah, it is, I think, a very important skill, maybe one of the most important skills and the new. People talk a lot about taste, but maybe this is a variation of taste. The taste is like the other side, right? It's like the rigor. It's like the, yes. Yes. In fact, if anything. Yeah, taste or rigor, which one's more important?

Well, I don't know. I think people, at least the way I am viewing it is, I mean, the aspect of that researchers are software developers, which there's a lot of overlap in a lot of fields. I'm seeing that software engineers, are, it's almost like, obviously, there's a lot of concern like, oh, no, what am I going to do? You know, coding seems to be, you know, getting automatic. So I think there's sort of both. I think there's a lot of people get pulled into, well, I'll be the creative source. So I'll try to figure out new science.

I'll try to figure out new products. I'll try to sort of really, be very recreated. And again, I'm strong believed to that. I don't think that's going to go away. There's also people sort of pull towards rigor, like, oh, I want to make sure this doesn't that this doesn't crash. I want to make sure this scales. I want to make sure this isn't wrong.

I think you need both. And I think you need people who are really good at both. But they don't necessarily have to be the same people. But yes, I think you need, I think this is even broader than science, just as sort of software engineering evolves. Yeah, it'll be the people who will bring the creativity and the people who will bring the rigor. And I think those will be sort of anchors. Some other things that I've seen are related work.

out there, there's a really cool leaderboard for, you know, like Claw leaderboard, agent leaderboard for scientific problems from Stanford. I don't know if you're familiar with it. It seems like a really interesting idea to me to have, you know, sort of different agents kind of competing on the leader. So it seems like if you squint a little bit, what ERA is doing is kind of a leaderboard, but it's internal and it's recombining ideas. Whereas what are your thoughts about this? And is that like a thing that you guys are working on? And is there problems with that or advantages to that? Ironically, you know, the whole era project actually started

because people may not realize Cagle is actually part of Google. Oh, yeah. And so it was called the auto Caggle problem. So it was actually like, that's what it was. Let's try to have a system that can sort of, you know, win at Cagle competitions. So that's sort of why it sort of has the shape. That's sort of how the project started.

And it goes back to sort of overfitting, right? If you've ever actually competed in a Kaggle competition. I have done Kaggle competitions and I've done one. It is a really interesting phenomenon because there's this overfitting is like rampant. Yeah, yeah. And it's really impressive how people can overfit to certain datasets in a way that is, yeah. Or even we had a, we have a fun project I can talk about. or if you like

that tries to mitigate contrails. Yeah, yeah. Talk about that if you want. And we had a contrail cattle competition and people actually beat us, but they found that we had a half pixel error in our labels. And they had to do with the center versus the lower left. Like where is zero zero?

Is it in the lower left of the pixel or isn't in the center? You know what I mean? Yeah. So they found that and exploited that and squeezed to whatever, a little bit extra stuff. Because it turns out when you make artificial data and you rotated, you have to make sure that you take into account

that half pixel offset. So yes, people themselves will be act like these LMs and try to sort of reward hack on these things. So it sort of goes back to what is it, Goodheart's law? Yeah, Goodhart's Law. Maybe we may quote it though.

Let's see, let's say any metric that becomes a target is no longer good as a metric. Yeah. And so that's the, I mean, it's good. And it's just that you have to be, you have to be very, very careful. And you have to, again, you have to have like layers of rigor. It's like, okay, but we'll do this and we'll optimize for this.

But you have to realize, okay, that's just now Goodhart's Law applies and you have to be careful. And so, yeah, that's a lot of reasons why the whole AI field has been kind of constantly exhausting these things. Because, again, Goodhart's law applies individually to every, to every leaderboard you make. So again, it's sort of you just have to step back and be very, very careful. I mean, that's not. No, no, no, that's really useful to my thinking. This is, you know, as we've had guests on, it's been a recurrent theme of how do you manage all this, the complexity that's introduced by LLMs in agentic science? I think my follow of question was about overfitting in gaggle.

Yeah, it is if you, you had an auto-caggle problem and then, the question is given auto caggle, how often was it successful? I mean, assuming you probably just ran this on like all of your Cagle competitions or something. Well, we tried it on various, like what they call playground competitions, and it did very, very well in the playground competitions. We've entered into different competitions, some of them. It turns out there's in the last few years, just the number of leaderboards and competitions and whatnot have just exploded far beyond Cagle.

So we've done very well, and some of them, like one thing we're super proud, of is the whole CDC set up this competition where you try to predict next weeks the number of COVID and flu cases that will happen in every state and territory in the U.S. And you try to predict a week in advance. And ERA did super well on that. It's funny because in some sense, Google invented the concept of using data to track disease progression with Google flu. So it's kind of funny that you were sort of going full circle 20 years later.

or something like that. So that did very well. Other ones where we've entered, we weren't quite as good often because people are very, again, you know, you have to sometimes, sometimes how well you're doing these competitions is a measure of how much sort of TLC you put into it

and how much you're willing to squeeze the last point, 0.01. And so it was, I mean, Irid did well. It got you pretty close, but we didn't close the jump in the last, whatever, 30 places or whatever, because no one was there. to shave the last, you know, point bowl one off the things. Yeah, yeah.

Is it very iterative like you get to it, you know, it sort of, I saw the charts in the paper and, you know, you sort of get these step changes as it discovers something and then, and then, and then, so is it very much human in a loop like, okay, you've stalled on the problem, like try this kind of thing? Yes. Okay. Yes. At the outer loop, which is I think that's almost like the more fun, creative part.

So yes, oh, here, look at this paper, oh, you're doing something bad, or you know what I mean? So it's almost like having a hyper-eager grad student or something who doesn't sleep, and you sort of tell it things and you sort of guide it around. How often does someone intervene versus, like, what is the outer loop look like actually? It might run for a few hours and come back and give you some examples. and then you would, you know, you can do it. As far as you like, you can sort of keep trying and keep poking at it. So that's, it's very much designed to be human in the loop then. Yes, yes.

Interesting, because a lot of the other tools that I've tried tend to be very one-shot. Well, I guess it depends on your definition, right? I mean, it's obviously you talk to it and you start it and it'll go for some number of hours and come back. But, of course, then you say it, but then, then, then, That's where the human creativity kicks in. And then you're sort of doing the outer loop where you sort of every, you know, depends if you want to sleep, but you know, every few hours you go and you give it an another try and you.

What kind of budget are you giving this thing? Like you blew through a million dollars accidentally kind of thing? I don't actually know because we're using sort of, you know, internal calls to Gemini. So actually, I don't actually know. So, but look, there's token budget, but then there's also like I'm solving a problem that is computationally expensive. Oh, yes. That also, essentially underneath it because the scoring function itself might have, you know, Monte Carlo estimation or whatever. Yes, so you actually end up, you can actually end up using a lot of compute to just even do, or simulation.

Like if you have a simulator inside, has to run a simulation. So, yeah, you can, you can, you can spend a fair amount of just CPU or GPU. So my little experiment with Era, Era and Cod is to build a neural network. for some classification problems. And so they obviously, like, if you have enough data, then larger networks work better, but they're more expensive to train. And you start to run into a question of how do I manage my budget,

if I have a fixed budget, so that I'm spending my dollars on the most effective solutions. That's right. And I think that's still something we need to figure out. But it's, of course, it's no different than if you have a grad student and they're trying to train a very, very large neural network or a very, very large data set. They themselves have to, there's something like,

oh, is there a scaling a law? Can I extrapolate? So it's not, I guess it's the same problem, but maybe more urgent because it just runs into this problem because it's so relentless, it runs into the problem much quicker than a grad student could. One of the things that you optimized was contrails. Can you talk a little bit about that?

Well, let me maybe spend a minute or two talking about the Contrails problem. Yes. For context, con trails, not Kim Trails, which is a conspiracy theory. Yes. Although you should also dislike Conrails, but maybe not for the same reason. So Contrails are, if you've ever seen those white clouds form behind jets, those are called condensation trails or contrails. And it turns out they add, at least according to the estimates that people have,

about 1% of all anthropogenic global warming is caused by contrales. Why is that? I can just talk about maybe the physics of that. So it turns out that there's actually two

countervailing effects. Contrails are well sometimes if you've ever seen them they're streaking and then they kind of go away. Those don't really do anything. But sometimes they last for a long time. You'll just see in the sky just almost like a waffle

of just persistent contrales they're called. And there's two effects. that they have. Those are thin, white clouds, so they reflect sunlight, but that only, of course, happens during the day. It turns out all objects emit something called blackbody radiation, and the earth does at whatever the temperature is, about 300 Kelvin. It's in the far infrared, around 10 microns. And at those wavelengths, confrales have very low albedo. They're almost essentially black. And so they'll absorb a little bit of the outgoing infrared radiation and then

reamid it both directions. So essentially, they'll work. reflect some of the outgoing heat, so it'll trap heat like a blanket. And so because that happens 24 hours a day, they tend to be warming. And it turns out it's a surprising, again, there's some uncertainty about it, but, you know, Contrail Cirrus, Cirrus that sort of comes from contrails might cover, especially in places like Europe, which has a lot of airlight traffic, a few percent of the actual sky is covered by sort of additional contrails, which so it adds, In those places like Europe, it adds about one watt per square meter of forcing, locally at least, which means that just to give you a sense, all of anthropogenic warming sort of averaged across the whole globe is about 3 watts per square meter.

So in places of high airplane traffic, it can be a lot of warming locally. So what can you do? Well, it turns out contrales are caused by areas in the atmosphere that are ice super saturated. They're a little bit like rock candy. So, like, when you have rock candy, you get a water solution that has too much sugar in it. And any little, you know, little bit of sugar in it will just crystallize all the sugar out. Just like in this contral, these regions, they tend to be kind of pancake shaped, only a few hundred meters tall. And if you fly through it, the jet exhaust has a little bit of moisture in it.

which will turn into droplets and then freeze. And then for every gram, if you're in this bad region, for every gram of water, ice or so you put out, it's about 10 kilograms of water gets sucked out. So there's this enormous 10,001 curing ratio. So it's a big problem. So what you can do is you can figure out where these regions that are invisible, of course, these regions of icy precessurated are,

and then tell the plane to go underneath. And you only have to drop essentially, what they call two flight levels. So it actually costs a little bit of fuel, but not very much to kind of avoid these sort of bad regions. So we built a system that sort of looks at satellite images

and tries to detect where contrales are. So we have essentially a continuous monitoring system. And then try to build a model of, because it turns out the weather models are not quite accurate enough to find these places of ice super saturation. So we built a custom model, again, like a convolutional net or a UNED or something,

essentially to predict where they're going to happen so that then we give maps to a flight planning software so that they can dodge it and inexpensively reduce the climate impact of aviation by a lot. What's the physics behind why you can predict that? Is it just, I see it in the satellite, and then tomorrow I think it will be there because planes go to the same place? No, it's because you're trying to detect these regions of ice supersaturation

because they're very, very persistent. Essentially, they're... Oh, they're persistent. Oh, yeah, yeah. I mean, no one knows exactly, but they could last for days. Essentially, they're caused by, they think, sort of warm, moist air

being injected just at the boundary of the tropopause between the, just at the bottom of the stratosphere, and then when humidity gets up there, it sort of sticks there for a long time and then gradually dissipates. Got it. So there's just sort of...

They're like bad spots in the atmosphere you don't want to fly through. Right, okay. And so once you've established that, it's probably good for a couple of days at least. Well, you have to keep predicting where that is. Yeah. And the models you're using are,

you mentioned like CNNs or something like that. That's right. And we haven't replaced those with era level models yet, but there was a very interesting problem that came up, which is you sort of want to know, well, just how much warming did this contrail make and how much did it add to global warming? because, for example, you might want to find the biggest ones

because there's some fuel cost and maybe it costs a bit of money for the airplanes to avoid it. So you say, well, gee, I'd like to kind of know how much it did. But that's actually what they call a counterfactual problem. Like, okay, you made a contrail and a certain amount of infrared radiation happen. So we can measure that if you're careful. But would have happened if there hadn't been a contrail there. That's a very difficult thing to estimate because you can't access the universe

where that didn't happen. That didn't happen. So you have to make these things called counterfactual models. And those are actually, I don't know if your listeners know, counterfactual models are actually pretty tricky to fit and make.

And we were, remember, there were two, there were two reflecting the sunlight and then there's the infrared. It turns out the measuring what the effect of reflection sunlight is actually more difficult. And we were actually stuck on it for two years.

We had a model that worked okay, conceptual model for the outgoing long wave radiation, but not for the reflected sunlight. Ira actually helped us find a model that sort of searched all the confounders and sort of figured out like, oh, how can we estimate it? Because we had, again, we even had like test code on sort of artificial, because you can kind of inject artificial, make artificial data sets where there's sort of injected contrales and sort of figure out, oh, well, we know how much it was because we injected it. And so, again, our own attempts didn't even pass our own tests,

but ERA's thing actually did and sort of unstucked this problem. So, yeah, we're in the middle of writing up a paper. We have a paper about the outgoing long-ray radiation, but we have a paper that it's not submitted yet, but we've talked about it at EGU, I think, where we actually solved this problem. And the models that error comes up with, Are they just like a big monstrosity of code?

Or are they like pretty basic? And it was just you needed the intuition to develop that. Yes, it's actually, in this particular case, it was actually more of the latter that it essentially sort of helped identify what the, it was a very simple model with some, just a, some number of confounders that we just hadn't, we just hadn't tried that combination before and it worked very, very well. So yeah, it actually sort of came up with the, and it was sane in retrospect. So that was, I think, a big win.

Yeah, that's interesting. I know you've done a lot of work in climate. What other stuff have you done? I think I talked about this, right? I talked about the CO2 thing. That was pretty fun because it's still quite a, the reason why estimating CO2 in the atmosphere is an interesting problem is we actually don't know what the carbon flux is into the bio, in and out of the biosphere. I mean, we do. We know that the biosphere captures, right, we emit a bunch of CERN. CO2 out into the atmosphere. And some of it gets absorbed into the ocean with sort of mostly in organic chemistry, some phytoplankton, and a lot of it gets absorbed on land. But the error bars about what happens are moderately large, and the error bars 50 years from now are very large. Like the models in 2100, we don't know how the biosphere will react to the ever-increasing

temperatures and CO2. so we don't actually know how much the CO2 absorbed. And the error bars are 300 ppm of CO2 just from the uncertainty of what gets absorbed. And just to point out, you know, right now there's about, what, 440, 450 ppm. So it's huge. I mean, it could be seriously amazingly awful or not great,

but, you know, the 300 ppm is like enormous uncertainty. So it would be really nice to figure out, you know, Can we reduce that? So this is like the CO2 concentration is like one step towards that, solving that. And I know that Google has made some really big improvements in climate modeling and weather prediction as well, right? I was at Nureps this year, this last Nureps. And I stopped by the climate track. And, you know, I maybe only had a chance to listen to talks or something.

but it really blew my mind the sort of step change. I think this happened in the past, I don't know what it is, maybe 10 years or whatever, in terms of climate modeling. I know a lot of that happened at Google. Can you talk a little bit about what has happened in Google and other places that has made that, allowed that really big transition in climate and weather modeling?

Okay, so let me, people often sort of collapse climate and weather together. Well, because they're fundamentally the same physics. Yeah. Although, at least for the atmospheric physics, obviously when you start having ice and land, climate is long-term weather. And so the complexity of a full-earth system model, which is a climate model,

is much, much bigger than an atmospheric model. Like you have to actually measure what's the water flux and the CO2 flux in and out of the land or what will happen with ice. And so there has been a step change with weather models. Sorry, sorry, I want to make this. So weather is up to 15. days, approximately, because, you know, weather itself or the atmosphere appears to be chaotic.

I'm hoping, I don't know if I should explain chaos, essentially it's the butterfly effect, right, that small perturbations, like a butterfly flaps its wings, and the weather will be completely different in, you know, two or three weeks. So weather is trying to predict the actual trajectory of the atmosphere over, say, two weeks. And that's now, that has been a huge step change. And that's because that's been a lot of not even the new LLM stuff that was based on the 2018 era machine learning stuff and just a large amount of data and a large amount of compute. So there's been a lot of sort of very clever work

and a lot of it from Google making new weather models and it's been great. And in fact, we had a really neat breakthrough because now we can apparently predict tracks of cyclones, tropical cyclones, much more accurately, many days in advance. And so places like Jamaica had got hammered by a terrible hurricane, and a lot of the classic models didn't actually predict it, partially because it's often, especially the intensification,

it's all being driven by what the surface temperature is. Because hurricanes people might not realize are heat engines. Essentially, they convert sort of heat in the ocean to big atmospheric motions. So weather's been great. Climate is much more difficult because you don't actually care about, you're not trying to predict whether it's going to rain in Seattle in 2070. You're trying to get kind of like averages. And what makes it difficult is that it's what they call non-stationary.

So that in fact, literally the, it's like the underlying physics or the underlying, like, you know, plants are behaving differently and ice behaves differently. And so it's a, it's very, very difficult to use sort of classical ML on. sort of true climate models. And so that's why sort of the whole discussion you guys had about, you know, what we're talking about about descriptive models and multiple hypothesis testing, that is incredibly severe in climate because we have no data from 30 years from now and we don't want to wait 30 or 50 years to find out whether we were right or that way overfit.

So whatever things we do, you have to be kind of careful and try to peel off sub-problems and the problem of exactly how do you inject, how do you build a big model that can predict into the future, but is still constrained by what we know. It's a fascinating problem. I think it's still unsolved, but it's a great problem to have because, again, these uncertainties we really would like to know

what will happen in 60 years to the climate. So it's still, it's a thing, it's a very, very interesting problem to work on. But so far, AI has not revolutionized it because it's very, very resistant again, because of this data problem. It's a low data problem. Does the butterfly affect chaotic nature of weather?

Does that also impact climate, or is the time scale so large that you have a closed system for which, you know, maybe it's oscillating between poles or whatever, but it's sort of when you look at that time scale, it's more stationary. It's unfortunately non-stationary in a different way, but the original sort of whole chaos thing was, well, maybe many people came up with it, but in meteorology, it was back to a person named Lorenz, who had this sort of very model, very simple model ODE.

So the difference between climate and weather is, where are you on the attractor? And climate is about the statistics itself of the attractor. Right. The problem with climate is that we're altering it, so the attractor itself is changing. It's moving, and there could be, and there could be, everyone talks about tipping points, that means that the attractor suddenly changes. And the trouble is that's very, very, very difficult to predict. So even the attractors shape changes quickly.

Or could. Could. And the trouble is when you run a simulator, you don't know, like, is it, did it go unstable because my model's not great or is it an actual physical instability? And it's extremely difficult to tell the difference. So what do you do, especially when, I mean, to me it strikes me that not only do you not have future data, you really don't have much past data. You can do some measurements and ice cores

and lots of stuff to try to do that, but there was nobody with an instrument 100 years ago. That's right. So if you have annual data or whatever, maybe you have, if you're lucky, 50 data points in any one location. Right. So whatever we do has to be very constrained by what we know,

but it's just very different. I'm just telling you sort of the horns, the dilemma people want. So people make these, In fact, people in applied science in general, I would say climate is the most extreme, make these things called process models where what you do, and I've seen the code, oh, well, you know, I'm going to be reductionist and I'm going to sort of take the horrible, complicated climate thing and sort of boil it down to a thousand pieces.

And then I'm going to, you know, find the expert who wrote a paper about, you know, piece number 763 and he fit a cubic to some data. Like, for example, one thing that's very mysterious, which is related to Contrails, is how does ice behave in clouds? It turns out, you might, again, everything is complicated once you dig into it. But it turns out that, like when you make a contrail, how long does it last?

Well, it depends on, because the way contrales can evaporate is ice starts to accumulate, as I said, and then the ice crystals get big, and then they fall. But of course, how quickly they fall depends on their shape, which is not known. And how much does the, the, the, the contral mix from the moist inside the contral out. Again, people have approximations, but they don't know. And so the uncertainties very much confound. And it's not just, oh, John, who cares about contrales? It turns out that the actual physics of microphysics of ice has very strong implications about what climate models do. And it's sort of we just don't know.

So I'm trying to say it's very gnarly and it's not a solved problem. My hope is that with tools, maybe not like today's era, but maybe tomorrow's era. Because it, remember, as I was saying before, it not only can fit data, it can read papers, right? Yeah. And the question is, it can read a lot more papers than we can. So maybe, maybe we can integrate all the data that or all the knowledge that people have carefully evaluated much more than any one person writing a piece of code and fit data.

I mean, that would be, that would be utterly glorious. We don't have that today, but that's sort of one of the hopes that I have, even for a more amazing tool in the future, is something that really can write code in a sane way, even much more sane, because it'll be constrained by all the scientific knowledge that we've accumulated so far. That would be amazing. We don't have that today.

I mean, that strikes me as being very similar to biology. Oh, yes. Oh, boy. Right? If you've ever played by biology or even looked at biology, There's so many exceptions and so many hacks in the biological systems. Yes. Oh, boy.

So, yeah, it would be amazing if we could have a thing that could really integrate all known scientific knowledge with data and try to synthesize sort of new models and new things. I think what you're saying is that AI can be an unlock here to some extent because the models are so piecemeal. They are necessarily piecemeal. And so that being able to assemble the jigsaw puzzle, not to mix metaphors, but to assemble like a really, this jigsaw puzzle, having just scale and capacity actually helps a lot. That's right.

The one thing that these AIS have is somehow, you know, humans, even I'm pretty well right, I think, but it's just difficult for me to kind of integrate across the N squared different paper. If N is the number of papers I read in the life, it's pretty big. And it's just difficult for me to even, do that n-squared thing, but somehow there's just so much data in those billions and billions

of parameters, and you can also give it access to read PDFs, that it can somehow start to pull things together that people wouldn't do it. So that's, again, I'm starting to see little indications of that inside of ERA. I'm not claiming that's what ERA does today. But yeah, that's sort of my hope of where this is going to go. I think I've heard a lot of people suggest something like the the route to intelligence is to combine LLNs with some form of search. So it's actually amazingly like ERA doing that, right? Something which, you know, is maybe a very, a very strong database lookup with a good search algorithm is one way.

And of course, I mean, there was the whole, I mean, people still do, I guess, the whole rag thing, of course. And if you think about it, Google itself, you know, the 10 Blue Links things, it was or is a form of AI even before we had LLNs, right? because it was like you could cast yourself back to whatever, 2010 or 2015. You could ask Google about literally anything and it will tell you stuff, right? Surprisingly well, actually. Yeah, surprisingly well.

Because somebody on the internet has written about it probably. That's right. So if you can match that. In fact, that was one of the reasons why I wanted to come to Google. It's just that was such an amazing thing, right? I wonder how much of our audience did a search pre-Google and just know how bad that experience was. Yeah, I agree. 1998, I think, I think, when Google, like, I was using Altimista.

Yeah. And, like, I don't know, Google just got released and I used it. And I just, sorry, a digital. Yeah, I remember that too. I just dropped, like a hot potato or something. It started immediately using Google. So it's, so that is sort of a form of AI. And so, yes, it might be, yeah, it could be that just having access to all of that

and sort of keeping it in mind at the same time. that model of sort of scientific discovery in as much as it pans out is kind of comforting too because it is reductionist so that you can look at the individual parts and understand them. So it found the exact things to assemble, but they're all actually maybe fundamentally things that people have invented or it's done iterations on. And so all those little pieces are individually understandable. And then you can also put them together into a coherent picture. I think for a lot of problems like biology or climate science, I don't think we would trust the answer unless it was in that shape. Yeah.

Because if there was some giant black box model that said, oh, this is how a cell works. It's like, do I believe it? I mean, I don't know if I believe it because I can't examine it. But I mean, to argue against it, though, like if it works really well. But you'd have to gather, you'd have to, I mean, you have to test it, obviously. But a statistical model, again, it has to extrapolate. Yeah, yeah, and it has to extrapolate to the extreme or the sort of the black swan events. That's right.

Yeah. So it's very hard. This is why things like self-driving cars are very, very, it's a very difficult problem. Right. It's all corner cases. Yeah. It's kind of amazing how well they've done. That's an interesting point.

It's thinking about when, you know, coming from the world of physics where a model was usually a single equation or a small number of equations, which uniquely define a system and everything about it. And you just crank, you know, just find a solution. of the system and you now know everything you need to know. And I think something like AlphaFold was kind of a shift for a lot of people where before they thought, oh, protein folding is a problem where you just, if we find the right force field and we have the right computational engine, we will solve protein folding.

And the thought of even really solving it in a data-driven way was only appeared a few years before, you know, AlphaFold. AlphaFold 1 came out. And it's interesting that I think it's sort of forced the new AI modeling has, I think, forced people to reevaluate almost what is science. Because Alpha Fold and similar models are incredibly powerful. There's a lot of things that they've opened up as tools. But at their core, they oftentimes don't give intuition in nearly the same way that, let's say,

that most physicists historically would have wanted. And so I guess it's sort of there's this old saying. all models are wrong, some are useful. Yes, box. Yeah. Box said that. Yeah, yeah. When do you find the data-driven models to be sufficient,

and when do you want sort of like something which is interpretable that humans can actually understand? I think it boils down to almost like the difference between weather and climate. If you're in a data-rich regime, like weather, or even proteins because of PDB, you can feel, oh, yes. In other words, I've got enough data to kind of, of cover. And so a statistical model like Alpha Fold should do. And so, in fact, a lot of people happily use Alpha, you know, I think it's really revolutionized by understanding. I'm not a

biochemist, but people seem to love it. And the end, and one amazing thing they did is they exhaustively, they just just ran it on all PDB and published it. It was just really, really cool. It's like six billion protein protections. The vast majority are actually quite accurate. Yeah. Yeah. So that's just amazing. So, but it feels closed, if you know what I mean. But when it's like climate and it's open and it's non-stationary or you have to make these be extrapolations, you have to be much more cautious or maybe biology. Again, and there's probably there may be parts of biology like, oh, there was this virtual cell challenge from the Arc Institute. That had a funny result.

I know that people were, there may have been some overfitting for at least that. Yeah. What would we say more? Sorry. Or I guess maybe at a high level, I think simple baselines for. It worked very, very well. Yeah, yeah, yeah. Yeah, just like in the...

One of the classic things in whatever you do biology is just always start with the simple baseline. Maybe this is probably just good ML in general. It's good ML in general. Yeah. Understand your simplest case in biology. There are many problems where they're extremely resistant to anything beyond the simple baseline, even if you have a lot of data. That's right. And in fact, I tell people the same thing.

I said, always just fit linear regression. Just do it. Just do it. Just do linear regression. Or SVM. SVMs are just a different... Different form of linear aggression, yes. So when do you need the more process-modely thing?

I think it's just when you have, I mean, sort of climate is on one end and I don't know, whether, maybe on the other end, that may be too extreme. But I think just where are you on the data richness thing? When can you feel like, oh, no, I really have a closed, I really have a closed problem. And I think I can actually cover it. Yeah, closed problem that data fully covered. is, yeah, I think that makes a lot of sense in what I've seen as well. Yeah. I think one thing I'm kind of curious about is when you're working on climate modeling,

you talked about contrails, you've talked about CO2 predictions. What are the broad things you're trying to accomplish? So one of them is, I guess making interventions, and the other one might be making predictions for things like insurance or like how do you help, like, adjust for some sort of climate change? Or, like, yeah, what are the, like, principal goals, I guess, for you specifically or the community at large? I think, you know, just like any community, there's probably many different goals. For me, I'm, or and my team, we're very, very interested in interventions.

So, like, which ones are possible at relative costs? I mean, Contrales was kind of amazing because it turns out the, the, intervention is quite low cost. And also one amazing thing about contrales is they are local, unlike things like CO2, because so if a country decides to fix controls over itself, it actually improves its, I mean, it has the global effects, but it mostly improves the climate a little bit over themselves. So they like that. I guess that if you were in a cold climate, you want to warp it up. This is now your own, you could know. Yeah, it turns out that it's a little bit asymmetric. The warming is constant essentially and global.

The cooling only happens when you're sort of at a good, the sun is at a good angle over you. So it's very rare that the uncertainty, there are contrales that where our uncertainty bounds in terms of the warming. There are many, many contrales mostly at night, of course, where it's like a largely warming and we're very sure in terms of like 2 sigma. There's very, not very many contrales where you say, oh, I know for sure that it's cooling and I want more of it.

like, like, so only over sort of the poles in polar summer, do you know that the contrails are cooling and therefore if you got rid of them, they would warm up. But there's essentially no flights over Antarctica and not that many over the poles in the summer. So no one's going to, no one who lives in a cold climate is going to use this maliciously. Well, yes, in that, in that, well, they wouldn't know for sure whether it was warming or cooling and so they would do stuff and things. So mostly we just, sort of ignore where we don't recommend that people fly those. You also brought up an interesting point about the economics. I mean, a lot of there, I think, was a lot of resistance historically about certain, you know,

to climate, certain climate change interventions, which have in some sense, the market is just taken over. Like, at this point, unambiguously, like renewables and batteries are just almost universally, unambiguously, just better than alternatives. For first. For non-mobile. I mean, free-formable. I mean, that's a really good point.

Yeah, yeah. Like, planes, we do not have a solution to... Correct. I mean, there are some battery-powered planes, but they're very small and have to go very limited range. They probably will never actually be a... It's hard to imagine. The physics would be very, very...

Unless we came up with something like nuclear batteries, which would be kind of amazing. But I don't know how to do that. Or even if we did, I think the risk of, like, people would be too afraid of a nuclear battery going wrong or something. Oh, yeah. Since we don't know what they are, we don't know what they are, we don't know what they're.

The risk is on. Yeah, so we don't know. So, yeah, that's the problem with, I talk about, I have given talks about climate change. And I talk about the pie chart of badness, my chart of sadness, which is there's no one silver bullet for climate change, right? There's so many different things that contribute greenhouse gases just from across our economy. So they sort of all have to be fixed or many, many of them have to be fixed. So there's no one single thing.

I mean, I've worked on fusion. Fusion's cool. And it might actually knock a lot of them out if it's cheap enough, which we don't know. because we don't know if it'll work yet. I mean, Fusion is one of those interesting things where the joke was always fusion is 30 years away, but I think it's actually now less than 30 years away, maybe. Yeah, no, I think there's a definite probability

that someone will make commercially relevant fusion even by the end of this decade. So I think it's like three years away, not 30 years away. It's very real. Interestingly enough, I think a lot of that, I'm going to not just stump or, advertise some of our other episodes, but a lot of that actually comes down to material science, interesting enough, in that... Well, I'm skeptical. Oh, superconduct, sure, sure, sure. Well, yeah, sorry, we can talk about Fusion if you make.

Yeah, yeah, there was actually two things. One is Fusion. The other is federal control systems, which I think is actually up there on. Yes, and in fact, Google DeepMind has been working on control systems for Tokomax to make sure they don't essentially go in stable and go disrupt. Yeah, that's right. Disruptions are quite interesting to themselves. Yes, yes. Yeah, it's basically the entire, all energy in the tokenac culminates into one little beam and then... And then hits your vacuum chamber, then you're very, very sad. You're very sad. Yes.

Yeah, yeah. I think people believe that either could turn on after $30 billion disrupt and then basically have a $30 billion brick or something. Oh, yeah, I guess you could try to patch it. I remember working, we, again, before LMs, we worked with a fusion company called T8. and I was in their control room. And yes, it was kind of sad. You have to be very careful. We were making systems to recommend new experiments.

And they were very, very skeptical and jaundiced, which, where they should, because I've even been there for that. Even under human control, it's like they were doing some experiment. Then you hear this big bang. And it's like, oh, no. And then it's like, you know, then the apparatus is down for two weeks as they patch. You were there during a disruption? Oh, no. This is, sorry.

They have filled reverse configuration. Oh, okay. So, which has its own, I mean, things, you know, there's some arc. So what is that? Sorry, I'm not familiar. Oh, oh, what's the field reverse configuration? Well, it turns out Tokomax are not, although perhaps the most studied form of plasma, there's many different kinds of architectures, essentially ways to try to stabilize and compress plasmas.

There was a shape. Essentially, it's essentially a self-contained football of plasma called the field reverse of configuration where essentially the magnetic field inside and outside are opposite. So they're separated by something called a separatrix. And that is sort of in theory unstable, but in practice stable. Like, for example, when you run Magnute Hydro Dynamics, MHD code, it's unstable under that assumption. But that's an assumption. That's not the way the real world works.

And so, yeah, it was kind of disfavored for many years. But TA and other people, I think Helion have FRCs because they are actually relatively robust. You can actually knock them against walls and they'll still stay stable and yes, but you can still get discharges and things that punch holes in your vacuum chamber, which is kind of unfortunate. For clarification, so you

have these fusion reactors, they are or trying to be reactors and... Apparitis. And you create a plasma. The plasma is magnetically charged. Oh, confined, yes.

So it's confined by a So you have some sort of magnetic system that is tunable by a computer, and then the computer tries to kind of maintain the confinement? Well, FRC is kind of, once you make them, they're sort of sustained. There's different ways of trying to make sure you. Okay. So all of Fusion boils down has been called the Lawson criteria. There's essentially, and it explains why Fusion is hard. Essentially, you can just very easily on the back of an envelope just show that,

the density, the temperature, and essentially the energy loss. It's called the confinement time. It's one over the amount of time it takes for the energy to decay away, one over E in a plasma. So the product of those three numbers has to be bigger than some constant. And then you can get fusion. And if you don't, then you don't. And that explains the fact that it's a product of three numbers explains why fusion is so hard because every approach has an Achilles heel where one of those numbers is not very big.

And then they tried to desperately make that. be higher. And every approach is different. Every approach to fusion is kind of different. And a lot of, you have to be a bit skeptical when there's all these sort of breathless news things about fusion because they'll say, you know, now confinement time is starting like stable for X minutes or whatever. And it's talking about like one of the three numbers, but you have to have all three

numbers before it you can get fusion. I think the whole field is making a lot of progress and it's very exciting. But you do have to, you have to be a little bit cautious about the breathless news article that only talk about one number. So what is the computational part of that? Oh, unfortunately, for better, for worse, it depends on the approach. So for Tokomax, as Brandon said, it's that there's this, it's mostly stable,

except that there's occasionally this instability that takes all the energy and smacks it into one place. And so you have to sort of keep everything sort of under control. So it's a control system. FRCs themselves have very simple instabilities. So, for example, they have what they call, a Z instability. So it's fine, it's stable, it'll just wobble,

literally wobble back and forth, but you just make what they call a PID controller that just keeps the football in the center of the reactor and things are fine. And it does that by adjusting the magnetic field. Yeah, it actually just, I think the electric field is sort of knocks it back and forth. The issue that people have really depends on

which sort of plasma architecture they're deciding to use. Climate is sort of political because of economics basically, probably, mostly, maybe other stuff. But the economics of it, you know, you have to persuade people to somehow spend more or you have to have a solution that has like this happy coincidence where it's both economically better and better for the climate.

That's hard. But in cases it's not. I mean, many cases it hasn't been hard. Yeah. Well, like you think about like, okay, predicting even whether, right? You can prep and you could see how that could be economically beneficial. So what kind of work are you doing with interventions and how does that kind of interact with economics? Like it sounds like the contrails one. We said in an analysis and said, actually, this is great because it is very low economic impact, but high value. That's right. So if you're trying to think, there's sort of energy intervention. So you have to be, you have to sort of compete with existing forms of energy. And that, That's not trivial unless there's a co-benefit or there's some sort of clever, you know, just co-benefit. Like this, again, is highly speculative.

It wasn't our work. There was a startup that was, I don't know if you saw the news. It was last year, I think, where someone figured out if you inject mercury into a fusion reactor, the neutron flux can actually transmute the mercury into gold. And then you can sell the gold, which I thought was very clever. It might not work. You know, as a physicist, the one thing I want out of a fusion reactor is helium. But that's a different story.

Oh, helium, it's three. Well, I mean, helium four is kind of boring, although it is getting, because the strategic reserve has been shut down. There's less of it. Yes. And, of course, I want helium three and four. Well, not even just a fuse, just to make dilution refrigerators for quantum. Or MRIs or so much, yeah. So much technology we think about actually just goes out the window if we run out of helium.

That's true. No one's thinking about it. Sorry, that's like a complete aside, though. Yes, the fact that the U.S. had a helium, strategy helium was to reserve was for a very important blimp fleeing. Yeah. But they kept it for decades anyway. So that was nice, but then we stopped.

And then we got rid of it all. It all went up in the air. Yes, yes, in balloons and stuff. Or out of natural gas wells. Yeah. Sorry, now we're talking with helium. Yeah. So interventions, what are some of the most exciting, interesting ones?

Well, I'm very excited by fusion. I mean, I don't know if it's intervention. that's sort of a source of energy. Because if we can make it work and we can make it be sort of low enough capital costs, that that will actually help a lot because at least the current models are, renewables are great. Ideally, you'd like to electrify everything, right? Which has problems because you can't electrify flights.

But you can try to electrify a lot of stuff. You know, there are EVs. You'd have to figure out how to electrify things like cement or steel. those are hard, especially things like making steel want reduction power anyway to essentially you're adding carbon and you're reducing iron ore. So there's a lot of things that are difficult about electrifying everything. But if you could electrify everything, then the amount of electricity required would grow by factor five. And you could try to grow renewables. Renewables plus battery, again, trying to squeeze all of it out, it starts getting ever more expensive because you just need

ever more you need like a huge number of batteries to sort of cover the last you know few percent or even you know 10 or 20 percent so we do need some sort of power that that can cover the last 20 percent something that's you know base load so fusion might be a thing for that so that's super exciting again there there's no one sort of silver bullet that can sort of cover all the cases so I'm happy to sort of talk about any specific case but it's sort of like the world the world is a very complicated place and the global economy is a very complicated place. So it's super hard to sort of talk about sort of interventions in general.

Yeah. Maybe instead of interventions, one thing I'm curious about is how does this make, affect decisions into, for example, like, where do we build? How do we build? I think you're from L.A., right? Or at least you... Well, I spent 11 years there. Yeah.

You're what, okay. So, yeah, you spent a lot of your life in L.A. I mean, L.A. just basically, large parts would just burn down. And maybe probably close to where you used to live. So this is something that I think a lot of people kind of saw coming, maybe partially due to regulatory issues, but partially due to other issues. And we were completely unprepared. And it seems like there is a lack of preparation about what to do next or to sort of adjust for this.

And I mean, have you worked on basically predicting like new risk assessments or suggestions like what do we actually change? to maybe harden, you know, society, even for what's coming, regardless of whether or not we actually do something to make solve the underlying problem. That's right. So, in fact, there's a big effort at Google into something called Crisis Resilience. And so we had a very fun project called FireSat.

I don't know if you know about this. So it turns out that for wildfires, a lot of these wildfires, if you only caught them early enough, it's very easy to put out a wildfire the size of this room, but even if it's like an acre, it gets much, much harder. And of course, under certain circumstances, they can grow exponentially from the size of this room up to an acre. So that might be hard to catch.

But they often sort of start small and spend a while. So we figured out that, oh, if you had a global constellation of low Earth orbit satellites that could detect in the midwave IR, which goes back to the black body, essentially, that's the temperature of fire. They stand, the fire stand out in the midwave IR. And so we designed a sensor that if you built,

it depends on exactly their orbits, but roughly 50 to 80 of them, you could actually find fires about the size of this room, about five meters on us. Well, maybe it's a bit large in this room, five meters on a side. And anywhere on the planet, And again, depending on how many satellites you had, within like 15 to 20 minutes,

you'd have to put a fair number up, like, you know, 80 to get them within 15 minutes. And then you could actually intervene. You could decide not to if you wanted to, you know, have the fireburn fuel and you thought it was safe, but if it was going to blow up to something unsafe, so he worked with a now a nonprofit called Earth Fire Alliance that we're part of. And so they're starting to, like we've launched one satellite, which is a prototype. We've worked with a company named Muon Space to actually, sort of make the satellite.

So that's cool. We have wildfire boundary detection, and we propagate that information out through Google. So we can actually sort of figure out from existing satellites and existing data fees where the boundaries of fires are. And then we sort of tell people through their Android phones or through search about fires. We've worked with the US Forest Service on making new models for how fires propagate. Because again, that goes back to these process-based models from the 70s. by a person named Rothromel.

So we've actually made a little neural network proxy model based on a new, essentially to sort of be able to run it very, very quickly. So we worked with the Forest Service on that. So yeah, yeah, we're very, very interested in trying to minimize because it turns out people might not realize the World Health Organization estimates that there are 300,000 excess deaths a year across the world from wildfire smoke. Yeah, I mean, I remember, it's been a few years since we had a really bad fire season,

maybe what, four or five years ago, there was this cloud of smoke, which like crossed all of northern U.S. and Canada and caused a lot of respiratory issues, I think. Yes. Yeah, and it's very hard to track. I mean, you have to get these.

You have to estimate these excess that's from statistical means. But yeah, it's a very serious public health problem. And also just very scary and it burns people's houses down and it's terrible. Yeah, yeah. My view is that climate change is sort of like a serious disease. Do you treat the symptoms, i.e. do you adapt? or do you try to attack the underlying thing?

And the answer is, well, if it's the reason of both. Yeah. Right. And so, yes, so we take sort of adaptation, especially around climate resilience very seriously at Google, and we try to get people informational tools to sort of help. That's part of the reason why we were working on weather and then sort of cyclone prediction and things.

So it all actually hangs together. So it's more than just, you right, it's more than just interventions. It's climate resilience, too. Yeah, having lived through four or five fire seasons on the West Coast, they can be quite nasty. And it used to not be, I mean, I have a cabin up in the Sierra Nevada Mountains, and yeah, it used to be, oh, you know, summertime, it's nice. And then now it's like, well, not every year.

But yeah, there's like, you know, winter, spring, summer, and smoke. Yeah, fire season. Yes. Yeah. I want to stay to the west of the fire line. Yes. And I, yes. So that is another thing that I'm interested in and that Google's also very interested in is climate resilient.

Are these IR sensors small enough that they could hitch a ride in like a micro satellite grid? Like would it make sense to, would it be almost cheaper just to hire, I mean to pay someone who's launching a constellation? Oh, they're not that small. The thing is you need refrigeration because it's midwave IR, so you have to keep it cool. So these would have to be their own special satellites. They're not super large. Okay, yeah. They're not like the, you know, the satellites in two synchronous orbiters.

giant monsters because of all the optics and who knows what. I see. Yeah. And they have, you know, they're basically just iron sensors with a resolution of five-by-five. No, no. That's the other cute thing is the resolution is about 50 by 50 meters. But you can use super-resolution because it's essentially multispectral and you sort of know where fires are.

So, yeah, there's a fair sprinkling of AI on top of them to reach that 5-5 meter. That's cool. Yeah, yeah. When you also have, you have a convolution. over what you're reading out, right? I forget the frame rate. The satellite is moving. I don't remember what the point spread function is.

I'm sorry. I don't. But you're right, they do move, but I don't remember how fast they, I don't remember how fast they, this also has been called a broom sensor. So there's this funny thing of trying to, you kind of spread out the spectrum one way.

And there's also, it's a somewhat complicated thing. It isn't just a, like a Polaroid. It's a complicated sensor. Yeah. Sort of switching gears a little bit. You've been at the intersection of AI and science for quite some time. I think you've sort of wound your way into and out of it back and forth.

How do you see the field has evolved? Because I feel like it's evolving very quickly now. And what are the sort of lessons that you've learned that you think the community is learned? And how do you think this should change, if you were a young scientist or young practitioner, how should this change what? how you should approach the future? Well, I think there has been a phase change in the last 12 to 18 months.

So, I mean, a lot of what we used to do, as I said, was build these specialized models to solve individual problems. And if you think that's your job, it's kind of fun. You find a problem you solve it. You find another problem you solve it. But now we have these much more general AI things.

And I think the whole AI for science community is kind of still feeling around. The fact that they're working is so new that collectively we're not sure like what's the best thing to do? Or maybe there's no one best. Maybe there's a tool chain. And I think we're all trying to figure out,

like, what should we do? And so there's a question of what should young scientists do. I think it would be, you know, I have a son who's just turned 20, and he's really into both sort of AI and coding and

chemistry and I look at him, I think he's doing the right thing because he's both learning a lot and trying to be a domain expert about RNA, but he's also sort of using vibe coding and using all the tools. I think that's the right answer is

because everyone's figuring it out still be a deep domain. I don't think domain expertise is going away because it goes back to a lot of people that said it goes back to taste and trying figure out how people get taste without doing all the grant work. That's an interesting open question, but develop domain expertise, but also try and play, I would say, with all the different tools that are available, because it's not like, oh, yes, we know what's going to happen and the smart old people are knowing what's like,

no, we're experimenting too. And so, yeah, so I would say definitely develop domain expertise and try to use these tools and try to solve big, hard, scientific problems as best you can. There's still the huge open issue about what do you actually do about physical lab work that is not going away because, you know, experiments are the ground truth. And a bottleneck. And a bottleneck. I mean, people are talking about a lab in the loop, but that's still very, very, very open

because how do you, no one has, I think, as far as I know, a general lab that does everything. Now, there's a lot of very specific, you know, labs that are controllable. So I think it's just we've gone through this phase change. It seems super exciting. Again, I would advise people to play with whatever tools are available and to develop sort of deep domain expertise and taste to the extent you can. And I would advise people also not to be scared. And let's just like, try stuff.

You know, I'm always happy we have student researchers at Google and they come and they do sort of wild and crazy things. And that's always just delightful. So, yeah, people should be trying. for wild and crazy things and see what happens. This may be a question without an answer, but when I think about how I developed extortes and how a lot of people developed expertise,

it was by starting with a, you know, a simple, defined problem and then hammering it. And then in that process of exploration, you learn more. And, you know, some ways you go broader, some ways you go deeper, but you still, the process of just banging your head

against a problem, which now would be instantly solvable, teaches you the skills you need to solve harder, problems. What advice would you give to your son for that? You know, I don't know. Maybe it's a bit like hiking, which is, yes, I mean, you obviously can't drive everyone or you could you could drive up the mountain. Or you could hike up the mountain and maybe it's okay, even fun to occasionally hike up the mountain even if you can drive up the mountain. Yeah, I mean. In the old days, I'm going to sound like a real old day. In the old days of six months ago.

Oh, no, no, I was even thinking of in the old days of the 80s and 90s. Like, you know, a lot of people take it like, oh, there's open source packages, there's, there's can learn. there's all sorts of things. We didn't have that. I had to write my own numerical library. I had to write my own machine learning. I've written boosting, probably written it four times, four different languages. And so now I know boosting, you know, it's like, and so maybe not taking the totally easy, obviously, I mean, there's this tradeoff like, oh, but I want to be as efficient and productive as possible, yes, but you also have to develop the muscles. So it's a little bit maybe like being an athlete, like there are places, there are times when you're actually doing exploit, when you're

trying to run as fast as you can, and then there's also training time. And so maybe people just have to train. And it's entirely plausible that if you spend time actually hammering away and doing the hard work, even if it goes slower there, that pays dividends into your larger, you know, productivity long term. Like even if locally that one moment you are not being maximally productive by not exploiting an LLM, that feeds into something. I hope so. I hope the thing I don't know is I people in their careers, it's hard because, right, the whole world seems to want to optimize everything. It's your fault. No, I know. It's my fault, but it's just sort of the, you know, and sometimes you have to set aside time. Like at Google, especially in my group, we have this

concept of 20% time, which I still, I very, very strongly in my own group try to protect. It's like, you can do whatever you, if you want to learn stuff, if you want to try stuff, you don't even have to tell me. You don't even have to tell me. In fact, probably shouldn't tell me. You know, just do stuff for exactly for learning. And also because that's where the sort of creative juices are. I don't want to so occupy people's time where they have nothing like, where they can't feel like they can play or learn or try new crazy things. So I know 20% time is unusual. And there just seems to be this strong impetus in the world to just like I said, optimize and squeeze everything out. But you do lose something when you hyperopt. You sort of overfit.

Yeah, you're overfit to productivity. Yes, that's right. So I know that my advice might be swimming upstream against perhaps cultural norms. The thing that always comes up for me here is that I've, there's a, the problem is not stationary. There's a new skill set that will be the, the right skill set for the future, right?

And the question in my mind is always just detangling, okay, is this a skill that is an enduring skill? Yes. Or like maybe it wasn't enduring yesterday, but today it will be enduring because like I've seen that, you know, like for just as an obvious one, you become sort of like a manager when you're using L. Absolutely. I tell people this.

A lot of those skills transfer well. Some of them don't, but a lot of them do. And so that as a manager, you lose track of the details of what's going. on and you trust your people or agents or whatever to have that managed so that they can report up to you and answer, you know, sort of the high level questions and get the judgment about the little things correctly. And so is that what we've come to? Is that we're just like middle managers now? Well, I mean, I don't know. Again, I have a little management work, but you can't, I don't want to be an empty suit.

In other words, because the things might get it wrong, especially LMs that are sort of really weird. They don't make the same kind of mistakes that humans make. And so you can't fully trust them. You have to be rigorous and, like, you know, poke at it and make sure. Although you should be poking at software that you write yourself too. I mean, you shouldn't trust yourself. That's one thing I've learned.

What did Feynman say, you know, you absolutely can't fool yourself. And you're the easiest person to fool. Yeah. So that might be enduring. I don't know if I can quantify what's enduring, but somehow fundamentalness, I mean, really learning a domain that is about the world, for example. So this is why I like, well, biology or physical sciences,

I think those are enduring sort of fundamental things. Math is very enduring. But even things like rigor and checking and that sort of thing, which goes back to maybe management that you want to really make sure that the LMs are producing the right things or they haven't cheated in some way. But again, you should be doing that to yourself too. Yeah.

So I think there's some enduring and also just the enduring value of sort of creativity and thinking out of the box. And so I think those are enduring. I don't know. There's something very fundamental about all of those.

So you mentioned Feynman, if you don't mind me changing gears. Yeah, yeah, yeah. I took a class from finding. Yeah, yeah, they were going to ask. No, not just any class. Yes, his sort of physics of computation class. He did it with, in fact, Hopfield and Carver Mead.

That was fun. At the time, I don't think, and maybe Feynman knew. I felt like none of us knew what even the problem was. I mean, I guess Feynman was trying to say, oh, let's do quantum simulation, which I guess it turned out to be the right answer. But yes, at the time, it was, well, I guess it was cool. First of all, DARPA funded it, so you were supposed to go once a week to get a prime rib dinner.

But I just went every week. Anyway, for the students. So just for a little bit more context, this class was basically the class right after Feynman and I forget who else proposed the concept of a quantum computer without really knowing what it was, but knowing that there was some sort of... Well, there was plenty of room at the bottom essay,

which I think was in the very early. This I took it in 82. Okay. That was when he taught it. and he did say there was plenty of room at the bottom. But the way the class was structured, it was like a guest lecture on Tuesday, and then Feynman would stand up on Thursday and explain why that was all wrong, which is pretty fun.

And then we encountered something, which other people have also encountered something which called the Feynman effect, because maybe he was so charismatic or something. He would explain things, and he would say, yes, yes, I understand. And then you walk out to thinking, no, no, I don't. see in any of this. Oh, did not stay. Yeah. So it was kind of fun.

But a lot of the guest people, I mean, maybe the sort of showed the chaos. I mean, a lot of people, I think it was, Danny Hillis came. There was all these interesting guests here. So I would say in the union of all of them, I think, showed the sort of mass confusion of what was going on because there was a lot of like,

oh, should we make computers reversible? Because we have to make sure, can they even be reversible? Can this sort of the bottom limit of the heat per operation be zero? Or is there some sort of thermodynamic limit? that was like a big deal. I don't think that's a big deal now. Wait, so you're talking about the land hour limit, right?

Or not what's on the? Well, at the time, it was there made with all this question about, like, can you have like billiard ball computers and can they be reversible and things? And also sort of like what kind of computing, that's why Danny Hillis came. Like, you know, I think that was in the era of the connection machine. And what the original thinking machine is not Miramaradis, but the original one in the 80s. So he came. What was the state of like general computation?

in 1982. I mean, at this point... To rounding error, we had zero... I mean, I remember when I got there. I was Carver Mead's cis-admin, and we had a Vax 11, 750 that maybe did a MIP. One million, one million operations.

And the whole research group shared an 80-megabyte disk drive that was the size of a dishwasher. It was very exciting. So what about complexity theory? What was it? Because I know that there's a lot of interest in quantum complexity and how it relates to gravity right now. And I wonder, I don't have

a time in my mind about weighing complexity theory. We could try to look it up. I don't think there's, of course, the whole, there's a whole hierarchy of you're right, quantum complexity classes. I think that was developed after that, because this was in 1982. In fact, I, I'm not even sure there was a gate, we never talked about the gate model. There was no gate model of quantum gate. Yeah. Yeah, yeah. Yeah, yeah. I forget. I'm sorry. I forgot. I think a lot of those, yeah, I don't think those came out until like the 90s or something. Yeah. I mean,

I remember reading Nielsen and Chung, the classic quantum information. Yeah, quantum information textbook, which brings up a lot of those points. Now I think about it, that textbook was written in what the late 90s or early 2000. Yes,

that's right. And I think that was the first textbook, which like put down kind of the general knowledge of the field. But I could be wrong. I guess I think now that when I was young, I don't think it, I took it just for granted that this is, This is the book that everyone...

Yeah, I know, but this is a provision. Well, you remember in the early 80s, I mean, there was a whole bunch of excitement around neural networks. And it's really interesting because, again, we really did not know what we were doing. No one knew what they were doing. There was like an interesting like neural networks, then SVMs, then neural networks.

Oh, no, no, this is before that. Oh, okay. In the 80s, everyone said, everyone said, this has the capability of revolutionizing computing, but what does? Well, I mean, there was a tremendous excitement around Hopfield networks. In fact, Nureps came out of because there was a workshop at Snowbird that was nominally private, but everyone tried to crash. And so they spotted up Nureps.

Snowbird is a skiing trip with a computation conference attached. That's right. That's why I learned to ski because I didn't know how to ski and then I kept going. Anyway, that workshop came out of the Santa Barbara workshop in 1985, which came out of some local things at Caltech, which is called Hopfests. So, yeah, people thought, oh, wow, something involved. In fact, ironically, there was like, yeah, yeah, something about associative memory. And if you actually dig down into what Transformers are, they are associated memories. So, in fact, there was even a paper called, you know, Hopfield Networks are all you need.

So, yes. There really was. Yeah, I forgot. I'm sorry. So was that title was a reference to that paper? To the era. So it's actually, it's like, it's like the whole thing is sort of come full circle. And in fact, I think we did.

We have collectively as a field revolutionized computer science, but it was the sort of the hopes and dreams completely outstripped the capabilities because, again, we effectively had zero compute. Yeah. I think the history of machine learning is different paradigms as compute versus memory scaling versus data become available in different levels. That's right. And the fact that I think people don't realize that the reason why

all numbers one is there the one compute limited thing, although again, with now that we have transformers, things are getting memory limited again, but and the fact that they ride on top of Blas, and so the fact that Blas was being optimized by things like GPUs, I mean, I don't actually know if, in fact, I'm pretty sure the brains don't work by matrix multiply, but it was just that the algorithms co-evolved

with the hardware and that's why we're here. Who knows, I mean, if we got down some other path where people really cared about some other compute. Who knows what architecture would have ended up with? I don't know. I mean, didn't a lot of this start with like, I think people were hacking PlayStation to train neural network or something or to do, I guess, maybe it was even before that for like super.

There's a funny story by a friend of mine from grad school, Brian Katanzaro, a video where he, Brian, sorry if I get this wrong, but he came to Nvidia and, and was having a lot of trouble getting traction. And he basically, they were doubled and tripled down on gaming. And he basically one day had a meeting with Jensen and convinced him, you know, like this is, we have all these people using Kuda for Blas, basically, and for deep learning in particular and convinced Jensen.

And he said it was like a 15-minute conversation and they pivoted the whole company. company to next day or whatever. Okay. I've never worked in. I mean, I have friends, Dave Kirk, who's the first chief scientist, and Bill Daly, were both friends of mine from grad school.

So, yes, but I don't know the details of way. Now, remember that people, like in Hinton's group, even in the round 2010, they were using GPUs to do the deep learning. But even in, I would say, 2007, 2008, they weren't that much faster than CPUs. Again, they only started

really exceeding. In fact, I think this not a coincidence in the era of the original image net and some of the speech recognition stuff. So I think that's not a coincidence that they really resurged when GPUs passed CPUs. So that's not a coincidence either.

I really want to know, how does one get the opportunity to name an asteroid? Oh, well, again, is Caltech. There were some wonderful people, Gene and Carolyn Shoemaker. And they we're teaching a class in planetary science.

I like planetary science. And so, yes, as part of that class, they took us through the sort of asteroid discovery thing. Now, this is in the 80s. So now there's all sorts of amazing, amazing systems. Well, there was for a while something called linear, which automated the thing. But this was before anything was automated.

So, yeah, it was just part of a class. And what you would do is you go to steps, even though we did it out of order in the class, but the steps we used to do, this was 40 years ago, is you would go to Palomar. There would be a fast telescope, which literally now a museum piece that's in their visitor center, but at the time it was a real thing,

you would put a piece of film in it, you would take a picture, and then you'd wait for a few more minutes and take a same picture of the same point of sky. You'd put in a stereoscope, like back at Caltech. You would see if anything, you would look around, you would see if anything floated, because it would move by a tiny bit,

so it would pop up. So you would be a different, Because it's different in different eyes, then you would be able to see it as something that was projected in a different place. That's right. So you would pop out at you literally. And then you would go to a measuring microscope and you would take measurements of known star references and where this floater is. And so if you got accurate enough measurement, then you would send it off to a person. I don't think it's him anymore.

He was his name Brian Marsden at the Minor Planet Center in Arizona. and then he had a big software system to do, to sort of piece together these are called apparitions. And if you found, and if you happen to have done an observation, which was the final apparition, which allowed his software to connect it into one big orbit, then you would get discovery rights

and you could name the asteroid. But now it's like amazing. There's this observatory now in Chile called the Vera Rubin Observatory, and there's this amazing telescope called the Simonee Survey telescope. Essentially automates this problem, essentially can just take many frames of the sky.

It's an utterly stunning instrument. And so they discovered 11,000 asteroids in six weeks. Oh, wow. Yeah. So it's now. How many are them are there? I mean, at least above a... Well, it depends, I guess, done.

You need that cut off. But yeah, so, but there are probably millions of asteroids. So there's still a lot of opportunity. It's true, but I don't even know if they bother them anymore. So I don't know. Maybe you don't mean, I don't know. But people are doing occult. Sorry, I'll talk about this forever.

There's stuff called occultation. Like one of my asteroids, I was talking, I was sitting email to amateur, if an asteroid happens to pass in front of a star, it dips. Like when they discover exoplanets, but if you're super lucky, it'll dip and then it'll dip again because there's a moon. And so one of my asteroids, this amateur, found a little moon around it. So that was, I think, last year. Around the asteroid. Yeah. Interesting.

So that's pretty cool. So there's still room for. Your asteroid has a moon too. Yes. In fact, apparently, it's actually, there's a little bit of a story there because it's not really my asteroid that I found two of them. I named one of them after my dad. I waffled. So Carolyn named it after a professor

at University of Washington and I said, oh, but I wanted to name it. And so I whined it. And she was nice. And so she gave me one of hers. And so then they named that one after my mom. And that's the one that has the moon. Oh, okay. So the asteroid named after my mom has a moon. How big is it? It's like the main body they think is about four kilometers across and trying to get the units right.

And the moon is about one kilometer. So it's actually, it's a pretty big binary. They're not quite twins, but yeah. Did other people in that class? Discovered? I think there was one other person. It's a little bit of a crapshoot, yeah. And the fact that I found two is quite unusual.

Was there a reason why you, was it just pure luck? It was just staying there all day, do it? Oh, I don't know. I know. I try to be careful. play. Yeah. How does one get an Oscar? Yeah, an Academy Award. Well, again, maybe I was at the right place. I

my thesis visor's named Al Bar, and I was a student intern, actually, in 1986 at a place called Schombardier, which is a well, a discovery place, but they had an AI lab. And so they were all doing sort of computer graphics research, and we were all thinking,

you know, at the time, this was revolutionary, I realized it's now considered incredibly boring. Like, you could actually use like physics simulators to make computer graphics movies. And at the time, it was like, wow, that's really cool. So I said, you know, you could use the theory of elasticity to make floppy things. And so I said, I'm sort of like, here's the theory of elasticity. And I wrote an elastic simulator.

And I made, you know, fabric and, you know, stretchy things and stuff. So they said, oh, wow, that's cool. And so you sort of descendants of that became a lot of the physics simulators that people used in, you know, Pixar in their various movies. So yes, I tell B. I tell an intern sort of half jokingly, well, if you do a really good job as an intern, you can get an academy award. So how long was that between the time you did that work? 20 years.

It was 20 years. Which is actually not atypical, right? Because they want to, when they give you an academy word, they want to make sure like, oh, yeah, it's sort of well used and everyone uses it and stuff. So yeah. But of course, obviously, like in that 20 years, like, well, everyone does this. It's obvious. But, you know. This was specific work done for a specific movie or something.

No, no. It was like a paper and sigraph. It was a paper, and then Pixar just became, like, I think, somewhat important chord to, like, a lot of their... Yes, and in fact, some of my friends, in fact, a lot of my friends did a lot of these simulators. So, yeah. I'm curious about since you have been quantum computing adjacent and worked on quantum computing directly at Google Applied Science as well, I think you're not currently on that. But I'm curious to see what are you... I'm still dabbling in quantum era.

Oh, you're still dabbling. Yeah, yeah. Yeah. Yeah. So what do you see the trajectory of quantum computing over the, over the, over the... years going because this is one of those things which I guess kind of like fusion was sort of had a sense of first like being very exciting and then seeming like it wasn't going anywhere for a long time and then maybe now we're seeing hence again of it being exciting and I'm or maybe that's like

my sort of quantum adjacent take on it. I think a lot of people have gone through that. I tend to average I tend to average things out over the day. I guess I'm old enough now or I sort of average things out over the decades and it's just on progress. But I mean like seeing going from you know, Starting from Feynman just trying to figure out even what this means as a concept all the way up to now where, you know, I guess there's this recent Willow result, a quantum error correction, which actually seems genuinely achievable with like the right scaling laws and stuff. I mean, does this, is this like a, do you still think that, or would you say that quantum is 20 years away or even until like some simple but practical algorithm which actually succeeds? Or, I mean, is it, do you see this accelerating or do you think this is still going to be something that's slog? Because, I mean, there's a lot of areas where we saw, oh, this is very, you know, this advanced very quickly and it was very unexpected. And I'm wondering if this is a thing that you think will be quick or will not be or is, I don't know.

I think sort of in between, it's not a purely software thing because we need to build systems that are large enough and stable enough to be able to, you know, be a quantum computer instead of a quantum. apparatus. We're in there what they called the NISC era. That was a noisy intermediate scale quantum, which was I think coined by John Preskill. It's not a very good term, sorry, but I guess that's this term we have. And there are, you know, the quantum team made this wonderful paper, which I think I'm co-author on, one of many, for something called the quantum echoes algorithm, which could be applicable now, which fundamentally, it's, it's in the style of Feynman's, um, proposal, which is essentially it's, well, the technical term is you could try to fit a Hamiltonian to observe data like it in NMR. What that means is, yeah, you have a physical model that's

parameterized and you use the quantum computer to kind of adjust the parameters and try to figure out inside of a loop to what the rate, so you could, for example, decode NMR parameters. So that is a practical thing that's used now. The question is, it will be big enough to make breakthroughs that's still TBD. The quantum team at Google has been very executing amazingly well against a roadmap that Hartman-Nevon laid out a few years ago and they're just continuing to march down this thing where they're scaling up

and when they hit their last milestone and they should be able to have a quantum computer that does amazing things and they're continuing to march it along. So, yeah, it's, I think, on the scale of a few to several years, I haven't sort of kept up on exactly what date they're saying. So you should ask Hartman exactly when that's going to happen. But, yeah, they're sort of marching along. So the interesting question is, will the superconducting computing be the winner or will one of the other alternate technologies? And that's still TBD, I still think superconducting is a very promising thing because it is very scalable.

So, yeah, no, I don't think it's 30 years away. I don't think it's tomorrow. I don't think there's going to be, I don't think, unless, well, even if there's a sudden hardware breakthrough, these are very finicky things. So it's like someone might have a brilliant idea, but it will still be a while before. Because fundamentally, at least in the NISC era, or for a while, these are fundamentally analog computers. And so that they tend to be very, very, very finicky.

So I wouldn't expect all of a sudden, you know, some phase change happens. No one has figured out how to scale up qubits in a way that, interact in just to kind of arbitrary size or still in the order of 100 to 100 cubits I think or you mean in terms of okay so in terms of well it's a little bit like yeah yeah exactly so for super connecting cubits um yeah people I mean although people are working on it the ratio of the number of physical cubits to logical cubits is still relatively large for the error rates that you need and so maybe there'll be a breakthrough there I don't know or there are other things that might have, you don't need that ratio to be so high because a lot of it has to do with the,

the, the, the, of the, of the chips that you lay out your cubits on a two-d chips. And for things like neutral atoms, they, in theory, in theory, can connect anything to anything else. But of course, in practice, we don't really know and we don't actually know what the limitations of neutral, at least I don't know the limitations of what neutral atoms. Maybe, maybe the super experts know. Superconducting qubits in particular have a sort of tension where you want to have nice clean resonators, which you get a clean resonator by decoupling from the environment, and then you get good interactions by coupling resonators,

which involves covering to the environment. It's true, but at least there, but there's, yes, but there's like the whole environment, and then there's like little tiny people that you want to go through to your neighbors. This is not like a fundamental uncertainty relationship or something. That's right. It's like this is a technological limitation,

that difficulty or something, but yeah. Yeah, the hardware. team in Google Quantum is very, very skilled. They're very, very skilled. So, yeah, they're really good at making these designs and making these things actually work. So I find them impressive. Yeah.

It'd be exciting to see that advance. Yeah. Yeah. I guess I'll just have to hold a breath and wait. Just wait. Yeah, I guess I'm just very patient. So I just start working. 20 years ahead of, you know.

That's what Dave Bacon, who runs the software team in Google Quantum. I'm always, he teases me like, John, like, oh, no, I can't work in quantum computing. This was like 10 years ago because it's going to be, you're always 20 years ahead of time. So I'm going to have to wait for 20 years. Like, okay, well, you know, 10 years have gone by. Halfway there. Halfway there. I don't take that as an absolute thing.

There's a hard rule. I also started working on Fusion 10 years ago. We'll see. Maybe you're actually causal. Like you start working at something and the reality just, you know, catches up. Catches up. Yeah. I guess so. Maybe, who knows? I don't know. I started working. I loved convolutional nets in the early 90s.

Jan Lecun claimed I coined the term convolutional net. And as far as I can tell, that may be true. Because everyone called it Lynette, because it was a very specific thing. They all worked for Jan. I said, well, I don't work for Jan. I don't want to call it Lynette. So I called it a convolutional net. So I don't know. That was a more generic term. It's a good term. Yeah. Before you go, is there anything you want the audience to take away any messages you want to deliver? I think Air is an example of it, but I'm really amazingly excited about the potential of AI for science. I think it's, I think it's going to be an amazing power tool for scientists. And I think scientists won't be replaced.

I think, in fact, I'm hoping that they'll spend all their time on, again, the creative stuff, on the rigorous stuff, on the philosophy stuff. And so I think it's going to, I think it's going to be way cool. I hope so too. Yeah. Yeah, my intuition is that a lot of, a lot of, I've heard a lot of people say that, that. I hope they're right. I hope it's not just sort of coping with a reality that's uncomfortable. The other question we almost forgot to ask,

if you could remove a bottleneck in your industry, which whatever you, however you wanted to find that, by fiat, then by magic. Yeah, by magic, what would that be? If I could get a magic wish, I would say, someone, please make the everything lab that you could like send J-San, on blob too and it will do any experiment at all. Okay, automated.

But it has to be anything. So essentially, so I guess we have to solve the sort of AI complete robotics problem, I guess. But if we did, then that would be stunning. Because right now, things like ERA, it's all computational. So someone has to gather the data. So yeah, if we could just break that, oh, oh, that would be so amazing. That would be so utterly amazing. Yeah, great.

So I really appreciate you taking the time to see us. And I think you like kind of flew in and adjusted your schedule a little bit. Yeah, I was sort of flying over San Francisco at home. And so I said I landed in San Francisco. So you made a big effort to be here. We really appreciate that. It was really fun to talk to you. Yeah, well, it's been a blast here.

Okay, cool. Thank you for having me. Yeah, you're welcome. Thank you.