The Great Simplification with Nate Hagens - Silicon Dreams and Carbon Nightmares: The Wide Boundary Impacts of AI with Daniel Schmachtenberger

Episode Date: July 17, 2024

(Conversation recorded on June 27th, 2024)    Show Summary:  Artificial intelligence has been advancing at a break-neck pace. Accompanying this is an almost frenzied optimism that AI will fix our most pressing global problems, particularly when it comes to the hype surrounding climate solutions. In this episode, Daniel Schmachtenberger joins Nate to take a wide-boundary look at the true environmental risks embedded within the current promises of artificial intelligence. He demonstrates that the current trajectory of AI's impact is headed towards ecological destruction, rather than restoration… an important narrative currently missing from the discourse surrounding AI at large.  What are the environmental implications of a tool with unbound computational capabilities aimed towards goals of relentless growth and extraction? How could artificial intelligence play into the themes of power and greed, intensifying inequalities and accelerating the fragmentation of society? What role could AI play under a different set of values and expectations for the future that are in service to the betterment of life?  We encourage you to explore the resources and research from The Civilization Research Institute on artificial intelligence compiled in this document: https://static1.squarespace.com/static/61d5bc2bb737636144dc55d0/t/66958505d89b99287c4ecab3/1721074950447/AI%2C+Climate+and+the+Environment-07-12.pdf   About Daniel Schmactenberger: Daniel Schmachtenberger is a founding member of The Consilience Project, aimed at improving public sensemaking and dialogue.   The throughline of his interests has to do with ways of improving the health and development of individuals and society, with a virtuous relationship between the two as a goal. Towards these ends, he's had a particular interest in catastrophic and existential risk, with focuses on civilization collapse and institutional decay. His work also includes an analysis of progress narratives, collective action problems, and social organization theories. These themes are all connected through close study of the relevant domains in philosophy and science.   Show Notes and More   Watch this video episode on Youtube Read the Development in Progress paper   To support ISEOF visit: https://www.thegreatsimplification.com/support

If there is a non-trivial possibility that the thing that we do will cause irreversible catastrophic harm, the burden of proof should be on proving safety, not the other way around. Right now, it's the opposite. Lead goes into the market, it gets put into the gasoline, it gets aerosolized everywhere, and only after a billion IQ points are gone in the U.S. and harm has happened everywhere irreparably does eventually the law regulate it. But as we're talking about tech that is radically more powerful and radically faster moving and scaling, especially as we're passing planetary boundaries, you don't get to say, oh, look at all the harm, let's reverse it. That'll never happen. So the precautionary principle says if there is really significant uncertainty and irreversibility of the consequences, then the burden of proof goes on proving safety.

You're listening to the Great Simplification. I'm Nate Hagen's. On this show, we describe how energy, the economy, the environment, and human behavior all fit together and what it might mean for our future. By sharing insights from global thinkers, we hope to inform and inspire more humans to play emergent roles in the coming great simplification. I would like to welcome my colleague Daniel Schmachtenberger back to the program to take a deep dive. on artificial intelligence, potential impact on the environment and particularly climate change. I have recently gone to some energy tech climate conferences where AI is in every other presentation at least talking about how it's going to reduce emissions and be a general boom for the environment. My gut and my conversations with Daniel, Tristan Harris and others tells me this is not likely the case.

So this conversation is a follow-up to our original one on AI and the superorganism. This was a deeply informative and oftentimes disturbing conversation, especially the 20 minute in the beginning before we got to AI and climate was talking about the broader macro story of artificial intelligence, human power systems, and what's happening. and some worst case scenarios. So you be warned that the first 20 minutes are kind of intense. Personally, I think Daniel is directionally correct on what he lays out in this podcast. I hope that the magnitude of what he is predicting is incorrect on behalf of life and the things that I care about.

I have to say, though, that since I met him three or four years ago, he's been very accurate on artificial intelligence and what's unfolding in our society with respect to AI. I think we're going to run into energy and other limits before what he suggests in this podcast manifest, but I defer to him on this topic. One final note is we keep this conversation at a high conceptual level, but his team has created a 20-plus page document of references, resources with actual data on what's going on in the ground. We will update that in the show notes on YouTube and on the great simplification.com website. So you can look there for further data. The conversation continues, my friends, please welcome Daniel Schmockenberger. Hello, Daniel. Great to see you. Good to see you, Nate. Of the hundred odd topics that you and I could discuss relevant to human futures and in service of life, we're going to talk about artificial intelligence today. And even on that topic, there are many, many subsets, how it affects inequality, profits and power and military and corporate. You know, different new technologies.

But today we are going to focus on what AI portends for climate change and the environment, because I don't think it's often discussed. So I just got back from a climate energy tech conference where many of the people presenting were offering this or that technology that was going to be better for climate because it was using artificial intelligence. So I know you and your colleagues, your team have been working on addressing this question. Why don't we start by giving an overview of how artificial intelligence in its current form could be good for the environment in theory? What are the main threads from there? Sure. There's a pretty comprehensive paper. Let me see the name, tackling climate change

with machine learning that was written last year that addresses pretty much all of the claims of how AI, machine learning, and the associated sensors and automation could help. And it's everything from being able to model natural phenomena better, like being able to model how icebergs are melting better to predict rates and things like that, to modeling, dynamics better for being able to do aerosol-based geoengineering more effectively, to being able to create efficiency gains in lots of areas that supposedly should create a decrease in energy or material use because of the efficiency gains. And then the really big areas are the idea of like AI being able to help fundamental scientific and technological breakthroughs, things

like nuclear fusion in particular. And energy generation, energy storage, way better batteries, you know, things like that. But, you know, nuclear fusion, room temperature superconductors, those are kind of like holy grail claims. But, you know, if you look at all of the areas where efficiency could be applied, if you think about all the places where you've got oil lines or natural gas lines, the ability to have sensors. all along them that monitors a leak and that can, you know, use some kind of machine intelligence to be able to identify where the leaks are happening to, and then at minimum to direct people there, but pretty soon to kind of have automated robotic servicing of it happening. Obviously, you'll have less leaks. And so there are lots of areas where if you're applying

optimization to an efficiency problem, you'll get some gains. And how many of those are theoretical and how many of those have you seen evidence that are actually manifesting already to some small or greater degree? The things that I think are for certain possible is wherever we're talking about sensors where there's lots of data and you need AI to be able to model all of that data, that's straightforward. Like, that type of technology exists, is getting better. Ironically, one of the major applications of it is to make the extraction of oil cheaper and more efficient. And so, you know, 92% of all oil companies are in major contracts with AI companies to develop their oil and fossil fuel extraction using AI techniques, everything from, like, Super technical to be advanced stuff, like being able to use AI on seismic records and underground acoustics to be able to see exactly where the underground resources are and stuff like that to be able to make the exploration and exploitation costs much cheaper. And all the big AI companies are servicing that industry.

Invidia is providing chips and working with big traditional energy and not on just the renewable side. And Microsoft services has whole contracts and products developed exclusively for oil industry, things like that. But I would say those types of things where we're looking at automation, well, excuse me, where we're looking at lots of data and the ability to make sense of that, that's a pretty solid thing. Where we're looking at also kind of marginal efficiency gains that don't require fundamental breakthroughs, that's going to happen. Where we're talking about fundamental breakthroughs, so the way Gates said it, I think, is kind of clear. He's like, the types of AIs we have so far, as far as material innovation, tech innovation, speed up stuff that humans are already doing pretty well with, but they're not doing the types of things we're really interested in that humans aren't making much progress with. And so if you think about like AI has been used in nuclear energy and fission for a long time to help both with stability of the fission centers and things like that. And it's the breakthroughs in that space have been marginal.

So things like will it really create massive breakthroughs in the speed to fusion or things like that? Those I would say there is no, we can see why people think maybe, but there's no precedent for high confidence, particularly in any timeline of those things. I don't know about you, but when I look at LinkedIn and I talk to people not in my core circle, but in the broader people paying attention to the world. I think the consensus is that AI is going to be good for the environment because of many of the things that you said. It's going to make renewable technology and batteries and clean tech more efficient. And on a narrow boundary view, which you and I don't specifically use, we like to look at a wider boundary. And we're going to get to that in a second. it's positive for using less energy materials.

Would you agree? I think that is a popular narrative. I think that there are a lot of people in the environmental space and the climate space who don't think that's true. But I think there's a lot of people who are broadly concerned about environment and climate, don't understand it in great detail. And I think there's a bunch of reasons to kind of buy that narrative. Obviously, from a marketing perspective, if I've got a product or service, then I look at all of the various market sectors and try to figure out how to market to them and why my thing will be useful to them. So AI and defense is one of the biggest topics.

And it's kind of like there actually is no defense conversation that is not AI in defense right now. It's AI for not just like autonomous weapons and drones, but it's also the precision targeting is AI. Also increasingly comprehensive battle planning is happening on more complex AI systems. intelligence gathering, intelligence gathering of huge amounts of intelligence that people have to sort through is not that useful, right? That's only useful with AI systems. And so similarly, if I want to talk about law enforcement, AI can actually show you where the bad guys are because now we can actually make sense of all of this data. Does that look like AI empowered ubiquitous digital technological surveillance? Yeah, which is pretty bad.

But if you look at pretty much every sector, right, AI is going to radically improve education because now we can have these AI tutors that know everything that get to tutor your kids directly. And AI is going to help old folks homes because now you get to have these chat bots to talk with old people so they aren't lonely and it's going to improve medicine. So the idea that AI is going to radically improve climate change, like, okay, so climate is an influential topic. It affects international policy. it affects national policy. So we got to get those guys to think this is a good thing. And so let's talk about all the ways that it's going to help. So from a supply side marketing to important demographics perspective, of course, there's going to be a lot of resource that goes into that.

And that's spin, right? So that's one thing. Now, when you look at just the rise of Nvidia's market cap and associatively everybody who's working in AI. So, you know, you look at the top 10 companies by market cap right now, and they're all AI companies, right? Like, so Apple's not an AI company, Microsoft's not an AI company exclusively, but fundamentally, like that's one of their, if not their primary plays right now. Arguably, you could say the only one that is in the top 10 right now that isn't is like Saudi Ramco. But increasingly, AIB, they're using their money from oil to fund. AI efforts massively and they're using AI to fund their to advance their oil efforts.

So, um, and obviously this is new, like that they're all AI companies. It was oil and defense and banking. Um, that market sector being as big as it is and those companies being as powerful as they are means that there's a lot of marketing and a lot of lobbying and a lot of public influence. Consilience project, we wrote a paper one time called Where Arguments Come From. And it was basically like the think tank industrial complex, you know, etc. Kind of conversation of so there's an argument circulating all over the place. That was not a natural selection of good ideas.

Good ideas don't get naturally selected. They get upregulated by whoever it is that has marketing budgets to get that idea out a lot and to be able to pay research firms to develop the research that. they want and authors and, you know, et cetera. So though, where arguments come from thing is really important to think about. And of course, if we're saying, okay, well, what about the environmentalists that are concerned about how much AI is actually going to radically damage the environment? Do they have comparable marketing budgets? And do they have, like, the correlation of force and means between those is as bad as it could be? It's kind of like

peace activists compared to the military industrial complex correlation of force and means. there's another reason that people want to believe it is not just that they're being sold it like crazy, but a couple reasons. One is it's kind of where all the money is happening, right? And so if I want to get ahead and at minimum not be left behind, I don't want to be against the thing that I can't stop and everybody who's writing is going to get ahead and everyone who's not going to be left behind. So I would like to believe that writing that thing is good. And I would like to believe that writing that thing is good. and maybe I can steer it in a good direction. Right? Like, duh. And that's why even in the just not AI and climate, just AI space, there's this joke in the AI safety community that the fastest way to accelerate AI risk is starting an AI safety org. Because, you know, Open AI was started as an AI safety org to try to protect the world against the dangers of deep mind.

and now it is radically accelerating, then Anthropic left because they were scared of its acceleration, I wanted to make a safety org, but then they're like, hey, in order to be able to really test our tech, we have to build tech, which means we need a lot of money,

so they took $300 million from Google, and now they're racing, and then Elias left because he was concerned, and now he's got a company focused on super intelligence. And so it, and having privately talked with, you know, many people in the space, in addition to whatever their other concerns are, one of them is my job will be automated by AI soon. Right now, all of my friends who aren't working in AI safety and are working in AI development are making boatloads of money.

They make their own little startup and get acquired for a stupid amount by one of the big ones. It does some nothing. It doesn't even matter what it is like AI applied to have a business. brilliant, brilliant friend who's basically doing AI image recognition applied to taking certain kinds of files and putting them online with like a couple percent more efficiency than the previous type of image recognition. And he's like, I can get acquired for like a tremendous amount of money for something I can build in a couple of years. And I'm probably not ever going to be able to have a job in the future. And so that's another reason to want to be on board with the like

AI is good story. And then the other one is that if you really care about climate change, you're probably pretty depressed. And the hope doesn't look that good. And any kind of hope. So it's like, why is the Jesus is going to come save us all narrative compelling for some people? Why is the maybe the UAPs that we're seeing are benevolent aliens and they'll save us all compelling? A has another version of that. Like, we're, there's no good sign that the answer is going to come from us.

We seem pretty intractively fucked. Maybe something radically smarter than us can solve all the stuff, which is, of course, kind of like a regress to a childhood psyche, still wanting a parent who's going to kind of figure it out. So, A, I does a very believable job of that currently for some people. Just listening to that brief summary that you just gave, it really sounds like a Twilight Zone episode. like science fiction that would have been written 30, 40 years ago, and yet this is our world that we're living right now. So wait, just real quick,

in terms of the sci-fi episode, um, we, we had somebody put a clip together of things that people who are, um, like running the top AI labs publicly have said, you clip it together and like many people probably seen where Sam Altman, you know,

has said, uh, several times on stage. AI will probably kill everyone, but it'll make a bunch of really profitable companies first. And- Did people laugh at that? And everybody laughs and invests,

and the market cap goes up. And then- What does that say about our species? So, and then, like, you know, Elon said many things on this topic, but, you know, one of the clips recently was,

him saying, I used to be kind of bummed out by this, like, AI will probably kill everything. But then I thought to myself, what I like to be here to see the AI apocalypse or would I not like to be here? And I guess I'd like to be here to see it. And, but, you know, these are the people who are building the most powerful AI systems in the world. And the argument is we can't stop it because of races. We can't stop everybody. If we can't stop everybody, then whoever gets to kind of AGI dominance first will run everything forever. And so if you feel like you could participate in that game and you think about it this way, there is only one game.

There's only one game, which is be the leader of or a part of the group that makes it to AGI supremacy first. Because if so, so one, if you can't stop it, every attempt to try to stop it is futile. It sounds a lot like the Borg or like Sauramon saying to Gandalf, like it's inevitable, you must join Sauron. Like it has that kind of feeling and you mentioned sci-fi. But so you hold this inevitability argument. And so now the fact that I'm not trying to stop it isn't an ethical issue because couldn't anyways. Right. And.

And then if you hold, and then this is one of the most fascinating motivated reasoning cases. Actually, I have never seen motivated reasoning. This, that like captured this many smart people, this intensely, this quickly. It is the superlative case of motivated reasoning in my life experience. Where everybody who ends up saying, I need to run an AI company. based on this argument says who the AGI, the emergence of AGI is now inevitable. We don't even need major breakthroughs and cognitive architectures. We're going to get there just by scaling the stuff we have.

Nobody could stop it because you can't stop everybody. You can't stop China, whatever. And it will probably be dystopic, maybe kill all. hydrocarbon life, like quite likely. In which case, if I get there first and it becomes totally out of control, it doesn't really, I'm not safer. But if there's any chance that whoever gets there first still kind of controls it or can merge with it, which is why, you know, neuralink brain computer interface, you know, Elon is

working on that, Google's working on it. Like almost all of the kind of big players are working on. And you'll hear Nick Bostrom say there are publicly recently. There are lots of reasons to prefer to be digital than biological. Biological things die and suffer and are limited and et cetera and our only work on this planet and AI things can work in space and be eternal and become digital gods. Ted Chu, who was the previous head of the Abu Dhabi Sovereign Wealth Fund was also the chief economist. of GM for a while, wrote this book called Transhumanism and Human Potential and basically said there's this kind of religious narrative of humans ascending into angels or higher

selves or overmins or gods or something like that. And AI and synthetic bio and brain computer interface is what actually delivers that, where we can do whole brain emulation and upload our consciousness on. to the cloud and move from being slime-based computational systems to crystalline-based computational systems that can, you know, live forever and et cetera. So that whack-a-doodle metaphysical idea is pretty universal and dominant in the AI acceleration space. And as a result, if somebody holds that by the consciousness and life are purely the results of computation that you know universe happened for who knows why big bang why are the constants what they are who knows eventually on this

earth life started to emerge somehow and then somewhere after life around neural network something called consciousness emerged and it emerged because of a complexity of computational process that's happening in neurons and whatever. And it's because this hydrocarbon thing can do computation. That's the kind of computational neuroscience models that everything that is happening in the body that matters is to support the brain and everything that's happening in the brain is basically to support computations. And that it's computation running on this hydrocarbon, i.e. slime-based computational substrate. And that as soon as we get adequately complex computational substrates on silica, you can run the same thing. But that the evolutionary process,

David Pierce is probably the guy that has done more on this than anybody. David Pierce at Oxford and Nick Bostrom created the transhumanism movement together. And David Pierce has something called the abolitionist project, which is that the moral imperative for humanity is to abolish the potential for pain and suffering from the entire universe by genetically engineering. opioid receptors out of humans that could experience pain and not just humans, but all animals and all life, genetically engineering predation out of universe, et cetera, et cetera. And of course, once we can get past genetically engineering us to just being computational beings, that's the ideal. So if, and he says evolution selected for us to be, to survive, which meant lots of suffering

motivations, not to be happy or whatever, we can do a better job now. So if, If you hold that by biology life is basically just the happenstance that was able to bootload itself on this planet, right? And Elon has said hydrocarbon life is probably the bootloader for silica life. It loaded first to be able to evolve to the human place, to be able to make the silica system that will then take all the hydrocarbon life's atoms and repurpose it into the silica system. That's what it means that it's the bootloader. because silica didn't automatically self-organize on the surface of the earth, but hydrocarbons did, and they figured out how to make the silica thing, and then the silica will use all the atoms to make the thing it wants to make. So the idea that we're the bootloader, but that's awesome because the hydrocarbon things basically just suffer and are very limited and have cognitive biases and are emotional and are kind of nasty and are limited to this planet, which is going to be,

but the AI systems can go interplanetary and, you know, be digital gods. You don't care about killing all life. It's a moral imperative to kill this thing and replace it with that other thing, right? So that kind of test-greal metaphysics, it's not that everybody is motivated by that. They're motivated by different things. But that one is very deep for a lot of folks. People should read Nick Land and Nick Bostrom and David Pierce and Kurzweil on this to see who are the philosophers that kind of all of Silicon Valley follows.

So there is an argument that whoever gets to AGI first, if AGI takes everything over, I'm not still going to be able to control it, but maybe I can merge with it. Brain computer interface, orienting towards whole brain emulation. But maybe I can control it. And if I can control it, then whoever gets to AGI first runs everything. So if you look at Alex Karp, the CEO of Palantir, which is AI applied to intelligence and military capacities, his public talk. recently. He says, U.S. has to get AGI dominance and then use that capacity to ensure that no one else ever gets it. Now, what that means, like, okay, well, how do you ensure that China doesn't advance? Well, because AGI dominance can turn their nukes off, can turn their banks off, can whatever you need to do, right? So it's actually like an open public

call for a global one world government infinite monopoly of power with no checks and balances on it, sitting next to Eric Schmidt and, you know, the heads of U.S. military and defense being like, yep, this is the imperative. So there's the idea, if we get to AGI and can control it, we run the world. If we get to AGI and can't control it, at least we have the best shot of knowing how to merge with it. If anybody else gets there first, not only are we losers forever, but we have no say in anything ever again because the AI will now run everything. So if I care about anything ever, the only play I have is get to AGI first. And whatever I have to harm to get there first is worth it because then that's the only way I can do stuff.

That is kind of an ubiquitous argument. And so why I say the motivated reasoning is everybody who makes that argument, argue, why. It's inevitable we're going to get there. It'll probably be dystopic. Maybe it'll be utopic. But I need to be the one to get there first because I'll make the least dystopic for everybody. And it's like, really? Really? Like, why everybody's argument is that they're the best ones to do it? Not like, I think that guy is the best qualified. Let me go help that project. Nobody makes that argument. And so you're like, do you believe that yourself? Or is that just the public rhetoric you're giving? That's the motivated reasoning story.

I told your colleague, Zach Stein, on our podcast that at times like this, I long for just the days of when I was only worried about peak oil. Because what you just described is not a science fiction novel. It's a Stephen King novel. It's in the horror genre. So I want to get back to the environment and climate change because that was. the focus of why we wanted to unpack this. But let me just ask you a follow-up. In the minds of those AI

accelerationalist humans who are working on this, what about Redwood groves and dolphins and hummingbirds and earthworms and bumblebees and the 10 million species that we share this blue-green earth with? So if I am being an AI accelerationist saying, China is rapidly on its way to AGI dominance in a war we have to get there first or lose or some other version of that, right? And you bring up that question.

Depending upon what marketing had I have on, I could answer a few different ways. One way, like the most true way is all of those things are good sources of atoms that the AI will use to build the things. it wants in the future. Seriously? So if people have not watched Eliaser Yudkowski talk about AI risk, Eliaser founded Miriam Machine Intelligence Research Institute

and I would say has put more time into the topic of AI risk than anyone is clearest on that. And he says, look, it's not that in most scenarios the AGI or the ASI superintelligence. It's not that it

wants to kill all humans. It just wants to do whatever its objective function is and it needs atoms to do that and things happen to be made of atoms. In the same way that most people, when they're relating to those other animals, don't have hatred and vitriol to the redwood trees and the whatever. They just like want wood. They want stuff and like the trees are made of wood and the animals are made of meat and the nature is made of energy and materials. And so we turn it into the substrate of the shit we want more. Whoa. So it's just a higher level scaling of that same dynamic.

I can, you know, chat GPT has all these guardrails on what it's not allowed to say. But you can figure out hacks to kind of jail break it and get it to say things. And then there are obviously open source ones that are not. So we were playing with it a little while ago and got it to make a bunch of answers about how AI can kill the whole world, just a bunch of things. How would it use synthetic bio to kill all humans? How would it use nanotech? How would it use on and on to just see what will I actually say publicly?

And it gave very, very detailed plans on how to kill everybody using all those techniques publicly in the non-gill broken version. And I'll actually share that document and you can link it in show notes if people want to have a look at it. And so one of the questions was I was asking about if it could start to run on a quantum computer and work on optimizing the number of qubits of the quantum computer on an advanced quantum computer running a more advanced version of its transformer, how intelligent would it be relative to humans? And its answer was, you know, while IQ is an anthropogenic human-centric score that is, you know, irrelevant and can't assess all of these things, if we were trying to use IQ, it would be something like millions of times what a human is. And it would be comparable to the intelligence difference between a single-cell creature and a human. And so humans can't begin to fathom the nature of what that thing is any more than a cell can fathom. can fathom what it's like to be a human. I have a pretty strong opinion on the way forward.

I'm going to hold on to that. You asked the question about redwood trees and bees and nature and the deepest answer is like the AGI question long term is those are all made of atoms and have energy and stuff in them. The answer that would be given probably, so if you're focused on AI military dominance, So after China did its military drill near Taiwan as a kind of a punishment to Taiwan last month, the U.S. Pacific Naval Commander initiated Operation Hellscape, maybe some of the people have seen. I like that it's not like called Operation Mockingbird or something. It's just Operation Hellscape, just kind of like openly acknowledging. and Operation Hellscape is put so many, fill the Taiwan Straits with so many automated robotic kill drones that anything that moves in there is dead instantly. And, okay, so if, so that's now, right? Like, that's not in the future. That's now. If people have not watched Anderil videos on what the state of integrated automated killwebs all.

and stuff like that. It's worth watching. So if you're focused on eminent World War III, where AI is the kind of primary game change are applied to everything, and someone says, but what about bees and redwood trees? Then you're like,

you can focus on irrelevant shit because you aren't actually playing to win, but there's really only one game that matters right now and it doesn't have anything to do with bees or redwood tree. and I have a limited amount of attention and all of my attention is allocated to what it must be allocated to. Now, the answer that they would give publicly

would be more like if you want to protect bees and redwood trees, you better support that we get to AGI dominance first because we are the good guys and the Chinese are the bad guys and we will protect environmental laws and et cetera, et cetera. So the only way to care about those things

is to support us in our AGI dominance. I'm actually feeling somewhat ill. like I have a pit in my stomach right now. I knew some of these things, but this is just, and by the way, the reason that you and I decided to schedule this is because we believe that the environmental movement, the climate movement is unaware of the magnitude of impact that artificial intelligence will have on the issues that we care about.

But I have to ask you a question first to finish. this section, which is, I'm, I fear your answer. But in the same way that the movie Idiocracy was a comedy and it ended up being kind of a documentary, Black Mirror was like this futuristic dystopian thing, but that actually is starting to be a reality with some of the things that were described in that show. So here, here's my question. I'm actually afraid to ask you this, but we're doing this. I'm doing this podcast. I'm trying to pass the pro-social baton to more humans to change the initial conditions of the future and to kind of fight power in service of life. But based on what you're telling me, what if 95% of humans

absolutely understand and concur about these risks? Are they just like gnats that can be swatted away because of the power dynamics that exist in this industry and what's happening? Not yet, but very soon, which is why now is important to talk about this. Let's talk very briefly, and then we'll move to environment. Let's talk very briefly about what's called ubiquitous technological surveillance. People can look up that term. Ubiquitous technological surveillance means you've got, all right, I'll do a little bit. Wi-Fi is bouncing around my room because I have Wi-Fi routers in here, right?

So that's just to make my Internet and phone work. But the Wi-Fi signals will bounce off everything in the room, create patterns just like echolocation or whatever, that are being monitored by the system where AI can interpret it. So AI systems have now been used to be able to see everything happening in a space by interpreting the signal on a Wi-Fi router. So it can watch me walking around the room know exactly where I am, whatever, anyone who, just by having AI's ability to access that kind of interference pattern on a Wi-Fi router, right?

That's one example of a type of technological surveillance that is only made really interesting and possible by AI. But also... Even if there are no cameras turned on or anything? No, we're just talking about the Wi-Fi itself. No cameras. Now, of course, the cameras and the sensors and the automated home sensors and the fact that all the sensors and new cars are all aspects of technological surveillance, whatever, all of what we call IoT, right, the Internet of things that all have sensors that are hooked to the Internet. But also, like, we have been able to gather people's texts and emails and communications and whatever for a long time.

But nobody can read all those. It's a lot of content. AI can read all those, right? And LLM can read all those. And now we have satellite images of the entire surface of the Earth that are pretty high resolution every single day. Right. There are many companies that do that. Planet Labs is a particularly advanced one.

And they're getting higher and higher spatial resolution, meaning being able to see smaller things. Temporal resolution, meaning more pictures per day, moving. And spectral resolution. meaning not just in the visible range, but being able to see mission spectra and stuff like that. So where that is heading, and it's not far from, is real-time video, you know, at like a human level of the entire surface of the Earth from satellite. And that's just satellite. That's not all the other sensors. So now take the satellite, take all the other sensors, take, etc. Can you have AI systems? It makes sense of all of that to provide real situations. assessment of what is happening. Totally. So ubiquitous technological surveillance at a global level. One of the things that limited autocracies and despots in the past was they couldn't

monitor everything. So a rebellion could foment in a basement. People could have, you know, dissenting ideas, whatever. So does AI applied to those types of things enable dystopias, not just catastrophes, but dystopias, unlike anything that anybody's ever been possible to do. I mean, people have imagined it. There are sci-fis of this. Yes, totally. And then if somebody looks up what are called automated killwebs, which is kind of a military concept of you've got a bunch of drones and various types of military vehicles and a bunch of types of sensors and all the sensor systems on the drones and everywhere are sharing information in real time. So if any of them are taken out, the other ones have all of it and can identify targets to automated kill.

You kind of see, put those together, put the satellite systems, put the space-based ultra-short pulse laser systems, etc. Put the automated kill web type systems together. And it's not that hard to see what the convergence of the technology is moving to. So when you ask, are we just Nats, in that world, yes, but we're not there yet, but we're moving there as rapidly as we can. There is an image that shows Moore's Law relative to Nvidia's GPU growth, and Moore's Law was the fastest growth rate of anything technologically ever. It broke all the kinds of records of like, whoa, continuous exponential growth. And now you see Mars law tracked and it looks like a flat line in comparison to GPU growth. And because even though they're both exponentials, they're exponentials with very, very different exponents.

If 95% of the world today said, holy shit, this is a world we don't want and they recognized how urgent it was and applied all of their energy to leaning into it, could it continue with only a small number of people wanting it, if everyone was being effective, no, it could not. So are we in a hopeless situation? No, are we in a urgent situation? Totally. Do you sleep? And if so, do you dream? And if so, do you dream about this?

Yes, sleep. Yes, dream. Often dream about these things. It's so intense. And you and I talk every few weeks and stuff, but because we're working on other risks too, like the nuclear thing and climate and the oceans, it's just sometimes,

uh, our ancestors didn't devolve to handle this amount of complexity and, and terror. You know, interesting. I, I dream, I have dreams about these scenarios because people dream about the stuff

they're thinking about. and I'm thinking about this stuff all the time. And sometimes they're helpful, right? Sometimes I actually get insights of like, oh, I hadn't seen that. That was really worthwhile. But I think even more than that, I actually have dreams about, like, my, I'm very fortunate in this way. I have dreams about like waterfalls and kids playing because that's actually what I'm thinking and feeling about underneath it all the time.

That's beautiful. That's a gift. and that's actually some emergence in our individual human minds, maybe something like that could scale in response to growing awareness about this stuff. Okay, so let's get back to the main thread. So to summarize, AI ostensibly could be positive for the environment because in a world that we're running out of cheap fossil. energy and fossil energy has a lot of carbon footprint and renewables are incredibly

materially needy and global complex supply chains, etc. We could invent nuclear fusion. We could have room temperature superconductors. We could find new ways to find lithium and things needing in the supply chain for renewables, all sorts of science breakthroughs. Okay, I can understand those things. But I think AI is both energy and ecology blind in its broader impacts. And as we are reading in the news, AI itself in the training, in the servers, et cetera, is incredibly energy hungry. So can you give us an overview of the, and we're going to talk two frames here, but on the on the microframe, what is the story, give us an up to date overview of the energy and material use needed by artificial intelligence?

Maybe we'll just up front speak to a few concentric circles of how to think about the environmental impact of AI. Okay. And we'll focus on energy use a little bit more, but also materials use. pollution, et cetera. So the central thing you could look at first that a lot of people are looking at is how much energy are the data centers that are running AI systems using. And a lot of people have had concern about this because it's a lot. There's actually a really, really nice piece in Bloomberg recently that made a very good graphic visualization on this and showed the size of the data

centers and how much energy they're using in the growth rates and it's a good reference. And the, you know, like a couple stats from it are that data centers collectively, right? The global data centers use more energy than every country in the world uses energy except for the top 16. So use more energy than Australia, use more energy than Italy, et cetera. they're also the single fastest growing energy demand in the world as far as all sectors go. So this is just data centers, not AI data centers. This is data centers and the fastest growing demand in data centers is AI data centers. And there are some areas that have lots of data centers.

They're huge ones in Texas. Virginia has massive ones, Malaysia, you know, etc. And like the ones in Virginia have had so much kind of intensity and growth that they, the overall electricity grid is beyond capacity and getting these kind of load balance errors very regularly. And there are lots of other companies that are on like multi-year waiting list to have access to energy because the energy demand so. far exceeds the energy supply. So that's on the topic of data centers. And so this is why a lot of the big AI companies have said the limit for our growth is not the ability to put together a huge GPU cluster. It's the ability to have enough electricity because right now, like, it's shutting, for us to try to run them in full, it'll shut down California's energy grid. And so we have to build new nuclear power.

plants to be able to power these that are our own plants because right now we're having to pull energy across multiple states and it's too slow and the transmission lines aren't big enough for it and stuff like that. That's the topic that a lot of people are focused on, which is the energy use of the AI systems directly. Could I ask a quick question there? If you have a data center, I assume I know the answer to this, but could it allow or tolerate any intermittence?

Like if there were lots of solar panels. and wind turbines, an AI data center presumably has to be 24-7 with not even a one-minute downtime, right? So it needs 24-7 guaranteed. Yeah, I mean, it would be listed as critical infrastructure. Critical infrastructure needs energy storage if the energy supply sources are intermittent, which is why there is renewable build for these, but there is more energy going into nuclear build. And of course, then if you need lots of batteries. That's another aspect of the material demands. Got it.

So beyond the energy use of the data centers and let's just say kind of what AI uses directly, then you have the material use and of the GPU, but also of the entire supply chain, right? So the materials needed for you've got, you know, gold and plastic. platinum and et cetera in the circuitry, you've got rare earth metals in the semiconductor doping and, you know, all those kinds of things. You've obviously got not just energy, but a lot of water needs for the cooling of these systems and land use. And because of then land use, it's you have what the local electricity grid dynamics are. So who, as the supply is not keeping up with the demand, the cost goes up. There are places and kind of bidding wars for property that can make huge data centers,

which also then drives up the electricity cost, driving kind of poorer people out of electricity access. So there's a lot that happens not just in terms of the increased energy use, but in terms of the change of power dynamics associated with who gets access to the increasingly expensive rate limiting resource, which is energy here. So that's kind of the first thing, is just that. And that by itself is a big deal because already Microsoft had had a commitment to have itself powered by renewable energy by 2030 that now they said they probably can't meet because of the more urgent imperative to be able to grow in these ways.

But this is not really the big topic when it comes to AI's effect on the environment. Well, it's, you know, you and I have discussed this before, that artificial intelligence is downstream in the human hierarchy of priorities than economic growth. And so as long as we're using AI for the same cultural aspirations as we have been, there is, is going to be the same Jevins paradox or rebound effect that we've had the last 50 years or so. Since 1990, we've become more energy efficient by 36%, yet our global energy use has increased 63%. And I don't see how AI is going to change this dynamic. And just for definitional purposes, just to clarify, the rebound effect is something that if there's a 10% improvement in technology and we get more efficient,

but there's an increase in the use of that thing, less than 10%, like say 6 or 7%, that's a rebound effect. But if you have a 10% improvement in efficiency brought about by XYZ technology, and it results in a more than 10% increase in overall energy use in the system, say 15% or 16%, that is called a backfire effect, which is the Jevins paradox. So you and I, when we've discussed this in the past, most people are looking at the narrow circle and saying, oh, well, AI will require a little bit more energy, but that can be overcome with efficiency. but if you're saying that this will be applied to all different processes within our world, and there's a little bit of a productivity boost on each of those, that extra productivity rebounds back and backfires so that the real impact on CO2, on climate, on the environment, on ecosystems is going to be this giant backfire on all

the things globally. So please tell me what you think about. that and give me some examples and insights. If people don't understand the Jevin's paradox phenomena, then efficiency is a very hopeful thing. I remember when I thought that. And then if somebody does understand it, then they recognize that efficiency is not a hopeful thing, right? Because on its own, because the gains in efficiency will lead to more total energy use, material use, et cetera, not less. And so that doesn't mean we're not interested in more efficient stuff.

It's just the more efficient stuff technologically has to correspond with legal and economic changes that also bind not growth. And so if like taken in isolation, we can look at a curve of the development of solar cells or wind energy or whatever and say, oh, look at this awesome curve of how much renewables are going up. except what you and I know and probably all your listeners know is that simultaneously fossil fuels use is going up with a steep curve. So it's not that that curve is taking a larger percentage of the total energy that year over year, we're using more fossil fuels every year as the renewables are growing. So you're like, okay, the renewables are not globally. Globally. Yeah.

Yeah. Which is the only metric that matters for global common issues like climate and ocean and stuff. But it is one of the nonsense things countries will do is kind of export their badness elsewhere and pretend that their country metrics are awesome. And so, but that is like, that's just statistical warfare. Like it's just, it's just goofy, right? And we've talked about this before, but like you can't say the genie coefficient is good. We don't have many people in extreme poverty, but we import shit from countries that are made by slave labor and not include that in your genie coefficient when your country doesn't make the stuff it needs to survive. And so genie coefficient only is a real thing if you do partial attribution analysis globally of your entire supply chain that you depend upon.

And ditto with your environmental footprint. Exactly. And so like these are just global numbers in a world of global supply chains and with no country producing all of its own stuff that it needs. So like country, country's not the right metric, right? It's a civilizational metric. What is the minimum unit of civilization? Well, it's the unit that can produce all of its own stuff. That's a sixth continent global supply chain. So this is actually a really depressing thing. But it can also be a clarifying thing, which is, The first scientific paper on climate change was published in 1938. Climate change has had so much attention from so many significant scientists, powerful thinkers, obviously like a vice president of the United States of America. Like most powerful country in the world took it on as his main jam after being vice president and got everybody's attention on it. International agreements have been made.

The United Nations Environmental Program was made over 50 years. go to get all the countries the world to work together on global commons issues. And like there are trillions of dollars in climate funding that are allocated. There are, you know, intergovernmental organizations that just work on this. There's support from NASA and NOAA and super powerful technological orgs. And yet pretty much every single year, fossil fuel use has gone up. And the couple little dips have nothing to do with environmentalism, right? Like the couple of little dips have to do with recessions.

Resessions. COVID for a second. And it's like, okay. So that's enlightening. So in a way we can say, if we just looked at the curve of like solar panel go up, we say, look, we're succeeding. But if you look at fossil fuel use go up, you're like, 100% of all the activity that has been done has not even slowed fossil fuel use. it hasn't even made the curve inflected down, right? And it has to obviously stop it and then reverse total energy needs and, you know, stuff like that.

So why this is helpful is to say, okay, we're actually not on track. Let's not like take bullshit narratives that somebody sells to inspire people. Like we're not on track. and the totality of the approaches we've employed so far will never get on track. They're not converging. What would it take? One of the things it would take is as long as I can invest a dollar and get more than a dollar back or invest a jewel of energy and get more than a jewel of energy back, there is an incentive to do so. right? Anytime there's a return on investment of something that has

adaptive advantage to have, there's an incentive to do it. So, if we make renewable energy, it's like, great, there's more energy. We need energy for everything. Energy corresponds so very closely to dollars. We want more dollars. That doesn't mean we stop using the coal energy. And the

natural gas and the oil. and everything else. It just means more energy we use all of it. And now actually, we have more energy to advance the AIs to learn how to oil exploit faster. And so if we had something instead that was like every time a new megawatt of renewable was created, a megawatt of hydrocarbon had to be shut down and there was international agreements to ensure that they were not created again, right? like something like that, which is a legal binding of the Jevin's paradox associated with the transformation from more toxic to less toxic industries and or movements and efficiency. You get a, you know, increase in efficiency somewhere and you have a legal binding that says that those gains don't get reinvested into more total use, but actually keep the overall domain at the same size and just use less stuff. But that wouldn't work unless we changed our cultural objectives. Because if we still wanted to maintain this 19 terawatt global metabolism, we could not physically swap those out because it wouldn't work.

So we could only swap them out if there was some, you know, agreement to use less and maybe different aspirations. So notice what we just did. We were talking about trying to deal with climate change just through tech, more efficient tech, renewable energy, realized. And that's infrastructure. And we said, no, you have to go up to social structure. You have to actually change the economic incentives and law. And then you went up to superstructure and said, actually, you have to change the culture of what is it people value that is the basis of what could get codified into law. Yes, it takes superstructure, social structure, and infrastructure altogether. And the idea that there is a technological solution that does not also require legal bindings,

changes in economics, and changes in culture and values, like, no, the technology plays a role, but it only plays a meaningful role as a part of that integrated suite of things. Totally agree. And this is the conclusion that I've come to yesterday. I did a podcast with a woman named Janine Benyis, who runs the biomimicry Institute. And they're partnering with Microsoft and they've got all kinds of corporations who want to learn from nature. But then I brought up the Jevin's Paradox rebound effect and she said that's a real thing. Even if we're imitating nature, but we're increasing scale. So to me, there are two challenges.

I am not anti-technology. I think we can use artificial intelligence or other technology or human innovation to do things better. But then there's a totally separate other conversation that has to happen in parallel, which discusses the scale of the human enterprise and tries to have some glide path

towards a livable biosphere, because we're, I mean, talk about energy and finance and geopolitics and war, we have limits there. But the natural scientists I've talked to, and I know you have many in your network, we're out of time to change. Otherwise, we're not going to have a livable planet in the next century. Time is a really important part of this conversation. Because if we look at all of the things that were outlined in the paper I mentioned at the beginning of how AI could help stuff, it's almost all very marginal efficiency gains that will not change the curves very much and will lead to Jevin's paradoxes.

And the real big possible changes have to do with stuff that we don't know if it will ever do. And if it does, it won't do soon, right, like fusion. So if we take a look at fusion, the big fusion companies have given dates by which they would have a viable plant and the dates keep getting pushed back. To what? So right now, 2030, but who knows what the next pushback will be. And so if you got the first viable one by then, which there's no reason to have a high confidence in, how long it would take to scale them to replace the existing energy is a very long time.

And yet the timelines we're looking at crossing irreversible tipping points on planetary boundaries is less time than that. So the timeline on planetary boundaries creates a constraint where it's like, okay, well, if we get that thing, awesome, but that does not solve any of the timely stuff we need to do. And if on the race to get that thing, we're actually using up the environment faster, then that's, that is misguided. And I would say there's a very similar argument for, we might not make it here. We're messing this biosphere up. And by the way, eventually the sun's going to take this place out. we have to become an interplanetary species. So let's really focus on the space deck to become an interplanetary species.

We're not close. We're not that close. Like, realistically, humans and microgravity, not a thing, right? Like, that's not going to go well, which is why the actual pursuit of people being able to live in microgravity situations, the exobiology of it, is human consciousness uploaded to AI systems that can do fine with it, assuming we get there, or genetic, genetically engineered humans that can do microgravity, like this is all, who knows. But O'Neill cylinders where you can actually get one gravity, we're not close. We're not close to putting a lot of the people there. So the idea that maybe our stand isn't here, we can make our stand in space because of the

issues here. Nonsense. And if we mess this place up more to say, let's take as many of the resources as we can to get out of space. No, no, we have to actually get it right here first. And if we get it right here, then maybe we can export that. on the planet. Totally agree.

But the order of operations really matters there because the timelines matter. I mean, the simple summary of what I've learned so far, even though I already believe this, is tech alone is not going to solve our biosphere climate. Tech alone makes it worse. Yeah, tech alone makes it worse. And I think a lot of people. By a lot. And I think a lot of people in the environmental movement don't recognize that per se,

at least the ones that these conferences I've been to. Because I know we're trying to do this quickly. I'm speaking quickly. And I want to say, I'm not an expert in these topics in terms of like all of the exact energy curve math on these technologies and supply chain and whatever. There are people who are. And we've talked to some of them. And I think that we could get into it. we're speaking at kind of a high general, like topological principle level.

But I think the principles hold. And if anyone has any good arguments as to why they don't, we'd love to hear them, of course. It would be awesome to be wrong about this. Awesome. But I want to just build out the model a little bit more. At the center, we can say when we're looking at AI and its effect on climate and then beyond climate to the environment. So how much energy does the data centers use? and then how much material does it take to make them and the supply chains that it takes to make them?

Right. So kind of integrated life cycle assessment on the material and energy needs of the AI. That's kind of the first phase. Second phase is AI applied to other extractive industries. So AI helping mining companies be able to exploit more efficiently. Because if you can do better seismic, and ground penetrating radar and all of the things to see exactly where the deposits are and not have to do as much expensive exploration and whatever, then there's a bunch of places

that were not quite profitable to mine with the old technologies where that increase in efficiency that AI gives means now it's profitable to mine it. And so now lots of areas that were ecosystems now become mines. And the same with oil. Lots of areas that were not profitable to exploit, now become profitable to exploit. And so, in the same way that AI applied to military, both has had to grow the military budget and increases the lethality of the entire space. So, of course, AI applied to supply chains where the economic incentive is still that you're allowed to externalize the cost.

If anyone else externalizes some of their costs and you don't, they beat you economically because there's no law forcing you to actually pay the real cost of the thing. We had, we have a really brilliant Earth scientists on the team, and he is the one that put together a lot of the research that we'll share on this. He did an analysis of the current state of Phafos that is fascinating that maybe we'll share. But there was this recent case where Dow got sued, I think it was, Dow got sued $10.3 billion or whatever for its role in PIFO pollution. But then looking at what it would actually cost to remediate Phafoss in the world currently, using known technologies, given that it's in every rain drop in the world and every organism, whatever. The estimate by some group of PIFOS scientists...

Greater than the GDP of the world? Yes. And it was basically $16 trillion a year to internalize the cost of the PFS used per year. Just each new year, if we wanted to actually take out of the environment, the PFS that's put there, it'd be $16 trillion. And that's not remediating all the back stuff, given that they're forever chemicals. So what it would take to remediate the back stuff was, you know, in the $150 trillion-ish vicinity, which is more than the GDP of the world. And this is just one family of chemicals. This is not all chemicals, right?

So if you have what that says is it's not like we pay most of the cost and we externalize a little bit. No, no. The default nature of the economy is that the only cost we pay for is the cost we absolutely have to. Like what we actually have to pay the people and the tech to extract the thing is the only cost we pay. 100% of the rest of the costs are externalized. And so sometimes it's like, and Pthos is not actually a very big industry. It serves big industries, but it was like $30 or $40 billion a year. And yet it would take $16 trillion a year to clean up the mess of it.

And you're like, oh, this is not like it externalizes a little bit of its cost. This is like it's orders of magnitude upside down if you actually had to pay the costs. Right. So in that market dynamic, AI applied to the materials economy means that with the incentive to continue to externalize all the things that you can means just accelerating everything we have. And everything that we have is more pollution per year and more species extinction per year and more unrenuble resource per year already passing planetary tipping points. And this is just more market efficiency on all of the market dynamics that did that just accelerates it. So we said the beginning is the energy and material use of the AI. Second is AI applied to extractive and polluting industries and their efficiencies. Third, out from there, is AI simply increasing the market as a whole, even where it's doing software and financial services and whatever that doesn't look like in extractive industry.

but the more dollars via the kind of Garrett relation type thing is more dollars means more energy demand to be able to make that whole thing work and more material demand. And so when you look at the predictions that Invidia that just became the most profitable company in the world that is, you know, said to be $10 trillion by 2030 and that one prediction I think it was in Bloomberg said, AI in the next a few years will generate $13 trillion new market growth. And Elon just said that Optimus will bring Tesla to a $25 trillion market cap, which is AI empowered robotics. That as we're talking about growth in the overall financial system that is still coupled to a materials economy, obviously the entirety of that, no matter where, even if the domain is not materially intensive itself, the economic effects of the increased currency are materially intensive. So if you want to say energy material use of the AI, energy material use

of AI applied to extractive stuff, but then energy material use of just being able to keep up with the finances, whether it's extractive or not, that's a good way to start to think about it. And that as long as the AI is in service to companies, they are companies themselves, and they are in service to other companies that are based on a externalized cost and invest everything where you can get a return on investment model, they will accelerate the conversion of the earth into capital. I agree. However, let me ask you this. A few months ago, I did, frankly, and it was labeled peak oil, artificial intelligence, and the straw. and I likened shale technology to a larger straw sucking out a milkshake because it doesn't really add a whole lot more milkshake or oil.

It does add some because it discovers some new, but what it really does is it's a bigger straw and it gets the liquid out faster. And I think AI is going to do the same thing to our ecosystems, our species, our biosphere. it's going to suck out more productivity in the three spheres that you just said. At the same time, that larger straw is going to diminish Earth's ecosystems and some of the things we've talked about. However, it has a recursive demand on energy. So in a weird sort of way, AI may make people in our society aware of peak oil in a way that peak oilers never were able. to do. Because very soon there's going to be brownouts and you were talking about Virginia and other places that have such a demand for energy, not just oil, but natural gas, coal-fired

electricity and other things, that we are going to go back to our dependence on energy and ecology and AI is going to accelerate that. Yes, it's going to get us more energy because of some of the better access to technology and efficiencies and sensors, the things that you mention. But depletion and rust never sleep. And we have a six and a half percent underlying depletion rate in global oil that all of the wells in the world that were already drilled are declining that six to seven percent. So the new stuff we're adding on top of that to maintain our access to 100 million barrels a day of liquid fuel. But if AI starts to, um, widen the straw, we're going to hit energy limits all that much sooner. What are your thoughts on that?

Obviously, the energy resources that we know about that are not profitable to extract with increases and efficiency that become profitable to extract, we go through the known resources faster. It is also obviously also true that the unknown resources become known. faster through the sensor tech and stuff like that. So Norway finding all the Arctic oil that it found was a big deal, right? Like that moved it Norway's GDP a lot, but also the energy dynamics of Europe. So how much Arctic oil is unexploited? I don't know what the best estimate is, but it's a lot.

One of the other things that I was just looking at was the fossil fuels amount in Antarctica that's estimated. And again, sensors are going to change this. And it's obviously not just fossil fuels. It's lots of mining assets. The conservative estimate is $50 trillion in minable assets that the one, climate change makes way easier to get to. like awesome melting ice caps. And two, AI and robotics and whatever makes a lot easier to get to. And the treaty that has kept us from going there was only because there was a lot of easier

stuff to get to anyways. But in the presence of dwindling resources, our answer so far has always been figure out how to get more of them and with increasing technological power. So, like, moving faster to there's no resources isn't awesome, right? Because we know it's not going to be like on the current trajectory, it's not proactive. Oh, look, there's not going to be enough resources. Let's all start using less. And specifically, let's start putting less energy and resources into military and make

international peace agreements and stuff. Like, no. The military gets to keep the energy and resources even if poor people can't eat. And super yachts go up even in recessions. and, you know, on and on. So I, the idea, like, if we actually, when, if we don't slow the curve on the way to losing resources, then you get terrible effects. If we simultaneously figure out how to find and get a lot more stuff and extend peak oil a lot further, okay, then you just go further into benesification. So they both suck. It kind of doesn't matter. They both suck. So,

So people seeing that they both suck, that that's the path that we're on, that the efficiency gains aren't going to matter. Now, wait, there's one more part I really want to say on this. If the hydroelectric energy plants had actually lowered fossil fuel use, one could argue that they were a worthwhile investment, even though they destroyed migratory pathways and they flooded ecosystems and killed species and messed up geopolitical power between countries and all those things. And if the materials that had to be extracted to make the solar cells had actually reduced fossil fuels, you could say the material extraction environmental burden of it was worthwhile. But if fossil fuels go up every year, if the goal is right from a climate perspective, if the goal is do those things to reduce climate and fossil fuels are still going up and you're also getting the damaging effects. of dams and the damaging effects of the new material supply chains on the other systems, and even all the nuclear energy, right? Like the nuclear energy that increases nuclear war risk and, like, nuclear waste and all those

types of things, and that didn't slow it down. You're like, wait, would we have just been better off with none of those and just fossil fuels and not the nuclear risk and the hydrodham issues and the, et cetera? So the new stuff that doesn't bind the old stuff is not interesting. New stuff that replaces bad stuff with a binding that ensures it replaces it can be interesting. But that requires social structure and superstructure. It requires people understanding that, caring about it and doing what it takes to do the binding. Otherwise, you do like as people's purchasing of organic food has gone up, total pesticide use every year still goes up.

So has it really helped? No, in that way. As people care more about veganism, total amount of animals and factory farms still goes up every year. So it's almost like those other things just become niche market diversifications. Cool, we'll add that market too. Like anything that sells is great. And so if it is truly going to be a benefit, it has to be that the new thing is replacing the old thing, which means it has to bind the old thing when the unit economics of the old thing still work. If AI or something separate from AI were able to do something on the social structure and the superstructure to change our legal systems, our aspirations, our cultural goals, then you might suggest that AI could help our environmental problems with efficiency and new technologies, et cetera. No? The big powerful AIs are not being built by non-profit think tanks thinking about new economics. Right. They are being built by for-profit companies with fiduciary responsibilities to maximize their profits that are not interested in decreasing profitability of the entire economy or decreasing growth. So the AIs are in service because they're being built by corporations to market dynamics. And the thing we have to bind is market dynamics.

There's no incentive for that thing to happen. So humans, from a superstructure point of view, have to say, oh, we have to bind market dynamics. We have to enforce the internalization of the externalized costs into the cost equation. and we have to be able to stop exponential growth of finance. Then, of course, if humans were doing that, could we use computational capabilities to help? Yes. But computational capabilities in companies, in service to companies, will always do the thing that it does. So what is the beating heart of the superorganism, or you like to use the word Mollock?

Is it AI now, or is it the market dynamics? and AI is just the going through the veins and arteries. It's maximizing returns on power. And we didn't have a lot of power in our ancestral environment, so there was no real maximization because we all just ate 2,500, 3,000 calories a day and then did whatever we had to do for that lifestyle. In the time of Alexander the Greater Genghis Khan, their expansion of empire was not mediated by markets,

as we understand them today. But it was mediated by maximizing returns on power, right? Let's use our military in a way that grows our military. Let's use our resources in a way that grows our empire and our access to more resources. And so the market ended up being an extraordinarily efficient way to do maximizing returns on power because you have this fungible token that applies to absolutely everything that confers power with high liquidity and et cetera. So far, AI, I mean, so far the market is the kind of most powerful thing, but the same dynamic was happening before the market.

And AI is the first technology that can actually replace finance. Because like what is the market trying to do? What is finance trying to do? So currency is traditionally three things, right? It's a store of value. It's a unit of account. and it's a intermediary of exchange. Can AI, can software do all those things?

Yes, of course software can do all those things. The idea that a centrally planned economy is bad, and that's why the invisible hand of the decentralized market is better than central planning. Well, that was when you didn't have AI. If you have a sensor system, like, it's a hard thing to index every single good in service and every resource in the entire system to know how much currency to make, right?

That's kind of like central banks have a hard job. But with sensors, it can literally index everything and can optimize the flows of everything and can factor whatever it is that it's seeking to optimize for. Do you need a market? No. So right now, the market is the dominant system and the AIs are being built by companies. So they're fundamentally in service to the market dynamics. and serving other companies.

But could AI actually replace market? Yes. Now, ultimately, the thing before market in the age of empires and whatever, and the market and the thing that could be after it, are all currently based on maximizing returns on power. And by power, I don't mean Jules. I mean game theory, of which Jules is a subset. Right?

one empire kills the people on the other side and takes their stuff. Somebody grows their population faster and they have more overall effect, you know, that kind of thing. That's, if I want to say, what is the heart of Moloch? It is basically maximizing, like people using agency to increase their agency, maximizing returns on power through all of the known technologies and methodologies. the forced arms races that drives. The thing when we talk about the Jevins paradox, the thing that we have to get out of

is the arms race being the underlying driver of humanity. We can't stop the thing. They're going to do it, therefore we have to race to get there. But they're saying the only reason we're doing it is because them. Now, one thing I'll say is the people who are at the front of an arms race, like who really can have a chance of winning, could bind it and not do the arms race. They have enough power to say, let's actually create an international agreement that stops us think. But they don't want to because they believe they can win. The people who don't have any chance of winning and are just going to be vassalized or whatever, that's a different

story. They absolutely have to race to try to create any security or whatever they can. But the one who could win could also bind it, but they are the ones that tell the story most vehemently. We have no choice. It's inevitable. We can't stop what the, you know, so-and-so is going to do. But that's kind of plausible deniability for a winner-take-all motivation. I want to ask you, what can we do about this? But let me preface that with another question that I'm personally interested in your answer. So fortunately, you and I know lots of people that are waking up to our ecological plight. How worried are you that AI or its owners will use AI to marginalize the environmental demographics of humans, including coordinated personal propaganda campaigns with curation algorithms to steer political

campaigns and do targeted influence on, you know, important people that are that are voicing opposition to what's unfolding? Is that something to worry about? let's say that for any topic, let's say that there is, we could make a law that banned Phaas or banned a particular kind of terrible pollution or something like that, right? We say we really don't care about raising public awareness. We only care about public awareness insofar as it gets to change the real thing. What we really want to do is something like that. Like it's going to come down to a law.

Well, not really because let's say that we work really hard and we change the law. But the forces that put it the other way in the first place, we didn't remove. Well, we just hurt their market dynamics by a lot. They have a lot of lobbyists. Now they're going to get more lobbyists and they're going to change the law back. And because it's not fixed, right? Like most laws don't last that long. You have a shelf life of them.

And so it's not about did we change a law. It's about are there more forces moving? the laws in the right direction or in the wrong direction, right? There's like a correlation of force and means, the kind of battle planning concept on the forces that are working to make it easier to externalize the costs versus the forces that are working on internalizing all the costs. Because a particular law, a particular, same with changing public opinion. Okay, so you make a documentary that's powerful, but then people will forget after three scrolls of Facebook and a couple things in the news and then what comes out next. So the question wasn't, did we do a thing that

moved awareness for a moment? It's, are there more forces moving awareness in one direction ongoingly relative to the other one? So of course it would be quite silly to think that if there are forces that like the world is the way it is for reasons, it's not the way it is for no reasons. And so the forces that are moving it that way don't want it to move a very different way. And if there starts to be success, they don't just stand there and do nothing, right? They respond to try to do the thing they want to do. So anyone who's thinking about strategy obviously has to think about the dynamics. And okay, one thing, there's something kind of problematic about the term AI is it makes it sound like a homogenous thing. Like there's one kind of technology, one kind

of application. There's really not, right? It's kind of like asking, like, am I, for or against AI is kind of like asking am I for or against like that's a ridiculous question there's lots of kinds of tech that I like and I think are important and other kinds of tech I think are net bad and other and some applications and so um the idea AI can do some important things therefore positive sentiment on AI therefore scale the fuck out of it all over the place like that's ridiculous right now having appropriate concern on the risks that are irreversible is really important that doesn't mean that we don't advance computational capabilities to help with things that are really appropriate. So it's important to have some nuance in this. Now, if there is possibly a,

if there's uncertainty about a future dynamic and there is a non-trivial possibility that the thing that we do will cause irreversible catastrophic harm. The burden of proof should be on proving safety, not the other way around. Right now, it's the opposite, right? Lead goes into the market, it gets put into the gasoline, it gets aerosolized everywhere, and only after a billion IQ points are gone in the U.S. And harm has happened everywhere irreparably does eventually the law regulate it. But as we're talking about tech that is radically more powerful and radically faster moving and scaling,

we don't actually survive the harms of continuing to do that thing, where you do the thing, you take the DDT out, you take the let out, the whatever, and regulate it after all the harm. The harm that occurs from AI systems with global surveillance and weapons and etc. Like you don't, especially as we're passing planetary boundaries, you don't get to say, oh, look at all the harm, let's reverse it. That'll never happen. So the precautionary principle says, and is really the right principle to apply here. If there is really significant uncertainty and radical consequentiality in that space and irreversibility of the consequences, then the burden of proof goes on proving safety. And that has to happen first to really get the approval to move forward with the thing. That's not the world. That would be a completely different approach to regulation, tech design, etc. But I will say, if we do not get that, I don't think we make it. But you just earlier in this conversation said the majority of the key AI players via motivated reasoning would follow the logic that AI development is an acceptable risk even in the precautionary principle because on the near term horizon we have nuclear war and ecological overshoot so it's worth the risk to continue towards AGI and my AGI is the best. So who would be the guardians of such a precautionary principle if it's not one of the key AI players like you mentioned?

No, the whole thing about after a war, the winner has a chance to be generous in a way that the loser does not have a chance to be generous. And if that happens, you can create a much better post-war world than if not. If the winner is not generous and the other people who are alive are in shit conditions and, you know, whatever, then that's going to maximize the enmity and everything they can do to be able to scale towards a future war. In the same way about only the winner can be generous in a really consequential way, the person or the group that has the potential to become the winner at an arms race is the one who can bind it. Because the other groups also know, and they're like, hey, look, we're probably going to beat you at this thing. But let's not.

Let's actually make a real agreement. We know that we haven't made real agreements in forever. All the agreements we make are kind of bullshit. Well, we know that we're both going to defect on them and spy on each other and do counterintelligence on each other's spies and whatever. But let's say we're going to try to make a real agreement. and actually build a situation where we could have real trust that we're not building the automated AI weapons and neither are you. What would it take to create real trust there? Could we do it? Yes, we could do it. The argument that we can't, and therefore we have to rush

ahead to win and there is no safe solution, is actually gibberish. But from a motivated reasoning point of view, if I think I can win, I like that world better than the one where I have to share power in a multi-polar way and don't have all the power and don't get to upload my consciousness to the cloud and become a digital god and control everything and whatever. So whatever degree of conscious or unconscious, motivated reasoning is at play. Nope, there's no way we could bind this arms race. It's going to happen. And so we better win. Otherwise, we lose. But it's not true. It's not true. And then you say, well, how much money goes into arms races? A lot. How much goes into really, really trying to figure out how to bind arms races? Almost none.

So what could humanity do to help? Because you said if 95% of the people woke up to it, the perspective that it is impossible to win the arms races the world is facing now. You don't get to win a nuclear war. You don't get to win a who inherits the scorched earth after getting total supply chain dominance where you cross the planetary boundaries but then you control stuff. Like who wants that planet? Nobody wants that planet. You don't get to win the, who gets the AGI dominance first? Who wins?

Nobody. Like that becomes a world that is, if not extinctionary, at minimum dystopic. So there is a place at which the win-lose model is always very destructive. The cumulative externalities and the total exponential power of the tech, we are now at a place where win-lose becomes lose-lose, kind of omni-lose. and either you figure out a different kind of win-win that is not winner take-all that is actually power-sharing, or you get Omni-Lus in the pursuit of winner-take-all. I would love for that worldview to really proliferate and for everybody to say, all right, I'm going to put all the pressure on the senators and congressmen and everything that I can

to say we want to decelerate all the war fronts that could leave. to World War III, all of them. And whatever the right long-term answer is, we don't get to figure it out if nuke start flying. So let's decelerate the acceleration pathways on those war fronts and then figure out the long-term geopolitics. Let's simultaneously decelerate all the things that move us past points of irreversibility. So like with AI, what is the right type of AI that is safe? Let's be able to figure that out before we've crossed the boundaries on it. So it's hard to say what can everyone do because someone who's listening to this is a lawyer who works at a lobbying firm who could be like, holy fuck, this is my kids, this is my everything. And I've actually been lobbying on behalf of the wrong things. I'm actually going to completely change this and start lobbying on behalf of other things. And somebody else is a journalist who could start doing investigative journalism into some of the things that are happening. and somebody else is a, so it's more like what can people do? Well, when it comes to AI risk, it's not just the environment, it's lots of things.

If people here are working on the environment, really study the upsides to AI for environmental issues, the downsides to AI, have a clear perspective and work to influence the sphere that you're around on that, realizing that the side that will try to influence the story on the other way has a lot more money. meaning you're going to have to, in order to have clear ideas proliferate, it's going to take a lot of work and a lot of creativity and a lot of energy, right? I would say for the people working on environment, that's critical. Really, really learn more about this issue. So you are not wrongly putting hope in the wrong places. And that where there is something that is actually really messing up the goals you're working on your fear on that and can start to think about what would addressing that look like. If you're concerned about AI risk in general, not specifically from the environmental place, like listen to, read the key people discussing AI risk and many of the key suggestions they're offering. Nobody has comprehensive suggestions because it's so many things because it's different types of AI and different types of national and international regulation

or whatever, but understand the space better to then be able to look at with your skills, your network, your resources, what you can do. Thank you. I have like 20 more questions, but I think this is the heart of it. Your three concentric circles on how we can consider AI's impact on the environment, climate change, in the natural world is much greater than I think is in the public sphere at the moment. So thank you for you and your team. calling attention to this, and we will put some of the links that you mentioned, and your Earth

scientist, you said, had some documents on this. This was supposed to be a, frankly, if you recall, you were going to do a 30-minute riff. I'm going to be traveling, and I was going to have you substitute for my, frankly. We went a little over 30 minutes, but thank you again for all your wisdom and traviles on behalf of the future of life. Do you have any closing words, my friend? Yeah, you know, reached out to me about us doing this topic a while ago and I've really wanted to, but I wanted to understand the data on the topic more and I still would like to understand it a lot more, but we, we, the high level principle assessment isn't in the conversation enough and it felt worth entering to the conversation.

I would say that we'll try to add into the show notes a few resources that people can look at. If there are people that have additional expertise in this space and can advance the conversation and either say, hey, the things you're saying are directionally right and here's a lot more facts and statistics and principles to strengthen the argument, awesome. Or if you're like, some of the things you're saying are right, but we have reasons to think that some of these other things are wrong or et cetera, awesome also. Like no vested interest in what is true here. There's just the best current understanding, but clear understanding about what is true here really matters. So if you have, if you watch this and you have information that you think materially influences this, I would say actually probably one of the main reasons to share this was a conversation initiation that would hopefully advance the conversation beyond the little scratch starting point we have here. The little people in the shire may still have a voice in this journey. And, you know, I really think that there are podcasts and blogs and whatever that are discussing AI risk.

And there are ones that are discussing environmental risk and there are ones that are discussing problematic market dynamics. But you can't, like none of those make any sense without putting them all together in the same way that it's like looking at your liver is not a good measure of your health. your liver can be great and you can have a heart attack or whatever. You kind of have to look at all of it and how they inter-influence each other. And so you looking at the AI issue from the point of coming from the energy background and saying, hey, look, this is and really getting kind of the Jevins paradox and all those things and saying, look, this is not just about data centers, right? Like this is about the total set of effects that it has where you have to understand market.

You have to understand energy to actually understand AI risk. this actually an AI risk conversation that requires understanding materiality. I'm so happy that you are doing that because there's almost nobody doing that. And I actually really hope that conversation really increases. Not to give any financial commentary, but the fact that NVIDIA is over $2 trillion in market cap and a lot of the energy companies are all over $3 trillion now is in the end. Energy companies are like 5% of the S&P. One of those two things is not right because AI is going to need a whole lot of energy coming up. So thank you again.

To be continued. This is an evolving story. What an amazing and perilous time to be alive. I appreciate you as a colleague sharing this journey and working on these things. While I still have you, what other topic? I'm sure we'll do another podcast later in the year. Is there any topic relevant to livable futures that's burning in your mind? Yeah, there's a bunch that are interesting.

We started to talk about a precautionary principle framework for safety. Yellow teaming. This is, it requires yellow teaming. It's actually a slightly different thing. We wrote a paper on advanced tech regulatory policy. And actually in the last talk you and I did about progress, we were going to get to more like brass tech stuff at the end and kind of ran out of time. So those things like what would a regulatory framework that would actually be adequate exponential tech in a world kind of crossing planetary boundaries be? What would it take to make systems where regulatory power was trustworthy?

And what would it take to, like, what are some really tangible things that could happen to make the market less perverse, internalize more externalities, et cetera, and deal with the kind of embedded growth obligations? Like, speaking more to what some of the solution space looks like in these areas, they're having built some of the problems out, would be really interesting to get into. Let's do it. All right, my friend. Thank you so much. Thanks, man. Really good to speaking with you.

If you enjoyed or learned from this episode of The Great Simplification, please follow us on your favorite podcast platform. You can also visit thegreat simplification.com for references and show notes from today's conversation. And to connect with fellow listeners of this podcast, check out our Discord channel. This show is hosted by me, Nate Hagan's, edited by, No Troublemakers Media and produced by Misty Stinnett, Leslie Batlutz, Brady Hyan, and Lizzie Siriani.