Plain English with Derek Thompson - The Future of War Is Here

Episode Date: May 16, 2023

Today’s episode is about how artificial intelligence will change the future of war. First, we have Brian Schimpf, the CEO of Anduril, a military technology company that builds AI programs for the Department of Defense. Next we have the Atlantic author Ross Andersen on how to prevent AI from blowing up the world. If you have questions, observations, or ideas for future episodes, email us at [e-mail]. You can find us on TikTok at www.tiktok.com/@plainenglish_ Host: Derek Thompson Guests: Brian Schimpf and Ross Andersen Producer: Devon Manze Learn more about your ad choices. Visit podcastchoices.com/adchoices

Hey, it's Sean Fennessey, one of the hosts of the Prestige TV podcast. HBO's Barry is back for a fourth and final season. And that means I'll be back recapping the show with co-creator and star Bill Hater to dive deep on the themes, scenes, and major moments in the series. Bill will provide insight into how every episode was made and why it's ending. New Prestige TV Barry recaps will go live every Sunday night when the episode ends. So make sure you're subscribed to the Prestige TV podcast wherever you get your podcasts. What do you think of when you think of when you think. Think about AI and war.

Maybe you think about Skynet and The Terminator. Maybe you think about killer drones going rogue. Or maybe you're a little bit more creative, and you think about misaligned AI acting as a terrorist, stealing money, ransoming military leaders, and generally creating habit. Or maybe for some reason, you think about peace. You think about a curtain of atomic robots,

hovering in the air, threatening to demolish any invading army with such gravitas that we all freeze in place, and we get world peace under the watchful eye of aligned AI. It seems to me that AI has a way of doing this in the common discourse of the day. It has a way of pushing everybody's ideas about the future into two clean and opposite categories. It's dystopia or it's utopia. Either everybody dies, or it's heaven on earth. But the world doesn't really ever work out like that, does it? You and I don't live in dystopia, and we never have.

And we don't live in utopia either, and we never will. We live in the messy in-between, where some things get better, and some things are shit. Today's episode is about trying to think about how AI will change the future of military campaigns. And this is admittedly a subject where I am so, so, so far from being an expert. So this is a show where two guests do a lot of my thinking for me. First up, we have Brian Schimpth, the CEO of Endurl. Named after the Sword in Lord of the Rings, Endurl is a young military technology company that builds AI systems for the Defense Department.

Most famously, drones and anti-drone technology. a lot of which we have sold to Ukraine. And Dural often notes that there is more AI in a Tesla than a typical American military vehicle. There's better computer vision in a Snapchat app than in a DOD system. And that's a problem because America's geopolitical nemeses are building AI into their military systems.

It's also ironic because historically, the U.S. military was a fount of software technology. This is how we got the internet. But presently we're behind the eight ball, Schimp says, and in particular we are very likely behind China. Among most people who share my politics, drones have a terrible reputation. They are killing machines.

They are tools of war, and war often goes haywire. So in this conversation with Schimp, we talk about not just how his technology works, which is very important, but also how we can ensure that its use is ethical and responsive to democratic processes. And next up, we have my friend, the Atlantic author, Ross Anderson,

who has done some very deep thinking about the next major question in AI and war, which is, how could all of this go wrong? And not just a little wrong. I mean disastrously wrong, at the scale of threatening human civilization and blowing up the world, wrong. So that's today's episode. light fare, how AI is already changing war, and the rules we need to set in place to make sure it does not blow up the world. I'm Derek Thompson, and this is plain English.

Brian Schimp, welcome to the podcast. Thank you very much. Pleasure to be here. First question, what is Anderrell? Why did you start this company? So, Anderle's goal is to create a defense technology company. So what does that mean and how is that different than what has been happening? in defense for the last, you know, 100 years. When you look at the state of defense today, what the defense base is really good at is building really big ships, really big planes, building tanks. Kind of the technologies that we have looked at as the key for how the military

operates for, you know, the duration of the Cold War and in a lot of ways, essentially unchanged since World War II. The, you fast forward 20, 30 years, what's going to start to be different, right? You start to see how software, how AI, starts to impact how warfare is conducted. That starts to look like more systems that are cheaper, that are more intelligent, that take fewer humans to operate. It enables the humans to do what they're actually very good at and make sort of informed decisions with the context and understanding of politics, of the consequences of their actions in a way that machines are always going to be kind of limited. But we want to get humans sort of out of this game of doing very mechanical parts, operating these big systems and working with smarter, lower cost systems. So that sort of focus on software and technology is really key.

The other part of it is our view when we started the company was the pace of getting new technologies into the hands of the actual soldiers, the sailors, like all those folks that are actually doing the, you know, sort of the operations and conducting military engagements, like, we want to get them real technology, but the system has evolved, again, on this mindset of, I'm building an aircraft carrier, I'm building a submarine,

I'm building a giant plane, and that is a very cumbersome, a very slow, and a very expensive process to get those things to work. When you start talking about, how do I take advantage of, you know, smarter drones,

smarter surveillance systems, more autonomous on man capabilities, you can move wildly faster. So a second part of what we've done as a business and, you know, beyond just the technology, is really focus on how can we apply a different business model that is kind of fit for the types of technologies that are going to be relevant for the next 30 years

and look at how do we invest our own dollars to be able to get things out there quicker? How do we find a way to kind of sort, you know, basically work through this process and get technology fielded as fast as possible? We're going to go a little bit deeper into just about all of that. But first, I would love to hear you answer this question at a personal level. Why did you, Brian, decide that this is what you wanted to do with your life? Of all the different kinds of startups that you could have been a part of, why were you so interested in military technology?

So I think the characteristic you note with a lot of the folks working in national security is they are passionate about national security. They really believe in the mission. They believe in why they are doing it. and they believe the importance of U.S. having the best technology. And for me, that is very much a huge part of the motivation. This is the sort of thing that, you know, I believe the world is better off when the U.S. is able to kind of keep a sane world order

where conflict is not the default way you resolve issues, where using military force is not going to be that effective. And so that's sort of the worldview I truly believe it. I think, you know, for better or for worse, conflict is a part of human nature and making it so that these things are unwinnable, that, you know, countries have the defenses they need to maintain their sovereignty. These are things that I think are incredibly important. And I think the U.S. of all the world powers is the one founded on human rights as an important construct. And freedom is an important construct. It's harder to say that for a lot of the other major potential world powers. So I think there is sort of a, for me, a moral imperative on working in these areas. And from a technologist's perspective, I think the ability to actually do these things in a way that's more intelligent, that's more proportionate, that's more limited, is actually a very good thing. You know, like, worse will happen for better or for worse.

And having the best technology that is the most limited and targeted possible, that feels like a good thing. and something I feel like I could move the needle on. I think it'd be useful for a lot of listeners if you to help us level set on what the American military is good at right now and where our deficit is so that we can understand why we would need new technology

to bring our capabilities into the 21st century. So again, what is the U.S. strength as a military? The U.S. has built up a way of conducting war, which is, I would describe this sort of this power projection idea. We have the ability to fight this sort of away game, where we can send our carriers. We have bases worldwide. We have like a very large scale, very experienced military that is able to conduct, you know, operations anywhere in the globe.

And we've done that by largely investing in these very large, very expensive capabilities. So things like aircraft carriers, stealth bombers, advanced fighters. And, you know, kind of since World War II, those were very effective strategies, if you believed you were going to be in this world, a sort of state-on-state conflict, and that was what you needed to provide.

I think the reality is there's a bit of a shift. So, one, we had the war on terror. That was what preoccupied us the most for, you know, the last 20 years, where that was very much about how do we kind of, do the best intelligence possible. We need to know very precisely where people were, how they were networked, how were they connected,

and be able to conduct very precise counterinsurgency operations, very limited, kind of like a very ground-heavy type of conflict. But all about that sort of precision and getting every move correct. Ukraine has kind of shifted this where the two major things, kind of happened in my opinion. So one, there was a resurgence of state-on-state conflict.

I think there was kind of people didn't really believe this was going to be a way a state would try to exert its will to try to get to its political ends. But as we saw, it still is, right? Like Putin thought this would be an effective way to accomplish, you know, whatever was and Putin's had of what he was trying to accomplish. So, you know, I think there is, that is back. and that is a well-funded state entity with real advanced capabilities that is, you know, looking to use military force to accomplish their end. The second is the U.S. did not directly engage, right? And so that is another thing that I think is a little subtle, but kind of we take for granted

a little bit where we've reset the norm where it was kind of a little unclear when it set out. how much support would we provide, would we get engaged, would NATO get engaged, what would trip these different conditions, like how much would this be the U.S. and Western Europe directly fighting? But I think the norm in a lot of ways was set where the U.S. and the Western allies would provide intelligence. They'd provide support financially. They'd provide support with weapon systems and everything that our allies need to defend themselves and conduct their own, you know, kind of try to withstand this aggressor, right? And so that was a big shift. But then you look at what's happened here, and it's actually quite interesting. It matches a lot of the

thesis we had of how these things would play out, where day one of the conflict, you had all of the air bases, all of the F-16s, all those things were knocked out, right? Like, they were not relevant to the fight. You had a massive dispersal of forces. So instead of having these relatively expensive, easy to target air bases and military compounds. You have the Ukrainian forces and the Russian forces, both learned very quickly that the optimal strategy was to break it up, to have essentially uneconomic targets. And you have a lot of this more smaller unit fighting, but then how you provide them intelligence, what do they need for drones to be able to surveil the adversary? The sheer volume of ammunition and weapons they are going through is huge. And so I think you saw a pretty

big shift where this isn't a large-scale naval battle. This isn't, you know, tanks are having limited effect. There isn't a real air campaign. There's no like major fighter jet pieces here. This is really looking like a lot of very tactical, shorter range engagements and a lot of them simultaneously. And that is a very different way of conducting warfare than we've been used to. But I think in a lot of ways, that's going to be the hallmark of what this next generation of military engagements looks like, which is highly defensive in nature. how do I withstand a stronger adversary trying to exert its will on me and prevent them from being successful, right?

I'm not going to decisively win this as Ukraine. I'm not going to knock out Russia. That is not the goal. But the goal is to make it so impossible to succeed that people take military options off the table as an effective means of accomplishing their political lines. I want to bring in the technology revolution that we're seeing as well because you make AI products,

you make unmanned autonomous products, the fact that you exist, the fact that a startup has to take the technological frontier here suggests that the U.S. military missed something, right? I mean, you wouldn't have to exist if the U.S. military was all over these technologies and clearly owned the tech frontier. So what happened? Why do you think this institution, U.S. military, which succeeded in big, manned technology did not do as well to pivot towards smaller

unmanned technology. Yeah, I think there's kind of like a process and incentives thing with any organization, right? And so in any organization, you've got sort of the process pieces of they built up a way

of ensuring safety, reliability based on what they were doing. So you look at the life cycle of a, you know, a submarine, like an attack submarine. You know, those are going to go into service. Next generation, let's say, goes in the service in 2030 with a lifespan through 2080. These are very long-duration platforms, and then the rigor you want to do on the engineering, the amount of effort you put into that, and the safety considerations around this are massive. But that all costs money. That all takes time. That is a very particular way of working when you need that to go.

And so the processes that have served them well, I'm building these big man things, are kind of the opposite of what you need in a lot of these lower cost systems, where when you look at what has happened in Silicon Valley, Silicon Valley has figured out, and the tech world writ large has figured out, how do I move faster? How do I just learn faster, iterate faster?

Velocity has become the predominant thing. And then figuring out all the process and technology pieces around that, to ensure that I still have quality, the system still work, I can manage thousands of developers simultaneously contributing on this without them stepping on each other's toes. Those are hard problems, and Silicon Valley has really figured that out. But they had a very different set of consequences and structures and incentives for how to do this. And so I think there was a genuine and real learning and innovation that happened

in how you conduct business, how you build technologies in the Valley, that is not obvious to a lot of folks. this idea that velocity sort of trumps nearly anything else in terms of what you're trying to accomplish. You just learn faster is a pretty subtle but very important idea. So the military, because of all their incentives, they built a system that was kind of dialed in for it, right? Like they wanted safe, predictable, large-scale things, and they got it. But then that is what works against you when you need to move fast, when you need to iterate, and when you need to actually do things where you can absorb more risk. You can have these systems just get out there faster and learn faster.

And the consequence from a cost perspective of, you know, like, hey, I, you know, this drone wasn't as effective as I wanted, but I can make another one very, very quick. Those, if you're not tuned for that world, it is very, very hard. So I really do think it was, it was not sort of inherently a question of like, were people sort of incompetence or, you know, did they not have the talent? You know, I think this is just purely a matter of the system that worked for so long is working against them now. And in a lot of ways, that's what I think startups can do well. They can say, I don't have this legacy business where I, you know, 95% of my money is coming from the platform I made 30 years ago. I have the opportunity to look at this differently, and my success is contingent on getting everyone to understand a different way of operating. Let's talk about your portfolio.

What are your contracts with the federal government, as much as you can talk about them? What are you selling to the federal government and how are they using your technology? So cross-cutting everything we do, we think of this as sort of more intelligent, you know, unmanned and autonomous systems. So for us, that is very much, you know, where are these areas where smarter, you know, networked, more intelligent, really sensor-heavy systems will move the needle in terms of providing capability. We have a software platform crossing all of this that we do and trying to take the best of

how tech companies have learned how to build those advanced software technologies into the military space, right? Like you invest heavily in these core technologies, you apply it kind of broadly. From a hardware perspective, you know, we build a lot of hardware systems and, you know, there's a lot of reasons for that. A lot of them practical around, like,

how can you actually get stuff out to field quickest? Like the military at the end of the day, they buy integrated systems. That's what they buy. They buy a capability. Just could I step you there? Like, when you say the military buys integrated systems, I kind of understand what that means,

but I kind of don't understand what that means. So, like, in plainer language, what are you saying? So, you know, there's sort of this view that it's like, you know, when you're selling like a Fortune 500 company and you're selling, for example, cybersecurity, you can sell them a piece of network gear or something like that and they'll have an IT shop that is able to pull all this together or, you know, you're selling them apart and the company is often doing all that integration to make like the whole system work. The military needs to buy it already working, right? When they buy a drone, they need to, you know, they're not buying like an airframe and then separately buying this and separate. buying sensors and network, and then they're going to bolt it all together. They need to buy a full system, a ground control system, the drone, with the sensors integrated, with the training, with how do I actually get people how to use it? So they need a lot of, kind of holistically, this has to be something they can train troops on, deploy, service, operate. So it's quite a holistic thing that you end up that they need to buy the end of the day. They don't have

the in-house capability to, you know, kind of pull all these things together. I would love to understand just at a really basic level, how drone and anti-drone technology actually works. So talk to me like I'm a moderately smart high schooler here. How does this stuff actually work? So when you think of drones, you've got kind of quadcopters, you know, so four-pro, you know, four blades on, you know, every arm. Those are like the commercial ones you see, like DJI makes these,

you can buy them at Best Buy. And, you know, you've got these bigger ones we think. of as like US military, right? Like the predator drones, these big planes that can kind of take off. A lot of what we're focused on is, you know, somewhere in between. How do we shrink these things down? Make them so that they can actually be brought by small numbers of troops to the front lines launched and go find things. And then you have the question of what do you want to even do with these things, right? Like, what is the point of having this? And by and large, the goal is surveillance and reconnaissance. I want to know where the troops are on the other side. I want to know

what they're doing. Are they getting ready to launch a missile at me? Are they moving their tanks, right? And like you kind of have these questions of kind of surveillance at the end of day. I just want to be able to see things, find things. Like the goal of the military is often just to find and identify where things are and what is their intent, right? That is the goal. So where autonomy starts to come in on this is how can I make it so that I have much less manual effort involved in doing this. So often a big constraint, when you look at this, is that two sticks.

I'm flying it around and I'm looking at a video screen. When we think about adding in more autonomy to this, I can now start to push more of that intent of what am I trying to do to these drones. I can say, go find tanks. And they're probably on those roads. You can go out, a lot of course, go look for them, use computer vision to say, like, yep, I think I found tanks here, here and here. report that back to the operator and decide what they want to do with that information.

So the drone space is, you know, kind of that we focus on is how do we make this smaller, how do we make this easier to employ, and how do we reduce the manpower? So it's not just people holding joysticks looking at a camera. They can now start to operate many of these while doing their other job, which is being a soldier, actually making sure they're secure, you know, conducting whatever operation they're on. So that's sort of how we think about the drone side. When you get to the counter drone side, you're trying to say, well, I don't want to get surveilled. And then the extension of this is some of these drones are kind of suicide munitions.

They are explosive. They are actually cruise missiles. These are things that an adversary can send in and say, hey, go hit this target. And now what do I have to do? I have to be vigilant and know where do these showing up, right? So I have to like detect and track and identify these to start. That is hard. These things are small.

They often look like birds on radar and on cameras at range. They are flying very low. So even being able to get a line of sight to see them is very, very hard. And they can strike anywhere, right? So I could not just looking at a big military base, but it could be a power plant we're seeing in Ukraine. And so the challenge you have is, you know, step one, identify them. Know where they are. No, it's not a friendly aircraft. No, it's not a bird. No, it's something I need to engage with. And then you have to decide what to do about it. And there's a handful of strategies you

can actually take. I can try to jam communications. I can jab their ability to navigate. And that can work to some degree. But there's a lot of countermeasures you can do against that. And ultimately, what does work is I can shoot it down. That can be a missile. That can be a gun. But I got to start with this process of detecting them, identifying them, deciding to act, and then taking that effect against it. And that whole engagement might happen in a matter of two minutes, max a minute. So your ability to very quickly understand what's happening, make a decision, and respond is the fundamental thing you've got to solve for a lot of these problems. And now doing it at scale with hundreds of humans that are very tired

and get very fatigued doing the same activity day in and day out and never making a mistake. So how can we use technology to automate more of that process? And again, the same analog applies. The historic means is guys with joysticks looking at screens. I can take a lot of those manual steps out, just present them with decisions when they need to know. That's a huge step forward in our efficacy

of how we can actually start to combat these problems at the scale we're going to need to. I'm really interested in how this technology actually works in a battlefield or along a border. So maybe the best way to get at that question is to ask a hypothetical like this. If your company started a decade ago or 15 years ago and it had reached full maturity before the Russia-Ukraine war had even started, what would you, and the federal government, government that's contracting with And Durrell, what would you have given Ukraine that would have helped them discourage an invasion by Russia?

Yeah. So, you know, I can, I think about this a lot. And the, there's kind of like a couple different facets to this when you think about, you know, like, what are these conflicts really going to look like? So the most obvious one is kind of the most visible is the weapon side, right? How do you provide intelligent weapons that are going to be able to hit the types of military targets and work at the sort of economics of what is feasible here? Let's get the end of the day. These can't be 10 million dollar missiles.

They have to be very cheap. There's going to be a lot of them. And you have to make it very clear that the volume can be sustained. That is a huge part of what we've seen in Ukraine. This is not a one-day conflict. This is a multi-year conflict. So how do you have the sort of ability to produce an operative scale? I think this would look like things like long-range, smart, loitering munitions.

So there's a couple of these in the world, but being able to have, you know, the javelin missile has been very effective against tanks. That works at a couple of kilometers away. That's a pretty dicey position to be in if you're a Ukrainian, like right up against the armor column coming in. You could provide that capability at 100 miles away. And I knew there was a column of tanks coming. and I could deploy 50 of these to go out and deter them.

It would be very deterrent, right? It's like, you will not drive on this route. It will not work. Okay, well, that really is a very clear indication of, like, well, that strategy is not going to be effective. So I think the weapon side is, you know, how do you kind of get very precise, very targeted systems working at long ranges really tips the scale?

I think the second is you need surveillance and an understanding of what's going on. So low-cost drones that troops on the front lines can deploy that work at relevant ranges that can spot these sort of enemy positions and be able to identify where they are, what they are, and be able to do this at scale without needing a huge amount of manpower is very, very key. And the other aspect is defensive in nature. So how do I provide the ability to counter all of those things I just described? Because if we can figure this out, the other guys are going to figure this out too,

which is counter-drone capabilities, counter-cruise missile capabilities. How do I provide this air defense picture that tells me what's going on, the ability to have all the counter-dron missiles I need or jamming capabilities I need to be able to make it so these things can't operate effectively? And then underpinning this is kind of the resilient communications, right?

We kind of take for granted that this is really important. A big part of this has been Zelensky, is able to communicate out to the world, communicate to his country, what is going on, tell the story, retain support, and that the troops can communicate with each other. That wasn't obvious when this started. The cell phone infrastructure was taken out.

Starlink has been a huge kind of lifesaver in this, but having a more proactive strategy for this is really, really key. We're kind of working in all of those spaces. Some stuff we have deployed today, some things we're working on and enhancing.

All of those are subtly different than what I think the U.S. has historically wanted to procure the capabilities they've kind of historically specked out. And a lot of the technology you're seeing there today is things that were built up in the 80s and 90s, right? Like these are not kind of cutting edge capabilities. These are things that we've kind of had solved since, you know, around the end of the Cold War. So, you know, I think there's a lot of, you know, how do we make sort of smaller, higher quantity systems that really work in this new paradigm of lots of distributed, you know, forces fighting. And then on the defensive side, I'm protecting whole cities. I'm not protecting,

you know, just a small group of troops. I'm protecting Kiev, protecting the critical infrastructure. How do I make that a reality, given all the tactics we've been seeing that the Russians have been employing? So just to reiterate, for my own benefit, you're talking about giving this sort of parallel universe Ukraine. This Ukraine that exists in February 2022 in the world in which Endurl was founded a decade earlier. You're getting them next-gen anti-tank missiles. You're getting them armed drones that provide a kind of curtain of surveillance that discourages a border crossing into the Dunbos in the first place. You're giving them anti-drone technology. You're giving them resilient communications technology. And it seems to me, just stop me if I'm wrong here, the goal is to discourage a Russian

invasion in the first place to not even get into a multi-year war that then unlocks the need for even more sophisticated and unmanned military technology. Is that all correct? That's 100% right. And so I think the goal is it has to be obvious that a conflict is unwinnable, right? And I think that's probably the right term for this. Like no one is going to decisively win these conflicts in the future. There will not be a, you know, sort of full and unequivocal surrender. that is unlikely to happen. And so if that is obvious, you know, wars are conducted to accomplish political ends.

And if it is sort of so obvious that the deterrent capability is there, that your invasion won't succeed, the other side won't capitulate, then the military option becomes far less attractive. And so I think that's a huge part of it, is kind of this ability to deter an invasion, to make those strategies off the table is very, very key. You know, that's one of the, like, more consequential ways,

You can see these conflicts playing out. I think it's very relevant to Taiwan. I mean, it's very relevant to Eastern Europe. I think writ large, they're all going to look at this with a perspective of how would I make this very clear from a display of force that, and a, you know, kind of a revealing of capability that any kind of means of driving large-scale conflict will not work, right? It's just not going to accomplish your political ends and it's not an winning strategy. That is a lot of how we've thought about this. And we, as you know, are going to get to Taiwan in just a second.

I want to double back and ask a question about how to, how countries should communicate to their potential adversaries exactly what kind of military technology that they have. So, for example, this is a question I think that we got in the audience when we talked in at the Progress Summit in Los Angeles, and I thought it was a really interesting question. You just made what seems to me like a perfectly valid point, a great way to discourage an invasion from, like,

let's just go back to discourage invasion from Russia into Ukraine, is for Ukraine to be able to project the fact that they have such an arsenal of military technology that victory is impossible. But the more you communicate exactly what military technology you have, the better Russia can plan to counter that military technology. So I know we're getting a little bit into the question of strategy. and communications and outside the realm of pure technology. But if you thought a little bit about this question of,

shouldn't a country want to keep some of its military secrets a secret? Yeah, I absolutely think there's this constant sort of debate on like a reveal-conceal perspective. And the conceal part is actually really valuable in the sense of if you know there, if the adversary knows you're concealing certain things, then anything is possible in of what could happen, right? And so there's kind of a tricky strategy question here. You know, I think the other dimension to this is a lot of these technologies do not present an easy counter, right? And so, you know, there's sort of always this question of technologies that either

favor the offense or the defense. And it's a little blurry, right? So it's like, you know, stopping an armored convoy coming in. Is that inherently an offensive action? You're taking against that convoy or defensive nation? You know, it's like it's a little bit tricky often to have a clean line there. But a lot of these just present very challenging problems where, you know, you're sort of just on the one side of this cost and efficacy curve and technology curve where in those contexts, revealing is very effective, right, in terms of your strategy you have to take if you kind of want to do an invasion will not work for all these reasons, which you can't really beat, where it's so expensive to beat that'll take you a decade. And the nature of

defense is, it's always this sort of like cat and mouse game, right? You're always kind of adversaries are learning, adapting, building technologies to counter you. You're learning what they're doing, building technologies to counter them. It is a constant race here for better or for worse. I think what is kind of shifted is the view of the race has moved from these sort of like strategic deterrence, like nuclear weapons and how do we have bigger, more nuclear weapons that are more secret. It's like nobody, you know, that's kind of reached stasis, right? That has reached an equilibrium.

Now I think we're seeing a similar thing on the conventional side. where kind of these sort of conventional conflicts will kind of get to a point of equilibrium, cyber, right, all these realms. I think new realms of warfare will happen. A lot of what I think we'll see is moving to these more kind of below threshold of conflict, armed conflict sort of engagements, kind of covert action, sabotage, weird sort of, you know, paramilitary, semi-military exertion of force, trying to extend your sea claims, like all these things that are very hard to have decisive action against,

and that will, I think, be in a very effective strategy as well. That'll kind of spur a new set of ways of how do we counter that, how do we kind of not escalate conflicts? These are all just, I think the nature will shift, but I think taking these large-scale military conflicts off the table is something we can reach a stable equilibrium on quite quickly here. All right, well, let's head right into the trillion dollar question. here, what are the sort of things that the U.S. could give Taiwan from your portfolio of technology

that could make an invasion of the island less likely? I think there's a lot of those things we described, right? So it's how do I have the defensive capabilities to protect what is going to have to be a whole island, right? It is no longer just like protect these one or two military bases. You can't, right? The volume of missiles that China can't launch at Taiwan is too high. So how do I make this as sort of uneconomic as possible, make this as dispersed as possible, enable them all the communications possible that they need? All those, all those pieces are sort of still there. I think when you look at the Taiwan scenario, it's a little more constraint, right, in good and bad ways. The logistics of an invasion are harder, coming across

a channel, an ocean, or you're coming via air. Those are both very hard to do. There's very real choke points on where those invasions can happen. And so I think there are kind of more constraints to this that make an invasion scenario kind of highly favor the defense of you're largely trying to deter kind of these maritime evidence, right? And so like anti-ship capabilities, things like that become really, really key. I think a lot of the autonomous undersea capabilities, things like that become really relevant as well.

So I think there's a lot of aspects where, you know, the other interesting angle for Taiwan is the Chinese strategy on this has been, you see, in the South China Sea. I think DOD released recently all the islands and the reefs that China has built up military presence on and have basically been incrementally inserting more claims into South China Sea. I'm just trying to just say this is our territory. We're going to have fishing rights, we're going to have mining rights, we're going to mineral rights. We're going to use kind of low-grade force and harassment to exert those claims. That's a hard thing to counter, right? And Taiwan has to have a response. Philippines have to have a response.

Vietnam has to have a response. And so figuring out ways to, you know, kind of provide those defensive capabilities, protect your rights, you know, as you're doing mining or, you know, oil and gas or anything like that, how do I protect those from, you know, sort of adversarial action becomes really, really key. So I think there's a lot of like subtle parts to this where a lot of the autonomous capabilities are really relevant where you're not going to have the number of pilots and boats you need. You're not going to have an economy to support, you know, the huge amount of capital ships you would otherwise require. So this has to look unmanned in a lot of ways. This has to be much more

cost effective to counter it. And those are, you know, kind of writ large, the types of technologies we're working on. And of course, if we arm Taiwan with a bunch of autonomous submarines and other technology that you've mentioned, there's a possibility that it makes Taiwan safer because it has exactly the effect that you've described, which is discouragement. There's also at least a small chance that China seeing Taiwan collect all of these new technologies says to itself, we might have to accelerate our military takeover plans. So how do you think about the fact that you are building tools that in some versions of the future are used to discourage war,

but in other versions of the future are used as a kind of cossus belly by policymakers to accelerate warfare rather than merely strengthened defensive systems? I think this is a really tricky foreign policy line for, you know, And largely, so, you know, one is a company, our view is aligned with the military, is aligned with, you know, kind of U.S. foreign policy positions, which is, you know, number one, the job of the military is to provide the president options, right? Like, that is their job.

They are not the policy people, like, they want to do things ethically, responsibly, and effectively, but how use of force is implied, that is inherently a question for civilian leadership. That's their job. And the same view, I think our view as technologists is that we have to provide these technologies in a way that are effective, they work, and we clearly communicate the limits and what these things can and can't do, and that we are employing the technologies in a way that makes humans more accountable, that gives them more agency, that gives them more control,

and reinforces kind of the policy doctrine we have, right, which is humans are accountable for their actions, that rolls up through elected officials, and that's a pretty good system. And so I think on the whole, our sort of framework on this is, you know, our job is not to conduct foreign policy. Our job is to provide effective technology solutions and do it as ethical as we can. On kind of the question of like, how do you thread this, you know, if I'm a policy person in Washington, like, what's the right way to think about this? It's quite tricky, obviously, right? I think there's a lot of ways to kind of analyze this problem. I think the view has been sort of a non-intangism strategy

and pulling them into this sort of Western rules-based system would kind of mute out a lot of the more authoritarian dimensions, would increase freedom, would kind of pull them in from economic interests into a more stable system. I think history has told us again and again. that's kind of a questionable strategy. I think, again, with Russia here, it was not sort of obviously rational

why you would invade Ukraine, but certain leaders, especially in these sort of more unstable authoritarian setups, have incentive to do so, right, in sort of weird cases. And so I think it's kind of hard to imagine a world where a completely pacifist Taiwan somehow is a more stable system

that is less likely to induce conflict than one that is kind of relying on the stability of it just won't work. Like, it's just not possible to work. Then how do you thread this sort of transition period where one power's rising, one power's declining? It's quite tricky.

Those are the most unstable times. And I don't think there's a perfect answer for this. Yeah, I want to hold on this topic. I know that it's thorny, but I think these questions are really important in keeping with these sort of ethical gut checks. So obviously, autonomous technology is the sort of thing that could be used for good or could be used for something really terrible. And a good example there is, you know, surveillance technology in Xinjiang in the Western province of China is being used not to, you know, discourage war, but to create a police state to subjugate an ethnic minority. How does the fact that this kind of technology can so clearly lead to a kind of, you know, digital or well-ean state, how does that make you guys think harder about the ethical guardrails

around this tech? So a lot of us is you can't get enamored with what can be done. You have to sort of really think about how would this technology be most responsibly employed and understand the policy frameworks that the government operates. operates in, right? And I think that's really key. I think people forget that the U.S. actually has extensive rules on use of force, extensive rules on, you know, kind of authorities here, how it flows down from the president, what legal basis we have to have in these different contexts, how much information we have to have, use of surveillance technologies, and like what sort of checks

and balances we have in the system. And I think you can disagree with those at times. You could, you know, wish them to be different. But at the core of it, the U.S. has, you know, we've kind of taken the position that this should be real policy. This is not an authoritarian system. It's accountable to a civilian leadership, and we will refine and enhance these use of force guidelines and intelligence guidelines over time. I think that's a good system.

I really believe in that system. So then as a technologist, I think our view is, where can these things be done responsibly and accurately represent what is possible? So simple examples of this are like, we've had people ask us to do long, facial recognition. We're like, it's not going to work the way you think it's going to work. And they're asking, honestly, right? Like, they're asking, like, if I could get, you know, 99.999%

confidence of like, you know, this to work. It's like, yeah, that that world might be believable, but that's not the world we live in. Here's the actual efficacy of this. So we can't, we can't be selling sort of a false promise of what these technologies can do. When we think about autonomous systems, it's very much about, you know, nobody is asking for, nor would we propose that these systems go out and autonomously choose which targets they're going to engage. It's too complicated, right? It's like it can make mistakes. The accuracy is not where we would want it to be for that to work. We need a human kind of vetting these things at the end of the day. And then on top of it, you're kind of relying on the system actually understanding all the

political, societal contexts and having all this information at its disposal that we hold humans accountable to. So our view has always been, this is about enhancing human accountability on this, like that's the goal. Reducing the fog of war, making them make better decisions, that feels like something that will lead to better outcomes and gives you more policy levers of how do you inform and make sure the decisions at the right level of confidence, not in these sort of like, you know, and get us out of these dire situations where you're sort of forced into making compromised decisions because of necessity. And I think that's sort of how we've thought about it on the whole. One of the ways that I'm trying to get smarter in the

frames that I use, the thing about technology, is to distinguish between what I've come to think of as, quote, mere mourness. Like, hey, here's a thing we can do so it's good. Here's a new thing we can do. We've pushed technology forward, therefore it's good. Here's a thing we can make more of, therefore it's good. That's what I would call mere mourness. And then there's technological progress.

And that's the ability to use technology to actually improve people's lives, to reduce pain and suffering in the world and raise the ceiling of human flourishing. And so I was interested in asking you, like, what's an example of a technology that you've explored at Enderrol? You came to realize, wait, I'm not sure we can stand by this. I'm not sure that we can support this within the ethical guardrails that you and I have just discussed. Just to go a little bit deeper, would you say that long-range facial recognition is a good example of a place where you explored and thought, you know what, this can't really be done in a way that is efficacious and ethical? Yeah, I think that's a great example where we are just like, the physics and technology does not enable this to work today.

This can't be done in a way that, you know, like the probability you're going to misidentify someone is too high. Like, it's just not reasonable. Like, you can't say there's like a given I found someone I think is, you know, a high-ranking drug cartel member, but there's like, actually it's only a 20% chance is actually him. Like that, like the math doesn't work out. And so it's, from our view, we have to actually be able to kind of look at this in the context of the system and what they're trying to accomplish. And we can do the engineering, understand the system, understand the math of this to say, here's what will work or what will not.

Or here's the probability you're going to have consequences that are unintended. Is that acceptable in this context or not? And so we can kind of accurately represent, if somebody can accurately represent those trades, I think that informs policy makers. to decision makers on where they want to be on this. They can make fully informed decisions. And, you know, there's, that has certainly been kind of our view to this on the whole. So I think that's a great example of one that we've, you know, kind of early on looked at and said, or had a request to look at and said, I don't think this is going to do what you wanted to do.

I want to move to 2023 in domestic politics. Tell me what contracts, and again, as much detail as is reasonably allowed, what kind of contracts have you signed under the Biden administration? How has the U.S. military continue to expand their portfolio of enderial products? So the vast majority of contracts we've signed as a company have been during the Biden administration. The biggest one to date in the U.S. has been with special operations to do their counter-dron work. So we provide kind of the capabilities to detect, track. identify drones and then counter them.

And it's structured in a way that we're able to evolve this to be able to respond to the threats that are actually being seen. We've done a lot of work with the drone capabilities writ large, like how does the military kind of take advantage and integrate these smaller, more autonomous systems? That's been a big part of the portfolio as well. And then a lot of work on what we would call kind of this joint command and control. So how do we think about the Air Force and the Navy and the Army and the Marine Corps, all kind of operating together, understanding what's happening in this battle space

and being able to respond and operate very quickly? We've done a lot of work there as well. All with the intent of the pace and nature of these conflicts has increased tremendously. It was wildly different than what we saw from kind of war on terror. And this needs to be something where we can make highly informed decisions very, very quickly and be able to respond. And so increasing that pace of how we operate as a country, as a military, has been a huge focus as well.

So it's been kind of quite comprehensive. I would say, you know, national security in our experience has been one of the most bipartisan supported efforts, right? People believe in a strong defense. They believe America needs to have these technologies. I think you see kind of like shifts on the margin from administration to administration. But on the whole, the military has, I think, kind of, tried very hard and by and large upheld its belief as a nonpartisan entity

that is serving the national interest and has not been as much of a political hot potato as many other issues. How's your anti-balloon technology? Oh, man, so many ideas. The balloon thing's actually wild, right? Because it's an example of, you know, one, I kind of think there's been kind of this question of, is China actually building military capability? Is this all just saber rattling?

And I generally believe that if the Biden administration says, hey, definitively we have this evidence that this is a military or intelligence balloon, like they're not lying. They're not, you know, they're actually quite honest and high integrity on the whole in terms of like really thinking, you know, releasing information they believe to be factually accurate, right? So I'm inclined to take them at face value that this is, in fact, to Chinese military capability or intelligence capability. And I think it's indicative of the types of capabilities that China's building very quickly.

It's a very cheap way to get effective surveillance capabilities that, on the whole, are actually kind of hard to detect and defeat. So there's a lot of really interesting nuances of why these things are actually quite hard to find and quite hard to shoot down. It really gets very weedsy fast. But it's a pretty effective technology area. So I think it's a really interesting indication of like, time is active. They're really looking.

They're really building a real military intelligence capability that's global. And that is something, you know, at the face of it, we should take very seriously and understand that they are a real, you know, entity that's able to play at this level of the game, essentially. You said that one of the biggest areas of contracting with the Biden administration is an anti-drone technology. Just one level deeper. where are we talking about anti-drone technology being employed? Are we talking about it along the border of the U.S.? Is it in Eastern Europe?

Is it in near China surrounding our allies in the region? Where is this anti-drone technology being employed? I can't really say exactly where it's being deployed. What, in that one, zip code and longitude and latitude? Yeah, exactly. Here's the specific base and geolocation of where it is. I would think of it as military-based security kind of within the U.S. And then, you know, kind of looking at foreign military installations overseas

has been like kind of overall like where we've been deployed to date. And most of these locations have had varying degrees of sort of drone threat, everything from actual, you know, kind of similar things you're seeing in Ukraine to more hard to attribute kind of commercial drones flying over air bases and things like that. So it kind of varies across the spectrum. But I think kind of the thing we've seen on it is, again, it's like technology that was historically, uniquely military technology that was very exquisite, it's extremely proliferated. And this is a big shift where the last 20, 30 years, the U.S. didn't have to think about air defense

there was nobody else who had air superiority in anything we were doing. And when we think of air defense, it's like protecting relatively few assets. And we have this huge benefit in the U.S. of geography. But now if these things are pervasive, they're cheap, everyone can have access to them, it suddenly gets very hard, right? Where now everything has this concern of protection, security, air defense in a way that's quite a bit different. Every unit going out is at threat from drones. That's a big shift from, you know, kind of where we were.

You look back to like the Iraq War, the threat was shooting down these scud missiles, right? You know, for the folks that remember that, big missiles shot at long range, targeting U.S. military installations. That's not the threat anymore. It can come from anywhere at any level. And that is a very hard problem to counter. So a lot of what we focused on with this is how do we make it cheap? How do we get the manning down as low as possible? Make this so it's as close as possible.

to, you get alerted, there's a drone, and here's your options of what to do about it. And it has to be that simple. Contrast it with where we are today, where you have humans times three shifts staring at multiple screens, manually correlating all the sensor data to figure out, what am I looking at? Is this a threat? And they have like, you know, seconds to respond. And then knowing what, you know, kind of techniques will be effective against this is incredibly subtle. And they may only see like five of these in their whole career, right? So it's a very tricky problem on this defense side to to actually crack at the scale we're going to have to operate this at, which is very different how we've historically looked at kind of high-end air defense, right? This isn't defending

North Korean missiles. This is defending against this pervasive low-end threat, which can harass and cause problems for nearly any unit at any time. My last question for you is about talent. I think it's ironic that Silicon Valley was in many ways berthed out of the defense industry and investments made by the defense industry in that part of California. But today, it seems to me like most people working in what I would call the tech industry do not want to work in this space anymore. They don't want Google to contract with the Defense Department. They don't want to be involved with war. We are living, at least relative to the 1940s, 1950s, in a kind of peace time, at least in the Western Hemisphere.

And we don't want to, I think tech people don't want to think of themselves as belonging to the international war industry. How do you think about balancing, on the one hand, you know, the virtue of wanting peace with the need to keep the military at the cutting edge of this kind of technology so that we aren't lapped by China and Russia

and its adversaries, to the next generation. So first off, I think what we've seen is that there's a very vocal, but very small set of the tech population that feels very strongly about that. And I love that in America. That is something we embrace and we want them to have, right? That is not an option in China so much, right? And so I think that's, you know, first and foremost, very important to keep in mind

that people certainly get this choice, and I love that about this country. But I think at the end of the day what we've seen is it's a relatively small minority of it. The second piece of this is that the war in Ukraine really changed opinions for folks, which was, I think there was a belief that kind of the sort of end of history view that conflict was over

and that sort of the economic stability would lead to kind of a stable international order. Many claims were made about that before World War. one or two, right? And it's like, I think it's just history tells us that that is inherently not true. And that I think the world has benefited from this sort of unipolar world where the U.S. on the whole is trying to enforce a rules-based order. You can disagree with specific conflicts, how we engage in it. But at the end of the day, we are not inherently trying to use military

force to massively extend our territory, our, you know, our rights, any of those sorts of things. And so I do think the world has benefited from this. And I think when people examine that and look at kind of some of the alternatives, they come to a similar conclusion and that a world where the U.S. has the ability to support that order is a beneficial thing. And there's going to be a lot of legs of this, right? There is a diplomatic angle and relations we have there. There is trade and economics.

And there's going to be military support. And all three of those have to be used in concert. to be effective. And on the whole, I think people get that. They get that this is actually pretty complicated, that this is probably not going away, and that they'd rather be in a world where the U.S. has the edge, and that is probably a net better world.

Then I think people do have very reasonable disagreements around use of force when we should employ it and how we should employ it. And that is a very important debate to have that I am very happy in America we get to have. And so I think on the whole, we found people are really gravitate to the problem, to the mission, to the impact it can have.

I think we've had no issue recruiting. There's a lot of people who truly believe this as a sort of imperative that the U.S. has this access to these technologies, and that it's not so simple as not participating makes everything peaceful. It's just empirically not true. And so I think a lot of folks choose to work in it. We've had no issue recruiting. and I think there's been a massive shift in perception of why this is important

and a reminder that for better for worse, conflict has not gone away. And we would like to get to a world where it is off the table, but that only comes through strength. And hard power deterrence is for better for worse, still part of the equation. Brian Schimp, thank you very, very much. Thank you. Convierte your passion in a business with Shopify and bathe records of ventas with the form of pay with better conversion

of the world. Has heard the best the best conversion of the

internet the thing of the website and the website on your

website and the music for your music for your business is

a great to start your period of a

year for a year on that was my interview with

Endurl CEO Brian Shemp and as Schemph said several different times, drone warfare is not meant to replace human decision-making. And Dural, you could say, is automating troops, not automating generals, right? They want elected

officials, or people empowered by elected officials, to make the key decisions for how to use these weapon systems. But here's where the story gets a little bit hairy for me. What we're seeing today with AI, the likes of Chash E.P.T. Is that artificial intelligence is encroaching on extremely human faculties like creativity, writing, and even logic. Over time, AI might climb that corporate ladder from grunt to manager in many industries, sports and music, entertainment, and defense. War. In an essay in the Atlantic Magazine, the author Ross Anderson recently examined the question of whether we should allow AI into the Situation Room, into the Cabinet, into the West Wing, whether

artificial intelligence should be allowed to make decisions about how the United States conducts war. The title and the subtitle of that piece is, quote, never give artificial intelligence the nuclear codes, the temptation to automate command and control. will be great. The danger is greater. Now that, now, and now here is the author of that essay, Ross Anderson. Ross Anderson, welcome to the show. Derek, thanks for having me. So before you and I talk about killer AI and the ethics of automating our military decision-making in a hypothetical future, I have a question for you about reality. In the current scenario, What happens if the U.S. president is informed that a nuclear missile is headed for America?

It's an interesting question, and I hope one that remains a hypothetical. You know, it's interesting that the answer to this question has changed over the years. It used to be, you know, at the start of the Cold War, or several years into the Cold War, once the United States and the Soviet Union both had large nuclear arsenals, they were carrying. by bombers, by planes, right? These things take hours to travel between Eurasia and North America. And, you know, both countries set up radar stations such that they would have a lot of warning that a nuclear attack was underway, you know, something like an hour and a half,

two hours maybe. And then over the history of the Cold War, what you see is both sides looking to innovate in their launch technology and the weapons themselves. such that instead of bombers, now they're using ICBMs, which not only make that same trip in just 30 minutes, but they come, you know, when they arrive, they come screaming down from space. You know, you really have kind of very little chance of stopping them. And so now the president, if he or she is faced with this devastating news that a nuclear attack on the United States appears to be underway,

they've got maybe 25 minutes, maybe 30 minutes. Then when you have nuclear submarines that have been perfected and they're patrolling the oceans everywhere, that decision window, as they call it, has now shrunk down to something like 15 minutes. And there are now technologies on the way that could shrink it further still to maybe like seven or eight minutes. So your piece opens with a twist on this absolutely terrifying seven-minute experience

that the American president has to, understanding that there's a nuclear attack on America. Your piece opens with a war game, a scenario that has been used to test our military's decision-making. Tell me everything about this war game. This war game was actually developed by Jacqueline Schneider, who's at the head of war games at the Uber Institution at Stanford. And it's about nuclear command and control. And here's the idea. You're hustled into the situation, room with the president, you get the absolutely terrible news that an adversary of the United States is considering a nuclear attack on our country.

And there's a wrinkle, however, intelligence officials tell you that they have good reason to believe that the enemy has developed a cyber weapon, whose purpose is to interfere. with nuclear command and control, the system that the president uses to get his or her nuclear commands out to his or her nuclear forces, right? Which means your ability to retaliate is an insurious question. And the people who played this war game were not like jokers like you and I, right? Like they're not like obvious to like to think about this stuff on the side. Like these were former heads of state.

These are NATO officials. These are former foreign ministers. These are people who, you know, have thought about this scenario, should have thought about this scenario, and really thought through what they might do if this were to happen. And chillingly, you know, some of them said, okay, well, in that case, we have to delegate launch authority out to officers out in the field at the actual, say, missile silos

or in the submarines. And look, that itself, like, disingen, aggregating nuclear commands from the president to lots of different individuals who might have different psychological temperaments or judgments and don't have the level of support of the president comes with a whole bunch of issues, which is a reason we don't do that. So that in itself is like, oh, wow, that's pretty wild that people are willing to do that. But even more chilling is that you had a, one of the regular answers was that people said, okay, automate it, have an AI make the decision as to whether to launch a retaliatory nuclear

strike. And I found that just shocking that people with that kind of life experience and career experience and presumably sobriety would be willing to give algorithms kind of control as to whether the United States would enter into a nuclear exchange. The title of your article is Never Give Articles. artificial intelligence, the nuclear codes. And when I saw the title, my reaction was, well, of course, we should never give, you know, an extremely evolved version of chat GPT the nuclear codes. That would be insane.

What's so wild about this story and about this anecdote of the war game is that it's not just, as you said, jokers like you and me and people on Twitter and Instagram saying, yeah, you know, maybe in some scenario in the future we should, you know, give a large language model or some advanced AI access to nuclear codes. No, these are military leaders who have played a popular war game, a commonly played war game that have essentially said in this not entirely unforeseeable scenario, we could imagine automating Armageddon. But as you add, Ross, there's precedent for the idea of automating Armageddon. Tell me a little bit about Russia's dead hand program. Yeah, yeah. And I, this is, um,

It's crazy. I mean, it's funny. I had some ambient knowledge about this that I'd forgotten. And I remember when I came across this again in the research, it was sort of like shocking to me anew. But yeah, it turns out that during the late Cold War, you know, again, I was talking earlier about both sides, you know, were constantly trying to innovate. And every time one side made an innovation, it disturbs that strategic equilibrium of mutually assured destruction. right? Because one side thinks, well, maybe the other has an advantage and they could achieve a first strike on us. And so what the Soviets did is they developed this dead hand technology that is very simple. And you could quibble with whether you call it AI, but the principle is it's not a human decision maker. The idea was that if this machine stopped receiving communications from the Kremlin, it would inquire into the atmospheric conditions about Moscow. And if it had seen, you know, bright flashes and, you know, high levels of radioactivity and the sorts of atmospheric conditions that you would associate with a large nuclear attack, it's not just that it's retaliating. It's the way it worked was it, it sent up an ICBM missile. And that ICBM missile sent a radio signal to all Soviet silos. And they're sending everything. So it really was kind of automating Armageddon. Now, its purpose, you know, to add some nuance to that was not because the Soviets were like, you know, let's go.

We really want Armageddon. I mean, it was to shore up their deterrent in the face of what they perceived to be a technological or a series of technological setbacks. And I don't want to suggest it's just Russia that has thought about or acted on this inclination to automate the nuclear process. Have any American researchers or military folks suggested that the U.S. should develop our own version of deadhand? Yeah, absolutely. There's two guys, two scholars who both worked in nuclear command and control. And they put out an article a few years ago, one of them, fairly senior at the Air Force Institute,

making an explicit argument that the United States, because of this shrinking decision window, again, because we're getting down from a world where we had two hours. Then we had one hour, or I'm sorry, 30 minutes with ICBMs. Now with submarines here at 15 minutes and, you know, missile technology development continues a pace. Maybe that's going to inch down to seven minutes, six minutes, five minutes. And their point was in that kind of world, you know, maybe, I don't want to say sleepy Joe Biden, but, you know, maybe it's tough to rouse Joe Biden from his bedside at three

in the morning or he's in the bathroom. You know, you're eating up two minutes, just getting the guy in front. of the nuclear football. And so maybe there's an argument these guys were saying for introducing our own dead hand. So just to catch people up, the idea of automating the nuclear process has already somewhat been done by Russia. It's something that Americans have already discussed. And now here is this war game that military leaders have responded to by saying we should actually increase the degree to which we automate our nuclear response process. Now, how much power is AI currently being given to make decisions on the battlefield?

Does the Pentagon allow the development of AI weapons that can make kill shots on their own? That's a good question. And I want to be honest, a lot of this is highly classified that the picture is murky. However, the Pentagon did recently adjust its policy to make clear that it can't,

it's funny, they sort of hide behind all these various caveads, but they are allowed to develop such a weapon that can make kill shots on its own. And look, in South Korea, along the border, they have these, you know, famously these robots that are able to shoot on their own. But the other places that you see AI kind of in the military chain of command right now is at the troop level, right? At the soldier level, you know, we're talking about drone swarms. We're talking about algorithms that can achieve dogfighting in the F-16 with maybe the human doing the shooting. You have tanks that can identify and engage targets on their own that are kind of always scanning the horizon looking for someone with an IED or a machine gun or something. And that's where this technology is today.

Like I don't want to suggest that lots of people are rushing that no one is inviting artificial intelligence into like the Joint Chiefs of Staff. However, it's not that far of a leap to imagine a world in which they would. But yeah, so now you have AI working at like on the battlefield, you know, and almost like a soldier I'm sorry, now you have AI working on the battlefield, almost in like a soldier role, right? But you could imagine at most, but you could imagine a world in the near future actually where AI starts to move up the chain of command. And why? Because AI has some obvious advantages over human commanders.

A really basic one is they don't need sleep, right? But also, you know, and we've seen this, you know, open AI before they got into large language models developed an artificial intelligence that played the game Dota. And Dota is unlike Go, you know, famously. AlphaGo, which beat the world champion, with this very brilliant creative strategic move that kind of blew everyone's minds five or six years ago. Go is not nearly as complicated

of a game as Dota. Dota is five-on-five, highly variable. The game space is three-dimensional. It's like just orders of magnitude more complex such that nobody thought that it was going to be beat by an AI, a team of AIs in this case, and they were able to do it. And you would imagine that in the future,

a commander that can take stock of an unfolding battle that has thousands of variables and positions and continually update its sort of strategic disposition and orders according to the changing of those variables, there's no human that can do that. And there may well be an AI that can do that sometimes soon. And it's easy to say, well, you know, despite that, I actually don't think we would hand over, you know, the ability to command an army or command, you know, a fleet of ships to an AI. When your adversary does it and is achieving, you know, genuine battlefield successes, you might be quite tempted.

Yeah, one way that I imagine AI possibly moving up the chain of command is that American decision makers get scared of China, that there's some future in which China seems to be developing AI that gives its military strategists some kind of super human capability to observe all these different variables that happen during war. And so they're going to be just masterful in terms of strategy. And there's some Wall Street Journal of Financial Times headline that's like China's military decision-making has now reached AI superhuman level. And that people in Washington, D.C. get freaked out and they say, we need to bring superhuman AI into the decision-making process in the Pentagon. And that leads to this scenario where we essentially invite artificial intelligence into, you know, the room with the nuclear codes. Do you have a similar sense that there's something about the fact that we are, are in this geopolitical fight against China, which is probably number two in terms of developing AI, that a kind of arms race between us and China could lead to a situation potentially

where AI is introduced into this decision-making process? Yeah, I do. I think that's really well put. I mean, we've been living in a world for the last 70 years or so where mutually assured destruction was this kind of neat symmetry between two parties. And now it's a three-body problem, right? Now there are three parties to this. There's recent reporting that suggests that China now has enough ICBMs to take out,

you know, every American city over, say, 2 million in population. And so, yeah, you no longer have this neatly symmetrical strategic equilibrium. The thing is messier and more complicated. And China looks to be, you know, in many ways, a much more formidable. long-term adversary, if it's proper to think of them as an adversary, it certainly seems like it is now, than the Soviet Union was just because, you know, its population is much larger, right? Its economy is more dynamic, or at least appears to be for now. And so, yes, I do think that, you know, we, it was really difficult to, to escape arms race dynamics with the Soviets

and now having China, I can imagine that that's only going to continue. There's other details from URSA that I won't entirely go into. I really appreciated this guy, Michael Clare, peace in world security studies professor at Hampshire College, who talked about the idea that there could be the equivalent of a flash crash in war if we automate too many of these intercontinental ballistic missile systems, the same way we had a Wall Street flash crash, where a bunch of algorithms essentially en masse and in hyperspeed sold off a bunch of securities

so that it created this enormous dumping of assets. You can imagine something similarly happening where you have a kind of domino effect of algorithms responding to each other and creating a kind of flash war scenario. That's hypothetical that's science fiction, but I just wanted to point out because I thought it was an interesting thing that you glanced toward in the piece. I want to end on solutions here, or if not solutions, then at least what models from history should guide our approach to ethically limiting artificial intelligence in command and control? Yeah, I mean, unfortunately, you know, we're in the position where the solutions are kind of unsexy, right?

They look like international agreements that are hashed out across many parties. But as you say, we do have a historical model for this, right? you know, after the Cold War ended or looks as though it was going to end famously, you know, George Bush Sr. and Mikhail Gorbachev signed the first START treaty. And that was the first in a series of agreements that, you know, nearly shrunk the nuclear weapon arsenals of the two major nuclear powers by an order of magnitude. Tremendous success. All those guys should have Nobel prizes, in my opinion.

And that's a model for this, right? And it's a model because what that represented was a recovery of human agency, of human choice, right? It was saying that, look, we are not going to allow the dynamics of technology of an arms race, of just pure kind of mindless technological competitive development to dictate how we arrange our world here on this planet, right? We are going to say that actually we don't want a world where all the incentives lead towards ever-expanding arsenal so that people have, you know, tens upon tens upon tens upon thousands of these missiles, you know, set on a hair trigger,

pointed at each other. That's a silly world to live in. And when we have these little moments of peace, you know, these windows, those are a good time to strike those agreements and try to get towards a world where there are fewer such weapons. And I think in this case, you know, we're not at a moment of peace right now. You know, our strategic disposition towards China has it hasn't been more hawkish, you know, really in decades. We have a hot war that the Ukrainians are fighting against Russia, that, you know, We obviously are bankrolling to a large extent.

And so I don't think you're going to see an agreement not to use AI and nuclear commanding control just yet. But I'm hopeful that as things potentially cool down, maybe there's a recognition that China doesn't need to be our adversary necessarily. Maybe there's new leadership in Russia and that relationship gets better. And we can sit at the table and say, hey, whatever our difference is, we don't want to live in the world where we have a bunch of nuclear arsenals, putting us, you know, one glitch away from the apocalypse.

The piece of history that I was reminded of as I read your piece and thought about what kind of action we have to take was the Montreal Protocol. So the Montreal Protocol was a 1987 international treaty to protect the ozone layer by essentially banning chlorofluorocarbons. And it took decades of scientists ringing the bell about the danger of CFCs for the world to come together and say, we have now determined that these chemicals are destroying the ozone layer and the only way to avoid the destruction of the ozone layer is for all the world to come together and ratify a treaty to phase out CFCs. With AI, it's like we've invented this technology, the equivalent of CFCs, but we're not entirely sure what the dangers are yet.

It's not entirely clear what proverbial ozone layer they're depleting. But still, we need a similar sense of international organization to ensure that we don't get the kind of arms races that you've pointed to. because I find your hypotheticals, as hypothetical as they are, right? The piece that you wrote and a lot of what we've just discussed is a kind of science fiction, but I find it such a plausible science fiction because we've just lived through the 20th century where we had a similar technology in nuclear weapons that saw through a cold war the proliferation thereof, and it's created the potential for enormous problems for the world. And I just hope we can find some way to get the equivalent of the Montreal Protocol in place,

for AI before these things become the equivalent of nuclear weapons. I'm kind of mixing a bunch of metaphors here, like, you know, AI and CFCs and nuclear weapons, but there are just some inventions that have global consequences. And it is really hard, I think, to get our arms around what the global consequences are of AI. But it's so incredibly important, I think, for governments, even in a period of geopolitical controversy to come together and talk about making sure that we don't put, you know, nuclear arsenals at the hands of systems that might go haywire and don't have this trigger of human decision-making to keep us from having this kind of flash war. Any last comment on

just this philosophy of non-proliferation in AIs that you've thought about as you wrote this piece? Yeah, I, well, first I just wanted to respond to your excellent example of the Montreal Protocol, another sort of great achievement. And, you know, unfortunately, you know, for each particular country, giving up CFCs, you know, it entailed some perhaps an economic setback, you know, a loss in productivity for some period of time, some inconvenience for your citizens.

and what's really dangerous about arms races is the thing that you're asking the other country to give up, especially in a geopolitical situation like we have right now where there isn't strong trust built up between the really important parties you would need to an agreement like this is you're making yourself vulnerable, right, to existential attacks, right?

You're not just giving out CFCs for, you know, 10 years and taking that on the chin economically. you're having to explain to your voters or to your party committee in the case of China that you have struck an agreement that makes you potentially more vulnerable to an adversary that is willing to use these tools. And so that doesn't mean that I'm necessarily pessimistic. You know, as I say, I don't, I'm not a techno determinist. And the Stark Treaty is a great example of it, right? Like people got together and said, we don't want to live in a world like that. And I think that's what it's going to take this next time around.

Ross Anderson, thank you very, very much. Derek, thanks for having to be armed. It's a blast. Plain English was hosted and reported by me, Derek Thompson, and produced by Devin Manzi. We'll see you back here every Tuesday for a brand new episode. Have a great.