Angry Planet - A Throw of the Dice: Games the Pentagon Plays

Episode Date: January 9, 2024

Welcome to the new year at Angry Planet.For the last 100 years, American defense policy has been aided by elaborate war games. SIGMA, the Cold War Game, and the Millenium Challenge are just some of the most famous. Sometimes these games are played with dice and boards, other times they’re purely electronic. Why do we do this, when did we start, and what does it all mean? More importantly, how do we make sure the board games don’t play us?Here to answer those questions is Director of the Hoover Wargaming and Crisis Simulation Initiative, Jacquelyn Schneider.What War Games Really RevealOne episode of The Crisis Game on YouTubeAngry Planet has a Substack! Join to get weekly insights into our angry planet and hear more conversations about a world in conflict.https://angryplanet.substack.com/subscribeSupport this show http://supporter.acast.com/warcollege. Hosted on Acast. See acast.com/privacy for more information.

Love this podcast. Support this show through the ACAST supporter feature. It's up to you how much you give, and there's no regular commitment. Just click the link in the show description to support now. This is an episode I've wanted to do for a really long time. And then I saw your piece in Foreign Affairs, and I've read it like three times now, what war games really reveal, which is kind of this really incredible survey. maybe surveys the wrong word,

this really incredible article about war games and their importance to the American military and also why we should be skeptical of that importance or at least kind of go in with an informed eye. So can I hear at the top, can you tell us, can you introduce yourself and tell us what you do? Sure. I'm Jackie Schneider, so I'm a Hoover Fellow at Stanford University and I'm the director of Hoover's war games

and crisis simulation initiative. So when we say war gaming, I think that brings a lot of different images to people's minds, what kind of game are we talking about? What does it actually physically look like? Yeah, and I'm always surprised. I get this question a lot. And I think the first image that people have,

that they haven't kind of done this world, is a video game. And the second image is risk. Yes, yes. I see like 20-sided dice, actually. Yeah, yeah. And you know what? Some games are like that.

But some games are also a bunch of people sitting around a table with a piece of paper that they are writing down their decisions. So games take a wide variety of different mediums. Most of the games that I'm talking about, about the games that have really influenced the American foreign policy. are games that are kind of a mixture of a lot of different mediums. So you have decision makers and military members that are sitting around military maps or computer-generated military maps, making big decisions. So this in some way would look like a group of people sitting in a situation room making decisions in a crisis, right?

So you have games that look like that. But those games on the back end of them, you have different what we call cells And there's a groups of players that are playing the bad guy, that are doing what we call adjudication, where you're determining, okay, the good guys did this and the bad guys did this, what's the next move? And then you have teams of what we would call, you know, facilitators or white cell. And these are people that are making sure that the game is working correctly. Are the, um, are the, um, are the, um, the dungeon master. Yeah. Well, I mean, you say that, you know, sometimes it's kind of like a dungeon master.

most of the time it's just somebody who's very good at program management and administration. That's a dungeon master. So games take on all these different forms. And actually, if you look at the history of American war games, sometimes they're actually completely computer simulated. And so what this word means has really changed over time. And it means different things to different people, sometimes based on kind of bureaucratic, interests. I think it's one of those little known facts

outside of kind of our circles that I think a lot of the public would find bizarre that people sitting down and playing admittedly complicated and complex board games shapes American foreign policy

and American defense policy. But like it's happened repeatedly over the last hundred years, right? Yeah, there's a lot of great examples of games that It made a really big difference in the choices that American presidents or American military commanders made. There's this great letter from Chester Nimitz about the influence of Naval War College gaming in between World War I and World War II. He was actually a student at the Naval War College during that time.

He wrote a little thesis about his experience playing these specific games. And what he said was, we played the games in so many different ways, in so many different times. that we were prepared for everything they threw at us, except Kamakazis. He said that was the one thing the games did not anticipate. So you have games like that, right, that drove what turned out to be the World War II American Navy, that drove the kind of focus on big carrier groups and a whole different set of tactics from what they had been leaning on, you know, in the previous war. So you have games like that.

And then you have other games, like, for example, the Sigma Games, which is, Games that were run at the tail end of the Kennedy administration and into the Johnson administration about Vietnam. These were extraordinary games. We're organizations across the U.S. government spent thousands of hours building scenarios, bringing in the biggest experts, having super senior and important decision makers play. And the results of the game were really prescient. They found that strategic bombing was not going to work, that the Vietnamese were probably going to be able to hold out a lot longer than we expected. depressing, but also turned out to be

relatively accurate results. But the games ended up having almost no influence on U.S. foreign policy. In fact, so far, no historian has found evidence that the games were ever even briefed to Lyndon B. Johnson. And we have countervailing evidence that some of the players,

including good old Air Force General LeMay, actually campaigned against the games. And we're lobbying through the halls of the Pentagon and in the White House about how the game was rigged and Air Force would have done much better in real life, right? So you have examples of games that have this huge influence on foreign policy. And then you have other examples where the game should have had huge influence,

but they end up not, right, for these kind of bureaucratic reasons. How do you make a game match up with reality? I mean, how do you find, what makes a game valid? Ooh, yeah. Well, you just said a social scientist sweet word valid. And in my world, you know, as a social scientist, I think of two, maybe three types of validity, right? One is internal validity. So are the findings that I find in my game, does the logic of how those findings work make sense, right?

Have I created something that doesn't have substantive bias in it? And then there's external validity or ecological validity. And that type of validity is about does how my players do, how my players play in the game represent how they would play in real life? And you need both, right? You need your players to play like real life. And you also need to build games that aren't substantively biased. But this is a really big tradeoff. Because sometimes the things that I do to make my players buy in and the things that I need to do to make them act like they would in real life also can create substantive bias.

And so as a game designer, you're constantly, if you're trying to make games that are valid, you're constantly dealing with this tradeoff. And the other kind of really big tradeoff is what I would is generalizeability and just pure numbers, right? For every game, there's a variety of different outcomes. And if you only play it once, you know that it's valid for that one time, but you don't know if somebody else played it or if we played it in a different week, if that would lead to different outcomes. And so that's also a real struggle with making sure that the results that come out of games have some sort of kind of valid. I don't want to say predictive, but that they're more or less valid representations of what the future could be. Do you have any examples of military board games where maybe the incentives were perverted or something went wrong and the outcomes were not valid? Well, this actually happens all the time because the reality is we don't know which outcome.

are valid or not. So it's really, really easy to bake a game because who's to know whether, you know, you've got it right for what we're talking about. I mean, nuclear war, right? Like, who knows? Did we get it right? Did we not get a right? Hopefully we never know. And so it is really hard to kind of check afterwards. And so this actually becomes a huge part of a lot of Department of Defense war games. So, I mean, there's examples going back into once again, kind of the inner war between World War I and World War II period where air power is becoming bigger and there are advocates within the Army Air Corps that are saying, hey, listen, this airplane's going to be a big thing. We need to invest more and then. We need to think more about the

integration of air power into the Corps Army. And then you have a bunch of kind of land power advocates that are saying, no, no, no, no, no. Like this technology doesn't make sense. Don't do it. And so there becomes a time when they're having this really big war game thinking about kind of the future of land power. And the air power advocates finally convince them to use an airplane. Like, yes, I've got my airplane and I'm going to prove to everyone how influential this technology is. But they set of rules in the game that limit the way that the airplane to be used. So you bake in rules that make sure that the airplane isn't going to have this kind of revolutionary effect. I mean, there's other examples where, you know,

we have a series of games run by the Office and that Assessment, which is a small office in the Pentagon that does kind of really big thought ideas. They were kind of the progenitors of the revolutions and military affairs idea. So they had a series of games in the end of the 1990s, early 2000s, looking at the future different types of technology. And the first few times they play it, they're not getting the results they want. They want people to play this game and be like, yeah, let's invest in unmanned.

Let's invest in, you know, information technology. They're not getting that. So they change the rules. They change the capabilities. And they are able to kind of change the way the players play. So not only does it change the outcomes, but it changes what the players learn from the game. So the players learn, hey, actually, these technologies are really revolutionary.

And I should invest in them. And we should use them on the modern battlefield. field. So those are just kind of two examples. But there are, you know, there's, it's, they're kind of almost all big games have some sort of intentional or unintentional bias baked into them because of who pays for them. It almost sounds as if they're political projects in and of themselves, these games. Oh, yeah. They're, they're extraordinary ways of influencing ideas. The act of playing in a game, a good game. is so immersive, so evocative, that the players learn from their experience.

That means that it's a really compelling tool to convince people about what you think you should invest in, for example, right? And then quite often the results of games feed into either campaign planning processes or budget and acquisition processes. So if I'm in the Navy and I want to buy more ships or I want to buy more aircraft carrier, you can imagine a game report. that comes out and says, hey, we really could use more aircraft carriers, that's going to be the type of game that you're going to want to report to Congress, right? So when the Air Force comes out and says, we need more stealth bombers, you're like, oh, okay, wow, I never would have thought that Air Force would design a war game, that the result would be we need more stealth bombers, you know? How old are games? I have, I mean, I'm not, you know, I know, I know everybody's played with hoops and sticks and, you know, and Mel Brooks talked about hitting a tree with a stick was a good game, you know, when he was a kid. But, Did the Romans play these kinds of predictive games? I mean, does it really go back a long way? Well, there's evidence of what we would consider abstract strategy games going back to early China, ancient China. And that's kind of Go, which is now a modern game of discs, right?

And those games were really developed not as a way to represent military campaigns, but as an elite game that honed your mind and made you think about big strategy ideas. And you see examples of this also in Egypt and Rome. And then moving into kind of the medieval times, you start seeing the evolution of things that look a little bit like chess, right? And so for the first time, you start seeing a more physical representation of military things. But this doesn't really become accurate enough to be used in an effective kind of military campaign analysis way until Prussia.

and with the advent of a game called Kriegschweil. So the interesting thing about Kriegsela is it's considered the first modern military war game, but it wasn't actually invented by the military. Prussia was a kind of matil, it was a society that was interested in war, that was interested in games. I remember this was before the TV. So, you know, a big parlor game at that time in Prussia's, you get these kind of the clergy and the lawyers and the doctors, you know, kind of the bourgeoisie,

and they're sitting around at night and they're playing these games. And so one of these guys realized is, hey, I bet this game would be even better if we got a bunch of military officers together and had them come up with some good rules. So these military officers come up with rules, and this becomes kind of the beginning of Kriegshvill. And what is amazing then about Kriegshvail is not that. Prussia learned how to fight better wars by, you know, playing these games, but that the arguments about the rules and the experimentation that young officers would have about developing rules and

testing rules actually turned into a really structured way to experiment with tactical innovation, and especially in a less costly way than actually having people on the battlefield. So that becomes like the advent of the first real kind of military war games used in a really institutionalized way. And then that gets adapted in like the late 19th century into Stratigos, right? It's like the American version. Right, right. And there's all, they become all these like these American or other European countries

start playing with this idea and adapting them to their own needs and adapting the rule sets. But what Creeksville does is it creates a structure, right? So now we know, hey, if I want to do. a game in an American context or an naval context, I'm going to take, I know the core elements that I need in my game, but I'm going to adjust the scenario. I'm going to adjust the rules. And so now you get this ability to basically take a recipe and then modify that for whatever your country or your unit's particular interests are. And then the deep nerd lore, Jason, and sorry, one more tangent, is that

in the Midwest, a couple of nerds are meeting every weekend and playing games and then discover in their library, like in the early 1960s, a copy of Stratigos. And they start using it as the basis for war games

that they're playing in their basement. Fast forward a few years, and this is eventually where D&D comes from, is out of these war games, kind of modified, and refined version of that. That's where we get Dungeons and Dragons. I'll close that aside off.

That's pretty awesome, though. We can move it back in proper. So when does this thing make the leap from this Prussian war game into Stratigos and then actually start being used by the Pentagon? Yeah, so in the U.S., it really takes on importance within the Navy. So the Navy becomes the one that significantly adopts wargaming. Actually, this dates back to one of the early world fairs.

I think the World Fair in Chicago, where the guy who's, if you've ever heard of a publication called Jains, which publishes information about weapons. Yeah. So the original Fred Jane was a British guy who did wargaming with the British Navy, and he brings it over in the World Fair. and the Navy begins to adopt it. So this is kind of like the beginnings. And the Navy really starts using these games as part of their process of thinking about what kind of systems and technology they should buy

and how they are the first ones integrated into their budget and planning process. And this starts happening at the Naval War College in Newport. And actually, this is a big theme about what ends up being successful in terms of American Wargaming is when the war games occur, little bit away from the center of power.

And so they're kind of forgotten. And people are just allowed to do good games. That's when they become the most influential. So that's really what started at the Naval War College was just kind of ignorance, right? Like the Big Navy didn't really like wasn't really super involved. It was like some dude, you know, admiral dude doing these things, you know, in this weird island. And because they had that kind of intellectual space, it ends up becoming. something that becomes more professionalized and a huge part of kind of the future of the Navy.

And so the Naval War College, you know, becomes one of the early adopters of war gaming. And I think probably sets the tone for what is DoD war gaming for the next hundred years. How do you go from you and I and Matthew? We can have lots of people at this game, let's say, but how do we go from the little kid game of, you know, and I watch my 10-year-old do this all the time. Okay, I attack your 10 guys and boom, they're gone. You know, how do you, I mean, what's the randomness? I mean, you don't actually in these kinds of games.

Do you use dice? I mean, what actually affects an outcome? Yeah, and it really depends. So there's a lot of different ways that you can try and, and adjudicates like the jargony war gaming word for how we determine what happens next. And one way of adjudicating it is a very fixed set of rules. So die are very helpful for that, right? Like it's a set probability of outcomes.

Those set probabilities can be tied to specific outcomes, right? And that would be what we would consider a very formalized rule structure. And a lot of games are like this. A lot of board games are like this. some campaign models are like this. And actually, this extremely formalized strict set of rules lends itself very nicely to computer automated gaming that started occurring in the 60s as, you know, the computer really advented. There are downsides to having what we would consider a very formal rule structure. The downside is, what if you got your probabilities wrong?

or what if you've limited, unduly limited the potential outcomes or the potential plays that somebody would have in real life? So there are other games that we would consider more free play, where players are kind of allowed the world of possibilities to play. And then there's a group of maybe subject matter experts, sometimes informed by set rules, sometimes not, who then look at those plays and determine the next set of outcomes. So a really good example of that are the political military games that were played in the 50s and 60s to think about nuclear war. So that's another example. And then there are other examples that are more of a hybrid where you do have some sort of formula about what the probability of kind of the next outcome is, but that

is also informed by a series of experts. And it's really the rules. It's like how you determine those outcomes where the most bias can end up in a game. And so it's, it is extremely difficult to make sure that you're making the right rules. And actually, when I run my academic games, my academic games, my goal is to get lots and lots and lots of people to play so I can have more generalizable results. I generally fake adjudication. So I want to know, I want them to think there's another move. I want them to be invested in the other move. But as soon as I introduce rules to determine what the next outcome is, I'm either making it very difficult to compare across the second move of the game or I'm inserting my version of

reality. So what I do is I tell them that they have a second move and then I give them all the same outcome. But those are, you know, there is a big difference between some of the academic and analytic games versus some of these DOD games. My academic and analytic games have a very defined question. And I am willing to sacrifice a lot of these kind of bigger complexities to answer a refined question. That's not what's happening in these big DoD games. The big DoD games, you've got like, you know, 20-30 questions that they're trying to answer. So it's a lot more complicated.

So tell a little bit about the games that you are, you know, doing that aren't military. I'm just curious what outcomes you're trying to determine. Is it just like in the corporate boardroom? So, you know, I, my research looks at emerging technology and conflict. So I spent a lot of my time looking at technology or scenarios that we don't have good data for. So cyber, AI, and the interaction of those technologies with nuclear crises. Thank God we don't have a lot of good data on that. So the first series that I ran was on cyber vulnerability.

vulnerabilities in nuclear command control and communications. And my question was, okay, if there was a vulnerability in NC3, would that make people use nuclear weapons earlier? And secondarily, if I give somebody a cyber weapon that they can use to attack nuclear command control and communications, will they use them? And so that game, we ran over three-year time period with over 580 players. each team is a group of fives that put us at about 115 groups. And we played with about 70% American, 30% foreign, lots of different types of expertise.

You know, you had venture capitalists, and then you had foreign policy decision makers, and then sometimes you had students, right? So we get a really heterogeneous, like, a lot of differences in the population. And what we found from that was that cyber vulnerabilities do not create this. like incentive to use nuclear weapons earlier. Instead, the danger is actually in misplaced confidence in cyber exploits. So we found that people really wanted to use cyber exploits to attack NC3. And that that had knock-on implications for instability and escalation.

And that when we gave a team a cyber vulnerability, the danger was that they would not use a nuclear weapon, but instead delegate or automate to lower, levels, the early use of nuclear weapons, which has significant concerns for accidents and inward escalation. But they were kind of non-intuitive relationships, but be able to play over a long period of time, allowed us to make some kind of generalizable findings about how we just as humans respond to this technology. And then apply that, you know, then, you know, you think, okay, well, how does this apply to real contexts? So that,

was like the big war game we just finished. It's funny. I just read a book by Iran scholar about artificial intelligence and nuclear command and control. And one of the big takeaways is that there's one of the things that makes deterrence work, quote unquote, as far as it works right now, is uncertainty. And that all of these new technologies and this automation will create overconfidence and make people think that they know more than they do and could have the knock-on effects that you're talking about that you've discovered in this game. It's pretty fascinating.

That's exactly what we find is that it is certainty and not uncertainty that causes the damage. or the danger. Right. Just because you're certain doesn't mean you're right. No. Yeah. And when you seed authority to these machines, you know, all sorts of bad things can happen. Yeah.

And so the war game series we're running right now with my colleagues at the Center for International Security and Cooperation at Stanford is actually looking at a U.S.-China scenario. And if there is an AI enabled weapon system that doesn't function properly. How do people respond? What controls do they put on the weapon system? And then how does that then affect the crisis scenario? So we're also looking at that. And we're having LLMs play the game and comparing that to human players to see, is it man versus the machine? What are the similarities and differences?

Which LLMs are you using? So the first iteration of this comparison, we used chat GPT 3.4. And now we're using chat GPT 4.0. And there's some other work that's been done by Stanford scholars and the computer scientists, looking at different LLMs, not against human B players, but against each other. And there are some pretty significant differences. I mean, chat GPT 3.5 is like, let's use nuclear weapons and we'll do it early for all these rational reasons. and like the norm button doesn't seem to work for that chat GBT.

But chat GBT 4.0 is like you're, he's overly polite. Heges a lot. Tries not to escalate. So there are like different personalities inside all of these LLMs. And also kind of the way you let the LLM play roles or not play roles, whether how much dialogue you kind of allow it to have. have before making decisions, that can also really drive big differences between the outcomes. Right. They react pretty heavily to your prompts. So it's a lot about how you've designed it to

play. Yes. Yes. And so the other thing is we use some of the direct demographic characteristics of our human players and of the groups that we put them in to see whether that significantly changed the outcomes. And that did not. But if you let if you let the LLM toggle, that, it goes crazy. It goes off the rail. So the longer the dialogue you let it have, the worse, it's going to be for the world. So at what point does it leap up, bang its shoe on the table, and say it's going to bury us? Is that about four moves in? I mean, yeah. I'm going to defer to my computer sciences colleagues who probably, if you ask that question, could tell you exactly how long the conversation went before things got a little netso. But I think it is a,

I think in general, what you found was that the longer we let them free dialogue before making the decision, the more escalatory the decision became. So that's not a good sign. I think we found that, we found that in general with LLMs, right? It's like the longer they move in one direction, the more prone they are to hallucinate. Yes. And for things to start getting strange. But you know, it's opposite on the human side. So the more my players talk, the more they mediate towards an average choice.

Interesting. So consensus. Yes. So a lot of I randomize my groups, which does create a bit of a problem for external validity because these people don't know each other. And so people are a little bit nicer to each other when they don't know each other very well. I'd love to compare that directly to like, you know, people who are actually. actually the type people who will be in a cabinet who have come in with preconceived ideas. And there's already antipathies and alliances.

And there's, you know, that's a much more complicated group dynamic than a bunch of people who don't really know each other that well. All right. Angry Planet listeners, we're going to pause there for a break. We're right back after this. All right, Angry Planet listeners, welcome back. Can we switch tracks here just a little bit? I'm wondering about what happens on like a metal level when there are, complaints or debates at the table about the rule set and how things are set up.

And yes, this is me setting you up to talk a little bit about the Millennium Challenge. Oh, which you know what? Okay, so yes, players can often disagree with rules thoughts. You know, LeMay says this when he is so angry about Sigma. He says, well, you bias the game by setting up bad rules about air power. We, in the war gaming community, you would say this is when the players are fighting the game. Maybe they're fighting the game because they don't like the order of battle. Maybe they're fighting the game because they don't like the scenario.

All Americans fight the game. My American players generally fight the fact that the war has occurred. Like, why do we have this happen? Why are we here? So there is always that element of fighting the game. And then there's the like, okay, you come back up. you've made a move and you see the outcome. You say that's not the outcome that would have happened.

I disagree with the way that was adjudicated. And that's very dangerous as a game designer. Because once you get, once players don't buy into the game, you no longer get those kind of super valid. You know, players then just start treating it like a simulation and less like they would in real life. So that can be pretty dangerous for game designers. Right, the illusion breaks. The illusion breaks down.

And when the illusion's broken down, like the data I would assume is kind of thrown up the window. Yep, exactly. And that's always the constant balance that you have to do when you're designing and running a war game. So one of the things I really like about the piece is that its main thrust is that we have these games, especially in the modern context, not necessarily or I guess maybe I should say the Pentagon and the Defense Department at the moment

that's see that's not even right the people you learn more about the people running the game than you do perhaps about what the results are based on the game right? Can you kind of walk us through this and I think the most pressure or the most current example is there's a lot of Taiwan war games

going on right now. We had somebody on last year to talk about one of them. Why? And what do they say about us in the current moment? Yeah. The Taiwan games are particularly interesting because you're seeing them being run at a wide variety of different institutions. So think tanks on both sides of what you would consider kind of a right and a left as well as the centrist think tanks. and they're occurring publicly, you know, they're on 60 minutes, which means that there's some level of interest or fascination coming from the general society and general public. And now you have games run, you know, by Congressman Gallagher, for example, who I thought, this is pretty remarkable, got a bunch of congressmen to stay after work and sit for hours with each other playing a war game about Taiwan. So that's pretty remarkable to see that many games. And I think one of the big differences I see between these games versus some of the war game references that you heard in like the 2010s is that this, while ostensibly being about deterrence, so not really deterrence games.

These are not, hey, it could end up in war. How do we avoid it? It's in war. They have invaded. How do we defeat? And that is a really, really significant shift. So if you're looking from the outside of these games, who's playing them, how often they're being played, what they're being played.

I think what it shows you is that the general consensus of America is a pessimism about China and less debate about the inevitability of China invading Taiwan. I think there's also a lot less strategic ambiguity about the U.S.'s willingness to support Taiwan. And while I haven't seen these public games call for American troops stationed in Taiwan, they're pretty explicit about the amount of support that Taiwan will need from the United States to include air support, naval support, as well as, you know, munitions and kind of like general weapons allocations. So I think it signals a pretty significant shift in U.S. foreign policy towards China. And it also signals that both sides, both Republican and Democrat, see this as a really,

really strong potential in the future. And the fact that it's not just coming out of the partner defense is also a really big deal, right? These are kind of civilian institutions. So just by playing the game, you're showing that there's been an outcome determined. right? I mean, they wouldn't play the game this way if they didn't think this was the way it was going to happen. Okay. Right. These are not games where they're like, will China or won't China. They're also not games that are should we or should we not. It's how. How do we support Taiwan? Not if. What? Another thing I'm thinking about right now is that art broadly does affect American foreign policy and reflects the way

the public and our officials are thinking about things. You know, we did another episode recently talking about the movie the day after and its effect on Reagan.

You know, Biden recently talked about seeing the new Mission Impossible movie and making him think about AI. But games are different. Games work in a different way. What is it that a game can

do that moves people in a way that like a movie or a television show or a book does not? Well, okay, so this is very, this is something I don't completely understand. To be fair, I don't really understand why a war game gets a lot more national press than, you know, a model or an analytic report. And actually, I'm exploring some of that with one of my colleagues at MIT, Eric Lynn Greenberg, we're trying to understand kind of why do people invest more in information if it comes from a war game than other types of analysis. So I don't completely understand that. But the act of playing and the visual nature, when we started this conversation with like, what does it look like to be in a war game,

the visual nature of that act had some sort of strong emotional response for us. in fact, you mentioned the role that kind of theater and art play on presidential decision-making. There was a game in, I want to say, in 1984 that was televised. It was like Ted Cople. And it was a multi-day series written actually by like Graham Allison, who's now at Harvard. And I think Les Gelb, who was at later was at the Council on Foreign Relations. So they write this like beautiful evocative game and you sit there and you watch and you can still get them. In fact, and my archivists have grabbed these TV shows.

You can watch these TV shows. And then the TV show ended up creating a whole different set of responses from society. So there was backlash and then there was also interest. And so we've actually seen this in the past where games have been used as theater as kind of, art in order to influence societal understandings about kind of big foreign policy issues. And there's something about this. You should watch the video. I mean, they're in this room with all their fancy kind of computers and pieces of paper,

all these white men sitting around a table making very important choices. You know, you see flashes of the different scenarios. And so there's something that I think people really connect to because we all kind of, we can put ourselves in the players' shoes. And then as players, you put yourself in the president's shoes in this immersive way that you don't do in other forms of analysis. Right. And it's supposedly not scripted, right? I mean, a good game isn't scripted, at least in terms of the outcome, right? Yeah, exactly. I mean, there's these shelling used to design a bunch of games.

games in the 50s and 60s Thomas Schelling. And he was talking about him. He said, you know, you do these games for five days. They did a series of games at Camp David over a five-day period. And they said that coming out of the game was like coming out of a dream because they so immersed themselves in the experience that it was really, really hard for them to then turn it off. And I know anybody who's done in Department of Defense War games, you build enough crisis scenarios and like ways in which the world goes really bad. And then something happens in real life. And you're like, no, we don't have to do this. And it becomes you immediately.

And I'm, you know, I'm someone who can be critical of war games, but I do it too. Something happens. I say, I played this game. You know, I know where this goes. I don't. I don't know where this goes. But I feel like I do because I lived it and experienced it. And a funny story when I was in college many years ago, two decades ago, I did a fantastic war game.

And that was a six-party talk. So it was like North Korea, China. And this is before North Korea. I'm a date myself. This is before North Korea went nuclear. And so I played on the China team. And I wrote a really great memo and I got an A.

I was like really proud of myself, right? And then my first job coming out of college was as an intelligence officer in the Air Force watch in Korea. So sitting in South Korea watching everything that North Korea was doing. And I remember I was working. I was on the night shift one night. And there was all these rumors. that Kim Jong-il was going to test a nuclear weapon. And I was like, I have played this war game.

And I wrote a memo explaining all these very well-reasoned explanations for why the relationship with China was too important. The North Koreans were never going to test a nuclear weapon. And at the end of my shift, I hit send. And wouldn't you know it, they tested that nuclear weapon like three hours later. Oh. But it was good as a total. 23-year-old to learn this lesson that you should not take too much learning from any one single

experience in a war game. The map is not the territory, right? Right. Exactly. And I tracked this down, and I will probably put, I think I found it all on YouTube, the crisis game. Yes, the crisis game. Yes, the crisis game aired in four parts.

and it's these experts kind of gaming out a Soviet invasion of Iran over the course of four nights. And we actually, we also grab the playbook. If you're interested, we can send it to you. We have that. Oh, I would love to see that. Yeah. It's like they wrote this.

It's like a little book that Graham Allison, and I think it was Lescalb wrote, and that has all of the scenario, the rules and stuff. Yeah, I would absolutely love to see that. Does, do we know if anyone else does this? Like, is China doing war games? Is Iran doing a war game? Good question. And when you read or you look at kind of the way China talks about war games,

it suggests that they don't really play games the way we do. The best American games have free play. So a lot of free decisions. What I have seen the Chinese talk about doesn't suggest that it's a lot of replay. It seems like it's pretty scripted. It feels more like an exercise. So an exercise is I know what I'm doing and I am going to execute and I'm going to practice the execution of it versus a game, which is I'm choosing the direction in which my world goes. I have most of what I see that look I don't speak or read Chinese so I'm just looking at what's translated

it looks more like exercises and a little bit less like gaming but we do I mean our allies definitely game there's some evidence that the Russians game so it happens in different countries in different ways I think definitely our NATO allies and our allies that we've had a strong relationship with for many, many decades, are constant partners in our major war games. So my other question is kind of related to that is what do you think China or Russia sees when they see, you know, a news report from us, a piece in the Washington Post, about, you know, us gaming out the invasion of Taiwan. How is one to respond to that? Yeah. And I think the Chinese look at what we're doing and say,

oh, that doesn't look very ambiguous. I think if I was China, I would watch what was coming out and think that the U.S. was pretty serious about defending Taiwan. And in some ways, that can be the point of doing these games at all, right? it becomes a costly signal for deterrence credibility. I'm not sure that the Chinese or the Russians watching these games understand that Americans, there's still a lot of room for choices and decision-making for Americans in war gaming. Like war games, if I'm a, you know, the reports I've seen of Chinese war games are like,

we did this and it means this, right? Like, it was very deliberate. And I don't think they necessarily understand that the way these games, these Taiwan games, are evolving is it's not all controlled, right? Like, there are debates that are occurring. The constituencies that are playing the games, that are advertising the games, are representing different sides. And I'm not sure that they know or understand enough about kind of American bureaucratic politics to understand that the politics that's happening underneath the games. I mean, I guess that makes sense. it just doesn't feel right.

I mean, if you have everything you see of China, everything that's written about China, which is, of course, all wrong is going to be, you know, it's a command and control totalitarian state. Everything comes down from the top. Everything is scripted. Everybody follows and locks up. Yeah. Which I'm going to guess is not 100% true somehow. Yeah, I would imagine that just as the news of us that gets over there is perhaps different.

the reality, right? I mean, I think the hard part always for decoding the U.S. I mean, for the U.S., we always feel like we don't have enough information about, you know, China or Russia because they control information better. But as a democracy, we have a lot of information out there. We have people talking all the time. It's easier to get information, but it is harder to know what is the actual opinions of American foreign policy leaders because there's so many of us talking.

And it's like the games, too. Okay, there's lots of games. Do any of these games represent what the U.S. military thinks? Do any of these games represent with the Biden administration thinks? Decoding that, I think, is a very complicated understanding. And you really have to know kind of who's who in the zoo, the, like, the people magazine gossip behind how these foreign policy decisions are made. Have games like this ever been played between actual adversaries? I mean, like, did we ever get a really cool game between like the Japanese and the Americans, you know, or the, anyway, just.

Yeah. And that's a good question because we have found games where the Russians came over and played with the United States at the Naval War College, but it was in the 90s? So was Russia a good guy, bad guy? Not sure about that. I'm part of a track two dialogue between the U.S. and China on AI, and we have been using scenarios as a way to structure useful conversations. They're not war games, but the scenarios and playing through scenarios, we've actually found to be extraordinary helpful to being able to kind of understand how to put yourself in the other side's shoes and have kind of a more open conversation.

we've also used war games. One of that academic war games that I ran, we ran it as part of a track to between India and Pakistan. And in that case, we actually put the Indians and the Pakistanis on, we mixed them on same teams, which is kind of a cool idea for trying to create trusts between people who don't normally see themselves as being on the same side.

But that's a good historical question. I'm going to track down and see if I can find any U.S. Soviet games. That would be awesome. What should the public, how should the public read the games that America plays? Like what basic knowledge should it be bringing to the table so that it is not kind of caught up in whatever propaganda purpose, you know, is intended by the game? Yeah. And I think the first thing is just understanding who's playing and why are they playing. if it's the Air Force and, you know, the, I know it's the Air Force,

and I know that they're using that testimony at Congress, you know, when the war game says by a self-bomber, that's going to make me say, well, let me show your work, show me the rules, show me this scenario, right? And that's kind of like anything. If it feels like it's too good to be true, you should question it, right? And then I think the other thing to look at is, do they show their work? So the CSIS game, which was a U.S. China, Taiwan game, they showed all their work.

They published everything. They published their rules. They published their scenario. They published their data. And so you can go through and you can disagree with some of the choices they make. You can agree with some of the choices they make, but the work's there. Right. And so that, if I'm looking at that, that to me says, oh, like this feels really, it doesn't feel like it was intentionally by. right? Like they're not trying to hide something. There may be biased there, but it's not, you know. So I think, you know, if somebody shows their work, that's really important. And then trying to understand, is this a pattern we see over many games or is this something that only occurs once, right?

Is it four people playing a game? Is it 300? Who are they? Are they students? Are they foreign policy decision makers? There's an example of a game that was played at Strategic Command where the city sec-deff played. That never happens. That is crazy. Right. And then they told everyone that the sitting sect-def played in the game. That's insane, right? That should clue you that whatever is coming out

of that public statement, that organization thought was really, really important. Okay. You know, and then you can say, well, okay, well, I can't see the work. It's classified. But maybe because that was such an unprecedented thing, I need to contextualize that entire war game finding. I have a question that I hope isn't too off the wall, but are these games fun? Not always, sometimes. I mean, the best ones are fun, right? I don't know if fun's the right word. The best ones are immersive and engaging, right?

You know, it's the ones where you, because I've played or participated in many that are about nuclear war. I don't think it's fun. you know, I don't think you should, I would hope that my players aren't trying to have fun when they're playing a nuclear game. And I'm kind of serious because sometimes like, yeah, yeah. People are just playing a game.

You don't want them to just play a game. It has to be different than that. It can't just be risk, you know? It needs to, people need to feel like the decisions that I am making have some sort of import. They matter. And so, but then there are games that really are like, are not fun. I'll be honest.

And those are the poorly design games. Those are the ones where you like, you, you disengage. And then for me as a game designer, that's the scariest moment. Like you put the game out there and you're like, you can't walk around. Like, are they engaging? Do they like it? And they, you know,

because you need them to, to immerse themselves a certain amount. But I would say, you know, if people want to play in games, you know, email me, let me know, I'll put you in our database. We run multiple games, always looking for new players, and then, you know, people can try it out for themselves, see whether they enjoy it or not. That is awesome, and I'm totally signing up. I'm not kidding. I absolutely would love to. Thank you for answering all these questions. Oh, thank you. I mean, when you've been researching it for a while, it's exciting to get to talk about it. And actually, we're just at the beginning of this. I have an archivist and a program associate, and we are,

going out and trying to bring games that have been lost to history back again. What are, like, what are you digging around and looking for? Well, we found when. I mean, some of these you don't know if they're a game or an exercise or what, right? But this one we're looking at right now, it's called an exercise. It's called high heels. Never heard of it. Okay.

But we found cables from the U.S. Embassy to the Soviet Embassy, warning them that they're going to do. this and not to take it as nuclear war. It's a nuclear command and control game or exercise. And then we found other memos from like the head of the DIA and the CIA to Kisinger and Laird and they're saying, don't do the game. It's going to make the Soviets go to war with us, you know? But I can't find the materials, right? But we found all these like great like ephemera of, I was surrounding it about how people

respond. And so we're looking into that trying to find, you know, okay, where is that? What was it? And then I guess Paul Bracken and some of the other folks have participated in it and then talked about it later. So it's one of the, well, where is it? You know? These rules are somewhere, presumably. Yeah.

Someone's garage. But so, yeah. And then we've compiled almost everything we can about Sigma. So what we're trying to do is we don't just compile the game. We're trained to also find any interaction that people have with the game. So memos, not without PowerPoints, but like briefing. so that we can actually trace, like, to anyone see this? What did they say about it?

And then we just got finding it's super exciting to do oral interviews. So now we can actually interview people about, like, did you play in the game? Did you use the game? Did you get this game in testimony, right? So we can actually trace down, like, why did you run the game? Who did you run it with?

What were the pressures that were around you as you're running the game? You mentioned Millennium Challenge. That's one of the games that we're tracking down. and it's like a relatively recent one, so it's kind of fun because a lot of people have really strong emotional responses to it. Yes, they do. So that's where we're kind of at the beginning of that right now.

And the goal is that all this stuff ends up becoming publicly accessible. We are building a database that will have a lot of metadata. So it would be treated, it would be much more searchable than what currently is where you are like literally. sifting through both digital and non-digital archives. And that's what I've got. I've got a team sometimes digitally sifting through archives. A lot of times physically going to like NARA or the presidential archives or the PME archives, people's personal papers, and trying to do all that hard work of bringing history back.

Well, I'm going to be watching your work. And I eagerly await being able to play Sigma or some of these other games or at least like look at the stuff. Maybe not play it. Maybe that'll just be depressing. I think the Sigma Games would be slightly depressing. But thank you for coming onto Angry Planet and walking us through all of this. Yeah. Thank you so much for having me. Really, it was a great opportunity. Thanks so much. All right. That's all for this week. Angry Planet listeners. As always, Angry Planet is me, Matthew Galt, Jason Fields, and Kevin O'Dell was created by myself. And Jason Fields. If you like us, please go to the Substack, AngryPlanentpod.com or AngryPlanet.com.

and kick us that $9 a month. It helps us keep the show going. There will be more bonus episodes this year. Also, if you are seeing the substack post for this episode, there is an invite into the Discord server. I've just set it up as I'm recording this just now. We'll see how that goes. I've also made a couple changes to this particular episode based on some user feedback that I've been getting, which I have invited.

I'm going to be a little bit more of an interactive show this year, too. Trying to get some intro music or intro noises that's fast that lets people know the show has started that the ads are over. So there's a little kind of click noise that we used from one of the past incarnations

of the show. We'll see how I feel about that and how the audience feels about that. Anyway, we will be back next week with another conversation about conflict on an angry planet. It's going to be a good one. It's going to be,

look at Russia and Ukraine from someone who has been inside Russia for a long time. It has a very interesting perspective, looking forward to it. We will see you next week. Stay safe until then.