The Team House - Burn-In: how AI will change the battlefields of tomorrow with August Cole and Peter Singer, Ep. 43

Episode Date: May 23, 2020

August Cole and Peter Singer are authors of the near future techno-thriller titled Burn-In about a FBI agent who is partnered up with a A.I. Support the stream on Patreon: https://www.patreon.com/m/TheTeamHouseBecome a supporter of this podcast: https://www.spreaker.com/podcast/the-team-house--5960890/support.

Being a parent can be really challenging. It's normal to feel uncertain about whether you're doing the right things to raise healthy and happy children. That's why Child and Family Resource Network focuses on connecting pregnant parents and those with kids under the age of five with free support services to help them build confidence in their parenting journey. Everyone deserves to have someone they can turn to for support with parenting. Visit child and family resource network.org today. There we go. on YouTube. This is episode 43 of the Team House. I am Jack Murphy here with co-hosts over there or down

there, Dave Park. Today, our guests on the show are authors Peter Singer and August Cole. They are the authors of a new novel called Burnin, which I finished last night. We're really excited to talk about. It is a book about the sort of, I guess we'll let them get into it, but the collision course between human beings and artificial intelligence. So, So it's a really insightful book, near future book. I wouldn't even call it science fiction. It's not like Star Trek or something. This is what we're going to be dealing with in the next 20 years.

So Peter is a strategist at New America. He has a PhD from Harvard in government. He's the author of seven books, including Ghost Fleet and Burnin, which he co-authored with our second guest here, August Cole, who is a non-resident senior fellow at the Brent Scowcroft Center on Strategy and Security at the Atlantic Council. and he is a former Wall Street Journal reporter. So both of you guys, thank you so much for joining us tonight. Oh, thanks for having us.

We really appreciate it. It's great to be here. So just to kick this all off, I mean, I think I'd like to ask you guys if you can give us a little thumbnail sketch of what this book is about, rather than me giving you the synopsis myself since you're the authors. You know it better than anyone. I'll jump in on that. And again, I really want to thank you for having us on. It's a great group and just really appreciate it. So, Burnin, the title is taken from the idea of when you push a new technology to the breaking point in order to learn from it.

So, for example, you know, how deep can you take a new watch before underwater, before it cracks or something like that? And so that's essentially the idea of the book. It's a smash up of a novel and nonfiction. So it's a techno thriller. It follows a veteran turned FBI agent who's hunting a terrorist through the streets of a future Washington, D.C. So you get those kind of thrills and chills and interesting things like that. But baked into the story are over 300 explanations and predictions that are all drawn from the real world. And in the back of the story are 27 pages of research notes to show, hey, this really is drawn from the real world.

This really can't happen. And so you get the fun of the story, but you also walk away from it, understanding, you know, everything from how does AI work, what are going to be some of the applications by military, by police, by terrorists, what are going to be some of the dilemmas that we're going to have to face. August and I are both dads, and we kind of liken it to sneaking fruit and veggies into a smoothie. So you get the entertainment, but you also get the good stuff at the end. Yeah, that can be a really, you know, dubious endeavor for, or with a lot of pitfalls for it, I feel like, for a lot of authors who they try to take their profession and weave it into a novel to, you know, reach a larger audience with it through a narrative story. But I feel like you guys pulled it off really well. And I would even say you did a good job at filling a void that was left by the late Michael Crichton. You know, this is, all of the characters in the book are very smart, which is something I, appreciated. And the protagonist is a female FBI agent who is a former Marine. And when she was in the

Marines, her job was working a lot with droids, with drones and different sorts of robotics that would, you know, help out a squad or a platoon of Marines. So it's interesting how she came to it, her interaction with the AI is very pragmatic. She sees it as another tool. But then to see how other people interact with it, like for instance, her daughter, like falls in love with this thing and just loves it like a big toy. You know, trying to characterize the human-machine relationship is a really interesting challenge. I mean, you know, writing a memorable or interesting character

that's human is, I think, hard enough. What we were trying to do was to, you know, use the way that this Agent Laura Keegan relates to the bot as a way to talk about some of these really big issues around trust about, you know, is technology good or bad? They're kind of like meta-thames, but we feel, you know, when you're looking, at trying to understand them, being able to see what it looks like in a real world context,

even if that world is, you know, 15 years from now. And, you know, one of the things, Jack, that's interesting you noted her background, that role that she has is actually the new role in Marine squads. And so, you know, again, we play that forward and go, okay, what does that person take from their experience in war, including in this new role that, you know, people are living out there right now. And how do they take from that and bring it back into their home life back into their next job? And just as August said, it also makes her a character where, and I think a lot of people have experienced this, you know, she's used the technology. She knows what it's good

for, but she also knows that it's not perfect, that it breaks, that there's flaws to it. she also brings a lot of the lessons that she's learned out in the war zone and applies them into this space. It makes her, I think, savier also makes her maybe some mistakes, too. And so again, we try and have it that she's a character with some depth to her. Yeah, and I would not go as far as to say that the book is dystopian per se. It's not a cyberpunk necessarily. It's it's more very near future, but it does address a lot of the fallout of the things that are happening right now and how that can project itself, you know, 20, 30 years having, you know, in her case, you know, endless wars. And, you know, she's talking about how she has this nerve damage

in her legs. And it's like, I know people who have that exact same problem, you know. So it is very relatable. But before we dive deeper into the book, I wanted to kind of address the larger issue of AI and kind of play devil's advocate here a little bit because your book is the premise that AI is the next revolution. This is the big thing that is going to hit the human race and we're all going to have to grapple with. Now, I'm 36 years old. When I was a little kid, virtual reality was the big thing that was going to change the world. It was going to change everything. And then it even had a little second renaissance like five years ago. VR was back. And then what are some of the other technologies that have come around and then it seems like they peter out and they go away.

Oh, when I was a teenager, nanotechnology, we were talking about universal constructors and we were each going to have the power of a God, be able to, you know, I remember reading one book about how you be able to move your hands and make your house bigger and then contract it, make the house smaller again, hold it in the palm of your hand. I mean, all this sorts of like very, like, utopian thoughts about technology. I was wondering if you could explain to us why this is different, Why this craze, if you will, or let's just say caution about artificial intelligence is the real deal, and it's something we all need to be taking seriously. I'll jump on to that, August, and you then hit next. So that's one of the, I think, values that you get by the combination of going after the fun, exciting story, but also backing it up with the research.

So we need to be clear here. there's no vaporware in this story. Every single technology already exists is already either deployed out there or already in the real world. And so I think that allows you, one, it gives you the sense of realism. It also means that you don't get stuck in those kind of pie in the sky things that you're talking about. It's basically saying, okay, let's just take where we are at right now with, be it Siri, Alexa, big data tracking,

face recognition software, maybe it's on the robotic side, what we've seen with prototypes of everything from the tiny handhelds that micro drones that are starting to be used in swarms to people have probably seen the videos of the Boston Dynamics ones

that, you know, do parkour or whatever. And again, remember, that's something, you know, those videos that are going viral. A lot of them are from like 2016, 2017. So he said, okay, what does that look like as you move that dial forward? And what we're seeing play out is essentially a new industrial revolution. And it's affecting, you know, all sorts of different roles, all sorts of different jobs. You know, frankly, coronavirus has accelerated a lot of this.

You are seeing telemedicine. It took us a couple of weeks to get where that industry thought they would be in 10 years. You're seeing drones rolled out in policing. You're seeing robots clean subways. You're seeing big data tracking. So again, I think it's a new industrial revolution. But it's not, again, it's not pine in the sky. It's going to have good effects. It's going to have bad effects. It's going to be used in lots of different ways out on the battlefield as a decision aid in the command post. Same thing. Your kid might have AI-infused toys. But I think it goes back to where you started. Where we come down on it is this idea that there's

a really fine line between utopian and dystopian views of the future. And it really kind of depends on where you are in that society, right? So you get a different perspective from the cop's view versus the person that's being policed. The cop in this case takes off her identity outside the door, goes into her house and now she's a mom. Now she's also, her husband is working remote. She's

seen it hit her marriage. And, you know, I think of that, August and I've been kicking around of like a lot of these things that you see get pushed out there by Silicon Valley or whatnot. It's often, you know, they're super excited about it and you're like, hey, that kind of feels a little bit creepy. It's cool, but it's creepy. And so again, that's the real world side of all this. you know there's this aspect to the AI sector that has always existed in a in a bubble of hype what's significant i think in the last you know six or seven years has been you know a couple different breakthroughs one is actually on the hardware side not just on the software side but the sorts of chips now that can essentially power what are called neural nets so imagine a computer processing capability that's based off of our own

brains. What's funny about that is we don't really understand, I mean, I'm a history major, I'm not a scientist, but scientists don't really even understand the full function of our own neurological systems. And yet we were able to create these really interesting architectures that allow computers to crunch huge amounts of data that they couldn't before, at speeds they couldn't before, and then also to learn by essentially whether it's playing against themselves, if you will, whether it's doing sorts of free association that's not like creativity as we see it, but is similar in that the way that they understand, say, the parameters of a goal might allow them to reach an outcome, you know, a robot that, for example, tips itself over to move more efficiently

than one that might try to walk, you know, as a biped. So we're at this really interesting, you know, inflection point where, you know, you're seeing those two technologies, you know, neural net and machine learning computing come together with this next generation of change. from companies like Nvidia, and then suddenly, along with that, the deployment of these technologies. And what's really interesting about a lot of the tech, I think, out there right now that is so transformational is you don't see it. And it's all around us. You know, it is a really big challenge, I think, when you have a software-driven world. You know, Mark Andreessen famously said that software is eating the world. And, you know, you've had people like, Or the Rocks, right? Software is eating the

war. You know, we're at this, you know, cognizant point where we acknowledge that we are moving past the hardware paradigm and not just computing, but conflict even. And at the same time, that rests upon, you know, access to and processing, you know, more data than we've ever produced. I think it's like, you know, you produce like a Kindle's worth of data every second individually or something. I should have that stat, you know, more nailed down. But the point is it's literally like an unimaginable amount of data. And, you know, as we try to conceive of, what is the import of that in our everyday lives. Well, we're starting to see it, as Pete said in the coronavirus response, you know, this well of data can allow, of course, something as simple as contact tracing, which I know isn't simple,

but it's a fairly upfront example. But yet all the, you know, movement to virtualizing industries like teaching, you know, this conference call, they're all powered in effect by these big breakthroughs in computing power. And so, you know, what makes this, this time is different. I mean, I lived in San Francisco during the 90s and worked, you know, in the tech sector during the dot-com boom. I've seen the hype build around not just companies, but whole kind of concepts, you know, that this time it's different. And in my gut, I always had this feeling because I'm a fairly optimistic person, but I'm quite skeptical and pragmatic too, that, no, that, you know, there were certain rules like, you know, gravity, if you will, that like weren't going to go away. But with AI, it's different in part because you're starting to break the bounds of, you know, the kind of linear ways that computers have worked in the past into a more exponential type of processing. So when computers are teaching computers, when they're writing their own code, which is we're getting very close to that point.

Right, right. It's like the machines are having sex with each other and it's changing the DNA. You know, so you're at this really interesting point where, you know, it is right to be skeptical, I think, about hype. I didn't mean to cut you off there. Sorry, David. But I think at the same time, and again, as someone who's spent a lot of time being a cynic professionally or a skeptic, I do, now that I'm in the midst of this, see it with, with a lot of potential and optimism, but as, you know, Pete is, I think, iterated in the book shows, we have a lot of concerns about how this is being put together in terms of how it affects society.

Yeah, there's one other, I'm sorry, Dave. I was saying off the one other thing that drives it, I think, relevant to your community is the competition side of this drives it forward. So it's not just the supply side, the tech getting better, and the like. It's the fact that you have all these applications, not just in the business, but into war. And again, it might be in visible ways. you know, ever more advanced unmanned aerial systems, UGBs, you name it. But it's also, you know, woven into, you know, I call them decision aids.

You know, I was at a talk that had recommendation engines for which route for people to go. It was essentially like a ways map, but for projecting expected casualties that we'd take if you went different routes. You already have that thought out. And then you have this larger strategic competition with China. And, you know, China has said they want to be the world leader in AI by the year 2030. In turn, the U.S. National Defense Strategy has said, no, no, no, no, we want to be the world leader. And so that also, you know, makes it different than like, you know, VR gadgets for kids or something like that. Mistakes are so high.

You know, I was a teenager in the 80s. So, you know, you had SkyNet, you know, Terminator, you had sort of all, you know, Bladerner, you have all these big visions of what artificial intelligence would be. Is there fear of sentience or actual sentience or is there just more fear of the collection of so much data that privacy no longer exists and that these decisions or these AIs are making decisions based on data that they really shouldn't have access to or whatever else? I think that the Terminator fear is so shaped by James Cameron's film in 1984, which is a great film. I think it was 84. But at the same time, I worry more about us, the people, you know, what we're doing with the data than I do about a... There's a book called Super Intelligence by Nick Bostrom, which is really interesting.

And it posits, you know, a lot of really kind of far-reaching scenarios, the kinds that worry, you know, people who do, you know, deep thinking, you know, Bill Gates, Stephen hawking. And, and, you know, yes, there are scenarios you, a person can't imagine where that is an existential threat to humanity. But I think in the near term, the next 20 years, the greater likelihood is that we're going to cause, you know, as much or more harm to ourselves, perhaps enabled by these technologies rather than the arrival of a superintelligence or a sky net. And the, the latter part of Burnin plugs into some of these ideas about the fusion of artificial intelligence and big data. And actually dovetails well with Peter's previous book, Like War, in that these social media behemoths know more about us than we know about ourselves.

And really, you know, I thought the novel really brought to the forefront some horrifying scenarios of what that could look like. And, I mean, not to put too fine a point on it, but you think about some of the industrial scale murder that happened in the previous century, with the Nazis, with the Soviets. Imagine something like that happening again, but with artificial intelligence and with biometrics and those sorts of technologies. And if that doesn't scare the hell out of you, I mean, I don't know what would. China is dealing with that, you know, or the Uighurs in China are dealing with that right now, right? They're being targeted as an ethnic group with the help of big data, which to me is

is terribly frightening and also risks creating a model that other nations or groups may seek to acquire, especially if they buy into kind of the Huawei framework. I think what you're seeing is also this shift. So, you know, as David laid it out, you have all of this different tracking and collection going on and it creates, you know, huge amounts of concern over privacy. And the entities that are tracking you are, you know, everything from, it might be face, recognition, we'll use face recognition as an example. It's been rolled out about, you know, there's a defense department program on it to use it for intelligence gathering and targeting, face recognition at a one kilometer distance. And you very quickly, you know, think of the applications

of that. Policing everywhere from U.S. cities to West Virginia at state police was rolled out face recognition. But it's also being rolled out on the business side. And, you know, everything from Saks Fifth Avenue, the retail company, to our favorite example of is Kentucky fried chicken. And so you get this, you know, a very obvious concern of Big Brother or in KFC's example, Big Colonel, you know, tracking everything that you do because it's not just, you know, who is this person by their face. It's matched back to, you know, everything that's brought up that in your life history,

all your posts, your bank data. now maybe your health data and you know not just what you posted but what everyone's ever posted about you wherever you've been there's a funny example that just surfaced to this where they were able to track movements of um uh u.s intelligence community special operations by a beer app uh people were signing in even at the um uh super secret um CIA uh training ground all off of that just one app. Okay, so take an AI. That's what humans were able to figure out.

Take an AI that sips through all that. And your whole history is laid out. But the big, interesting, really cool, really scary. What comes next is not just tracking your history and where you are right now, but making prediction, prediction based off of all of these analytics. And it might be prediction of everything from what's your next physical, move to what are you next going to buy, what route you might take as a soldier, you name it, but then you also with all of this get the next phase of it, which is not just prediction,

influence. That is, I can steer you to do something. And I might steer you in very obvious ways. I pop ads up at you when August goes into Starbucks, when social distancing is over, And they recognize his face. And they also read the emotions on his face because it can do that too. And they say, oh, looks like you need a double shot today. And they pre and you go, thank you very much. That's awesome. But it also might be an influence that, oh, we figured out how to maybe make you vote a certain way that you don't even know that you're being steered.

And that's, you know, again, to me, that's all real. You know, we got the footnotes to back it up. but what's great about fiction is you can play that out and, you know, see how useful it would be for an operator, for a police officer. You can also see, you know, how a parent goes, oh, I don't maybe like all of that for my kid, you know, that back and forth of it. Yeah, some of the concepts that you unpack in the book, it couldn't help but remind me of Tom Logote's book, The Conspiracy Against the Human Race, where he has this thesis that free will is a myth that it doesn't really exist. And it feels like, you know, if an AI can predict

every single thing we do, you have to ask yourself these really unsettling questions. Like, am I just a expression of the programming in my DNA? I mean, what are these decisions I'm making if a robot knows all of them? But anyway, your novel, you know, the protagonist is Laura Kegan, who's a former Marine-turned FBI agent. But the show stealer of the book, of course is TAMS, which is this robot used with an onboard AI. And I felt the way that you wrote it was really believable. It felt very real. Like if my kid joins the Marines, like she could be working with something like this. So I've wondered if you could tell us a little bit about TAMS in your novel, what this robot is, what it represents. I think, you know, one of the ways to

think about a robot is, you know, do you trust it or not? And when you think about the the elements that go into creating trust with technology, it usually comes down to experience. But the interesting thing about a bot like Tams is, you know, the morphology, the shape of it is really important too. And, you know, the use of smaller and smaller robots on the battlefield, I think will be more of a paradigm than, you know, big lumbering, you know, I don't know, 20 foot tall mecca kind of things or 30 foot tall mecas, in part because of power, in part because of survivability and, you know, hideability. And so Tams is kind of a few. of some of those concepts. And the interesting thing in trying to write that character is really

thinking a lot about the physical description and allowing being a parent can be really challenging. It's normal to feel uncertain about whether you're doing the right things to raise healthy and happy children. That's why child and family resource network focuses on connecting pregnant parents and those with kids under the age of five with free support services to help them build confidence in their parenting journey. Everyone deserves to have someone they can turn to for support with parenting. Visit child and family resource network.org today. You know, the reader to imagine enough, but also to be able to picture it is a believable,

you know, entity. And then fusing it with just enough personality, I think, that it seems like the kind of technology that, you know, we can see today, but then draw a line forward and think that, in fact, the way that my, you know, operating system with my phone will increasingly be able or Alexa, for example, in your house, increasingly able to learn you and learn to anticipate what you want and what you need, whether it's, if you're hungry or not, you know, and the, the data of really everything with the internet of things is moving more and more to the clothing we wear, to maybe the food we eat even. So suddenly a machine like that is going to know you

better than you know yourself. You know, all the biases and kind of willful blindness that we have about our own, you know, weaknesses. You know, that, that machine's going to have like this X-ray vision and just see right through that. The question is, what does it do with that information? And that was something that was always really, really kind of, I think, interesting to explore. And also tricky because, you know, in the same way, like the cell phone, I think, and then the smartphone has totally changed like a detective story

because you have the answer to every single question and every bit of research, you know, right there. You know, Tams kind of has that same potential peril for the story of a robot like that. One of the other things that's funny is August, you called Tams a character. And it is a character, but it's all. also not. It's a piece of technology. So, you know, essentially it's take your Alexa or your Siri on the software side, move that forward, but also take the physical robotics that we've seen, you know, we mentioned that you've seen the YouTube videos or whatnot, move those slightly forward. But it's still, it's just a technology. And it's a technology, though, that both we, the readers,

but also the character can't help but put our own like human emotions and identity on it. So when it says something, it's just the machine saying it, but you read what you think the emotion is. And also there's sort of a, you know, a wonderful sort of telling moment that reveals, I think, a little bit about the human character, Kegan, you know, the first time she meets it, it talks to her in a voice that's female gender. And, you know, just like what Alexa or like. And she's like, you know, screw that. No, reprogram change.

And it's this wonderful little moment that sort of reveals her background and how, you know, she looks at the world of this experience, but also how we also are kind of putting our own, like, different identities and the like on top of them. And again, this was pulled from the real world, probably the starting point for this was a nonfiction book project. I did way back in the day called Wired for War that looked at how unmanned systems were being used out in Iraq and Afghanistan.

And I came across these, you know, you had a unit in Afghanistan that gave their packbot that had gotten blown up, a military funeral. And, you know, we laughed, but, you know, you would never do that for a Humvee, but that system had saved their lives on several occasions. In other situation, in Iraq, a PackBot got stuck under heavy machine gun fire, and someone ran out about 20 yards. It was stuck in the mud to rescue it. You would never rescue, you know, a shovel or a Humvee, but again, they felt this sort of teeming with it. And this is for like PackBots that are remote operated. They've got no autonomy. They've got no voice. And yet people were kind of imprinting

these emotions on them. And, you know, again, so moved it forward. Okay, what is it like when it actually can talk? What is it like when it's got a slightly more human form? And yet, just like our character, you know, you'll be aware of it, but you will keep slipping back into, you know, thinking about it that way as beyond just a tool. It's like an it. I mean, if we'd give them, Tam's a Copenhagen habit, you know, it would have been pretty,

I think, authentic. For sure. August, you said that, you know, the, this AI,

you know, they have, they collect all this data, or they have this data. And the question is, what they do with it, what determines maybe now and possibly in the future, what an AI does with the data.

Is it the programming? Is it conclusions that it comes through through its own, like logic systems? like what makes it a eye and then you know again I guess this is kind of approaching the sentient thing, the sentience thing, but you mentioned, you know, you mentioned what it does with it. So what determines that? And what do you see determining that in the future? So right now, really the bounds of what your, you know, algorithm can do is the amount of data and also the quality of the data. You know, it is easily, you know, briefed that, you know, you can teach a neural net to

identify, you know, an individual in a megacity. The reality is, you know, trying to get a machine vision system to see what you want it to see, you know, whether it's determining that a Toyota, you know, pickup is not a tank, is really hard because you have to train that system on, on millions of images and there are large repositories, one is called ImageNet, which have their own problems because of the way that data is categorized. So there's a little bit of a perception bias in terms of like how capable some of these systems are. That said, you know, companies or groups that invest in, you know, they call it cleaning that data, which is a really manual process, can kind of do whatever they want. I mean, we don't have really clear ethics, norms, or laws around the usage of data

as it applies to this really, you know, vanguard of computing. There's a way I've been thinking about it in the tech sector. There's been a phrase for a few years now, you know, data is the new oil, right? And I think it's a pretty good way of encapsulating kind of the way that our own activity online, you know, what we buy, what we choose to do in the real world can be monetized by companies, you know, there's no such thing as truly free email. But I think about it, of course, always in the context of conflict and even war. And I'm like, data is a new ammunition. You know, and when it comes to

hunting a terrorist, even in the U.S., you know, the utilization of data, you know, you can talk about Title X authorities or whatever, but like, you know, how you not just, how you approach that problem, not just with people, but information, you know, which is an ongoing, you know, conversation in our society, you know, should Apple encrypt its, its phones, for example, or how hard should that encryption be? But we're getting to the point now where some of the information that you might rely upon getting from the data that's resident in someone's cloud account or phone can be figured out without even having to have that in the first place because these really complex computer systems can make logical leaps almost that we wouldn't otherwise or just

hoover up so much information and create associations and connections with like machine learning software that suddenly you're able to find someone in a megacity individually not by their face but perhaps by a mix of bio data, a mix of browsing activity, a mix of consumer information. So I don't know if I answered it as cleanly because it's a complex, for me, at least trying to figure it's a complex subject, but the answer is, you know, it's limitless right now, and I don't expect limits to be put on it.

And that's the world we created in burn-in, right, where that happens commercially, it happens in tech, it happens in government as well. And also, you know, adversaries get that kind of access to. So I think the other thing I want to ask about Tams in very much what your novel is about is the fusion, the relationship between human beings and these AIs. And I think it was, I had a conversation with John Rob who said something that it's going to be like our relationship with, you know, horses or domesticated animals in the past, that there's going to be this symbiosis between humans and AIs. And I don't know if you agree with that or what you think that relationship will be. but since the novel is so much about that and how the two are going to integrate,

because in the book, you know, again, it's about an FBI agent paired up with an AI during this detective story. So like literally her cop partner, if we're looking at it as a buddy movie, is an artificial intelligence. I mean, how do you think that sort of relationship between us and our creations will hash out in the coming decades? That's a great issue. And it's actually one of the core questions, not just for, you know, military doctrine moving forward, but really humanity moving forward is this idea of human machine, the human machine relationship or sometimes called human machine teaming. And you will see, and we use the book to explain all sorts of different forms of that teaming. So one part of the form

might be complete task delegation. You send out a small set of drones to survey the ground in front of you. A company uses a drone, or, you know, I saw a marine landing exercise that do the same thing that uses a drone to autonomously deliver something. In that case, it was an MRE,

but it might be, you know, Amazon, I mean, Amazon recently rolled out a test version of this, Virginia. So it might be it out there, it is out there doing its work on its own without you, tasked to do it. It might be instead the idea that it is, you know, you'd use the parallel of the horse. The one that's more often talked about is like a police dog. So you're partnered with it, but it goes and does something. It's tasked out. So you're partnered, but it's tasked out. You each do what you're really good at. Another model is that it's a decision aid. It's not even a physical

form robot. It might, it's a equivalent. Like the predator drum. No, what I'm getting out is it's more like a human concierge or like a staff officer in a headquarters. It's churning through all the data and providing recommended courses of action. Or it might be physical out there advising, sifting through data, translating it for you. Another one is the idea that it's a wingman. And literally the Air Force, the program is called that, a robotic wingman. It's like a true partner. It's right there alongside you.

And so when we're getting at you have these totally different visions. And each of them are being pushed out. They're being pushed out by different organizations, different ideas. Guess what? Some of them will work well. Some of them are not. Bad guys are going to take advantage of it.

go after one way or another. But one of the interesting things, and unlike that idea of like, you know, the horse or whatever, is that it's a learning machine. So it's not just about your vision of the partnership right now. You're always training it because it's always observing you. And it might be training it and you get that kind of back of your head like, ooh, am I training it to replace me. And this isn't futuristic, right? There are radio DJs that train the AI that replaced them last year. But there's people worried about that on the military side or whatever.

So it might be I'm trained to replace me. Or the other is the kind of the role that any leader has or any parent has. It's always watching me. Am I training it like poor, am I giving it, you know, by example. Right, right. The old, you know, the goofy, um, drugs commercial. I learned it by watching you, dad. Um, you know, so if, if, if this system is always watching you, it's also going to pick up, you know, all your bad habits or other people's bad habits. Um, or, um, it might be trained off of, um, situations or extraordinary. One of the funny versions that, that literally we're all going to deal with, um, is that all the shopping, uh, so your, your AI steers you ads, um, on you know, Amazon alike.

What's happening in coronavirus is warping all of that. So all the shopping algorithms are being, you know, trained by this extraordinary situation. So, you know, basically they're going to be steering you bags of rice for like the rest of your life

because they think that's what humans want. And you're like, no, I only wanted it for like that one two week period when I was freaking out. And, you know, so we've got sort of a similar and we'll plot spoil. There's no pandemic in the book. But there is sort of this big emergency moment. And, you know, one of the things that are human character, she's like, hold it.

Like, is this machine going to learn, like, from this disaster playing out and think it's like, this is what humans do? Well, there's a part of the book where she takes the AI to a very deviant type of strip club and has to tell it. Like, filter all this, take all this data out because this is like such an abnormal human experience. It poisons the data if you subject. Like you said, like if it was. as a kid, like you're exposing them to things they shouldn't be exposed to. I mean, the way that, you know, we're going to engage in that learning process is probably going to be more passive than active. Like, you're not going to realize necessarily that you're training a system to, it doesn't

have to even be replaced you, but to, you know, optimize, you know, as someone who's maybe running a company that's going to do that for you, you know, they're trying to quote unquote optimize your world. or in the military context, you know, you might be working with a decision aid that is, you know, shaping you and has the near-term objective of, you know, accomplishing whatever, you know, task or mission, but also might be thinking about your career because it also knows the metrics that you might be promoted on or not promoted on. You know, there's a really complex web of wadings that get figured in. And what's interesting is there's this whole black box aspect where, You know, if you're going to say the relationship between a human and a piece of software or an actual bot or a jet, you know, like Firefox and Clint Eastwood, because I'm also a child of the 80s, what are the motivations of those who are setting up those parameters?

You know, how is the system weighted? You know, what are its priorities? And that's really, really opaque. And some of it is because we're creating systems like Google Translate that one of my favorite things to do is occasionally, you know, they'll roll out a new addition. it'll work twice as good if not more. And the article will say, and the people who gave us this breakthrough don't know why it's been working better now, but it is.

That's fascinating, right? I mean, we don't, you know, usually if you have like a more efficient engine, you know, in an automobile or even a, you know, a jet turbine, like or a turbo fan in a commercial like aircraft, like that's the product of hundreds and thousands of hours of like human and computer, you know, partner engineering.

So that's an expression of that. This is something altogether different and much more difficult to come. of ascertained. What exactly is going on? I'm sorry, go ahead, Peter. Yeah, I was going to riff off that black box idea. It's important in two ways. So one is this notion of the very, we can understand it.

We don't know why it comes to the conclusions that it does, but that's the very value of it, because if we could understand it, you know, we could do it ourselves, right? And I heard, for example, the head of, there's an AI program for special operations command, and you sort of put it that way, is there challenges that they want to buy it, they want to use it, but no one literally can understand it. So how do you buy it under the U.S. government acquisition system? But there's another part of the black box problem that's a little bit is what drove us forward

with the book, is that it's a whole technology area that all of these different organizations say is so crucial to their future, whether it's that single military command or the over national defense strategy to, you look at every Fortune 500 company. They all say AI is so key to my future. And yet, we don't even understand kind of the basics of it, the terms, the applications. And, you know, I can illustrate that with numbers. There was a survey taken of leaders. And only 17% percent of them said that they had even a familiarity with AI, let alone its applications. And we know leaders lie. So it's probably not 17 percent.

But again, if all these organizations are saying it's important and only 17 percent say they even have a familiarity with it, then we've got to disconnect. We've got a problem. And so for us, you know, the way that we go after that is strangely enough with a novel. it's the idea that most people are not going to read an academic paper about how AI works. No one ever said, man, that was such an awesome PowerPoint. You ought to read it by the pool too or on your next flight. But they will read a novel.

They will talk about it, share about it with someone else. And so the idea is that we can go after the second black box problem, the human side, by tapping into that sort of human need for story. They, you know, I had read that, the article that Henry Kissinger and Eric Schmidt had written earlier in the year, and they mention how AI on cars, it like starts moving the car forward, like creeping forward at red lights, just like actual drivers do. But they, no one knows why the AI starts doing that. Rep, like mimicking, are they mimicking the behavior of the other drivers?

Like, what is it? And I mean, so beyond the obvious kind of layperson problem that I'm not an engineer, so of course I don't understand AI and machine learning on that level. But there's something, there's a phenomenon taking place that even the experts don't totally understand what the hell is going on here. It's a really great phenomenon to see because it doesn't point us towards any one direction. And what I, you know, as someone who doesn't have technical background, and, you know, gets to work in and around these issues.

issues, I kind of try to use that as a way to create space for people who can bring perspective that a true, like a pure engineer may not have. Because honestly, you know, if you don't have a much larger conversation, especially at like the society level, or, you know, take the example of within a, you know, a government agency or, you know, in a military, you know, combatant command, right? Like, you know, being able to have a very fulsome conversation about the true capabilities, about the mysteries, right, about the performance and kind of objectives that you have with the capability that isn't necessarily as expensive as traditional acquisitions either, which is part of the challenge too because it doesn't fit quite in the same dollar

bucket as an F-35 or something like a large submarine. So the point is you're at this, at this like entering the zone where you're dealing with technology that's difficult to understand. It is even a challenge to come up with a common framework or set of base level definitions about what AI is because there's, of course, many different permutations, which makes it very difficult again to kind of have this very defined, you know, rigid interaction with like a bureaucracy that needs to acquire that capability. And yet also creating space for that surprise and kind of that mystery, because especially in a conflict context, you know, when people do

unorthodox or audacious things, those are often actions that are closely linked to not just tactical, you know, success, but like strategic victory. And so you, I think, are going to get into a really interesting zone where we're going to be probably experimenting more than we're either admitting or are comfortable with, in part out of necessity, especially, you know, if the conversation right now is about great power of conflict, and that means China, which has a very clear and well-funded AI program for not just its PLA, but for a whole society. You know, we're going to be in a position where we don't get to take baby steps and decide that is a fast follower, you know, to use the tech jargon that waits to see what works because of the super speed of a,

you know, decision-making, you know, neural net or, you know, targeting systems that are used by swarming under sea capabilities, like you can't go second. And that's something that I think we haven't quite figured out yet with doctrine, at least in the, you know, the public sphere. And the other subject, I'm sorry. I'm just going to add the other subject that we also have to compete with is the Chinese have very little ethical considerations. It seems when it comes to something like gene editing. So there's the whole biological side of manipulating our genetics as well that we may not feel like it's something we want to do, but other actors around the world may not feel the same way that we do about it. But I'm sorry, go ahead, Peter. No, no, it hits it perfectly because

what you're bringing in is something that, you know, anyone on the military side would recognize, which is the enemy gets a vote. So you have your plans, you know, so you've got the plan for the AI, you've got the AI out there, but you also have, you know, Klausowitz, you've got fog, you've got friction. And that fog and friction comes from both, hey, guess what? In the real world, plans don't always work out the way that you expect. And also, you have a thinking adversary who's constantly going against them. So, you know, that example of the cars, yes, you know, we'll have more and more driverless cars.

And they'll be doing, you know, on one hand, there'll be massive efficiencies. And then they'll be acting in these little strange ways that you said, you kind of a little hopping forward. But oh, by the way, the cars will be owned by different entities and different people who will program them differently to different priorities. some of them will be operating completely on their own and some of them will still have a person behind the wheel and the outcome of that is guess what you'll still have traffic jams just a different kind of traffic jam right um and and you know so we have a lot of fun with that but again I think um it's it's important you know I both um a lot of military plans a lot of military war games as also a lot of, you know, sci-fi has this idea that everything works out the way that you planned.

You know, all the technology is clean and perfect. And instead, no, you know, things break, things don't work. The enemy goes after it. It's unevenly distributed, you know, back to we're talking about the different privacy concerns. Guess what? Different government bureaucracies will buy different systems. They won't communicate perfectly. You know, so while we can't predict the future perfectly, I think we can predict that we'll have a bad U.S. acquisition system still in the future, right?

There will still be Windows XP, right? I mean, you know, AI, you know, hover tanks, but we'll literally probably still have Windows X2. Yeah, DOD's acquisition system will still be broken in the year 2100. I'm positive of it. Which is where it's most like China might have, you know, might have an advantage in that sense because, you know, I mean, they control it all. They determine what product or they don't have to go through sort of the democratic process of acquisitions or, you know, or whatever. A quick question that made me think, because you're talking about cars, completely like automated cars. And being a parent can be really challenging. It's normal to feel uncertain about whether you're doing the right things to raise healthy and happy children. That's why Child and Family Resource Network focuses on connecting pregnant parents and those with kids under the age of five with free support services to help them build confidence in their parenting journey.

Everyone deserves to have someone they can turn to for support with parenting. Visit child and family resource network.org today. I don't know what the origin of Myspace was. And then if Facebook was just going to be MySpace, if it was ever, there was ever the intent to be, become data brokers or if they figured out that along the way. But it seems as though almost everything at some point will be a data collector. If your car is automated, then your car may send information on every location you go to, how long you spend there, every conversation or keywords of conversations you have in the car. And that may not matter to Toyota where you go. But are there like big data

clearing houses that collect stuff, like it is maybe Tinder. Are they kind of collecting data on what type of people you like? And then it all goes some, because they don't maybe care, but somebody cares because somebody's using all these different data points to create this aggregate. Are there these places like that that basically buy raise information and put it all together? I'll jump on that first August. What you lay out is basically the two different models of kind of strategic competition as well as just different economic models and the like. So on the China side, you have a government that not only has access to all data, but can steer companies towards different directions. So you have, you know, it's chosen,

it's tech powerhouses, the by-doos and the like, but also, they have to share across any and all data. So it's out there supporting them, but that's part of the deal. And so you get the result of what's called the social credit system is emerging there. It's an idea that you get a single score, just like your financial credit score, but it's for your, and it's literally called your trustworthiness in the eye of the government. And that score going into it is everything from your online. what you voice online to what you buy.

If you buy diapers, your score goes up because you're a good parent. If you spend over an hour playing video games, your score goes down because you're goofing around. It's not just you. It's your network. So if your brother complains about the food at the hospital, which is government run, your score goes down. And so then you go to your brother, hey, knock it off.

And the reason is because the score has real impact. It's being woven into everything from if your score is not high enough, you don't qualify for a certain job application to if it's not high enough. You don't get a bed on an overnight train. It's even creepily, the underlying software from it is pulled from dating software, online dating software, match software. So if your score is not high enough, you don't get matched with someone attractive. So that's the China model, sort of this mass comprehensive one.

The U.S. system is we don't choose our technology champions. They more kind of emerge up. And so each of them is both plowing into AI and also hoovering up all that data and trying to access as many of the different databases that they have, but they kind of aren't, you know, they don't perfectly overlap. So you'll get these situations, and we're already sort of feeling this, where a Facebook might have more data on you than the FBI agent would. Or the special operator out in the field might be able to pull up more with a open source web search than they could from their classified network search. And so you get that kind of mismatch that's out there. But I got one more bad news for you when you were talking about like, oh, is, is, you know, my driverless car going to collect with me?

If you have a car that's, you know, a couple of years old or new, it's already collecting on you. It doesn't have to be robotic for that to happen. And it's collecting not just your movement data. It's collecting, even some of them collect voice. And, you know, that's used to train up AI. So anything that's connected now has a sense. sensor and it's collecting on you. There's this other aspect, too, where, you know, we'd like to say that consumer

choice will drive or dictate, you know, the boundaries that we put on companies in regards to how they treat our privacy. But I don't think the reality and really, honestly, the value of that data, you know, probably from a corporate perspective, is going to eclipse, you know, that kind of good sense, unfortunately. I did a little research on this little while back trying to figure out which of the automakers, you know, had better data policies. And from what I could tell Toyota had a clearer and fairly articulated one that it was essentially going to be more protective.

But other automakers, it was quite clear that like anything the car collected, they would consider their property. And this is a really interesting thing when you think about another issue that we raise this in the book, but you see it today in terms of how do we handle half of working Americans being out of work. Should we be paid for the data we produce? It's the sort of thing where when we're trying to kind of think about a post-employment economy, you know, in the mid-20th century, excuse me, 21st century as, you know, AI and software do more for us. There is this question of, you know, what is the economic value of data and who owns it? And that's actually quite a big fight that's looming. And it may be one that is, you know, suppressed or sort of, you know, kept from actually becoming a full on contest

because it has extreme economic implications. You know, some of America's largest technology champions would have very different business models if they had to, you know, cut a check or, you know, give you like a royalty, if you will, in the same way of, you know, a recording artist might. So, so there's, there's certainly, you know, what you're saying is so much truth in that. And, you know, we just don't know. And of course, there are also, you know, the same data brokerage firms today like Axiom and others that work in the consumer credit market that, you know, feed information to corporations. They're continuing to use, you know, advanced, you know, machine learning systems to, you know,

not just obviously gather data, but to process it. And, and there's an interesting. like special operations aspect of this too where you know the ability to go into areas that have like a 8 to 80 bubble or you know are denied and find people in ways that don't require the conventional you know satellite a drone being able to tap into those commercial databases i feel like is going to be something that will be a discriminating advantage in especially a great power conflict or or a mid to low intensity conflict in northern europe and yet we're not not really having a conversation about what the rules are. And when you look back to the history of special operations, Second World War, it was very much like break things on the fly and get stuff done, especially the British. And I feel like when we consider the implications

of not being able to do what we were expected to do, strategically when America sets out a objective like that militarily and says, OK, we're going to use small units to do that, watching what happens in terms of the rules that will have to be broken to get those missions accomplished. I feel like the more we can talk about and consider that today, the better a chance

we're able to be able to handle it ethically and also effectively, because you made the point too about our adversaries are gonna be pushing and past those boundaries and breaking those rules before us. So we have to, I think, be in that position where we've at least given consideration. And law of armed conflict, humanitarian law,

like we have frameworks that kind of speak to this somewhat already, but there almost has to be like another, another iteration or another version of that that accounts for all this data. When, and not just those that we had actually from the Army, on one of our first shows, we had somebody from psychological operations, SIAPs, Army SciAPs on our show. And they were under so many restrictions in terms of doing sort of cyber warfare or battle with other countries because of how it might affect U.S. citizens.

So are, and I'm not saying this is necessarily a bad thing. I think it's hard to say. but our government really puts the leash on on our own sort of operations you know when it comes to I don't know I mean we were talking more about propaganda but I could see how we would boil over to dad and everything else because we are supposed to respect privacy even though you know that's questionable now but it's it's a very interesting topic I'll jump on that in part too because one of the things that's fascinating to me is the idea that you'll have so William Gibson is an awesome

sci-fi writer, and he talked about how the future is already here, it's unevenly distributed. And we play with that in one way of like, okay, the book has its technologies that are already here, even some of the attack modes that are already here. One of the scenes has a certain kind of cybersecurity breach that literally just happened in Israel, two weeks ago.

We'll plot spoil a little bit. Hackers went after the chlorine level in Israeli water treatment plants. If you live in the greater Washington, D.C. area and you think that the little small towns up the Potomac River have really good cybersecurity on their water treatment plants. It's better than Israeli cybersecurity. I've got really, really bad news for you. But so you got that one kind of technology distribution, different forms. You've got another, which is, you know, income level.

Some people have advanced technology. Some people will still have the old version of it. You will have, and it might be individuals. It might be organizations, which, you know, when at an emergency, different government units show up at the party, and they have different levels of tech. But then there's also what you bring up, which is they'll have different rules that guide their different systems. So someone in one agency,

their system is going to be allowed to do X. Someone in a different government agency is going to be allowed to do Y. Someone working for private sector, it might be a whole other. And then one of the other, so you've got sort of the different organizations will be allowed to do different things.

I mean, look, we already lived out. You know, you got Title X and all that sort of stuff. So apply that to our next generation of technology, but you also have like different geographic areas where you will be, different rules will mean different things are rolled out. So there might be some places that you go where there's mass amounts of face recognition cameras all over the place. And there might be others where people have sort of decided that local community or universities right now are saying, we're going to ban it on our campus. So it might be like in a city that has tons of cameras everywhere,

but on the campus in it, there's not. And, you know, that cacophony, from a fictional standpoint, it's really cool to have your character sort of go through that journey from place to place to place. But also, I think, you know, for the operator, it means, you know, again, you're not going to have this smooth battlefield. The battlefield environment will have, you know, these different areas that you deploy into,

and you've got to be ready for each one. I wanted to also address with you guys. there's a very interesting subplot in your book. And as I was reading it, I felt almost like this could have been like Andrew Yang's campaign book, because so much of it is about how AI is not only going to affect the battlefield and law enforcement, national security, but it's going to put a lot of us out of work. And, you know, for sure, there are already, you know, blue-collar Americans out there who have felt the effects of globalization and their jobs being displaced overseas.

But now we're also talking about white-collar America. The protagonist in your novel, her husband is a lawyer, and like almost overnight, he's out of a job. And I was wondering if you guys had ever seen Ghost in the Shell's Stand-Alone Complex, where there's one episode where, you know, it's a cyberpunk sci-fi show. And it's a court, like the scene is like a courtroom drama. And the prosecutor, you know, gives one of those like, Your Honor, I object. And the judge looks over to a bank of like six different computers, like these black boxes. and they're all boop, blue, blue, blue, blue, blue, and then some of them light up red, some of them light up green, and the judge is like, oh, objection sustained.

And it sounded, that was in the 2000s when that came out, and I thought it was a little goofy at the time. But now seeing where we are today and reading your book and the things you had to say about the legal practice, kind of getting wiped out by AI, or at least much of it, it's like, ooh, that's not so science fictiony anymore. It's really interesting when you research how we perceive that risk today. and, you know, we've scoured countless reports trying to kind of come, you know, closer to the truth of what's ahead. And the thing that always fascinated me was every time I read a report, it seemed to be that, you know, AI or software driven or robotic driven, you know, job replacement was going to happen to other people, but not the kind of people who wrote

the reports. Not the people who go to like Ivy League schools and they did everything, quote, unquote, right in life. Right. And so, you know, the way I'm looking at that is, you know, from, I used to be a business journalist, so I've seen a lot of, you know, how companies think and make these kinds of investment decisions. It makes a lot of sense to replace very expensive people with very inexpensive software. And though a lot of the narrative is on a person who has an hourly job or a mechanical job that can be automated with, you know, an armed robot, I actually feel like in finance law, particularly in medicine too, to some extent, now that we're creating so much telemedicine, well, that's training data for the next generation of not deep fake doctors,

but you know, sort of telepresence physicians that are synthetic personalities. The point is, you know, this sort of automation-driven replacement isn't going to happen to other people. It's going to happen throughout the economy. We will, of course, I think, have a societal discussion that acts like this is inevitable. And it's not, right? I mean, we have to know that we have agency in figuring us out. We have a voice.

And one of the worries, I think, when we talk about big tech trends like this, it's as if it's looking at a tsunami coming at you, you know, and there's nothing you can do. But rather, we can decide what we value and what we don't in society. It's very difficult when we're as fractured as we are. And I don't expect that to get any better, to be honest. So we may miss out on an opportunity to steer the march of technology in a way that is still creating, you know, a kind of a social contract that, that, you know, resembles maybe not the one of the 20th century, because I don't think that's possible, but one that at least feels equitable. And I don't mean in like necessarily in a universal basic income sense, but in that people understand what they can,

do with work, even if it isn't as economically valuable as it would have been in prior generations before this industrial revolution, but will allow them to at least have a sense of value and purpose in American society, because that's a crucial issue in keeping this country, I think, cohesive and unstable in the 21st century. You use that term post-employment society earlier. What does that mean? What does that look like that, you know, instead, because Americans, we define ourselves by our jobs, so many of us. You know, you have to want to. or spiritually and ideologically as well, how do we define ourselves?

It's like some of those things that maybe Murray Bookchin wrote about post-scarcity society. How do we define our lives and find meaning in a world like that? I mean, it's a sense of almost vertigo if you don't have the ability to orient yourself around, how many times when you meet someone, first thing that says, so what do you do? And that's obviously an easy way to connect

because we, of course, prize our professional roles in society. But at the same time, you know, we are going to have to, I think, consider what it means to be alive, to raise, you know, families, to connect, you know, what is worth fighting for and kind of, again, my kind of, you know, future conflict, you know, mindset. What does the nation truly value, not just at home, but abroad if it's not as economically engaged in the, in the ways that we, you know, used to, you know, say in the 20th century? You see these different, um, uh, professions, occupational specialties being altered by this. And, you know, just like August mentioned, we pulled all

these different studies. And they, you know, so sometimes people look at the overall economy and other times they break it down into occupational specialties. Oxford, for example, did a study that found there was, you know, 702 different kinds of jobs out there, you know, truck driver versus surgeon versus whatever. And, you know, almost all of these have military parallels to them. And what they show is just like what you were saying is that, you know, you may have, you may have seen the story in the past of, you know, the factory assembly line worker losing their job to a robot. Actually, the data shows that each robot on the assembly line meant 3.3 less workers. That's backward looking data in the past. But moving forward, you know, it hits these, it's not broken down by like income level or even sometimes past education level. And you can see, you know, some of the most elite field being hit by this.

Look at the military, right? The pilot. And yet we are seeing that replaced. But on the flip side, you are seeing a value for things that you can't teach a machine, human intelligence. So, you know, yes, we have robotic pilot. No one's talked about having a robotic special operator. Same thing, the medical equivalent of that. surgeons, the top gun of that field, they used to be measured by, you know, basically how they could

tie little tiny strings, you know, how still they could hold their hand. We already have robotic surgeons, but robotic pediatricians, we aren't talking about that right now because the role of pediatrician is like consoling, not just the kid, but the parent. And so, you know, what I'm getting at is that you'll, you'll see these different levels replaced. What the number, it's not just people don't think it might happen to them, but these reports, they're just numbers, right?

And what you can do in a story is make that weirdly real for people. Okay, how does it affect that, you know, lawyer who did everything, right, went to that Ivy League school, got the, got the good grades, It's got a job that pays a couple hundred thousand dollars a year. And that profession that he has in the book is that's all true. And it's also one that is identified that's going to be replaced. So how does that mean he, you know, how does he take that out on his marriage or what is it? So, you know, what August is bringing up?

How does that express itself in politics? Yang, you know, again, utopian, dystopian, good, bad. On one hand, he, you know, brought attention to some of these trends that are playing out. On the other hand, I think kind of, you know, very typical of someone from the tech field had sort of an overly optimistic of if we just do X, we configure it. This universal basic income concept, you know, it'll solve a lot of problems. On the other hand, think about all the anger of just Obamacare for like 8 million people. Now you're talking about, you know, essentially putting the whole population under, you know, what's in a sort of equivalent of what we would have passed called the adult. That's going to be really, really controversial, right?

So you have, you know, that might be, the solutions would be controversial. The flip side is, you know, the people that lose jobs, you know, not everyone's going to stay at home. Some are going to go protest. Well, both of you guys have brought up some really interesting points. And I think, you know, then I'd like to ask you, how do you think we can kind of pat? I don't want to say pat around, but provide some cushioning to society as these technologies advanced. What are some things that we could do to try to protect our citizens? I mean, I think a lot about societal resilience, you know, is having different layers.

You know, the vulnerability that we have right now in our technical infrastructure is pretty well known. It also, I think, carries over to the way we're connecting our physical infrastructure to the Internet of things. And that's something, I think, that has an almost, you know, very kind of linear approach. the more challenging thing is almost what we were just talking about, whereas how do you create a society that can over a generation, because many of the changes will happen, I think, quite quickly, how do you essentially create fields that don't exist today? Is a bot trainer a field in 2040, like in the same way you have a dog trainer today?

But that's not enough to keep a whole society from coming apart at the seams. And so there is, you know, the approach does America end up looking more like a European, like Germany, you know, in terms of what it provides in terms of the social safety net? Because that's what it takes to keep people, you know, fed safe and healthy. You know, the question even right now, I think in the midst of a pandemic when we can't quite agree on whether people should be subsidized by the government and receive more than they might have made in their hourly, you know, roles as workers, you know, speaks to the difficulty in trying to trying to do that. But I would honestly start to go with the fundamentals of what makes a society able to weather a pandemic, what makes it able to weather a conflict. So it's, you know, healthcare, creating an excellent healthcare system. Education is vital, especially as we start to see those same algorithmic forces being used by both internal and external actors to try to, you know, create riffs for either profit or, you know, strategic objective. It's not a very, you know, tech-rich or super-exciting, you know, set of prescriptives because we've been having many of these same debates and will continue.

But hopefully if we can start to see the imperative, like the why now, we might actually get progress. And that's, of course, expensive. Those are not measures that it come cheaply. But if you think about the cost that we would truly bear to a country that's, you know, fighting against itself year after year, year, year, you know, simmering, kind of boiling itself, you know, like, the frog metaphor. That's not going to be in America that can withstand China's rise in the 21st century. That's not an America that can be a good ally to Europe, which will need, you know, as Russia continues to be more aggressive in that sphere. So, you know, I think those are really elemental, you know, aspects of, you know, U.S. resilience. I love that idea of resilience.

And, you know, to put some numbers on it, like on the internet of things side, we're recreating all of the mistakes that we made with regular cybersecurity, a generation back with the new internet of weaving in, you know, everything from smart homes, smart power grids to smart military bases, smart weapon systems, you name it. We're not baking in security. We didn't bake it in for our communications back then

and we've spent the last, you know, 20 plus years working through all that. We're not making it in for the physical objects to put some numbers on it. A study was just out this year that found 98% of all Internet of Things device traffic is unencrypted. So if someone penetrates it, they get it. And 57% of Internet of Things devices are vulnerable to medium or high severity attacks. That's crazy that we're allowing, we're building out this world that's going to be,

so brittle and so open to attack. It's the same thing. You know, the other side of resilience, though, is the people side of the resilience. And that's what, you know, August was talking about in terms of particularly, some of it may be safety nets. Some of it, I think a big thing is retooling our education. And it within the military, you know, just like we had to retool education to deal with cybersecurity starting 15 years back, you know, and for some people is we're going to make you you're going to specialize in it. You're going to go into cyber command or information warfare. But for the rest of it was also, hey, whether you're infantry, whether you're a logistician, you've got to understand a little bit too. You've got to have a better password or whatever.

It's the same thing moving forward for these technologies that we talked about. We need to change, you know, again, everything from the PME side all the way to the doctrine. But there's a parallel to that on the civilian world side. We have this crazy mismatch between what we teach people. people and the types of jobs that are going to be out there. To give you like a tragic example of it that looms, there's a program in Indiana for, it was for factory workers that have been automated. And so they go, you know, we're retrain them.

And it's good. You know, not everybody gets that opportunity, but they're retraining them to be truck drivers, which is like the next on the list to be automated. So they're like, you know, setting them up for a fall, right? And so it may also mean that we need completely different models of education. You know, August, you scared me with like the total Germany safety net model.

What I like from the German model is something different, which is they have an apprenticeship model for there. You know, and in the U.S., the only things that are apprentice model, and apprenticeship is like where you mix school and work like at the same time. And you get like on the job training. Yeah. And the only things that we have for that is like plumber. That's like the only like, but in Germany they have like apprenticeship for like

cybersecurity. And and I think that's a something we can learn from other nations and, you know, bring it over here. So again, complete agreement with August. There's like no one single silver bullet solution that's sexy and easy because guess what? We're going through a really complicated change. Dave, is it cool if I read off some. questions?

Yeah, I got them right here. All right. This one is from Zach. I don't know if you guys know this or not, but he says, what do each of you see as the prospect for startups like Winston Privacy that help users obscure their online footprint and control their data? I don't know which of you feels more comfortable tackling. If you want to start out and then August you can still in.

It actually goes to that idea of the, you know, in the future you may have William Gibson, but you'll also still have Klausowitz. And so that you'll have this like back and forth. Someone get that quote and like tweet it out for us. So you have this like back and forth. And look, we already start to see it. And just like you see in insurgency, the adversary, their response to your high tech might be a high tech response.

It might also be a low tech response. And so you might have these, and, you know, there's a product, there's a market for more of this obscuring. There's other ones that also try and wipe your web history, you know, all that kind of stuff. Some of them try and delete. Others try and flood with misinformation. And the same thing, we play with this in the book. We're not going to plot spoil a lot, but like face recognition technology and pulled from the real world,

there's both high-tech ways of tricking it. And then there's low-tech ways. And, you know, it might be anything from a makeup that has microbeads in it that are reflective, that trick the camera, to harlequin makeup, block white and black, a lot like the gestures for World War II and World War I history fans, that old school camouflage that you would have on the name. Navy ships, the sort of... Basel-dazzle. Exactly, right? And so people putting that on their face that screws up the camera. And some of the people that do it will do it because they are bad actors.

Others will do it because they are the equivalent of gestures. And I love the parallel of Doc Martin shoes. Doc Martins started out as they were used by, you know, basically anarchists and police for street battles. And then punk rockers copied them. And then they became, you know, something that everybody was doing. You know, teenage girls were wearing their, their Doc Martins. And so what I'm getting at is a lot of this, like, back and forth.

Some of it may even become fashionable. So stylish, like mainstream where it goes from. Yeah. You know, does privacy and secrets? you have social cachet, you know, right? You know, are you going to date somebody who's not like got their obsec in order? And, you know, and I think there's kind of a today version of that, which is, you know, animizing and things like that. But as Pete pointed out, you know, the equivalent in the digital information realm of pixel spoofing, you know, a machine vision sensor.

So like, can you put a pixel or two out of place on a vision snapshot of a stop sign to trick a car to accelerate, not break, you know, doing things in the information space, you know, do you have an AI manager for not just like your email, but you're a whole social media presence, right? That is almost either a coach, kind of an angel and a devil on each shoulder, or, you know, something that can passively, you know, kind of wash over your data to give you a different presence online. Again, not for anything nefarious, but just to either stay on the right side of a social credit system, you know, whether it's, you know, called, called one as they do in China or whether it's an equivalent one like we're kind of inching towards here.

Great question. And just to piggyback on Peter's point about high tech versus low tech attacks, it's interesting to see how they're updating sniper doctrine at Fort Benning. They actually envision a world of electronic warfare on the battlefield, but the sniper utilizes, you know, bolt action rifles and semi-automatic rifles. The technology is over 100 years old, can't be jammed. So they envision sending, you know, snipers stripped down of their technology down to, you know, a sniper and a spotting scope and making hand drawings for reconnaissance,

going into an electronic warfare battlefield and taking out whatever kind of enemy jamming devices they have and kind of paving the way for the main force. So it's really interesting to see how all that develops. But then, of course, you have to worry about AI. It's going to be able to spot the snipers and cut through their camouflage like nothing ever before. So. And look in the book, we've got the plan for the next gen sniper rifle. Yeah. Yeah. And, yeah. We won't reveal, but it's not August in my dream of it. It's literally what's planned to come. I had to look that up, Peter, because even I was calling bullshit on that, but no, that that is the thing. Well, I know, I know somebody was experimenting the sniper defense system. It was just an array of ears basically connect, you know, that I guess kind of a neural network or whatever of ears that were connected to, you know, these rapid guns.

So if a sniper took a shot, these guns, I mean, these ears would triangulate on the sound and immediately, you know, like zoom in and shoot. Yeah, yeah, and blast the area. So it's a challenging situation all around. And you asked about ethics, you know, you said different ethics. That system, so it automatically sloughs towards wherever it detected the fire coming from. It's not a technology question as to whether it then shoots. It's a. law of war. And so one actor might say, U.S. military might say, okay, even though you automatically slew, a human still has to approve it, someone else might say, you know what, no, because guess what, by the time the human approves, it's gone. And then one of the other things that, you know, when we talk about laws of war and application, it's not just about

different nations and how they interpret it, it's also frankly about how you're doing in the war. And, you know, I use the example of, in World War I, the U.S. decided that unrestricted submarine warfare completely violated the laws of war. It's actually why we entered the war against the Germans. And then December 7, 1941, we get hit Pearl Harbor. And you know how long it took us to change our mind? Five hours. That thing that we said was complete violation. Five hours after Pearl Harbor attack, order goes out.

commit under significant submarine warfare against Japan. It's because we were losing and we were mad. Right. So Alex wants to know, what's the least likely jobs to be replaced by AI? I mean, I'd like to say science fiction writer, but that's probably one of the easiest things to automate. I mean, I think Pete's a pediatrician example is really interesting because if we see value in human-to-human interaction, So the start of life, perhaps even the end of life, you know, those are two areas where you might find that we do decide that, you know, the inefficiency of having a, you know, a human doctor is, is less than, or the value, you know, the drag of that is less than the value of, you know, feeling in that kind of magical human way, you know, safer or more connected to a caregiver that, you know, is like us. But I honestly don't think that you could say the role of doctor, for example, can't be automated. And I really do believe this rise in telemedicine, depending on how you're taking all that information that you're gathering and feeding it, what kind of networks you're feeding it into, you're really going to be at a point where you can create the kind of telepresence synthetic personalities that can do much of the work, especially for worried well, especially if those people are heavily censorized already.

So the AI kind of knows what's going on because of the way it's been listening to them by other smart devices in their home. perhaps it knows what they've been buying. It knows how often they've been exercising, you know, what they've been drinking and where to use the untapped example. So I wouldn't really say there's too many fields I can see being off limits. But again, it's really up to us to decide what's important. Paintings in fine art are a really nice example too. You can do some crazy neural net art right now.

It has value. People are paying half million dollars, you know, as they did for the obvious art collectives, one of their works last two years ago. But also we might not see the same value in machine art as we do the, you know, the paint strokes of the human. So that's just going to be one of those tradeoffs, I think, and that we'll hopefully do actively and not passively. There was a military, U.S. military study at this and it had two sides to it. One was they asked, they sort of did a survey of people like, what did you think would never, you know, which is more likely to be automated or not?

So what did people think in the military? and then the other was like, okay, what's actually most likely to be? And it was fascinating. It's like the respondents were saying things like, you know, I don't think cooks will ever be automated. And you're like, they're kind of already are starting to be, right? Not, you know, the number of people working in the kitchen, whether it's at McDonald's or on a warship, you know,

is like whatever one-one hundredth of it used to be. Like if, you know, if you go to the old World War II ships, you know, the galleys there had massive numbers. Now it's, you know, it's a handful. McDonald's, the people in the back has gone from roughly like 60 to like four. So they were like, you know, this might never happen. But when they looked at like what the least likely as opposed to, you know, pilot or, you know, your sniper example is a really good one because it used to be that someone who did spotting

whether it was for the sniper or a forward observer calling in an air strike, artillery strike. They used to be a task that required, you know, if you're talking about a forward observer, you know, that was a officer with huge amounts of training. Now it's just anyone with a laser pointer or any machine with a laser pointer, right? So we see that change, right? But the area that they said, okay, we're not going to see that no one could contemplate civil affairs because it's, you know, it's all about that human and human intelligence, the social side of human intelligence and in turn the trust from the other side. Now that civil affairs may be pulling in all sorts of

AI and robotics to make them more effective, you know, be it translator, be it data that will allow them to know the life history of the person that they're interacting with. But at the end of the day, there's still that sort of feel to it, much like in our story, you know, there's there is robotics, but there's also still police because guess what, there's certain things that each do. So last question, or maybe two questions here from Max. He says, I work in defense, AI, and robotics. Can you speak to the technical references for Burnin? I'm super excited to check it out. and does an AGI have rights? Ooh, good question.

So the book has, as Pete said, like 27 pages of end notes that have sources that range from, you know, governmental reports to, you know, news clippings. You know, we'd like to, you know, be able to have someone have their mind blown in a scene and be like, oh, that can't really happen. And then realize, oh, wait, there is an actual end note for that. So it's a deliberate choice with those end notes to be transparent about how we do our research,

but also because we hope that it's helpful too in going further in figuring some of those things out. As to whether AGIs have rights, man, that's like a theological question that we probably have to start thinking about, even if it is something that could be 80 or 90 years away. Saudi Arabia is obviously played with robot rights with some of their kind of synthetic, robotic AI personalities. So the conversation is starting on that. But I think there's another aspect, too, is who decides that?

You know, is it a UN question? Is it, do robots have rights in certain countries or even in certain communities, for example? So there are a lot of different threads you can pull as a writer that I get really excited about. And I don't have a clear answer, but I love the question. I'm sorry, I was going to give this sort of a fun way of ending it that goes to, we're going to end on what the U.S. military and sex bots have in common. and it's from this question. And so you'll have all these looming applications,

and they're already starting to be out there. And as a result, you get all of these different, and we've talked about some legal and moral, ethical questions that come out of it. But one of the new ones that we don't have any history, anytime you get a new technology, a bow and arrow, a submarine, you get these new questions. But we get a new kind we never have before,

some of which are, you know, machine permissibility. What should the machine be allowed to do without us? But the other one is, what should humans be allowed to do to machines, which has never been like, no, no, you're not, how dare you kick your car? And this is emerging both in the idea of sex bots, which they're a thing and going to be more of a thing. but also the laws of war as they apply to unmanned systems used by the U.S. military. In both, the question is what can humans do to them? And the U.S. Air Force has concluded that unmanned systems have the same inherent right of self-defense that a manned platform has.

that is not only if you shoot at it, it can shoot back, but even more, if it looks like you're going to shoot at it. So, for example, if you light it up with a targeting radar, that we have the right to fire first because you might actually, the decision cycle is so quick when you light up a radar. And the U.S. Air Force has taken that position that robots have rights of self-defense. Now, I don't think they kind of meant it, but it's a real thing and actually I'm out there. And so welcome to one of those awesome discussions from science fiction that's now like a real issue for war.

It's fascinating. That's fascinating. Jack, we have one more question. We missed up. T. Barr gave us a donation, but then he posted the question underneath. Can you discuss the relationship between data acquisition tech and data management tech? In Operation During Freedom, it seemed like we had more intelligence surveillance reconnaissance than ability to analyze and process it.

Peter, do you want to start that in August? Yeah, and we try and depict that through, again, through the scenes. And one of the great scenes in terms of illustrating that is actually early on. And our main character is, it's this scene where they're in the train station and they're flooded with data coming at them from all of these, you know, sensors that are out there, face recognition background. And she describes how, you know, she gets that familiar feeling from when she was back in the military of, you know, basically the fire hose of data trying to sip from it. and that the real problem that we have, and again, in the real world that we play out in the fiction, is not so much data availability as trying to pick out what's relevant, what's useful or not. And again, that's sort of the idea of people think the technology is going to give you some kind of solution.

In other cases, it can actually overwhelm you. And so then to the management side, ironically, we turn more and more to AI to manage that info overload for us. And we already do it right now in the way your email inbox is sifted for most important messages or it can auto-reply, it auto-fills out answers to you or to give a military version. There's an AI that it will assess the situation that's flowing into the talk and based off of all the prior like situations where a fire order then came

out of it, it will pre-write the fire order. It won't say, okay, it won't order artillery to fire, but it'll pre-write the order and then say, hey, human, I've sifted through all the data for you and I'm managing for you. Yeah, or send the order out to that unit. And so is that, kind of, you know, the back and forth, I guess what I'm getting at is that on one hand, all the data leads to info overload, but then the flip side is, is that more and more you turn to a machine to make more and more of the decisions about it. And yeah, that's something that, you know, we live in our personal lives, but you might see it in battlefields too. Yeah, I can just tail on, you know, quickly. I think that's on the processing side of information

where you're going to see the greatest inroads on, you know, military AI, you know, and sooner too than, you know, Terminator stocking the battlefield, in part because that's where the demand is so high. Well, guys, I think we already kept you past your time. Yes, Dave. We got one last question. Brendan just sent it in. So the technical side of doctor, nurse, or paramedic can be automated, but not the human

interaction. 100% agreement. Yeah, totally agree. And again, you know, we play that out in the book. There's certain scenes that are actually set in the hospital. But we're all seeing that right now surrounding us in terms of you were seeing, as August mentioned, more telemedicine. You're seeing robotics deployed into hospitals. But then there's other roles that, you know, you just can't right now automate. And so again, you know, some of it's going to be completely automated. Some of it will be the human. aided. Some of it will be the human at a distance. You'll have all of that and what will be that, that those different, that variation will be the case or is already in the case in medicine, but it's the same thing in everything I've just said applies to warfare, right? We're seeing the same thing in warfare. And the question is, how do you best navigate it to the competing demands of everything

from what's my best doctrine? What's, you know, the most cost effective? what, oh, by the way, is, you know, legal, moral, ethical. And then finally, what about the prior, you know, organizational mindset, unit culture, you know, all the sort of things that steer us in, you know, to maybe not operate the best way. Put it bluntly, that's the way we always did it. You're like, well, maybe the world's changing, right? And so all those aspects are the case in war. They'll be the case in medicine, you name it.

So guys, the book is called Burnin. Authors here, Peter Singer, August Cole. Guys, tell us where everyone can go and find your book. You can pick it up at local bookstores. You can pick it up at Amazon. It drops on Tuesday, but pre-orders are obviously something that every writer loves. So we'd love to have people get engaged with the book and then talk about it. You know, reach out to us on Twitter, other social media platforms.

You know, this is an ongoing conversation that we're having to be. heavily invested in. So let's keep it up. Share this interview with your friends, folks. Hey, thanks everybody to join us. Please subscribe with the channel if you haven't already. Hit the bell sign for your notifications and join our Patreon. A dollar a month keeps us deep in booze and our rent paid. And I just have one final shout out, although it's apropos for this interview.

I told my friend I'd mention hackasat.com. The Air Force is actually sponsoring a hacking competition and whoever the winners are they're going to get the chance to ethically hack a satellite in orbit. So that's going on right now, hackassat.com, if you want to go and take a look at it and a friend of the show will be participating in that competition. So Peter and August, thanks so much for coming and spending, you know, an hour and 40 minutes with us tonight. Thanks a lot for the chance to connect. We appreciate it. It's a lot of fun. Yeah, I hope we can do it again. Me too. Take care of. Have a good night, everyone.