The Daily - Silicon Valley’s Military Dilemma

Episode Date: March 6, 2019

Across Silicon Valley, tech companies are pursuing contracts with the Defense Department. But seemingly lucrative deals can come with hidden costs. To explain, we look at a company that sold something to the military and later came to regret it. Guest: Kevin Roose, who writes about technology for The New York Times. For more information on today’s episode, visit nytimes.com/thedaily.

From The New York Times, I'm Michael Barbaro. This is The Daily. Today. Across Silicon Valley, tech companies are pursuing contracts with the U.S. military. Kevin Roos on a cautionary tale of where that can lead.

It's Wednesday, March 6th. So I'm a tech columnist. I spend a lot of time talking with and looking at the big tech companies and where they're going. And one of the most interesting threads that's come up in the past year or so is this question of how Silicon Valley should engage with the government and with the military specifically. And I think we're at a point right now where the government is saying, we want the things that

you're building. We want you to be part of our modernization efforts. And within some of these companies, there's a sort of growing divide between the management of the companies and what they want for their business and some of the people actually building this technology, the engineers who are working on these advanced systems that are in hot demand by the military and other government agencies. And we just saw this play out in a pretty meaningful way at Microsoft. What happened there, exactly? So last year, Microsoft signed a contract with the Department of Defense to use a technology called HoloLens in the Army. Good morning, everyone. A few years ago, we started asking ourselves,

could we make things so simple that technology would just disappear? And HoloLens was originally developed. It's kind of like a headset for augmented reality. Could we place your digital content right into your world, right into your life, with more reality than ever before. I saw a demo of this a couple years ago, and they were touting it as kind of the next generation of gaming. Welcome to a new era of Windows. Welcome to Windows Holographic.

People were wearing HoloLens, and you would be swinging your imaginary sword at imaginary monsters, but it all looked like it was sort of happening right there in your living room. And last year, Microsoft signed a contract to develop HoloLens technology for use on the battlefield. to give soldiers the ability to wear these headsets and get information about their surroundings and to increase the lethality of the soldiers on the battlefield. So this technology will specifically be deployed within the military so that American soldiers are better at killing the enemy. That's the way it's described in the contract, yes. And so I'm guessing that this was something that Microsoft engineers objected to. And then there's

Microsoft. They're receiving backlash from their own employees over new groundbreaking technology. Well, a few of them did, but they did it pretty vocally. Yes, so more than 150 Microsoft employees signed a letter demanding that the company cancel a $480 million contract to build a HoloLens for the Pentagon, saying they, quote, Dear Satya Nadella and Brad Smith, we are a global coalition of Microsoft workers and we refuse to create technology for warfare and oppression. And it goes on and basically they argue that this sort of crossed the line into weapons development. This was basically taking something that they had built and turned it into something that was designed to help soldiers on the battlefield kill other soldiers. Not what they designed it to do. Right. Not their intent, not what they were told they were going to be working on, and not something they were comfortable with.

They were told they were going to be working on and not something they were comfortable with. I mean, Kevin, thinking about this, it's kind of hard to imagine that the U.S. military would not use U.S. technology, American-made technology like this in warfare. That seems kind of inevitable, doesn't it? Well, right. I mean, as long as there has been a U.S. tech industry, there's been a partnership with the U.S. government and with the military. I mean, the original internet came out of a Defense Department project. And since then, lots of advances in technology have been spurred by this collaboration between the Pentagon and Silicon Valley. And a lot of these companies already provide technology to the military. I mean, a lot of military computers run Windows operating system. But this was a sort of a bridge too far for these employees. They say, we're okay basically providing general use

technology, the same stuff that you and I could go and buy. But the specific development of our technology for use in this specific case crosses a line for us, and we don't want to be a part of it. So, Kevin, what's interesting to you about this story? Well, I'm interested in the ethical debate around, you know, the use of artificial intelligence and other technologies. But I'm also a business reporter. I think this is a decision that is not just being made along ethical lines. These companies are also thinking about their businesses and the amount of money and the

sort of profitability of developing these systems for the military. So I wanted to look at it along those lines as well. Like, let's just bracket the ethical discussion for a second and say, like, does this make financial sense? Well, not just bracketing, right? Isn't a company often going to make decisions about moral dilemmas based on the implications to their finances? Exactly. These are not philanthropies or think tanks.

These are for-profit corporations that generally operate on a sort of cost-benefit calculation. And so just on that plane, I wanted to see, does this make sense for them? And so I did a little bit of digging and I went back about 50 years in history and I found an example of a company that sold something to the military and later came to regret it. Which was what? something to the military and later came to regret it.

Which was what? It was the case of Dow Chemical. In 1965, Dow Chemical is this small, sort of little-known chemicals company based in Michigan. They make lots of different things, household chemicals, agricultural chemicals. Now, at your favorite grocery store, the most amazing food wrap ever developed. They were probably best known for making saran wrap. Heard of it. Have you tried saran wrap?

It really is amazing. And look, saran wrap clings like magic. Saran wrap is a product of the Dahl Chemical Company. Vietnam. United States helicopter gunships backed up ground forces in a strong assault on a Viet Cong position. Meanwhile, the Vietnam War is happening. And in 1965,

Dow Chemical wins a contract with the Department of Defense to produce a new chemical called napalm B. And napalm, which is kind of a highly flammable sort of gel that binds and sticks to things and then burns them, had been in use, but napalm B was kind of the new formulation, and the military was very excited in using this in Vietnam. Troops followed up the advantage gained by the air support

to knock out the BZ. They thought it had a lot of promise on the battlefield. It was sort of hard to contain and hard to put out. So it was a very effective form of weaponry, a very horrible form of weaponry, and it produced very, very bad burns on people. They would basically be burned alive. And so there's not much noise about this

for the first little while. And then Americans start seeing images from the Vietnam War. And for a lot of people, this is their first time seeing the effects of napalm. Most famously, there's this photograph. You've probably seen it. It won the Pulitzer Prize.

It is a photograph of a nine-year-old girl who's been badly burned by a napalm weapon. So far, we may have killed a million of them, mostly children. And it sort of shocks the American public, and it galvanizes the anti-war movement. What do they think as we test out our latest weapons on them, just as the Germans tested out new medicine and new torches in the concentration

camps of Europe? And they decide to go after the use of napalm by U.S. armed forces. And the way that they do that is by going after Dow Chemical. This is Monday afternoon, November 20th, on the San Jose State College campus. What you're watching is the result of a demonstration that began about an hour ago. They stage demonstrations outside the company's headquarters, and they go to dozens of colleges where Dow was recruiting students to come work for them. The professors against the war and the students for a democratic society had marched in front of the administration building

to protest the recruitment on this campus by Dow Chemical Company of employees. And they protest the recruiting sessions. They have signs like, Dow deforms children, Dow deals death. They're not letting anyone in. They're the only people that are allowing in. I don't see them acting like they're with Dow. My favorite story from this period is that the Dow recruiters, the people that would go to campus to talk to the students, they would get locked up in classrooms.

They would sort of get like held hostage by the activists. And at one point, one of the recruiters sort of got so used to this that he started bringing a sandwich in his briefcase so that if the inevitable happened and he got locked in a room with a bunch of angry protesters, he would survive the day. He would at least have lunch. So this sounds like a complete corporate nightmare that your company is so associated with war and with death and not with the household product that you make Saran Wrap. Right. And Dow was particularly offended by this because Napalm was a very small part of its business. I mean, it accounted, I think, for a half of 1% of its annual sales. And yet, it had sort of come to take over its corporate identity. So there was some internal discussion, there were some debates within the company and at the board level about what to do, whether to sort of stop manufacturing napalm. And there were lots of reasons for that. I mean, there were customer boycotts. They did take a

financial hit and they had to do a lot to sort of counter their corporate reputation as a warmonger. But what they really worried about was recruiting. They worried that on these visits to college campuses, they weren't going to get the best people. They weren't going to get the scientists and the engineers who would help them in the years ahead. And as a result, they would have long-term costs, not just financial, but also in terms of what they were missing out on. The future employee with moral objections to Napalm who doesn't come work at Dow Chemical and make the next blockbuster product. Right. But ultimately, they decided to kind of stand their ground. They said, you know,

we have a duty to produce Napalm for the U.S. government. It's our patriotic obligation. And we can withstand the controversy. Huh. Dow Chemical feels like such an extreme case. And it feels quite different from what's going on with these tech companies today. Yeah, absolutely. These are not totally comparable. I mean, the Vietnam War at this point was extremely unpopular. You had these horrific images coming back. There's no good use for napalm, right? There's no like pro-social healthy use for napalm. It's a weapon. Whereas a lot of these technologies, AI, facial recognition, image recognition, augmented reality, these can all be used for very productive and healthy things.

It's only when you put them into a certain context that they can be used for harm. And that would seem to be the case for most tech companies, right? They are developing products not explicitly for the military. They just happen to be used that way. Exactly. Artificial intelligence, drones, warfare, and Google. It's a mixture that caused an uproar inside the tech giant where the early motto was, don't be evil.

There was a big controversy at Google last year about this military contract known as Project Maven. Google is developing artificial intelligence to analyze drone video data. And this was a Defense Department program that basically used AI and sort of image recognition technology to interpret video images. So, you know, the same kind of AI that Google would use in Google Photos or in, you know, its self-driving car unit to sort of recognize images out on the road. to make drone strikes more accurate, to be able to recognize certain people or certain buildings and be able to direct a drone at that target specifically. Again, not what Google engineers probably ever thought the technology would be used for. Exactly. I mean, they didn't sign up for that.

And, you know, some of them didn't even know that this was happening. Thousands of Google workers now protesting Google's involvement in a Pentagon drone program. So there was a big uproar. There was lots of sort of heated internal meetings. At least 10, maybe 12 employees did resign.

Some resignations. Wow, people quit the company over it. Yeah, people quit the company over it. Meantime, thousands of Google employees have signed a letter protesting the company's participation in an artificial intelligence project by the Pentagon. There was a petition that was signed by thousands of Google employees. Calling for the company to not only not enter into this contract, but to stop making tools of weaponry altogether. And ultimately... Tech giant Google will end a project with the Pentagon.

Google bowed to the pressure. They pulled the contract. They said they wouldn't renew it. They put out a statement saying, you know, we'll still work with the military on other projects, but we're not going to manufacture weapons and weapons sort of related technology. And that seemed to sort of quiet the unrest. I feel like people would be surprised that a huge tech company like Google would back down like that. Yeah, I mean, it does speak to

the difference between a conventional sort of contractor like Dow Chemical and these tech companies. I think there are sort of three main differences there. The first is that these companies are beloved. They have loyal followings and they don't want to do anything to upset that. idealistic. And in Silicon Valley, this is a big deal. Companies market themselves not just as profitable enterprises, but as sort of humanitarian projects. And anything that kind of compromises that hurts them. Right. Kind of world changers. Totally. And I think the biggest difference is that employees at these companies, especially the engineers who work on very specialized projects like AI development, have a ton of leverage because there are not very many of them. They make a ton of money and they're really hard to recruit. If a couple of chemical engineers walk out of Dow Chemical in protest, you know, they can probably find some more where that came from. But if hundreds or thousands of AI engineers

are upset about a project that Google is working on, that's a real threat to their business. It doesn't take many people staging a protest or threatening to walk out to really get the attention of senior leadership. So these three differences that you just laid out explain why Dow didn't back down despite images of dying children affected by their products, whereas Google did back down long, long before anything like that happened to it. Right. And I want to be clear, like what happened to Google was the exception, not the rule. This is not happening in any vast quantity across the tech industry. At Microsoft, for example, they came right out and said, you know, we appreciate the objections of this small group of employees.

You know, they can move elsewhere in the company if they want, but we're committed to seeing this contract through. And they've sort of defended themselves by saying, we have a patriotic duty to provide technology to the military. We think that we can sort of advocate for the ethical use of technologies like AI if we're at the table in these discussions. And they say that, you know, basically these technologies could be good. These could save lives on the battlefield if you have, you know, the army has said it wants to use HoloLens,

not just for combat training, but for things like, you know, measuring vital signs and monitoring soldiers for concussions and having some sort of hearing protection on people's ears. So those are the kinds of arguments they're making to justify staying in these contracts. So the argument here being that engineers who came to these companies to do good, having their technology used by the military does not necessarily mean not doing good. Yeah, I mean, that's one of the arguments that they're making is that basically if the U.S. military doesn't have

the best technology available on the market, our adversaries will, and we will be endangered as a result. I guess the question is, do the engineers buy that argument? Well, some of them probably do, but as we've seen, a number of them don't. as a result. I guess the question is, do the engineers buy that argument? Well, some of them probably do, but as we've seen,

a number of them don't. So, Kevin, we started by talking about this intersection of the moral and the financial as these tech companies navigate this question of should they work with the military. What's your understanding in Silicon Valley about how these companies are thinking about that? So I think a lot of these companies have really shifted their view on this in the last couple of years. A lot of Silicon Valley companies used to just think of themselves

as toolmakers, right? We build the stuff and we put it out into the world and people use it. And our responsibility basically stops at the moment of sale. To me, entrepreneurship is about creating change, not just creating companies. And now I think we're seeing that these companies are being held responsible by the public and by their own employees, not just for the tools they're building, but for how they're being used in the world. their own employees, not just for the tools they're building, but for how they're being used in the world. And this is not just about the military. At this hour, some 300 programmers are threatening to leave Microsoft unless the tech giant drops its contract with the U.S. Immigration and Customs Enforcement Agency, known as ICE. I mean, it's also companies are being scrutinized for working with ICE or working with law enforcement. An Amazon worker is pushing the company

to stop selling facial recognition technology to law enforcement. Social media companies are being held responsible for bad things that happen on their platforms. Facebook defending Alex Jones this morning, saying the platform will continue to allow the video blogger to run video on its site, even as it claims to crack down on fake news.

This is really happening across technology. And as technology embeds itself into kind of every part of our lives, we're seeing that these companies are having to make new kinds of decisions. It's not just a dollars and cents profit and loss calculation anymore. They also have to consider what might happen out in the world once they release these technologies. Which is another way of saying that the morality element of this is now playing a much bigger role than it has in the past, and perhaps is not beating out financial, but rivaling it. It's certainly part of the discussion now, in a way it hasn't been before. It's not just about how much money we'll

make or how many resources it will take. It's about what we're building and why we're building it and who we're selling it to. Kevin, thank you very much. Thank you for having me. We'll be right back. Here's what else you need to know today. The system is well beyond capacity and remains at a breaking point. On Tuesday, the Trump administration said that the number of migrant families crossing the Mexican border has broken records for the fourth time in five months and risks overwhelming government staff and facilities. Based on the experiences of men and women on the front line, this is clearly both a border security and humanitarian crisis. During a news conference, the Commissioner of Customs and

Border Protection, Kevin McAleenan, said that more than 76,000 migrants crossed the border without authorization in February, more than double the levels from the same period last year. Within these numbers, we are confronting challenging new smuggling cycles, patterns, and methods, so-called caravans, where 500 or more migrants form groups in Central America, mostly in Honduras, and travel together through Mexico to our southwest border. The record numbers are likely to complicate the already partisan debate between Republicans and Democrats over the president's plan to build a border wall by declaring a national emergency.

And for just the second time since the AIDS epidemic began, a patient appears to have been cured of HIV, the virus that causes AIDS, confirming that a cure is possible. The elimination of the virus in the patient, an anonymous man in London, came after a stem cell transplant, which was intended to treat his blood cancer, also removed the HIV virus. It was the same procedure that seemed to cure a different HIV patient 12 years ago. We waited 16 months before stopping in the post-transplant period just to make sure that the cancer was in remission, the patient was well, and that the measures we had of the HIV reservoir in the body

showed that there was very little virus there, if any at all. And at that point, we stopped the treatment. How a stem cell transplant could translate into a broader cure for HIV remains unclear, since the treatment used for the London patient can only be applied to those with both cancer and HIV. But several teams of scientists are working on gene therapies that could someday be applied to anyone with HIV. And so we're now 18 months in, and we're confident that this will be a long-term remission, but it's too early to say as to whether this is a cure or not. That's it for the day.

I'm Michael Barbaro. See you tomorrow.