Sea Control - Sea Control 377 - Intelligence Analysis and Analytic Reflection with Dr. Brian Holmes

Episode Date: September 4, 2022

Links1. “Analytic Reflection: Measuring the Attributes of Open and All-Source Intelligence,” by Brian Holmes, Homeland Security Today, June 8, 2020.2. “Feedback Driven Decisions and the Evolution of Intelligence Analysis in the United States," by Brian Holmes, The Strategy Bridge, January 23, 2020.3. “A Short History of Maritime Intelligence Integration,” by Dr. Brian Holmes, NMIO Technical Bulletin, March 2019, p. 4-5.4. Brian Holmes, Author at Hstoday.

Hey folks, it's Jared. Anna McNeil is back as host this week, and she's got Dr. Brian Holmes on to discuss, among other things, open source intelligence and intel analysis. This episode was edited and produced by Nathan Miller. I'd like to pause here to highlight our local chapters. Whether you're in South Korea, Egypt, Singapore, France, New York, India, or the Caribbean, chances are there's a SimSec local chapter near you. You can find a full listing of local chapters and contact information on our website at simsec.org, so if you're interested, please reach out and get involved. Finally, I want to take the opportunity to recommend our partners in the SimSec Podcast Network, The Bilge Pumps. You can find Alex, Jamie, Drack, and a pile of Iron Brew bottles wherever you download your podcasts. And with that, Kimber's Mountain. You're listening to Sea Control, hosted by the Center for International Maritime Security.

Hey folks, thanks for tuning back in to Sea Control. Today, we're talking with Dr. Brian Holmes about his recent articles on analysis. Dr. Holmes became a National Intelligence Officer for Emerging and Destructive Technologies in 2021 in support of the National Intelligence Council and Director of National Intelligence. As an NIO, he supports the policy community, Congress, and military decision makers through community-coordinated production of all-source strategic analyses on key topics of interest. Dr. Holmes has studied at the University of Delaware, earned his PhD at Clemson University, He served two years in a postdoctoral associate at the University of Minnesota's Department of Chemical Engineering and Material Science, where he researched organic semiconductors, and has worked with the U.S. Naval Research Laboratory in Washington, D.C., studying single-electron nanotransistors.

Dr. Holmes has also served as an all-source analyst and branch chief at the Defense Intelligence Agency's Counterproliferation Support Office and in the Directorate for Analysis Staff Operations Division. He also was a direct commission intelligence officer in the U.S. Navy Reserve, where he served from 2007 to 2015 in DIA's Directorate for Intelligence. Dr. Holmes worked for seven years at the National Intelligence University in Washington, D.C., where he was the associate dean of the School of Science and Technology and later became the dean. Dr. Holmes, thank you for joining us today and welcome. Is there anything you'd like to tell the audience about yourself and your background? Only really that my framing responses are based on a cross-disciplinary career that spans both science, academia, military, and intelligence. And in the spirit of kind of the maritime domain, I did want to communicate that although I'm not in the position anymore, when I was associated with the National Intelligence University, We did a really good study in which we looked at the short history of maritime intelligence integration through what is known as the National Maritime Intelligence Integration Office,

in which we looked at the kind of history of the National Intelligence University and the importance of both leadership in the maritime domain across everyone from board of visitors to staff and faculty to students, as well as integration with a number of different maritime based offices. And it was a lot of fun doing it, and we could certainly share that with your audience if they're interested further. Yes, sir. I'll make sure we put a link to that in our show notes. That sounds perfect. Thank you. And as a reminder to our listeners, all views expressed are our own and not representative of any institution with which we might be otherwise associated. Dr. Holmes, you've written extensively about all-source analysis. So can you please define that term and explain why it's important for decision makers to understand the difference between all-source analysis and open-source intelligence? Sure. So when I first kind of joined the intelligence community, this was all the way back in 2006, I joined as what is known as an all-source intelligence analyst.

And I work with the Defense Intelligence Team, the Accounting and Proliferation Support Office, for a number of years trying to learn what exactly all that is about to include not only definitions, but really tradecraft. which is just as important in terms of AllSource. And then more recently, last fall, I became very fortunate and became a national intelligence officer with the National Intelligence Council and had to rediscover AllSource intelligence analysis as a practitioner. In between those years, I studied it and then wrote a number of different articles on it as an academic living inside the U.S. intelligence community. So the manner by which I kind of view it and define it

and the way in which I talk about it in articles, like you pointed out, It's really predicated on those kind of two areas, both as a practitioner and as an academic whose job is to really evaluate it. So all-source intelligence, in this case is analysis, is really the fusion integration of multiple or more than one source or type of intelligence. And we do this to ultimately apply essentially logical argumentation based on our tradecraft. So we're attempting to form a judgment. We call them assessments sometimes. We use a process of reasoning. So reason can be everything from inductive, deductive, or abductive.

And we use the intelligence essentially as a form of evidence. So if you simply understand logical arbitration and the really basic components, you have evidence plus reasoning equals claim. That is a real core tenet of what we do and how we do it. Now, there's lots of different types of intelligence that seems to be growing depending on where you sit. There is, as you mentioned, OSINT, that's open source intelligence. There's SIGINT that signals intelligence. There's human, that's human intelligence. So I think spies, like in a classic sense.

There's also things like MAZINT, which is measurement and signals intelligence and geospatial intelligence as well, which kind of grew up out of the world that looked at imagery and mapping. So, for instance, in the U.S., we have the National Geospatial Intelligence Agency that looks at geospatial intelligence. Now, what's important maybe for this audience is that if we use intelligence as a form of evidence to build a claim, then what we know, particularly from our own tradecraft and kind of longstanding use of intelligence, is that every type of intelligence has both a strength and a weakness. And if you're trying to build a claim, one of the most important things about intelligence, what we do is actually really building confidence in what we're communicating. And so there's no real right or wrong to the kinds of intelligence that you integrate as a form of evidence. So you can use all the types that I included, or in some cases, which I see in my current job, you may only use two or three because that's the best information that you have at that point in time. But you use it knowing that you're really trying to bolster your argument and improve your confidence in that argument.

So open source, you mentioned, for instance, what's the difference between open source intelligence and open source information? Well, frankly, first of all, regardless of the type of intelligence or int that you're talking about, one of the things I don't hear people communicate very often is the fact that what makes something an int, I would argue, is based on the fact there's actually a collection requirement sitting somewhere in the background using a tool record that an analyst and a collection manager put into the system for formalization. So what's that mean? Well, if I'm an analyst and I say, I need to understand where the blue cars are across the world or some country, then technically you need to go to a collection manager and you need to formalize that and say, analysts need to understand where all the blue cars are around the world. And that goes into a tool of record.

And then in some way, shape and form, you use tools of intelligence to gain and collect information to figure out where those blue cars are. So one of the biggest differences that's not communicated very often about open source intelligence in relation to open source information is, at least in the U.S. intelligence community, there's a collection requirement communicating, I need information on X. And then they go out and they use open source information, which is generally publicly available information that you and I and everyone else can use. And they go and they use that information to try to respond to that collection requirement, essentially provide an answer to the requirement in terms of information they bring back through

the intelligence cycle. Now, strengths and weaknesses, we could go through this, you know, really for hours, which I won't. But one of the things you hear about, of course, is that open source information and the volume of information that's available in that source is so big and so growing exponentially that it's almost overwhelming if you think about it. To some extent, the same could be true about like signals intelligence. We have a great apparatus, certainly in the US, gaining signals intelligence into to a lesser extent imagery and even to a lesser extent there human intelligence the dilemma right now our challenge with open source information is first of all it's great in the sense that there's readily available information you can just reach

out and grab to a certain extent as long as you're doing it based on the legal authorities that are appropriate at the same time you have absolutely no idea in some cases where that information is coming from. And we see that, of course, in the news. And we constantly are trying to understand if something was seeded or planted in terms of the information or manipulated, or if there's propaganda, or in some cases, there's some sort of formal narrative that the original author was trying to propose, which there's nothing right or wrong about it. But they're trying to put that information out in the public source for what could be ulterior motives. So it's plentiful. But there's also a lot of information, you have no idea where it comes from, and what to do with

it, which is all the more reason why you want to use that information as a complement to the other classic types of intel like SIGINT and UNIT, which have strength allegiances too. So the more information you might have in which you're answering a key intelligence question, which is this classic kind of thing that we do in intelligence and all-source analysis, is to try to find complementary information through other sources in which you can ultimately bolster your confidence in your judgment or assessment. Now, in a perfect world, you get intel from all of the different types of int, but as I mentioned before, you don't always get that.

But really bolstering that judgment or assessment is your ultimate goal, and you want to do it using multiple sources of information. What a fantastic roll-up of all-source intelligence as a confidence builder for the decision maker. So in an era of data-centric security and big data, and you talked about volume briefly, how can analysis add value above and beyond your basic data integrity checks? Yeah, so I would argue it can, and it's actually, it's probably more important now than ever before. I know there's a lot of different questions about this. You know, it's funny. If I went back five years and asked an all-search intelligence analyst if they had a problem with big data and went five years before that and five years before that and five years before that and kept going,

every single analyst would give you the same answer. They'd say, yes. We have a huge problem with all the information that we have, and it's growing, it's growing, it's growing, it's growing. At the same time, there's plenty in the community that say, well, there's so much information, particularly in open source, that everything we need is in the open source. But I would argue you still have these challenges in which you have strengths and weaknesses in information. And the nice thing is about certain kinds of intelligences, ultimately intelligence is used to penetrate and get behind those kind of secret doors and barriers that different adversaries or competitors are attempting to hide information behind, to gain more confidence in that information. One of the things I've seen in my current job is it really comes back to that confidence issue.

So really savvy customers sometimes realize they're getting information from a lot of different places. And sometimes they go to the intelligence community and say, OK, here's the issue that we see. Here's our questions. Can you answer by using all source information? Oh, by the way, at least in our case, what does the rest of the community say about that? This really comes down to credibility as well as accuracy, as well as confidence in the information. So when we use these different types of intelligence and then we apply our tradecraft, at least currently anyways, the customer is much more satisfied with the answer that we provide. And sometimes it's not perfect, but there is a certain level of confidence that they have in that answer and how it was crafted and used. And certainly I would argue in the report, the CICI report on the elections in 2016, that is in fact one of the issues that I write about in one of those articles for Homeland Security Day, in which I would argue the CICI is communicating these very things in that report that came out from the commission.

they're communicating we applied our tradecraft in a way in a manner that was responsible they communicated that we use all sources of information in the way in which we provided and created a judgment so they actually commented you used both open source and other types of information and all this led to far more confidence in what we're doing and how we reached our conclusions so i think this still stands and in fact it's growing because there's is almost so much information that you can't even read through it. And so you need another group, in this case, intelligence officers, to provide something that has a little bit more confidence and, frankly, a little bit more logical interpretation behind it that follows those

judgments. Absolutely. You had written of the intelligence officers that the ethos or the character appeal of a finished intelligence product still serves as an important type of appeal in finished intelligence, and it has to be maintained to ethically persuade an audience. Your articles are written about analysis as a trade, but many of our listeners may be more accustomed to being intelligence consumers. So from your perspective, what role should a good consumer of intelligence play in the intelligence cycle? Yes, so there is, you know, in a perfect world, it's really two or three, I would argue. So in a different article I wrote, probably my, probably article I loved the most was an article written for War of the Rocks. And it started and it was called Pathos for Art Now.

And what I did for that article was I took the tradecraft standards that the U.S. has under what's called the Telus Community Directive 203, and I looked at all those different tradecraft standards. And then on the left, and basically on a whiteboard, and I wrote them all down. And then on the right, I wrote what are known as Aristotle's Three Appeals, the Jesus, Athos, Logos, and Prophets. And then I drew lines, some straight and some dotted, and then in some cases none, between what was on the left in the tradecraft standards and in Aristotle's three appeals. So what are the three appeals and why does it matter for this conversation? Well, ethos is about credibility. Pathos is really about emotion. And then logos is about logic. And in Aristotle's case, if you believe what you read, he was a huge fan of logos and logical argumentation. And what's unique about all sorts of intelligence in our tradecraft standards is that analytic tradecraft standard six is about logical argumentation.

It's the core tenet of what we do and how we essentially reach our conclusions based on a certain process. In terms of customers, I mean, if customers can be savvy enough to understand how these three appeals work, and how, in fact, intelligence community and all sorts of analysts do what they do, if only based on logical argumentation, then i would argue one of the great things that can happen is that they they value in particular the fact that we're applying certain level of logic logic identification in terms of our analysis and attempting to be very inclusive and use lots of different experts both inside the community and outside to gain far more credibility in what we're doing as well as different kinds of information perspective and apply that to our process and understand exactly what happens when you're influenced by a lot of emotion-driven information, whether it's visual or descriptive

adjectives and other kinds of things, and how what we do in and of itself can be very different in process and outcome than what others do when they go through different types of information as a service. So that's one. So value logical argumentation. Second, I wrote another article, Zopfiebeck, which you mentioned, I think, earlier. it's not feedback driven decisions so certainly in the article and i would argue that even more today is that as you go in terms of intelligence from what we would call tactical to operational all the way up to strategic intelligence because strategic being far more estimative in nature so you're thinking further out you're thinking consequences what we know is the intelligence

cycle becomes less and less of a real cycle so the intelligence cycle in like textbooks and images looks like a circle and there's all these different components to it it's really not a circle um i would argue and as you reach up all the way up to a strategic it's more like a c with a kind of fuzzy blob in between each end point of the c and the reason it's not a full circle is because you don't really complete that loop because of the feedback issue so if there was a perfect cycle you would gain continuous consistent and standardized feedback and that would integrate right back into the cycle in a way that's it's very iterative over time that we can use we can learn from and then we can improve upon but right now that's very tricky and there's very good reasons frankly

sometimes we don't gain feedback our customers depending on the level and what they're doing don't always want to communicate everything that they're seeing and doing from particularly from policy standpoint in many cases to maintain a certain level of secrecy right until it becomes more public in a very deliberative process so in this sense the more feedback the customer can provide the better it can be knowing that it's not a perfect cycle and i think certainly at the national intelligence council i've learned what feedback looks like and how it can be incorporated fairly well into the the overall process so sometimes feedback is just hey i wish i could get more information on this and that could turn all the way into you know you give them a product

they provide feedback on their product and then maybe everything from substance to tradecraft to something something else depending on how savvy the customer is but working really hard to gain that feedback which is not always an easy thing to do is very important and at the same time as the customer trying to provide substantive feedback in and of itself is one of the best things you could do to really improve what that analyst is doing and how they're really trying to cater that information in an appropriate way. So in addition to building feedback into that consumer-to-producer relationship or into that fuzzy part of the intelligence cycle, how can understanding the intelligence process

and the needs of the people who are involved in producing intelligence help leaders improve the lives of the analysts supporting them? Yeah, so I think, you know, what was interesting when I came into the community, and I think I write a little bit about this in my Home On Security Day articles, and I use this term all the time, you know, the business of intelligence is about bad people doing bad things. And unfortunately, business is good. The military understands this in a way, actually, it's even probably more profound than the home screen does, particularly if you're kind of a strategic intelligence officer, you're not really on the ground all the time, the way the military can be. but the fact is when you come into this world you are suddenly handed these keys to lots of unique information which is great and it's unique and it's intimidating at the same time nine times out of ten you're going to be on a portfolio that is frankly could be quite shocking depending on

what you're used to or not i came in as a chemical warfare analyst and i think i wrote about it within a week i remember seeing all kinds of things about chemical warfare which is not the nicest or funnest thing to see so your intelligence analysts in particular as well as collectors and support officers they're thrown into this world right off the bat and they're expected to do their job and to some extent there's not a lot of frankly information or support sometimes in the back end in terms of everything from what they're seeing and doing and how that affects them to all the way through like how has this affected them over time in an iterative way and how are they're doing with it. But I think to some extent, suppliers need to be empathetic with their teams

in whatever shape and form and constantly checking in with them to say, how are you doing? What's the information look like? Do you have any other issues or concerns? And trying to provide that kind of one-on-one level support. The one great thing about the intelligence community in particular that certainly can be learned across different professions is that they are really big about development and training. And so there are resources available to some extent that can support a lot of your teammates in a way that's very helpful. But I think to some extent, it's really on the supervisors and the leaders to more proactively connect essentially the resources and the training with some of the things that intelligence officers are facing. Additionally,

and this is the last point, treating people like humans is kind of an obvious assumption. Sometimes we kind of treat, whether you're military or whether you're an intelligence officer, you kind of treat them like machines to some extent. In an intelligence production office, it's, here's an issue, here's a key intelligence question, go work on it, do your research, do smart research, now get this product out, which at the end of the day is your job. But the fact is, everyone's got a life, lives to live, they have families, they have kids. and all these things come up no matter how dire the situation and how difficult or timely the information is needed based on your day job and so also making sure that you're being empathetic to what their personal life looks like as well as their professional life is probably as crucial as anything else you can do again a lot of times it comes down to checking in with them being

empathetic trying to stay flexible and building and connecting support if their support either or formal, particularly formal, where it exists or if there's not, then trying to be as good as you can with them from an employee standpoint and trying to figure out if there's ways to bolster some of that support. You've got the well-trained brain who knows their portfolio and you're trying to help support them. As a large part of the future workforce, either for the intelligence community or the military, hopes to work remotely, either full or part-time, is there any hope at all for giving our analysts

more time at home? and maybe a better work-life balance as a result? Yeah, I think so. I think there's some sort of middle ground. And to some extent, it really does depend on the portfolio. You know, there are certain things that simply are secret for very good reasons and need to be secret. And the only place to do that, at least for the moment,

frankly, is behind the loop, all the skip doors, in which, you know, sometimes you don't quite see the light of day. But I think the biggest trend I've seen is, you know, I think a lot of managers, certainly in the environments that I work in and a lot of the people I collaborate with, are far more open and flexible to what you would consider like a mixed kind of work environment. So on the one hand, certainly in intelligence, you could argue, yes, you need access to certain secret information.

You need to work behind the scenes and get on the systems. At the same time, there really is other kinds of information in ways, for instance, you can do outreach, you can do unique research in which you can use open source information or maybe something that's a secure database to try to do some of these kind of things at home in an environment that might be a little bit more flexible based on the circumstances. I'm seeing this blend a lot. And I think culturally speaking,

I think supervisors, that's the biggest thing. I mean, culturally, they're more, frankly, open to this. So if I went to my supervisor and said, hey, tomorrow I want to do a WebEx and really speak to different customers and learn a lot from them, then they would say yeah that's what the new job duties go ahead and do it or you know tomorrow i really want to do a lot of like scientific research because all that stuff exists in a different place

and here's what i'm doing you're very clear and transparent about what you're doing i think many supervisors are very open to that at this point and they kind of script it in and you have certain expectations of course that you have to meet but i think the flexibility between the two worlds is far better now than what it was before now whether or not someone's going to be universally at home versus the other, again, I would argue that really depends. But certainly the door has cracked quite a bit. And I think the technology is coming in a way that they may be able to open up some more of the door as well. Those are fantastic examples too. And I hope that they'll reach some of our listeners and they'll have some good ideas for ways that they can balance out their lives.

Sir, I think my next question you already answered, but I'll repeat it just in case you want to add anything? Has the time since these articles were published led you to any changes in opinion? Or since many offices have returned to work, do you feel that the intelligence community has made the most of the opportunities that increased telework has given? Well, I'll answer just a couple different ways that really kind of reinforces what I mentioned before. So what's interesting, when I wrote these articles, I was an academic and better teams have used intelligence. So my perspective was I'm going to write a lot about our analysis. And here's what I see is some sort of quasi-internal external researcher essentially when i kind of looked at these things now i'm doing all sorts

intelligence again as an international intelligence officer so in one sense my again my perspective based on the importance of all sorts in and of itself is really bolstered at the same time you know as you mentioned before the flexibility in terms of our work life balance everything from our work life balance to be able to to do some more open source kind of research and outreach is far more available and normalized at least culturally now than i've ever seen before and i see this even in a place like the nick where the nick tends to be an environment which you do have to do a lot of things in a classic system but there's plenty of flexibility across across the board to make sure you're doing things across not only government but industry partners

gaining more information and really getting out there as well as doing telework and everything in between so i think the trend is there i'm fascinated to see where the trend goes in certain cases i love to see some more innovation behind it and what people can do but to some extent that really has to do with some of the technology behind the scenes and how secure some of that technology can really be anywhere that you'd like the audience to follow you on social media i am on linkedin i'm probably not as active as i used to be in terms of producing more kind of unique public analysis and reports like we did before or research studies, but I still really look pretty hard at a lot of different things, particularly against my portfolio,

which is emerging tech. You can find me there if you're curious about some of the things that I care about and post. Thank you, Dr. Holmes, for not only being a wealth of expertise for us to speak with today and to have on the podcast, but also for being a leader in the community and advocating for your folks. Thank you. To our listeners, thanks for tuning back into Seat Control. © transcript Emily Beynon Thank you for watching.