Screaming in the Cloud - Cloud Compliance and the Ethics of AI with Levi McCormick

Episode Date: August 24, 2023

Levi McCormick, Cloud Architect at Jamf, joins Corey on Screaming in the Cloud to discuss his work modernizing baseline cloud infrastructure and his experience being on the compliance side of cloud engineering. Levi explains how he works to ensure the different departments he collaborates with are all on the same page so that different definitions don’t end up in miscommunications, and why he feels a sandbox environment is an important tool that leads to a successful production environment. Levi and Corey also explore the ethics behind the latest generative AI craze. About LeviLevi is an automation engineer, with a focus on scalable infrastructure and rapid development. He leverages deep understanding of DevOps culture and cloud technologies to build platforms that scale to millions of users. His passion lies in helping others learn to cloud better.Links Referenced:Jamf: https://www.jamf.com/Twitter: https://twitter.com/levi_mccormickLinkedIn: https://www.linkedin.com/in/levimccormick/

Hello, and welcome to Screaming in the Cloud, with your host, Chief Cloud Economist at the Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. Welcome to Screaming in the Cloud. I'm Corey Quinn.

A longtime friend and person who's been a while since he's been on the show, Levi McCormick has been promoted or punished for his sins, depending upon how you want to slice that. And he is now the director of cloud engineering at Jamf. Levi, welcome back. Thanks, Greg McCory. I have to imagine internally you put that very pronounced F everywhere, and sometimes where it doesn't belong, like your IAMP policies and whatnot. It is fun to see how people like to interpret how to pronounce our name.

So it's been a while. What were you doing before? And how did you wind up stumbling your way into your current role? When we last spoke, I was a cloud architect here, diving into just our general practices and trying to shore up some of them. In between, I did a short stint as a director of FedRAMP. We are pursuing some certifications in that area, and I led kind of the engineering side of the compliance journey. That sounds fairly close to hell on earth from my particular point of view, just because I've dealt in the compliance side of cloud engineering before, and it sounds super interesting from a technical level until you realize just how much of it revolves around checking the boxes. And at least in the era I did it, explaining things

to auditors that I kind of didn't feel I should have to explain to an auditor, but there you have it. Has the state of that world improved since roughly 2015? I wouldn't say it has improved. While doing this, it did feel like I drove a time machine to work, you know, where we're certifying VMs rather than container-based architectures. There was a lot of education that had to happen from us to auditors. But once they understood what we were trying to do, I think they were kind of on board. But yeah, it was a journey. So one of the things that you do,

in fact, the first line in your bio talking about it, is you modernize baseline cloud infrastructure provisioning. That means an awful lot of things depending upon who it is that's answering the question. What does that look like for you? For what we're doing right now, we're trying to take what was a cobbled together, part-time project for one engineer. We're trying to modernize that, turn it into as much self-service as we can.

There's a lot of steps that happen along the way, like a new workload needs to be spun up. They decide if they need a new AWS account or not. We pivot around like, what does the access profile look like? Who needs to have access to it? Which things does it need to connect to? And then you look at the billing side, compliance side, and you just say, you know, who needs to be informed about these things? We apply tags to the accounts. We start looking at lower level tagging, depending on if it's a shared workload account or if it's a completely dedicated account. And we're trying to wrap all of that in automation so that it can be as click button as possible.

Historically, I've found that when companies try to do this, the first few attempts at it don't often go super well. We'll be polite and say their first attempts resemble something artisanal and handcrafted, which might not be ideal for this. And then, in many cases, the overreaction becomes something that is very top-down dictatorial, almost, is the way I would frame that. And the problem people learn then is that, oh, everyone is going to route around us because they don't want to deal with us at all. That doesn't quite seem like your jam from what I know of you and your approach to things. How do you wind up keeping the guardrails up without driving people to shadow IT their way around you?

I always want to keep in mind that even if it's not an option, I want to at least pretend like a given team could not use our service. I try to bring a service mentality to it. So we're talking accounts as a service. And then I just think about all of the things that they would have to solve if they didn't go through us. Are they managing their finances? Imagine they had to go in and negotiate some kind of pricing deal on their own. All of these things that come with being part of our organization, being part of our service offering. And then just making sure those things are always easier than doing it on their

own. How diverse would you say that the workloads are that are in your organization? I found that in many cases, you'll have a SaaS-style company where there's one primary workload that is usually bearing the name of the company, and that's the thing that they provide to everyone. And then you have the enterprise side of the world where they have 1,500 or 2,000 distinct application teams working on different things. And the only thing they really have in common is, well, that all gets billed to the same company eventually. They are fairly diverse in how they're currently created. We've gone through a few acquisitions. We pulled a bunch of those into our ecosystem, if you will. So not everything has been completely modernized or brought over to,

you know, standards, if you will. If such a thing even exists in companies, you know, you may pretend that they do, but you're probably lying to yourself. Right. But, you know, there know, there are varying platforms. We've got a whole laundry list of languages that are being used. We've got some containerized, some VM-based, some serverless workloads. So it's all over the place.

But you nailed it. Like, you know, the majority of our footprint lives in maybe a handful of, you know, SaaS offerings. Right. It's sort of a fun challenge when you start taking a looser approach to these things, because someone gets back from reInvent, like, well, I went to the keynote, and now I have my new shopping list of things I'm going to wind up deploying. And that never goes well, having been that person in a previous life. Yeah. And you don't want to apply too strict of governance over these things, right? You want people to be able to play. You want them to be inspired and start looking at what's something that's going to move the needle in terms of our cloud architecture or product offerings or whatever we have.

So we have sandbox accounts that are pretty much wide open. We've got some light governance over those, more so for billing than anything. And all of our internal tooling is available. If you're using containers or whatever, all of that stuff is in those sandbox accounts. And that's where our service offering comes into play. Sandbox is still an account that we try to vend if you will out of our service so people should be building in your sandbox environments just like they are in your production as much as possible you know it's a place where tools can get the the tires kicked and and smooth out bugs before you actually get into roadmap impacting problems.

One of the fun challenges you have is, as you said, the financial aspect of this. When you've got a couple of workloads that drive most things, you can reason about them fairly intelligently. But trying to predict the future, especially when you're dealing with multi-year contract agreements with large cloud providers, becomes a little bit of a guessing game. Like, okay, well, how much are we going to spend on generative AI over the next three years? The problem with that is that if you listen to an awful lot of talking heads or executive types, like, oh, yeah, if we're spending $100 million a year, we're going to add another 50 on top of that just in terms of generative AI.

And it's like, press X to doubt, just because it's... I appreciate that you're excited about these things and want to play with them, but let's make sure that there's some there there before signing contracts that are painful to alter. Yeah, it's a real struggle. And we have all of these new initiatives, things people are excited for. Meanwhile, we're bringing old architecture into a new platform, if you will, or a new footprint. So we have to constantly measure those against each other. We have a very active conversation with finance and with leadership every month or even weekly, depending on the type of project and where that spend is coming from. One of the hard parts has always been, I think, trying to get people on the finance side of the world, the engineering side of the world, and the folks who are trying to predict what the

business is going to do next, all speaking the same language. It just feels like it's too easy to wind up talking past each other if you're not careful. Yeah, it's really hard. Recently taking over the FinOps practice, it's been really important for me for us to align on what our words mean, right? What do these definitions mean? How do we come to common consensus so that eventually the communication gets faster, but we can't talk past each other. We have to know what our words mean. We have to know what each person cares about in this conversation or what does their end goal look like? What do they want out of the conversation? So that's taken a significant amount of time. One of the problems I have is with the term thin ops as a whole, ignoring the fact entirely that

it was an existing term of art within finance for decades. Great. We're just going to sidestep past that whole mess. The problem you'll see is that it just seems like that it means something different to almost everyone who hears it. And it's sort of become a marketing term more so than it has an actual description of what people are doing. Just because some companies will have a quote unquote fin ops team that is primarily going to be run by financial analysts. And others, well, we have one of those lying around, but it's mostly an engineering effort on our part. And I've seen three or four different expressions as far as team composition goes, and I'm not convinced any of them are right. But again, it's easy for me to sit here and say, oh, that's wrong, without having an environment of my own to run.

I just tend to look at what my clients do. And, well, I've seen a lot of things, and they work poorly in different ways is not uplifting and helpful. Yeah, I try not to get too hung up on what it's called. This is the name that a lot of people inside the company have rallied around. And as long as people are interested in saving money, cool, we'll call it FinOps. I mean, DevOps is the same thing, right? In some companies, you're just a sysadmin with a higher pay, and in some companies, you're building extensive cloud architecture and pipelines.

Honestly, for the whole DevOps side of the world I maintain, we're all systems administrators. The tools have changed, the methodologies have changed, the processes have changed, but the responsibility of keep the site up generally has not. But if you call yourself a sysadmin, you're just asking to please pay me less money in my next job.

No thanks. Yeah, where's the exchange server for me to click on, right? That's the... If you call yourself a sysadmin... God, you're sending me back into twitching catatonia from my early days. Exactly. So you've been paying attention to this whole generative AI hype monster. And I want to be clear, I say this as someone who finds the technology super neat and i'm optimistic about it but holy

god it feels like people have just lost all sense if that's you my apologies in advance but i'm still going to maintain the point i've played with all the various you know toys out there i'm very curious you know i i think it's really fun to play with them, but to make your entire business pivot on a dime and pursue it, it just seems ridiculous to me. I hate that the cryptocurrency space has pivoted so hard into it. All the people that used to be shilling coins

are now out there trying to cobble together a couple of API calls and turn it into an AI, right? It feels like it's just a hype cycle that people are more okay with being a part of. Andy Jassy on the earnings call a couple weeks ago saying that every Amazon team is working with generative AI. That's not great. That's terrifying. I've been playing with the toys as well, and I've asked it things like, oh, spit out an IAM policy for me. Or, oh, great, what can I do to optimize my AWS bill? And it winds up spitting out things that sound highly plausible, but they're also just flat out wrong.

And it feels like a lot of these spaces, it's not coming up with a plausible answer. That's the hard part. It's coming up with the one that is correct. And that's what our jobs are built around. I've been trying to explain to a lot of people how if you only have surface knowledge of the thing that it's telling you, it probably seems really accurate. But when you have deep knowledge in the topic that you're interacting with this thing, you're going to see all of the errors. I've been using GitHub's Copilot since the launch. It was in one of the previews, and I love it. It speeds up my development significantly. But there have been moments where IAM policies are a great example.

I had to crank out a Lambda functions policy, and it was just frankly wrong in a lot of places. It didn't quite imagine new AWS services, but it was really close. The API actions didn't exist. It just flat out didn't exist. I love that. I've had some magic happen early on where it could intelligently query things against the AWS pricing API, but then asked it the same thing a month later, and it gave me something completely ridiculous. It's not deterministic,

which is part of the entire problem with it too. But it's also, it can help incredibly in some weird ways I didn't see coming. But it can also cause you to spend more time chasing that thing than just doing it yourself the first time. I've found a great way to help me write blog posts with it. I tell it to write a blog post about a topic and give it some bullet points and say, write in my voice. And everything

it says, I take issue with. So then I just copy that into a text editor and then mansplain, correct the robot for 20 minutes. And now I've got a serviceable first draft. And how much time did you save? Right? It is fun. It does help because that's better for me, at least, than staring at an empty page of what am I going to write? It gets me past the writer's block problem. Oh, that's a great point. Yeah, just to get the ball rolling, right? It's easier to correct something that's wrong, and you almost are spite-driven at that point, right? Like, let me show this AI how wrong it was, and I'll write the perfect blog post. It feels like the company's jumping on this. If you really dig into what

we're talking about, it seems like they're all very excited about the possibility of, we don't have to talk to customers anymore because the robots will all do that. And I don't think that's going to go the way you want to. We just have this minor hallucination problem. Yeah, that means it lies and tries to book customers to hotel destinations that don't exist. Think about this a little more. The failure mode here is just massive. It's scary. Yeah.

Like, without some kind of review process, I wouldn't ship that straight to my customers, right? I wouldn't put that in front of my customer and say, like, this is, I'm going to take this generative output and put it right in front of them. That scares me. I think as we get deeper into it, you know, maybe we'll see, I don't know, maybe we'll put some filters or review process or maybe it'll get better. I mean, who was it that said, you know, this is the worst it's ever going to be. It will only get better.

Well, the counter argument to that is it will get far worse when we start putting this in charge of you know safety critical systems which i'm sure is just a matter of time because some of these boosters are just very very convincing it just think of how could this possibly go the worst it's not good yeah well i mean we're talking impact versus quality right the the quality will only ever get better. But, you know, if we run before we walk, the impact can definitely get wider. From where I sit, I want to see this really excel within bounded problem spaces. The one I keep waiting for is the AWS bill, because it's a vast space, yes, and it's complicated as all hell, but it is bounded. There are a finite,

though large, number of things you can see in an AWS bill. And there are recommendations you can make based on top of that. But everything I've seen that plays in this space gets way overconfident, far too quickly, misses a bunch of very obvious lines of inquiry, I'm skeptical. Then you pass that off to unbounded problem spaces like human creativity, and that just turns into an absolute disaster. So much of what I've been doing lately has been hamstrung by people rushing to put in safeguards to make sure it doesn't accidentally say something horrible, that it's stripped out a lot of the fun and the whimsy and the sarcasm and the approach of i at one point i could bully a number of these things into ranking u.s presidents by

absorbency that's getting harder to do now because nope that's not respectful and i'm not gonna do it is basically where it's where it draws the line one thing that i always struggle with is like how much of the models are trained on you on intellectual property or when you distill it down, pure human suffering, right? Like this is somebody's art. They've worked hard. They've suffered for it. They put it out there in the world, and now it's just been pulled in and adopted by this tool. How many of the examples of give me art in the style of right and you just see hundreds and hundreds of

pieces that i mean frankly are eerily identical to the style even down to the signature in some cases yeah exactly you know and i think that we can't lose sight of that right what like these tools are fun and you know they're they're fun to play with. It's really interesting to explore what's possible, but we can't lose sight of the fact that there are ultimately people behind these things. monitoring, and security, protecting the entire application stack from build to runtime. Scalable across clusters and multi-cloud environments, Panoptica secures containers, serverless APIs, and Kubernetes with a unified view, producing operational complexity and promoting collaboration by integrating with commonly used developer, SRE, and SecOps tools. Panoptica ensures compliance with regulatory mandates and CIS benchmarks for best practice conformity. Privacy teams can monitor API traffic and identify

sensitive data while identifying open source components vulnerable to attacks that require patching. Proactively addressing security issues with Panoptica allows businesses to focus on mitigating critical risks and protecting their interests. Learn more with Panoptica allows businesses to focus on mitigating critical risks and protecting their interests. Learn more about Panoptica today at panoptica.app. I think it matters on some level what the medium is. When I'm writing, I will still use turns of phrase from time to time that I first encountered when I was reading things in the 1990s. And that phrase stuck with me and became part of my lexicon. And I don't remember where I originally encountered some of these things. I just know I use those phrases an awful lot. And that has become part and parcel

of who and what I am, which is also why I have no problem telling it to write a blog post in the style of Corey Quinn and then ripping all part of that out and put anything that's left in there. Cool. I'm plagiarizing the thing that plagiarized from me. And I find that to be one of those ethically just moments there. But written word is one thing, depending on what exactly it's taking from you. But visual style for art, that's something else entirely. There's a real ethical issue here. These things can absorb far much more information than you ever could in your entire lifetime. So you can only quote unquote copy, borrow, steal from a handful of other people in your entire life. right? Whereas this thing could do hundreds or thousands of people per minute. I think that's where the calculus needs to be, right? How many people can we impact with this thing? This is also nothing new. Where originally in the olden times, great, copyright wasn't really a thing because writing a book was a massive, massive undertaking. That was something that

you'd have to do by hand. And then, oh, you want a copy of the book. You'd have a scribe go and copy the thing. Well, then suddenly the printing press came along and okay, that changes things a bit. And then we continue to evolve there to digital distribution, where suddenly it's just bits on a disc that I can wind up throwing halfway around the internet. And when the marginal cost of copying something becomes effectively zero, what does that change? And now we're seeing, I think, another iteration in that ongoing question. It's a weird world. And I don't know that we have the framework in place even now to think about that properly. Because every time we start to get a handle on it, off we go again.

It feels like if they were to be invented today, libraries would absolutely not be considered legal. And yet, here we are. Yeah, it's a great point. Humans just do not have the ethical framework in place for a lot of these things. You know, we saw it even with the days of Napster, right? It's just, like you said, it's another iteration on the same core problem. I don't know how to solve it. I'm not a philosopher, right?

Oh, yeah. Back in the Napster days, I was on that a fair bit in high school and college because I was broke. And, oh, I wanted to listen to this song. Well, it came on an album with no other good songs on it

because one hit wonders were kind of my jam. And that album cost 15, 20 bucks or I could grab the thing for free. There was no reasonable way to consume. Then they started selling individual tracks for 99 cents. And I gorged myself for years on that stuff. And now it feels like streaming has taken over the world to the point where the only people who really lose on this are the artists themselves. And I don't love that outcome.

How do we have a better tomorrow for all of this? And we're a bit off topic from, you know, cloud management, but still, this is the sort of thing I think about when everything's running smoothly in a cloud environment. It's hard to get people to make good decisions when they're so close to the edge. And I think about when I was, you know, college age, scraping by on minimum wage or barely above minimum wage, you know, it was hard to convince me that, oh yeah, you shouldn't download an MP3 of that song. You should go buy the disc or whatever. It was really hard to make that argument when my decision was buy an album or figure out where I'm going to, you know, get my lunch. So I think as now that I'm in a much different place in my life, you know, these decisions are a lot easier to make in an ethical way because that doesn't impact my, my livelihood nearly as much.

And I think that is where solutions will probably come out of, right? The more people doing better, the easier it is for them to make good decisions. I sure hope you're right. But something I found is that, okay, we've made it easier for people to make good decisions. Like, nope, you've just made it easier for me to scale a bunch of terrible ones. I can make 300,000 more terrible decisions before breakfast time now. Thanks.

And no, that's not what I did that for. Yet here we are. Have you been tracking lately what's been going on with the HashiCorp license change? A little bit. We obviously use Terraform in the company and a couple other Hashi products, and it was kind of a wildfire of, you know, how does this impact us? We dove in and we realized that it doesn't, but it is concerning. Yeah, you're not effectively wrapping Terraform and using that as the basis

for how you do MDM across your customer fleets. Yeah, we're not deploying customers written Terraform into their environments or something kind of wild like that. Yeah, it doesn't impact us. But it is concerning to watch a company pivot from an open source community-based project to, you know, you can't do that anymore. It doesn't impact a lot of people who use it day to day, but I'm really worried about just the goodwill that they've lit on fire. is that their entire write-up on this was so vague that it was, there is no way to get an actual piece of, is it aimed at us or is it not, without a very deep analysis and hope that when it comes to court, you're going to have the same analysis that is sympathetic. It's what is considered to

be a competitor. At least historically, it was pretty obvious. Some of these databases, okay, great. Am I wrapping their database technology and then selling it as a service? No, I'm pretty good. But with HashiCorp, what they do is so vast in a few key areas that no one has the level of certainty. I was pretty freaking certain that I'm not shipping MongoDB with my own wrapper around it. But am I shipping something that looks like Terraform if I'm managing shipping MongoDB with my own wrapper around it, but am I shipping something that looks like Terraform? If I'm doing, if I'm managing someone's environment for them, I don't know.

Everything's thrown into question and you're right. It's the goodwill that currently is being set on fire. Yeah. I think people had a impression of Hashi that they were one of the good guys, you know, the quote unquote good guys in the, in the space, right?

Mitchell Hashimoto is out there as a very prominent coder. He's an engineer at heart. He's in the community, pretty influential on Twitter. And I think people saw them as not one of the, the big faceless corporations to, so to see moves like this happen, it,

I think it shook a lot of people's opinions of them and scared them. Oh yeah. They've always been the good guys in this context. Mitchell Armand were fantastic folks. I'm sure they still are. I don't know. This is necessarily even coming from them. It's market forces.

What are investors demanding? They see everyone is using terraform. How does that compare to HashiCorp's market value? This is one of the inherent problems of being direct of the end stages of capitalism, where it's, okay, we're delivering an awful lot of value. How do we capture ever more of it and growing massively? I don't know.

I don't know what the answer is, but I don't think anyone's thrilled with this outcome because it's, let's be clear, it is not going to meaningfully juice their numbers at all. They're going to be setting up a lot of ill will against them in the industry, but I don't see the upside for them. I really don't. I haven't really done any of the analysis or looked for it, I should say. Have you seen anything about what this might actually impact? Any providers or anything? Because you're right. What kind of numbers are we actually talking about here?

Right. There are a few folks that have done things around this that people have named for me. Spacelift being one example, Pulumi being another. And both of them are saying, nope, this doesn't impact us because of X, Y, and Z. Yeah, whether it does or doesn't, they're not going to sit there and say, well, I guess we don't have a company anymore. Oh, well, shut the whole thing down and just give their customers over to HashiCorp. Their own customers would be incensed if that happened and would not go to HashiCorp if that were to be the outcome. I think on some level, they're setting the stage for the next evolution in what it takes to manage large-scale cloud environments effectively.

I think basically every customer I've ever dealt with on my side has been a Terraform shop. I finally decided to start learning a lot of the ins and outs of it myself a few weeks ago. And, well, it feels like I should have just waited a couple more weeks, and then it would have become irrelevant. Awesome.

Which is a bit histrionic, but still, this is going to plant seeds for people to start meaningfully competing, I hope. Yeah, I hope so too. I've always awaited releases of Terraform Cloud with great anticipation. I generally don't like managing my Terraform backends. I don't like managing the state file, so every time Terraform Cloud has some kind of release or something, I'm looking at it because I'm excited. Oh, finally, maybe this is the time I get to hand it off, right? Maybe I start to use their product.

And it has never been a really compelling answer to the problems that I have. And I've always said, like, the cloud journey would be Google's if they just released a managed Terraform service. And this would be one way for them to prevent that from happening. Because Google doesn't even have an infrastructure-as-code competitor. Not really. I mean, they have their plans or their projects or whatever their infrastructure-as-code language was. Isn't that what Stackdriver was supposed to be? What happened with that? It's been so long. No, that's a logging solution. their projects or whatever their infrastructure as code language was. Isn't that what Stackdriver was supposed to be?

What happened with that? It's been so long. No, that's a logging solution. That's the thing. It all runs together. It was their operations suite that was formerly Stackdriver. Yeah, now that does include some aspects. Yeah, you're right.

It's still hanging out in the observability space. Yeah, this is the problem. All this stuff conflates, and companies are terrible at naming, and Google likes to deprecate things constantly. But there is no real competitor. CloudFormation, please get serious. Hey, you're talking to a member of the CloudFormation support group here, so I'm still a huge fan. Emotional support group, more like it, it seems these days.

Ooh, good, it got four loops recently we've been asking for basically that to make them a lot less wordy only for what 10 years yeah i mean my argument is that i'm operating at you know the account level right i need to deploy to 250 300 500 accounts show me how to do that with terraform that doesn't you know stab your eyes out with a fork it can be done but it requires an awful lot of setting things up first. Exactly. That's sort of a problem. Like, oh, yeah, once you have the first 500 going, the rest are just like butter.

But that's a big step. Step one is massive, and then step two becomes easy. Yeah. No, thank you. I'm going to stick with my stack sets. Thank you. I'm going to stick with my stack sets. Thank you. I really want to thank you for taking the time to come back on and honestly give bits about the state of the industry with me. If people want to learn more, where's the best place for them to find you?

Well, I'm still active on space normally known as, formerly known as Twitter. You can reach out to me there. DMs are open. I'm always willing to help people learn how to cloud better. Hopefully trying to make my presence known a little bit more on LinkedIn, if you happen to be over there.

Reach out. And we will, of course, put links to that in the show notes. Thank you so much for taking the time to speak with me again. It's always a pleasure. Thanks, Corey. I always appreciate it. Levi McCormick, Director of Cloud Engineering at Jamf. I'm cloud economist Corey Quinn,

and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice. Whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice. And along with an insulting comment that tells us that we completely missed the forest for the trees and that your programming is going to be far superior based upon generative AI. If your AWS bill keeps rising and your blood pressure is doing the same, then you need the Duck Bill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duck Bill Group works for you, not AWS.

We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.