The Changelog: Software Development, Open Source - Shift left, seriously. (Interview)

Episode Date: January 26, 2024

This week we're going deep on security and what it takes to shift left, seriously. Adam is joined by Justin Garrison (co-host of Ship It), plus two members of the BoxyHQ team — Deepak Prabhakara, Co...-founder & CEO and Schalk Neethling, Community Manager and DevRel as well as fellow Changelog Slack member. We discuss how to shift left, the role of the developer and the burden of security, the importance of tooling, the difference between authentication and authorization, and a mindset change for when security takes place — it's a matter of "when" not "who."

Transcript
Discussion (0)
Starting point is 00:00:00 This week on The Change Law, we're going deep on security and what it takes to shift left seriously. and two members of the Boxee HQ team, Deepak Prabhakara, co-founder and CEO, and Scott Kneethling, community manager, and Devrel, and fellow Changelog Slack member. By the way, our community is free. Join us in Slack, changelog.com slash community. We discuss the idea of shifting left, what it means, how to do it, the role of the developer, and the burden of security,
Starting point is 00:00:44 the importance of tooling, the difference between authentication and authorization, and a mindset change for when security takes place. It's a matter of when, not who. A massive thank you to our friends and our partners at Fly.io, the home of changelog.com. Launch your app near your users. Fly transforms containers into micro VMs that run on their hardware in 30 plus regions on six continents. Antarctica is coming soon. Launch an app for free at fly.io. What's up friends this episode of the change log is brought to you by our friends at vercell and i'm here with lee robinson aka lee rob vp product over at vercell so lee let's talk about
Starting point is 00:01:39 vercell's front end cloud i know vercell over 35 frameworks, but the question I often hear is, is the happy path on Vercel Next.js? Are you optimizing the best experience, the sugar on top, so to speak, for those using Next.js? Yeah, it's a great question because, of course, we're biased because we make Next.js, that we really like Next.js. But the idea behind the front-end cloud and the idea behind a concept we call framework-defined infrastructure is that we want to make it as easy as possible for you to write code in the frameworks that you prefer and still get to take advantage of all of the cloud infrastructure primitives and the developer experience platform that Vercel provides in that same model. So tangibly, this framework-defined infrastructure concept means that I write code in my framework-native way,
Starting point is 00:02:28 and Vercel is going to do the heavy lifting of transforming that into managed infrastructure. So in Next.js or in Nuxt or in SvelteKit or Remix or all of these frameworks that we support through framework-defined infrastructure, I'll write my code in their framework API. So I want to do server-side rendering. Great.
Starting point is 00:02:48 I'll do it through their conventions. When I deploy to Vercel, our platform is going to transform that code to work with our managed infrastructure that's very scalable and also flexible for what developers need. Okay, so help me break down what you mean when you say the front-end cloud.
Starting point is 00:03:04 I get Vercel, front-end, front-end frameworks, all that good stuff, but break that down for me. What exactly is the front-end cloud? We believe that there should be tools designed and built by and for front-end developers. We want to have tools that help enable you not only from the framework and the code that you're writing with React or Vue or Spelt, all the way up to the platform that allows you to deploy your code, not only in US East, but around the world as well too. And I think historically, a lot of tools for developers have been built backend first, have been built for the backend developer. And if you think about AWS, right, it's a great backend cloud. It has a suite of tools that you can pull from and really
Starting point is 00:03:47 craft and build anything that you want. But when you're building the front end of your application, there's a lot of things that are undifferentiated. It's like, I don't really want to have to stitch together all of these different pieces and set up my pipelines and set up all of this front end infrastructure. I just want to build a really good product. And really what the front-end cloud means to me is focusing more on the product experience and focusing more on the user experience and just letting a platform like Vercel's front-end cloud take care of all of that infrastructure and tooling for you.
Starting point is 00:04:18 Very cool. I love it. With zero configuration for over 35 frameworks, Vercel's front-end cloud makes it easy for any team to deploy their apps. Today, you can get started with a 14-day free trial of Vercel Pro or get a customized enterprise demo from their team. Visit vercel.com slash changelogpod to get started. That's V-E-R-C-E-L dot com slash changelogpod. So we're here with Deepak. Don't know how to say your last name, but I didn't ask you.
Starting point is 00:05:13 Can I try? Probacaria? Is that right? Close, close. What's the proper way to say it? Probacara. Probacara. Okay, I was pretty close on that one.
Starting point is 00:05:23 And Skulk Neithling? Is that right? Yeah, that's good. I mean, if you want the proper Afrikaans pronunciation, then it's Neithling. But Neithling works in a pinch. Okay, I'm in a pinch here, so I'm going to work with that one there. And we also have Justin Garrison here from ShipIt. How's it going, Adam?
Starting point is 00:05:41 Nice to have you here as a co-host. It's super cool to get you onto the network, but then also to get you onto this show with our friends over at Boxie HQ talking about security-related things. I mean, I think DevSec slash Ops sometimes is in my back pocket of things to talk about with folks when I meet them, and shifting left is always good for devs to be more mindful, I suppose, of, you know, the code we write and ship into production. And maybe this is more in the ship it lane, Justin, is that just like, how do we put the code we put out into production out there in a more secure way? So a lot of fun topics planned for today. I think maybe the one to open up with is maybe the easy button, the shift left. It seems to be a common in quotes phrase for folks to say,
Starting point is 00:06:25 I get it. How do we define shifting left? What does that actually mean to shift left? Does it have a different meaning depending upon who says it and where? I guess that question's for everybody. I think the fundamental, there's different layers to it, obviously.
Starting point is 00:06:40 So very high level, it's really about where the responsibility of anything right because it could be shift left testing could be shift left security shift left ops is where does that responsibility lie and left would mean more towards the developer or you know the the person responsible for it in some ways or or rather responsible for shipping something, to keep it simple. And on the right is really when it goes to, you know, it gets deployed and gets used.
Starting point is 00:07:11 So a lot of things traditionally have happened more to the right, which is after deployment or closer to deployment. Shift left is really about bringing that closer back to, you know, the person really building things today. Or, you know, the person really building things today. Or, you know, building software, right?
Starting point is 00:07:28 Of course, that's what it's all about. That would kind of be my fundamental description of what shiplap is. Okay. So if we shift this towards the source, whatever created the thing, and do we get rid of the right? Is this like a, hey, you no longer need
Starting point is 00:07:43 those security compliance tools or teams after the thing's running? Or is this now we do it in two places and we have double the work? So it's definitely not getting rid of the right. I mean, there's obviously those classic questions about, you know, who does it right? Can you actually, like, for example, in this case, development on one side, security on the other. Where does it, can you do away without, you know one side, security on the other. Can you do away without a traditional security team? And the answer is actually no, because you can't have the developer doing everything, of course. That's one thing. There's not enough mind space for anyone,
Starting point is 00:08:16 no matter how big the team is, because it's really about the focus. So that is really more about how you coordinate rather than getting rid of one side or the other. Yeah, I think I see that in the accessibility world cloud the same way, where you do want to shift left and give some of the responsibility over to the developer and the designers, but you're never going to be able to get rid of your accessibility specialists who can really make sure that all the nuance has not been missed. But, you know, I think it's kind of maybe another way one can look at it
Starting point is 00:08:52 is when you have a bunch of stuff running in CI, so when you open a pull request, it can like take care of a bunch of things that the maintainer now doesn't have to review because they can kind of run the CI things and it'll throw up like, oh, your formatting is not is not quite cool yes learn can scream at you because you did something and you can take care of that but then you get into the the minute things that the really detailed things and that's where your specialists come in and really have additional knowledge i suppose if
Starting point is 00:09:21 if we think that the shift left movement is more towards putting more responsibility on the developer, I feel like developers already have a lot of responsibility to then also take on one additional hat or half a hat or a portion of a hat is obviously going to be better for the product and better for the service and the company. But I just wonder, do we put too much burden already on developers to cover so much? I mean, even with NPM, like if you're a front-end developer pulling code from NPM, it's like a wild, wild west out there
Starting point is 00:09:54 in terms of like packages these days. Like we just had this major everything package that was meant to be a prank, but really caused some major harm. And that's totally possible. And so we have friends like Frost Book DJ who runs socket.dev, not a promotion, love him by the way,
Starting point is 00:10:10 that really is helping developers to shift left, right? To be able to pull packages from the, in quotes, open source supply chain, which is not something I really believe in, but it is a chain of supply. So sure, open source supply chain in a way that doesn't require the developer to dig into every single dependency every single line of code because it's just impossible right
Starting point is 00:10:29 it's impossible so when we talk about shifting left and giving more responsibility to developers how do we do that in a way that doesn't add that extra burden like what are you what are things you're doing a boxy execute to do that and what do you think about the holistically to not overburden the developer and say, well, you got to learn it, you have to have this responsibility. But how do we do that in a way that doesn't just add more to them? Yeah, yeah, no, I mean, I'll take the example of, you know, shift left security, but kind of applies to shift left in general. If you think about the spectrum, it's basically, you know, in a startup, it's very different from, you know, a very large enterprise
Starting point is 00:11:04 that has to do shift left, right? Mainly because in a startup, everything's very different from a very large enterprise that has to do shift left, right? Mainly because in a startup, everything is fused into one. You have one or a bunch of people just responsible for everything, shipping, deployment, thinking about security in the middle, usability, all of that, right? So in many ways, you're already wearing multiple hats. That's what a startup is all about. But then as you grow and as you know you start to say okay now actually i should have a separate function here then it just becomes how do you now start to bridge the two uh so that
Starting point is 00:11:36 spectrum is kind of fused together in smaller companies and kind of starts to spread apart in larger ones smaller ones is much easier because you kind of already have a bit of the context, the knowledge. It may not be full, but enough to kind of do it. Whereas as you start to get bigger, you need enough tooling, automation in the middle to kind of help with, as you said, how do you unburden folks with everything
Starting point is 00:12:02 that they already do to have to do one more thing. So it's more about providing them the right tooling to kind of give them enough context, give them just enough to kind of deal with what they have to do at that point in time. But then have enough resources to go back later. And then if it's not a deadline that they're dealing with or you know fighting a fire then you know they can always come back to it and they then have that foundation to work upon so that's how we think about it at voxie hq it's really about process tooling and you know people right so those three kind of come together to really help you do the shift left and it's not possible without you know all of them coming together
Starting point is 00:12:51 and i think um like what you mentioned adam like what the folks said socket.dev's doing i think jackson's doing some of that right because i think one of the ways to um allow a shift left is to ease the burden with tooling and i think you've hinted at this deep up. And so, you know, when you start talking about SSO and, you know, single sign on, and then you start blowing that, exploding that, and you get into all the nitty gritty and the nuance there, and you then start as a startup, for example, having to deal with stuff like SAML, which is, you know, an older way of doing it. It's not, you know, OAuth 2 or whatever. There's a lot of complexity that's being introduced there. And now you're thinking like, oh my goodness, how am I going to be able to deal with... And that is where something like Jackson comes in and it says, well, you just talk the
Starting point is 00:13:33 modern protocol and let us deal with talking to the more older sample-based system. And Deepak, you can dig into that a lot more than I can. But what I'm trying to get at is this idea that I think tooling can play a role here. And I think Boxee is building some of these building blocks, these security things in the open using open source to ease the burden. What's Jackson? I'm not familiar with the tooling there. It needs to have the full name, honestly. You need to say the full name. Yeah.
Starting point is 00:14:03 So Samuel Jackson is effectively our enterprise single sign-. You need to say the full name. Yeah, so Samuel Jackson is effectively our enterprise single sign-on product. Nice, okay. Say my name. And you know, because we don't have a marketing department to veto things, it kind of stuck. That was very good.
Starting point is 00:14:18 People say naming things is hard, and then you just nailed it. That was just, yeah. Where is the line here where you're talking about moving responsibility to developer and adding tooling for them. But like at some point when I go down, when I try to write some code, there's too much tooling.
Starting point is 00:14:32 Yeah. Everything gets in the way, right? Like I got chat GPT, I got my reliability tooling. I got this, you know, some sort of like Docker helper. I got a cloud helper. I got something. And then I got the security helper. At some point, like as a dev,
Starting point is 00:14:45 I just need to write some code and I need to like not think about all of those things for a hot minute. So I can just get the text in the editor and then I can go back and like say, oh, was this right? Or how does this actually apply to things? Where does the tooling get in the way
Starting point is 00:15:00 versus help a developer? You know, I come from, you come from the pre-cloud world. So early 2000s when I started my career was we had to go into a data center and then spend about a week there, hanging by the window, trying to get some Linux commands sorted out.
Starting point is 00:15:18 Then you set up a rack and then you can be useful, right? But today, of course, you can do all of that on the back of a click of a button on AWS or GCP or any cloud provider out there, right? So, you know, we've taken that, let's say, a week, as an example here, of going to a data center, setting up infrastructure.
Starting point is 00:15:39 You compress that down to a couple of minutes on the back of tooling automation. So that's how we think about it, right? You're taking a building block, packaging it up together. It has some interfaces that you deal with, but you don't need to now know how the actual infrastructure underneath works or is even set up. Except for when it breaks, right? When something's not secure or breaks.
Starting point is 00:16:02 Because that week in a data center taught me so much about, hey, when something broke, I was like, oh, I bet it's this cable. I was like, I know it's this plug because that plug was loose. And now I can go back and I can find exactly where it was, right? Because like that click of a button helps eliminate all that learning. And at some point, like I just need to go deep and like, let me go to a data center and like find the actual physical server. So I feel like those tools are helpful to accelerate some things.
Starting point is 00:16:27 But I'm curious there, like, because it feels like we need to be able to give someone the data center experience to understand what's happening under the hood and how it's actually built. Absolutely. I mean, at least for me, the way the best way to learn is like you mentioned, right? You're knee deep in there and something's broken and you're going to have to fix it. Doesn't matter where in the stack it is. But I think that's pretty hard
Starting point is 00:16:47 today because there's so many layers of abstraction. And I say it in a good way, right? Because, you know, how much time can we spend in this fast-paced world today to go and say, let's give a data center experience to, let's say, our SRE. You now just say, okay, I'll spin up another EC2
Starting point is 00:17:03 instance and just move stuff over and carry on because you can't be bothered to go figure out what exactly happened at an infrastructure level. So in many cases, you don't even have access to it because it's impossible. You have abstracted away so much that the only interface into that is a virtual machine. And you're not too concerned about where exactly it runs
Starting point is 00:17:24 and how it's orchestrated because the cloud provider is kind of doing that for you. And that's what it is today. If you're building software, it's taking all those abstraction levels, building on top of each other, and then saying, okay, my business logic is here, so I'm just going to go focus on that. As you mentioned, if something breaks, now you certainly have to say where in the layer it is. That's getting better, of course. You've got enough tooling to say, actually, this is in my application and not anywhere else.
Starting point is 00:17:53 That's, I think, where we want to, and that's where a lot of tooling can get in the way, but at the same time, it also helps you get out of it. At a high level, you need to understand which layer you need to jump into when something's wrong. And as you said, a lot of it is built over
Starting point is 00:18:09 with experience, but in other places, it's just about specialization. You're saying, okay, as an SRE, my responsibility is just looking at what's actually happening at an infrastructure operating system level,
Starting point is 00:18:26 but anything beyond that, now I don't understand because it's application level and then you go and call somebody else on the next level saying. And pretty much all those layers, right? They're all divided by who built that layer. Exactly. Right, because like the SRE is like, I built that layer,
Starting point is 00:18:37 so I understand how this piece works. And the developer is like, I built this layer, so I understand how that one works. Where does security as a shifting left paradigm, who builds that layer? If it's a tool to help a developer, who's the person that goes and debugs it and knows how it works?
Starting point is 00:18:53 So security gets very complex by definition because it's extremely layered, right? You've got security at the infrastructure level, even though you may not see infrastructure. Like, for example, what kind of network has opened up to the wide world? I mean, today, if you think about it as full-stack engineers, we don't really need to dive into networking, right? But at the same time, we don't fully understand the implications
Starting point is 00:19:17 of opening up something to the outside world. I always use this kind of security by default example. In effect, at least from a technical standpoint for a developer, things are only as secure as the default option. Because if you think about MongoDB, when they kind of came out and were getting popular,
Starting point is 00:19:39 they made it super easy to deploy something. I had to, right? Because from a developer experience, you wanted folks to just get up and running. And I've had this I had to, right? Because from a developer experience, we wanted folks to just get up and running. And I've had these cases many times where I've helped other companies kind of figure out what exactly went wrong. And most of the times it was because they deployed MongoDB somewhere, but the default option is that there's no authentication. And that's how 20,000-odd databases were out there. And then, you know,
Starting point is 00:20:06 boxes came in, scanned it, got in, because the port is well known. No one's at real fault here, right? Because by default, an even simpler example is your modem. Username is admin, password is password. You always get that. At my parents' place, they've never changed it until I went in and figured things out and gave them the right, you know, you take away the default, which is insecure,
Starting point is 00:20:32 and then put in something which then causes a usability issue, but at least, you know, it's secure. So that trade-off between what's easy to use versus what's secure
Starting point is 00:20:41 but a little more difficult to use. How do you understand how it works, right? Because like that, that default password on the router, it's probably okay as long as it's not open on the WAN, right? And that's what default providers started doing. It's like, oh, look, all these routers are available on WAN access with admin, admin, and guess what?
Starting point is 00:20:57 People can do whatever they want. And those defaults, yeah, that matters a lot until it's like, did I plug in the data center or did someone else or did I click a button? And where does it in for BoxEHQ, like what sort of defaults are you changing there? What are you actually relying on a developer to understand and know versus just doing it for like, hey, guess what? Like Mongo's insecure by default, or at least it was, right? Like, so we're just going to do that for you.
Starting point is 00:21:21 But you have like three more steps to set up authentication. You need an environment file, something like that. Yeah, exactly. So that's obviously fairly, you know, depends on the context as well, because for us, that's secure by default. I think we've learned it over time. Right now, of course, Mongo is also patched set up. So, you know, by default, you will have some kind of a reasonable layer of security, and
Starting point is 00:21:44 then you can kind of build upon that. So I think that kind of awareness has already come in. We also had cases where, you know, Internet of Things got popular maybe a decade ago. And because, I mean, from the outside, we always think an attack is quite sophisticated, mainly because we don't truly understand, you know, how that actually went about. But the reality is people get in through very simple things. And they try a lot of times and they find one or two weaknesses and then it kind of cascades into more things internally. I think classic case was a fish tank in, I think, a hedge fund
Starting point is 00:22:21 or a casino, I forget exactly. And that was IoT enabled and connected to Wi-Fi, and that was open. So they got into the actual internal network on the back of a fish tank that was connected. Could you imagine, right? Who didn't VLAN that network, right? Come on.
Starting point is 00:22:40 That's not a default in most networks, though, right? Like, VLANs are, like, something that you definitely, you know, because every network is designed, right? It's flat by default, I suppose. So not secure by default. And then you unflatten it by VLANing and you would have an IoT VLAN that has blocks, allows and disallows, of course. But I mean, who's responsible for that, right?
Starting point is 00:23:00 Like at a startup, it's flat and who knows, but like does BoxE HQ VLAN my network? Probably like that. I still need a network admin for that, right? startup, it's flat and who knows, but does Boxee HQ VLAN my network? I still need a network admin for that, right? Yeah, exactly. And even if you did all that and somebody installs it and you don't really know then, because as the person responsible you may not know that there's a fish tank that's
Starting point is 00:23:18 labeled, unless you have the right tooling to scan that and say, okay, now we've got another asset that I did not know about. So I think that's why I was talking about the layering, because you've got your external security, then you've got your internal security, and then more deeper things like infrastructure and things like that. Because ultimately, if somebody gets into your infrastructure, you're in big trouble anyway. They can do whatever they want to. And every layer, you're trying to protect them from
Starting point is 00:23:44 getting deeper into your stack. So the way we think about it very simply is building blocks, a bit like Lego, right? Who's constructing it and where that kind of responsibility lies. And of course, it's a very large spectrum, as I mentioned, and our entire focus is on not so much preventing hack, but assuming that you will get hacked as a baseline. And then you need to think about, is my product truly secure if they get to this level? Do I have the right set of primitives there to prevent that? Like, for example, escalation of privileges, can they get to the user management database? Can they get to our sensitive data? So that's kind of the different building blocks we think about at a high level and then kind of
Starting point is 00:24:32 try to make that as developer friendly at the same time having those reasonable secure by default. And as a power user, then you can build upon that with your own security construct. So that's kind of our approach. Trying to be opinionated, of course, so that you have, that's why, you know, the minimum viable security that we spoke about broadly. And this is an initiative from Google, Security Scorecard, and a bunch of really great security companies. They call it the MVSP. It's minimum viable secure product. And the fundamental idea there is that you need some level of basic security in place at a minimum for everyone.
Starting point is 00:25:14 And then you can build upon that based on your context, the vertically on end, the kind of requirements you play with. So that's how we think about it. We give you those minimum viable security blocks, and you then have that foundation to build upon. Sorry, I think one of the interesting things that I learned since joining, because I'm new at Boxee, like I'm here for like two, three months maybe,
Starting point is 00:25:38 is the significant difference between authentication and authorization and where um jack saddle jackson plays the sso role so that's your single sign-on role and where that is focused on authentication not authorization and i think the authentication is this first level of security almost where this gets you into the system but at that point the system then does not know it knows that it's you but not what you're allowed to do yet so that's that was interesting so i don't know if we want to talk a little bit more about the nuance between authentication and authorization because i found that when i've spoken to people before is that oftentimes those are used
Starting point is 00:26:22 interchangeably and when they're actually nuanced and different the one gets you into the system the other one deals with now that i'm in the system and i'm i've been authenticated that i am who i said i am what can i do inside the system is that something that you find developers are writing often that interchange between things is that something that's like a common thing for developers that are using boxy hq to like hey you know i do this all the time i need this help or Is that something that's like a common thing for developers that are using boxy HQ to like, Hey, you know, I do this all the time.
Starting point is 00:26:47 I need this help. Or is that something that's like kind of specialized for, Hey, we set this up once. And then once they, they are, you know, we have like a structure in place for authorization and we can say,
Starting point is 00:26:58 Hey, we, we trust that someone else said you, you are who you are. They have a key to the door. Now they're in the car. Do they have a driver's license? Can they actually drive the car? Those are different things that like my kids can are who you are. They have a key to the door. Now they're in the car. Do they have a driver's license? Can they actually drive the car?
Starting point is 00:27:06 Those are different things that like my kids can get in my car, but they can now drive the car. Is that something that you're looking at and that's something that's common? Or is this more of a framework helper for people that are like, hey, what can I do now that I have access to the application? Yeah, so there's, I think, always two aspects to it. One is more infrastructural in the sense that what is the foundation there, right? Can you give folks the building blocks to say, you can now build, for example, an authentication
Starting point is 00:27:36 policy around this? So in our case, we do that with just purely enterprise single sign-on, which is a way to get into any kind of system on the back of well-known established identity providers like Opta, Azure Active Directory. And just to even do that, you have to decipher all the protocols in between how they talk to each other. There's massive amounts of security vulnerabilities on the back of that, simply because of the protocols involved and not necessarily knowing how to do it right. So even within authentication, to get authentication right is a lot of work.
Starting point is 00:28:13 You can make things work, but then are you doing the right things in terms of being as secure as you can? Of course, there's nothing like 100% security, but as much as you can. Would that be a maximal security viable product? Is that like the opposite of minimal? I think that's pretty hard because, you know,
Starting point is 00:28:32 if you look at the breaches, even the best companies in the world get hacked and they've got all the resources, right, that they could think of. So I don't think there's a maximum level to it. It's more your risk management around it and how much you put into the most priority risk that you have. So it's always that trade-off in security.
Starting point is 00:28:53 So you can have minimum, but I don't think, because the maximum is never being able to get into your system. If I unplug it, it is maximally secure, right? Exactly. Go offline and don't let anyone into the room then. But at that point, it's pretty unusable. So, and because systems are evolving so quickly
Starting point is 00:29:14 and you kind of have to, I mean, like, just over the holidays, right, even though I didn't take, like, a long break, but you come out of it and the number of NPM updates you have to do is, like, insane. Who was shipping code? Who was doing this?
Starting point is 00:29:31 Dependabot, I'm on break, all right? This is something, just chill for a minute. You mentioned rights. You mentioned authentication being right, but systems always change. Is there a right and wrong is there a a there's a right for there's a correct way for right now or for this level of access because i feel like if there was a solid definition of this is the right way that's the wrong way the right would be default and i would hope it would be default but i don't think it is and at least from my experience that constantly
Starting point is 00:30:01 changes depending on the people that need to access the app, how the app is interfacing with other applications. So how does BoxEHQ or shifting lefts move that definition from, oh, this was right last week, but this commit changes what the new correct way of doing it is? Exactly. So there, I think the important bit there is a context, because you're building new things, requirements now change. Obviously, security also changes with the incoming requirements. A lot of the times, you're just getting something out, but not necessarily fully thinking about what are the implications of doing that, right? So in a lot of ways, first order effect is very
Starting point is 00:30:41 easy because you kind of see it, right? It's cause and effect. But second, third, higher order is very, very difficult even for people with experience, right? Because a lot of the times you may not have seen it or it manifests in a very different way, which then leads to other things, right? So that second, third order, higher order effects is quite difficult. Security by default is an example there because why would you think about that when you do something like a default option? So in many ways, that evolves as well. And so does your security on the back of the resources
Starting point is 00:31:16 available to you. So you're a startup now, you know, you've found product market, but you didn't start to scale up. Then it's a way different way to approach security because now, you know, you're popular, someone will try to hack you, you know, denial of service attacks, DDoS attacks, because it's so much cheaper to carry out attacks today than actually prevent it. There's a huge differential right cost differential. That's kind of how you start to think about this. And that's and that's how security evolves for you as well as a company. As you mentioned, is there a right way?
Starting point is 00:31:49 There's always the right way for the moment and that then quickly changes. That's why shift left as a process is also important because you're constantly thinking about it. Now that the vectors have changed, you now say, okay, actually now I have to go back and rethink a lot of things that I did not right. And can I get it to a phase now where I get it right enough that I can carry on to the next thing? So that trade-off between making something work to making something secure is quite a
Starting point is 00:32:22 broad spectrum and changes very quickly on the basis of risk and resources available. And sometimes you know it's there, but you know that it's not exploitable. So in which case it's fine because you've kind of dropped that risk down. So that's how everything you do is kind of evolves. And it's iteration, right? At the end of the day, it's security as well. For sure.
Starting point is 00:32:44 And that's why I think it's important to think about different layers as well. You can't just make your product secure without that external testing, that external approach to it. Someone who doesn't know your system might actually have a much better understanding of how to break it than you who are in the weeds and think that you have everything covered. Very simple things, right? Like API, if you think about API keys, you may not have thought about everything. So you let someone else come in
Starting point is 00:33:12 and kind of, you know, do that, right? So that could be tooling, that could be people, process. That's why that is important and you have to do it continuously. Otherwise, it just breaks down eventually. Always be improving, Kaizen. We do around here at least.
Starting point is 00:33:28 Absolutely. So it's super hard, but I'm going to give it a try. I'll start with... And then... and then that was pretty cool hey friends this episode is brought to you by our friends at Sanadia. Sanadia is helping teams take Nats to the next level via a global, multi-cloud, multi-geo, and extensible service fully managed by Sanadia. They take care of all the infrastructure, management, monitoring, and maintenance for you so you can focus on building exceptional distributed applications. And I'm here with VP of product and engineering, Byron Ruth, and David Gee, director of product
Starting point is 00:34:30 strategy. So when you think about connectivity being the first thing to consider, someone push back on this and say, we'll think about it later. What competes with a mindshare of connectivity? Just like an HTTP developer, you actually just download and run the NAT server. Whereas an HTTP developer, if you're building an HTTP set of endpoints, you typically have to implement or use an HTTP library. And then whether it's a Go standard library, Python, whatever it is, and you're actually implementing endpoints that register into the HTTP server. And then now you have to go deploy this HTTP server and ensure that it's like performant. So it's a slightly different model, but like you download the NAT server, it's a standalone
Starting point is 00:35:10 binary. It runs on, you know, the majority of platforms. And then you have a handful of client SDKs across all the major languages. You download that. And we even have a higher level API that is akin to what HTTP developers have of like defining a handler, for example. We just call it our services API. And you basically have a few boilerplate things that you register your handler in the NATS
Starting point is 00:35:37 context. And out of the box, it actually supports sort of a general request reply setup. And then you get all of these other benefits out of the gate. But the experience and like the onboarding is arguably just as simple as any other HTTP onboarding, with the exception that you're technically deploying a client application that implements these NAT services in addition to the NAT server. But that's where the Saniti Cloud, it's already managed instance, you know, and we even have the demo server for you to just try it out. It's a public endpoint that you can literally connect to. So you can still build a simple client application, use the demo server as the endpoint, and then you can play with that and use that as sort of the server deployment.
Starting point is 00:36:17 Well, if we talk about it just from, you know, the central view of applications, forget networking, all that kind of packet-based stuff, you were calling them HTTP developers, which kind of stalk instead of API devs. I mean, what do people do? They either glue it together at a primitive level. So the primitive being HTTP, they move up the stack in their mind's eye and they go, oh, we're going to do some gRPC, which is kind of still point to point. So it's a lot of point to point stuff versus broker assisted connectivity, which is way simpler. You connect to an endpoint, you get told about other endpoints. It's like connecting to a hive mind. What we're trying to do is move people away from coordinated point-to-point connectivity
Starting point is 00:36:52 to easy connect to anything securely and connect to your other stuff securely. Instead of having to coordinate the whole rat's nest of where to connect to them, then you've got to negotiate, well, what do we do then? Now we've got to go get the schema information and can we even connect to this thing? And does it even work? And you know, what version is it and all this stuff. And what we're trying to do is transform that and flip that to Unify to make it much simpler. So I think we're trying to go from a rat's nest of point-to-point connectivity in the application space to making everything on net. And it's like connecting to a hive mind. And what we're kind of
Starting point is 00:37:23 asking people to do is think about applications the same way you would video conferencing. So if me and Byron are going to have a chat, we might do a huddle on Slack or jump on a Zoom or something. But if we want a colleague to join, we ask them to join the same course. We can have a point to point conversation by the same medium, or we can have a party line by the same medium. So, you know, it's request, reply or pubs up, but it's on the same platform. We don't care about what Zoom server we connect to. We join, we to the service, and we coordinate our communications over the fabric. There you go. Yesterday's tech is not cutting it.
Starting point is 00:37:51 NATS, powered by the global multi-cloud, multi-geo, and extensible service, fully managed by Synedia, is the way of the future. Learn more at synedia.com slash changelog. That's S-Y-N-A-D-I-A dot com slash changelog. How does security as code shift left, I suppose, the shift left? Because security as code seems very similar where you just treat security like any other workflow when you're a software developer the full entire lifecycle.
Starting point is 00:38:35 It seems like shift left, but how does that change the lens of how you look at security when you think about that? Yeah, so security, I think that's quite new as well. You know, security as code in itself is really about automating certain things which fundamentally you couldn't do earlier because you had not built it in the right way to be automated.
Starting point is 00:38:57 So an example is, you know, we're talking about authentication authorization. Let's say you build an authorization engine or you pick one off the shelf somewhere. That fundamentally means you build a policy around it. And now that policy, of course, will change over time. But then because the policy is now in a way that actually can be embedded in code,
Starting point is 00:39:17 it's quite easy for you to go in and automate tests around that. So now it certainly goes from just having been a policy to you being able to tweak those things around. So that kind of is an example of security as code, right? Just like infrastructure as code. Earlier you built infrastructure, but didn't have a way to replicate it. Infrastructure as code now allows you to kind of bring up the same set of instances if you lose something. Same way with security.
Starting point is 00:39:47 You codify it so that you can kind of rebuild the whole thing again and then tweak some parameters if you wanted to. Let's say you had to add a new role. And now you're in the position to say, okay, now with this new role, can I go back and test everything that I had before and then improve upon that? Because now I know that, for example, if you add a billing role, then within your application, you know what is kind of segmented off. And now all your tests can kind of utilize that security as code
Starting point is 00:40:16 to kind of automate things around it. So security as code is really just giving developers the ability to kind of automate things, as opposed to kind of being a utopian way to say, we can change everything around it, because that's fundamentally not the case. Because every time you change something, you need to have enough context around it to kind of do the right kind of testing, right kind of, you know, release cycle,
Starting point is 00:40:43 and post-deployment as well, right? Are you testing fully end-to-end? So that's why security as a code is fairly new. You'll hear about it in DevSecOps. That's kind of more after you've written code and it's going through your CICD pipeline to get deployed. And at the same time, you can have a similar thing within your product on the back of, you know, policy.
Starting point is 00:41:10 And the interesting thing there is because fundamentally, you know, your product has usage on one side, but also there's compliance, potentially, or, you know, regulatory frameworks you deal with. Most of the times you need to collect some kind of evidence around the fact that you're doing something correctly, right? Or at least you're following a process around compliance. And most of the time, this is, you know, just boring evidence that you're collecting, like, for example, access control log, or the fact that, you know, you have an access control policy, but who understands it?
Starting point is 00:41:45 Is it just the product people or can your compliance manager also come in and get a full understanding of what this was about? So in many ways, security as code helps you make that bridge, that jump from people building it and know what it is about, know the full context.
Starting point is 00:42:02 Saying from an external point of view, can I now automate that piece and let them know that an admin is responsible for XYZ and here's evidence that will follow that. So that's where we see that the real value of security as code. You're automating things on one side, but you're also opening up evidence gathering on the other side, which is a crucial part of compliance.
Starting point is 00:42:26 Now, the first automation I ever wrote was a wiki page. I wrote documentation. Documentation was the first form of automation because I could write it down once and someone else would do it. And that was automation to me because I could say, hey, I need a new server set up this way.
Starting point is 00:42:40 I would point them to the wiki page and they would do it. And that was the best, the very first automation. And that had all the context in there. It had the screenshots and it had the paragraph of why we checked a box versus not checking a box. And that enabled someone else to go do that work. It was automated for me, not for them, right? They had to manually go through and then we're like, oh, let's take this wiki page and like
Starting point is 00:43:02 make it a bash script. And we can take that and we can say, hey, we can take out all of the context and we don't need to, we just trust that the steps are correct at some points. And someone that runs the bash scripts, they don't know what's happening. They don't know why it's happening. They just know something happened. And in security is code sense, I feel like we're removing some of that context. We're saying, hey, you just have to trust that we got this right in a meeting that we planned three years ago. And now you're able to just run the script and have this interface for, don't worry about it. It's a magical bash script
Starting point is 00:43:37 that does the secure thing. But now my question is like, who is responsible? Cause you're saying very specifically that shifting left for security and developers writing that that security is code versus the compliance manager and i feel like the compliance should never be a developer's responsibility a developer is responsible for i'm doing the secure thing by default that i know with my context but as we're talking we say layers right like the infrastructure layer changes, they update OIDC on the layer below you and your thing has to change and the person above you has to change. And so the only piece I wrote was in the middle there. And so now somewhere, someone, a compliance manager or some team needs to go through and say, hey, does the full path
Starting point is 00:44:20 through all of those layers still meet the requirements of whatever security posture we need to have, whatever compliance checks, whatever regulations we have. Someone else needs to be able to do that full stack because their context isn't what does the application do? Their context is, is this path and this token and whatever, is this part secure? Is that right? Yeah, yeah, absolutely. So we'll take an example, right? Like let's say you're building a high frequency trading platform. I would never do that. That sounds difficult. Let's say you are. There you are. Let's say your key metric is latency, right? You need to reduce latency as
Starting point is 00:45:00 much as you can. But there's a flip side to when things go wrong. There's obviously a financial aspect because you're losing money now because your trading system is broken. But then there's a lot of regulatory burden on the back of that, not just for you as a company, but externally as well. The SEC is going to come knocking on your door. So now you've kind of moved that whole this one from a system broke and wasn't available to why did it break? What went wrong? Was there a security implication of this breakdown? Did something get stolen?
Starting point is 00:45:35 Was it a deliberate attempt to kind of, you know, benefit financially for someone? So you start to get into all of this external context that your product may not have. So you now suddenly see that what is traditionally an availability incidence in any other B2B context is no longer that, right? Because of the regulatory environment you're playing in. So there, the burden to do this kind of automation plus knowing the context, you mentioned documentation. They do a lot here, right? Because error codes get documented.
Starting point is 00:46:10 You kind of have to follow that path on what actually went wrong, which system spoke to the other, and that whole traceability around where did the request start, where did it break down, which system was responsible. And on the back of that, figure out everything else, right?
Starting point is 00:46:26 So combination of logs, your own metrics around it, the code as well, because at that point, you need to know the exact code that did things. So there, you see that the burden around security as a code is very different from, let's say, someone building a CRM application, right? That kind of context, security, I mean, anything as code will help you there because, but then, of course, you know, you've written more code. So what happens to that as code that you've written? Whether it's a structured as code, security as code, or, you know, ops as code. So in many ways, that's why compliance exists
Starting point is 00:47:03 because they're now trying to say, here's your process. that with your process, your people, the regulatory burden you carry. And that also changes the pace at which you can move as well. The way to deploy a trading system is way different from traditional SaaS application, where you can have multiple deployments a day because you can always roll back, come back. So in many ways, as you said, documentation, you write documentation, it gets old. Then there's a disjoint now between where code lives and what the documentation says about it. So a lot of times it is about how close you can couple them because when one changes, you're then automatically forced to change the other. You're absolutely right. Like documentation does become,
Starting point is 00:48:06 it's a static thing and it doesn't match reality. But the people we hire are like the reason, reasonability of like, oh, I see where they went with this. I see how it is now. But my infrastructure as code does the exact same thing, right? Like my Terraform code isn't the exact state of reality for my infrastructure,
Starting point is 00:48:22 but even it's not like some of it can be changed, but also I'm not holistically defining every last piece for my infrastructure. But even it's not like some of it can be changed, but also I'm not holistically defining every last piece of my infrastructure because that's too much context that I need. And with Kubernetes, we do what's called infrastructure as software. It was a term my co-author and I coined a few years ago, but it's like,
Starting point is 00:48:39 it's continual reconciliation of what does the state of the world look like and what did we want it to define as? And as soon as one of those things get out of sync, it goes and it changes it, right? GitOps is the exact same thing. GitOps does infrastructure as software where it says like, hey, that thing that's applied didn't apply back here. I either need to make a commit back to Git or I need to change the world back to what
Starting point is 00:49:00 I think it should be. Is there a tool for infrastructure as software for security that says, hey, we're going to continually reconcile to keep that state as close as possible. So when the human looks at the code in Git, it is reasonably assumed that that is actually what's in the real world. Good question.
Starting point is 00:49:19 There's something that exists today. I think it's a combination of things, right? Because there is tooling today, but it's abstracted away, right? You spoke about Kubernetes. There's a lot of tooling around Kubernetes cloud native. Then you think about cloud. There's a bunch of things around cloud security
Starting point is 00:49:34 that you can plug in. But again, each layer kind of does specialize in what it does. At the end of the day, for you externally, you're thinking about cloud security, you're thinking about product security, application security across your DevOps pipeline, and then post-deployment and all of those things. So you have today the tooling that's fairly fragmented, and that's why security is quite an exciting field today, it's not yet reached a place where, like infrastructure was, right, a decade ago, where a lot of exciting things were happening, but you didn't have that holistic approach to it. And the question whether you can get that is also a big one, right? Because
Starting point is 00:50:15 security is very fragmented. Your responsibility in each layer is quite different. So you get tooling in each place. And the bigger question that we are also addressing is, can all of that be brought under, if not one single umbrella, but at least helping join those pieces together, which today is done quite manually on the back of compliance. You want to get a SOC 2 or an ISO 27001.
Starting point is 00:50:41 Again, compliance is not really security, but for a lot of practical purposes, they kind of blend together. One doesn't necessarily mean the other, but it's kind of assumed to be. That's why the larger companies will say, do you have this? In which case, we check all our boxes and we can carry on. Because today, there's no easy way for them to actually say, does this company have true internal security? So they kind of rely on some external signals to say, okay, they think about it, they're serious about it,
Starting point is 00:51:11 therefore we trust them with it. At a very high level, there isn't one tool that kind of unites everything, but you have enough tooling in each area of security you're dealing with, which then has to be brought together. And that's why that's so hard for the CISO and its team today because they are not building things, but they're defining security around it.
Starting point is 00:51:35 And I think that that mismatch is what is causing a lot of the context gathering that is quite manual today, which on the back of security as code should improve as you kind of go along, because as we saw, you can kind of expose a bunch of things to the other side saying, okay, now automated, you can gather X, Y, Z and feed that into your compliance. Also, maybe with the shiftless thing, I think other than trying to shift things onto a different crowd, splitting it off saying like, oh oh, this is another responsibility for you. I think part of that is maybe just changing your mindset.
Starting point is 00:52:12 So you're kind of thinking about security. You don't necessarily need to be a security specialist, but at least it's part of your thought process when you do something you're kind of like you stand back for a second you're like what could i potentially compromise by by not doing this or by doing this thing it's like if you use a llm and you stick your security uh your what your id number or whatever in there like now you've exposed this into where does it go before you do that you know you stop and say like if i put this in do i even know where this goes like if you have a key like a api key access key like you have to think about if i put it just in git and it goes to github this is not a good thing i don't want that that's why we have tools that'll say whoa whoa don't commit this this there's something in here that that that said
Starting point is 00:53:01 so i think it's thinking about that but then i think to what you were speaking justin this whole thing about syncing it made me think of react and all these new things where like this state no you think i know i'm sorry you know like this whole idea of when something changes like everything has to be synced back so that your ui is a reflection of your of your data and all that kind of stuff. I think one of the ways that this happens is like if you think about a system where you have a single sign-on process, so you've signed on, but now it brings in a bunch of users and things can change externally. So now how do you ensure that the central system, the SaaS system, for example, is in sync with all the things that happen outside? So I think something like directory sync comes into
Starting point is 00:53:45 play there, where it has a responsibility of picking up on webhooks that happens, that sees this event happened, like somebody joined the company, somebody left the company, somebody was promoted. And so as you are promoted, you now have more access and this needs to be synced through. But then at the same time, as all of this stuff happens, there needs to be seen through but then at the same time as all of this stuff happens There needs to be some kind of auditing and that is where something maybe like ordered logs can come in Which collects all of this action that happens and then over time if something an incident happens Now you have a source of truth so to speak that you can interrogate and ask like okay on this day between these hours What happened within my system
Starting point is 00:54:26 that caused this breach to potentially happen? So, I mean, I said a whole bunch of things there, but I think the shift left, security is different to that, but the shift left thing is changing just your mindset. So just when you're doing stuff on a daily basis, you kind of just stop for a moment and consider whether this,
Starting point is 00:54:43 like if I go back to front-end code, like, if I'm going to have a button, like, I should just use the HTML button, right? I shouldn't use a div and make it be a button. Don't overcomplicate it, please. If you're listening, you may remember the early days of the internet where open networks like HTTP and SMTP led to an explosion of websites and online communities. Building a fan site and connecting over shared passions led so many of us to careers in software. Back then, it seemed like anything was possible because the internet of the 90s was built to democratize information, not consolidate it with a handful of big tech companies.
Starting point is 00:55:32 Read, Write, Own, Building the Next Era of the Internet is a new book from startup investor Chris Dixon that explores how network architecture plays out in our online lives and the decisions that took us from open networks governed by communities of developers to massive social networks run by internet giants. ReadWriteOwn is a playbook for reclaiming control and for reimagining applications so users can own, co-create, and even profit from the platforms they use every day. From AI that compensates creators
Starting point is 00:56:05 to protocols that reward open source contributions, this is our chance to build the internet we want, not the one we inherited. Order your copy of Read Write Own today or go to readwriteown.com to learn more. What you're saying is resonating with me because I kind of feel like it's a win versus who and we began the conversation with shift left and the idea was you know deepak you mentioned it's not after deployment it's pre-deployment and so we began with the conversation assuming oh we're putting more burden and pressure on the developer which yes kind of unless you do with what you just
Starting point is 00:56:57 said which is it's more of when versus who it's at development not so much by the developer it's meant to happen prior to going to production so that you're not shipping the defaults in in the old mongo days or other scenarios like a modem you've got insecure defaults it's so that you can think about those things prior to shipping and it doesn't mean that it has to be the developer necessarily it's the team behind development which includes, but also may include compliance. It may include this, you know, MVSP, which is more of a kind of policy in a way of how to secure cloud applications. And as I'm reading this, there's like security headers in there. There's how you allow your customers to test the system. There's all these sort of protocols that are in there that can be tested. But this is something that the whole team says, okay, this is what we
Starting point is 00:57:44 want to do when it comes to shipping a cloud application that's secure. And that happens before development, that happens during development, but prior to production. So it's more of a when versus a who. That's kind of how I see your perspective shifting, at least the way you describe it.
Starting point is 00:58:00 Absolutely. It seems like a lot of that really depends on that minimal piece, right? Like the bare minimum unit, because like the context that I need to remember as I'm writing code is very different if it's a trading application versus my personal website. Way different, yeah. Yeah, that minimal piece. And it depends on like expectations, right? Like, am I going to be expected that this will never get hacked or this is only secure against this level of sophistication or something? And even what the user is going to expect out of it. Because just an example of users, I go change my password.
Starting point is 00:58:36 And I've worked in enterprises for 10 years now. And every single one of them, I go change my password. And the prompt says, please wait 24 hours for your password to replicate across the entire ecosystem of all of our apps. And like, that is the expectation of a user that I had that I'm like, oh, I may go to a site and have to use my old password for the next day. And that is okay that that state is not immediately synced because of the amount of pressure and scalability and all these other things we need to say like, oh, if I need that replicated everywhere right now, guess what? That's actually like a big, a big task to have everyone scale up their systems to get,
Starting point is 00:59:10 receive all those webhooks versus just like a little prompt on it that says, hey, just like wait 24 hours. And guess what? Like all the scaling problem goes away. All of that like expectation of this is everywhere right now. And it's going to be the same, not security, but the same password everywhere can be just different based on whatever minimal means in your context, at your application level, in whoever's writing that code. Because I can solve those problems at different layers. Well, this is a minimum viable secure product. So in this case, the word minimum is still used
Starting point is 00:59:40 there, but it's five scrolls for me, three or scrolls so it's a lot so this is minimum in comparison to a cloud application is that you know metric system scrolls or is that standard system scrolls i don't know if this is uh it's my digits scroll but that's that's a cloud application so i would say a minimum version of this for a personal website would be like one liner like don't get hacked yeah or i hope. Don't get hacked. Or hope it doesn't get hacked because it's personal. Not that big of a deal unless your email's attached to it and you've got other things happening as
Starting point is 01:00:12 a result. But anyways, I digress. We've been pretty bad at this as a software industry. We've stored passwords as plain text. We've done... I mean, it still happens. It's crazy. But there's legacy systems that can't change. And your password complexity, it causes more pain than actually securing it.
Starting point is 01:00:32 Because from a usability perspective, you're just denying the user. And then they just take the easier path out, right? So GDPR, for example, the reason it had to come in was, it's just common sense, right? Secure the data that you collect and don't use it in bad ways. You would think that's common sense, but I don't know. Yeah, exactly.
Starting point is 01:00:52 But that's the reason, right? The reason that came about is because companies were misusing it. They were not doing the right thing. They were just throwing it out everywhere. Could they even use it the way they used it? I mean, we don't even know today, right, where our data sits and how it's been used. So GDPR was kind of that attempt to say, okay, now just, you know, hold on, think about it.
Starting point is 01:01:12 You've collected a lot of data, but, and it's not even just about sensitive data, right? If you hold 6,000 email addresses, that's a big problem because that leak is effectively, and it's also the quantity you're collecting, not just specific things. So I think that's why all of these compliance regulations exist, because we just don't do it the right way until we're compelled by an external event to kind of make that happen. So security is kind of has to be forced upon, right? Otherwise, you kind of happily say, you know, if I don't have to do it, I won't or I'll do it later. And that data kind of comes.
Starting point is 01:01:48 Probably never. In many ways, from an external point of view, it feels like it's slowing things down, but it's more about saying, think about it. If we don't force you to think about it, you're never going to think about it. So I think that's where we've been pretty bad. And that kind of mindset that we were speaking about has to kind of come back because, you know, we have a lot of responsibility, right, as data collectors. And that's the crown jewel that we're collecting. And, you know go point to the hard drive that had the email addresses on them.
Starting point is 01:02:27 Like that's where it is. It's not leaving here, but now I can't do that. Right. Like I have like key value stores in cloud flare. I'm like, where does that exist? I don't know. Like what, what is my reasonable amount of expectation to secure that data? I was like, well, like I have an access key and that's how I get to it and how I can write to it. So no one else should be able to read it.
Starting point is 01:02:46 But I have to trust that my provider is doing the right thing beyond the layers that I have control of. Like I have the access key and they do everything else. But like dynamic is very different from a globally distributed key value store versus a hard drive in my closet. Like those are very different things around how I think about security and what my responsibilities are for that security. And so I guess it really depends on, again, those layers of like, I have a key to the door to the data center
Starting point is 01:03:13 and no one else has card access to it versus I have a key to the API to give me access to the data versus, you know, like, and those are just bits, right? Like if someone stole those bits, I would never actually know. But if I lose my keys, I know immediately. Like my physical keys, not the, like, and those are just bits, right? Like if someone stole those bits, I would never actually know. But if I lose my keys, I know immediately. Like my physical keys, not the API keys, right?
Starting point is 01:03:29 It's very different how we go through those mindsets. It's similar too, also to the build versus buy aspect, right? Because in the aspect of the data center and pointing to the hard drive, you've likely built the model. You've built the servers, you've racked and stacked, you provisioned all that good stuff. Whereas from built the servers, you've racked and stacked, you provisioned all that good stuff. Whereas from the API perspective, you have the key and what's behind the API is a
Starting point is 01:03:50 black box to you. You're focusing on trust. So you've in that scenario chosen the buy versus the build. And I think security is one of those things where, wow, you really have to have a lot of awareness and knowledge to do the build part and not a ton aside from judgment and maybe peer pressure to do the buy part right to buy a decent solution is like okay what are the what does google tell me when i say work os versus it is boxy hq in that list like i know michael greenidge from work os big fan of them and y'all are an open source solution to a similar problem where they're proprietary. And then it's like, well, do I trust open source more?
Starting point is 01:04:29 Do I trust the proprietary less? Do I build? Do I buy? Do I want to even learn SAML? Do I want to learn scheme? Do I want to learn all these protocols that do I want my team focusing on that versus the product we should be building?
Starting point is 01:04:41 And that's where you really sort of almost are forced by market nature to buy versus build. Even if you're not that knowledgeable to even know what to buy, you have to buy the most trusted default or trusted solution that comes up in. Yes, I still use Google, chat GPT isn't for everything. So when I am learning something like,
Starting point is 01:05:01 hey, what's well-known brand versus? And I usually choose from that because that's what the wisdom of the crowd has said. But let's open up that, like the buy versus build, because I feel like there's two different paradigms there. You can be knowledgeable enough to build, but does the market allow you to even do that? Maybe you have to be at a certain scale to say, we should build this versus buy this. The first stepping stone there was really infrastructure, right?
Starting point is 01:05:27 The data center, shift from data center to clicking a button on AWS. Right. If you didn't do that, let's say this was like the early 2000s, right, when AWS came out. If you chose to go to a data center, that means you were like at least a week or two behind, right? Just at the get-go from your competitor who chose to build on AWS. At the same time, as you kind of had to maintain it,
Starting point is 01:05:49 you would probably lose more time as you kind of evolved. So it kind of just immediately becomes a competitive advantage of speed. And today we know speed for a product is super important, right? You kind of get left behind. If you're not at the right place at the right time with the right product that you have.
Starting point is 01:06:08 That fundamentally, like, you know, Stripe changed that fundamentally as well. You were building payment systems, which took forever, didn't quite work as well, to now just integrating Stripe and, you know, carrying on, right? So what is core to you as a business
Starting point is 01:06:23 versus everything else? Twilio is a great example as well. You wouldn't build audio video today. Just pick up Twilio. Not if you're smart. Exactly. So I think that is that whole question of what am
Starting point is 01:06:40 I building upon? And there's other questions that come up. You're dependent on a platform and all of that. But as you start off, you're kind of then saying, if I had to build everything like Twilio and then do my value out on top, it's going to take me forever. So that has changed a lot now. You can roll things out over a weekend with almost no cost. That, I think, has become a competitive advantage question now.
Starting point is 01:07:07 What do you build? What is most essential to your core versus anything that's undifferentiated and commoditized, I think. So that, I think, is that buy versus build question that comes about. That's why SaaS has kind of taken off right in the last decade. I think average company uses about 80 SaaS vendors on average, which is quite a lot.
Starting point is 01:07:31 It is a lot, yeah. We see that as well. It's a small company. We have about 30, I can think of, that we use internally. Now, how is that different from what it was in the 90s, right? Because I would go buy box software. And that was like the 90s version of a SAS was like, I went to a, you know, circuit city or whatever. And I bought a box off the shelf and that was the software that I was buying. And I would run that
Starting point is 01:07:52 forever as like, this is the software I didn't have to write. And I feel like the SAS term throws around as like, it's something new, but really it's been around for a long time. That model of like, I don't have to build this part of it. I don't need to buy, build my own Excel. I just go buy Microsoft Office in a box and I paid for it once. And I know plenty of people that use Office 2003 for two decades. And it was the thing that ran for them versus a SaaS product where they have to keep updating and retraining and doing things. There is an amount of, I can buy this once and own the software and move faster long-term because I don't have to retrain people off of when I remember when office 2007 came out the ribbon, right? Like it was a huge thing. Like I was working in the
Starting point is 01:08:36 help desk and like, I wrote, I don't know how many pages of documentation about like, Hey, guess what? The ribbon is here. And you're going to have to change how you use Microsoft word because everything is different now and nobody wanted to move to it. And you're going to have to change how you use Microsoft Word because everything is different now and nobody wanted to move to it. And so it's like that slowed us down as a company because we had to stay with what the SaaS was providing. You know, we had to upgrade or whatever. And I feel like what you mentioned before of like, if you go to a data center versus
Starting point is 01:08:59 AWS, like you're going to be a week behind, except for the fact that like, if I go to AWS, I have to learn an organization and I am and security groups. And I have so many more layers to learn versus I plugged in a machine and installed Linux. And when I was working in a Disney animation, we, we had a render farm and we would, we could rebuild the entire render farm, like four or 5,000 machines in a day. And I went to Disney plus and we were in AWS. And if I wanted to replace our four or 5,000 machines, that was like six days. And like the difference between like how fast, like to re-image and to refresh the system, there was a speed
Starting point is 01:09:35 difference there. And owning the hardware and owning software is a very different mindset than what a SaaS and someone else can say what you can or cannot do and how fast you can do it. And how does that apply to security? Because I see like on your site, you have a self-hosted option. I could go download the software. It's open source. I can run it in my data center on my machines and I can run it for, I don't know. I don't know if there's, you know, even if it's not supported by you, I can run it forever. As long as the hardware is there versus the SaaS, I can get started now and see how this works and you're going to take care of the layer below
Starting point is 01:10:08 me. Maybe I don't need that. Maybe the actual long-term effects of running this myself and actually just having a little bit of expertise on how that runs and troubleshooting it is valuable to my company to be able to say like, oh, I don't need to open a support ticket to fix this. I can go find the open source and have my developers do it
Starting point is 01:10:24 right now. You bring up a very interesting point. I think go find the open source and have my developers do it right now. No, you bring up a very interesting point. I think to your point, one thing we always overlook, you know, build versus buy is can you hire the right set of people to do that, right? Internally. So your choice of, for example,
Starting point is 01:10:39 today React versus something else, typically, or, you know, even like, you know, Golang versus Java, right? It's fundamentally about who can you bring on board to kind of support it, right? Even if you build something, it needs to be supported in the long run. So I think a lot of the build versus buy choices also happen. It may not be the best one, right, that you pick up, but it could be the one that has
Starting point is 01:11:02 the most talent pool out there. Or you think that you will reach a point where this thing is going to get big. A lot of people bet on React when they saw Facebook was behind it, saying that this will become big. So we will have enough people who picked up the skills that are on. So I think to your point about that on-prem versus SaaS, we can't cannot talk about Salesforce as an example. They were the first ones who kind of, well, they did
Starting point is 01:11:29 a great thing with their whole no software in the cloud. You don't have to worry about it. And today, if you think about it, it helps you more fast. You've got not just a CRM, but a whole platform around it. You can now do pricing quotes on the back of that. You can now do, you know, quotes, pricing quotes
Starting point is 01:11:45 on the back of that. You can align your account management team around it. Customer support can happen there. So you've certainly gone from, you know, just a CRM to a whole range of things within that same platform,
Starting point is 01:11:58 so to speak. Do you find that distracting for companies? I mean, most companies have to do it. So I've seen this evolve. Your founder led sales, and you figured out BMF,
Starting point is 01:12:09 you hired a sales team, now Salesforce is where they live and breathe. Nobody likes Salesforce, but everybody has to be there for sales, marketing, and customer support. Can I plug something real quick? Because I do not like Salesforce. And I do our sales for us.
Starting point is 01:12:26 Yeah. And I started to use a thing called Pipe Drive. I think it's awesome. Yeah, exactly. I love it. I'm just like so in love with this product. Like I just want to mention it right now because they're not even paying me to do it. It's phenomenal. And I could use it as a one person sales show. And now if I had to scale and hire somebody next year to help me with sales, which we've considered, I would bring them in the pipe drive and I can take a step back, but still be involved. Anyways, I'm not going to plug them too far. They're just amazing.
Starting point is 01:12:52 And Salesforce has never, it's just been too big, just too big, you know, anyways. And that's the distraction I'm talking about. Like at what point do you need Salesforce, all of the features? Cause like, I can't tell you how many people over the last four years asked me what, how do I learn AWS? I'm like, there are over 200 services. You do not learn AWS. You learn one piece that you want to do and everything else is a distraction. Right. And I feel like Salesforce is the same thing where it's like, Hey, you wanted to do CRM and that's it. Like go to pipe drive. Like don't do all of the things that Salesforce does for a large enterprise, for someone that knows they need all the feature sets.
Starting point is 01:13:24 That makes sense. But for the smaller people, for the people getting started, for the people like, Hey, I want to solve this problem. Or indies, man. Yeah. Yeah. Like maybe Excel is too difficult, right? Like if I'm like, go look at Excel and I'm like, actually, there's too many functions here. I don't need all of them. I can hide most of them. But at some point, like I just need like maybe a calculator because actually what I need, I didn't need Excel to do that thing. Yeah. And I think that's why we are so focused in. We're not trying to do everything explicitly.
Starting point is 01:13:53 We're not trying to do everything. That's why we have these couple of protocols, I think is what you call, that we're focusing on, like enterprise SSO. We're not trying to solve the entire authentication authorization problem. We're trying to solve a very specific area in that. And I think that the reason, Adam, why you previously said the MBSP is so long, I think the reason why that is so long is because that is for when you want to sell to enterprise. Because now you get all kinds of... Right. And I expect it to be long too. That was not a knockout. I expect it to be. I mean, if you're going to
Starting point is 01:14:24 secure a cloud application, it should be long too. That was not a knockout. I expect it to be. I mean, if you're going to secure a cloud application, it should be pretty holistic. So the many scrolls was expected. Yeah, yeah. But I think one of the reasons why it is as complex and as vast as it is, is because who you're building this cloud application for is most likely a big company
Starting point is 01:14:40 that has all kinds of compliance rules around all of this. And to even satisfy the minimum is a big list exactly and it's it's a trade-off right like you mentioned salesforce versus pipe drive you can move fast with pipe drive but there might come a time when you need something that it doesn't have because there's integrations and things like that and it's possible but it kind of becomes a trade-off then you say okay now maybe i have to go back to i mean i have to migrate to salesforce i've seen this happen as well. As sales crew, they kind of were almost forced back into going to Salesforce, right? Or you're doing it in an Excel sheet for a while and that's fine. And then you're
Starting point is 01:15:15 going to go somewhere. So we see this in security as well, right? It's a trade off. We turn away a lot of prospects because we tell them that we can't do everything, right? And if you're thinking about doing everything, we're not the right product at this stage. And that's a trade-off as well in the build versus buy. It's okay. Do I pick this up from Box.chq? Do I go and build it?
Starting point is 01:15:38 Or do I find another vendor who does everything, right? So these are all the questions that constantly come up. And we obviously will evolve over time as well. As I mentioned, you go from single product to multi-product to kind of a platform. So that's kind of the bigger vision around solving this holistically, but that's kind of a 10-year timeframe, right? Not those three years. And what Salesforce did to the on-prem CRM Pipedrive is kind of doing it to them, right?
Starting point is 01:16:07 It's the evolution and there'll be a next generation. You might think so, that's for sure. I know I've ran CRMs on a desktop in a network environment when I had no idea what a network was and I was just like, just the worst anyways and running local software.
Starting point is 01:16:25 So thank God that Salesforce came along and used the cloud to make the software run in the cloud network versus on-prem, the LAN network. It revolutionized things. But I agree with Justin, like it just does so much. You don't learn Salesforce, you learn, can I just have the CRM? And that's just too complicated
Starting point is 01:16:43 because it thinks you want everything. And they have this massive scale enterprise large sales team mindset, which I suppose is more of a juxtaposition of how we think about security. You know, is it developers only or does that shift left include other folks who can help developers in the development process to get to where they need to go? The one thing I think that is really interesting, at least with Y'all's Choice, is the open source nature, right? You can go and self-host Boxee HQ yourself. You can do, I think, a premium version of it too, which I'm not sure how you license that,
Starting point is 01:17:17 but there's two different flavors where you can self-host. I'm assuming both built on open source, but then you can also go and do the SaaS version of it. But I think what I'm trying to promote more there is less about your tiers and how you're selling it, but more so that there needs to be back to that default, that secure by default scenario where can I choose a viable open source version that is free to me, free to contribute, free to use, free as in everybody else can use it as well. And it becomes this model that seems to be a version of yes for you all. But at the same time, does that then make it one single point of failure? Like if everybody starts going the boxy HQ way to do SSO and secure their SaaS products for enterprise,
Starting point is 01:17:59 does that mean that you become more of a centerpiece for an attack? Maybe yes. That comes back to the build versus buy make the right choice and like you said before tupac it's just constantly iterative when it comes to security iteration always improving being aware and fine-tuning and making changes as that changes but i'm also very happy that you've chosen open source can you talk about the choice to i mean because you can kind of cannibalize yourself by yourself by being open source, right? How did you choose that? Why was open source super important to you all?
Starting point is 01:18:32 Yeah. So two fundamental points for us when we were thinking about, you know, our go-to market, you know, we are serving the developers and as a developer myself, and, you know, having seen my team multiple times before, it's the best way to go and evaluate something, right? You're either thinking about, do I build this or do I pick up something that gives me some kind of foundation to build upon? But then you can also evaluate it without any of the having to talk to anyone, no sales, marketing language. You can truly see it functional. You see the value. And that's how you're, it's like the freemium version, right,
Starting point is 01:19:11 for SaaS, effectively. Somebody's there, we don't even know who they are, they're trialing it, and then, you know, they might come back around for more questions, or they might stumble into an issue and they, you know, they're on our community. So that was kind of the one major aspect to think about how we're discovered. But at the same time, we also realized that we're quite close to infrastructure and commoditized, right?
Starting point is 01:19:35 We're building on top of protocols. We're not trying to reinvent anything. Fairly interoperable protocols. We wanted to keep it that way. And that's the best way to kind of bring in contributions from the community, right? Integrations, you know, let us support 20 odd providers, some providers we can't get access to. We had a lot of help from community who were using that and were trying us out and ran into trouble and could help us troubleshoot and submit changes to
Starting point is 01:20:02 make it work. Those two were kind of the main reasons why we chose to go. And our license is Apache 2, so no restrictions, right? Fully free to use it. And what we've done is in an open core model, there are certain premium features we identified that only the larger companies need. And those are kind of locked off
Starting point is 01:20:22 as requiring an enterprise edition. But that doesn't prevent anyone else from not being able to use the full functionality of everything else. So that was our thinking here, you know, close to infrastructure, how developers discover and evaluate us. But in the process, what we really found insightful was we get used in the tech stack in ways that we didn't anticipate before.
Starting point is 01:20:47 Because they can lift and shift us, right? So they've used us in places that we did not even think was a use case before. So that was quite insightful to see the new use cases kind of emerging. How do you deal with that case where someone uses
Starting point is 01:21:03 you in a way you didn't think, but then maybe they want to contribute what you would consider an enterprise feature. How do you deal with open, but open to contributions? We obviously tie that in with our roadmap that we think about at a very high level on where we want to evolve as a company, both from a product and kind of who we serve perspective. And all the contributions, ideas coming in is kind of evaluated and prioritized based on that. Because there's many directions we can go, like Scott mentioned, we could do all of authentication, but that's a very different problem to solve versus going deeper into developer security.
Starting point is 01:21:41 That's the path we've picked. So we're quite strong about saying no to things that we think is not kind of core to what we're doing. But at the same time, they have the APIs to extend it and use it for the bespoke use case. And wherever we see repeated use cases, we kind of
Starting point is 01:21:58 then go out and see, there's a lot of verticals we don't have experience in, but it's kind of being used there. And then we try and sit down and see if, you know, that's a vertical we could just go after from your go-to-market perspective. Very cool. I was just clicking around on the open source.
Starting point is 01:22:14 I love seeing commits. When something is open source, I love just digging into GitHub and looking at the commits. I didn't do that. I didn't dig into the actual source. I looked at it, but it didn't dig in. No, I just want, Deepak, I wanted to commend you.
Starting point is 01:22:26 This is actually amazing because you have not the most commits. Dependabot obviously has the most commits. Dependabot is just a 100x developer out there. But Deepak, of your commits, you have 516,000 lines added, lines of code. But even better than that,
Starting point is 01:22:41 you have 470 removed. And that's amazing. That is a great stat right there. Like you are only 46,000 away from like breaking even on like, you could just have zero code left. I mean, obviously it's a bit skewed by the package lock and stuff, but I love removing code, you know, wherever my kind of sometimes a metric is a more random there is, you know, it's better. I always strive to be neutral in my code. Like by the time I leave a company, I want to have as many lines written as removed.
Starting point is 01:23:17 And like that is how I know that like I made an impact of like getting rid of stuff and adding the, you know, like whatever it is. But like break those even and get that in a neutral spot is a goal of mine. Interesting. There's a funny anecdote. I mean, I won't name the company I was at, but wherever I was, they came up with this crazy metric to just reward people. I'm more recognized, right? Not even monetary reward or anything. They said
Starting point is 01:23:35 the one where they developed the most number of lines of code, so commit, and this was back in the days of CVS, right? Before Git. And people gamed the hell out of it, right? It's, you know, changed the whole formatting to leave like two lines between every piece, line of code.
Starting point is 01:23:53 And then it became a game of, you know, who can game it better to get to the top. Semi-colons go on their own line. And then we have two lines at the end of the file. Yeah, you can totally, like this is, if this is in my promo doc, I am getting more lines, that is for sure. Well, let's leave it there then.
Starting point is 01:24:08 I think we covered most of it. Definitely love hearing about the journey and the security journey, I suppose, for developers and how they think about it. I mean, I think one takeaway for me was really just the shift left of my shift left, which is that it's when, not who. I think that was really an eye-opener
Starting point is 01:24:24 because every time I hear the term shift left, always think in almost a defensive way like gosh can developers really just do more can we just keep piling on more but i think now that i think about it like skulk said just from a different perspective that it's really about when which was my words uh but really synthesized from his meaning was when versus who it's pre-deployment that it's everyone's job to think about it, but when we're actually building the application, not once it's built and shipped do we then begin to think about security. Because we're going to build better companies, we're going to build better products, we're going to build better teams,
Starting point is 01:24:58 which might even mean keeping people around longer, paying them more, profiting more, all the mores that you can put into that stack seem to be better if you can shift that left to the win versus just the who. So that's a takeaway for me. That's super insightful. I didn't think about it that way at all. Well, that's why we had you on here,
Starting point is 01:25:16 so we can give you some feedback. It usually happens to some degree. All right, well, let's leave it there. Thanks for coming on. Thanks for sharing this security journey. And Justin, thanks for joining me, man. It's been fun gabbing with you a bit, man. It's our first time together.
Starting point is 01:25:30 Maybe we'll do it again sometime, you know? Who knows? Yeah, this was great. Awesome. Yeah, I mean, honor to be here. It was an absolute pleasure. Very cool. Thank you.
Starting point is 01:25:38 Thank you so much. Thanks so much. Okay, it is time to shift left. Seriously, hopefully this conversation today gave you some things to think about in your teams, environments, etc. Make sure you check out BoxieHQ at
Starting point is 01:25:55 BoxieHQ.com and big thanks to Deepak and Skulk for coming on the show today. By the way, Jared and I are on the road next week, so expect maybe a few delays in our shipping schedule. We'll be at that conference. No, for real, that conference.
Starting point is 01:26:13 Go to thatconference.com. We'll be here in my backyard in Texas celebrating that front-end polyglot way. If you're there, we'll be in the hallway and on stage. Thanks to our friends at Cloudflare for hooking us up and making that conference possible. Coming up on ChangeLogging Friends, we have a deep conversation with Jose Valim
Starting point is 01:26:36 on the state of Elixir and what's happening with being gradually typed and his thoughts on artificial intelligence. We just can't avoid this conversation. And I guess next week on the change, we're also talking about AI deeply as well. So stay tuned for that. Of course,
Starting point is 01:26:52 a big thank you to our friends and our partners at fly.io, the home of change.com. Launch your app for free at fly.io, as well as our friends over at century. Get $100 of the team plan for free for 3.8 months. It's kind of cool. Use the code change log to get that once again, century dot IO S E N T R Y dot IO a hundred bucks off the team plan for free a hundred bucks off the team plan. That's pretty awesome. And of course, the Beat Freak in Residence Breakmaster Cylinder.
Starting point is 01:27:26 You know, next week we have something coming out. We'll be announcing. Hopefully our conference travels won't impact that much at all. But get excited. A new album awaits. Also, there's a bonus for our Plus Plus subscribers. If you're not a Plus Plus subscriber, do so at changelog.com slash plus plus. It's better.
Starting point is 01:27:47 It is better. Ten bucks a month, a hundred bucks a year. Drop the ads, closer to the metal, directly support us. And of course, that sweet bonus content. Okay, that's it. The show's done. Bye for now. Game on.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.