The Changelog: Software Development, Open Source - Shift left, seriously. (Interview)
Episode Date: January 26, 2024This week we're going deep on security and what it takes to shift left, seriously. Adam is joined by Justin Garrison (co-host of Ship It), plus two members of the BoxyHQ team — Deepak Prabhakara, Co...-founder & CEO and Schalk Neethling, Community Manager and DevRel as well as fellow Changelog Slack member. We discuss how to shift left, the role of the developer and the burden of security, the importance of tooling, the difference between authentication and authorization, and a mindset change for when security takes place — it's a matter of "when" not "who."
Transcript
Discussion (0)
This week on The Change Law, we're going deep on security and what it takes to shift left seriously. and two members of the Boxee HQ team, Deepak Prabhakara, co-founder and CEO,
and Scott Kneethling, community manager,
and Devrel, and fellow Changelog Slack member.
By the way, our community is free.
Join us in Slack, changelog.com slash community.
We discuss the idea of shifting left,
what it means, how to do it,
the role of the developer, and the burden of security,
the importance of tooling, the difference between authentication and authorization, and a mindset change for when security takes place.
It's a matter of when, not who.
A massive thank you to our friends and our partners at Fly.io, the home of changelog.com. Launch your app near your users. Fly transforms containers into micro VMs
that run on their hardware in 30 plus regions
on six continents.
Antarctica is coming soon.
Launch an app for free at fly.io.
What's up friends this episode of the change log is brought to you by our friends at vercell and i'm here with lee robinson aka lee rob vp product over at vercell so lee let's talk about
vercell's front end cloud i know vercell over 35 frameworks, but the question I often hear
is, is the happy path on Vercel Next.js? Are you optimizing the best experience, the sugar on top,
so to speak, for those using Next.js? Yeah, it's a great question because, of course,
we're biased because we make Next.js, that we really like Next.js. But the idea behind the
front-end cloud and the idea behind a concept we call framework-defined infrastructure is that we want to make it as
easy as possible for you to write code in the frameworks that you prefer and still get to take
advantage of all of the cloud infrastructure primitives and the developer experience platform
that Vercel provides in that same model. So tangibly, this framework-defined infrastructure concept means that I write code in my framework-native way,
and Vercel is going to do the heavy lifting
of transforming that into managed infrastructure.
So in Next.js or in Nuxt or in SvelteKit or Remix
or all of these frameworks that we support
through framework-defined infrastructure,
I'll write my code in their framework API.
So I want to do server-side rendering.
Great.
I'll do it through their conventions.
When I deploy to Vercel,
our platform is going to transform that code
to work with our managed infrastructure
that's very scalable and also flexible
for what developers need.
Okay, so help me break down what you mean
when you say the front-end cloud.
I get Vercel, front-end, front-end frameworks, all that good stuff, but break that down for me.
What exactly is the front-end cloud? We believe that there should be tools designed and built by
and for front-end developers. We want to have tools that help enable you not only from the
framework and the code that you're writing with React or Vue or Spelt, all the way up to
the platform that allows you to deploy your code, not only in US East, but around the world as well
too. And I think historically, a lot of tools for developers have been built backend first,
have been built for the backend developer. And if you think about AWS, right, it's a great
backend cloud. It has a suite of tools that you can pull from and really
craft and build anything that you want. But when you're building the front end of your application,
there's a lot of things that are undifferentiated. It's like, I don't really want to have to stitch
together all of these different pieces and set up my pipelines and set up all of this front end
infrastructure. I just want to build a really good product. And really what the front-end cloud means to me
is focusing more on the product experience
and focusing more on the user experience
and just letting a platform like Vercel's front-end cloud
take care of all of that infrastructure and tooling for you.
Very cool. I love it.
With zero configuration for over 35 frameworks,
Vercel's front-end cloud makes it easy for any team to deploy their apps.
Today, you can get started with a 14-day free trial of Vercel Pro or get a customized enterprise
demo from their team.
Visit vercel.com slash changelogpod to get started.
That's V-E-R-C-E-L dot com slash changelogpod. So we're here with Deepak.
Don't know how to say your last name, but I didn't ask you.
Can I try?
Probacaria?
Is that right?
Close, close.
What's the proper way to say it?
Probacara.
Probacara.
Okay, I was pretty close on that one.
And Skulk Neithling?
Is that right?
Yeah, that's good.
I mean, if you want the proper Afrikaans pronunciation, then it's Neithling.
But Neithling works in a pinch.
Okay, I'm in a pinch here, so I'm going to work with that one there.
And we also have Justin Garrison here from ShipIt.
How's it going, Adam?
Nice to have you here as a co-host.
It's super cool to get you onto the network, but then also to get you onto this show with our friends over at Boxie HQ talking about security-related things.
I mean, I think DevSec slash Ops sometimes is in my back pocket of things to talk about with folks when I meet them, and shifting left is always good for devs to be more mindful, I suppose, of, you know, the code we write and ship into production.
And maybe this is more in the ship it lane, Justin, is that just like,
how do we put the code we put out into production out there in a more secure way?
So a lot of fun topics planned for today.
I think maybe the one to open up with is maybe the easy button, the shift left.
It seems to be a common in quotes phrase for folks to say,
I get it.
How do we define shifting left?
What does that actually mean to shift left?
Does it have a different meaning
depending upon who says it and where?
I guess that question's for everybody.
I think the fundamental,
there's different layers to it, obviously.
So very high level,
it's really about where the responsibility of anything right
because it could be shift left testing could be shift left security shift left ops is where does
that responsibility lie and left would mean more towards the developer or you know the the person
responsible for it in some ways or or rather responsible for shipping something, to keep it simple.
And on the right is really when
it goes to, you know, it gets
deployed and gets used.
So a lot of things traditionally
have happened more to the right,
which is after deployment or closer
to deployment. Shift left is
really about bringing that closer back
to, you know, the person
really building things today. Or, you know, the person really building things today.
Or, you know, building software, right?
Of course, that's what it's all about.
That would kind of be my fundamental description
of what shiplap is.
Okay.
So if we shift this towards the source,
whatever created the thing,
and do we get rid of the right?
Is this like a, hey, you no longer need
those security compliance tools or
teams after the thing's running? Or is this now we do it in two places and we have double the work?
So it's definitely not getting rid of the right. I mean, there's obviously those
classic questions about, you know, who does it right? Can you actually,
like, for example, in this case, development on one side, security on the other.
Where does it, can you do away without, you know one side, security on the other. Can you do away without a traditional
security team? And the answer is actually no, because you can't have the developer doing
everything, of course. That's one thing. There's not enough mind space for anyone,
no matter how big the team is, because it's really about the focus. So that is really more about
how you coordinate rather than getting rid of one side or the other.
Yeah, I think I see that in the accessibility world cloud the same way,
where you do want to shift left and give some of the responsibility
over to the developer and the designers,
but you're never going to be able to get rid of your accessibility specialists
who can really make sure that all the nuance has not been missed.
But, you know, I think it's kind of maybe another way one can look at it
is when you have a bunch of stuff running in CI,
so when you open a pull request,
it can like take care of a bunch of things
that the maintainer now doesn't have to review
because they can kind of run the CI things
and it'll throw up like, oh, your formatting is not is not quite cool yes learn can scream at you because you did something
and you can take care of that but then you get into the the minute things that the really detailed
things and that's where your specialists come in and really have additional knowledge i suppose if
if we think that the shift left movement is more towards putting more responsibility on the developer,
I feel like developers already have a lot of responsibility to then also take on one additional hat or half a hat or a portion of a hat
is obviously going to be better for the product and better for the service and the company.
But I just wonder, do we put too much burden already
on developers to cover so much?
I mean, even with NPM,
like if you're a front-end developer pulling code from NPM,
it's like a wild, wild west out there
in terms of like packages these days.
Like we just had this major everything package
that was meant to be a prank,
but really caused some major harm.
And that's totally possible.
And so we have friends like Frost Book DJ
who runs socket.dev, not a promotion,
love him by the way,
that really is helping developers to shift left, right?
To be able to pull packages from the,
in quotes, open source supply chain,
which is not something I really believe in,
but it is a chain of supply.
So sure, open source supply chain
in a way that doesn't require the developer to
dig into every single dependency every single line of code because it's just impossible right
it's impossible so when we talk about shifting left and giving more responsibility to developers
how do we do that in a way that doesn't add that extra burden like what are you what are
things you're doing a boxy execute to do that and what do you think about the holistically
to not overburden the developer and say, well, you got to learn it,
you have to have this responsibility. But how do we do that in a way that doesn't just add more
to them? Yeah, yeah, no, I mean, I'll take the example of, you know, shift left security,
but kind of applies to shift left in general. If you think about the spectrum, it's basically,
you know, in a startup, it's very different from, you know, a very large enterprise
that has to do shift left, right? Mainly because in a startup, everything's very different from a very large enterprise that has to do shift left, right?
Mainly because in a startup, everything is fused into one.
You have one or a bunch of people just responsible for everything,
shipping, deployment, thinking about security in the middle,
usability, all of that, right?
So in many ways, you're already wearing multiple hats.
That's what a startup is all about.
But then as you grow and as you know you start to say okay now actually i should have a separate function here then it just becomes how do you now start to bridge the two uh so that
spectrum is kind of fused together in smaller companies and kind of starts to spread apart
in larger ones smaller ones is much easier because you kind of already have a bit of the context,
the knowledge.
It may not be full, but enough to kind of do it.
Whereas as you start to get bigger,
you need enough tooling, automation in the middle
to kind of help with, as you said,
how do you unburden folks with everything
that they already do to have to do one more thing.
So it's more about providing them the right tooling to kind of give them enough context,
give them just enough to kind of deal with what they have to do at that point in time.
But then have enough resources to go back later.
And then if it's not a deadline that they're dealing with or you know fighting a fire then you know they can always come back to it and they then have that foundation to work upon
so that's how we think about it at voxie hq it's really about process tooling and you know people
right so those three kind of come together to really help you do the shift left and it's not
possible without you know all of them coming together
and i think um like what you mentioned adam like what the folks said socket.dev's doing i think jackson's doing some of that right because i think one of the ways to um allow a shift left is to
ease the burden with tooling and i think you've hinted at this deep up. And so, you know, when
you start talking about SSO and, you know, single sign on, and then you start blowing that, exploding
that, and you get into all the nitty gritty and the nuance there, and you then start as a startup,
for example, having to deal with stuff like SAML, which is, you know, an older way of doing it.
It's not, you know, OAuth 2 or whatever. There's a lot of complexity that's being
introduced there. And now you're thinking like, oh my goodness, how am I going to be able to deal
with... And that is where something like Jackson comes in and it says, well, you just talk the
modern protocol and let us deal with talking to the more older sample-based system. And Deepak,
you can dig into that a lot more than I can. But what I'm trying to get at is this idea that I think tooling can play a role here.
And I think Boxee is building some of these building blocks, these security things in the open using open source to ease the burden.
What's Jackson?
I'm not familiar with the tooling there.
It needs to have the full name, honestly.
You need to say the full name.
Yeah.
So Samuel Jackson is effectively our enterprise single sign-. You need to say the full name. Yeah, so Samuel Jackson is effectively
our enterprise single sign-on product.
Nice, okay.
Say my name.
And you know, because we don't have
a marketing department to veto things,
it kind of stuck.
That was very good.
People say naming things is hard,
and then you just nailed it.
That was just, yeah.
Where is the line here where you're talking about
moving responsibility to developer and
adding tooling for them.
But like at some point when I go down, when I try to write some code, there's too much
tooling.
Yeah.
Everything gets in the way, right?
Like I got chat GPT, I got my reliability tooling.
I got this, you know, some sort of like Docker helper.
I got a cloud helper.
I got something.
And then I got the security helper.
At some point, like as a dev,
I just need to write some code
and I need to like not think about all of those things
for a hot minute.
So I can just get the text in the editor
and then I can go back and like say,
oh, was this right?
Or how does this actually apply to things?
Where does the tooling get in the way
versus help a developer?
You know, I come from, you come from the pre-cloud world.
So early 2000s when I started my career
was we had to go
into a data center and then
spend about a week there, hanging
by the window, trying to get some Linux commands
sorted out.
Then you set up a rack and
then you can be useful, right?
But today, of course, you can do
all of that on the back of a click of a button on AWS or GCP
or any cloud provider out there, right?
So, you know, we've taken that, let's say, a week,
as an example here,
of going to a data center, setting up infrastructure.
You compress that down to a couple of minutes
on the back of tooling automation.
So that's how we think about it, right?
You're taking a building block, packaging it up together.
It has some interfaces that you deal with, but you don't need to now know how the actual
infrastructure underneath works or is even set up.
Except for when it breaks, right?
When something's not secure or breaks.
Because that week in a data center taught me so much about, hey, when something broke,
I was like, oh, I bet it's this cable.
I was like, I know it's this plug because that plug was loose.
And now I can go back and I can find exactly where it was, right?
Because like that click of a button helps eliminate all that learning.
And at some point, like I just need to go deep and like, let me go to a data center
and like find the actual physical server.
So I feel like those tools are helpful to accelerate some things.
But I'm curious there, like, because it feels like we need to be able to give someone the
data center experience to understand what's happening under the hood and how it's actually
built.
Absolutely.
I mean, at least for me, the way the best way to learn is like you mentioned, right?
You're knee deep in there and something's broken and you're going to have to fix it.
Doesn't matter where in the stack it is.
But I think that's pretty hard
today because there's so many layers of abstraction.
And I say it in a good way, right?
Because, you know, how much time
can we spend in this fast-paced world today
to go and say, let's give a
data center experience to, let's say, our
SRE. You now
just say, okay, I'll spin up another EC2
instance and just move stuff over and carry on
because you can't be bothered to go figure out
what exactly happened at an infrastructure level.
So in many cases, you don't even have access to it
because it's impossible.
You have abstracted away so much
that the only interface into that is a virtual machine.
And you're not too concerned about where exactly it runs
and how it's orchestrated because the cloud provider is kind of doing that for you. And that's what it is today.
If you're building software, it's taking all those abstraction levels, building on top of each other,
and then saying, okay, my business logic is here, so I'm just going to go focus on that.
As you mentioned, if something breaks, now you certainly have to say where in the layer it is.
That's getting better, of course.
You've got enough tooling to say,
actually, this is in my application
and not anywhere else.
That's, I think, where we want to,
and that's where a lot
of tooling can get in the way, but at the same
time, it also helps you get out of it.
At a high level, you need to understand which
layer you need to jump into when something's
wrong. And as you
said, a lot of it is built over
with experience, but in other
places, it's just about specialization.
You're saying, okay, as an SRE,
my responsibility is just
looking at what's
actually happening at an
infrastructure operating
system level,
but anything beyond that,
now I don't understand because it's application level and then you go and call somebody else
on the next level saying.
And pretty much all those layers, right?
They're all divided by who built that layer.
Exactly.
Right, because like the SRE is like,
I built that layer,
so I understand how this piece works.
And the developer is like, I built this layer,
so I understand how that one works.
Where does security as a shifting left paradigm,
who builds that layer?
If it's a tool to help a developer,
who's the person that goes and debugs it
and knows how it works?
So security gets very complex by definition
because it's extremely layered, right?
You've got security at the infrastructure level,
even though you may not see infrastructure.
Like, for example, what kind of network has opened up to the wide world?
I mean, today, if you think about it as full-stack engineers,
we don't really need to dive into networking, right?
But at the same time, we don't fully understand the implications
of opening up something to the outside world.
I always use this kind of security by default example.
In effect, at least from a technical standpoint
for a developer,
things are only as secure as the default option.
Because if you think about MongoDB,
when they kind of came out
and were getting popular,
they made it super easy to deploy something.
I had to, right?
Because from a developer experience,
you wanted folks to just get up and running. And I've had this I had to, right? Because from a developer experience, we wanted
folks to just get up and running. And I've had these cases many times where I've helped other
companies kind of figure out what exactly went wrong. And most of the times it was because they
deployed MongoDB somewhere, but the default option is that there's no authentication.
And that's how 20,000-odd databases were out there. And then, you know,
boxes came in, scanned it, got in, because the port is well known. No one's at real fault
here, right? Because by default, an even simpler example is your modem. Username is admin,
password is password. You always get that. At my parents' place, they've never changed
it until I went in and figured things out
and gave them the right,
you know,
you take away the default,
which is insecure,
and then put in something
which then causes
a usability issue,
but at least, you know,
it's secure.
So that trade-off between
what's easy to use
versus what's secure
but a little more difficult to use.
How do you understand
how it works, right?
Because like that, that default password on the router,
it's probably okay as long as it's not open on the WAN, right?
And that's what default providers started doing.
It's like, oh, look, all these routers are available
on WAN access with admin, admin, and guess what?
People can do whatever they want.
And those defaults, yeah, that matters a lot
until it's like, did I plug in the data center
or did someone else or did I click a button?
And where does it in for BoxEHQ, like what sort of defaults are you changing there?
What are you actually relying on a developer to understand and know versus just doing it for like, hey, guess what?
Like Mongo's insecure by default, or at least it was, right?
Like, so we're just going to do that for you.
But you have like three more steps to set up authentication.
You need an environment file, something like that.
Yeah, exactly.
So that's obviously fairly, you know, depends on the context as well, because for us, that's
secure by default.
I think we've learned it over time.
Right now, of course, Mongo is also patched set up.
So, you know, by default, you will have some kind of a reasonable layer of security, and
then you can kind of build
upon that. So I think that kind of awareness has already come in. We also had cases where,
you know, Internet of Things got popular maybe a decade ago. And because, I mean, from the outside,
we always think an attack is quite sophisticated, mainly because we don't truly understand,
you know, how that actually went about. But the reality is people get in through very simple things.
And they try a lot of times and they find one or two weaknesses
and then it kind of cascades into more things internally.
I think classic case was a fish tank in, I think, a hedge fund
or a casino, I forget exactly.
And that was IoT enabled and connected to Wi-Fi,
and that was open.
So they got into the actual internal network
on the back of a fish tank that was connected.
Could you imagine, right?
Who didn't VLAN that network, right?
Come on.
That's not a default in most networks, though, right?
Like, VLANs are, like, something that you definitely, you know, because every network is designed, right?
It's flat by default, I suppose.
So not secure by default.
And then you unflatten it by VLANing
and you would have an IoT VLAN that has blocks,
allows and disallows, of course.
But I mean, who's responsible for that, right?
Like at a startup, it's flat and who knows,
but like does BoxE HQ VLAN my network? Probably like that. I still need a network admin for that, right? startup, it's flat and who knows, but does Boxee HQ VLAN my network? I still need
a network admin for that, right?
Yeah, exactly. And even if you did all that
and somebody installs it
and you don't really know then, because
as the person responsible
you may not know that there's a fish tank that's
labeled, unless you have the right tooling to
scan that and say, okay, now we've got another
asset that I did not know about.
So I think that's why I was talking about the layering, because you've got your external
security, then you've got your internal security, and then more
deeper things like infrastructure and things like that. Because ultimately, if somebody
gets into your infrastructure, you're in big trouble anyway. They can do whatever they want
to. And every layer, you're trying to protect them from
getting deeper into your stack.
So the way we think about it very simply is building blocks, a bit like Lego, right?
Who's constructing it and where that kind of responsibility lies. And of course, it's a very
large spectrum, as I mentioned, and our entire focus is on not so much preventing hack, but assuming that you will get hacked as a baseline.
And then you need to think about, is my product truly secure if they get to this level? Do I have
the right set of primitives there to prevent that? Like, for example, escalation of privileges,
can they get to the user management database? Can they get to our sensitive data?
So that's kind of the different building blocks we think about at a high level and then kind of
try to make that as developer friendly at the same time having those reasonable secure by default.
And as a power user, then you can build upon that with your own security construct. So that's kind of our approach. Trying to be opinionated, of course,
so that you have, that's why, you know, the minimum viable security that
we spoke about broadly. And this is an initiative from
Google, Security Scorecard, and a bunch of really great
security companies. They call it the MVSP. It's minimum viable
secure product. And the fundamental
idea there is that you need some level of basic security in place at a minimum for everyone.
And then you can build upon that based on your context, the vertically on end,
the kind of requirements you play with. So that's how we think about it. We give you those
minimum viable security blocks,
and you then have that foundation to build upon.
Sorry, I think one of the interesting things
that I learned since joining,
because I'm new at Boxee,
like I'm here for like two, three months maybe,
is the significant difference
between authentication and authorization and where um jack saddle jackson
plays the sso role so that's your single sign-on role and where that is focused on authentication
not authorization and i think the authentication is this first level of security almost where
this gets you into the system but at that point the system then does not know it
knows that it's you but not what you're allowed to do yet so that's that was interesting so i don't
know if we want to talk a little bit more about the nuance between authentication and authorization
because i found that when i've spoken to people before is that oftentimes those are used
interchangeably and when they're actually nuanced and different the one gets you into the system the other one deals with now that i'm in
the system and i'm i've been authenticated that i am who i said i am what can i do inside the system
is that something that you find developers are writing often that interchange between things
is that something that's like a common thing for developers that are using boxy hq to like hey you know i do this all the time i need this help or Is that something that's like a common thing for developers that are using
boxy HQ to like,
Hey,
you know,
I do this all the time.
I need this help.
Or is that something that's like kind of specialized for,
Hey,
we set this up once.
And then once they,
they are,
you know,
we have like a structure in place for authorization and we can say,
Hey,
we,
we trust that someone else said you,
you are who you are.
They have a key to the door.
Now they're in the car.
Do they have a driver's license? Can they actually drive the car? Those are different things that like my kids can are who you are. They have a key to the door. Now they're in the car. Do they have a driver's license?
Can they actually drive the car?
Those are different things that like my kids can get in my car,
but they can now drive the car.
Is that something that you're looking at and that's something that's common?
Or is this more of a framework helper for people that are like,
hey, what can I do now that I have access to the application?
Yeah, so there's, I think, always two aspects to it.
One is more infrastructural in the sense that what is the foundation there, right?
Can you give folks the building blocks to say, you can now build, for example, an authentication
policy around this?
So in our case, we do that with just purely enterprise single sign-on, which is a way
to get into any kind of system on the back of
well-known established identity providers like Opta, Azure Active Directory. And just to even
do that, you have to decipher all the protocols in between how they talk to each other. There's
massive amounts of security vulnerabilities on the back of that, simply because of the protocols involved and not necessarily knowing how to do it right.
So even within authentication,
to get authentication right is a lot of work.
You can make things work,
but then are you doing the right things
in terms of being as secure as you can?
Of course, there's nothing like 100% security,
but as much as you can.
Would that be a maximal security viable product?
Is that like the opposite of minimal?
I think that's pretty hard because, you know,
if you look at the breaches,
even the best companies in the world get hacked
and they've got all the resources, right,
that they could think of.
So I don't think there's a maximum level to it.
It's more your risk management around it
and how much you put into the most priority risk that you have.
So it's always that trade-off in security.
So you can have minimum, but I don't think,
because the maximum is never being able to get into your system.
If I unplug it, it is maximally secure, right?
Exactly. Go offline and don't let anyone into the room
then. But at that
point, it's pretty unusable.
So, and
because systems are evolving so quickly
and you kind of have to, I mean,
like, just over the holidays,
right, even though I didn't take, like, a long break,
but you come out of it and
the number of NPM updates you have to do is,
like, insane.
Who was shipping code?
Who was doing this?
Dependabot, I'm on break, all right?
This is something, just chill for a minute.
You mentioned rights.
You mentioned authentication being right, but systems always change.
Is there a right and wrong is there a a there's a right for there's
a correct way for right now or for this level of access because i feel like if there was a solid
definition of this is the right way that's the wrong way the right would be default and i would
hope it would be default but i don't think it is and at least from my experience that constantly
changes depending on the people that need to access the app,
how the app is interfacing with other applications. So how does BoxEHQ or shifting lefts
move that definition from, oh, this was right last week, but this commit changes what the new
correct way of doing it is? Exactly. So there, I think the important bit there is a context,
because you're building new things, requirements
now change. Obviously, security also changes with the incoming requirements. A lot of the times,
you're just getting something out, but not necessarily fully thinking about
what are the implications of doing that, right? So in a lot of ways, first order effect is very
easy because you kind of see it, right? It's cause and effect. But second,
third, higher order is very, very difficult even for people with experience, right? Because a lot
of the times you may not have seen it or it manifests in a very different way,
which then leads to other things, right? So that second, third order, higher order effects is
quite difficult. Security by default is an example there because why would you think about that
when you do something like a default option?
So in many ways, that evolves as well.
And so does your security on the back of the resources
available to you.
So you're a startup now, you know,
you've found product market, but you didn't start to scale up.
Then it's a way different way to approach security because now, you know, you're popular, someone will try to hack you,
you know, denial of service attacks, DDoS attacks, because it's so much cheaper to carry out attacks
today than actually prevent it. There's a huge differential right cost differential. That's kind
of how you start to think about this. And that's and that's how security evolves for you as well as a company.
As you mentioned, is there a right way?
There's always the right way for the moment and that then quickly changes.
That's why shift left as a process is also important because you're constantly thinking
about it.
Now that the vectors have changed, you now say, okay, actually now I have to go back
and rethink a lot of things that I did not right.
And can I get it to a phase now where I get it right enough that I can carry on to the
next thing?
So that trade-off between making something work to making something secure is quite a
broad spectrum and changes very quickly on the basis of risk and resources
available.
And sometimes you know it's there, but you know that it's not exploitable.
So in which case it's fine because you've kind of dropped that risk down.
So that's how everything you do is kind of evolves.
And it's iteration, right?
At the end of the day, it's security as well.
For sure.
And that's why I think it's important to think about different layers as well. You can't just
make your product secure without that external testing, that external approach to it. Someone
who doesn't know your system might actually have a much better understanding of how to break it
than you who are in the weeds and think that you have everything covered.
Very simple things, right?
Like API, if you think about API keys,
you may not have thought about everything.
So you let someone else come in
and kind of, you know, do that, right?
So that could be tooling,
that could be people, process.
That's why that is important
and you have to do it continuously.
Otherwise, it just breaks down eventually.
Always be improving, Kaizen.
We do around here at least.
Absolutely.
So it's super hard, but I'm going to give it a try.
I'll start with...
And then... and then
that was pretty cool
hey friends this episode is brought to you by our friends at Sanadia.
Sanadia is helping teams take Nats to the next level via a global, multi-cloud, multi-geo, and extensible service fully managed by Sanadia.
They take care of all the infrastructure, management, monitoring, and maintenance for you so you can focus on building exceptional distributed applications. And I'm here with VP of product and engineering, Byron Ruth, and David Gee, director of product
strategy. So when you think about connectivity being the first thing to consider, someone push
back on this and say, we'll think about it later. What competes with a mindshare of connectivity?
Just like an HTTP developer, you actually just download and run the NAT server.
Whereas an HTTP developer, if you're building an HTTP set of endpoints, you typically have to
implement or use an HTTP library. And then whether it's a Go standard library, Python,
whatever it is, and you're actually implementing endpoints that register into the HTTP server.
And then now you have to go deploy this HTTP server and ensure that it's like performant.
So it's a slightly different model, but like you download the NAT server, it's a standalone
binary.
It runs on, you know, the majority of platforms.
And then you have a handful of client SDKs across all the major languages.
You download that.
And we even have a higher level API that is akin to what HTTP developers have of like defining
a handler, for example.
We just call it our services API.
And you basically have a few boilerplate things that you register your handler in the NATS
context.
And out of the box, it actually supports sort of a general request reply setup.
And then you get all of these other benefits out of the gate. But the experience and like the onboarding is arguably just as simple as any other
HTTP onboarding, with the exception that you're technically deploying a client application that
implements these NAT services in addition to the NAT server. But that's where the Saniti Cloud,
it's already managed instance, you know, and we even have the demo server for you to just try it out. It's a public endpoint that you can literally
connect to. So you can still build a simple client application, use the demo server as the
endpoint, and then you can play with that and use that as sort of the server deployment.
Well, if we talk about it just from, you know, the central view of applications,
forget networking, all that kind of packet-based stuff, you were calling them HTTP developers, which kind of stalk instead of API devs. I mean,
what do people do? They either glue it together at a primitive level. So the primitive being HTTP,
they move up the stack in their mind's eye and they go, oh, we're going to do some gRPC,
which is kind of still point to point. So it's a lot of point to point stuff versus broker
assisted connectivity, which is way simpler. You connect to an endpoint, you get told about other endpoints.
It's like connecting to a hive mind.
What we're trying to do is move people away from coordinated point-to-point connectivity
to easy connect to anything securely and connect to your other stuff securely.
Instead of having to coordinate the whole rat's nest of where to connect to them,
then you've got to negotiate, well, what do we do then?
Now we've got to go get the schema information and can we even connect
to this thing? And does it even work? And you know, what version is it and all this stuff.
And what we're trying to do is transform that and flip that to Unify to make it much simpler.
So I think we're trying to go from a rat's nest of point-to-point connectivity in the application
space to making everything on net. And it's like connecting to a hive mind. And what we're kind of
asking people to do is think about applications the same way you would video conferencing. So if me and Byron
are going to have a chat, we might do a huddle on Slack or jump on a Zoom or something. But if we
want a colleague to join, we ask them to join the same course. We can have a point to point
conversation by the same medium, or we can have a party line by the same medium. So, you know,
it's request, reply or pubs up, but it's on the same platform. We don't care about what Zoom
server we connect to. We join, we to the service, and we coordinate our communications over the fabric.
There you go.
Yesterday's tech is not cutting it.
NATS, powered by the global multi-cloud, multi-geo, and extensible service,
fully managed by Synedia, is the way of the future.
Learn more at synedia.com slash changelog.
That's S-Y-N-A-D-I-A dot com slash changelog.
How does security as code shift left, I suppose, the shift left?
Because security as code seems very similar
where you just treat security like any other workflow
when you're a software developer the full entire lifecycle.
It seems like shift left, but how does that change the lens
of how you look at security when you think about that?
Yeah, so security, I think that's quite new as well.
You know, security as code in itself
is really about automating certain things
which fundamentally you couldn't do earlier
because you had not built it in the right way
to be automated.
So an example is, you know,
we're talking about authentication authorization.
Let's say you build an authorization engine
or you pick one off the shelf somewhere.
That fundamentally means you build a policy around it.
And now that policy, of course, will change over time.
But then because the policy is now in a way
that actually can be embedded in code,
it's quite easy for you to go in and automate tests around that.
So now it certainly goes from just having been a policy
to you being able to tweak those things around.
So that kind of is an example of security as code, right?
Just like infrastructure as code.
Earlier you built infrastructure, but didn't have a way to replicate it.
Infrastructure as code now allows you to kind of bring up the same set of instances if you lose something.
Same way with security.
You codify it so that you can kind of rebuild the whole thing again and then tweak some parameters if you wanted to.
Let's say you had to add a new role. And now you're in the position to say, okay,
now with this new role, can I go back and test everything that I had before and then improve
upon that? Because now I know that, for example, if you add a billing role,
then within your application,
you know what is kind of segmented off.
And now all your tests can kind of utilize
that security as code
to kind of automate things around it.
So security as code is really just
giving developers the ability
to kind of automate things,
as opposed to kind of being a utopian way to say, we can change everything around it,
because that's fundamentally not the case.
Because every time you change something, you need to have enough context around it
to kind of do the right kind of testing, right kind of, you know, release cycle,
and post-deployment as well, right?
Are you testing fully end-to-end?
So that's why security as a code is fairly new.
You'll hear about it in DevSecOps.
That's kind of more after you've written code
and it's going through your CICD pipeline to get deployed.
And at the same time, you can have a similar thing within your product
on the back of, you know, policy.
And the interesting thing there is because fundamentally,
you know, your product has usage on one side,
but also there's compliance, potentially,
or, you know, regulatory frameworks you deal with.
Most of the times you need to collect some kind of evidence around the fact that you're doing
something correctly, right? Or at least you're following a process around compliance. And most
of the time, this is, you know, just boring evidence that you're collecting, like, for example,
access control log, or the fact that, you know, you have an access control policy, but who understands it?
Is it just the product people
or can your compliance manager also come in
and get a full understanding of what this was about?
So in many ways, security as code
helps you make that bridge,
that jump from people building it
and know what it is about,
know the full context.
Saying from an external point of view,
can I now automate that piece and let them know
that an admin is responsible for XYZ
and here's evidence that will follow that.
So that's where we see that the real value of security as code.
You're automating things on one side,
but you're also opening up evidence gathering on the other side,
which is a crucial part of compliance.
Now, the first automation I ever wrote was a wiki page.
I wrote documentation.
Documentation was the first form of automation
because I could write it down once
and someone else would do it.
And that was automation to me
because I could say,
hey, I need a new server set up this way.
I would point them to the wiki page and they would do it.
And that was the best, the very first automation.
And that had all the context in there.
It had the screenshots and it had the paragraph of why we checked a box versus not checking
a box.
And that enabled someone else to go do that work.
It was automated for me, not for them, right?
They had to manually go through and then we're like, oh, let's take this wiki page and like
make it a bash script.
And we can take that and we can say, hey, we can take out all of the context and we don't need to,
we just trust that the steps are correct at some points. And someone that runs the bash scripts,
they don't know what's happening. They don't know why it's happening. They just know something
happened. And in security is code sense, I feel like we're removing some of that context. We're
saying, hey, you just have to
trust that we got this right in a meeting that we planned three years ago. And now you're able to
just run the script and have this interface for, don't worry about it. It's a magical bash script
that does the secure thing. But now my question is like, who is responsible? Cause you're saying
very specifically that shifting left for security and developers writing that that security is code versus the compliance manager and i feel
like the compliance should never be a developer's responsibility a developer is responsible for
i'm doing the secure thing by default that i know with my context but as we're talking we say layers
right like the infrastructure layer
changes, they update OIDC on the layer below you and your thing has to change and the person above
you has to change. And so the only piece I wrote was in the middle there. And so now somewhere,
someone, a compliance manager or some team needs to go through and say, hey, does the full path
through all of those layers still meet the requirements of whatever security posture we need
to have, whatever compliance checks, whatever regulations we have. Someone else needs to be
able to do that full stack because their context isn't what does the application do? Their context
is, is this path and this token and whatever, is this part secure? Is that right? Yeah, yeah,
absolutely. So we'll take an example, right? Like let's say
you're building a high frequency trading platform. I would never do that. That sounds difficult.
Let's say you are.
There you are. Let's say your key metric is latency, right? You need to reduce latency as
much as you can. But there's a flip side to when things go wrong. There's obviously a
financial aspect because you're losing money now because your trading system is broken.
But then there's a lot of regulatory burden on the back of that, not just for you as a company,
but externally as well. The SEC is going to come knocking on your door. So now you've kind of
moved that whole this one from a system broke and wasn't available to why did it break?
What went wrong?
Was there a security implication of this breakdown?
Did something get stolen?
Was it a deliberate attempt to kind of, you know, benefit financially for someone?
So you start to get into all of this external context that your product may not have.
So you now suddenly see that what is traditionally an availability incidence in any other B2B context
is no longer that, right? Because of the regulatory environment you're playing in.
So there, the burden to do this kind of automation plus knowing the context,
you mentioned documentation.
They do a lot here, right?
Because error codes get documented.
You kind of have to follow that path on what actually went wrong,
which system spoke to the other,
and that whole traceability around
where did the request start,
where did it break down,
which system was responsible.
And on the back of that,
figure out everything else, right?
So combination of logs, your own metrics around it, the code as well, because at that point,
you need to know the exact code that did things. So there, you see that the burden around security
as a code is very different from, let's say, someone building a CRM application, right?
That kind of context, security, I mean, anything as code will help you there
because, but then, of course, you know, you've written more code.
So what happens to that as code that you've written?
Whether it's a structured as code, security as code, or, you know, ops as code.
So in many ways, that's why compliance exists
because they're now trying to say, here's your process. that with your process, your people,
the regulatory burden you carry. And that also changes the pace at which you can move as well.
The way to deploy a trading system is way different from traditional SaaS application,
where you can have multiple deployments a day because you can always roll back, come back.
So in many ways, as you said, documentation, you write documentation, it gets old. Then there's a disjoint
now between where code lives and what the documentation says about it. So a lot of times
it is about how close you can couple them because when one changes, you're then automatically forced
to change the other. You're absolutely right. Like documentation does become,
it's a static thing and it doesn't match reality.
But the people we hire are like the reason,
reasonability of like,
oh, I see where they went with this.
I see how it is now.
But my infrastructure as code does the exact same thing, right?
Like my Terraform code isn't the exact state of reality
for my infrastructure,
but even it's not like some of it can be changed,
but also I'm not holistically defining every last piece for my infrastructure. But even it's not like some of it can be changed, but also I'm not holistically defining
every last piece of my infrastructure
because that's too much context that I need.
And with Kubernetes,
we do what's called infrastructure as software.
It was a term my co-author and I coined
a few years ago, but it's like,
it's continual reconciliation
of what does the state of the world look like and what
did we want it to define as?
And as soon as one of those things get out of sync, it goes and it changes it, right?
GitOps is the exact same thing.
GitOps does infrastructure as software where it says like, hey, that thing that's applied
didn't apply back here.
I either need to make a commit back to Git or I need to change the world back to what
I think it should be.
Is there a tool for infrastructure as software for security that says,
hey, we're going to continually reconcile
to keep that state as close as possible.
So when the human looks at the code in Git,
it is reasonably assumed
that that is actually what's in the real world.
Good question.
There's something that exists today.
I think it's a combination of things, right?
Because there is tooling today,
but it's abstracted away, right?
You spoke about Kubernetes. There's a lot
of tooling around Kubernetes cloud native.
Then you think about cloud.
There's a bunch of things around cloud security
that you can plug in. But
again, each layer kind of does
specialize in what it does.
At the end of the day,
for you externally, you're
thinking about cloud security, you're thinking about product security, application security across your DevOps pipeline, and then post-deployment and all of those things.
So you have today the tooling that's fairly fragmented, and that's why security is quite an exciting field today, it's not yet reached a place where, like infrastructure was, right, a decade ago,
where a lot of exciting things were happening, but you didn't have that holistic approach to it. And the question whether you can get that is also a big one, right? Because
security is very fragmented. Your responsibility in each layer is quite different. So you get tooling
in each place. And the bigger question that we are also addressing is,
can all of that be brought under,
if not one single umbrella,
but at least helping join those pieces together,
which today is done quite manually
on the back of compliance.
You want to get a SOC 2 or an ISO 27001.
Again, compliance is not really security,
but for a lot of practical purposes,
they kind of blend together. One doesn't necessarily mean the other, but it's kind of
assumed to be. That's why the larger companies will say, do you have this? In which case,
we check all our boxes and we can carry on. Because today, there's no easy way for them to
actually say, does this company have true internal security?
So they kind of rely on some external signals to say,
okay, they think about it, they're serious about it,
therefore we trust them with it.
At a very high level, there isn't one tool
that kind of unites everything,
but you have enough tooling in each area of security
you're dealing with, which then has to be brought together.
And that's why that's so hard for the CISO and its team today
because they are not building things,
but they're defining security around it.
And I think that that mismatch is what is causing
a lot of the context gathering that is quite manual today,
which on the back of security as code should improve as you kind of go along, because as we saw, you can kind of expose a bunch of
things to the other side saying, okay, now automated, you can gather X, Y, Z and feed
that into your compliance.
Also, maybe with the shiftless thing, I think other than trying to shift things onto a different
crowd, splitting it off saying like, oh oh, this is another responsibility for you.
I think part of that is maybe just changing your mindset.
So you're kind of thinking about security.
You don't necessarily need to be a security specialist, but at least it's part of your thought process when you do something you're kind of like you stand back for
a second you're like what could i potentially compromise by by not doing this or by doing this
thing it's like if you use a llm and you stick your security uh your what your id number or
whatever in there like now you've exposed this into where does it go before you do that you know
you stop and say like if i put this in do i even know where this goes like if you have a key like a api key access key like you have to think about if i put it just
in git and it goes to github this is not a good thing i don't want that that's why we have tools
that'll say whoa whoa don't commit this this there's something in here that that that said
so i think it's thinking about that but then i think to what you were speaking justin this whole thing about syncing it made me think of
react and all these new things where like this state no you think i know i'm sorry you know like
this whole idea of when something changes like everything has to be synced back so that your ui
is a reflection of your of your data and all that kind of stuff. I think one of the ways that this
happens is like if you think about a system where you have a single sign-on process, so you've signed
on, but now it brings in a bunch of users and things can change externally. So now how do you
ensure that the central system, the SaaS system, for example, is in sync with all the things that
happen outside? So I think something like directory sync comes into
play there, where it has a responsibility of picking up on webhooks that happens, that sees
this event happened, like somebody joined the company, somebody left the company,
somebody was promoted. And so as you are promoted, you now have more access and this needs to be
synced through. But then at the same time, as all of this stuff happens, there needs to be seen through but then at the same time as all of this stuff happens
There needs to be some kind of auditing and that is where something maybe like ordered logs can come in
Which collects all of this action that happens and then over time if something an incident happens
Now you have a source of truth so to speak that you can interrogate and ask like okay on this day between these hours
What happened within my system
that caused this breach to potentially happen?
So, I mean, I said a whole bunch of things there,
but I think the shift left,
security is different to that,
but the shift left thing is changing just your mindset.
So just when you're doing stuff on a daily basis,
you kind of just stop for a moment
and consider whether this,
like if I go back to front-end code,
like, if I'm going to have a button,
like, I should just use the HTML button, right?
I shouldn't use a div and make it be a button.
Don't overcomplicate it, please.
If you're listening, you may remember the early days of the internet where open networks like HTTP and SMTP led to an explosion of websites and online communities.
Building a fan site and connecting over shared passions led so many of us to careers in software. Back then, it seemed like anything was possible because the internet of the 90s was
built to democratize information, not consolidate it with a handful of big tech companies.
Read, Write, Own, Building the Next Era of the Internet is a new book from startup investor
Chris Dixon that explores how network architecture plays out in our online lives and the decisions
that took us from open networks governed by communities of developers
to massive social networks run by internet giants.
ReadWriteOwn is a playbook for reclaiming control
and for reimagining applications so users can own, co-create,
and even profit from the platforms they use every day.
From AI that compensates creators
to protocols that reward open source contributions,
this is our chance to build the internet we want,
not the one we inherited.
Order your copy of Read Write Own today
or go to readwriteown.com to learn more. What you're saying is resonating with me because I kind of feel like it's a win versus who
and we began the conversation with shift left and the idea was you know deepak you mentioned it's
not after deployment it's pre-deployment and so we began with the conversation assuming oh we're
putting more burden and pressure on the developer which yes kind of unless you do with what you just
said which is it's more of when versus who it's at development not so much by the developer it's
meant to happen prior to going to production so that you're not shipping the defaults in in the old mongo days or other
scenarios like a modem you've got insecure defaults it's so that you can think about those
things prior to shipping and it doesn't mean that it has to be the developer necessarily it's
the team behind development which includes, but also may include compliance. It may include this, you know, MVSP, which is more of a kind of policy in a way of how to secure
cloud applications. And as I'm reading this, there's like security headers in there. There's
how you allow your customers to test the system. There's all these sort of protocols that are in
there that can be tested. But this is something that the whole team says, okay, this is what we
want to do
when it comes to shipping a cloud application that's secure.
And that happens before development,
that happens during development,
but prior to production.
So it's more of a when versus a who.
That's kind of how I see your perspective shifting,
at least the way you describe it.
Absolutely.
It seems like a lot of that
really depends on that minimal piece,
right? Like the bare minimum unit, because like the context that I need to remember as I'm writing code is very different if it's a trading application versus my personal website.
Way different, yeah. Yeah, that minimal piece. And it depends on like expectations, right? Like,
am I going to be expected that this will never get hacked or this is only secure against this level of sophistication or something?
And even what the user is going to expect out of it.
Because just an example of users, I go change my password.
And I've worked in enterprises for 10 years now.
And every single one of them, I go change my password.
And the prompt says, please wait 24 hours for your password to replicate across the entire ecosystem of all of our apps. And like, that is the expectation of a
user that I had that I'm like, oh, I may go to a site and have to use my old password for the next
day. And that is okay that that state is not immediately synced because of the amount of
pressure and scalability and all these other things we need to say like, oh, if I need that
replicated everywhere right now, guess what?
That's actually like a big, a big task to have everyone scale up their systems to get,
receive all those webhooks versus just like a little prompt on it that says, hey, just
like wait 24 hours.
And guess what?
Like all the scaling problem goes away.
All of that like expectation of this is everywhere right now.
And it's going to be the same, not security, but the same password everywhere can be just different based on whatever minimal means in your context, at your application level,
in whoever's writing that code. Because I can solve those problems at different layers.
Well, this is a minimum viable secure product. So in this case, the word minimum is still used
there, but it's five scrolls for me, three or scrolls so it's a lot so this is minimum in
comparison to a cloud application is that you know metric system scrolls or is that standard
system scrolls i don't know if this is uh it's my digits scroll but that's that's a cloud
application so i would say a minimum version of this for a personal website would be like one
liner like don't get hacked yeah or i hope. Don't get hacked. Or hope it doesn't
get hacked because it's personal. Not that big
of a deal unless your email's attached to it
and you've got other things happening as
a result. But anyways, I digress.
We've been pretty bad at this
as a software industry. We've stored
passwords as plain text.
We've done...
I mean, it still happens. It's crazy. But
there's legacy systems that can't change.
And your password complexity, it causes more pain than actually securing it.
Because from a usability perspective, you're just denying the user.
And then they just take the easier path out, right?
So GDPR, for example, the reason it had to come in was, it's just common sense, right?
Secure the data that you collect
and don't use it in bad ways.
You would think that's common sense,
but I don't know.
Yeah, exactly.
But that's the reason, right?
The reason that came about
is because companies were misusing it.
They were not doing the right thing.
They were just throwing it out everywhere.
Could they even use it the way they used it?
I mean, we don't even know today, right, where our data sits and how it's been used.
So GDPR was kind of that attempt to say, okay, now just, you know, hold on, think about it.
You've collected a lot of data, but, and it's not even just about sensitive data, right?
If you hold 6,000 email addresses, that's a big problem because that leak is effectively,
and it's also the quantity you're collecting, not just specific things.
So I think that's why all of these compliance regulations exist, because we just don't do it the right way until we're compelled by an external event to kind of make that happen.
So security is kind of has to be forced upon, right?
Otherwise, you kind of happily say, you know, if I don't have to do it,
I won't or I'll do it later.
And that data kind of comes.
Probably never.
In many ways, from an external point of view,
it feels like it's slowing things down,
but it's more about saying, think about it.
If we don't force you to think about it,
you're never going to think about it.
So I think that's where we've been pretty bad.
And that kind of mindset that we were speaking about has to kind of come back because, you know, we have a lot of responsibility, right, as data collectors. And that's the crown jewel that we're collecting. And, you know go point to the hard drive that had the email addresses on them.
Like that's where it is. It's not leaving here,
but now I can't do that. Right.
Like I have like key value stores in cloud flare. I'm like,
where does that exist? I don't know. Like what,
what is my reasonable amount of expectation to secure that data?
I was like, well,
like I have an access key and that's how I get to it and how I can write to
it. So no one else should be able to read it.
But I have to trust that my provider is doing the right thing beyond the layers that I have control of.
Like I have the access key and they do everything else.
But like dynamic is very different from a globally distributed key value store versus a hard drive in my closet.
Like those are very different things around how I think about security
and what my responsibilities are for that security.
And so I guess it really depends on, again,
those layers of like,
I have a key to the door to the data center
and no one else has card access to it
versus I have a key to the API
to give me access to the data versus,
you know, like, and those are just bits, right?
Like if someone stole those bits,
I would never actually know.
But if I lose my keys, I know immediately. Like my physical keys, not the, like, and those are just bits, right? Like if someone stole those bits, I would never actually know. But if I lose my keys, I know immediately.
Like my physical keys, not the API keys, right?
It's very different how we go through those mindsets.
It's similar too, also to the build versus buy aspect, right?
Because in the aspect of the data center
and pointing to the hard drive,
you've likely built the model.
You've built the servers, you've racked and stacked,
you provisioned all that good stuff. Whereas from built the servers, you've racked and stacked, you provisioned all
that good stuff. Whereas from the API perspective, you have the key and what's behind the API is a
black box to you. You're focusing on trust. So you've in that scenario chosen the buy versus
the build. And I think security is one of those things where, wow, you really have to have a lot
of awareness and knowledge to do the build part and not a ton
aside from judgment and maybe peer pressure to do the buy part right to buy a decent solution is like
okay what are the what does google tell me when i say work os versus it is boxy hq in that list
like i know michael greenidge from work os big fan of them and y'all are an open source solution to
a similar problem where they're proprietary.
And then it's like, well, do I trust open source more?
Do I trust the proprietary less?
Do I build?
Do I buy?
Do I want to even learn SAML?
Do I want to learn scheme?
Do I want to learn all these protocols
that do I want my team focusing on that
versus the product we should be building?
And that's where you really sort of
almost are forced by market nature to buy versus build.
Even if you're not that knowledgeable
to even know what to buy,
you have to buy the most trusted default
or trusted solution that comes up in.
Yes, I still use Google, chat GPT isn't for everything.
So when I am learning something like,
hey, what's well-known brand versus?
And I usually choose from that
because that's what the wisdom of the crowd has said.
But let's open up that, like the buy versus build,
because I feel like there's two different paradigms there.
You can be knowledgeable enough to build, but does the market allow you to even do that?
Maybe you have to be at a certain scale to say, we should build this versus buy this.
The first stepping stone there was really infrastructure, right?
The data center, shift from data center to clicking a button on AWS.
Right.
If you didn't do that, let's say this was like the early 2000s, right,
when AWS came out.
If you chose to go to a data center,
that means you were like at least a week or two behind, right?
Just at the get-go from your competitor who chose to build on AWS.
At the same time, as you kind of had to maintain it,
you would probably lose more time as you kind of evolved.
So it kind of just immediately becomes
a competitive advantage of speed.
And today we know speed for a product is super important, right?
You kind of get left behind.
If you're not at the right place
at the right time
with the right product that you have.
That fundamentally, like, you know,
Stripe changed that fundamentally as well.
You were building payment systems,
which took forever,
didn't quite work as well,
to now just integrating Stripe
and, you know, carrying on, right?
So what is core to you as a business
versus everything else?
Twilio is a great example as well.
You wouldn't build audio video
today. Just pick up Twilio.
Not if you're smart.
Exactly. So I think that is
that whole
question of what am
I building upon?
And there's other questions that come up.
You're dependent on a platform and all of that.
But as you start off, you're kind of then saying, if I had to build everything like
Twilio and then do my value out on top, it's going to take me forever.
So that has changed a lot now.
You can roll things out over a weekend with almost no cost.
That, I think, has become a competitive advantage question now.
What do you build?
What is most essential to your core
versus anything that's undifferentiated
and commoditized, I think.
So that, I think, is that buy versus build question
that comes about.
That's why SaaS has kind of taken off right in the last decade.
I think average company uses about 80 SaaS vendors on average, which is quite a lot.
It is a lot, yeah.
We see that as well.
It's a small company.
We have about 30, I can think of, that we use internally.
Now, how is that different from what it was in the 90s, right?
Because I would go buy box software.
And that was like the 90s version of a SAS was like, I went to a, you know, circuit city or whatever. And I
bought a box off the shelf and that was the software that I was buying. And I would run that
forever as like, this is the software I didn't have to write. And I feel like the SAS term throws
around as like, it's something new, but really it's been around for a long time. That model of
like, I don't have to build this part of it. I don't need to buy, build my own Excel. I just go buy Microsoft Office in a box and I paid for it once.
And I know plenty of people that use Office 2003 for two decades. And it was the thing that ran
for them versus a SaaS product where they have to keep updating and retraining and doing things.
There is an amount of, I can buy this once and own
the software and move faster long-term because I don't have to retrain people off of when I remember
when office 2007 came out the ribbon, right? Like it was a huge thing. Like I was working in the
help desk and like, I wrote, I don't know how many pages of documentation about like, Hey,
guess what? The ribbon is here. And you're going to have to change how you use Microsoft word
because everything is different now and nobody wanted to move to it. And you're going to have to change how you use Microsoft Word because everything is different
now and nobody wanted to move to it.
And so it's like that slowed us down as a company because we had to stay with what the
SaaS was providing.
You know, we had to upgrade or whatever.
And I feel like what you mentioned before of like, if you go to a data center versus
AWS, like you're going to be a week behind, except for the fact that like, if I go to
AWS, I have to
learn an organization and I am and security groups. And I have so many more layers to learn versus I
plugged in a machine and installed Linux. And when I was working in a Disney animation, we,
we had a render farm and we would, we could rebuild the entire render farm, like four or
5,000 machines in a day. And I went to Disney plus and we were in AWS.
And if I wanted to replace our four or 5,000 machines, that was like six days. And like the
difference between like how fast, like to re-image and to refresh the system, there was a speed
difference there. And owning the hardware and owning software is a very different mindset than
what a SaaS and someone else can say what you can or cannot do and how fast you can do it.
And how does that apply to security? Because I see like on your site, you have a self-hosted
option. I could go download the software. It's open source. I can run it in my data center
on my machines and I can run it for, I don't know. I don't know if there's, you know, even if it's
not supported by you, I can run it forever. As long as the hardware is there versus the SaaS,
I can get started now and see how this
works and you're going to take care of the layer below
me. Maybe I don't need that.
Maybe the actual long-term effects of running this
myself and actually just having a little bit of expertise
on how that runs and troubleshooting
it is valuable to my company
to be able to say like, oh, I don't need to open a
support ticket to fix this. I can go find
the open source and have my developers do it
right now. You bring up a very interesting point. I think go find the open source and have my developers do it right now.
No, you bring up a very interesting point.
I think to your point,
one thing we always overlook,
you know, build versus buy is can you hire the right set of people
to do that, right?
Internally.
So your choice of, for example,
today React versus something else,
typically, or, you know,
even like, you know,
Golang versus Java, right?
It's fundamentally about who can you bring on board to kind of support it, right?
Even if you build something, it needs to be supported in the long run.
So I think a lot of the build versus buy choices also happen.
It may not be the best one, right, that you pick up, but it could be the one that has
the most talent pool out there.
Or you think that
you will reach a point where this thing is going to get big. A lot of people bet on React when they
saw Facebook was behind it, saying that this will become big. So we will have enough people who
picked up the skills that are on. So I think to your point about that on-prem versus SaaS,
we can't cannot talk about Salesforce
as an example. They were the first ones
who kind of, well, they did
a great thing with their whole no software
in the cloud. You don't have
to worry about it. And today, if you think
about it, it helps you more fast.
You've got not just a CRM,
but a whole platform around it.
You can now do
pricing quotes on the back of that. You can now do, you know, quotes, pricing quotes
on the back of that.
You can align your
account management team around it.
Customer support can happen there.
So you've certainly gone from,
you know, just a CRM
to a whole range of things
within that same platform,
so to speak.
Do you find that distracting
for companies?
I mean, most companies
have to do it.
So I've seen this evolve.
Your founder led sales, and you
figured out BMF,
you hired a sales team, now Salesforce
is where they live and breathe. Nobody likes Salesforce,
but everybody has to be there for
sales,
marketing, and customer support.
Can I plug something real quick? Because
I do not like Salesforce.
And I do our sales for us.
Yeah.
And I started to use a thing called Pipe Drive.
I think it's awesome. Yeah, exactly.
I love it. I'm just like so in love with this product. Like I just want to mention it right
now because they're not even paying me to do it. It's phenomenal. And I could use it as a one person
sales show. And now if I had to scale and hire somebody next year to help me with sales,
which we've considered, I would bring them in the pipe drive and I can take a step back,
but still be involved. Anyways, I'm not going to plug them too far. They're just amazing.
And Salesforce has never, it's just been too big, just too big, you know, anyways.
And that's the distraction I'm talking about. Like at what point do you need Salesforce,
all of the features? Cause like, I can't tell you how many people over the last four years
asked me what, how do I learn AWS? I'm like, there are over 200 services. You do not
learn AWS. You learn one piece that you want to do and everything else is a distraction.
Right. And I feel like Salesforce is the same thing where it's like, Hey,
you wanted to do CRM and that's it. Like go to pipe drive. Like don't do all of the things that
Salesforce does for a large enterprise, for someone that knows they need all the feature sets.
That makes sense. But for the smaller people, for the people getting started, for the people
like, Hey, I want to solve this problem. Or indies, man. Yeah. Yeah. Like maybe Excel is too difficult,
right? Like if I'm like, go look at Excel and I'm like, actually, there's too many functions here.
I don't need all of them. I can hide most of them. But at some point, like I just need like maybe a
calculator because actually what I need, I didn't need Excel to do that thing.
Yeah.
And I think that's why we are so focused in.
We're not trying to do everything explicitly.
We're not trying to do everything.
That's why we have these couple of protocols, I think is what you call,
that we're focusing on, like enterprise SSO.
We're not trying to solve the entire authentication authorization problem.
We're trying to solve a very specific area in that. And I think that the reason, Adam, why you previously said the MBSP is so long,
I think the reason why that is so long is because that is for when you
want to sell to enterprise. Because now you get all kinds of... Right. And I
expect it to be long too. That was not a knockout. I expect it to be. I mean, if you're going to
secure a cloud application, it should be long too. That was not a knockout. I expect it to be. I mean, if you're going to secure a cloud application,
it should be pretty holistic.
So the many scrolls was expected.
Yeah, yeah.
But I think one of the reasons why it is as complex
and as vast as it is,
is because who you're building this cloud application for
is most likely a big company
that has all kinds of compliance rules around all of this.
And to even satisfy the minimum
is a big list exactly and it's it's a trade-off right like you mentioned salesforce versus
pipe drive you can move fast with pipe drive but there might come a time when you need something
that it doesn't have because there's integrations and things like that and it's possible but
it kind of becomes a trade-off then you say okay now maybe i have to go back to i mean i have to
migrate to salesforce i've seen this happen as well. As sales crew, they kind of were almost forced back into going to
Salesforce, right? Or you're doing it in an Excel sheet for a while and that's fine. And then you're
going to go somewhere. So we see this in security as well, right? It's a trade off. We turn away a
lot of prospects because we tell them that we can't do everything, right?
And if you're thinking about doing everything,
we're not the right product at this stage.
And that's a trade-off as well in the build versus buy.
It's okay.
Do I pick this up from Box.chq?
Do I go and build it?
Or do I find another vendor who does everything, right?
So these are all the questions that constantly come up.
And we obviously will
evolve over time as well. As I mentioned, you go from single product to multi-product to kind of
a platform. So that's kind of the bigger vision around solving this holistically, but that's kind
of a 10-year timeframe, right? Not those three years. And what Salesforce did to the on-prem
CRM
Pipedrive is kind of doing it to them, right?
It's the evolution
and there'll be a next generation.
You might think so, that's for sure.
I know I've ran CRMs on a desktop
in a network environment
when I had no idea what a network was
and I was just like, just the worst anyways
and running local software.
So thank God that Salesforce came along
and used the cloud to make the software run
in the cloud network versus on-prem, the LAN network.
It revolutionized things.
But I agree with Justin, like it just does so much.
You don't learn Salesforce, you learn,
can I just have the CRM?
And that's just too complicated
because it thinks you want everything.
And they have this massive scale enterprise large sales team mindset, which I suppose is more of a juxtaposition of how we think about security.
You know, is it developers only or does that shift left include other folks who can help developers in the development process to get to where they need to go?
The one thing I think that is really interesting,
at least with Y'all's Choice, is the open source nature, right?
You can go and self-host Boxee HQ yourself.
You can do, I think, a premium version of it too,
which I'm not sure how you license that,
but there's two different flavors where you can self-host.
I'm assuming both built on open source,
but then you can also go and do the SaaS version of it.
But I think what I'm trying to promote more there is less about your tiers and how you're selling it, but more so that there needs to be back to that default, that secure by default scenario where can I choose a viable open source version that is free to me, free to contribute, free to use, free as in everybody else can use it as well.
And it becomes this model that seems to be a version of yes for you all.
But at the same time, does that then make it one single point of failure?
Like if everybody starts going the boxy HQ way to do SSO
and secure their SaaS products for enterprise,
does that mean that you become more of a centerpiece for an attack?
Maybe yes.
That comes back to the build
versus buy make the right choice and like you said before tupac it's just constantly iterative when
it comes to security iteration always improving being aware and fine-tuning and making changes
as that changes but i'm also very happy that you've chosen open source can you talk about the
choice to i mean because you can kind of cannibalize yourself by yourself by being open source, right? How
did you choose that? Why was open source super important to you all?
Yeah. So two fundamental points for us when we were thinking about, you know, our go-to
market, you know, we are serving the developers and as a developer myself, and, you know,
having seen my team multiple times before, it's the best way to go and evaluate something, right?
You're either thinking about, do I build this or do I pick up something that gives me some kind of foundation to build upon?
But then you can also evaluate it without any of the having to talk to anyone, no sales, marketing language.
You can truly see it functional.
You see the value.
And that's how you're, it's like the freemium version, right,
for SaaS, effectively.
Somebody's there, we don't even know who they are,
they're trialing it, and then, you know,
they might come back around for more questions,
or they might stumble into an issue and they, you know,
they're on our community.
So that was kind of the one major aspect to think about how we're discovered.
But at the same time, we also realized that we're quite close to infrastructure and commoditized, right?
We're building on top of protocols.
We're not trying to reinvent anything.
Fairly interoperable protocols.
We wanted to keep it that way.
And that's the best way to kind of bring in contributions
from the community, right? Integrations, you know, let us support 20 odd providers,
some providers we can't get access to. We had a lot of help from community who were using that
and were trying us out and ran into trouble and could help us troubleshoot and submit changes to
make it work. Those two were kind of the main reasons why we chose to go.
And our license is Apache 2,
so no restrictions, right?
Fully free to use it.
And what we've done is in an open core model,
there are certain premium features we identified
that only the larger companies need.
And those are kind of locked off
as requiring an enterprise edition.
But that doesn't prevent anyone else from not being able to use
the full functionality of everything else.
So that was our thinking here, you know, close to infrastructure,
how developers discover and evaluate us.
But in the process, what we really found insightful was
we get used in the tech stack in ways that we didn't
anticipate before.
Because they can lift and shift us, right?
So they've used us in
places that we did not
even think was a use case before.
So that was quite insightful
to see the new use cases kind of emerging.
How do you deal with that
case where someone uses
you in a way you didn't think,
but then maybe they want to contribute what you would consider an enterprise feature.
How do you deal with open, but open to contributions?
We obviously tie that in with our roadmap that we think about at a very high level
on where we want to evolve as a company, both from a product and kind of who we serve perspective.
And all the contributions, ideas coming in is kind of evaluated and prioritized based on that.
Because there's many directions we can go, like Scott mentioned, we could do all of authentication,
but that's a very different problem to solve versus going deeper into developer security.
That's the path we've picked.
So we're quite strong about saying no to things that
we think is not
kind of core to what we're doing.
But at the same time, they have the APIs to
extend it and use it for the bespoke
use case. And wherever
we see repeated use cases, we kind of
then go out and see, there's a lot of verticals
we don't have experience in, but
it's kind of being used there.
And then we try and sit down and see if,
you know, that's a vertical we could just go after
from your go-to-market perspective.
Very cool.
I was just clicking around on the open source.
I love seeing commits.
When something is open source,
I love just digging into GitHub
and looking at the commits.
I didn't do that.
I didn't dig into the actual source.
I looked at it, but it didn't dig in.
No, I just want, Deepak, I wanted to commend you.
This is actually amazing because you have
not the most commits. Dependabot
obviously has the most commits.
Dependabot is just a 100x developer
out there. But Deepak,
of your commits, you have 516,000
lines added, lines of code.
But even better than that,
you have 470 removed.
And that's amazing. That is a great
stat right there. Like you are only 46,000 away from like breaking even on like, you could just
have zero code left. I mean, obviously it's a bit skewed by the package lock and stuff,
but I love removing code, you know, wherever my kind of sometimes a metric is a more
random there is, you know, it's better.
I always strive to be neutral in my code.
Like by the time I leave a company, I want to have as many lines written as removed.
And like that is how I know that like I made an impact of like getting rid of stuff and adding the, you know, like whatever it is.
But like break those even and get that in a neutral spot is a goal of mine.
Interesting.
There's a funny anecdote. I mean, I won't name the company I was at,
but wherever I was, they came up with this crazy metric
to just reward
people. I'm more recognized, right? Not even
monetary reward or anything. They said
the one where they developed the most number of
lines of code, so commit,
and this was back in the days of CVS, right?
Before Git. And people
gamed the hell out of it, right?
It's, you know, changed the whole formatting
to leave like two lines between every piece,
line of code.
And then it became a game of, you know,
who can game it better to get to the top.
Semi-colons go on their own line.
And then we have two lines at the end of the file.
Yeah, you can totally, like this is,
if this is in my promo doc,
I am getting more lines, that is for sure.
Well, let's leave it there then.
I think we covered most of it.
Definitely love hearing about the journey
and the security journey, I suppose,
for developers and how they think about it.
I mean, I think one takeaway for me
was really just the shift left of my shift left,
which is that it's when, not who.
I think that was really an eye-opener
because every time I hear the term shift left, always think in almost a defensive way like gosh can
developers really just do more can we just keep piling on more but i think now that i think about
it like skulk said just from a different perspective that it's really about when which
was my words uh but really synthesized from his meaning was when versus who it's pre-deployment that it's everyone's job to think about it,
but when we're actually building the application,
not once it's built and shipped do we then begin to think about security.
Because we're going to build better companies,
we're going to build better products, we're going to build better teams,
which might even mean keeping people around longer, paying them more,
profiting more, all the mores that you can put into that stack
seem to be better if you can shift that left to the win
versus just the who.
So that's a takeaway for me.
That's super insightful.
I didn't think about it that way at all.
Well, that's why we had you on here,
so we can give you some feedback.
It usually happens to some degree.
All right, well, let's leave it there.
Thanks for coming on.
Thanks for sharing this security journey.
And Justin, thanks for joining me, man.
It's been fun gabbing with you a bit, man.
It's our first time together.
Maybe we'll do it again sometime, you know?
Who knows?
Yeah, this was great.
Awesome.
Yeah, I mean, honor to be here.
It was an absolute pleasure.
Very cool.
Thank you.
Thank you so much.
Thanks so much.
Okay, it is time to shift left.
Seriously, hopefully this conversation
today gave you some things to think
about in your teams, environments,
etc. Make sure you check out
BoxieHQ at
BoxieHQ.com
and big thanks to Deepak and
Skulk for coming on the show today.
By the way, Jared and I are on the road
next week,
so expect maybe a few delays in our shipping schedule.
We'll be at that conference.
No, for real, that conference.
Go to thatconference.com.
We'll be here in my backyard in Texas
celebrating that front-end polyglot way.
If you're there, we'll be in the hallway and on stage.
Thanks to our friends at Cloudflare for hooking us up
and making that conference possible.
Coming up on ChangeLogging Friends,
we have a deep conversation with Jose Valim
on the state of Elixir
and what's happening with being gradually typed
and his thoughts on artificial intelligence.
We just can't avoid this conversation.
And I guess next week on the change,
we're also talking about AI deeply as well.
So stay tuned for that.
Of course,
a big thank you to our friends and our partners at fly.io,
the home of change.com.
Launch your app for free at fly.io,
as well as our friends over at century.
Get $100 of the team plan for free for 3.8 months.
It's kind of cool. Use the code change log to get that once again, century dot IO S E N T R Y
dot IO a hundred bucks off the team plan for free a hundred bucks off the team plan.
That's pretty awesome. And of course, the Beat Freak in Residence Breakmaster Cylinder.
You know, next week we have something coming out.
We'll be announcing.
Hopefully our conference travels won't impact that much at all.
But get excited.
A new album awaits.
Also, there's a bonus for our Plus Plus subscribers.
If you're not a Plus Plus subscriber, do so at changelog.com slash plus plus.
It's better.
It is better.
Ten bucks a month, a hundred bucks a year.
Drop the ads, closer to the metal, directly support us.
And of course, that sweet bonus content.
Okay, that's it.
The show's done.
Bye for now. Game on.