Software at Scale - Software at Scale 19 - Vanta

Episode Date: May 4, 2021

Christina Cacioppo and Robbie Ostrow work at Vanta, an automated security and compliance company with a mission to secure the internet. Vanta sets up monitoring via a set of continuous tests to ensure basic security best practices, like mandatory MFA for employees. Each test bubbles up to one or more compliance standards like SOC-2 so that companies can rapidly move their audits and unlock deals.Apple Podcasts | Spotify | Google PodcastsThis episode is special because of two reasons: I currently work at Vanta, and it’s the first combined interview with both the CEO and the first engineer at the company, which led to an interesting conversation with multiple perspectives.As usual, the episode focuses on the technology and business of Vanta, and I’ve tried to not go easy on them, even though there’s an obvious bias involved :)HighlightsMy notes are italicized2:00: “In order to work on a security company, you’d actually best start with compliance company” - compliance is a “hair-on-fire” problem for companies since it helps unlock deals, whereas security is often an afterthought. Solving compliance helps make companies safer since the incentives align better. This idea and the headache of SOX compliance at my previous job convinced me to work at Vanta.5:00 - Continuous security monitoring vs. snapshots that are double-checked in audits11:00 - How Vanta was initially built.17:00 - Should security reports be standardized or extremely customizable per company?20:00 - How does someone decide on the set of security policies? Do customers ask for advice?31:00 - How should engineers think of developer productivity for their startups? What has the impact of initial choices like MongoDB and GraphQL been as the company has grown?40:00 - At what point should a founder decide to hire an engineer? What qualities should the engineer have? At what point should the founder stop interviewing engineering candidates?52:00 - How to effectively build a brand for a security company? Experiences over the past few years. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.softwareatscale.dev

Welcome to Software at Scale, a podcast where we discuss the technical stories behind large software applications. I'm your host, Utsav Shah, and thank you for listening. Thank you for joining me on another episode of the Software at Scale podcast. I have Christina Cassioppo, I think I pronounced that correctly, and Robby Ostro. You got it. Christina is the CEO and co-founder of Vanta, and Robby is the first engineer. And Vanta is, I like to think of it as like outsourced head of security. I think that's one of the phrases you use in one of the onboarding calls.

It automates security and compliance. And we'll talk about why that's important in a bit. And can you talk a little bit about yourselves for the listeners? Maybe we saw Christina. For sure. So I've been working on Vanta for the last about three or four years right now. Prior, I was a PM at Dropbox working on Dropbox paper, trying to bring it to market and getting a little stymied by some of the

reasonable but hard to achieve security and compliance requirements. Prior to that, I taught myself to code, so a sort of shoddy self-taught engineer, and started by studying economics, which I still very much like. And Robbie. Thanks. Yeah. Christina's always giving me tidbits from Matt Levine and all that stuff. So that's really exciting. Yeah. I've been at Vanta for less time than Christina, about two and a half years. And before Vanta, I started my own company, which quickly crashed and burned. And I panicked and decided I had to join a company that was like mine, but better. And before that that I was finishing up my master's in AI and biocomputation and working at a political risk consulting firm. So been a bit

all over the place but it's been a great two and a half years at Vanta so far. That's a super interesting background and like for both of you like why work on Vanta? Like what is Vanta Salt and? What is automatic security and compliance? Yeah. So one of the many things I really like about Vanta is the sort of incentive alignment piece. So sort of our joke, just not actually a joke, is if you want to start a security company,

you'd actually best start a compliance company. And that's because for many businesses, especially kind of faster growing, smaller ones, it's really hard to stop and prioritize security, but it's much easier to prioritize the things your customers are asking for. And unfortunately, we'd like to change this, but unfortunately today, they don't ask for security as much as they ask for compliance or proof of security. And so we figured, hey, the best way to kind of get small companies and growing companies to care about their security posture is actually use it to help them kind of grow their businesses through compliance. It makes a lot of sense. And Robby, why pick Wanda?

Yeah, I absolutely agree with everything Christina said. I think security is incredibly important, but the incentives are misaligned at the moment, and hopefully Vanta can serve to align them a little bit. But on the engineering side, the reason I chose Vanta was computer security is kind of in itself a nebulous term, right? Does it mean that your systems are invulnerable to attack from evil corpse supercomputer that's trying to break RSA slash a nation state? Or

does it mean that you didn't actually accidentally leave a window unlocked? There are a ton of really brilliant companies doing work in the first category, making sure the cryptography is working, making sure that the math behind all this stuff was working. But there are very few companies doing work in the second category, which is once you've bought the expensive lock, the really sophisticated cryptographic algorithm, you have to make sure that these locks are actually in place. And these systems today are so incredibly complex, you know, thousands of S3 buckets and servers and whatever it is, no one person or no human, even a team of people can ever be confident that they've gotten all of those things. And I see Vanta as a really good solution to that.

Like all the cryptography has already been built. It'll continue to improve over time. Vanta is in a place that can actually make sure that people who don't have 17 PhDs in cryptography can implement this stuff in a way that is going to actually secure their systems. Yeah, this reminds me of like the SolarWinds hack. I don't know if you all read about the password they use for their build server like solar winds one two three like implement a password policy please and there's so many of those things that need to

be solved and that this definitely seems like a higher leverage thing to work on so what is the goal for like vanda like what does success mean like 10 years down the line like has vanda taken over the world and like what what does that mean exactly definitely taken over the world um more seriously so we'd love to get more of the software industry thinking about um security and kind of proving security what we now call compliance the way the way we do right so that means that um when you're thinking about a company security, whether it's someone you'll do business with or someone you're thinking about joining, you think about it kind of more like, I don't know, continuous, right?

Like kind of the way we think of the rest of engineering. It's not, you know, a point in time last Tuesday, someone went to the company and asked them if they were secure and the person said yes. It's, again, something closer to like kind of a status page or again, just this kind of continuous outlook. And that's how we start thinking about the security of software companies. Robbie also has take over the world ambitions. So curious what he thinks. Yeah, I have substantial ambitions for Vanta.

Ultimately, it can be boiled down to software company A can't sell software to software company B unless the first one has a Vanta. Right now, they ask for various other things, security questionnaires, compliances, all that stuff. All that stuff is a point-in-time estimate in the best case and a checkbox ticking exercise in the average case. If all companies need Vanta to sell software,

all companies need Vanta full stop. And that I think is a great place to be. Yeah. So maybe you can expand on that a little bit. Like today, it looks like companies mostly ask for something like a SOC 2 or another kind of compliance, like an ISO or like a HIPAA. Why is that not enough?

One thing you mentioned is like a continuous model, right? So how does it work today? Like if I have to get certified? Yeah, one of my favorite anecdotes is Equifax definitely had a SOC 2. And so the way these things work is you, and I'm sure they pass their audit, right? And so the way these things work is kind of step one, you write down all your security practices

and compliance or security speak, these are called controls. You sort of write down all the things you do and then you hopefully make sure you do them all. And you, especially to the Equifax point, want to make sure you do them all when the auditor comes and checks. So this is, you know, again, they'll pre-COVID,

they actually literally came into your office and would say, again, pre-COVID, they actually literally came into your office and would say, hey, pull up the dashboard, show me on this Tuesday afternoon, we do these things. And so we'd look, when you know the auditor's coming in, you're going to make sure you do all the things. I think what's harder is a few weeks or months later when there's just a quadrillion other competing priorities and things your doors and windows may or may not be locked and I think again when we're sort of doing these point in time once a year twice a year verifications we'll think about security as making sure everything's okay on Tuesday but if it was sort of this continuous status again we thought about this like we think

about a lot of the rest of engineering as something continuous, we'd actually get much better outcomes. It's not just continuous, but also everything, right? Today, it's a point in time for some things on Tuesday. Vanta is everything all the time. The way I think about it is hackers, script kitties, whatever you want to call them, are going to look at all of your resources, right? Not just the ones that the auditor happened to pick. And a sampling exercise is incredibly useful. It's better than nothing, right? But if you have 100 S3 buckets and only 99 of them are encrypted, right? The auditor is going to choose three of them. They're going to see that all three are encrypted and right? The auditor is going to choose three of them.

They're going to see that all three are encrypted and then they're going to pass you on your audit. But it's the one that an automated system is going to see, is going to fetch, and then all of a sudden Citibank leaks all of their customer data to everybody. And so Vanta checks all of them and prevents that from happening. Yeah.

Go ahead. I was going to say, just kind of building off of that, right? I think a lot of the auditor checking three of them and checking them on Tuesday made a ton of sense before we sort of lived in the cloud API world, not to be a total trope here, but kind of actually when like sort of the only way you could see this was someone came to the office and checked the servers in the closet and saw what was going on and couldn't check them all because you know had to go to somebody else the next day um but yeah in the world we're in it's it's pretty easy to write the software that checks all 100 buckets um uh and see if they're configured the way you want them configured this sounds like auditors have basically just gone from checking books to checking software like manually is that roughly accurate yeah i think that's roughly accurate yes and i can i can totally understand so like the motivating example you had was like an s3 bucket like an engineer might add a new bucket with user data it's unencrypted and just because there's not

been an audit recently that just gets lost right and that's what vanda can prevent it's like you can get an email notification or something like that as soon as an S3 bucket that's unencrypted shows up. Is that roughly correct? Yeah, that's exactly right. We in fact have hundreds of tests we run against your whole infrastructure. And in some sense, they're greater than the sum of their parts. We don't just check for encrypted S3 buckets.

We check for access controls from person A to resource B and a million different things. One of the things you will get is an email notification. You will also get a notification with Invanta and all sorts of other beautiful UI tricks that we're improving every day. But yeah, somebody will be alerted that something has gone wrong

and then you will be able to fix it, and you won't have to wait. And you also check for things like is my employee's laptop, do their drives have encryption or not? How does that work? We run an optional agent on a lot of our customers' workstations and servers. This agent is based on a really cool open source project in the Linux foundation called OS query and that allows us to query certain metadata about their systems.

Only non-sensitive metadata. We've made sure to remove all the sensitive parts. But this allows us to do things like figure out whether all of the workstations are encrypted, figure out whether they all have a password manager, make sure that there's only one user account on them, make sure that they don't have any malicious applications and so on. At the end of the day, all of these things are really important

to prove security and to be secure. Once again, if only 99 out of 100 machines are encrypted, the hundredth one gets stolen and you're doomed. But at the end of the day, it's just another integration, right? If you use an MDM provider, or you have a policy of you somehow ship out to your all your computers, it's the same thing as our agent for all intents and purposes, and we can integrate with that instead. Okay. So you're pulling in data from both like cloud APIs and also people's laptops and basically

from wherever you need to prove that your customer is secure, integrating into like one place and then showing it to like a user saying, here's a checklist or here's a list of things to make sure that you need to basically pass in order to be compliant and be secure. That's really interesting. And how does all of this work behind the scenes? So you started Christina, you started this in 2017. Is that, that's what you said, right? Yep.

How did that, yeah. How did you build out the initial version and how how do you know what integrations to prioritize? And what did it look like at that time? For sure. So the initial Vanta was, of course, built in a spreadsheet, as you do. And it sort of had these lists of controls. And then we had to ask people whether they did them or not. And we just gave them the spreadsheet back. Like, is this useful? And we sort of didn't think it would be, but it turned out it was.

So we did this for two companies initially. And the idea for doing them for, well, the first company was, can we even make a spreadsheet that's like possibly reasonable, that kind of has the right stuff on it? Would they believe it? Would they spend time with us? We didn't for a second company. The trick there was we used the old spreadsheet and was like, will they be able to tell? They couldn't. So that actually felt good from a like, oh, we can standardize some of this perspective. And then the thing that actually happened is then we got an email from someone that was funny where they were like, weirdly, you seem like you've become these compliance consultants and like crazy life choice. But will you come? Can I pay you to do it for my company, please? Also, maybe we should get a drink and figure out what you're doing with your life.

So at that point, we started writing code. The first code we wrote was what we called at the time the AWS scanner. And roughly what that did was take AWS credentials and then go call all of the AWS APIs and get configuration information about all the AWS resources in someone's account. Built scanners, as we called them, which Robbie can explain what these scanners, these benighted things were, but kind of did things for G Suite and Slack and GitHub and sort of what we thought of as a common startup stack of tools. So as we were building those, we were also kind of still making these spreadsheet sort of things for people. We turned them into webpages. We called them Vanta reports. The trick here, the subterfuge here, was we would make these Vanta reports

and we told people they were powered by code, right? Give us your AWS credentials, they'll populate the report. That was actually a lot of code that we hadn't written. And so the reports were written manually. So we'd both get the AWS credentials, we'd do the API calls, like look at the raw JSON, and then write the report and sort of tell people it was all magic. It just took a day to scan everything, whereas actually it took a day to go madly write everything behind the scenes. But that was the first Fanta. I did that for six or seven companies, startups,

and then went back to them and was sort of like, so what did you think? And like, what was this? Can you kind of play back what this thing is? Which is actually really helpful because they generally were like, yeah, you checked out all my security stuff and told me what to fix and gave me a document that explained it all. And you're like, oh, yeah, that is it. Like, you got that. That's great. Built a lot of products where people couldn't explain what I did. So that felt good.

And then it was just sort of an exercise in how much of that painful report writing could we write code to do instead? That is super interesting. So you have like these AWS scanners, and you would read like the JSON output, as you said, and it just reminds me of like, you know, do things that don't scale for like a really long time. And then eventually, I'm guessing you built out some kind of like report thing thing that would read this JSON and actually write it to something that people... You know, you'd think so. That people are still... Kind of, yeah. We would also like sort of concatenate all the different API calls for

a service and then plop it in like one really big JSON file in S3 that then we'd read from. So yeah, Robby could tell you all the delights of that, but that was about as fun as it sounds like it was. Yeah, Robbie, go for it. Well, first, I got to point out to all the listeners, of course, that we do not store non-encrypted credentials anymore. That was very true. I have customers. But yeah, so day one, not to air our dirty laundry too much, but we were storing these several megabyte JSON files in S3 and we were actually using that to power these reports that Christina just mentioned, partially automatedly and partially when she would get the email and frantically type things in.

But also to actually power our UI, which was a disaster, loading a giant JSON file in from S3 and trying to say, hey, look, you have this current state of your system. Anyway, we have totally revamped that and have, I think, an incredibly beautiful and elegant resource fetching system now. We don't call them scans anymore. We call them fetches. It's basically the same thing. But instead of writing your entire AWS state to an S3 blob in JSON, we now write marginal parts of each of your services states to our database and update it continuously and

read from it for our tests and our UI and everything you need there. Are these reports generally standard across companies or do they have a lot of customization? Oh, yeah. So ours are generally standard. And this was one of the kind of core product hypotheses of Vanta that I think have been borne out, both in our experience and kind of how we're seeing other companies move, which is basically so prior to Vanta, every company's sort of compliance report or these things all look different, right? And kind of step one of one of these projects was like, go call someone up who has some experience

with these, spend 15 hours with them, tell them about what you do, and they will write down what you said and give it back to you. And we sort of looked at that as kind of PM and engineers and we're like, well, that's goofy, right? Like two companies' security practices probably shouldn't be that different and they should probably both just follow engineering best practices. So we kind of put a stake in the ground and tried to standardize on a single set of controls or list of these things. And I think it's served us well. We've gotten savvier and we've upgraded some of our controls, but I think kind of the fundamental idea of you can

and should standardize on best practices has actually served us well. Yeah. And that makes sense to me. Why should two companies that are basically building the next consumer like app, mobile app, have completely different security practices? Yeah. And even a step beyond, if we think about, I don't know, Dropbox, how they store customer

data or user passwords or something, and how Robinhood stores right? Like, these shouldn't actually be that different. Like, there are just, again, standard ways to do these things. And innovating on a lot of the stuff is just not as useful as it used to be. That makes sense. This comes back, of course, to like computer science education itself, right? If you're a software engineer at Robinhood, chances are you maybe took one security class in college. If you're a software engineer at Snapchat, you probably didn't. Maybe you did. But the point is, none of the people, the brilliant people writing the software have enough experience to know what the best practices here are in every case. And so every few weeks, you see a story of like, this company forgot to salt the passwords, this company encrypted their

passwords using, you know, ECB mode instead of counter, whatever, right? Like, you shouldn't have to know this, we should add, you know, security to computer science education, but it failing that we can we can really help without requiring you to have a security audit of every single PR. That makes sense. And I have so many more questions just from this. It's like, how do you decide what good security practices are? So I'm guessing some of it is inspired from the audit language or like what the SOC 2

requires, but I'm sure you have additional things on top of that, right? So how do you all decide on that? Yeah, I think especially when we're working with one of the sort of recognized compliance standards, there's stuff they include that we need to include, no matter how we kind of feel about it. I don't really feel reasonable about it, but yeah, that's just part of adhering to some of the recognized standards.

I think one of the really neat opportunities for Vanta is getting to define some of what reasonable security looks like, or we use this sort of locking your doors and windows analogy, and there might be different levels, but sort of getting to define and promote that to just kind of bring more clarity and make some of this kind of easier to accomplish. Okay. Robby, if you have anything to add to that. Yeah. I mean, at the end of the day, it just comes to being aware of the latest developments in best practices and just having sanity checks, right? 90% of the tests we run, if you talk to any engineer, they would say, oh, yeah, of course, you should encrypt or of course, you should make sure that your VPC doesn't allow traffic from anywhere or

whatever it is, right? So the 90% are just automated stuff that anybody would say. The remaining 10%, we have to make sure that we're up to speed on all the latest standards and all the latest security best practices and eventually become a body of our own that is opinionated about these things. So it's simultaneously just empowering our engineers to say, this is a good test, we should enforce it and empowering auditors, whoever else to say, you know, this is new best practice like Vanta, you should consider telling your customers that it's a good idea. Yeah, and you can move the industry forward and make it more secure through this thing, which is kind of great.

And I'm curious if like customers end up asking you for guidelines, like, yeah, I know Vanta provides this much security for me, but do you all have ideas on like what processes I should have for security? I'm curious if like customers are already asking you about that. I get this question a lot of like, we're just kind of starting up. What sorts of things should we do? What sorts of things should we keep in mind?

I'll give you one good story and then Robby, you should jump in. One of the very first things actually we checked for when we had our G Suite scanner was two-factor on everybody's G Suite account. And sort of to Robbie's point, prior to actually doing this, we'd ask the company like, oh, by the way, do you have two-factor on everyone's accounts? And 100% of companies said yes. Everyone was like, of course. Like, I'm not, you know, a doofus here. And then we, you know, called the APIs and turned out, yeah, most of the companies didn't have two-factor on everybody's accounts. But there are sort of these interesting oversights

where it was everyone but the founders, right? Because the cobbler's children have no shoes sort of thing. It was everyone but the office manager who is in fact responsible for everyone else, but didn't have that. And I think it just speaks to how even the simple obvious things are actually kind of hard to operationalize, or it's just helpful to have the continuous check in there. But Robbie, what two founders on a couch, what should they be doing? Well, of course, they should be installing Vanta. And so Vanta can tell them what they should be doing. But it's actually a really interesting question, because a lot of the things we do are semi-optional, right?

Like, we check to see if you've had a pen test, right? And if you're two founders on a couch, you haven't had a pen test yet. And trying to display to our users when something is important is a really tricky problem that we're still working on. But in terms of answering your question directly, which is, you know, two founders on a couch, what should you be doing? It is logging as much stuff as possible and encrypting as much stuff as possible and making sure that you use single sign-on and multi-factor authentication and all that stuff. Because at the end of the day, you're probably not getting targeted by a nation state. You're probably getting targeted by somebody who is scanning the internet and trying to find open

Redis ports, right? And so so use Vanta to make sure that you don't have any open ports, and use Vanta to make sure that you remember to encrypt the places that you needed to encrypt. And what what about when you go from like two founders on a couch to like a 200 person startup and you still don't have that much experience in security because I don't know, the co-founders just a regular software engineer. Like, have you thought about or does Vanda do something around like, you know, having a certain security process? Like how should founders or just heads of engineering think about security when you've kind of expanded and you're at that point a big risk vector for companies that are growing maybe not quite at

the 200 person stage but even at the 30 40 50 person stage and even compounded at the 200 1000 person stage is uh managing access um phantom makes it really easy for you to see who has access to what and can also alert you when somebody who doesn't have access to something should or shouldn't have access to something does. I'm sure you've all had jobs before where you left the job and two years later you tried to log into the company Slack or something just for kicks and you still had an account because it's so hard to operationalize removing all this access so just doing regular access reviews is a is is tiring and and in general hard um having that automated for you and making sure that people are deprovisioned when they're supposed to be prevents yet another um vector of attack which is you know an ex-employee or a

current employee who is either careless or or mad you know, probably careless. But that's just something you should always principle of least privilege, give the fewest people keys as possible. Makes sense. And it feels like with the proliferation of like more and more SaaS tools for everything, it's going to be harder and harder to manage this. And that's why companies like Okta and Odd odd zero are doing so well yeah i want to step back from here and ask some of the questions around the technical stuff go back to the aws scanner and all that so you mentioned that you've changed all of that from storing s3 blobs to actually having a well-architected piece what are some like key tech choices you all

made at that time? And how do you go about deciding this is what we should do? And I'm curious how they've played out three years later. Yeah, I'll start, then Robby, you should jump in. So we tried to choose things in 2017 that seemed like that were not so cutting edge as to break what we were doing, like kind of new but reasonable. So we actually started writing Vanta and JavaScript and lasted about a day and moved to TypeScript. Robbie can continue this journey

because it was not fully typed, but sort of the server side was typed kind of pretty quickly. GraphQL and React, you know, like the GraphQL client we used with Apollo because it like seemed like the one with the most traction. So I think that was trying to, yeah, like to use kind of market pull there. What are other interesting choices? Containers, certainly. And I think that was honestly, it was probably an optimization around personal development. And I mean, also deployment, but actually literally kind of personal deployment is

how can we get this stuff running? In a moment of great frustration at a MacBook Air that was purchased. We ended up with remote development, especially developing on AWS machines with the codes are synced from a laptop to the machine. And that was literally the joke was the laptops had become panini presses and we needed them to be laptops. And so that ended up being a sort of a, I think, helpful choice, at least at the time. Mongo was our sad choice.

And then that was like, it was honestly a holdover from a prior project. And it was like, oh, the config's already set up. Let's go. And then, yeah, it's kind of one of the lessons of when you're making prototypes, you should actually assume sometimes things work or you'll get stuck with your choices.

I think, you know, in retrospect, would have used a SQL database, but never got to the point where, yeah, you know, when something's actually working or it seems like it's working, there's always things to build. So you don't go back and fix that. You just shake your fist at it.

It was also definitely like self-hosted Mongo, which has since been fixed. I don't know. Probably what else am I missing? Yeah. I mean, it's always an interesting question, kind of a multi-armed bandit question of should you build your own thing? Should you invest in developer tooling and safety? Should you just build a new feature in JavaScript that probably has a million bugs and build it away? And we generally try to build features as quickly as possible. But as a company, especially that doesn't want to give a ton of surprises to our customers, tend to invest a little bit more in developing developer tooling and safety and all that stuff

than some other companies at our size. So we have an incredibly robust TypeScript code base, which we're all pretty proud of. A lot of type annotations that are very sophisticated. We use a lot of the built-in services on AWS, but abstracted in such a way that if for some reason we needed to move off of them, it wouldn't be too hard. At the end of the day, we generally gravitate towards buying

instead of building when somebody else has already built something. But we also want to make sure that if we need to change our decision down the line, that we have the tooling to do so. Yeah, I mean, everything is in Docker containers, on Fargate, on AWS, communicating with our APIs via GraphQL and with each other, using various message passing frameworks. Lots of interesting things that we could get into probably in a blog soon, but probably too technical for an hour long podcast.

I think that's all like interesting. And I wanted to ask about that push towards like developer tooling and developer productivity, right? So yeah, generally you would not imagine like a 50, 60% company focusing too much on developer productivity.

So what's your framework on deciding? And first of all, I think it's great that you care about this stuff. But what's your framework on deciding, you know what, I should not prioritize this product feature and focus on improving our tooling? Yeah. At the end of the day, for me, it really boils down to how much time this tooling is going to save in the long run. And it's not just time.

It's also how much developer frustration it's going to save that day. So obviously, features are the most important thing because they're the things that people buy Vanta for. But if it's going to be build one feature today and then not be able to build two features tomorrow or build half a feature today and build half a feature today and build half a feature every day thereafter.

We're a startup. We're a company that needs to continue to iterate, not just build everything today. So there are actual numbers that you could put in that framework. But at the end of the day, it's weighing how much time is this going to save in the next week versus how important is it that we get this feature out this week? Interesting. One just thing on this that I don't think Robbie will brag about, but he should, is I mentioned the server side of our code was typed. And then Robbie came in and he saw, I think it might have been somewhat abhorred by some amount of debugging and prod on the GraphQL typing in particular.

And so one of its first commits was adding typing to GraphQL and it was life-changing, the responses in particular. Yeah, we do a lot of code generation, which has been really fun because it allows us to type not just our code in our own code base, but also everywhere where code comes in from APIs, from our Mongo layer, from GraphQL, from whatever other data stores we're using for other services. And this allows us to have an incredible amount of type safety. And the runtime errors we hit tend to be like resource allocation errors instead of like you accessed this field that you shouldn't have accessed.

So what have been the implications of GraphQL? Not that many people use it. It has been increasing. GitHub offers a GraphQL API now. Do you think it sped you up in terms of feature development or it's been all right? Would you use it again? One thing that's really nice about GraphQL is that it allows us to define very concretely an API. I have worked before at places that have a REST API that is constantly changing the shape

of responses and makes it really hard to deal with on the client side and on whoever else is ingesting that API. GraphQL makes it much easier to do that and really requires best practices. When we started off, our GraphQL interface was very much implemented like a REST API. It wasn't really a graph. We just had a bunch of endpoints that had response values. Over the last years, we have been improving this and are continuing to improve this such that we are really expressing relationships between entities at Vanta as a graph. So I think it probably slowed us down in the beginning, because we're basically implementing

a REST API as GraphQL. But now that we people on the team really understand the fundamental power of GraphQL, and that we have a bunch of tooling around code generation there and type safety and all that stuff, it allows us to express relationships between entities really elegantly. And this is really powerful, especially at a tool like Vanta where things like access controls matter because we could say, hey, look, this user has these machines, these machines have access to these things, and this is really nicely expressible in a graph. Just to underscore that from a product perspective and kind of a security perspective, I think it's one of the places where Vanta is and can be really powerful, right? If you just have a laptop management tool that can tell you all about the laptops, but again, to Robbie's point,

Vanta's sort of power and elegance lies in saying, well, this person who joined on this date, who's this sort of employee has this laptop that is this SSH access anyway, and just like kind of being able to reason about that full chain of kind of things that happen in the real world. Yeah, I can imagine when like a company has like a hundred, like a thousand servers or whatever, trying to narrow that down super quickly and show exactly the relevant piece like GraphQL would help with that and you don't have to add new endpoints for every extra functionality which also seems really useful and you spoke about MongoDB and I've seen the memes like MongoDB is like web scale and all this but I would have imagined that you know it's

been four years or five years since those memes have come out and things have gone better. What's your experience been? Yeah, by the time I stopped coding, we'd like to complain about Mongo. But Robbie probably has a better point of view. I mean, when I joined, we had self-hosted Mongo and we didn't really know how to self-host Mongo. And so we ran into all sorts of issues there with bad configurations and things like that. Eventually, we decided to move to Atlas, which is the cloud service that Mongo provides,

which we've actually had a lot of success with. However, there are some database patterns that are kind of anathema to MongoDB that are really common in SQL databases. Things like having foreign keys in documents instead of embedding the documents. Things like complicated transactions and stuff like that. And there are certain features of MongoDB that we wish we had,

and that it still seems like a piece of software that is going to continue to improve over time. We still have occasionally our database having an election that causes some issues or something like that. But ultimately, now that we have hosted Mongo, and now that the team understands the limitations and the power of this database, instead of thinking about it as just a SQL database. It's been successful.

Interesting. And any other decisions you feel that you made early on that have played out in interesting ways? Language and database is one thing, but anything else that has been interesting? The types point is really nice because I've worked in a dynamically typed code base before and it's grown and grown and grown. It's been really bad. Anything else that stands out? Anything interesting to say on React or how we use it? So I tend to steer clear for the most part of our TSX files, but Kevin actually, the other first engineer who started on the same day as I did,

has been owning our React and it's been overall a really solid tool. It comes back to your point from earlier, which is use the thing that's heavily supported and isn't going to break immediately. When we started React, I believe it was 16 or 17, hadn't come out yet. And we were using React components and nobody really knew how to use those, and eventually we learned. But then React hooks came out, which has simplified our coding a great deal, especially with more code generation allowing us to have automatic React hooks

for any one of our GraphQL queries, makes our frontend much, much easier to develop, and debug, and cache cache and all that stuff. There are a million frameworks. I recently did some playing around with Svelte and things like that. I always joke around about writing a backend in Haskell and a frontend in Elm, but I'm happy with our React

and TypeScript stack today. Maybe one service at some point can be a Haskell, fully pure, doesn't deal with any inputs at all, just does something in memory. That's cool. Okay, Christina, question for you. So at some point you noticed that, you know,

people are buying Vantaa, right, early on and you had to make your first engineering hire. How did you decide you know who should I hire and like how should I evaluate them and it's like so early oh yeah um we I mean Robby should also speak to the other side of this um but I think at the time um I think we felt actually reasonably about our ability to sort of put someone through an interview loop and that we just had a lot of experience of that from Dropbox. I think the part that was a little goofier was, you know, we're kind of only two people. And so what sort of interloop? Do you

like excuse yourself from the room for one interview and like walk back in with a hat? You know, like, what do you do here? So we actually didn't do any of that. And what we did was invited folks in for a day to sit next to us and code something. We sort of pull a project off the list, literally, and hand it to be like, here's our code base. Here's the thing we're working on. Here we are. Here's the room. What's going on, right? And so folks got a very clear picture of what it would be like. I think one of the reasons,

I mean, there are a few reasons we didn't continue with that interview format and why folks kind of don't interview this way. A few of the downsides where it's sort of impossible to compare across candidates really just depends on what project you you pull um and sort of what their experience is so actually you know one of the folks we hired has been tremendous um you know had never written typescript or javascript at all um and so learned a lot that day but i you know mostly we got signal

on that person's um ability to learn kind of brand new things with brand new people versus sort of any sort of coding output. I remember Robbie took this desktop script that I used basically to generate policies. So in the web UI, we'd say, hey, you can click a button to generate policies and then hang on for a moment. And what that button did was emailed me and then I would like run this script on my laptop and then email them the policies or like upload them behind the scenes. Cause I was kind of, I could sort of write the script, but I didn't know how to like put it in a container and go. So that actually ended up being Robbie's project. And it was amazing because those emails used to stress me out. Yeah, I think we finally got rid of that microservice a few weeks ago.

So my interview project is no more, but it was the only interview for which I've ever been paid, which is a funny little thing because I came in and worked for, you know, nine to five with Christina and it was great. Yeah, we did pay because they were like doing real work for us that we wanted to use. Yeah, we did pay because they were doing real work for us that we wanted to use. So, yeah.

Okay, so it was like one workday and you would pay the candidate to do it and you'd get some amount of signal, which was not easily comparable, but some pretty good signal on how collaborative they were and can they pick stuff up. Yeah, and also kind of, yeah, I think it was, I mean, kind of, yeah, I think it was, I mean,

kind of a litmus test of like ambiguity and just like, again, you're sitting in a room with two new people going and stuff. How does that feel? And is this, you know, on the like exciting to wanting to run from the room? We didn't have anyone actually run from the room, but I think it would have been a, you know, not entirely unreasonable reaction. How did you decide that you needed to hire someone?

Like both of you, like you and your you needed to hire someone like you both of you like you and your co-founder were engineers and you're working on this like oh at what point were you like i just need someone else yeah i mean kind of later like i think we were both um i mean stubborn in a good way but sort of like uh stubborn and sort of had this high bar for like for bringing someone else on and we're sort of like we can spend time trying to figure out if there's product market fit but when we bring on someone else want to make sure we're not going to be like, oh, just kidding. We're now Uber for dog walkers.

Right. So one of the sort of preponderance of confidence in what we were doing, which we eventually got to, but kind of by the time we got there, it was like, ah, there's 14 things we want to do yesterday. Now we need to go recruit. So yeah, I think it was by the time we kind of came around to it, it was actually blindingly obvious. That makes sense. And Robbie, what was it like joining a startup with like two other people? Well, it was wonderful because every day you came in and you had to figure out what you needed to do and build it from scratch.

Ultimately, it was really exciting because I had my company, which technically I think was really cool. But on the customer side, it was very much a solution in search of a problem. Like we had all the code written and it worked, but nobody needed what I was building. Whereas Vanta had the opposite problem. Christina was running scripts to send people policies, but there were customers knocking down our door saying like, why can't you support me? Like you're a tech company, right?

Like we need a hundred more customers today, right? And so Vanta was very much a, like we had a good problem, but we just hadn't written all the code yet to solve it. And so every day at Advanta, even through today, it's very clear that we're just constrained by product and engineering. We're never constrained by we don't know what to build because there are so many things

that we're going to be able to do today and in the future to make our product better and more valuable for our customers. Yeah, that makes sense. It's like, there's like technical bottlenecks. There's not like market risk in a sense. You know that there's a big market and that you need to build a product. It's just about building the right thing and quickly enough. Yeah. And I think kind of across Fanta, there's also a lot of prioritization sort of to that point. Like there's a lot of ways we can help customers and we kind of want to help them in all of the ways eventually. But which ones are we working on today versus next month versus in next year? And those are the decisions that are tough.

And it's also really nice to work on a problem that customers really need solved, right? Because they need to unlock deals. Right. Yeah. For me, the best comparison point I have is working on Dropbox Paper, which is a team collaboration tool. And so we weren't even selling it, but without dollars,

we were sort of selling the idea that your team would be more efficient. And that was really hard. And you're sort of like, well, why? If you don't use Google Docs, you use this other thing. And you're sort of like, well, why, if you use a different, if you don't use Google Docs, you use this other thing. And you're like, well, because there's emojis, but actually they're really helpful for team collaboration. And they are, but, you know, when you, when you say that out loud, it sounds a little goofy. And working on Vanta has always

just been substantially different where it's like, it's something people know they need. It's something they don't want to do. And so they end up being extraordinarily appreciative. Like somebody is more appreciative than, than I think they should be. But that's been really fun. That makes sense. And then I'm sure at that point there were no like task trackers or anything because people were just doing whatever work they needed to do on that day. At what point do you start like introducing process or do you hire someone

who's been in this position before? And like, how do you think about these things? So we didn't have task trackers initially. And then we used Notion as a task tracker collaboratively for the team. And sometimes other founders ask me like, Oh, you know, my team doesn't want to use a task tracker. What's your secret? And the secret is I think to you, task tracker everyone hates for a while and then they actually kind of like really want a good task tracker so that was that was our journey i don't know if it's um

if you know in some ways we ended up in a very good place but i think there was a lot of we put differently a lot of teams have teeth mashing around like should we use a task tracker and ours was actually around like oh god can we use one that actually suits our needs? Makes sense. We didn't have a formal task tracker for a long time. But we were always a company that kind of prioritized building process, even when we were three people, like we knew we were going to have a six person company at some point. And we were six people, we hoped we would have a 20 person company at some point. And you know, now that we're 50 or 60, you know, we have to build process for 100 or 200. So we were never totally blindsided. But sometimes we had to wait a little bit longer to implement the thing that was probably necessary. Nowadays, at least on the engineering

and product side, we use we use clubhouse, which has been the task tracker that's been least hated that I've ever that I've ever encountered. So I think that's high praise for a task tracker. And do you interview candidates still? Christina, do you interview engineering candidates? At what point did you stop? I still, so I never interviewed them for engineering acumen, which is a good thing for everyone involved. We have an interview that sort of focused on folks' prior experiences and

particularly kind of work that stood out to them or found meaningful, which we really like for kind of getting a sense for what people find meaningful in their work. So I still do that interview with a reasonable number of candidates. And Robby, how do you think about the interview process? Do you interview like most candidates or have you delegated your responsibility at by this point? I interviewed every candidate until a couple months ago, where now we've ramped up a couple of engineers on all the questions I was asking, and I feel very comfortable with any of them running these interviews. I still go to all of the interview debriefs,

just to get a sense of what we're looking for and how we're doing it. But at the end of the day, it's really important to have a team where everybody trusts and respects one another and like, doesn't feel like if they write a spec that it won't get implanted in a reasonable way. And we've done a really good job of building up that team so far.

And I think interviews are really the only way, ultimately, that you can continue to build up a team that's so strong and kind and respectful. And that's why I think it's so important. That's cool. And I think I haven't mentioned this to listeners but i work at vanta so full disclosure i should probably edit this in and put this like earlier in the show but it's it's not too late for that um in general like i think i've i've understood a bunch of

pieces that you mentioned around hiring around technology how do you think other startups can think about like security and compliance like if you had to start a different startup today like either of you what would you do from like day one like that's like different in terms of security or in in maybe in any sense compared to what you started with in like 2017 don't use manga no not actually um um i think largely actually would have started i mean this is judging the the the score and not the shot but i think would have started engineering hiring earlier um that's sort of been been kind of again to robbie's point a persistent um blocker which is again tremendous blocker to have kind of rather that than market, but blockers in general are frustrating.

I think from kind of a starting up security perspective, we talked about two-factor and like broadly the locking doors and windows. So Robbie mentioned encryption and Robbie mentioned logging. And I think those are both great and useful things that are easy-ish switches to flip. I know there's more nuance than that, but they're closer to that than a lot of other things that actually get you pretty far.

Yeah, I've learned that the operational aspect of security is surprisingly important. Maybe it shouldn't be surprising, given that I work for a company that tries to codify operational security. But it's as simple as when you do a thing that has a security implication, write it down somewhere where everybody knows about it.

Whether that means like I am giving Christina access to this database, or I'm whitelisting this new IP address or I'm about to do this thing. And just having an audit log of all those things, even if that audit log is not a super technical log or even if that audit log is not implemented in Vanta, is something that you're going to be able to go and look back at and remember everything that you did.

Because when you're two people or three people, you feel like you don't need a trash tracker because everybody knows what's going on. But even in my personal projects, if I don't keep a list of things that need to be done or things that have been done, I forget between one day and two weeks. And it's important to keep track of that stuff. I think one thing we did relatively early, well, probably relatively early on

and sort of to the process early point were code reviews. We wanted more than two engineers. I think four or five maybe is when we started doing them. And some of that was actually seeing Dropbox struggle to implement code reviews at like a hundred person engineering team and sort of took them like three quarters just because it was like disruptive and hard

and you had all these people on board. But it's kind of the magic thing about when you do it early is then you hire your, I don't know, whatever, even 50th engineer or 15th engineer. And they come in and they're like, oh, it's a code review company. Cool. Guess I'll do that. Anyway, moving on. I actually think we did have code reviews from day one. Like when I joined, by that time we had code reviews. Code reviews, okay. And we even had CI and automated testing and all that stuff by then.

We, of course, invested a lot more in it since, but just doing it from day one is super important. Okay. And my last like category of questions is just around like brand. Like how do you all think about, you know, building a brand? Because I think Vanta is a product that you sell to like ctos or heads of engineering um how do you get them on board and have get them convinced that you know this is a solution that they need not

just for compliance but also to secure themselves yeah i think one of the core tenants that we think about here is it's we very much want to build Vanda as the kind of trusted security advising, you know, compliance, proving security brand in this space. And so part of that is you kind of can't play fast and loose with a lot of it, right? It is a trust game and the trust kind of only, it's hard to build and easy to lose. And so where I think you can kind of see that is we tend to be a little, we tend to have kind of more rigid standards for when we'll launch something or when we'll say we support something. And again, it just comes from this idea of like, you can't tell people you're going to secure them.

You can't tell them you're going to get them a specific certification and then kind of fail to secure them. You can't tell them you're going to get them a specific certification and then kind of fail to do so. I mean, you can and people do, but just kind of from day zero, day one, being like this is kind of fundamentally unacceptable and not who we are. And so we're going to skew a little bit more in what may seem like a tentative direction or conservative direction, but it actually comes out of this desire to be trustworthy and sturdy and reliable and around for the long term and longer term focused broadly. Okay, so it's like a don't move too fast and break things. But then what are the product implications of that? Because you also want to ship quickly

and you have competitors looking at your heels. It's a great question. Yeah, I mean, it's like all these things, it's a tough balance to strike. And I think for us, and to Robbie's point, we've invested and try to invest more and more in tooling that lets us move fast without disrupting customers as much.

Right. And so practically, this kind of looks like ways to release things that is not to everyone all at once on a random Tuesday, right. Which we sort of did have or, you know, which is, I think a thing that given the space we're in and the stakes and what we're trying to do being correct is,

is quite important. Robbie, if you have any thoughts. Yeah. I mean, move fast, but don't break too many things. The trust comes down to what I said at the beginning of my vision of Vanta in the future, which is you're never going to have a CISO of a big company asking for Vanta proof instead of SOC 2 or in addition to SOC 2

if you don't have a really robust brand. And it's something that I'm glad we have Christina and a whole marketing team to think about because I've never been good at that sort of thing. But it's more and more important as we grow because ultimately security is about trust. And does that mean in practice you have really advanced feature gating and all of that to make sure

that you're not rolling out something broken to everyone at the same time? What does that mean in terms of when I have to actually implement this? Advanced is a strong word. I would say we're working towards really advanced feature gating and things like that. And it's certainly prioritized for us more than it might be prioritized for another B2B company. But we are still a small company that is moving very quickly.

And maybe you'll get to build some of that. That makes sense. And a final question for you. Has there been any fun or interesting outage that you're okay with sharing? Oh, outage. Yes. That's a good one.

I won for after we stopped recording. I mean, it sort of implied this. So before Robbie went and typed the GraphQL responses, you know, I was joking about debugging and prod, but there's definitely some debugging and prod and like not in the like the cool kids observability sense, like just in the like go to CloudWatch, figure out the errors, go fix your type,

but like definitely some of that. Those are sort of outages. We kind of break a page for a single customer and sort of hope they wouldn't notice. Yeah. There have been, we generally have had a pretty robust system. We don't have like 10 nines of SLA. We, we haven't announced any of that stuff, but you know, we,

we work really hard to make sure that we're up most, if not all of the time. In the past, we have had issues with our self-hosted Mongo that took us down for a while that we haven't... We've had issues with... It's interesting because we rely a lot on external APIs, and when external APIs start returning error codes that are unexpected, that can cause a lot of upstream issues because we try to make it really easy for our customers to know, for example, when their credentials are no longer valid. So if an upstream API suddenly starts returning your credentials are invalid when they are valid, we have in the past sent emails to a bunch of customers saying your credentials are invalid.

And we've done some work to minimize the risk of that. But ultimately, when we're a company that's kind of based on aggregating external APIs, we're only as robust as those APIs. And we have to build a lot of tooling to figure that out. And I also have some fun stories for you. Yeah, it sounds like there's going to be a lot of technical challenges, right? Because you have to just keep integrating with more APIs. There's going to be more and more customers.

And it sounds like there's going to be more and more interesting challenges on the way. Definitely. If there's any listeners interested in working at Vanta, just let me know. Yeah, anyways, thank you both for being guests on the show this was a lot of fun thank you so much for having us