Lenny's Podcast: Product | Career | Growth - How Snyk built a product-led growth juggernaut | Ben Williams (VP of Product at Snyk)

Starting point is 00:00:00 Being able to identify the various micro and macro loops, how they're all connected, being able to document them in a qualitative model to communicate a shared understanding of how you grow, it's really powerful. Augmenting that then with, you know, the quantitative side of things that helps guide quarter to quarter focus and ensure you can be intentional about where you're investing, that becomes a big enabler. You know, you're never going to have a shortage of ideas in a high-performing growth team. So knowing where to focus amidst that kind of see of ideas is a really important role of the strategy.

Starting point is 00:00:40 Welcome to Lenny's podcast. I'm Lenny, and my goal here is to help you get better at the craft of building and growing products. I interview world-class product leaders and growth experts to learn from their hard-won experiences building and scaling today's most successful companies. Today, my guest is Ben Williams. Ben is a VP of product at Sneak, which is likely one of the biggest and most interesting companies that you've never heard of. Sneak makes it easy for developers to catch security issues in their code, and there's a lot to learn from how Sneak got started. It started through product-led growth, evolved into product-led sales,

Starting point is 00:01:12 was very community-driven, and was also laser-focused on developers, which has become one of the most lucrative markets to go after. In our conversation, we cover how the founders of Sneak got their first 100 users, what they got wrong when they tried to monetize early on, when they hired their first marketing and salespeople, how they structured and grew their growth in product teams, what they figured out about what should go into freemium and what shouldn't,

Starting point is 00:01:35 and so much more. As you'll soon hear, Ben is British, and so the episode is automatically going to sound more sophisticated, and I can't wait for you to hear it. With that, I bring you Ben Williams. This episode is brought to you by Coda. Coda is an all-in-one doc that combines the best of documents, spreadsheets and apps in one place.

Starting point is 00:01:56 I actually use Cota every single day. It's my home base for organizing my newsletter writing. It's where I plan my content calendar, capture my research, and write the first drafts of each and every post. It's also where I curate my private knowledge repository for paid newsletter subscribers. And it's also how I manage the workflow for this very podcast. Over the years, I've seen Cota evolve from being a tool that makes teams more productive to one that also helps bring the best practices across the best practices across the

Starting point is 00:02:23 tech industry to life with an incredibly rich collection of templates and guides in the Cota Doc Gallery, including resources for many guests on this podcast, including Shreos, Go Cool, and Shashir, the CEO of Cota. Some of the best teams out there, like Pinterest, Spotify, Square, and Uber, use Cota to run effectively, and have published their templates for anyone to use. If you're ping ponging between lots of documents and spreadsheets, make your life better and start using Cota. You can take advantage of a special limited time offer just for startups. Head over to coda.io slash Lenny to sign up and get a $1,000 credit on your first statement. That's Codda.io slash Lenny to sign up and get $1,000 in credit on your account.

Starting point is 00:03:14 This episode is brought to you by Athletic Greens. I've been hearing about age you want on basically every podcast that I listened to, like Tim Ferriss and Lex Friedman, and so I finally gave it a shot earlier this year, and it has quickly become a core part of my morning routine, especially on days that I need to go deep on writing or record a podcast like this. Here's three things that I love about AG1. One with a small scoop that dissolved in water, you're absorbing 75 vitamins, minerals, probiotics, and adaptogens.

Starting point is 00:03:45 I kind of like to think of it as a little safety net for my nutrition in case I've missed something in my diet. Two, they treat Age 1 like a software product. Apparently they're on their 50-second iteration, and they're constantly evolving it based on the latest science, research studies, and internal testing that they do. And three, it's just one easy thing that I can do every single day to take care of myself.

Starting point is 00:04:09 Right now, it's time to reclaim your health and arm your immune system with convenient daily nutrition. It's just one scoop and a cup of water every day, and that's it. There's no need for a million different pills and supplements to look out for your health. To make it easy, Athletic Greens is going to give you a free one-year supply of immune-supporting vitamin B and five free travel packs with your first purchase. All you have to do is visit Athletic Greens.com slash Lenny.

Starting point is 00:04:34 Again, that's Athletic Greens.com slash Lenny to take ownership over your health and pick up the ultimate daily nutritional insurance. Ben, welcome to the podcast. Thank you very much, Lenny. Thanks, first of all, for inviting me. It's a pleasure to finally meet you. I've got to say that kind of what you're creating with the newsletter and now the podcast, which is such awesome resources for the product growth and wider tech communities.

Starting point is 00:05:01 So it's a real honor to be invited onto the show. And I hope there's a few useful things that people can take away. Awesome, man. I really appreciate that. I have no doubt there will be many useful things. People will be able to take away. I'm really excited to chat about Sneak and the things that you're building there. I feel like Sneak is this very under-talked-about company and also super fascinating company,

Starting point is 00:05:21 especially in terms of how it got started, how it's scaled. And it's also at the center of so many trends, product-led growth, community-led growth, focusing on developers to grow. And also just security in general is really interesting. And so I'm really looking forward to our chat. Me too. Before we get into all of that, I'd love to spend just a minute on your background. So you're currently VPR product at Sneak.

Starting point is 00:05:44 And I'm curious how you got to that role and just some of the other wonderful things you've done in your career along the way there. I'll try to keep it reasonably short. My education is in computer science. Graduated from the University of Manchester back in the late 90s. I'd already landed as a job as a developer, but ended up having a bunch of other interesting offers, one of which was to join a small startup building requirements management tooling

Starting point is 00:06:09 for product and engineering folks. This was all pre-agile, but that whole space at developer tooling, engineering tooling, it was something I just felt naturally drawn to, and it's where I've really specialized through my whole career. I actually joined that company as a solutions engineer, so lots of demos and so on. I was really close to the product team, but also speaking with customers every day, which, as is quite common, I think, was my path into product there.

Starting point is 00:06:33 Several years and a few acquisitions later, I find myself in IBM with the Rational Software Developer Tooling Group. I was leading parts of the product strategy there and a bunch of initiatives. I actually learned a ton of useful stuff there, but also that I had a strong preference for working in smaller, fast-growing orgs versus 350,000 people, be a month. After an interest in unexpected interlude, leading a DevOps transformation at a fintech and some consulting and advising, I then joined Cloudbees. They're a DevOps startup with an open core model. I was leading product, design, and growth there. Stayed with them for three years through several raises and a period of really fast growth, before finally joining Sneak to build

Starting point is 00:07:17 out our growth organization and lead our developer experience initiative. I have a broader remit at Sneak now than just the Growth Ork, but yeah, that's how it started. Awesome. To give folks a sense of just how successful and big Sneak has gotten, one is just what a sneak do? We haven't even talked about it. And then two, just some stats about the scale of the company and the business at this point. The developer security company. We make it ridiculously easy for developers and their teams to improve their security posture while still moving fast.

Starting point is 00:07:48 So sneak can find and automatically fix vulnerabilities in code, open source dependencies, containers, infrastructure and cloud configurations, and all underpinned by the best security intelligence data in the market with a laser focus on developer experiences, which is why we're really different. It's also an amazingly fast-growing business with some stellar PLG focused investors and board members from the likes of Ed Sim at Bold Start to Tamar, Joshua, Slack, Ceport. We were founded in 2015. Last valuation after our Series F was at 8.6 billion. We're securing the software of millions of developers now, well over 2,000 paying customers, now around 1,300 people,

Starting point is 00:08:33 of which around 500 in R&D, with nearly 70 folks in our product talk. And the people here just create this amazing culture, and all in all, it's just a really exciting place to be. Okay. Did you have any sense it would get to this scale when you'd join? joined early on? Yes, I think because I wasn't looking for a new role when I ended up joining Sneak when I was first approached by them. But everyone I spoke with along the interview process just became more and more impressed with not only the caliber of people here, but the vision, the mission for the company and

Starting point is 00:09:09 all those things you mentioned in terms of PLG, community-led growth, focus on developers, the security space. These were all kind of all things that if you create a Venn diagram of all the things that I'm really interested in professionally, super interesting, super exciting to me, then kind of Sneak ended up in the middle of it. So it's pretty cool. Awesome. So that's a good segue to where I wanted to start is how Sneak started and how Sneak got their first 100 users. And I know you weren't there necessarily for that, but I'm curious what you can share about how the founders found their first, say, 100 users. How did they get to their initial developers and get people excited?

Starting point is 00:09:46 I think it's a really fun story. So if you don't mind, I'll just take a moment because I think it's important to set the stage by looking back at it. Just look back at the market in general. So PLG or bottom up and security, these were never words that were kind of known to have been spoken together in a single sentence, right? So security has always been a centralized function. Security programs were historically more about audit and policing and enforcement versus developer enablement or empowerment. A sales motion you saw, it was always top down, it was seen as immovable, CSOs, other AppSecLeads were the buyers, security tooling that was out there at the time,

Starting point is 00:10:26 they all catered to this dynamic. And these were tools that slowed developers down. They created a lot of frustration, they were met with a lot of resistance, not the ideal recipe really for consistent adoption and strong engagement and retention. And ultimately, app security programs based around those kind of solutions just weren't effective as they really needed to be. So it was a realization around this, timed with adjacent market shifts happening with DevOps that sparked the ideas behind Sneak. So the founders, Guy, Danny, and Asaf, they saw a real opportunity just to do things differently. They believed that the most effective way

Starting point is 00:11:03 to improve application security posture was a developer-first approach. They knew that the developers were increasingly caring about the security of their code in the same way that they cared about, performance and functional quality of their code. But they also knew that to empower developers to own that security, they needed just much better tools with way less friction than they ever had before. And their approach was, I think, looking back super smart, focus on a community. It wasn't the full extent of what we'd think of as community-led growth now, but it was close. They started with a really narrow early focus. It was a single persona, single context, single use case, and what that meant for was developers building applications using Node.js, who wanted to ensure that the open source

Starting point is 00:11:50 dependencies they were pulling into their apps were secure. Now, open source software, it's a huge accelerant in building modern software. The average software application today is at least 75% of open source libraries and components. So this was increasingly becoming a primary attack vector for malicious actors who could find a single vulnerability in an open source component and then find it and exploit it in every single application that was using that component. And at the same time, at least back then, you know, open source software was much less tested for security vulnerabilities and the maintainers of open source libraries were often less security aware. So you get that context and then at the same time Node.js as a runtime was gaining

Starting point is 00:12:36 traction. So there was this increasing adoption in the enterprise, more and more dedicated conferences and the like. But the community was still small. enough that Sneak could meaningfully influence. And Guy and the others just went all out on being deeply involved in that community. They were presenting at dev conferences, meetups, they were building online content and so on. And the question that they repeatedly posed to the community was, do you have known vulnerabilities in your apps? And sneak was there to help them answer that question. And fun kind of fact on the side, if you search for Sneak in the Urban Diction, you'll see it's an acronym

Starting point is 00:13:14 enforce so now you know. But all of this kind of, I think really only worked because of the parallel product-led approach. So while the answer to the question about how does your product monetize users was much less clear-cut in the early days for Sneak, the answer to the questions, how does your product acquire and retain users has always been product-led. And the initial version of Sneak, it was a command line tool, it was a tool for developers, and it could be run locally or easily integrated into CICD pipelines for early feedback. It allowed devs to assume more responsibility for the security of their apps. And that was just very different from the typical incumbent technologies that were run by security teams late in the dev process, long feedback loops, issues thrown

Starting point is 00:14:02 over the walls, inevitably just frustrating developers. And all of this was just built on this fundamental belief that the only viable path, and by viable I really mean to sustain, sustainably effective, the only sustainably effective path for software-centric organizations to meet the challenge of becoming and staying secure was for them to take this developer-led approach to that challenge. So really kind of complete disruption of the industry. And developer adoption for that reason was always a key priority for us. So with that in mind, Sneaker's just been free to use in some capacity from day one.

Starting point is 00:14:38 And the early strategy was always about creating something valuable that was readily available, something that solved a real problem in a uniquely differentiated way, and making it pervasive. So with dev adoption, you know, this core concern, a freemium go-to-market strategy was just the obvious approach. So eventually getting back to your original question, that's all of the context and where they went and how they did it. But the first 100 or so users really just came from the founders engaging with the Node.js community and the interest that drove.

Starting point is 00:15:09 I think we probably had maybe around 5,000 free users before there were any attempt. at monetization. Awesome. There's a bunch I want to unpack there because what's interesting, the way you describe it, maps to the series that I recently wrote about

Starting point is 00:15:21 consumer growth strategy and how the first three steps after you have an idea is to come up with your super specific who, kind of your target persona, come up with a hook that catches them, gets them excited,

Starting point is 00:15:32 and then go find them where they are and pitch them, your hook. So the super specific who for sneak was, you said, open source developers

Starting point is 00:15:40 working on Node. Is that right? Well, There was Node.js developers and Node.js developers were building their applications using open source node components. Got it. Was there any other constraint to that, do you know, or those were like the two to three attributes of a user?

Starting point is 00:15:58 That's really it. The community was growing, for sure. It was big enough to have a decent opportunity there, but narrow enough. Technology-wise, it didn't mean, it meant that a product could be brought to market in a reasonable time. Yeah, that's great. And then the hook was basically, do you have known vulnerabilities in your code base? Which, if I were an engineer, like, I don't know, shoot, I don't know.

Starting point is 00:16:20 Get off my. Exactly. And then, so now you know, right? Yeah, exactly. Okay, cool. And then the where. So you said that they went to open source communities. Do you have any more specifics about what those were?

Starting point is 00:16:31 Was it like a specific forum? Was it like GitHub somewhere? Was it Reddit? Do you know any idea where those communities lived? Yeah, it was less about a particular place, but more about being. the community of developers themselves, we focused on Node.js. So a bunch of early evangelism was really at conferences. It was, I think, the Velocity Conference in Amsterdam where Dianazaf kind of first unveiled sneak

Starting point is 00:16:56 to the world and kind of, yeah, it went from there. I see. Interesting. So it was in-person events, conferences and meetups probably focused on NodeGS developers. Yeah. Okay. Awesome. What happened after that? So that was like the first 100.

Starting point is 00:17:09 It was just roughly the next stage of growth. or did it focus on that for a long time? What was the next stage, roughly? Yeah. I think that focus was the really important kind of element there, if I can kind of latch on that. Starting with that narrow focus and building around community engagement, I think it's a well-proven playbook now, particularly in the developer tooling space, like New Relic did it notably with Ruby Community, for example. But ultimately, it was important because of, you know, this kind of depth versus breadth approach. And that depth first approach that Sneak took was important to be able to effectively validate the solution on the path to product market fit.

Starting point is 00:17:53 A JavaScript developer just won't care if you support Golang or Rust, but will absolutely care if a key feature like automated package upgrades just isn't available for their ecosystem. Of course, the bigger problem of vulnerable components in open source across all languages and all ecosystems, that's a very widespread problem. It affects the industry at large. But that just spoke to the potential opportunity that was there to be unlocked. But the key for sneak, I think, was just not to go too wide too early. So they focused on nailing that initial use case for that specific community of Node.js devs, like I say, narrow enough to be able to really focus on quickly building a compelling solution to a real problem, but also wide enough to be

Starting point is 00:18:37 something viable from a growth perspective. And even back then, the NPM, the Node Package Manager, hosted around 200,000 open source packages. They were downloaded something like two and a half billion times a month by over two million devs. And the typical Node app would have hundreds of dependencies, mostly indirect and so hidden, less immediately visible. But each of those dependencies brought with it some security risks. So yeah, I think nailing that narrow and deep use case before expanding wider was absolutely critical and generally just sound advice around finding product market fit and building solid momentum before casting a wider net. And, you know, it's difficult to maintain that focus for sure as the lure of that bigger tam can be really tempting, but ultimately you have

Starting point is 00:19:21 to build a service and market well to capture it. Do you have a sense of when that focus expanded to an adjacent group, like how many years into the growth story that happened, or was there some kind of milestone there. Because I know everyone imagines, yeah, we'll expand. The question is when and when does it make sense and when is it too early. Do you have any insights into that phase? Yeah, for sure. First, I think if we're talking about PLG, the story with Sneak, and I'd actually like that definition of product-led acquisition from Julian Shapiro when you spoke with him. Beyond that maniacal focus on finding product market fit for your product, founders really should be thinking about how their product is going to grow. And that's important, of course, as you think about

Starting point is 00:20:08 taking that next step. So let's assume in a simplistic definition that you found product market fit as demonstrated by a strong retention. And then the real question is, where are the new users for your product going to come from? And founders really should have, I think, strong hypothesis around this you risk essentially adopting. And if we build it, they'll come approach. So if your acquisition strategy is product-led, then understanding and being intentional about your early acquisition growth loops, I think is an essential founder's responsibility. Dedicating time there to design those loops into the product, it's key. And when I say product, I really mean the whole product experience, considering every touch point in and out of the core application that your users and customers

Starting point is 00:20:50 might have with you. Sneak, for example, believed very early that we could build out powerful content loops via fixed pull request that we raise on GitHub. New users, they'll sign up for Sneak, they'll connect their GitHub accounts, sneak will scan their code, we'll find vulnerabilities, will automatically create Sneak branded pull request to fix those vulnerabilities. Other devs in the repo will see and interact with those PRs. Some of them will follow links to Sneak, create accounts, and some of them will connect their own repos, and so the loop continues.

Starting point is 00:21:21 It's a company-generated, company-distributed content loop. It's actually really powerful for us because it's both an acquisition loop and an engagement loop. Over the course of time, we extended that loop by adding support for other source control systems beyond GitHub. We layered on a bunch of new loops. And I think if founders can be intentional about this, as you're developing early product iterations, then you're going to have a bigger amountage when the product clicks with the market. And that was built in to sneak as we went along. So that was kind of ready as we found product market fit.

Starting point is 00:21:55 But I think to talk about specifically how Sneak took that next stage, it was kind of a function of when we were chatting before this, talked a little bit about some of the failed experiences spinning up a self-serve revenue channel. Actually, before we get there, which I definitely want to get into all that. I love the story you just shared. I hadn't heard of this growth tactic of basically the connector GitHub account. You find all the vulnerable abilities, push a fix. People see that Sneak did this for them, and it just provides all its value. And you're saying that was really effective. It's an example.

Starting point is 00:22:30 It's an example. That integration in terms of connecting your code with security scanning like that was a first-of-a-kind integration. Like, no one had done that before. But the key was that we ultimately controlled the contents. And not only was the fix, poor request, doing something useful in terms of the code, but all of the description of the poor request was explaining about the vulnerability educating users. And it was all sneak branded and saying, if you find this useful, click here, come and learn more about the vulnerability, sign up for an account if you don't have one. It kept existing users coming back and it brings new users, a lot of new users, in fact.

Starting point is 00:23:08 You'd be in there all like, do you have known vulnerabilities in your code base? Click here to find out. So is that responsible for much of the early growth, that loop? I think that was one of the loops. We also have a couple of, from fairly early on as well, other content loops that are more kind of programmatic SEO assets that have both been pretty instrumental in terms of new user growth here. It'd be cool to hear about those if they're relatively straightforward to explain and then

Starting point is 00:23:37 we can get to the thing that didn't work, the self-serve monetization piece you were going to get to. We have a bunch of loops actually at this point. Like you hit off. Yeah, it's, I'm a bit. big loopess, funnily enough. But yeah, we have a bunch of loops. Company-generated company- distributed content loops have actually worked really well for us. We have a sidecar product called SneakAdvisor. SneakAdvisor, it's basically a service that developers use to search and

Starting point is 00:24:08 find open-source packages when they're considering integrating some within their software applications. The unique thing about it is it indexes all of the package managers. It learns about those packages. It augments the data about them with a bunch of metadata, including, of course, sneak security scans, but we also find out how actively maintained the software is on the source repo, on GitHub or wherever. And so we build this kind of package health score. So anyone's searching on Google for a package that does XYZ or a specific package by name, sneak advisor will be right up there in terms of the search results. They'll land on there. They'll get a good idea about that package. They can look at similar packages, and it's all, of course, a sneak website. And we have

Starting point is 00:24:52 CTAs to say, you know, if you want to secure your application on a perpetual basis, then just come and join us. So that's a great loop. And that's all kind of a programmatic asset. There are hundreds of thousands of these package pages, but they're just automatically being generated continuously. Got it. So it's programmatically generated better indexing of open source libraries that you can integrate with. That is so smart. It's programmatic because you can inform on the security vulnerabilities and then the maintenance and activity.

Starting point is 00:25:24 Interesting. Yeah, that makes sense. That's all data you could just gather. That's awesome. Okay. So there's that. Is there anything else that's worked really well for you guys to help you grow self-serve? One of the recent ones that's really interesting is security education.

Starting point is 00:25:37 We think of Sneak as a change agent in helping DevSecOps transform. And it's fine kind of having this capability, but what we really want to get to is this position where developers truly understand and can be better place to prevent security vulnerabilities being injected into their code. And so one of the things that is, again, something that's pretty different from the industry, from an incumbent perspective, is that we believe it's really important to democratize security education. And so we have been building this bunch of really high quality, but bite-sized lessons about developers, focus on developers about security issues and vulnerabilities. And we surface them. Again, they're out there in the public domain. There's no paywall to get access to those.

Starting point is 00:26:25 All the traditional solutions you need to sign up. You need to pay to get access beyond more than a couple. But these were just, they're all out there in the public domain. So that works really well for us from a company generated company. company distributed loop as well. So cool. So SEO and then integrating to GitHub in that interesting way. I imagine there's also a lot of intra-company variety when someone uses sneak-out-a-company and they spread it to their colleagues. Yeah, I mean, I didn't talk about those. I think those are pretty well understood. We have both referral loops and invite loops as well. Okay, awesome.

Starting point is 00:26:58 Coming back to what didn't work, and I think you mentioned that there was a monetization attempt that was self-service oriented and that had some challenges. Can you talk about that? At the time, you know, a few things were in place. Valuable product check, a strong developer user growth check, strong retention check. But the first self-serve monetization efforts only really saw traction with individual developers paying $100 a month or purchases in larger companies, they just didn't happen, as everyone had hoped. There was a really critical part of Sneaks history. At the time, a bunch of investors didn't lean in, perhaps shying away from early conviction

Starting point is 00:27:39 with the founders on building strong usage without a proven path to monetization at that point. Ed at Bolstar, who I mentioned previously, he was one of the first kind of true believers and was, I think, really key in helping with providing runway during that time. But it was clear that there was a lot of work still to do. The team dived in, they really figured out what the constraints were and through that process really learned about the importance of catering for the broader governance needs of the enterprise buyer. And that meant a couple of things. First, there was a need to build out table stakes features around governance at scale, just things that companies of a certain scale in size expected, reporting,

Starting point is 00:28:20 robust user management and so on. And second, that it was time to move beyond that depth first approach, right? So that depth first approach was absolutely critical in getting to that point, but it wasn't good enough to take the next step. If you think about it, there's a point in a company scale where you start to see diversification of tech stacks and all of those tech stacks need securing. It's obvious in retrospect that only supporting developers using a narrow slice of those tech stacks

Starting point is 00:28:49 wasn't going to meet the needs of the security teams who were ultimately the people who were held accountable for the security of their entire application estate. So the teams worked hard over the next few months, starting to build support for additional languages and ecosystems and adding those table stakes features. I think back then, Sneak were simply ahead of this inevitable curve of developer-first security. At the time, the only buyers were security teams, and Deb First Security for the most part, wasn't something that CSOs and Apset leaders were driving.

Starting point is 00:29:25 But if you look at Sneak, through that lens of, as I mentioned, being a change agent, being a key piece of the transformational journey of our customers' devsecops journeys, you realize how important it was for us to start to build relationships with those security leaders. So it was that time also that it was the right time to bring in the first sales and engineering hires as well. So you basically found it couldn't work, self-serve, independent of sales, being involved in convincing the folks at the top, which makes sense. How do you trust a company with your security, the people at the top responsible for security aren't bought in. Makes sense.

Starting point is 00:30:06 Today, it's less like that. There are organizations where the buying center is still very much APSEC, but there are also many organizations where kind of technical leaders on the buying decision around security investments. What was always true, though, even back then, was the influencing power of developers, regardless of where the buying center was. And imagine as the brand has grown, it's gotten easier to convince people like, oh, yeah,

Starting point is 00:30:34 look at all these other logos using this. It's probably going to be okay. So just to understand, in your experience with Sneak, it never really worked self-served monetization. It worked as a way to get into a company and then developers started using it in small scale, but you needed sales and marketing to really grow monetization.

Starting point is 00:30:52 Is that what you found? I think back then, yes. Now it's a very different story. We have a lot of self-serve-only customers scaling pretty large. Got it. That's so interesting. I rarely hear that you start out with sales being important and then it becomes less important. Or I imagine it's still very important, but there's like a segment that has emerged that can self-serve.

Starting point is 00:31:15 Fascinating. Yeah, I think it is important to acknowledge, though, that the product has always played a really key part in the sales process, for sure. That touches on something I wanted to ask. So, Ed Sims, you've mentioned him a couple times. He's got this awesome newsletter. He talks about you guys all the time. I think he's very proud of the progress of sneak. And he talks a lot about that for developers,

Starting point is 00:31:35 you've got to win hearts and minds of developers to build something that works. Any lessons or pieces of advice for folks that are targeting developers to win hearts and minds and get engineers, developers, excited about what you're building? I think there's two things, right? So, first of all, fundamentally,

Starting point is 00:31:54 for someone to get excited about using a product, they've got to care enough, right? They've got to have a problem that you're solving. So I think there's two things. One, there is a shift that is happening and still happening. I think there's still a long way to go for developers to really care about security as an integral part of their job

Starting point is 00:32:17 in the same way they think about functional quality or performance. I think that we're still, we're making strong progress there. It's changing all the time, but there's still a long way to go there. But the reality is that I think in most companies, developers have to care about security because their companies need to be secure. So the key then is how do I make the job of being secure for these developers as painless

Starting point is 00:32:46 as it absolutely needs to be? And that means really meeting them where they are, integrating with their tools, finding ways to take security to them instead of trying to pull them out of their workflows. Flow is just this incredibly important concept for developers, and you want to strive to keep them in that flow for as long as possible. So the GitHub kind of poor requests are a great example of that. Someone can sign up for Sneak, and they could theoretically be the only user of Sneak and connect their repos.

Starting point is 00:33:19 All of a sudden, we're protecting, securing those repos, suppose 100,000 developers could be working in GitHub with that code or benefiting from Sneak without necessarily needing to sign up. That's that example of kind of taking the product to users without pulling them out of their workflows. And I think that's absolutely critical. As an outsider hearing all this, it's a product that magically helps you avoid security issues, very little work, does a lot of the work for you.

Starting point is 00:33:50 it's hard to imagine it not working, looking at it now. And I'm curious what it was about the early days that just felt like maybe that people didn't believe in this working. Is it just there was doubt that it would be smart enough to find your security vulnerability issues? Was it the timing wasn't right? People weren't ready. We weren't like concerned about security enough. What do you think it was that created challenges early on? Because looking back, it's like, of course this is going to work.

Starting point is 00:34:16 How could it not? It sounds just like a magical all-win product. First of all, I don't think the challenges were there in terms of the developer adoption. So even when those first kind of forays into self-serve were struggling in terms of breaking into some of the larger customers, the developer adoption, the free user base was still growing at a really good pace. That momentum was just constantly building, and it's that momentum that is ultimately fueled the sales-led business as we've gone through the years. But it was just those few things, I think, that I mentioned earlier in terms of stumbling blocks that needed to be overcome. Because when those first sales and marketing hires did join us and we started having conversations and we also tweaked some of the things in the product to meet some, you know, had some breath, had some additional languages, ecosystems, building those table stakes features.

Starting point is 00:35:11 Then it kind of really unlocked. And there was kind of rocket ship time from then. Got it. So sounds like the biggest issue. was monetization. Can we make money doing this? Developers love it. They're using like crazy, but will people be convinced to scale this and inside an organization

Starting point is 00:35:26 and pay us a bunch of money for you? Okay, got it. So I want to dive a little bit deeper into your growth team and product team and how you think about organizing teams like that in a product-led growth, sales org. So the first question, just, how did the growth team start at Sneak? What was kind of the early days? And then what does it look like today? There was some ad hoc effort.

Starting point is 00:35:48 happening in various places. We had a small growth marketing function. We had DevRrel. We also had some ownership of key growth surfaces in R&D. There was a team that owned the new user onboarding flows, for example. But it wasn't until I joined that we really formalized the notion of a growth team. It was very kind of ad hoc before then. When I joined, we created what we call the developer growth group now. Before then, there maybe wasn't as strong an understanding about what a growth team needs to look like, how they might need to work differently to the core product teams. And I'd say overall, it was much later than you typically expect to see and at a bigger scale. You normally are going to start growth teams, one or two people, three or four maybe and scale

Starting point is 00:36:35 out from there. But we started much bigger than that. But at the same time, this bottom-up developer-first approach, it was baked into the company DNA in terms of how teams think and operate. So, yeah, we were growing fast even before we spun up the growth group. I think the significant change that happened there was it was a transition from a simple freemium approach to a holistic and well-coordinated PLG strategy. It's much more common to start earlier, much more common to start at a smaller scale that we did at Snec, but it worked for us because of this kind of perfect storm of where we had a product with that bottom-up growth built in from the beginning. We have founders with a deep appreciation for how the product could grow. And there was strongly exec alignment and sponsorship for scaling the

Starting point is 00:37:21 motion. So the problem when starting the growth group, it was really the most part more oriented to how we can get the flywheel spinning faster, as opposed to getting it moving in the first place. So where did you initially focus that team, which part of the flywheel? Right now, we have dedicated teams focused on acquisition, activation, and monetization, along with a supporting team who own our growth platform, including all of our data and experimentation stack. But the macro structure, it's changed over time to enable us to focus on the biggest constraints in our growth model.

Starting point is 00:37:55 So at the beginning, we just focused on acquisition and activation, intentionally deferring any investment into specific monetization initiatives around the self-serve revenue channel until we felt confident that, A, we'd built the necessary growth muscles to scale, and B, we'd figured out some of the more pressing issues that were present earlier in the user journey. So it was important that we felt really confident about our ability to effectively connect developers to Sneak's value in such a way that introducing and optimizing a self-serve revenue channel would make sense. I also really wanted to avoid one of the common failure modes I've seen around cross-functional collaboration and growth. So when I joined, there was an inherent tension built into the system.

Starting point is 00:38:38 It was particularly noticeable between R&D and the growth marketing team. I had amazing people in both teams, a ton of really great ideas, but many of them were just not being executed on. And it was leading to a lot of friction, a lot of frustration, ultimately caused by misaligned incentives between the different functions. So when creating the growth group, we resolved this by ensuring that each of the growth teams were truly cross-functional in nature, with everyone in each team aligned around common objectives and KPI

Starting point is 00:39:08 every team has engineers, an engineering manager, a product manager, a designer, a growth marketer, decision science support, and a basic shape of the growth teams that will be familiar to most. But I spoke to a bunch of people over the last couple of years, and I've actually learned to my surprise that inclusion of growth marketers in the product teams is not all that common. And I personally think there's just a lot of opportunity being missed there, and I'd expect that to start to become the norm rather than the exception over time.

Starting point is 00:39:37 Okay, I want to talk about that, but before we get there, you said there's a decision science person on the team? What is that about? That's cool. That's right. So it started off from a fundamental BI data analyst perspective, but over time, we wanted to apply a much deeper level of analysis on the data such that we could start to build in predictive models that could help us make better decisions and can ultimately, fuel. and power some of the in-product experiences. So, yeah, we spun up a decision science function, and, yeah, those folks are very smart. Is that similar to data science, or is that a separate? It's cool that you call them decision science people versus data science because that's so much more actionable. Yeah, I think so, yeah.

Starting point is 00:40:31 Wow, that's cool. All right, I hadn't heard that before. It makes me think of Annie Duke and all the stuff about how to make better decisions. And I love... Have you... Is this something anyone else does? Or is this something you came up with calling...

Starting point is 00:40:43 I'm not sure. I don't necessarily think what we're doing is revolutionary there. But maybe the name. I'm not sure. Yeah, the name is cool. I haven't heard that before. It implies a bias towards action

Starting point is 00:40:56 versus just... We're going to do a bunch of cool stuff and data. Interesting. Okay. And then you said that there's a growth marketer embedded on each team. So maybe just broadly makes up these teams,

Starting point is 00:41:06 which you touched on briefly, and then what have you learned is the value of having a growth marketer embedded within each team? It's important to have balanced teams with strong diversity across multiple vectors. Focusing on functional diversity at the moment, which is kind of what you're asking about, with having growth marketers on the team, one of the big benefits you get is a broader palette of ideas, but also a bigger toolbox when it comes to execution, which generally translates in an ability in a growth team for them to test and learn faster with more parallel yet at the same time aligned threads.

Starting point is 00:41:43 Perhaps I can give a recent example there. So having a growth marketer in an acquisition focus team led us to some lightweight experimentation on the website in creating an SEO optimized page. It was something that was really high performing, both from the perspective of traffic and conversion, but it didn't require any engineering resources to create. So the growth marketer in the team, and they decided. together, this was something worth pursuing, but the growth market was able to kind of execute that independently while engineers were working on other things. But then based on the success

Starting point is 00:42:14 of that, the team went on to build out a functional sidecar product that allowed users to basically try sneak without needing to sign up by simply pasting their code in for us to scan and giving them some results there and then. So, you know, we saw really great results with that. Visitor traffic saw a significant increase. Sign up rate dropped a little bit as we'd expect it would, but overall new users had a big bump, and those users had much higher intent, which we saw play out with increased activation rates. Awesome. Okay.

Starting point is 00:42:44 And so there's essentially four teams under the growth umbrella. There's acquisition, activation, monetization, and then this kind of experimentation platform team. Is that right? Yeah, that's right. And that team is also responsible for making that data available elsewhere in the organization as well. Product-led sales is a really important motion for us.

Starting point is 00:43:05 And so taking the knowledge we have, the insight we have around behavior with users and their teams and their companies within the product and making that available to the GTM teams outside in smart ways, allowing them to focus on the things that are most important to focus on, that's a really important part of what that team does. It's interesting. You guys are the epitome of product-led sales. That's this new trend of from PLG to PLS for sales. It's obvious that they're a big part of this whole process and the fact that monetization happens almost all through sales is interesting.

Starting point is 00:43:40 So that's interesting. Cool. Okay. That's not a question, just a thought. Talking out loud. One other thought I had is, so you talked about SEO being a really important part of your growth. What is the person team like to do the SEO piece, the right content? I imagine they're on the acquisition team.

Starting point is 00:43:58 There's maybe a content person. that lives within that team? We have actually one of the smartest SEO people I've ever met within that group. What's their name? Let's give them a shout-out if you want. Anna. Yeah. Cool. Go Anna. Well, she's part of growth marketing, but she works extremely closely with the growth teams, and she's got a few people in her organization, and we bring them into specific SEO-focused initiatives when we're looking to build loops around that. Yeah. Incredibly important to have someone like that who understands that at a far deeper level than I could ever hope to, how SEO works, and particularly in terms of keeping on top of some of the things that Google are constantly doing

Starting point is 00:44:41 in terms of their algorithm changes. And does she actually do the writing for editorially, for I guess even the programmatically main pages, or there's someone she outsures it? No, but what she does do a great job of is providing the kind of continuously updated guidelines on how content should be structured to lead us to good results. And then it's just engineers and PMs that end up writing the things. Wow, cool. That's exactly right.

Starting point is 00:45:08 This episode is brought to you by Vanta, helping you streamline your security compliance to accelerate growth. If your business stores any data in the cloud, then you've likely been asked, or are you going to be asked about your SOC2 compliance? Soct 2 is a way to prove your companies taking proper security measures to protect customer data and builds trust with customers and partners, especially those with serious security requirements. Also, if you want to sell to the enterprise, proving security is essential. SOC2 can either open the door for bigger and better deals or it can put your business on hold.

Starting point is 00:45:45 If you don't have a SOC2, there's a good chance you won't even get a seat at the table. But getting a SOC2 report can be a huge burden, especially for startups. It's time-consuming, tedious and expensive. Enter Vanta. Over 3,000 fast-growing companies use Vanta to automate up to 90% of the work involved with SOC2. Vanta can get you ready for security audits in weeks instead of months, less than a third of the time that it usually takes.

Starting point is 00:46:12 For a limited time, Lenny's podcast listeners get $1,000 off Vanta. Just go to vanta.com slash Lenny. That's V-A-T-A-com slash Lenny to learn more and to claim your discount. Get started today. What advice do you have for folks building growth teams in maybe there's similar space or just B2B PLS oriented businesses in general? What have you learned about what is important to get right? This is a big topic.

Starting point is 00:46:42 I have a lot of thoughts here based on what I've seen work well and also not so well. I think I can broadly bucket things here into maybe three main topics. So the first will be people in process, the second. Second would be strategy, and the third would be data. Now, they're all related, and they all have to be working well to be effective in growth. Starting with people, I'd maybe touch the first element there in terms of balance teams. You've got to have those balanced teams with diversity and ability to create great ideas, but also ability to execute.

Starting point is 00:47:19 That's the first thing, but still on the topic of people, I have this kind of mental model that potential at its core is unbounded, but that a bunch of things situationally prevent people from fulfilling their potential. You know, it might be how they're thinking about something. It might be organizational. It might be in relationships with co-workers or in broader team dynamics. It might be in them not even being in the right role even. So fundamentally, I think it's the role of managers and leaders to help them identify those things and work with them to find ways to thrive and grow. I've seen this have particular significance in a growth org where people are just naturally less good fit. Now, that doesn't mean that they're not amazing, talented humans, right?

Starting point is 00:48:02 It just means they aren't going to do their best work in a growth context. So when you're starting a growth team, you'll often be doing so with internal moves. So this can be something that may be a little bit easier to miss than with external candidates where you're likely testing for those things explicitly during the hiring process. I'll share an example with developers. So the devs that really thrive in a growth context are the ones that are motivated by moving quickly, iterating to create measurable impact, they're not attached to their work, they embrace imperfection as part of the process, they happily discard their code, their ideas even.

Starting point is 00:48:40 You know, they're curious and they're always looking for ways to be closer to their users. Now, those are the folks that generally make great growth engineers. And I've also known incredible engineers that are most motivated when they're working on really deep technical challenges and love the process as much as the outcome, and they've struggled in growth. Of course, it's never that simple. In reality, there's nobody who's really at either extremity of that spectrum, but it's really important to try to answer the question, can this person do their best work in this environment? And that's a really key part of it. And you need to also make sure they're well equipped from this kind of skills and knowledge perspective as well. So they need the right skills and knowledge to be able to do their best work in the context of your growth process.

Starting point is 00:49:27 So the ideal state is that every growth team member has common vocabulary. They're comfortable with the growth process. They can work well with data and experimentation platform. They understand the data. They have the right skills and mindset. Education, I think, plays a big part here. So we're big fans of Reforge, but we've also developed a bunch of internal programs to align and up-level the teams. Something we learned, I think, as an example, is the importance of starting simple and going deeper as the teams build experience.

Starting point is 00:49:58 So, for example, when it comes to experimentation, don't try at the beginning to introduce multivariate testing or concepts like sequential sampling as an alternative evaluation approach. teams are still trying to just dip their toes in this water. It's kind of a recipe for a lot of mistakes. So, yeah, that would be some advice there. But people also need to be well aligned. I've talked about this actually on another podcast that I did recently, but what I mean by that is their execution needs to be aligned with an evolving growth strategy. The growth strategy needs to be aligned with and at the same time influence where the company's going.

Starting point is 00:50:39 The growth strategy needs hooks into the product strategy, and ideally there's some overlap and alignment of KPIs so the growth teams and the core product teams are swimming in the same direction. The skills and experience in the team need to be aligned with the strategy. If you've got to focus on activation and acquisition, then someone who's, you know, a plans and pricing expert probably isn't the right set of skills at that point in time. But you also, leaders need to plan to ensure those skills are available as focus of the growth team inevitably share. trips over time. At the end of the day, I think everyone needs to be able to easily answer the question why they're there, why the work they're doing is important. I don't know if it's too far

Starting point is 00:51:22 off track here, but I have a vision and mission framework that I like to use that leads to, I think, great simple statements of an imagined better future state and your role in getting there. So I can talk about that a little bit if it's a... Yeah, let's do it. That sounds too good to pass up. Cool. So the vision is the Nirvana state that you aim to enable for your users and customers in five to ten years. It's something that could equally be enabled by your competitors if you don't execute effectively or efficiently or quickly enough. You can always prefix a vision statement with a in the future dot, dot, dot. Now it's something that should be bound to your target market, so not too wide and not too narrow. And critically, it should not mention your company, your product, or anything solution related at all, completely agnostic of those things. And then the mission, that's the thing that you're going to relentlessly iterate on to take you incrementally closer to the Nirvana state described in the vision. It should answer how you'll realize the vision by describing what your fundamental approach

Starting point is 00:52:24 will be. In other words, what you will do and how you'll do it. It should ideally aim to encode any unique differentiating advantage you have. So if I think about sneak at large, you know, when it comes to, comes to advantage, we might mention our unequalled security intelligence and knowledge of application context. And one of the neat things about this framework is, if you find utility in doing so, you can apply at every level from the company level all the way down to individual team. It doesn't mean the process is not difficult and you need to spend a lot of time, but it gives

Starting point is 00:52:56 you this set of kind of framework, a set of bounds that help you really come out with well-crafted statements that kind of stand the test of time. Is there an example you could share the mission vision, whether it's Sneak or something else? Yeah. Yeah, I'll give you one for the growth group of Sneak. So the vision, every developer securely unleashes their creativity. And the mission is to connect every developer and their organizations to the value of the SNEK platform with frictionless self-serve adoption and expansion. Interesting how the vision is so big beyond Sneak's current focus.

Starting point is 00:53:31 And to your point, this kind of trickle down, right? There's a company vision and mission, and it's the growth team's mission vision. Awesome. Exactly. I want to shift a bit and talk about your product org. So we've talked a lot about the growth org. I'm curious what the broader product org looks like.

Starting point is 00:53:48 How many teams do you have? How do you structure? It is a product focused, user focused, outcome focused, and then just how does that team work with the growth team? Sure. Happy to go into a bunch of that stuff. I know I'd mentioned earlier, kind of three areas of building a growth team. I don't know if you want me to cover those other two

Starting point is 00:54:07 because we talked about. Oh yeah. Okay. Let's do that. Let's finish that thread. Cool. So people in process was the first and we'll cover process really quickly. I think on the process side, at a minimum here, you need well understood documented growth processes, practices, working cadences. Teams in growth need to work differently in some ways from R&D teams working on core product, assuming otherwise and just giving growth responsibility to an existing team without working. working with them to implement appropriate ways of working, I think it's a common trap. Ideally, you get to a point there where the growth process is something that's continually refined and iterated on, trying to build in more predictability.

Starting point is 00:54:45 But what I've found, I think, most, singularly most important in a growth process is facilitating a rapid learning cadence and providing the means to socialize those learnings, surfacing them in the right place at the right time so they can be leveraged in other context. And if you think about experimentation, it's not about delivering outcomes. It's about generating learnings that the organisation can leverage effectively to deliver outcomes. Might not be now, might not be tomorrow or some point in the future. But the sad reality is that without good process, learnings easily end up unused and gathering dust. And you have to ask then what was the point?

Starting point is 00:55:23 So that's people in process. And you know, you know when that's on the right track, when you start to see enthusiastic sharing of learnings, when you see regular contribution of ideas coming from everyone in the form of well-crafted hypotheses that are based on data and learnings. And when you see a wide variety of folks just assuming end-to-end ownership of managing and running experiments instead of delegating that to the product manager or an engineering tech lead. So yeah, that's people in process.

Starting point is 00:55:52 And strategy is the... Let me throw in a question real quick before we get to the last piece. Sure. So learnings, just to kind of touch on that. I've heard more and more pushback on the idea of learnings being an outcome because a lot of, like, as a leader, you're not going to be like, cool, we learned so many things, but nothing really got done. How do you think about the tension between, yeah, learnings? We want to learn. We also want to, like, move metrics, grow the business, and learnings are a way to inform decisions more than even just learn.

Starting point is 00:56:20 How do you think about that kind of balance? Ultimately, you're there to create impact, right? There's no getting away from that. But learnings is the means. It's the same as there's a quote that I love from Farid Masawa around focus on the user's path to value, not on monetization, because if you focus on the form of the latter will follow, and that's the same thing with learnings and impact here. You know, if you try and focus on the impact itself, it might struggle.

Starting point is 00:56:45 If you focus on the things you need in terms of learnings to take you step by step, that will pave the path to creating impact. I imagine your OKRs and goals are still, like, move this metric, some percentage, but you Think of that as an output and the input is let's learn a bunch of stuff about what works and doesn't work and use that to inform what we're going to double down on. Exactly. Yeah. Sweet. Okay. You've got to focus on when you build a strategic opportunity, you're thinking about the outcomes, but you don't just go right to the end state. You've got to think about what's the quickest way we can test this hypothesis. And from there, what do we learn from that and what do we take into the next set of experiments?

Starting point is 00:57:25 and it kind of you're paving this path along the way. You know kind of that that rough destination. How you get there, you don't know at the start. And that's the part that the learnings take you on. Cool. Okay. And then strategy we're talking about. Strategy, yeah.

Starting point is 00:57:41 So strategy, it's a good one. At a very basic level, you need to be able to answer questions. How do you acquire users? How do you retain users? How do you monetize them? I know you talked with Elena on that in more depth, but from there you need, you know, you need more detail that's going to guide the growth teams and where they look for strategic opportunity, how they approach that.

Starting point is 00:58:06 The best way I've found to articulate a growth strategy that fulfills the promise of usefully guiding the team's execution, it's the loop-based model. We forge as specific kind of documentation that I think is great around that. being able to identify the various micro and macro loops, how they're all connected, being able to document them in a qualitative model to communicate a shared understanding of how you grow. It's really powerful. Augmenting that then with, you know, the quantitative side of things that helps guide quarter to quarter focus and ensure you can be intentional about where you're investing.

Starting point is 00:58:44 That becomes a big enabler. You know, you're never going to have a shortage of ideas in a high-performance. and growth team. So knowing where to focus amidst that kind of see of ideas is a really important role of the strategy. And early on, you know, you'll probably have one or two core loops, but inevitably you'll need to layer on and connect new ones over time. And having a framework for doing that and having a framework for regularly revisiting that in the context of growth team learnings, changes in broader company strategy, evolution of the product, new features being added and so on, market shifts, that becomes a big up-leveler for teams to be able to create timely impact.

Starting point is 00:59:25 So the model you create there is what enables you to know at any time where the biggest constraints to your growth are and allows you to balance your growth investments accordingly. At Sneak, for sure, you know, we've had times where we've focused on a broad set of constraints and opportunities, and other times where we've had a much narrower focus, for example, on driving improvement to activation and engagement, even more narrow, like doing that through empty states, for example. I want to unpack the strategy piece real quick. How do you operationalize this idea of you have this growth model, growth loop you figured

Starting point is 00:59:56 out, here's the ways we're growing? How do you tactically connect that to a strategy? Is it here's how we're growing? Here's where we're going to focus this quarter, optimize this part of the loop, or is there some other way you write it down, basically? Very simply. It's, first of all, alignment on how you grow, the different loops, how they work. the roles of different teams and the roles that they play in fueling those loops and getting

Starting point is 01:00:24 them to spin faster. I think that's the first thing, having that common vocabulary and understanding. Second is understanding the way they work in terms of what is constraining them, what are the inputs to them, what are the opportunities for making those loops spin faster, how can multiple loops work together in a macro context? And particularly using the data there to be able to identify and understand and get alignment around where are the biggest constraints in your growth model overall. And therefore, where are the things that we need to focus on as a team over the next quarter, two quarters and so on? And it's constantly changing.

Starting point is 01:01:11 As I mentioned, there's a bunch of stuff that needs to feed back into that model in terms of growth loop performance or the learning. learnings you're making and so on. That means it's changing and you're constantly re-evaluating and it's guiding kind of quarter to quarter planning. Cool. So just to make it even more concrete for folks that are listening, so one of your loops is this integration with GitHub where you automatically fix their code and create a PR, you may find at some point somebody clicking that link to becoming a user is too high friction, too many people are falling out. So one strategy for one of the, say, the activation team could be, we're going to optimize this. part of the funnel.

Starting point is 01:01:49 So I'm clicking from GitHub to sign up as a user and get them to a point where they found value. Right. Yeah, exactly that sort of thing. Or earlier on, for example, it might have been taking that same example of that same loop. And this is sufficiently far in the past that it's easy for me to talk about.

Starting point is 01:02:06 But we start with GitHub. We want to now expand the scope of that loop. We've seen success with it. We see the performance of that loop growing. But we think there's opportunity in saying, let's expand. that by adding support for Bitbucket. And now all of a sudden we spin up that same loop with Bitbucket and then GitLab and so on.

Starting point is 01:02:27 Awesome. Okay. We could go a whole hour on just that piece. So we'll move on and we'll save that for a fee too. Data was the last thing. Oh, yeah. Growth team just can't function without data. Even with early stage products before you have the needed volume of traffic to be able to run

Starting point is 01:02:45 formal tests in reasonable timeframes, you still need to have. have signals that help you learn and inform your decisions, both Qant and of course, Qual through speaking with users. My advice there is really just to invest early, the infrastructure, the tooling, and at a given scale, the dedicated people as well, they're going to be core to building out the growth team and they'll enable data to ultimately pretty much inform all critical decisions. So sneak was interesting that we didn't have a problem of not having enough data. In fact, there was an abundance of data, like too much. We collected absolutely everything. And the problem I identified very early was that we didn't have enough behavioral

Starting point is 01:03:25 specific data. And we weren't intentional enough about the data that we were collecting and how we were collecting it, which made the data hard to trust. And that becomes a big problem. So we invested in tooling and processes for building out event tracking plans. And now we test conformance to schema of the instrumented code in our CI processes. So we have have absolute confidence in the data. So that was, you know, getting to a point of trustworthy behavioral data was absolutely key. But the reason I say invest early here is that to remember that it also takes time to accrue enough data that you can learn about and make some key decisions around retention, for example, or so that you have enough data to run regressions on, be able to

Starting point is 01:04:07 inform definition of your activation metrics or engagement states and so on. So the early you can start collecting the better. So yeah, people in process, strategy and data. And you absolutely need all of those things to build and run and effective growth team. When you get all of those things working, it's like rocket fuel for focus creativity. But at the same time, you know, slowing down to put the maturity in place as an augks scales at the pace sneak has, it's pretty much impossible. You still have to get stuff done while you're kind of building all that stuff out, right? You have to accept that you're going to make a bunch of mistakes along the way. You have to be 100% comfortable with that and you have to treat those mistakes as learning opportunities that

Starting point is 01:04:50 provide levers for improvement. And it's also useful to, you know, you asked about KPIs and what the team are responsible for. And that is one way in which a growth team absolutely needs to make impact and it's the way that primarily they're going to be held accountable. But it's also, I think, useful to think about the efficacy of the growth org, not just in terms of the impact they drive via experiments and new product experiences and on core growth pick KPIs, but also how they enable and up-level the rest of the org. So, for example, our entire product-led sales process, it's powered by an evolving model that describes our understanding of users, teams, accounts, the usage and adoption patterns and

Starting point is 01:05:31 signals that best inform where and how our GTM teams focused. You know, insights from growth teams often have utility far beyond growth, but people can't know if something is useful to them if you haven't shared it with them. So the learning's made in the growth teams, even those from mistakes or failures, we socialized them widely and visibly. We want every R&D team to be leveraging experimentation where appropriate to learn, to create business impact. So one of the things we did here is creating a paved row for adoption of behavioral analytics and experimentation stack, coach teams on getting started to make it as easy as possible for anyone to start to reap the benefits of the platform, built out internal education programs on on data-driven development, on experimentation, built internal tools to help with metrics design and so on.

Starting point is 01:06:18 And then building core platform services as well that are useful for people outside of growth. So we built services that power contextual onboarding. And originally that was the intent, but now those same services can be used anywhere in the product to give contextual experiences. I know that was a bit of a ramble, but hopefully there's one or two useful things in there. Yeah, there's two things I wanted to quickly follow up, and then we can talk about the product team.

Starting point is 01:06:43 You said you socialize learnings and experiment results. Is there anything you could share there about tips to do that? Do you have a tool you use that for? Is there someone posts stuff in Slack? Is it an email that goes out? How do actually socialize learning such that, say, a salesperson can do something with it? Yes.

Starting point is 01:07:00 So first of all, there's a bunch of Slack channels like, Sync runs on Slack, basically. So there's a bunch of Slack channels. Even when we're planning experiments, those are kind of wide and in the open, and we invite collaboration on those. But from a ceremony perspective, we try hard to institutionalize ways to generate and leverage learnings. It's something, you know, I feel pretty strongly about. We have these team-level impact and learnings reviews loosely modeled from a blog years back down,

Starting point is 01:07:29 I know, six years I want to say, that Brian Balfour wrote about, a similar ceremony from these HubSpot days. And if I had to pick one meeting that as the most important in the growth team, it would be this. The teams continuously document any learnings from data exploration, from experimentation, from user research and so on. They document that in their weekly impact and learnings document. Some teams find it better to pair up in advance into dedicated learning sessions to deep dive on specific relevant topics, but however it comes together is all kind of put into that document. When it comes to the meeting itself, it's usually run by the PM.

Starting point is 01:08:06 Most of it is spent discussing the learnings that have been documented, their implications, how they can be leveraged in follow-up work, where they might have relevance to other teams and so on. A relatively smaller part of the meeting is also spent looking at key metrics. Some teams have actually split that out entirely into a separate meeting. And then no time at all is spent reviewing what the team have actually been doing. It's more on kind of the outcomes and the learning. So that's at the team level. but then we run that same meeting at the group level on a monthly basis.

Starting point is 01:08:34 So that's run by the product, engineering, or marketing director for the growth group. And that's where all of the growth teams come together. They share some key learnings. Can't share everything, of course. What they're doing is picking specific learnings that have potential relevance and utility across the other teams. So also as a standing agenda item for I user research team to share what we call developer insights. That's one of my personal favorite meetings to attend. It's always recorded and socialized with the rest of the company afterwards.

Starting point is 01:09:03 And yeah, I'd say that's really important. But there's a bunch of different ways in which we're fanning out that information constantly. So cool. And this is a meeting that anyone can come to, like a sales person comes to, to see what interesting stuff the product team has learned recently from experiments. How cool. Would you be able to share a template of that document that you put together that we could include in the show notes? Yeah, for sure. Sweet.

Starting point is 01:09:25 And then the meeting, you said it's run by product, basically like some of the product functions. and ideas to share things they've learned in recent experiments, say, in the past month. Typically, the team level ceremony, it's PM-led. That's more from a facilitation perspective. The learnings are all brought forward by the various folks in the team, and each one of them who's contributed to learning will talk about that learning with the group and facilitate the discussion from there. And then at the group level, those are read by the directors for the growth group,

Starting point is 01:09:57 And each team, it might be an engineering lead, it might be a tech lead, it might be a designer, it might be the product manager. We'll talk about some key learnings from that month. Makes sense. If there's nothing PMs are good at, it's facilitating meetings. So that makes sense. Okay. And so that's a good segue to just chatting a bit about the product org.

Starting point is 01:10:16 I'm curious just like how you structure the product team and then how that works with the growth team. That's good. Yeah, so the product org, what do you want to know? Broadly, how do you structure it? Just like how many teams do you have? have, do you align it across by outcome or is it by a surface area of a product? And then is the growth team like adjacent to this product org? Is it like a unit within the product org?

Starting point is 01:10:38 Or is it integrated, but I don't think it is. So yeah, just how do you, what does that look like org chart wise? Yeah, I think we've got a fairly common pattern for how we structure our product and wider R&D org. So most of the org is functional ownership based. There are a lot of really complex domains in the core products. So having that localized knowledge is important to be able to kind of own and run the code that team ship. The growth teams on the other hand are structured by outcomes. We talked about that already, owning areas like acquisition, activation, engagement, monetization.

Starting point is 01:11:14 These outcomes and team remits change over time, as we talked about. But the teams here are often working on areas of the product where they don't. own the code, which I think is the key difference between how we structure our growth teams and product teams, with some exceptions, you know, one of the growth teams actually owns the onboarding flows and so on. So that does require a lot of trust. It requires very transparent communication mechanisms built into how we work. One of the meetings that we have regularly is experiment plan reviews. They're kind of ad hoc, ad hoc meetings. They're led by the experiment lead, could be the PM, could be anyone else in the team. But the important thing is a bunch of people

Starting point is 01:11:55 will be invited there, especially stakeholders from other teams where we might be experimenting on their surfaces. And that won't be the first time they've learned about this. We'd like to try and actually include them in co-designing the experiment plan so they're kind of fully on board with it, but absolutely kind of inviting them into those experiment plan reviews really key. If we're going to run an experiment on that surface, you know, we need to make sure that everything in that experiment plan is watertight, especially from a scheduling perspective, because the last thing we want is a week and a half into an experiment for some change to happen within that surface from the product team, unaware that an experiment's happening and completely invalidate the experiment.

Starting point is 01:12:36 Cool. And then in terms of just org-wise, do you have a lieutenant that is responsible for just, say, the product team and then someone responsible for the growth team or the directors report up to you and you have a bunch of reports? How do you, has that structure? So our CPO on the exec team, Manoj, he runs the entire product organization. He has four, I want to say, VPs that own different areas. So there's a couple of VPs that own different areas of the product. So first of all, our application security products. Secondly, our cloud security products.

Starting point is 01:13:11 Third is our platform. And fourth is what we call developer journeys, which is the area I own. which has a few groups within there, one of which being the growth group. Got it. Okay. Makes sense. Okay. There's just a couple more questions I want to ask that are very tactical and specific

Starting point is 01:13:29 before we get to a very exciting lightning round to close this out. One is with a freemium product, there's always this question of what to put into free and what to put into the pay plan. Is there anything you've learned about how to think about that, what should be in free and what should be behind a paywall? Was it on your show with Elena that she talked about? about kind of things that promote your growth model being good to land in free and things that add friction. Yeah, that's right. Yeah, so I really like that guidance. I'll add that for many

Starting point is 01:14:00 businesses, there might be some cost of service element to consider as well. You know, of providing a feature to free users is cost prohibitive due to the volume, then that's obviously something you're going to want to reserve for paid. I mean, ultimately, that was the whole reason cited behind Heroku recently removing their free plan entirely. I think your plans from free to the top, they should have well-defined understanding of the target customer, their use cases. They should map out, or you should map out the motivations for motion between each. You're really clear about what are the drivers for someone to take a step from one plan to the next. For sneak, the real drivers to move from free to a paid plan, for example, is when you want to secure business critical code.

Starting point is 01:14:44 And you start having needs around governance and compliance. I think the other interesting dimension is how you approach trials. Like with most things, I think we're still figuring that out at Sneak. I don't know that there's ever a perfect answer or even a correct answer here. Certainly different from product to product. We have a self-served trial to support time-boxed evaluation of some of the capabilities that are reserved for our paid plans. But we're intentional in revisiting the model periodically.

Starting point is 01:15:12 It's important, I think, to regularly challenge yourselves to ensure you don't fall into the trap of simply assuming what was best fit in the past is best fit now and in the future. What if the trial duration was limited by some dimension of usage instead of time? What if we didn't have a trial at all, but put more into the free plan with appropriate limits? And might making those changes impact our growth model. It's not always easy to answer those questions, but I think there are some ways that you can help test there. For example, you know, you might cohort trial users and take.

Starting point is 01:15:46 teams who have low engagement and don't convert during the trial. And then when the trial ends, drop them back into a new enhanced freak plan and monitor engagement there. So there's some things you do, but I think that habit of continuously challenging yourselves and reevaluating whether the model, the specific delineations between the plans and how you support evaluation and the motion between those plans, I think it's really important to do that. And also, when it comes to PLG and sales, we talked about the self-serve motion.

Starting point is 01:16:20 Obviously, it's big and important for Sneak, but the sales-led motion, critical-critically large as well and significantly impactful. You need to think about the plan design and those motions across both aspects of PLG and the sales-led motion. When you have a strong PLG foundation that is inclusive of a product-led sales motion, you're going to be in a really powerful position from the perspective of having a significant volume of highly qualified leads that are coming from the product. We actually track a metric that we call product-driven revenue, which basically accounts for all revenue in customers where we saw meaningful value-based activity in the product before there was any sales contact.

Starting point is 01:17:08 And that really tells actually a super interesting story about the PLG efficiency of your company across all revenue channels, self-serve and sales led. And what's fascinating there is that the product-driven cohort contribute a relatively greater amount to net retention. So when you think about packaging, you know, you really need to think about and understand that macro-level contribution of the freemium motion and know what you're trying to optimize for, kind of balancing revenue today versus potential future revenue. Is that increase in net revenue retention from product-led leads, mostly because they started the cheaper price?

Starting point is 01:17:45 Do you think, or is it more that they just end up being better customers? It's a great question. I'm not sure I have a good answer to that now. No problem. I still trying to figure that out. It's a good problem to have people adding more customers in seats. You talked about the importance to figure out the trial length and what to put in the trial, freemium and things like that. Is there anything that just has surprised you, something you've learned

Starting point is 01:18:10 from iterating on that comes to mind? I wouldn't say surprised me per se, but it's something that I think is perhaps obvious in retrospect, and that is that companies of different sizes, of different complexities of different industries, from those that are, you know, very highly regulated to the other end of the spectrum, they're going to take different lengths of time. to need to evaluate properly. So being able to cater for those in some way, whether it might be dynamic trial lengths, or whether it be trial lengths that are based on usage, things like that,

Starting point is 01:18:52 I think it becomes really important. That's something to be thinking about for sure. Awesome. That's a great learning. I know Elena talks a lot about how trials often screw you because you don't give people enough time to really evaluate at a company. So that makes sense. Last tactical question that I wanted to touch on is around activation and activation milestones. I'm doing a survey right now with Yuri that I think will come out before this episode airs.

Starting point is 01:19:18 But anyway, in real time, I'm curious how you think about setting what the activation milestone is for a new sneak user. So maybe just share briefly how you think about what is an activation milestone. Why is that important? And then how you define that for sneak. Like, what is the milestone of a user is activated for your product? First of all, what is activation? So for us, activation is indicative of the team forming a habit around the usage of sneak. And when I say the usage, I actually mean deriving core value, which is ultimately fixing vulnerabilities.

Starting point is 01:19:57 It's not just logging in, right? It's not even just finding vulnerabilities. It's fixing vulnerabilities. So building a habit around that. And the reason I say team instead of using is, and we actually base most of our definitions of activation engagement around teams. It's really important because ultimately security is a team support. That team might be one person, which case a user is equivalent to team, but often a team is multiple people. And we actually expect different people to fulfill different parts of the team activation journey.

Starting point is 01:20:26 We also want to enumerate aggregate level activity around fix that sometimes happens off platforms. where we can't explicitly measure it at the user level. So in the activation process, we have set up moments, aha moments and habit moments. And our habit moment that we define as a team being activated, it's related to fixing vulnerabilities within 30 days of team creation. And the reason that is chosen, it's because there is a significant correlation with downstream

Starting point is 01:21:01 and in that case with activation, three-month retention and retention again based on, again, not just coming back and logging in, but still fixing. So teams that fix a vulnerability within their first 30 days are much, much more likely to still be fixing three months later. That's really interesting. Yeah, I love that. How did you come up with that number? Was there a decision scientist that looked at some kind of inflection of like at 30 days? And it's probably not exactly 30 in real life, but it's like a nice round number that's close enough, Yeah, that's it. So there absolutely was a decision scientist involved, thankfully. We had to

Starting point is 01:21:39 collect a lot of baseline data first. So after we built out the data platform, we needed actually to wait a bit to build a good data set. We did a huge amount of quantum analysis, a lot of spulunking of the data, applying ML models, along with a bunch of supporting call research as well. But we started really with identifying the corpus owners and their use cases, different roles of different users within the team-based activation journey. Defining our retention metric, which is still fixing, it's whether a team is fixed, along with natural usage behavior and expected natural usage frequency.

Starting point is 01:22:18 And then we found the habit moment that ultimately most strongly correlated with improved long-term retention. And most of kind of the numbers side of things came out of our MLOPS platform. But after that, we then worked back to figure out that's the habit moment. That's what we see is activation, but there's a set of steps that teams need to get there. So what's the aha moment? Before that, what's the setup moment? And what are the individual steps that the team needs to go through to reach that set up moment?

Starting point is 01:22:45 So yeah, that's the overall process. And ultimately, it's a really strong model that allows us then to feed in the set of user-level behaviors that we know can influence those different steps on that path to activation. Awesome. I'm excited to get this post out on that. That's a really good anecdote of how a company comes up with it and what they said. I also just thought of another question. I may start asking everybody.

Starting point is 01:23:10 I know I keep saying we're going to wrap this up. But here's a question. And if it doesn't work, we'll get rid of it. You mentioned a bunch of tools that you use. And I'm curious, if you have to think of what are the five most important slash valuable SaaS products to your organization, other than like the obvious ones like Salesforce, what comes to mind? When you say organization, do you mean sneak at large or do you mean the growth? Let's start with growth in Syracuse. Okay.

Starting point is 01:23:37 So I'm going to say amplitude, first of all, a segment as a means to be able to get our data from the products to amplitude and to everywhere else that cares about it, whether that be a kind of downstream BI system, Snowflake and Looker on top of that or system might. marketing automation systems like Marquetto and stuff like that. So I'd say amplitude. I will next say full story, which is, you know, absolutely fantastic for kind of session replays, of course, and kind of it bridges that gap between qual and quant, right? I would say user interviews.com, which is comparable to usertesting.com, both of them. amazing services in terms of getting fast, curated access to individuals to participate in user research. Sprig is another one. So Sprig is a fantastic kind of in-app survey platform, which is

Starting point is 01:24:48 what we primarily use it for, but it also does a bunch of other cool stuff in terms of being able to test UX designs as well in-app. How many have I said? I think four. If there's anything else, You could add a fifth. Okay. If not, we can move on. That's awesome. This is a really interesting list. Yeah, I'll stick it at four.

Starting point is 01:25:06 Okay. Is there anything for the wider product team that also comes to mind that you guys find really useful? A table. In fact, air table for growth and products. Just so flexible. It's just you can do anything. In fact, if we think about growth, I should have mentioned that first because that's where we keep most of our experiment plans and knowledge base and our user research base. Okay.

Starting point is 01:25:28 I like this question. I'm going to start asking it. This is great. Okay, that was a precursor to our very exciting light and round. I'm just going to ask you a bunch of rapid-fire questions. Just answer whatever comes to mind. We'll keep it short and quick. Question one, what are two or three books that you recommend most to other people?

Starting point is 01:25:44 I have been dreading this question, as there are too many. So I'm just going to pick a couple that I've enjoyed reading in recent months. So for product and growth geeks like me, or in fact, anyone with more than a passing interest in data, I'll recommend how to measure anything by Douglas Hubbard. Second up, you had Marty Kagan on, and he mentioned the book, Sprint by Jake Knapp and John Zaraxki, and I love that book, but personally, their make-time book, it was something that radically changed my relationship with information, and I recommend that to all time-star product people out there.

Starting point is 01:26:22 And for number three, I'm going to say, this is how they tell me the world ends by Nicole Pearl Roth, which gives an amazing view into the world of digital espionage. Oh, I read that book. Tim Ferriss recommended that at one point. That is a wild-ass book. Very beautiful. Cool. I love these recommendations.

Starting point is 01:26:40 All right. That's it. Cool. Great choices. Keep going on. Favorite podcast other than the podcasts you're currently on? Maybe acquired with Ben Gilbert and David Rosenthal. I just wish I had enough time to listen to them more.

Starting point is 01:26:52 Yeah, they're very long. It was just in an event where they interviewed someone live as like a live podcast. That was very cool. Those guys were pros. What's a favorite recent movie or TV show? So, movie turning red on Disney Plus, which we love watching with our kids. It was just fun. TV show, the most recent curb season, had me in tears, as usual.

Starting point is 01:27:13 They haven't had a new one in a while, right? So that's a little bit out there. Yeah, a little bit. Cool. I'm ready for the new one's coming. Oh, it is. Season 12, yeah, they're ready. I don't love watching that show.

Starting point is 01:27:24 My wife loves it. Cringe, painful. But anyway, watch it. Anyway, and my wife's actually the same. She can't watch it because she cringes too much. Okay, we're reverse. I love the awkwardness. It's good for a product leader to have that enjoyment. Favorite interview question that you like to ask, candidates? Give a couple here if it's not cheating too much. First is one I like to ask when hiring for anyone, actually, really.

Starting point is 01:27:49 It's fast forward three years, what's different about you then? A lot of people will default to telling you where they aspire to be in terms of, of role or title. But what I'm really looking for is signals of humility, of self-awareness around areas of personal and professional growth. So, you know, people who can be open about where they think they need to work on to grow themselves as people. I love that. Also, just generally, throughout interviews, I'm looking for curiosity. So day-to-day, good PMs will be asking why as much as my six-year-old son does, which is a lot. So I'll try and discern that through the course of the conversation. It's not really a question, but something I'm looking

Starting point is 01:28:35 for. And then maybe I want to flip it because building on something that Adam Fishman was saying, his theme of evaluating the people, dimension of folks you're potentially going to work with when you're interviewing with a company. And this was a question I got asked myself recently by a candidate, which I just thought was brilliant. And that was, tell me about the diversity, equity, inclusion, and belonging initiatives that you've recently personally been involved with. And that, it just felt like a really great way for them to be able to test alignment of their personal values with those of someone they'd be working with really closely. So I love that. Awesome. By the way, I love how many callbacks to other episodes you're making. You're definitely

Starting point is 01:29:14 a power adopter of the podcast, and I really appreciate that. There we go. Last question. Who else in the industry do you most respect as a thought leader, as a leader in general? So maybe I'll cheat on this one a bit too. And I'm not going to combine it to the, when you say industry, I think security industry, but I'm going to look at the product domain. Yeah. And specifically product operations. And in my mind, there's not many people who know more in this area than Christine Itwaru at Pendo. So if you ever get chance to talk with her, I know that would be a fun conversation. Many gems would be dropped, I think. Wow, I will get her on this podcast. That is my new goal. I had not heard of her and that's awesome. Thank you for the suggestion. Ben, this has been awesome. So many nuggets and stories and insights. I so appreciate you being here. Two last questions. Where can folks find you online if they want to reach out or learn more or maybe come work at sneak? And then how can listeners be useful to you? Firstly, I just want to say a big thanks for having me on, Lenny. I love talking about all this stuff and really appreciate you of being

Starting point is 01:30:19 willing to let me bend your ear a little bit. It's been my pleasure. As to finding me, I'm a bit of a twit when it comes to Twitter. I generally spend a bit more time over at LinkedIn, but you can find me on both of those platforms at Semantic Ben. And in terms of how people can be useful to me, I'm starting to take on some additional clients in advisory capacity, so feel free to get in touch if you think I could help.

Starting point is 01:30:46 I can't not ask about Semantic Ben and what the story is there. and before we let you go. Actually, when I was at IBM, it was a focus that I had on linked data. Yeah. Became semantic Ben. All right. There we go. It stuck.

Starting point is 01:31:03 Awesome. All right, Ben, thank you so much for being here. And thanks for listening. Thanks, Lenny. Take care. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcast, Spotify, or your favorite podcast app.

Starting point is 01:31:17 Also, please consider giving us a. rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lenniespodcast.com. See you in the next episode.

Lenny's Podcast: Product | Career | Growth - How Snyk built a product-led growth juggernaut | Ben Williams (VP of Product at Snyk)

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.