Lenny's Podcast: Product | Career | Growth - Everything you’ve ever wanted to know about SAFe and the product owner role | Melissa Perri (author, founder of Product Institute)

Episode Date: November 10, 2024

Melissa Perri is the founder of Product Institute, author of Escaping the Build Trap, and host of the Product Thinking Podcast. She has worked with startups, Fortune 50 companies, and everything in b...etween to help them build better products and level up their product teams. In our conversation, we discuss:• The history of the product owner role• The differences between product owners and product managers• How to transition from product owner to product manager• The evolution of and problems with the SAFe framework• How large non-tech companies can improve their product practices• Much more—Brought to you by:• Pendo—The only all-in-one product experience platform for any type of application• OneSchema—Import CSV data 10x faster• Coda—The all-in-one collaborative workspace—Find the transcript at: https://www.lennysnewsletter.com/p/product-owners-melissa-perri—Where to find Melissa Perri:• X: https://twitter.com/lissijean• LinkedIn: https://www.linkedin.com/in/melissajeanperri/• Website: https://melissaperri.com/• Product Institute: https://productinstitute.com/• Podcast: https://www.produxlabs.com/product-thinking—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Melissa’s background(02:12) The rise of the product owner role(06:37) Understanding Agile and Scrum(08:27) Challenges in Agile transformations(10:41) The history of the product owner role(13:58) The Scrum Guide(15:43) Product owner responsibilities(21:01) Adopting Scrum in organizations(26:21) The origins and implementation of SAFe(35:20) Why Melissa doesn’t recommend SAFe(40:33) Advice for implementing a digital transformation(49:12) An example of SAFe adoption(51:27) The value of experienced product leaders(56:53) Career paths for product owners(01:04:14) Transitioning from product owner to product manager(01:06:41) Be careful relying on certifications(01:11:43) Evaluating existing product owners(01:16:55) Final thoughts on Agile and product management—Referenced:• Escaping the Build Trap: How Effective Product Management Creates Real Value: https://www.amazon.com/Escaping-Build-Trap-Effective-Management/dp/149197379X• Lean UX: https://leanuxnyc.co/• Scrum: https://www.scrum.org/• What is Extreme Programming? https://www.agilealliance.org/glossary/xp/• Capital One: https://www.capitalone.com/• The Agile Manifesto: https://www.atlassian.com/agile/manifesto• Ken Schwaber on X: https://x.com/kschwaber• Jeff Sutherland on X: https://x.com/jeffsutherland• Kanban: https://www.atlassian.com/agile/kanban• What is a kanban board?: https://www.atlassian.com/agile/kanban/boards• Ron Jeffries’s website: https://www.ronjeffries.com/• Jeff Patton on X: https://x.com/jeffpatton• The Scrum Guide: https://www.scrum.org/resources/scrum-guide• OpenSky: https://www.openskycc.com/• SAFe: https://scaledagileframework.com/• Dean Leffingwell on LinkedIn: https://www.linkedin.com/in/deanleffingwell/• Capital One scraps 1,100 tech positions: https://www.reuters.com/technology/capital-one-scraps-1100-tech-positions-source-2023-01-19/• Product management theater | Marty Cagan (Silicon Valley Product Group): https://www.lennysnewsletter.com/p/product-management-theater-marty• Marty Cagan on LinkedIn: https://www.linkedin.com/in/cagan/• Jeff Gothelf on X: https://x.com/jboogie• Shruti Patel on LinkedIn: https://www.linkedin.com/in/shruti-patel-32bb573a/• Product Thinking Podcast: Mastering Product Focus: Balancing Legacy and Innovation with Shruti Patel: https://www.produxlabs.com/product-thinking-blog/2024/9/25/episode-190-mastering-product-focus-balancing-legacy-and-innovation-with-shruti-patel• Melissa Douros on LinkedIn: https://www.linkedin.com/in/melissadouros/• Mind the Product: https://www.mindtheproduct.com/• Athenahealth: https://www.athenahealth.com/• McKinsey: https://www.mckinsey.com/—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.—Lenny may be an investor in the companies discussed. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.lennysnewsletter.com/subscribe

Transcript
Discussion (0)
Starting point is 00:00:00 There's this whole concept of SAFE, basically scaled agile, right? So scaled agile framework came out of the desire to figure out how do we scale Scrum and different processes. I do not recommend using SAFE. Every single person I have talked to who, like SAFE, found success with SAFE, they ended up ripping it up and making it into something else. You've been up close and personal with a lot of companies working with product owners, scaled agile, and all these things. This product owner role did not emerge from product management as we know it today.
Starting point is 00:00:28 It was a way to help the do. developers prioritize what to work on. I ended up going to a ton of Agile conferences and speaking about product management. And I started to learn that there was this product owner role in Scrum. It feels like it's growing. Like more and more companies are adopting this as the way to work. A lot of large companies turn to Scrum or to the frameworks. And it's because they traditionally didn't grow up building software.
Starting point is 00:00:51 When you look at Agile methodologies, what we're really saying there is we want to be able to move quickly and deliver great value to customers. if you embrace those principles, you're going to do well. Today, my guest is Melissa Perry. Melissa is a legend in the product management community. She's the author of the foundational book, Escaping the Bill Trap, and her most recent book, Product Operations.
Starting point is 00:01:16 She's also the CEO and founder of the Product Institute, which trains product managers at all levels. She's trained PMs at almost every Fortune 500 company at this point. And in our conversation, we dive deep into a topic that I don't spend a lot of time on on this podcast. product owners, scrum, scaled agile, and building product at very large non-tech companies. Melissa shares the history behind these ways of working, what she's seen work and not work when companies roll out these frameworks, and most importantly, what you can do as a leader of one of these companies and as a product owner working in one of these companies to level up your organization and yourself.
Starting point is 00:01:53 I learned a ton from this conversation, and I'm really curious to hear what you think, since we don't cover this kind of stuff on this podcast too much. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. It's the best way to avoid missing future episodes, and it helps the podcast tremendously. With that, I bring you Melissa Perry. Melissa, thank you so much for being here, and welcome to the podcast. Thanks, Lenny. Thanks for having me. First, let me give a little context on this conversation that we're having. I think it's going to be a little bit unique.
Starting point is 00:02:26 So I was doing a deep dive on the job market in tech. and I saw something that was really surprising to me that the product owner role was the third fastest growing role in tech. And this was just in the US, the data I was looking at, but I think it's probably true broadly. And this was extremely surprising to me because I've never worked with a product owner.
Starting point is 00:02:47 I've never, I don't hear anyone in my circles talking about product owners. I've never wanted to hire a product owner. And it feels like it's just kind of this very different part of the tech ecosystem that you don't hear a lot about on podcasts like this. and it's clearly growing. So I felt like it'd be really helpful to spend some time
Starting point is 00:03:03 helping people and helping me understand this part of the world. And so I asked you to come on to talk about this. You've been up close and personal with a lot of companies working with this way of working product owners, scaled agile, and all these things. And so I couldn't think of a better person to have on here to help us understand what's happening here and also just to help people do this better.
Starting point is 00:03:23 So, Melissa, thank you again for coming on and helping us understand this. Yeah, I'm excited to talk about this. I have been really passionate about this topic for many years, and I've been talking about it in both agile circles and product management circles, so I'm pretty excited for the listeners to hear what else is going on out there. This episode is brought to you by Pendo, the only all-in-one product experience platform for any type of application. Tired of bouncing around multiple tools to uncover what's really happening inside your product.
Starting point is 00:03:55 With all the tools you need, in one simple-to-use platform, Pendo makes it easy to answer critical questions about how users are engaging with your product, and then turn those insights into action, all so you can get your users to do what you actually want them to do. First, Pendo is built around product analytics, seeing what your users are actually doing in your apps so that you can optimize their experience. Next, Pendo lets you deploy in-app guides that lead users through the actions that matter most. Then, Pendo integrates user feedback so that you can capture and analyze what people actually want. And the new thing in Pendo, session replays, a very cool way to visualize user sessions.
Starting point is 00:04:34 I am not surprised at all that over 10,000 companies use it today. Visit Pendo.io slash Lenny to create your free Pendo account today and start building better experiences across every corner of your product. P.S., you want to take your product-led know-how a step further? Check out Pendo's lineup of free certification courses led by talk product experts and designed to help you grow in advance in your career. Learn more and experience the power of the Pendo platform today at Pendo.com.com. I'm excited to chat with Christina Gilbert, the founder of One Schema, one of our longtime podcast sponsors.
Starting point is 00:05:12 Hi, Christina. Yes, thank you for having me on, Lenny. What is the latest with One Schema? I know you now work with some of my favorite companies like Ramp, Vanta, Scale, and Watershed. I heard that you just launched a new product to help product teams import CSVs from a especially tricky systems like ERPs? Yes, so we just launched one scheme of file feeds, which allows you to build an integration with any system in 15 minutes
Starting point is 00:05:37 as long as you can export a CSV to an SFTP folder. We see our customers all the time getting stuck with hacks and workarounds, and the product teams that we work with don't have to turn down prospects because their systems are too hard to integrate with. We allow our customers to offer thousands of integrations without involving their engineering team at all. I can tell you that if my team had to build integrations like this, how nice would it be to be able to take the software?
Starting point is 00:05:58 off my roadmap and instead use something like one schema, and not just to build it, but also to maintain it forever. Absolutely, Lenny. We've heard so many forer stories of multi-day outages from even just a handful of bad records. We have laser focused on integration reliability to help teams end all of those distractions that come up with integrations. We have a built-in validation layer that stops any bad data from entering your system, and
Starting point is 00:06:20 one schema will notify your team immediately of any data that looks incorrect. I know that importing incorrect data can cause all kinds of pain for your customers. and quickly lose their trust. Christina, thank you for joining us. And if you want to learn more, head on over to one schema.co. That's one schema.com. So before we get into the history,
Starting point is 00:06:40 is there just anything broadly that you think might be helpful to share before we dive into like the history of the product owner role and all these things? Maybe I'll help to set some context of, you know, where did I see all of this start emerging as well? When I started in tech,
Starting point is 00:06:54 I was very much a product manager, never heard of the product owner role before in my life. And in escaping the build trap, I talk a lot about how we use scrum when I started working with this team in a startup. and it was the first time I ever heard of it. That was like 2011. And at the time, the person who was teaching me about Agile was very like, hey, this is, this is flexible. We're just going to break things into sprints.
Starting point is 00:07:17 We're going to sit there and actually talk about the work, right? This is made for us to actually get better at our jobs. So we were pretty sold on it, and it was never dogmatic. And as I worked at other companies, I found that they were being a little more dogmatic with their scrum, with their stand-ups, how we actually run things. And then I started speaking at conferences. And one of the first conferences I spoke at in New York City was called LeanUX. And there was a bunch of people from the Agile world there too.
Starting point is 00:07:45 So I learned that this was much bigger than what we were learning in my company and what we were doing in these companies. There's this whole group of people out there practicing Agile. And I was like, oh, this is cool. I want to learn how to do things better. So like teach me about your philosophies. And I ended up going to a ton of Agile conferences and speaking about product management. And at the time, people were really excited to hear more about it. And I started to learn that there was this product owner role in Scrum
Starting point is 00:08:10 where I was talking more about how we traditionally talk about product management, like understand your customer, go out test things, make sure there's a hypothesis, don't just blindly build what you want to build. And I found out that that was not the case in a lot of these companies who were adopting scrum and introducing a product donor role. So I started doing a lot of trainings through my school Product Institute, and I'd get called into these large companies, like all these large banks, probably around 2014, 2015, to help them learn product management.
Starting point is 00:08:40 And I was really excited about this because before they didn't want anything to do with it. They were like, I don't know what product management is. I don't need this. So I go in to train people, and I found that a lot of them had been going through an agile transformation, and they had all of these new product donors where they came in and they basically said, hey, like, you're a product donor now. your whole role has changed, and they came from all different backgrounds. Some were developers,
Starting point is 00:09:04 so a lot were business people who worked on the banking side. A lot were business analysts. Some were project managers. But they just collectively took a bunch of people and said, to death, today you're going to be a product owner because we're going to do Agile now. And I would come in and help train these people. And what I found was that there was a really big misconception about what those people should be doing compared to what they teach in Agile and scrum versus what we all consider great product management. So I've been trying to fill that gap for the last, you know, almost 10 years working with these companies here. And then I would go to their leaders. And at the beginning of these agile transformations, I'd be like, you know, you can't just do scrum.
Starting point is 00:09:44 That's not going to make you amazing at delivering products. There's so much more to this. And the leaders didn't quite understand that. And I'm noticing this really big shift in the industry where we're finally getting there. A lot of companies are doing it well now. Capital One is a great example that took their agile transformation and started adding product management on it, and they've really turned that around in the card business. But so many organizations are still at the beginning of this journey, and they're at the place where I saw people 10 years ago. So I think there's still a lot of companies out there. Maybe we take it for granted in tech or in Silicon Valley about how many companies are doing this and how big this scope is, where they're making these roles, but they're not really
Starting point is 00:10:26 doing product management end to end. So that's kind of where I've seen all of these areas, and I've been trying to help organizations for the last 10 years really set up robust product management practices. So it's not just one piece of development. It's how do we actually build better products. I love this example of Capital One is just like it can work really well. Yeah, and you can get to a place where it's actually really productive.
Starting point is 00:10:48 So there's a few ways we can go about this. Do you think it would be helpful to talk through the history of the product owner role, just like where it initially emerged? Yeah. I think that's a great place to start. I think it brings a lot of context to what's happening. And people forget about the history here. So when I explain it to people, I say, you know, we had product managers in Silicon Valley, right? They were in Google. They were in all of these companies, Amazon. And they were born out of this business role.
Starting point is 00:11:14 And from a software-native company, your software is the business, right? It's what you sell. It's what you actually look at. So our product managers in Silicon Valley are doing, you know, they're doing market research. You're talking to customers. They're working with developers. they're iterating, they're doing the end-to-end product management. But what happened on the other side of things, especially in large companies, is the emergence of product management from scrum, from product ownership. And that's usually the first time these companies were introduced to product management was from implementing a product owner role and then going, hey, we're still not meeting our goals.
Starting point is 00:11:49 Like, we're still not, are we building the right thing? And then they started thinking about product management. And where that role came from is scrum. So if we go back and talk about the history of Agile, Agile was a movement that got started by software developers. So in 2001, the Agile Manifesto was written. A bunch of developers got together in Park City, Utah. They were all skiing together, and they said,
Starting point is 00:12:14 hey, we've been independently all working on how to develop software better. Some people were practicing Scrum, as we know it today. That was Ken Schwaver, Jeff Sutherland. There were people who were doing different types of Agile frameworks as well, con bond where you were, you know, moving it through a con bond board. There was a behavioral-driven development. There was feature-driven development. That was the style of agile. There was XP, extreme programming that was started by Ron Jeffries. All of these people kind of found each other saying, hey, we've been trying to push the boundaries of how do we develop better software,
Starting point is 00:12:46 and they got together and they wrote the Agile Manifesto, as we know it today. So the Agile Manifesto is really just a guideline on how they're striving to not just be people who code what people want, but like building better products. How do we build better products through software development? The premise to remember this is, and I keep saying it, but they're all software developers. Nobody was a product manager who went to that meeting, right? Nobody who wrote the Agile Manifesto was actually a product manager. And I've spent a long time talking to these people as well over the past 10 years just saying, hey, how did this come about? Where did this come from? The one person who was really close to them, who was a product manager, was Jeff Patton. But he never signed the
Starting point is 00:13:28 Agile Manifesto. He wasn't at that meeting. So he talked to them a lot. He was able to see what was going on, but all of this was approached from a how do we build better products from a development perspective. So that's really important to know. To the people who signed the Agile Manifesto, Jeff and Ken, as I was talking about, they were independently kind of coming up with Scrum on their own and their different companies. And they got together and started to codify it. And they said, this is how I'm doing it, this is how I'm doing it, and they ended up writing the scrum guide. And the scrum guide is what a lot of people base their agile practices off of today. And in the scrum guide, it outlines a bunch of roles that you would do on the development team. And then it says how you should
Starting point is 00:14:09 be developing products. So most people out there are working in scrum today. And what they say is let's break things down into two-week sprints. You can change the length of your sprint if you want to, but two-week sprints is pretty standard. At the beginning, we define what we're going to work in the backlog. It's the product owner's responsibility to define what goes into the backlog, write down the user stories for it, do all that. Then the development team comes in, they discuss it, the product owner prioritizes it, they ask questions, and then the development team commits to what they want to build, and they go out and do it. And at the end, the result is a potentially shippable product, not necessarily shippable, but potentially shippable. So they're trying to
Starting point is 00:14:48 break it down as small chunks and build things instead of what they had been doing in a lot of companies, which was building stuff for like three years and then releasing it a big bang. And what all of the people who signed the Agile Manifesto realized was if we do it the other way, if we do a waterfall type environment, so Agile Waterfall, that's where we go across. There's a lot of risk because we don't test it with the customer and we don't get feedback on it if we spend three years building it and never show it to somebody. So it really approached a different way of building software. So it said, let's chunk it down and try to get feedback faster.
Starting point is 00:15:21 really noble intention. In the Scrum Guide, though, it introduces these new roles. So we have developers, as we know and love them. We also have a product owner. Then we have a Scrum Master. And the Scrum Master is in charge in Scrum of actually helping people do scrum better. That's literally their job. How do I do Scrum better?
Starting point is 00:15:42 How do I make sure that the team is working well together? So they host things like retrospectives where at the end of a sprint, we say what went well, what didn't go well, how should we actually inspect and adapt our process? The product owner is where things get murky. So the product donor in general first showed up with scrum. And if you go and you read the first scrum guide, which I pulled up and started reading,
Starting point is 00:16:04 because I've been very fascinated about how this is described, it says that the product owner is responsible for maximizing the value of work done. The team does the work. Interesting, because now the product owner is not quite part of the team. The team consists of developers, with all the skills to turn the product owner's request into the potentially shipable increment each sprint. The team is usually seven plus four minus two members. And then when you go further
Starting point is 00:16:30 into the first version of the scrum guide, it does say that the scrum master works with the customers and management to identify and instantiate a product owner. The scrum master teaches the product owner how to do his or her job in order to optimize the value of the use of scrum. If they don't, the scrum master is held accountable. Then it's got another tip if we go deeper into this. It says for commercial development, the product owner may be the product manager. For in-house development efforts, the product owner could be the user department manager. What's interesting is that that was the first version of the scrum guide, and again, to arguments about the scrum guide with people all the time. Twenty-13's version, though, the more updated one
Starting point is 00:17:11 that you could go and find, is the one that almost every company has run an agile transformation off of. It loses that thing that says the product manager could be the product owner. doesn't say it anywhere in that guy. So this was the first version, and you can kind of tell it was like an aside. It's like, oh, by the way, the product owner in Scrum doesn't need to be a product manager. It could be the customer. It could be a developer. It could be, it's usually the customer. But when they were writing this too, sometimes the customer was an internal person, like at a bank or somewhere where we were building software who was asking for the software. They were like, go build me an internal tool, go do this.
Starting point is 00:17:46 So now we're just asking for requirements inside a company, and that's where you can start to see how the product owner role kind of evolved into somebody going to ask, like, hey, what do you want me to build? What's required here? And then just listening to somebody come back and say, I need this feature, I need this feature, I need this feature. Scrub doesn't describe how to get the stuff into the backlog.
Starting point is 00:18:10 And it didn't in the 2013 manuals. The manuals have all been a little bit better, They've all kind of been updated since then. And they do describe like it has to start with the vision a little bit more. You have to break down the vision for the product and get in there. But none of that existed in the early versions of scrum. So when people got trained on how to be a product owner, what was happening here is, and this is the whole other world of scrum over here.
Starting point is 00:18:37 When people get trained to be a product owner, it's usually a two-day class where they teach them, hey, this is how you break down a backlog. This is how you do stand-ups with your teams. this is how you think about prioritizing work. This is how you manage your backlog, prioritize it for the developers. This is how you work with the retrospectives. But it doesn't teach them about experimentation.
Starting point is 00:18:58 It doesn't teach them about market research. It doesn't teach them about data. It doesn't teach them about any of the things that we need to be a product manager. So then what happened was we went into these agile transformations at these companies. And they said, hey, let's adopt scrum. because scrum was built as a way to build better products faster.
Starting point is 00:19:20 It's literally the tagline. And everybody was like, yeah, I want to build products faster. Okay, great. Like, let's do scrum. And all these large organizations back in the early 2010s in the 2000s said, oh, we got to be better at software. How do we do this better? Because otherwise we're going to lose when it comes to innovation.
Starting point is 00:19:37 So they adopted scrum as a way to build software faster. Now what happened is in order to do scrum, Scrum basically sells training. That's what Scrum does. So all of these Agile coaches would come in and teach your product owners, newly minted product owners, took all those people, made them into product owners, put them through a two-day class, and then say, go. And that was the beginning of all the Agile transformations.
Starting point is 00:20:02 And that's where a lot of companies still are today. So this product owner role did not emerge from product management as we know it today. It was a way to help the developers prioritize what to work on. but that was it. And the product owner was held accountable for making sure that they were working on the most pressing things, or the highest value things.
Starting point is 00:20:24 They do say that. But to me, if you look at it from a developer perspective, it's also the person where you can say, hey, well, you told me to build that. Right? We didn't build it wrong. You told me to build that. And it almost gets into like consulting territory. We're like, okay, if the product owner
Starting point is 00:20:39 prioritized all this stuff for me and told me what to do, I can't be held accountable if it was a wrong thing to build. And some of that stuff does come up in a lot of teams that struggle to adopt Agile to adopt Scrum. So I feel like there's a big misunderstanding out there
Starting point is 00:20:55 about what is this role and what should we be doing? But the premise of this is when we talk about Scrum is just one piece of the puzzle. And when people talk about Agile now, they almost always associated with scrum. I was actually Googling Agile Methodologies.
Starting point is 00:21:12 And like I said, The other one's Combonds and Agile methodology. XP's an Agile Methodology. They don't have product owners. They do not exist in those methodologies. They're for developers to work on things, for teams to work on things. And XP would consider product managers in the teams,
Starting point is 00:21:29 as far as I know it. But Scrum kind of sees it as a separate thing. So Agile Methodologies, everybody says, oh, they're scrum now. So it gets a bad connotation out there about what to do with it. I think scrum, if you do it well, is bad. But you have to understand that it's just one piece of building great products. It's not the whole thing.
Starting point is 00:21:50 And companies will adopt it like it's going to radically transform everything. And to be fair, a lot of times it's sold that way too. There's kind of like a bigger picture question that's coming to mind as you talk about this. And I'm imagining founders listening to this and smaller companies listening to this. Why do we need any of this? We're just like, we're just going to like especially, you know, Silicon Valley startup. We're just going to build stuff. We know we don't need these like frameworks.
Starting point is 00:22:15 We don't need a scrum master. Like we just have awesome developers and product managers. And we're just going to build awesome stuff. And I don't know anyone that has worked this way that has built amazing things. Can you talk a bit about like who ends up looking for solutions here? Like where this even comes from, what companies need help here versus like I just don't need any of this. A lot of large companies turn to scrum or to the frameworks. And it's because they traditionally.
Starting point is 00:22:41 didn't grow up building software. So they're looking at how do I implement something that has rigor at scale? And that's where you see a lot of scrum come up. Now, I've seen startups using scrum. Some of them do it fine, right? They understand that it's just more about trying to get things out the door
Starting point is 00:22:57 every two weeks to test it with customers. I think if you keep that philosophy, like I said, I used it. And we didn't have a lot of rigor around it. That was fine. When we were doing scrum, when I did it with my team back in Open Sky, we got to this point where we were like,
Starting point is 00:23:10 two weeks is too long. just going to ship things every week. And we just talk to each other. We skipped stand-ups, which is like sacrilege and scrum, but we skipped daily stand-ups. We didn't need to stand around and talk about it. We talked to each other every day. For me, what was amazing and where I see teams actually thrive when they start using scrum is when you go and talk to people, right? You're having the conversations about the work. You're breaking it down. You're understanding it. So the developers and the rest of the team can go hit the road running and people can ask important questions. If you're not doing that, that's where I think things like a framework help.
Starting point is 00:23:44 But if you already are doing that, you don't need a framework, right? Like you don't need scrum. You don't need to be prescribed to this two-week sprint or anything like that. As long as you have a methodology or it doesn't even have to be a defined methodology, as long as you have a way of working that gets things out to customer as well, who cares, right? Who actually cares? And where there's a lot of, I think, baggage in the industry and where I think where I hear product managers get really frustrated in other people as well, developers too, is that when you do scrum
Starting point is 00:24:14 by the book or how people teach it and how they write about it, there's a million meetings. And I know they were put in there so that people were forced to talk. But when you already know what you're supposed to work on, why do you need to keep doing meetings, right? Like, shouldn't you just go do some work? So a lot of developers complain, a lot of product managers complain that scrum has too many meetings and they don't actually get to do work. And that's where I think you have to go back to the inspect and adapt part, and a lot of people who are very religious about Scrum will come and yell at me about this. They're like, oh, well, that's not how it's supposed to be. You're supposed to inspect and adapt. Agree, right? But a lot of people aren't doing it. And that's the piece where you go
Starting point is 00:24:49 and you say, is this serving us? And if not, let's get rid of it. Right? Let's not do those types of things. So when you're a small startup, I don't think you need a lot of this overhead. It's really much designed for larger scale companies. And those are the ones that you see really adopting it. And from what I've seen on the data, it's also companies that are, as you, I think, alluded to at the beginning, not like necessarily software first, product first companies. Like, it feels like it's very common in like banks and telecom companies and companies that aren't like product first and software first. There are staff companies that do scrum out there and they like it. And I don't think they're very dogmatic about it. Right.
Starting point is 00:25:27 But they do it for a reason of just trying to provide some more context to their teams about how to work together at scale. I've also seen places where, you know, they don't prescribe whatever methodology you want to work with for the teams. But instead, they'll spend a lot more effort breaking down, you know, the roadmaps, thinking about what are we going to do each quarter trying to like set those themes. And then they just let the teams run. And that works as well. I think it really depends how they adopt it. But I would say it's not a hard and fast rule that no startups are doing that. Some are doing it. I just don't know how it's going for them. it might, to me, it might be overkill if you're doing that with like a team that's pretty experienced in doing this. Yeah. And what I meant to what I was insinuating is less just like scrum as a general idea and more like very structured rigid. Oh yeah. Processes and also product owners. And then there's this whole concept of safe that we can talk about. So should we
Starting point is 00:26:26 get into that or should we talk about product manager versus product owner and just like the challenges people that have there? Let's let's talk about safe because that's, where a lot of it started to get confusing. So scaled agile framework came out of the desire to figure out how do we scale scrum and different processes and bring it to organizations at scale. So it originated from a more structured approach to agile two called rational unified processing. Now, Safe wasn't the first thing that started at scale. There was also less, which is a scaled agile framework.
Starting point is 00:27:03 and then Jeff Sutherland, who did scrum, has scrum at scale. So it's not the only scaling framework out there. There's actually a lot out there. But SAFE was one that was marketed the best. And the way it's marketed is we'll tell you everything you need to do to do all of your agile teams with scrum and put them all together. So the idea behind SAFE was that Dean Leffingwell came up with it. He wanted to really show how you tie multiple teams together at scale in an organization.
Starting point is 00:27:31 and how do you bring some rigor and process to that? And the executives at really large enterprises, we're talking tens of thousands of people, right? They love safe because it prescribes a lot of an operating model of what to do when it comes to development. But it also gets billed as like, hey, this is the whole model for you to go do software. And if you look at it, it's a big map
Starting point is 00:27:58 that everybody kind of makes fun of a little bit. and it describes like all all different things on it if you look at the map and you can click in and you can see the definitions and you can see what's going on in the areas the safe image has gotten bigger and bigger over time that I think that what is this version six and I do know a lot of people who worked on safe so I know a lot of trainers and I've worked with companies the first time I was introduced to safe it was when I was working with a bank back in 2015 I came in to train their product managers. I'm doing my training. We're like sending them up on how to like go talk to customers, talk about hypotheses, MVP. And somebody came up to me and they said, hey, Melissa,
Starting point is 00:28:37 all of this is great, but I don't have time to go talk to customers because I'm a product owner. I was like, well, what are you doing on a day to day basis? Like what? Why don't you have time for? I got to write my user stories. Like, okay, how many user stories do you write per day? And this is for the developers to have a full backlog so they could all work, right? she's like, oh, yeah, I spend like pretty much 40 hours a day writing user stories. I'm like, on what? We're like, what are you controlling? And she's like, the login API for our bank.
Starting point is 00:29:07 I'm like, can you log in? She's like, yeah. And I'm like, so what else are you working on? Like, is there a new initiative? Is there a new thing? And it's like, no, I was reorganized into a team where I became the product owner. I have a product manager who goes and talks to customers, but then she comes and she tells me what to build.
Starting point is 00:29:24 And then I write the user stories around it and I put it in. the backlog for the teams. And I was like, what is this? Like, and then they said, that's safe. This is what we're working towards. So that's what my first experience was safe. And then I ran into another company that did it, another company, same thing over and over and over again,
Starting point is 00:29:40 where all these product owners were just basically trying to keep these backlog fulls for developers. And they were working on such a narrow level. And when a lot of organizations, too, I saw it reorganized into agile teams. They did it by component. So everybody was over every tiny, little feature and these teams were massive, right? Like super huge scope. And some of the stuff was just not prioritized, right? It was done. Like you didn't need to work on it. So they were finding work to do
Starting point is 00:30:07 so people wouldn't get fired, right? Like that's how the product owners operated. So there was all this legacy baggage sometimes in companies where they were all re-put on things by by component. And they're just making up work to do. So SAFE introduced this kind of split between product manager and product donor. And if you look at the map, the product owner is part of the agile team where they sit with the scrum masters, which is a team coach that they call here and the developers. And then the product manager is sitting with a system architect and what they call a release train engineer. And what SAFE does is they pull a bunch of agile teams into a release train. And you get on the train when you're ready to ship things and you make sure that it all goes pretty smoothly to get to that
Starting point is 00:30:53 potential shipable increment or that big feature launch that you would be doing with SAFE. And SAFE's really good at prescribing how to do that. They're great at describing how to do the release trains, how to bring those teams together, how to put them on it. And then they do this thing called big room planning where they get the entire release train together, all these teams, they put them in a room, and then every quarter you're kind of breaking down what we're all going to work on. But where I hear frustration from teams every time I come in and train them is that when
Starting point is 00:31:20 you do big room planning, a lot of times it's a commitment. So you start at the quarter. They haven't been doing good discovery because remember, these people have not been trained on good discovery. So they don't really know what they should be working on. They haven't been out talking to customers a lot of times. They kind of scramble. They figure out what needs to happen. Usually they have a backlog of stuff that does need to happen because it just has to get done.
Starting point is 00:31:42 They map it all out in a big room together. They commit to it. And then that's the quarter. And they asked me, when am I supposed to do discovery? I'm like, well, before that, ideally, right? Like you should have a vision, you should be breaking it down. You should be putting discovery into that vision, talking to customers, feeding that in there. And then I hear, we don't have time to do that because we sprint back to back.
Starting point is 00:32:03 And I was like, what does that mean? And they're like, as a team, we go and we basically do two-week sprint into two-week sprint, into two-week sprint, and I got to make sure my developers are full. I got to make sure they have things to work on. So if I go take time off to go talk to customers, which also is not my job as a product owner, it's my product manager's job. They'll feed me in what the customers are saying. Then I break those down into features
Starting point is 00:32:26 and I can work with the developers on it. So that's how all of this stuff starts going. And what happens in organizations that they don't understand here is that it's not the most efficient way to work, right? You should be, I see a lot of developers out there become almost like, how do you describe it, reliant on the product manager or product owner to tell them what to do.
Starting point is 00:32:52 And even though you build them this great vision and you explain what needs to happen, they go, oh, I can't work on it because the product owner hasn't prioritized it. And then they asked me, if I don't have enough for the developers to do on feature work, what are they supposed to do? I said, I guarantee you there's a ton of tech debt that could be working on. Like, you don't have to scope that out. Let them choose what's the most important thing. They should be working together as developers and architects to figure out how to tackle some of that
Starting point is 00:33:17 tech debt, how to get into it, right? While you're figuring out, is this the right thing to be building? So with all of this stuff, it feels like people feel like they don't have time because they're in a million meetings. And the expectations of these companies is that every sprint, we're like delivering software towards these roadmaps that we promised in the last quarter. And we're not checking to see if they're right. We're not checking to see if they're actually helping us move it forward. And a lot of times the organizations are not set up with the right feedback mechanisms, the right user research, and the right data to tell us if everything is working so they can feed that in to the next release planning. So they're just planning, planning,
Starting point is 00:33:55 planning, breaking it down into sprints and going. And SAFE is not good at describing how you do all that other work. So in a lot of this stuff, too, they started putting on, there's like pieces that they put onto this map of SAFE where they're like, hey, you should do OKRs. And it's like, this is what OKRs are. You should do a roadmap. This is what a roadmap is. But like, how all of that cycle kind of works together where you're balancing discovery and delivery and feeding it in, it's really confusing in organizations. And then what it's basically saying is a lot of the discovery work goes into the product managers. And the product managers, the product owners report into the product managers. And what I've seen that doesn't work here is that you are basically making these product
Starting point is 00:34:38 owners order takers. They are extremely tactical. And then when it's time to, actually be more strategic, let's say you want to be promoted to a product manager. Well, some organizations, that's not even the same business line. That's not even the same career path. It's product owners go over here and product managers go here and they report into different people. And if you ever want to move from product owner to product manager, a lot of times you don't get experience with the strategy, like figuring out what customers want, breaking it down,
Starting point is 00:35:05 looking at the market research, determining, like, is this valuable? Is this what we should be working on? And they're not even getting exposure or a chance to do that because safe. is like, no, that's the product manager's job. Your job is to go really deep and work with the developers. Wow. Okay, so a lot of this sounds quite absurd
Starting point is 00:35:24 as someone hearing all the details and looking at this image. That being said, many companies are adopting this. It feels like it's growing. Like more and more companies are adopting this as the way to work. I imagine the incentive is, we just want to build great software
Starting point is 00:35:43 and we don't know exactly how, and there's this process we can plug in, and it'll help us do it. I guess thoughts on that, and do you find it can work or often works or often doesn't work? What is your experience with people adopting this and how it goes? I know a ton of companies that adopted SAFE about eight years ago and have gotten rid of it. Capital One just came out and said they got rid of all their agile roles, all their SCROM roles. They were an early adopter of SAFE. They don't do it anymore. And they wrote about that in the newspaper.
Starting point is 00:36:16 I've seen it happen more often. Now, in a lot of organizations, too, I'll see parts of them do safe and other parts not do safe. So it could change business line to business line. I don't think, though, that people grasp how much it's still out there. I get questions unsafe every single day on the podcast.
Starting point is 00:36:36 Like, everybody asked me, like, why are we still doing this? And it's for what you said. executives buy Safe because it is the only framework out there that basically draws them a map and says plug and play do this. And that's why everybody's so excited about it because it's the only thing that specifies things to this level. And they went, oh, it's something I can understand. It's something that actually has definitions around it. And to be fair, that was like a great thing for Safe to do as a marketing tool. You know, like Bravo, they created this thing that everybody wants. Like, it's a good product to sell. But It's overkill. And that's what I keep hearing from organizations is it's basically taking the responsibility away from leaders to go figure their stuff out themselves as well. And if you're a new leader and you've just been dropped into this role, I have tremendous empathy for them because, yeah, where do you get started? Right? Like, how do you try to run a technology organization? Somebody came and told me, the CIA came and told me I'm in charge now, right? I'm in charge of all of the
Starting point is 00:37:38 developers or I'm in charge of all the product managers now, where do you start? Right? I can totally tell why people adopt SAFE because you're like, oh, I've been looking for the handbook, right? Like, I've been looking for something to do here. But the problem is it's only solving a little bit of the puzzle, which is bringing those teams together. And people do say it does release trains well. But it doesn't tell you also how to do your job as a leader. It leaves it all out. So they talk about portfolio visions and portfolio management and safe there too. But more often, not, I come in and I find everybody above product owners and product managers. Let's talk about directors of product, VP is a product, right? They don't know what they should be doing as a VP of
Starting point is 00:38:17 product or a director of product. It's like, what's my role? What should I be feeding in here? And SAFE doesn't even have that in there. That's not even a role. Like product manager going up into those levels is not really there. So what do you do when you own a whole product line in an organization? What do you do when you're the head of product for a credit card at a bank, right? Like, what's my job? Doesn't say that. So there's a lot of people out there in these organizations that I've been working with who are, I'm like, you're supposed to be doing strategy, right? And this is how you do strategy. This is how you go out and talk to customers. This is the patterns that we have in software. Are you doing a platform strategy? Do you need APIs? How do you think about your app
Starting point is 00:39:00 strategy rolling it out? How do you do this here? All of that stuff doesn't quite come from ways of working, which is what Safe is doing, right? It's about how do you do your jobs in those areas. And a lot of organizations who adopt SAFE don't realize that you need a head of product, right? Like you need somebody to actually be feeding that vision all the way down and making sure it's breaking up around the teams and controlling that portfolio vision and doing all of these things into it. So I have not seen SAFE slowing down by any means out there for people adopting it. I see more and more organizations adopting it. And I think we take it. take for granted too in Silicon Valley, like how many people are just starting on their journey
Starting point is 00:39:43 for digital transformations. There's a lot of like pharmaceutical companies, banks, insurance companies. They outsource their development or they had an IT team, but they never had to really think about it before because digital wasn't as important. Now they do. And some of these companies, most of these companies are Fortune 50 companies, right? Fortune 100 companies. And I think a lot of like the ones I see at least banks, realized early on, hey, you know, when it comes to apps and how people interact with our stuff, software is important. But there's a lot of companies that did not catch that train. And they're just starting. And then they turn to things like SAFE because it gives them a guideline. Hey, I've never done this before. I've been in this bank for 40 years. All I know is like waterfall
Starting point is 00:40:28 type development. What do we do? And then we'll go. We'll go look at SAFE. I love that we spend time on that because I think it's really important. Like you can be cynical about all this and be like, what the hell are people thinking? This is crazy. But as you describe, people just have like a problem to solve. They've never done this before. They look for solutions. They find something that seems right. They see other people doing this and like, okay, let's try this thing. And what you've seen is it rarely actually works out, the safe, specific approach. So let's talk about people that are, there's like a few ways I think we can help folks. One is someone trying to do, say, an agile transformation or digital transformation,
Starting point is 00:41:06 like how your advice for how to actually do that better. And then I want to talk about like, say you're a product owner or a PM within a organization that works like this, what can you do? So maybe let's start with the first. Say someone's like trying to figure out what we need to build better products. Something's not working rate. Safe as an option. Your suggestion is don't maybe do that.
Starting point is 00:41:25 What should people do? I know this is like a, they're not going to have the answer in like a short answer, but generally how should people approach this? Yeah. When I've worked with companies on digital transformation, you have to, you want a development operating model, right? And that's where a lot of these agile methodologies came out of. You have to understand that's just a development operating model.
Starting point is 00:41:45 That's not actually going to help you with go to market, with launching your products and with product management. So what I advise for companies do is first sit down and say, hey, how do we think about building our operating model? And when I think of, you know, product operating models and what I do with companies is like we break out, how do you determine product strategy? Do you have a good product strategy, right? You look at your organizational design.
Starting point is 00:42:10 How are we actually organized around our products? Do we have good coverage of product managers and do we have skill of product managers up and down the organization? Then we want to look at product operations. Do we have the infrastructure in there to help support these teams? Like, can they get the data to make decisions? Can they actually be in touch with customers? A lot of these large organizations haven't actually thought through many of those
Starting point is 00:42:34 steps as well that enable product managers and development teams to be successful, right? They don't have ways for them to go and talk to customers. So that's why they're not doing it. So I have a lot of empathy for people in these organizations as well who can't do product management well because of the bureaucracy or the things around it. So leaders need to solve that, right? They need to understand what the role is and they need to open it up. And then we got to look at our culture and incentives. like are we just rewarding people for shipping as many things as possible, right? Which is like, hey, just put everything you possibly can into that release train or that backlog. Or are we coming back and saying, hey, is this valuable?
Starting point is 00:43:13 Is this tying it back to our business? Many organizations do not have a great product strategy, many large organizations that I've worked with. And it's that tying it back to the value piece, right? Tying it back to, is this going to reach our contract? company goals. If you're a huge organization, and let's say, you're making billions of dollars a year, and your goal is like expand geographically, what are you doing in your portfolio to actually enable that, right? Like, what products are you building to expand geographically? So many organizations don't have the transparency to actually even see that. One, like crazy thing,
Starting point is 00:43:51 a lot of people give large organizations a lot of flack for, and I know Marty does this too, for focusing on processes. I don't. think processes are the enemy here. So for example, if I hear somebody really worried about getting a road mapping tool in there or something like that, I'm like, yes, you need that. Does you have no idea what your 4,000 teams are doing? And if they're actually coming back to the business goals, right, you have no infrastructure in there to be able to see that transparency. Those types of blocking and tackling is absolutely necessary for a transformation, right, for a organization to be stood up around software product management. You have to have the transparency to actually see those things.
Starting point is 00:44:28 you do need to have enough process so that you as an organization can be efficient in getting things out the door. And that's what I think Safe was trying to do, but it's not working because it's not solving the problems of the product management. And it's not solving that problem of connecting the value back to the product teams. Instead, it's seen as a role that almost like babysits developers or kind of tells everybody what to do. But where's the discovery? Where's those things come in. And I know with the safe image that we got over here, like, they try to drop things like Lean UX in there, which Jeff Gauthel think is hilarious, but it's not really pulling it all together of how do we do this on a cadence, right? How do we help people go out there and actually talk to
Starting point is 00:45:12 customers? How do we enable them to do it? So if you're starting a transformation, it's not just thinking about how do we build the product, but you should also be thinking about how do we launch the product and how do we make sure this is the right product to do? Right. That's the big pieces of it, and that's where all that product strategy comes in. And you should also look at the career paths. And this is what really bothers me about agile transformations and what bothers me with Scrum and safe is that when we organized in these large organizations into agile teams, we made all these new roles called a product owner, and so many organizations don't have a career path for them. So they email me and they go, what's my career path? And what do I do next?
Starting point is 00:45:57 right, where am I supposed to go? And I've been saying for like 10 years, like this is not a team role. It's not just a team role, right? Like, it's a business role and it rolls all the way up to helping you further your business. And you have to make sure that people on teams can be promoted to running multiple teams, can be promoted to running an entire product line. And to us, that's so simple in Silicon Valley native software companies, but it's still unheard of in other organizations. And what happens too, and this is where I think leadership and C-suite needs to really pay attention, because we're transforming in this way of working, what happens is some of the roles that we had before do not serve us now. Maybe we don't need a million project managers.
Starting point is 00:46:43 Maybe people in the business who decided what we're going to build, are they the right people to bring with us on this next phase, right, into product management? Can they learn, right? Can they grok software? or do they understand those pieces? That's what we have to ask. But organizations are so afraid sometimes to put these career ladders in because it kind of overhauls their traditional ways of working. And then they've got people who've been in these organizations for 40 years.
Starting point is 00:47:09 And now you're saying like, hey, you're actually not in charge of that. The product manager is in charge of that. And that's scary. And a lot of them get in the way because of that. And if you really want to transform, though, the C-suite has to be like, hey, we're going in this direction and just put it down. because I've seen it run by a lot of middle managers, like a transformation run by tons of middle managers. And those are the jobs that are usually in most jeopardy when you start transforming and you have to
Starting point is 00:47:34 reskill and you have to figure out what to do. And they don't want to do that, right? Like they're not going to be the ones who jump up and down and say, hey, let's do this. There's a lot of people out there, I think, pushing organizations to try harder and to internally as well. Like I've worked with a lot of people who run these transformations who just really want it to work. and I think they do it with the best of intentions. But the C-Suite has to understand, like, it's not just a transformation project, right?
Starting point is 00:47:59 Like, this is a whole new way of working. And if we want a whole new way of working, we have to really rise to that occasion. This episode is brought to you by Coda. I use Coda every day to coordinate my podcasting and newsletter workflows. From collecting questions for guests, to storing all my research, to managing my newsletter content calendar.
Starting point is 00:48:19 Coda is my go-to app and has been for years. Cota combines the best of documents, spreadsheets, and apps to help me get more done. And Cota can help your team to stay aligned and ship faster by managing your planning cycle in just one location, set and measure OKRs with full visibility across teams and stakeholders, map dependencies, create progress visualizations, and identify risk areas. You can also access hundreds of pressure-tested templates for everything from roadmap strategy to final decision-making frameworks. See for yourself why companies like DoorDash, Figma, and Qualtricks run on Koda.
Starting point is 00:48:54 Take advantage of this special limited time offer just for startups. Head over to coda.io slash lenny and sign up to get six free months of the team plan. That's coda.coma.com slash lenny to sign up and get six months of the team plan, coda.coma.com.com. There's a few things I want to pull out from which you just shared. One is, just to clarify, just to clarify, you recommend not using something. safe. You don't think that's a good approach. I do not recommend using safe. I think there, yeah, great. There are people who like safe. So let me just say this. It's, there are people who found
Starting point is 00:49:29 success was safe. Every single person I have talked to who likes safe found success was safe, they ended up ripping it up and making it into something else. So I'm like, it's not actually safe by the book. If you do that, fine, right? Like, that's any process, right? Like, if you ended up adopting safe and you want to go back and look at it and say, actually, let's just get rid of all the stuff that's not working and keep the stuff that is, fine, right? But being open to understanding, like, this is not the way that we do good product management. Like, there is not a lot in safe about doing good product management. Like, that's the stuff that we have to understand. It could help in certain areas, and I do think it does help in certain areas, bring some, you know, some rigor to things. But
Starting point is 00:50:13 if you take it too far, it will destroy things. There's actually a great story about a water company in the Netherlands, right? And they decided to adopt safe. And this was on the news a couple months ago. They decided to adopt safe in their IT teams and start working with it. And they ended up going bankrupt. And the reason they ended up going bankrupt is because the teams were learning the processes for safe, they were taking so long to deploy their new invoicing system in payment collections that they couldn't collect payments from customers because it got so caught up in the process. And that's what I see happen a lot in these organizations. Instead of talking about what's really important, which is, hey, how are we serving our customers? How are we winning in this market?
Starting point is 00:51:02 How are we, you know, how do we stem churn? How do we do all these things? We're talking instead about like, what standups are we doing? And, oh, how do we do this release, release planning? oh my God, you guys didn't sprint back to back, like, you did it wrong. And that's not, we're talking about work about work, but we're not actually getting into what are we achieving here. And that, that's the part I do not like about rigid processes when it comes to this. So that touches on the other theme I wanted to bring up is it feels like the stuff is, uh, kind of replacement for skilled, talented people, like a product leader that understands how to do these things and has product taste and has organized teams to build great product.
Starting point is 00:51:48 It feels like people are just, we don't have that, so we're going to create this. This process is going to fix all our problems. Can you talk about just the importance of that, like the people you hire to run these things as key to this, if that's true? In a lot of organizations, the people who buy safe, they have not run large-scale technology organizations before, or they're new to this way of working. So they adopt safe and they hope it works because it looks like a nice plan, like we said, to go out and do things. And when you're doing a transformation, a lot of companies are pulling people into these roles for the first time.
Starting point is 00:52:21 I've said since day one that I've been working with companies, it's okay and I think it's noble to want to train people and put them in different roles. Like, cool. If you're going to spend money upskilling your people, like do that. But you also have to intersperse people who know what they're doing. And I think at leadership, it's really important to bring in somebody who knows what they're doing to help run this type of thing. So there are more people out there and more leaders who have done this before because we've been doing this for 10 years. So there's Shruti Patel. She's chief product officer at U.S. Bank for small business banking.
Starting point is 00:52:54 I just had her on the podcast. She worked at Shopify. She saw how great teams worked, right? And then she was able to come and help apply that at a bank. So she's experienced, right? She's an experienced product person who can help. in to help. Melissa Doros is the CPO of Green Dot Bank. And she had worked at Discover Financial, leading the transformation there, you know, did all that work and then could bring it to Green Dot Bank.
Starting point is 00:53:18 Like she can see what needs to happen, what needs to actually go on here. So we've got more and more people out there who have done this before, who are looking for these opportunities to do it in the bank. And I think it's important for C-suite to bring them in, right, to actually look at that. And where I've seen transformations be the most successful in all these organizations is when you do that mix, right? You keep, you keep some of your people, but you also bring people in to learn. And I get hired all the time to come in and train. I've worked with almost every like Fortune 50 company at this point, Fortune 100 company too. And I get in, I come in to train a lot of product managers.
Starting point is 00:53:56 And we do it through, you know, Product Institute. And we'll train everybody. and what used to happen about like eight years ago is they train everybody and they would say go. Where do you go after you bring in the consultants to do training to keep learning, right? How do they watch other people in the organization do great product management if there's nobody in the organization who's done it before? And luckily, I think a lot of organizations are realizing that. So more leaders are out there who are saying, hey, I've got to actually intersperse skills here. I need to bring in some more directors who are experienced here, some more individual contributors
Starting point is 00:54:32 who are experienced here. And those organizations, I think, are wildly successful because they recognize it and they say, I've got to make sure that people can learn from others. Because that's how you keep developing, right? Like, that's how we all keep developing. It's not just doing all external classes. And that's where I think these things become powerful. And you could do that at all levels. You don't have to just do it with the teams. You could do it at director level. You could do it at VP level. That's how we should be thinking about this. Now, there are some CPO out there and some VPs of product in these large organizations who are new to this way of working, but they've committed themselves to learning and to trying to figure out how to do it best.
Starting point is 00:55:09 And they're not saying, hey, I'm just going to adopt safe or I'm just going to do whatever it is over here. They're actually saying, what don't I know? And I'm watching them go out to talk to other CPO's, do all these other things. And they usually have great market knowledge, great business knowledge, and they're fantastic at strategy. and then they hire people underneath them who are great at the other pieces, like the execution and getting the software at the door. And I think those people are successful in it as well because they notice their skill gaps and they hire for it just like any great leader would. So in these organizations, I do see sometimes safe or something being a crutch for people
Starting point is 00:55:47 who don't know what they're doing to bring in. And if you really think about, hey, how do I make this better and have that continuous learning mindset and that way to want to propel this forward, I think you'll consider other options and start to think about broader than just safe, broader than just agile, what do we need to make this successful? But the key part of this too is recognizing that product management is not just this role in scrum. And I see this in my talk to. I say, take scrum away. You still need product management, right? Like, product owner doesn't exist without scrum. That's not a thing, but you still need product managers. And that's why all product owners should be product managers. They should be fundamentally product managers.
Starting point is 00:56:30 And that's why I do not like these career trajectories that keep them separate. Like, sure, if you want to have a principal, I see product manager like they do in a lot of large Silicon Valley companies, perfect. Like let people keep working on those things. They don't have to go into management. But that doesn't mean they're different between a I see product. owner and an IC product manager shouldn't be different there. Perfect segue to where I wanted to go next, which is say you are a product owner today listening to this and you're like, man, the sound, this is exactly my life. What can I do?
Starting point is 00:57:04 What's your advice to folks in that role right now about how to potentially become product managers, build the skills they need to be, to not just be stuck in this career path that doesn't go anywhere? Yeah, I think the first thing is bringing awareness to that your role is more than just working with the developers. a lot of leaders argue with me that we need product owners because it just doesn't scale. And I've seen, you've seen massive companies at scale where they don't have any product owners. So I do not understand that. It's a weak argument to me. It's a very weak argument.
Starting point is 00:57:35 It just means you don't know how to distribute the work evenly and give a little bit of strategic guidance to product owners so that they can go, or product managers on a team so that they can go and build visions and cut down features and stuff like that. If you're a product owner and you're like, hey, I don't have the opportunity to talk to customers. My product manager does that. I'm just working with the teams. I want to be more strategic. I want to think longer term. I'd say, try to take some ownership over that and push back on the things that are being given to you. I was doing a workshop for mine the product, like back in the day. And I had a product owner in my workshop. And she said, I don't think that things we're working on. They were doing safe are the right things to work on. And I said,
Starting point is 00:58:15 you should bring this to your manager. And she was like, I don't know, I'm going to get fired. I don't think it's the right thing. What am I going to work on if we're not going to work on this? And I'm like, well, this is the beauty of product instead of project, right? We stay with the product. Just because your project ends doesn't mean that you lose a job. So she put together this whole thing, went and said, I don't think we should be working on this.
Starting point is 00:58:35 And they promoted her. So they were like, fantastic, right? So she took this leap of faith and went out there and started saying, this is more. We need to do more. I think if you're a product owner and there's no career path for you, start asking leaders what your career path is because it's going to make them go, oh, great question. What should the career path be? And there's a lot of literature out there about how we make career paths.
Starting point is 00:58:57 So you can start there, right? Ask what's next for you after this product owner role. I would ask the product managers, if they're doing all the customer research, see if you can do some customer research with them, right? Go sit in on that. And a lot of them will say, oh, I don't have time. I don't have time to do this. Strip back. All the user stories you're working on.
Starting point is 00:59:16 Stop thinking about it as a quantitative metric that needs to just go up and up. Instead, really think about the value you're delivering with your team. Is this the right thing? And when you talk to leaders and when you present your case, you say it in a way of,
Starting point is 00:59:29 hey, I'm working on X, Y, and Z feature. What's the goal here? When we release this, what do we hope will happen? I think that's one of the best questions anybody can ask if they're worried their company is not focused on outcomes, right?
Starting point is 00:59:41 what do we hope will happen when we release this? What metrics are we going to change? How do we instrument it to make sure that's true? Then we can go back and actually see if it changed. One simple question to get alignment on it, and then you can start to say, oh, that didn't work, or this did work, great, why did it work? And you can open up those conversations. So I'd say there's a lot of things you can do to help move your companies forward. And I have seen in a lot of these organizations too, a groundswell of product owners and product managers saying, hey, you know, like, what's next for me? What's going on? That makes the organizations go and figure it out. I was working with one like Fortune 10 company, not too long ago. They're C-suite. And I've been
Starting point is 01:00:24 working with them for a very long time. And they're finally like, hey, we're going to codify the product manager role. And we're going to have it all the way up and down our organization. We're going to make roles. We're going to make responsibilities. To me, that was music to my ears. But the reason they were doing it too is because they noticed there was a lot of churn in the organization in that role. And they also realize it's a critical role. So they're losing good people because people from the outside are coming because they want to work for this great, you know, big organization that's doing super well, like fantastic. But they get in there and they go, where's my, like, where's my career path? Right. What am I supposed to do? Where am I supposed to go?
Starting point is 01:00:58 So a lot of leaders are out there now realizing like, hey, we do have to get our stuff together. and the only reason they're coming to this conclusion as well is because they're looking around, seeing other people doing it, and they hear it from the teams, right? They hear it from the product managers. So I don't want people to think, like, hey, I have no power. I'm in a 10,000 person organization. The more you bring this up, the more your leaders will respect it because they don't want to lose you, right? They don't want to lose good people. So if you want to be great at your job and you need more support there, like speak up, right? Speak up. At a certain point, I do tell people this, if you feel like you can't do great product management in your organization, try to find
Starting point is 01:01:37 another organization. And I know that is hard to say. And I respect like people are tied to insurances and, you know, it's hard to change jobs. But if you do have the opportunity to look for another organization that does it well, I would go there. I would also say in large corporations too, I've seen certain business lines and certain divisions do it super well. And then others not. So if you're in a large corporation, maybe think about moving internally, like laterally to a different team and seeing if you can work there. I'd find the leaders who know what they're doing and go work for that. That's usually the best move here.
Starting point is 01:02:11 The point you made about how a lot of companies don't have any product owners and have scaled very wide, just to reinforce that in the data dive that we did on job market trends, like no tech company has a product owner, basically, like no top tech company. And I know there's an important distinction here. These are tech software first product first businesses where their business is the software they're building. And a lot of the companies we're talking about here are not that. They're like banks and telecoms for pharmaceutical companies.
Starting point is 01:02:39 So I get that it's like a very different world. But I think it's important to highlight. Like you can become very big. Like Google has no product owners as far as they know. Amazon, Microsoft, Netflix, like no company you've heard of that's a tech company has a product owner. They have all product managers. They're all product managers.
Starting point is 01:02:56 Yep. And I don't want people to think that there aren't people who build great software in these large corporations too, because there are, right? There's pockets of people who are doing it super, super well. And if you are one of those people who's been pushing the boundaries, doing great work, and your title is a product owner, what I always tell people on your resume, if you're looking for your next job, so that you're not like pushed out, let's say, of these large corporations like a Google or somewhere like that. And that's where you want to work in a tech firm. Make sure you describe how you did your job from a value perspective. Do not talk about your agile cadences. Get scrum out of there, right? Talk about what value you brought to the users and what metrics you moved, and that's how your resume should be laid out. Because I do read tons of resumes to hire people. And also, like, Chief Product Officer, same thing.
Starting point is 01:03:46 If I see immediately like implemented scrum processes across the organization, I'm like, nope, that's not what I need, right? That's not what I was looking for. I was looking for what are you going to do to push the strategy in that part of the organization? What are you going to do to actually build better products for customers? And then when you get into the interview, you can talk about what things you did to do that. But you want to make sure that you're focused on really understanding the customer and translating that into great products and the outcomes that we were looking for when you do it on your resume. I think that's important.
Starting point is 01:04:15 Okay, I want to spend more time here because this is so important. So this is kind of highlighting here's the difference between a product owner and a product manager. And so if you want to move into product management and become a great PM, if you're a product owner today, even if you're not a product donor and just want to get into product management, can you again just highlight here's like the big difference and here's the skills that people value most in a PM versus a product owner? Yeah. When I see product owners write out their resumes or describe their job functions, they always approach it from a process standpoint. I prioritized the backlog. I worked with the developers to break down the work. I checked the developers work and did the acceptance criteria.
Starting point is 01:04:54 I wrote the user stories. All those functions is what I see it over and over and over again in product owner resume. What you want to do instead is say, I led the, we can even do the login API, right? I led the work around the login API. The problem that I was solving around it was, you know, trying to enhance security for our login protocols to meet regulatory requirements. I interviewed a bunch of users. I got up to speed on the regulatory requirements. I worked really closely with our legal teams and our compliance teams to translate that into something that was going to secure our bank. And when we launched, we were able to meet our compliance, save our bank,
Starting point is 01:05:31 like a couple million dollars. And we smoothly transitioned into these new security requirements without disrupting any service for customers for four million customers, right? Way different story than I prioritized backlog and I shipped it off to developers. Take it a step further. If you're working on customer facing things, who are your customers? Did you go out and talk to them? By, you know, interfacing with customers and understanding them, I was able to solve X, Y, and Z problem with them, which resulted in a measurable amount of X, Y, and Z metric going up for the business. I ran this function. I ran this feature. I launched this feature. I watched it through. I iterated on it. I did the stuff that was needed to make this successful. That's what I want to see
Starting point is 01:06:13 on a resume. So even if you have a product owner title, I'll still read the details and everybody else will too. But I will say there's sometimes a poor connotation when you have that title, unfortunately, because of the baggage that's associated with Agile. So even just like on resumes, I would say do product owner slash product manager, right, like in there, just to let people know that I know how to do this and I've been doing this well, right? But if you do that in your bullet points, that shows up as well. There's also this whole concept. that we didn't even get into about certifications. And people keep asking me if I want to transition into product management,
Starting point is 01:06:50 should I get a certification and Agile certification? And I feel like these were bigger a couple years ago, but they're still big. So if you ever see somebody with a CSPO on the end of their profile, which you probably have seen on LinkedIn, it's a certified scrum product owner. Now, one thing to remember, and this is about all agile stuff, is like, we call it the Agile Industrial Complex. Agile coaches and agile trainers, so like the whole scrum team, scrum.org, Scrum Inc, safe, everybody.
Starting point is 01:07:22 The way they make their money is through consulting to teach you these processes and by having people be trained to get these certifications. So they come in and they say, all your people need to be certified scrum product owners. So it's like, give me, give me $2,500 per person. And then they get a certificate at the end of a two-day class that says they're a certified scrum product owner. Doesn't necessarily mean they could do the job. Doesn't necessarily mean they could do product management.
Starting point is 01:07:47 But let's think about, like when we were talking about too, and should we adopt safe in general, should we adopt these things? Think about how these organizations make money. They're selling certifications, right? Of course they want more and more people to adopt it. Like that's the idea here. So they're selling you this dream that, you know,
Starting point is 01:08:06 you just certify all your people and then you can be working on it. So they take all these people, they put them in two-day classes or whatever, and they turn them out, and then they say, go, like, you're a completely new role. It doesn't work that way, right? Like, that's not in the best interest of your company. That's not really what we're looking for here. And that's why all of this stuff needs to go deeper. So if you've done a CSPO class and you have that certification, it may help you get hired at another large enterprise that is adopting scrum and safe, right? That will probably help you there.
Starting point is 01:08:35 if you want to transition into tech and go into the companies that we talk about, they're probably going to look at that and say, this person doesn't know what they're doing, right? Like, this is not here. So if you do know what you're doing and you did that for a reason, because some people need that to get promoted, some companies actually require it, which is crazy. But like, to get promoted or be that thing,
Starting point is 01:08:55 I have tremendous sympathy for that. You're going to have to do a lot of work explaining in your resumes and stuff as you transition that you know more than that. Right? You're not just a CSPO. You're with a test. two-day class, you have done the were. And that's where all of this building up on your resume becomes really, really important. It's not just about getting certified. That's why I had people
Starting point is 01:09:15 ask me, they're like, can you just certify product managers? And I'm like, no, if people take my course, I give them a certificate of completion and it's a completion. So you finished a course, just like any other course. But I will not certify product managers because I do not think you can ever say somebody is prepared and able to do their job from a short class. Now, if you are doing, there are some agile agencies that do a lot more training where you have different levels. And what they would do is they would train people, but then they would make them go do work and they had a coach that they worked with, and they go back and forth and they could demonstrate that they could do the work over time to get to the next level. And it's almost like the
Starting point is 01:09:57 PMP, like the project management certification where you have to have time actually doing the job. That's different, right? Like, that's a different type of skills, a different type of certification. But if you see any like CSPO's, it's typically a two-day workshop that they went to and then got certified. So that's the difference with this. And I would say, be careful if you are a product owner wanting to be a product manager of just certifications. Like all the large tech organizations I know too, they're not looking for certifications in product management or product ownership to hire people. They're looking for experience. But the organizations that might not know what they're doing, they are looking for CSPs, right? They are looking for that. And then if you're, it's required by your
Starting point is 01:10:39 organization, you might have to ask, like, do, are we all well set up here to do our best job, right? Is this the place where I'm going to learn how to be a better product manager? I also feel bad, though, for people, because it's hard to be a product manager. It's hard to get your foot in the door, right? So I'm so torn on it because there are organizations that hire people with a CSPO and they require it. So of course, if it gets your foot in the door, you know, and it helps you do it. But if it's not going to help you and it's not going to put you in the career path you want, I don't think it's worth money. I think one interesting thread throughout this whole conversation is rarely is a plug-in playish easy solution going to be the solution, the answer to your problem. whether it's plugging in safe and it's going to help us build great software,
Starting point is 01:11:27 taking a class, helping you become a great product manager. He's skeptical of that. Yeah, there's no quick way to doing any of this, right? There's no fast track. We don't get to skip over all the hard things. Yeah. Bummer. I know.
Starting point is 01:11:41 I wish. One question I wanted to clarify. So when you come into an org and they have product owners, do you encourage them to get rid of product owners as a title and roll and make them product managers, or do you keep product owners? I say that we should have a hierarchy, right? So you would have a, all product managers on a team would either be an IC, right? It's an individual contributor. So they're either an associate product manager. And that's if they don't have all the discovery experience or, you know, maybe they know basic scrum, totally fine. You can be an associate product manager. But if they don't know how to talk to customers, digest what the customers are saying and turn that into a, you know, feature direction and a backlog and this is how we're going to work. All that stuff needs to be. be in there, right? If they don't know how they should be measuring things, you're not quite a product manager yet. So associate product manager and then product manager and then a senior PM,
Starting point is 01:12:31 right? A senior PM can work on a scrum team as well or a development team. But I do encourage them, I say, get rid of these two different titles because it's confusing everybody. I've seen companies, I hope a lot of people with career paths too, I've seen companies with 47 different titles for product managers because somebody in this organization is called like a product associate and somebody over here is a product manager and that person is a platform product manager and that person is a platform product owner and this person is an API product owner like you know you can have product manager product senior product manager all that stuff but I would not confuse people with the two different titles of product owner versus product manager but importantly there's a bar for who is
Starting point is 01:13:15 called a product manager because if you take a product owner as you've said and say your product manager, they're going to be a terrible product manager. So your advice is make them an associate PM, and then once they reach a certain level, they've graduated to product manager. It depends. So I don't say this as a hard and fast rule because there are product owners out there who've been doing their job very well. So just because they have a title of a product owner doesn't mean that somebody can't do the job of a product manager. There are plenty of people out there who are just named this because the company names them that and they know what they're doing. So don't look at that hard and fast. When we do this, we typically will say,
Starting point is 01:13:49 everybody's got product owner on their title changes to product manager. What we would go back and look at was the actual skill set. So when I've come in to work with companies on transformations, what we typically do, the way I get introduced usually is we come in and they're looking for some kind of training for all of their product teams. And then we bring in Product Institute and we do our online training. And then afterwards, I work with the organization and I say, now that everybody's been baseline and trained,
Starting point is 01:14:14 we have to figure out who is going to be a great product manager and who's not. So there's an awareness. You know what's interesting. interesting. Once people figure out what the job's about, a lot of them opt out. So when I did the transformation at Athena Health, we had 365 product managers, and they all had different titles to, product innovation all over the place. So we turned them into product managers. We trained everybody. We gave everybody the opportunity to learn, which I think is great. And then we gave them opportunity to practice their skills. But what happened is at the end of the training, a lot of people raised their
Starting point is 01:14:43 hand and said, oh, no, no, this is not what I wanted to do. Like, I thought it was very different. I did not realize how much, you know, people work it was. I have to go influence these people. I have to do all this stuff. Like, this is actually not really what I wanted to do. And we found other roles for them that were, you know, what they wanted to do or they decided to leave. And that's, that was up to them. We didn't, we didn't cut anybody.
Starting point is 01:15:07 But we moved some people into operations because they wanted to be a little more heads down to find process, right? Like, that was their, their thing. We moved people into data roles who were good with SQL and we're good with data analysis. We move people into user research roles because they wanted to talk to customers, but they didn't want the responsibility and the accountability that came with the product management piece because they realized it was so intense. So I've seen this happen a lot in large organizations. It's like you baseline everybody, you give them the experience, you show them what the role is, right? And then you let them go practice it. And then at that point, some people will
Starting point is 01:15:38 opt out. And then you have to go back through your people and say, okay, how do we level set now, right? Who's, who's doing really well? Who's not doing so well? which teams need to have more experienced people on it because they don't have anybody to learn from, right? And that's where we would say, hey, let's hire some ICs over here or a director of product management who can help train these people and help them keep growing, right? And that's been super successful. I've seen people bring in some more directors, scatter them around, and they've leveled up these product donors who were not necessarily doing great product management. And now, they're doing fantastic. And I've watched fantastic leaders in these organizations that are
Starting point is 01:16:17 you know, not software native or doing these transformations, you bring in the right person, you can make amazing product managers, like give them a year or two and completely turn it around. So it's totally possible. It's totally possible to take people and train them. And I firmly believe in that. But you got to get them exposure to what good looks like. And if you're in an organization and you cannot see what good looks like anywhere, that's a red flag, right? That's where leaders need to look at their organization and say, do I have people with these skills interspersed around so that everybody can start to learn, right, so that everybody can actually be on the uptake of this and make sure that we are well balanced.
Starting point is 01:16:56 That was an awesome nuance and addition. So I'm just looking back at all the things we've talked about. We've covered so much ground, kind of the history of agile and scrum and safes and product owners. We've gotten into all kinds of advice on how to do this better as a company leader, as a product owner, as someone thinking about even moving to product management. Is there anything else that we have not covered that you think might be helpful to touch on or wrap up on? I think I would tell people to remember that when you look at Agile methodologies, and if you look at it with small Agile, what we're really saying there is we want to be able to move
Starting point is 01:17:34 quickly and deliver great value to customers. If you embrace those principles, you know, you're going to do well. But if you think of Agile as just like a defined super cut and dry process where you have to follow every single little step here and there, that's not going to serve you, right? Because you're not getting back into
Starting point is 01:17:52 why are we doing this. And there are some great agile coaches out there. There are first principal approaches to doing great product work and doing great development work, right? They're there because they are not just selling scrum. They are there to make people better. And those agile coaches, I think,
Starting point is 01:18:08 can make great, fantastic teams. And I've worked with a lot of them. And I think they are really there to just make it a better environment for the people who are working and to help the company produce better products, right? Like, that's their whole goal. But then there are people, too, who are wedded to, this is, you know, this is safe and you will do it by the book. This is scrum, you will do it by the book. And I'd be very skeptical of that. Because what's, what's the end goal? Right? What's their end goal? And like I said, a lot of people out there
Starting point is 01:18:37 get paid by certifying people and by consulting on these processes, right? McKinsey made a huge division to do this. And a lot of SAFE was actually introduced to organizations from McKinsey and from large organizations, consulting organizations like that. They came in and they said, yeah, this is what you do. I'll teach you how to do SAFE. I'll teach you how to do SCRUM. And they have been building those consulting agencies off of agile transformations. And we always laugh. Like when I talk to other people who have been doing transformations and coaches, they say, I go, why is McKinsey always coming in here and doing this? And I followed them into so many organizations. been like, oh, wow, they screwed this up. Like, let me go fix it. And there's always a saying that
Starting point is 01:19:18 nobody gets fired for hiring McKinsey. So McKinsey's big. People trust their name. A lot of the people at McKinsey haven't done this before, right? They've never been inside the organization trying to transform it and stay there for a long time. And that's why I'd be really skeptical of who's selling that. So if you approach agile from a perspective of I want to be agile because I want to release things quickly and get feedback from customers and make sure that that's great, look at all of your processes like that and say, is that serving the best interest of our company and our, in our culture and our customers, right? Is this making our customers proud of us, right? Is this helping our customers receive good value? And if your processes aren't, fix them, right? Change them.
Starting point is 01:20:05 Inspect and adapt is a lowercase agile principle. So we should be looking at every process we do and saying, is it working and not be afraid to change it? And I think it's important to highlight this can work. There are ways to reorganize the way you built product at a very large company where you can actually deliver great product consistently. You've seen that happen a lot. Anything you want to add there to give people a hope? Yeah.
Starting point is 01:20:31 I mean, the biggest pushback I hear from large organizations, especially ones that are not software native as banks or insurance companies, whatever it is, is that we're not like Google. We're not like SaaS companies. And it's true, you're not a SaaS company. So the way that you do it is going to be slightly different than the way that Amazon runs or the way that Google runs. I don't see many companies actually run the same, right?
Starting point is 01:20:55 And just because it works at Google doesn't mean it's going to work at an insurance company that's 100 years old, right? That's okay. But you could still learn from people who've been producing software at scale for many, many years, right? You can still learn what makes them successful, and then you can take some of those principles and apply them to your organization and then figure out where we need to adapt. And that's what I would look at, right? I see software strategies and digital strategies becoming so intertwined into the strategies of companies in general, whether you're an insurance company or a bank or anything, right?
Starting point is 01:21:32 They're becoming so heavily entwined in the C-suite strategies. And the first thing I would say, and where I see a lot of companies struggle with this, is you have to make sure that it is thought of at the C-suite. And what some organizations do is go, oh, no, that's like IT work. Let me push all the software strategy down. And if you're doing that, you're missing out on innovation, right? And this is where the product role comes in and where I think more organizations need chief product officers, because they're the person in the C-suite saying, how can software enable us to be 10 times better and crush our competition? whether we're a bank or insurance company or a pharmaceutical company. What can we do with our software that's going to put us ahead of the competition,
Starting point is 01:22:14 that's going to put us ahead of the market? And if you're not having those conversations when you're thinking of your long-term company strategy, you're behind, right? You're behind because that is what every small, high-growth startup is doing that's coming out to try to disrupt these big companies. And I think the big companies are successful. They have a lot of money, so they don't have as much urgency as sometimes these smaller companies do to change
Starting point is 01:22:37 or somebody who's failing to change. So that's why things are going so slow. At the same time, if we don't pay attention to that and we're not considering it, that's where the danger comes in. I imagine people are going to have a lot of questions and hope to get even more advice. So a couple of final questions,
Starting point is 01:22:52 where can folks find you online if they want to ask more questions along these lines, maybe get your help on some of this stuff? And then finally, how can listeners be useful to you, Melissa? You can find me on LinkedIn. So if you go search for Melissa Perry with an I, P-E-R-R-I, you'll find me there. My LinkedIn-P-N-P-R-I. Happy to connect with people there.
Starting point is 01:23:12 My school is called Product Institute. So if you're interested in doing product management training or getting help with some of the stuff, go to product institute.com and reach out to me. And then I run a podcast, too, called the Product Thinking Podcast. So we answer questions every week, and a lot of them are around safe and agile and all these topics. So if you have a question and you're saying, hey, how do I do this? definitely reach out and let me know. Awesome. Melissa, I learned a lot.
Starting point is 01:23:36 I feel like a lot of people listen to this podcast. I had no idea about any of this. And now I understand all these terms that I kind of hear occasionally. So thank you for coming and sharing all of this with us and especially the advice for folks that are in this world right now. Yeah, thanks for having me. Bye, everyone. Bye.
Starting point is 01:23:54 Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lenniespodcast.com. See you in the next episode.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.