The a16z Show - Innovating While Scaling: Lessons from Amazon

Episode Date: May 18, 2022

In this episode from February 2021, early Amazon execs Colin Bryar and Bill Carr -- in conversation with a16z's Sonal Chokshi -- go beyond the well-known artifacts of Amazon innovation, like the memo and the press release, and share the leadership principles, decision making practices, and operational processes that helped Amazon continue to innovate, invent new products and learn from its mistakes, as it scaled. It’s all based on their book, Working Backwards: Insights, Stories, and Secrets from Inside Amazon, drawing from the 27 years combined experience of being in the room where it happened at Amazon. Stay Updated:Find a16z on YouTube: YouTubeFind a16z on XFind a16z on LinkedInListen to the a16z Show on SpotifyListen to the a16z Show on Apple PodcastsFollow our host: https://twitter.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures. Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

What can today's startups learn from Amazon's so-called, quote, invention machine, and the culture of growth and innovation there that led to several successful and diverse lines of products? As it scaled, how did Amazon define and adapt its culture and practices to promote innovation even as it got more complex and large? In this episode from February 2021, early Amazon execs Colin Breyer and Bill Carr go beyond the well-known artifacts of Amazon innovation, things like the memo or the press release before the product has even been built, and share the leadership principles, decision-making practices, and operational processes that helped Amazon continue to innovate, invent new products, and learn from its mistakes, all while it scaled.

It's all based on their book, Working Backwards, Insight, Stories, and Secrets from Inside Amazon, drawing upon the 27 years of combined experience they had of being in the room where it happened at Amazon. Hi everyone, welcome to the A6 and Z podcast. I'm Sonal and today I have another one of our special exclusive first looks at a new book episode and it is both a very timely and evergreen topic because a new book coming out this week is titled Working Backwards, Insights, Stories and Secrets from Inside Amazon. In it, authors Colin Breyer and Bill Carr, who between them have a combined 27 years of experience in the company, where Bill was vice president of digital media, founded and led Amazon music, Amazon Video, Amazon Studios for a decade, and where Colin started out in the software

group was a technical vice president, and then notably was one of Jeff Bezos's earliest shadows, a legendary program there. Fun fact, the first shadow before that, I believe, was Andy Jesse, president and CEO of Amazon Web Services, and now to be CEO of all of Amazon. The book actually shares the origin story of AWS, among other businesses, there, which we touch on briefly. Though, as a reminder, none of the following is investment advice. Be sure to see A6NZ.com slash disclosures for more information. But in any case, our focus today is really on what is the Amazon way. And can other companies really adopt certain best practices to? In fact, as fast-growing companies establish and find their way, how do they define and scale their culture

processes and more? We actually spend most of the episode digging in detail into two operational practices in particular, the infamous memos instead of PowerPoints and working backwards from our press release and FAQs, given the presence of those two topics in tech folklore and lots of misunderstandings as well. So I actually probe for the origin stories, the specific details of how they do and don't work, and other nuances so organizations of all kinds can take what they want or need. Finally, we also debate within this episode the tradeoffs of lean and minimum versus maximum on viable products and whether the emphasis is on the wrong letter there, whether product managers have a role in companies organized like this and more topics throughout. But we start by very briefly

discussing the foundational principles. And actually, the first question I want to start with, especially since I hate the why did you write this book question, Colin, Bill, honestly, there's a tendency for folks telling these stories, these kind of narratives to do a sort of hindsight is 2020, not accounting for attrition data or the failure cases as well. So part of me is skeptical that startups can do what Amazon did. And what's most noticeable, too, is that Amazon had not just one or two product lines, but literally entirely different yet successful lines of businesses all under one roof. So what drives that and can startups really relate to the Amazon story then? Well, one thing is that the businesses you mentioned, they are substantially different.

They require different expertise. They're different customer sets. AWS is B-to-B. There's streaming video. There's that e-commerce business. But they all have one thing in common, and that's what we talk about in the book, and we call it the invention machine, which was the process and principles that Amazon used to develop these businesses. And the ones that we talk about, they started off as ideas on a whiteboard or emails, which many people were skeptical we should even do. Some of them grew into household names, but they all started off very small and with just one or two people. I would first start like going back in time a little bit and placed you sort of

of where Jeff Bezos was where all entrepreneurs start out. So at the beginning, Jeff worked out of a simple office building with a handful of employees, and he would be hands-on for everything. The very first customer support emails like Jeff rode or co-wrote. He could review the work and think about all the policies. He could direct the team and set the tone and the pace. Well, that works just fine when you're an early stage company and there's fewer than 20 of you and you can all do a stand-up each morning and give me hands-on. But guess what? That completely breaks once you start to grow like a weed and you've found product market fit and suddenly you look around and realize you've got a team of 130, 200, 500, and you realize that there's all kinds of

decisions and meetings happening around you. You can't be part of every decision. And so to me, what's most remarkable, notable, is that Jeff sought to figure out ways to actually inject his lens of thinking into all those meetings and then sought to create a bunch of processes that would reinforce the way he would think about the work or do the work itself. Is this idea of operationalizing the invention machine, as you guys are describing it, some of those principles and processes at a high level to quickly summarize just that context for our listeners. They range from customer obsession, ownership, invent and simplify, our right a lot for leaders, learn and be curious, hire and develop the best, insist on the highest

standards, think big, bias for action, frugality, earn trust, dive deep, have backbone, disagree and commit, deliver results, which I love, and we don't have to unpack all of those because that will take, like, all day, and it's a whole reason your whole book exists. I do want to ask about one before we go into some of the other practices, which is around leading. And the one that really intrigued me was number four are right a lot. And you basically write that leaders are right a lot. They have strong judgment and good instincts. They seek diverse perspectives and work to disconfirm their beliefs. And I love this because I always think to myself, yeah, damn it, leaders should be right all the time. Their instincts should be damn good. Tell me more about

that one, because that one made me chuckle a bit. Yeah, this is, and just to be clear, of course, we did not write that. That is Amazon's words that Jeff and the S team being his direct reports painstakingly reviewed to come up with that exact, language to define that principle, as with all 14 others. And they sweat it over the details of every word, every sentence. And this principle write a lot, in some ways it's very straightforward, which is that as you move up in early stage CEO, as they grow and progress, they need to be more in the mode of delegating important work and auditing work, but their most important job, frankly, is to make decisions. And many decisions will come to you, whether it's presented

with a spreadsheet, a document, PowerPoint. There's all kinds of things. of data that we presented to you with those decisions. But there are very, very few problems where the data gives you the answer. At the end of the day, you're going to have to use your judgment. What Right-A-Lot refers to is, number one, when those leaders are confronted with those decisions, that more often than not, they pick the right door. But the second part of the definition refers to, more importantly, how those leaders make decisions, which is a lot of people think that leadership is about their very strong opinion, arguing their opinion, and winning that argument. And what great leaders do actually is they can stake out a position,

but they are willing to update, change their mind on what is the right answer based on new information. And even Steve Jobs talked about this at Apple, where some product that they launched, It ended up being a mistake. And there was one of his reports had been telling him all along, we shouldn't do this, we shouldn't do this, we shouldn't do this. And then after it launched and it failed, he came back to that person and said, why didn't you talk me out of this? And the guy said, wait a minute, I tried to talk you out of it.

What are you talking about? And he said, well, you didn't do a good enough job because you didn't present the evidence to me in a persuasive enough way to make me realize why your point of view was so important. And so it's thinking about it and framing it that way, about bringing forward the right data and the right information. information, and then it's also the job of leader to solicit that to make the best decision. Well, that is a perfect segue to a question I'm dying to ask you guys about because so many

companies, their culture is like the mission statement and the values that you shove in a drawer. And so people spend so much time talking about the words and the precision of what they want those principles or values to be, but not actually how to operationalize it. So in that vein, because your book really does outline how to operationalize that through the processes and practices that you guys share. one of the ones that comes up all the time in Silicon Valley folklore is the infamous no PowerPoints write a memo. So let's tackle this one first because what you just shared, Bill, about convince me, share the evidence in a persuasive way. The point is to be effective and be heard, you have to do that well.

So what is your best advice about how leaders and people in the group can share that information? So Amazon started experimenting with writing narratives in 2004. And it was a result of weekly meetings, four hours every week with the S-Team, Jeff's direct reports, where two to three teams would come in and give either updates on their business. It could be a decision that needed to be made or investigating new areas to go into. And the business was growing fast and we realized that we were not making the right types of decisions. Some of the meetings would go over. We never really accomplished what we wanted to.

I was Jeff's TA at the time. we had been looking at other ways to conduct meetings, and we're big fans of Edward Tufti, who's Professor Emeritus at Yale. He came to Amazon to speak a couple of times. And after one particularly painful meeting, it was later on in the week, it was toward the end of the day, Jeff said, let's stop doing PowerPoints at these S-Team meetings. It's the wrong tool for what we're trying to do. And let's switch over to narratives, which are really just now a six-page. memos. One thing that is a little bit mentioned understood is these ideas don't come out fully formed. They started out as four-page memos. It ended up that six-page was about the right length

for an hour meeting. But we did it because narratives convey about 10 times as much information. You know, the pixel density is about seven to nine times the pixel density. People read faster than people talk. And you can have multi-causal arguments in a narrative much better than a hierarchical PowerPoint, but we realized we just needed a better way to analyze complex situations and make better decisions. So we just experimented with this. And the first ones were not good. Just to quote, you guys, because this actually made me literally laugh out loud. You guys write, the first few narratives were laughably poor when evaluated by today's standards. Some teams ignored the length limit, blah, blah, blah. And I was like, yes, people are not. I mean, we may be wired to be good

storytellers, but writing is actually hard skills. I do worry that it becomes a bit performative for the best writers, not just the best presenters. Because one of my colleagues, when I used to work at Xerox Park used to call this pissing on paper, which is like this idea of like, you know, how dogs piss around their territory. There's also a real time component where people who are performing in real time, like leaving comments in the middle of the meeting. So a common misconception is, well, now it's just the best writers instead of the best presenters that's going to win out in narratives. We haven't found that to be the case. It's really the best thinkers write the best narratives. And some of the best narratives I've ever read are by people

whose first language is not English. Yeah, in fact, the best narratives, many of them were written by software development engineers who may not have even focused on their writing skills. A good narrative, it's a very data-based and fact-based document. And writing good narratives is way harder than making a good PowerPoint. And I think, honestly, a lot of companies don't do this just because it's hard when Colin and I brought this to other companies. But What tends to happen mostly is that the author will vomit out about 25 pages of just sort of raw data at you. Getting that person to then shape it and narrow it down to six pages of well-thought-out

narrative is really hard. Yeah. By the way, if you spend 10 hours a day reading detailed narratives, like, I got to tell you, it can be mentally exhausting. But when people muse, like, how does Amazon do it? Like, how is it possible that they can effectively manage such diverse businesses? one of my number one arguments is that they use narratives to conduct meetings, not PowerPoint. And actually a former colleague of mine, Derek Anderson, who is now the chief financial officer at Snap,

I think he made the observation once that Amazon has like a narrative information multiplier. It's a strategic advantage that the company has over other companies. Because their executives are like seven to eight times better informed about what's happening in their company. and they're able to give super granular specific feedback to those teams. It allowed the S team to stay connected at a much deeper level. As Amazon started to move into more and more businesses, the span and control of the S team didn't grow by 100X. It's still a relatively small team,

and they are just as involved when it was a small company as they are now when it's a large company. The other thing I'd add about narratives is it does remove a lot of bias from the process where you can have a charismatic speaker with a so-so or even a bad idea that convinces your company to do something that you should not do. The converse is also true. If you have a shy engineer, it's a great idea, but it's a boring presentation. It's one way in which Amazon removes bias to make better decisions.

And the idea floats to the top rather than how good of a talker the person is. And we could go on about this forever. but the other part is it actually is a great way to get your team more engaged from top to bottom, especially in a COVID time. The way that these meetings work is you share the document, and whether it's with G Suite or with Word, then everyone can then use the comment feature, and it doesn't matter whether the person making the comment is a C-level person

or a fresh out of college individual contributor. All of those comments get seen and heard. And then when you get it to the discussion phase, all those people actually can then understand, you know, why we're making the decisions we're making. When you do a PowerPoint, you have to wait to get to the punchline. And while you're waiting, you're not sure like, well, what are we actually doing in this meeting? With a narrative, you just take all that in. And so then you can have high quality discussion versus that interruption and disjointed conversation you have with a PowerPoint. And if you miss the meeting, you can just read that document.

So there are so many ways in which the narrative method is superior to PowerPoint that, as you can tell, we could never go back. Okay. So how does one do meetings then based on these memos? One of the things you guys said is that sometimes it's shocking because the first 20 minutes of a meeting would be silent. And that made me chuckle, by the way, because one of the co-founders of Rome, the note-taking app, was sharing that sometimes they have entire meetings that are silent because they're just sharing notes with each other in Rome. Yeah, so Amazon meetings with narratives, they are strange to the uninitiated. It'll be the chit chat before meeting, you know, when they were in person or, you know, this online people are starting to come in.

But then there's silence for about 20 minutes. And during those 20 minutes, people are just focused on reading. There's no sound. They're entering questions and sometimes the presenting team can answer the quick questions right in the comments. So for that 20 minutes, really there's a huge, transfer of information that you can't see. And then the rest of the 40 minutes is really just high-quality Q&A and discussion on the questions that have already been entered or that come up.

The six-page narrative is a forcing function to where you can cover that amount of information in a one-hour meeting. Tell me a little bit more about what needed to change in the specifics of meetings in terms of running them because the premise of this conversation is not everyone starts out as Amazon and that startups can do these things. I'm trying to really tease apart. Like, are we just replicating the meeting dynamic in the memo? And then the same thing happens anyway. People would begin reading them in the meeting. But then how would the decisions happen? Like, what happens next? There are a variety of different kinds of documents you'd review in a meeting. It could be an annual operating plan. It could be a monthly business review.

It could be PRF aQ about a new product. So the conversation, literally what you do, once people stop reading, is you just go page by page. Or alternatively, if it's a smaller group, and let's say there's just 10 or 15, you could literally just go around the room and say, okay, Sonal, you're first. What questions and comments do you have on the document? And so you would get the feedback questions and comments from all participants, and then the document itself would present some sort of specific proposal. It's asking for some budget amount. It's asking for, are we going to launch this new product? And at the end of the meeting, you are tying it up and wrapping it up and saying either,

no, we agree that what you've written in this document, that plan works, approved, go ahead. Or you debate and discuss it. One of the things you might do at the end of the document is make a clear section after you've presented all the facts to say, what are the decisions we need to make today and then list those out? Or what are the important parts of this plan where we need your feedback and input? it. Like, we're not sure whether we should go down path A or path B. A good document will clarify what are the decisions we're going to make. The one thing I would add is that a lot of first timers to narratives, right after people read, they say, let me walk you through the narrative. And you stop that

right away. You know, you've just had that 20 minutes of high fidelity, Ben, why dumb it down with a two-minute verbal walkthrough of the document? You want to get to that feedback loop as quickly as possible just talking about the document. We didn't actually really say what goes in the memo. You observe that the memos can vary in form and format by function and purpose. But can you at least describe what goes in the memo specifically? Like, even the ingredients would help. Amazon has different types of memos for different purposes.

So if it's some monthly or quarterly or annual review, there's the typical what were the key wins, what did we do wrong, and what can we do better? and what are the key initiatives coming up for the next year. You can have appendices too, and the appendices are supplemental data that everyone's not required to read in the narrative meeting, but if you do need to go jump to it to answer a question, you can. It would include tables with like, here's a summary of our financial results from the prior year, here's a table of the summary of our plan for the next year.

They also worked very well with design, which you may not think about at first, but if you're going over mockups, either UI mockups for an app or physical prototypes of some hardware or process that you're going to build, having a narrative actually helps set the stage to say, before we take a look at anything, here's what we're trying to accomplish with the user experience. Here are our goals. Here are the challenges that we're trying to solve. You know, I've been at mock-up meetings where everyone thinks they're a UI expert, but if you don't have that before and they comment on, you know, three or four things where we should move this over here. but you don't do that if everyone's on the same page with reading a short narrative beforehand.

Another thing that can be in a narrative that's really powerful, especially for something that's going on on an iterative basis, as if you're refining an idea over time, are tenants. And those are really, okay, what are the design criteria that we are not going to compromise on or that we're going to fall back on when we have to make tough decisions? And that's front and center, usually up in the beginning of the document, before we go over Amazon's pricing policy, I want to remind you, here are the four tenants that we're following to make the following decisions. And to get to those tenants, it was difficult.

It took several meetings just to say these are the correct set. That's a good cashing mechanism because if you're a manager at a company that uses narratives, you're going to be contact switching and reviewing multiple ones every day. And sometimes we only meet with a team once a quarter, and you want to be able to very quickly get up to speed. There's another important technique where you can actually add at the end of the document a fact section for frequently asked questions. So you're actually anticipating the kinds of questions that the audience are going to ask you. And a talented senior leadership team is going to ask you hard probing questions about things like, well, you say in your plan that you're going to go do X, but it seems to me that there's an important hurdle. here of you're going to need this important partnership, you have this important dependency.

What gives you confidence that you can actually solve that problem? And you would answer it. Not only does that help reflect whether the author and the team have good mastery of the issues, but it also helps speed up the discussion and the decision making. A lot of my work leading, you know, Prime Video was making very expensive multi-year agreements with motion picture studios to acquire their films and TV shows for the service. These were big numbers, a lot of money involved. If we're going to go spend that much money, we're going to go write up a deal memo that describes like,

okay, we think we should acquire these films and TV shows. Here's what the package works. Here are the detailed metrics that show why we think this is a good investment or why not in some cases, because sometimes we would review it and say, we've looked at this deal and we're not going to do it and why. And again, just because it's a word document It doesn't mean you can't put it in a chart, a graph,

some in the appendix put an Excel table. All those things can still be in there. But then the narrative needs to clearly state with a clear beginning and middle and end. Here's what we're proposing to do or not proposing to do. And why? Yeah, I think the thing I found most compelling is how much this mimics how writers think, obviously.

One of the things that I found when I first came to a non-media company and was working at one previously was, oh my God, I have to make so many freaking PowerPoints. And I hate it because it's a muscle that's not ideal. for storytelling as much as people think it is. It's not. It really isn't. But the next, what I love about what you're saying there is that the FAQs is actually the equivalent of doing the inoculation technique, which is what really good op-ed editors will really build in. And that's not just because you're trying to inoculate the counterparty to the arguments they're going to make.

It actually what you're really saying there, and I want to pull this thread is you now can have a better, deeper discussion because you've laid to rest all the common things that you can literally get out of the way in like a memo in 20 minutes. Yeah, I mean, and a good document, that all comes out. I want to actually go back to the tenants for a quick second and then pick up on a thread that you brought up. One of the things Colin, you described it as a caching mechanism for the leaders where they have to do a lot of context switching so they get to kind of revisit that cash when they have to get back into that context. Yes. But I was thinking about it from the point of view of the group in the room, not just the leader. And how it really sets the shared context for

the room to have the most collaborative mindset possible while disagreeing within that framework. And in your book, you write that tenants give the reader an anchor point from which to evaluate the rest. But here's the best part. If the tenant itself is in dispute, it's easier to address that directly rather than take on all the logical steps that derive from that position. And sometimes you guys spent meetings after meetings just debating to get the tenants right, which I think is really great because it allows you to actually separate the thing that the tenant is and then what flows from that. Yes, I spent 15 years at Amazon and I've since gone on. And one of the things that shocked me was that certain topics get like recycled for debate constantly, right? Yes. Yes, exactly. We don't, we don't do that.

We didn't do that back at Amazon. Well, why is that? And one is the use of a tenant, which is that all that debate and discussion can land on a fundamental issue about what this company should or should not do. And if you don't come to a common agreement on it, then you will be constantly, you'll waste so much time with relitigating these issues. The second reason is that what the narrative process forces you to do is by actually putting down a piece of paper not only the tenant, but then like here's, so here's the specific plan, then you set the date for the meeting and everyone reads it and then you decide. And I realized that a lot of those things get relitigated.

So it was one of the ways which Amazon was very effective, which we don't realize as a management company. Yeah, on that note, does it then serve an archiving function for new hires and onboarding that you scale that tacit knowledge that's been made explicit? Yeah, how does that work? Great point, because the other thing I also saw is like,

whenever you hired someone new in, within a matter of a week, they would want to go relitigate. They, too, would trip across, well, why don't we do blank? Or why are we doing it's like, oh my gosh. And you can just say, look, here's the tenant, here's the document, you know, take a look at this, and then come back and, you know, talk to me. Right.

And then when you do decide to relitigate, it's an actual intent versus an accidental, every new hire repeating recycling the conversation. Okay. So the other big Silicon Valley folklore when people talk about best practices from Amazon is this idea of working backwards from the press release. Now, I know that you guys talk about working backwards well beyond. It's a reason it's a title of your book, obviously. But I really want to probe specifically into the mechanisms of what that means because, like, how does that happen? Is it just, oh, I write a press release for this thing I want to do?

It's a product. It's an idea. It's a service. I really would love to hear what, where, how, and why. And also I'd love to hear the origin stories if you have any specifics there as well. At its heart, the working backwards process is really starting from the customer perspective and everything you do works backwards from that. The PRFAQ, the press release and FAQ, that is the tool that Amazon uses to achieve the perspective of starting from the customer.

It's different than how a lot of companies develop ideas and products. A lot of companies use a skills forward approach. What are we good at? What are our core competencies? What are our competitors doing? How do we nudge into this adjacent market? How much market share can we get? When I was in business school and taught to think about like how you expand and grow,

as Colin already described, you create a SWAT analysis. Was it strengths, weakness, is opportunities, threats, right? But there's no C, there was no customer. Right. Amazon has put in deliberate mechanisms to make sure that the customer is front and center from the very first iteration of an idea. So if anyone says, raise their hand and say, I have a great idea, the first thing that the manager or the person in group will say,

that's great. Why don't you go write a working backwards document? And what that is, it's two things. It's a press release and then a frequently asked questions document. The press release has to clearly explain to the customer what it is your building, what's the problem you're trying to solve for the customer, and why does it make their life better? It can be something very small or it can be moving into a brand new industry.

It's a fractal process, which is great. Another thing is these PRFAQ documents, it's an iterative process. First of all, most of the ideas that go through it don't make it out on the other side. And second is that it takes several, several iterations and feedback to refine before the project gets greenlit. In fact, the origin story of how we got to the PRFAQ, or at least a part of it, both Colin and I were present for this because in 2004, I landed on a new role working as one of the founding members of the digital media team at Amazon. For perspective, at the end of 2003, 77% of all... all of Amazon's worldwide revenue was media products, but it was all physical media products.

It was books, CDs, DVDs, VHS tapes. And the writing was on the wall that, like, this business is not here to last. There were already a couple million iPods that were sold. Millions people were using Napster to while sharing. It's pretty clear that, like, okay, now that we have the Internet, it's just a matter of time before people consume their media digitally. But we didn't know how. So what I did, pulling out my bag of tricks from business school, is marched into meetings with Jeff with like, here's our projections for, you know, how big the ebook business will be over the next 10 years.

And here's our projected market share. And here's what the pro forma P&L looks like. And here's the kind of deals we'll make with publishers. And here's the competitive landscape. And I was so proud of all this work. And he looks up at me and says, Bill, where are the mockups? And I didn't have any mockups at that point. I was doing, you know, like, oh, here's the projection that all get started and we'll start

like working on like launching a better e-book store. And by where the mock-ups, what he meant was, well, you know, this is all super interesting or not really actually is what he was saying. But what's more interesting is like what is going to be the customer experience? And more to the point as we started to debate and discuss different ideas, whether it be in digital music or e-books, the discussion was about, well, why would we bother building like a me-to versus what Apple's already got, the iPod and iTunes. So what's ended for the customer to have just a knockoff of that service?

Like, what can we build that actually creates real value for customers? Something new. We should invent on their behalf. And we tried mock-ups for a little while, but frankly, there was sort of like so many questions and so many details that we hadn't thought out. And Jeff one day said, okay, I got a better idea.

Why don't we, everyone in this room, write up the idea for what they think we should go build in digital. We'll write those things up and we'll come back in a few days and we'll read those and we'll go from there. And this is before we had done narratives. We had not figured out this PRFAQ concept yet. But once we did that, everything changed and suddenly we were reading one document was describing like a puck. They would sit on your countertop and you would talk to the puck and could order groceries from the puck.

I had described a document about what we might go do in digital music. Another one was describing an early version of what Kindle might be. become. But some of these documents are like eight pages long. We need to get this to be, you know, more pithy and clear. And Jeff said, I know, let's write the press release for each one of these ideas instead. Ooh, neat. And he said, you know, we should write the pressure release because normally that comes last. And he said, you know, normally the engineering group and the product group, they go and they come up with the idea for the product. And then at the very end, when it's time to sell it, they chuck it over the wall to the marketing team and say,

okay, figure out how to go sell this thing. And he said, you know, But what if by the time that thing got to the marketing team, they said, yeah, well, the thing you built, for that really to work, we actually need that to have a price point of $99, but you've built it in such a way that it's costing us $150 to manufacture each one. So, yeah, we're not going to be able to sell too many. In other words, if you had known up front that you needed to hit a bomb of less than $99 to make this product go, then you would have designed the whole thing very differently and understood the constraints. It might have a whole lot of, there might be vaporware concept, there might be concepts that you don't know how you're going to solve, make business model problems. But then in the FAQ, then that defines, okay, what are the hard problems we're going to have to go tackle to make this exciting product a reality? So we switched to that method and it was halting progress. And have we not really taken that approach? Like we would have not ended up with what turned out to be a breakthrough product, which was the Kindle. That's fantastic. So a couple of questions to just quickly probe on a few nuances.

First of all, I know what you're really saying is flip the perspective from inward out to outward in, which I think is really interesting because that's something I constantly think about content, like oriented in the value to the reader. But what would you say on the flip side to the crowd that often says that part of the problem with, quote, working backwards, well, then you're not really inventing what they don't know what they want. And how does one write a press release for the startups out there thinking, oh, well, you know, we're creating a new category. This is not something that exists.

Tell me more about how you might address that crowd with this PRFQ approach. I would submit that, in fact, that's what this process is actually designed for. In 2004, when we started on digital media, the way e-books were you could only read them on your PC. There were no offline readers and tablets and devices in those days. They were priced way too high, like the same price as the hardcover book. The e-books had existed for four or five years before the Kindle launched in 2007, but it was a tiny, tiny business. And it was a tiny business because no one had imagined, well, what do I need to create? What's the new thing I need to build to make e-books work?

We defined what would be the ideal reading device, everything from it would be always connected to the internet, which, by the way, devices weren't that way back in 2007, that you wouldn't have to create some separate account or link your account to some mobile carrier, that when you got it out of the box, it already knew who you were. And so if you had actually bought a bunch of e-books online, they were. magically already uploaded onto your device. All of those kinds of things would be described in the FAQ section to say, yes, here's hard problem one, and here's how we're going to solve this problem.

Or we don't know how we're going to solve this problem yet, but here's our path, how we're going to go work on that. Yeah, I would just say there are two very real use cases where Amazon used the working backwards process to create something from scratch. with AWS, you know, cloud computing didn't exist. And as we were working through the Kindle issues, you talk about context, which in an hour later, we'd go to another conference room, and we'd be with Andy Jossie, who's the CEO of Web Services, and we'd be reviewing Word documents about what would eventually become cloud computing. And it took us about two years to come up with

what are we actually trying to create? And, you know, the first two were centered around storage and compute eventually. And where the press release, one area where it really helped crystallize everyone's thinking as we came up with saying that we want people in a college dorm room to have the same access

as an Amazon developer to world-class infrastructure. And that was just a powerful metaphor about, okay, so what are we creating? And what is this infrastructure that we really need? And starting from the customer experience, we had many, many

small teams throughout the company, just screaming at Jeff at the infrastructure team, I've built my service, it's taken me too long to deploy it and get it out and ready for customers. And that's where it started off as provisioning, but it kind of morphed through this working backwards process to compute. And then in terms of storage, there's a whole bunch of different types of storage that narrowed down to simple storage service, was the very first thing then we would build out from that. I love that example. it really emphasizes this point that when you start with the PRFAQ, you're essentially starting with the differentiation because you're really thinking in terms of the value,

because there's a million options for people to pick from. So that's when you go from provision to compute because you're thinking if the customer is this kid in the dorm room having access to, you know, we often talk about the power, like actually Mark Andresen in his original 2011 Y Software is Eating the World Op-Ed points to the power of this movement. Like, you know, AWS is being huge in bringing, we even have this op-ed about why every company is a fintech company. We actually call it the AWS moment in fintech.

It's quite amazing the impact that AWS has had in the industry, and there's no question about that. But that precise point of starting with the PRFAQ to take what could have just been like, oh, here's some storage, and here's how to provision the services you need, to here is how to create an entirely new business. That's a whole different. gang. The other thing that's really important to note where the PRFAQ process is misunderstood or

in conflict with the startup community today, sort of a lean and agile approach. I want to hear about this. Love it. Yeah. So have some fights, Bill. All right. All right. So here, let me tell you what the problem is with the lean and agile approach. I love Eric Reese for the record, but he himself is the first, and he's actually said it on this podcast, that sometimes people get a little culty and follow the letters of the rule instead of are the principles of it. I actually personally, I'm a big believer, having worked at Xerox Park, in the maximum viable product sometimes instead of the minimum viable product. Yeah, it's really actually, it's the V part. The problem is, is that people focus on the M, the minimum,

and they don't focus on the viable. So what is the definition of viable? Yeah, what is the definition? What would you say it is? To me, the definition is like, oh, if I go build this thing, I have created a, insert, size of business here, $100 million billion business. I have enabled, if I'm right, then this opens the path for like something really big. And I see instead happening, okay, I've got a sprint, I've got a couple weeks, what can I get done in a couple weeks, or, oh, what can I launch quickly to sort of test and learn, right? Where then you've created completely different constraints. And, oh, by the way,

if your whole dev team thinks of their whole roadmap and it break into these little chunks of how do I iterate quickly, test and learn, then you're going to launch a lot of small things where the actual size of the viable business on the other side of it might be like sized in like the $1 million range or $2 million range. They're like the actual potential good outcome. is very, very small. Now, there are plenty of places where this approach is totally applicable. Search, where you're like, how do I test and learn with, like, changes to the algorithm or changes to the logic, a new AI model. But if you're thinking about, I'm starting from scratch and trying to create a new business, the problem with this, the MVP approach where the viable part isn't

really thought out and mapped out is that people haven't really thought through, like, well, what could this really become and why might this not work? And in many cases, they could have actually, if they instead spent more time up front in the planning process, thought much bigger about what does the customer really need? Or, wow, I've really created some significant value versus, frankly, these little incremental changes don't really even move the needle at all. People don't do press releases for incremental releases. They do press releases for big advancement. That's actually an excellent point. If you, if you perish, into a new company and you look at their product roadmap and there's not one thing on there

you'd write a press release about, then you've got a problem. Yeah. To me, viable means you read the press release of what you're trying to build and you want to buy or use the product. If it's not something that you want to buy or use, it's not viable. Yeah. I mean, I would also push back on the definition of viable because what I heard from Bill and Bill, you should correct me if I'm wrong on this.

Colin, when you say like that obviously the customer is going to want to use it, But to me, viable is not just that it's something a customer would use because I think there's a lot of dumb things customers would use, quite frankly. I heard it more as the enabling conditions to really make something bigger and then also to address a deeper, more underlying opportunity sometimes instead of the surface opportunity. Because not to get all jingo-ish on this, but I, of course, think of Clayton Christensen's jobs to be done framework. And it's sort of about what is the underlying job to be done that is being fulfilled for the. this person. So that's how I heard the viable. What does it take to make this happen, whether minimum or maximum? And then also not only thinking about the product, but all the enabling conditions to get to addressing the deeper opportunity. In other way, where I've seen that the MVP

process being misused, because it can be useful, is that the process itself becomes the goal. Yes. Because I'm supposed to launch every two weeks, or I have a MVP. So this MVP card is go to the front of the line and get whatever resources I want because I'm doing an MVP. So don't let the process itself become the end goal. And then don't let the MVP get in the way of understanding your customers. Because some people actually forget the customer problems that they're trying to solve. Which is the whole point of doing an MVP in the first place. I mean, the whole point is to be able to get that feedback so quickly because the whole reason that that idea came about was that you don't want to have that very long delay in the

feedback loop. Yes. And if you've watered down the feature set so much just to conform with what the process is, it's a shallow, pathetic version of what you're actually trying to test. And so you say nothing here when you never really gave it your best shot. What's really interesting is you actually in your book outline the total addressable market or Tam. And I found that to be very interesting, in connection to the discussion about how to think about the biggest possible market for this product working backwards. Yeah, we spilled a fair amount of ink on the TAM part in the book because people with less experience sort of get this part wrong. To not only think about how many people have this problem, but like how big is the need? And for how many consumers is this problem really big enough that they're willing to spend money to do something about it?

How much would they be willing to spend? and how many of these consumers actually have the characteristics or capabilities to actually make use of this product. And a lot of people don't really step through that clearly. The thing is that you want to ask all of the hard questions up front. And as we talked about, this is an iterative process. So in the review meetings, you know, if there are a couple of questions that you don't know the answer to,

you just append those to the document and say, great, let's come back in two weeks, however long it's going to take it. The FAQ, you can basically break it up into two different components. One is an external FAQ. You're explaining how does this product work? How much does it cost? Why should I use this service versus what's out there on the market or why do I need to change my behavior? And then the internal FAQs, what are the things that we need to go organize and solve in order to make this product or feature a reality? And then, like, you touched on how big is this opportunity? You know, what is the total addressable market?

Then you can size up the opportunity and that'll tell you whether it's worth doing or how much to invest in it. Because one of the things that you ask when you go through the working backwards process is, is this big enough to be worth doing? So it may be a good idea, but it just may not have the impact and scope on your customers or the organization to make a difference to be worth devoting resources to. when we were figuring out in digital media in AWS, what are we going to go build?

And you parachuted into either one of those teams and said, hey, guys, how's it going? We said, oh, my God, we're so frustrated because we just want to go and build this thing and get it out there that Jeff, you know, is insisting that we, you know, go through this PRFQ process and figure this all out in advance.

And I was among the people who found this frustrating. And it was only in hindsight that I realized, like, how smart Jeff was, to slow us down. And we co-opted the Marine Scout Cyber Sting, which is that slow is smooth and smooth is fast, or measure twice, cut once. Or sometimes Jeff would refer to himself

as the chief slowdown officer when teams were sort of itchy to go pull the trigger and build and launch something. What I came to appreciate in retrospect is that to actually create value, you really have to take the time and think big, that's what the PRFAQ process does. And if everyone can go write PRFAQs and you've got 30 or 40 different ones to review,

then I'm here to tell you that the best ideas are going to bubble to the top pretty clearly based on this process, whereas the MVP lean process really is just about cranking stuff out rather than first filtering, thinking through, refining what to do. What to do. People confuse speed and activity with effectiveness. Yeah, exactly. What I find so fascinating about that, by the way, is just coming back to where we started,

is how the 14 leadership principles balance each other. Because you describe Jeff in this context and a lot of these processes as ways to slow things down up front, but then you also have principle number 10 or whatever number it was for bias for action at the same time. You think of this as a whole system. And it's really interesting because it resonates to me with Ben's, Ben Horowitz's book on culture, because he describes like the value system of the samurai warriors where they would have something

that was like Bushido, where there'd be something that was incredibly kind and generous and then something that was incredibly vengeful in the same like framework. It's very interesting how those motions are kind of opposite, but yet in balance. Yeah, they do work together. We started off the conversation about the R-R-R-A-Lat principle. Well, the one right below that is learning, be curious, and the one right above it on the list is invent and simplify. And you need to do both of those in order to be right a lot. The one thing I would add, a lot of people say, well, Amazon,

they either have so much money or so much time they can afford to do all these things. Long-term thinking doesn't necessarily mean it takes longer to get to your end goal. You know, Amazon built $100 billion business faster than any company. AWS got to $10 billion in revenue from zero faster than Amazon the retail business did. So sometimes to slow down to move fast, even with smaller organizations, you're more than likely will get to where you want to be quicker if you take some of these steps. Typically what you're doing is you're doing off-path distracting activities that, by the way, are taking away from your bottleneck resources, which are typically software engineering resources

at a lot of organizations. Yes, exactly. That are meant to build up long-term value. Yeah, and just to pick, the AWS thing is so remarkable. Think about that for a minute. Because we just told you, I mean, what was it calling? 18 months, 24 months? How many months from Go before the team even wrote the first line of code?

I mean, it was at least 18 months. And I did have software engineers after some of the meetings come to me and say, hey, can you remind Jeff that our job is to write software code, not documents? And they were itching to go and it took a while to figure out what we should build. And that time was very well spent writing word documents versus writing C code. And not like that, it's empirical fact that it was well spent because the company set the record for the fastest company to a revenue milestone by spending the first 18 to 24 months planning. And you talk about distractions.

We did not know at that time what other. people were doing. You know, web services were no secret. There were other companies who, by their own right, should have gotten there first. We didn't know if someone would come out with a set of developer APIs at that point in time. I know. That scares me thinking about that, actually. Like, almost two years, like planning, like that's scary. Think how scared they were. But Jeff had the fortitude to say it's not ready yet. This is not what we want to build. It's not viable. You know, talk about what makes Amazon special. You can read these principles. Sticking to them is sometimes quite challenging, either when things are going really, really well, or when they're not

going well or when there's uncertainty. These are not just posters on the wall. They're woven into the DNA of everyone who's been in Amazon for any length of period of time. I'm just struck at all the parallels and all the things you guys are saying, because essentially every business today is a creative business. What really strikes me is I'm not going to make a comparison between podcasts and AWS, but I will say that I talk to a lot of podcasters for how to make their podcast stand out, differentiate, like how to do editorial strategy comes up a lot. And one of the things I constantly say is if you can't be first or leading, then you have to be very differentiated. It's really interesting when you talk about the bottleneck resources. I think of things in terms

of opportunity costs and return on energy or what I call ROE. And similarly, you have to pick which things you're going to do for the greatest possible hits, or you'll never punch above your weight or be heard above the noise, which I think is so fascinating. We had Jeff Lawson from Twilio on the podcast recently, and he's like, you know, we need to give them problems, not just like specs and things to work on. And so when you describe that they're like, we're spending, you know, 18 months writing planning documents that is treating software developers as creatives. That also creates alignment for the teams that will go out and build. Because sometimes it requires more than one team. If they're not aligned on what the problem is, they're going to

solve different problems and those components aren't going to fit. Right. I want to switch into talking about some specificities, some advice for startups. One tool you can use for startups, and I've actually done this at other companies. If you're trying to figure out what to build, you can write competing press releases with different takes on the problem. And then it becomes pretty apparent when you've got two or three or four different approaches that we really need solution A or I'm going to take the first part of this solution and combine it with a great idea that came up in the second one and this is actually what we want to build. So it's a lightweight way to crystallize your thinking and also get alignment. Yeah, think about how inexpensive it is to

write a one-page press release versus how expensive it is to build mock-ups. To Collins' point, when you read them all as a group? Like the best ones are going to be clear. You know, again, this is where I just can't get over how creative the book is in terms of its application to all kinds of creative fields, companies big and small, because to me,

both the memo strategy, the PRFAQ, and then this idea that you both shared of being able to write competing press release and then harness kind of the wisdom of the group. It's actually about the power of narrative for really helping instantiate things that you know when you know,

I think a lot about how tools change our thinking and vice versa. And you guys are essentially describing these practices for how that plays out in organizations, especially as a scale. On this note, you have a section in your book, the subhead is better coordination was the wrong answer because that's one of the common myths that people have as a scale is we need to coordinate better. And in fact, it creates layers and layers of crap to deal with people create entirely new roles, like dedicated chiefs of staff just to like, you know,

create stitching and seeming between teams. And it's just the most ridiculous thing I've ever seen and heard. And I just want to hear more from you guys, especially because a lot of listeners and we're talking about what happens when you scale and grow very quickly. You guys really did see this phase at Amazon. Yeah, this is a case where Amazon faced the same question

that a lot of growing organizations do. We're adding more people. It's just, it seems like it's taking longer to get things done, but came up with a different answer. Some companies would say, let's build coordination tools, let's collaborate better. And Jeff said, I'd love to eliminate coordination altogether. Now, in practice, you can't eliminate it completely.

So you break up into loosely coupled teams. And this was a hard, hard problem to solve because it required to change the way we built Amazon. Technology-wise, decouple it into what's now services-based architecture. That was hard to do back then in 2000. in 2001, especially as you're growing super fast. But then there's also the organizational component, too, about how do decisions get made? It was a multi-year effort, to be quite honest. But if you're going to grow 10x and 100x, we're not going to spend any time building.

We're going to spend all of our time coordinating. So, you know, nip it in the bud. And Jeff wanted Amazon to be a place where builders can build. So the ironic thing is that we started off with the process. that was the conventional way, except the traditional wisdom, it was brought in what we called NPI, new project initiatives. It was horrible. I mean, basically, it was another example of the process becoming a thing,

where you had to come up with, like, what's your new project initiative? You had to write it all up. You had to project what are the financials behind it, most of which were totally wrong and guesses, like almost all pro forma PNLs are, and then put it in front of a big committee and try to take it upstairs.

Not only was this process of massive way, waste of time. But then you'd have these frustrating business reviews with a team saying, yeah, we really need to go build X, Y, and Z. And Jeff and S team saying, yeah, I agree. Go build X, Y, and Z. But they say, well, I can't because I need brows to do this. And I need the team that works on the checkout, the order pipeline. And they already have like these four other projects they're working on. So I'm just sitting here in line and I can't go build it. And so you've deempowered your teams. The senior management, they become like referees of between teams, who's going to go do what.

And frankly, it's not much fun. So this was a huge breakthrough for the company, was breaking down, you know, not only before breaking down the code into APIs, but then breaking down the teams into single-threaded focus teams. There's some roles that don't actually fit too well in this type of paradigm. So, for instance, like a chief product officer doesn't really fit in this role because if you are responsible for making every product decision in the company, from what's going on in the warehouse to what should the, how should the apps be built, to how can we decrease delivery time, there's no one person who can be obsessed with all of those details.

Right. And so that role typically doesn't fit as you separate into these small, separable, single-threaded teams. The fallacy is as your company grows, the chief product officer isn't really doing that anyway because they can't get that much high quality information to make all of those important decisions. You do want that the team's closest to the customers making those types of decisions anyway. Yeah. I call that bare metal decision making. Like who's the closest to the metal of the thing? And that's what I think is super valuable about what you just said. And you're essentially describing these small units as like every team is its own mini. That's right.

GM of every mini unit is a leader and then every product, you're like so close to the core in that decision-making framework. That's right. The idea is for each one to be like a self-contained unit. And in search, that's all engineers. But in some other business like my digital video business, it was like a combination of engineering and marketing people and people working with studios. But you have all the resources you need. You don't have these dependencies that basically slow you down. During this time frame where we were making the transition, we were using narratives. And one of the questions that we actually required was for the teams to put in, what things that are not under your control that you wish you had under your control and how are you going to

organize and create APIs so that can happen. Again, this is work that's below the tip of the iceberg of where Amazon really put a lot of effort into how can you still grow, and be as nimble and agile as we were when we started out. And it's not easy, but it only gets harder, so you may as well start now. Yeah, and don't take away from this that Amazon and some Nirvana world where you never had to worry about coordination with our teams. It was just that we worked so hard to minimize the degree to which we did. I'm very fascinated by how organizations evolve in general to be effective.

And when you think of any large company, it becomes a complex system. And you're essentially describing how modular self-contained units can thrive in these complex systems. It's a lot like evolution. And these pieces all fit together because then we have to loop back to like, oh, the reason that Amazon could do that is because those teams wrote narratives and PRFAQ so they made it abundantly clear. If Jeff wanted to see what is this team doing, oh, here you go. You read this document and in two hours you knew exactly. Right. He makes him the chief ecosystem officer essentially, not just the chief slowdown officer. Okay. So, Colin, so a question I have for you.

you, it's really unique that you were able to shadow Jeff Bezos and be his shadow. And, you know, I don't want to make this about the glorification of Jeff Bezos. I'm so not interested in that because I think there's just plenty of narratives out there. What I am interested in, however, is this question of what it takes for a CEO and a leader like that to evolve. And what you saw in shadowing him on that front and then also the act of shadowing. And that itself as a mechanism for learning mentorship, apprenticeship, if you will, I'm very, very, very, fascinated by this whole thing. Sure. So I mean, my role is Jeff Shadow was primarily two parts. One was just to make him more effective CEO on a daily basis, making sure that the right issues are surfaced, that the people

coming into the meetings, we cover the right topics and have enough information to make the right decisions on afterwards that it gets followed up on. So you're kind of bookending the day. But then the Second and more important part, the longer term part, was it was a training role. And the way Jeff put it is, I want us to be able to model each other and, you know, how we would think in different situations. One of the enlightening things that I realized during my time with Jeff is what he chose to work on during the two years, I was his shadow or technical advisor. And it wasn't the biggest businesses at Amazon. about half of the time were spent on what would become AWS and digital. And those businesses had revenue of effectively zero for those two years.

And he told me more than once, you want your top leaders and your most impactful people working on your biggest opportunities. And that may be different from what your biggest current set of activities are. How Jeff changed during that time and just to become a best. better CEO. One thing you said is he learned how to become a better operator, becoming more operationally efficient. Fortunately, that is a teachable and a learnable skill. And Jeff had some great people at Amazon like Jeff Wilkie, who's a great operator. He's the CEO of the consumer business. And now in his own right, he is his great operator. And you insist on the highest standards

is one of Amazon's leadership principles. He holds people accountable, himself included by the way. And I would say that I also learned that sticking to these core principles is harder than it sounds, but the time when you need them most is the times when it's easiest to ignore them. So for instance, Amazon Prime, he made the call that, hey, we're going to go launch this shipping project in the holiday season. And it was not very popular at the time, but when he explained is thinking that, you know, customers were basically giving us a B-minus on our three to five-day shipping, even though we had just gotten reasonably competent and spent a couple hundred million dollars doing that. Jeff looked at it, you know, long-term and said, well,

we're becoming a smaller and smaller share of the overall e-commerce industry. So we've got to make the change now. And so it wasn't that Jeff had this insight that comes once in a generation for Amazon Prime, he just stuck with the leadership principles and took them to their logical conclusion when it was tough to do in the face of the holiday season of what our quarterly results were. By the way, for the listeners, because we don't obviously have time to go into the whole book, but half of the book, the first half is about the principles and these practices and mechanisms, but the other half is actually showing through these very detailed case studies and examples that you both participated in a witnessed or oversaw firsthand in this invention

machine at work, and that includes the Kindle story, the AWS story, Prime video, and what you just mentioned, Colin, which is Prime. And what's really interesting in that Prime section is that not only that Jeff had the wherewithal to push through, but what's really interesting in the Prime program story is that it's all the iterations involved to actually get it to what it is today, because there was like a 1.0 and then, you know, the loyalty program it later became. So I think that's a really great part of the book for those that want to learn more. And Bill, anything to add there on what you saw in the evolution? I do. In fact, Jeff, I remember Jeff, at least more than once, talking about the way a leader needs to evolve as the company grows. And I think about this

frequently, which is that you said at the beginning, the leader needs to really focus on what. Then they really need to focus on how. And then eventually their focus really just becomes who. Wow. What is, you know, what is our business? What's our product? What are we building? What are the details of that? right? How is what processes, how do we do the work? What is the filter, the lens through which we make decisions? And then who, of course, is who are my leaders? Who have I, how are I making sure I assemble the right team? And in doing so, how do I delegate responsibility to those people so that they can carry out those details? That is a classic and challenging transition for any

entrepreneur owner who starts off being in control of every detail and that they have to slowly let go of those details. And then they have to figure out how they put in place the right mechanisms to be able to properly audit the work of the teams. And that's really what, you know, as we tried to describe in the book, is actually this management science, frankly, that Jeff and the team developed to really solve this problem. Working backwards. Insight, Stories, and Secrets from Inside Amazon by Colin Breyer and Bill Carr. Thank you so much, you guys, for joining the A6th and Z podcast.

Thank you. Thanks, Tonal.